Sharing a single target to EWF-protected Windows initiators

[digitalis99]

I'm running the latest free build to share out a single target to multiple initiators. The initiators all use the target as a boot volume, and they're running Windows 7 Embedded with the EWF filter driver filtering out the writes to the volume. My initiator names are specified by iPXE, and I'm only using the iPXE default so each machine name announces itself uniquely to the target. I can boot, run, and shutdown a single initiator normally without trouble. I've verified and tested that EWF is enabled on boot in the image provided by the target. I have multiple initiator support checked in the target I created.

If I have one initiator booted and start a second, both systems eventually bluescreen. They *seem* to bluescreen when Windows switches from reading from the iPXE provided disk to connecting directly to the target with their own initiator. The only warnings I'm seeing in the StarWind console are that "journal switching is cancelled, because there are no changes made in the IBV volume since the last switching." I do see a new journal being established on each connection.

1) Multiple initiators pointed to the same target is supported, right? :^)
2) Any idea why the two hosts both bluescreen? Kind of seems like EWF *isn't* filtering all of the writes to the volume.
3) What would you guys need from me to help diagnose or troubleshoot this?
4) Has anyone ever tried anything like this? I'm basically trying to create a golden-image scenario where a bunch of hardware boots off a common, shared image and they just keep their few writes in memory until they shutdown and discard them.

Thanks in advance for your help. Hopefully I'm not crazy.

[anton (staff)]

I have an impression your filter driver somehow gets bypassed and both system try to write to the same volume. Of course NTFS does not like this and you have a nice BSOD.

There are a couple of ways to go:

1) Use StarWind built-in volume protection. This means "golden image" would reside on target and also target will take care of all writes going to do the different places. More
traffic on network but you're 100% safe and don't need to configure anything @ initiator side. Let me check did we include this feature into our iSCSI boot package from our
experimental branch.

2) Configure target on StarWind as read-only and continue with your filter keeping doing writes to some other place. Let me also check do we export "read-only" tag.

[digitalis99]

I'm going to try a read-only target to see if that's even possible to boot from when EWF is enabled. On paper, it should be, but I have a feeling that the EWF driver is being loaded too late in the boot process to filter *all* of the writes.

[anton (staff)]

File system is mounted as read-only during early boot phase so properly written disk filter driver should have zero issues. I don't think MS shipped one is broken but... you never know for sure Hold on, I'm still waiting for response from my team (weekend).

I'm going to try a read-only target to see if that's even possible to boot from when EWF is enabled. On paper, it should be, but I have a feeling that the EWF driver is being loaded too late in the boot process to filter *all* of the writes.

[Bohdan (staff)]

Remove and re-add image in read-only mode.
For Image Files:
http://www.starwindsoftware.com/tmplink ... SIBoot.pdf
Select "Mount existing virtual disk", check read-only mode (step 9 on the 6-th page).

For IBV (Snapshot and CDP): re-add device in Read-Only mode.

[anton (staff)]

Discard mode? When we'll be handling all the data @ target (no EWF driver @ all)?

[digitalis99]

I'm going to try read-only and redirect on write with discard, but I have another question. How am I supposed to easily update the image? It would be nice to have the read only or write redirect defined on the *target* rather than on the *device*. That way, I could have two targets pointed to the same device/file/image, one that all the slaves used to boot from with their writes redirected to /dev/null but a regular target that has normal read/write operations so I can update the image and have it persist.

As it sits now, it looks like I'm going to have to do a lot of copying and pasting of rather large files to update an image that is read-only or write redirected.

[digitalis99]

Tried all three CDP modes (read only, redirect write, and redirect write with discard), and I couldn't get anything other than a botched target. I don't know how, but the target itself gets corrupted, and then nothing can boot from it citing an inability to find winload.exe. Further, the cited step 9 on page 6 refers to an "image file device", which does not contain any of the options that I'd probably need. The relevant options are only exposed to Snapshot and CDP devices.

I think this falls under the category of: it doesn't work, it never worked, you couldn't have tested it.

My previous statement regarding defining the limits on the device level (in your product) definitely still stand. That seems like fundamentally the wrong place to make those definitions, at least from an operational and administrative perspective.

[anton (staff)]

What you do is:

1) Prepare "golden image" with one initiator
2) Make 1) read-only
3) Boot from it using "redirected" mode using multiple initiators
4) Enjoy

You cannot update "golden image" after you're done as there are block-level dependencies. If you really need to do this you have to
disconnect all active initiators, re-boot with one initiator, update "golden image", turn it back to read-only mode.

I'm going to try read-only and redirect on write with discard, but I have another question. How am I supposed to easily update the image? It would be nice to have the read only or write redirect defined on the *target* rather than on the *device*. That way, I could have two targets pointed to the same device/file/image, one that all the slaves used to boot from with their writes redirected to /dev/null but a regular target that has normal read/write operations so I can update the image and have it persist.

As it sits now, it looks like I'm going to have to do a lot of copying and pasting of rather large files to update an image that is read-only or write redirected.

[anton (staff)]

Let's go step-by-step (see my upper post). At what step do you starting to have issues?

We don't have exact this scenario in production so you have a great chances to end with solution you want.

I don't understand why it's "fundamentally wrong place". Could you please clarify?

Tried all three CDP modes (read only, redirect write, and redirect write with discard), and I couldn't get anything other than a botched target. I don't know how, but the target itself gets corrupted, and then nothing can boot from it citing an inability to find winload.exe. Further, the cited step 9 on page 6 refers to an "image file device", which does not contain any of the options that I'd probably need. The relevant options are only exposed to Snapshot and CDP devices.

I think this falls under the category of: it doesn't work, it never worked, you couldn't have tested it.

My previous statement regarding defining the limits on the device level (in your product) definitely still stand. That seems like fundamentally the wrong place to make those definitions, at least from an operational and administrative perspective.

[digitalis99]

Anton, I already followed your steps, since that's the only way to generate the image to be shared out in read only or write redirected mode. The problem is that when I shut down the initiator that I use to setup the image, delete the device, create a new device pointing to the same .ibv with the write characteristics I desire, and boot any initiator off of it, it fails to boot early in the process (before the Windows logo appears) citing an inability to read or find winload.exe. In short, it can't find anything on the volume. If I reverse that whole process, turning the device back into a regular Snapshot and CDP device, the initiator will boot from it normally. I'd say read only, write redirect, and write redirect with discard modes are all presently useless, since you can't create the image they will share out and an image created in a more normal mode can't be shared out successfully using any of the 3 modes.

As to my comment regarding where you define write permissions, it would be substantially easier on an admin if you defined these write permissions on the target rather than the device. Since you can only have one device per file, but multiple targets per device, it would make life much easier to have 2 targets pointed to the same device. Set one target to be write-redirected or read only, and leave the other target with full permissions to read and write. That way, to update the image, all an admin would have to do is boot off of the target with the full permissions, make their changes, and shut down. As it is now, there's far too much clicking around in the GUI to accomplish the same task by deleting devices, creating devices, re-attaching to targets...in short, far too many points of failure. Telling a machine which target to boot from would be a couple clicks, and wouldn't present any possibility to mess things up accidentally.

[anton (staff)]

1) At this moment QA staff is testing all the scenarios listed below. So after they will make sure they work as expected they will get back to you with a demo or remote session or just detailed step-by-step guide. For now we need to be 100% sure it's not something stupid on our side entirely.

2) I see... The problem is - you cannot have target shared even with one writer and multiple readers so "master" mode will automatically disconnect everybody else to avoid data corruption. Also please understand it's target with a bunch of LUNs (devices in our terminology) and not vice versa. For now we'll figure out how to make whole concept working and then we'll think about automation. If you don't mind

Stay tuned!

Anton, I already followed your steps, since that's the only way to generate the image to be shared out in read only or write redirected mode. The problem is that when I shut down the initiator that I use to setup the image, delete the device, create a new device pointing to the same .ibv with the write characteristics I desire, and boot any initiator off of it, it fails to boot early in the process (before the Windows logo appears) citing an inability to read or find winload.exe. In short, it can't find anything on the volume. If I reverse that whole process, turning the device back into a regular Snapshot and CDP device, the initiator will boot from it normally. I'd say read only, write redirect, and write redirect with discard modes are all presently useless, since you can't create the image they will share out and an image created in a more normal mode can't be shared out successfully using any of the 3 modes.

As to my comment regarding where you define write permissions, it would be substantially easier on an admin if you defined these write permissions on the target rather than the device. Since you can only have one device per file, but multiple targets per device, it would make life much easier to have 2 targets pointed to the same device. Set one target to be write-redirected or read only, and leave the other target with full permissions to read and write. That way, to update the image, all an admin would have to do is boot off of the target with the full permissions, make their changes, and shut down. As it is now, there's far too much clicking around in the GUI to accomplish the same task by deleting devices, creating devices, re-attaching to targets...in short, far too many points of failure. Telling a machine which target to boot from would be a couple clicks, and wouldn't present any possibility to mess things up accidentally.

[digitalis99]

Believe me, I'll stay tuned. I need this thing to work!

[anton (staff)]

OK, so we're on the same page.

Believe me, I'll stay tuned. I need this thing to work!

[digitalis99]

As regards 2), I understand how it currently works...I'm just saying that's very difficult to work with from an administrative perspective. Why would it be so difficult to have two targets pointing to the same LUN, one with read only or redirected writes and the other with normal write capabilities? Sure, if you tried to write to the image while it was being used, that would make the whole setup unstable, but lots of admin tools give you the ability to shoot ones self in the foot. That's not a good enough excuse to withhold the feature entirely, though.

The read only and write redirected device modes aren't nearly as helpful without some ability to "update" that device without destroying the entire configuration and rebuilding it from scratch, twice, every time you want to update the image. I think the possibility for admin error (and resultant data loss) is greater the way things work now than the way I propose. Of course, as you point out, the modes need to work at all for us to even have that discussion.

My $0.02.