how to set up vsan with hyperv?

[danswartz]

I've been reading various whitepapers and such here, and am a bit lost. I've moved off ESXi to proxmox, and am wanting to move off that to hyperv. I have 3 hosts with 3 1tb nvme drives, but want to go with a 2-host setup. I seem to be seeing that the linux VSA doesn't support running on hyperv? I saw some article on startwindsoftware.com that said 'no longer supported for hyper-v or kvm hypervisors"? If this is the case, I'd have to use the windows install and use storage spaces? I plan to pull 2 of the 1tb nvme from host 3 and move 1 of each to host 1 and host 2, and set up a 2x2 raid10 on each host. Is that doable? Some clarification would be much appreciated! Thanks!

ETA: this is the article I mentioned:

https://www.starwindsoftware.com/resour ... h-hyper-v/

[yaroslav (staff)]

The plan you mentioned makes sense and is doable.
As a side note, I'd like to mention that StarWind CVM can be deployed on Hyper-V as described here
CVM https://www.starwindsoftware.com/resour ... ng-web-ui/
VSA https://www.starwindsoftware.com/resour ... ws-server/
Speaking of the CVM, you can use the Management Console (https://starwind.com/tmplink/starwind-v8.exe) for monitoring only.

You can also install bare-metal StarWind VSAN from https://starwind.com/tmplink/starwind-v8.exe with storage spaces.

[danswartz]

ok, so what does that red-line comment about not being supported on hyper-v and kvm mean?

[yaroslav (staff)]

I think I misunderstood your initial message. I sincerely apologize for any misunderstanding caused.
VSA is the deprecated product that was re-shaped into StarWind VSAN for vSphere (it can also be deployed on Hyper-V in some scenarios) and in its new generation into StarWind CVM (can also be deployed on Hyper-V).
If you need software RAID (MDADM), go CVM path.
If you still prefer Storage Spaces as software RAID, can use hardware RAID in Windows, or can configure the HA device per disk, go with bare-metal installation.
Hope I resolved the confusion caused.

[danswartz]

ok that makes sense. you said 'mdadm' can i use zfs instead?

[yaroslav (staff)]

Sure.
But ZFS may cause a performance drop.

[danswartz]

How much of a drop? I don't care about small decreases, but really want the security of bit-rot protection, as well as transparent lz4 compression (I've found virtual workloads to compress as high as 2X).

[yaroslav (staff)]

Got it. ZFS seems to fit well in your system then.
Speaking of compression and dedupe, we are planning to move forward with it. Stay tuned!

[danswartz]

before committing to this, I wanted to test performance in a virtual appliance setup. my storage: 2 1TB samsung 960 nvme drives. i configured a zfs appliance on local datastore (omnios, since it does zfs natively). i passed the 2 nvme drives to the the VM. in the VM, once up and running, I configured a zfs mirror using them. i set sync=disabled to see what the best case was. i set up a vswitch (not external) so hyper-v could see the LUN i provided (which was on a file, not a zvol, again for best performance, since historically, zvols seem slower than files on a dataset). i then spun up a ws2019 VM on that datastore, installed and ran crystaldiskmark (not the best I know), and got about 500MB/sec read/write. I thought an NVME mirror should be better than that, so I took down the zfs appliance and created a storage spaces mirror on the same 2 nvme drives. migrated the ws2019 appliance there and re-ran the tests. 3GB/sec read/write! i understand there is SOME overhead in a virtual storage appliance, but this seems excessive, no? i tried jumbo frames and it didn't really help. i wonder if this is related to iSCSI? i've read in starwind forums about multiple iSCSI sessions or something like that. Any ideas would be appreciated...

[yaroslav (staff)]

Thanks for your update. Yes, multiple iSCSI Sessions is the key. for an NVMe disk, I'd be using 4-5 iSCSI sessions per link.
For improving large blocks, align FirstBurstLenght with MaxBurstLenght (regedit -> HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4d36e97b-e325-11ce-bfc1-08002be10318}\000X\Parameters
Also, did you try to run FIO on the ZFS pool inside the VM?

[danswartz]

Ah, thanks! I'll give that a shot!

[danswartz]

The plan you mentioned makes sense and is doable.
As a side note, I'd like to mention that StarWind CVM can be deployed on Hyper-V as described here
CVM https://www.starwindsoftware.com/resour ... ng-web-ui/
VSA https://www.starwindsoftware.com/resour ... ws-server/
Speaking of the CVM, you can use the Management Console (https://starwind.com/tmplink/starwind-v8.exe) for monitoring only.

You can also install bare-metal StarWind VSAN from https://starwind.com/tmplink/starwind-v8.exe with storage spaces.

I have to admit, the bare-metal route is tempting. I intended to use 4 1TB nvme drives on each side of the mirror, in a 2x2 raid10, but am seeing conflicting stories online about whether storage spaces supports raid10. If using SS, can I assume vsan just sees one large 'disk' on each side of the mirror? I'm used to zfs where I can set compression, checksum, etc, so still learning the hyper-v stuff...

[yaroslav (staff)]

If using SS, can I assume vsan just sees one large 'disk' on each side of the mirror?

VSAN sees the storage underneath as one "array" to put the disks on. It can be a Storage Spaces pool, or it can be a ZFS pool.
In storage spaces you can do mirrored volumes, I believe. All I want to mention is storage spaces may be tricky, especially when it comes to Windows Server updates.

Let me know if there's anything I can help you with!

P.s. do not ever run cluster validations on the storage with storage spaces (that kills an array).

[danswartz]

If using SS, can I assume vsan just sees one large 'disk' on each side of the mirror?

VSAN sees the storage underneath as one "array" to put the disks on. It can be a Storage Spaces pool, or it can be a ZFS pool.
In storage spaces you can do mirrored volumes, I believe. All I want to mention is storage spaces may be tricky, especially when it comes to Windows Server updates.

Let me know if there's anything I can help you with!

P.s. do not ever run cluster validations on the storage with storage spaces (that kills an array).

thanks, i'll stick with zfs for now. the delay getting this going was due to having to migrate things from proxmox....

[yaroslav (staff)]

That's quite a project if it involves cross-hypervisor migration.
Good luck! Keep me posted.
Have a nice weekend.