Using Microsoft Dedup behind Starwind flat files

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

Post Reply
insomniac
Posts: 3
Joined: Sun May 19, 2019 4:36 pm

Sun May 19, 2019 4:49 pm

Hello,

I am presently running 2 HA servers which share 6 disks, configured as Microsoft CSV's. I have to all flash 70TB raid5 volumes that have multiple 6-10TB starwind flat files. I am deduping these volumes from my scale out file servers presently (over ISCSI). My concern is that this will become too much of a performance hit on the front end servers during heavy IO times. So my question is:

Is it supported to use Microsoft Dedup direct on the starwind servers(directly on the 70TB master volume)? Thus keeping most of the additional IO off of my front end servers.

Is there a best practice here? LSFS + dedup is off the table for me.
insomniac
Posts: 3
Joined: Sun May 19, 2019 4:36 pm

Tue May 21, 2019 2:26 am

Just bumping for an answer.
Oleg(staff)
Staff
Posts: 568
Joined: Fri Nov 24, 2017 7:52 am

Tue May 21, 2019 8:14 am

Hello,
The best way will be to run Microsoft Dedup on CSV's. In order to avoid performance issues, you can enable deduplication task during out of working hours.
insomniac
Posts: 3
Joined: Sun May 19, 2019 4:36 pm

Sat May 25, 2019 3:45 pm

Thank you for the reply, but this doesn't really answer my question. There is no such thing as off hours in my organization. So let me give you a scenario that I am considering moving to.

My present deployment if a 2 node HA that runs in a Hyperconverged cluster for VDI. This is all operating well at the moment, but I find myself in the same situation with Microsoft clustering being a major weak point, as we as introducing stupid issues such as Dynamic MAC mishandling, etc. I'm about done with Microsoft clustering TBH. So that issue aside, here is what I am considering and presently testing:

2 NODE starwind HA either locally mounting disk, or redundantly serving disk to additional nodes that host the VDI machines. The VDI machines will not be redundant, but will be using redundant disk. The disk will be presented to both machines as say the D: drive. There will be no split brain, as the VDI Hypervisors will not be configured in a cluster. Should a server die for any reason, I will run a scripted redeployment of static VDI guests on the other machine, or by utilizing UPD technology and pooled desktops, I will have to do nothing at all.

What I want to know is if I can enable Dedup against the actual drive that hosts starwind flat files. Thus keeping all overhead off of any front end boxes that host VDI (If i do end up using front end boxes due to performance). And to be honest, I have done this before in testing. I just want to know if there are any known issues with this.
Oleg(staff) wrote:Hello,
The best way will be to run Microsoft Dedup on CSV's. In order to avoid performance issues, you can enable deduplication task during out of working hours.
Boris (staff)
Staff
Posts: 805
Joined: Fri Jul 28, 2017 8:18 am

Tue May 28, 2019 7:52 pm

Using deduplication on a partition hosting the StarWind flat disk files is an unsupported scenario for now.
Also, I would recommend you avoiding connecting an NTFS partition to different iSCSI initiators (i.e. different hypervisors) without making it clustered (like CSVFS). You may end up with data corruption if multiple iSCSI initiators continuously write to the same NTFS partition.
Last edited by Boris (staff) on Sun Jun 09, 2019 4:42 pm, edited 1 time in total.
divyamane
Posts: 3
Joined: Fri May 31, 2019 11:12 am

Sat Jun 01, 2019 7:20 am

The plate will be introduced to the two machines as state the D: drive. There will be no part mind, as the VDI Hypervisors won't be arranged in a bunch. Should a server bite the dust in any capacity whatsoever, I will run a scripted redeployment of static VDI visitors on the other machine, or by using UPD innovation and pooled work areas, I should do nothing by any stretch of the imagination.

What I need to know is on the off chance that I can empower Dedup against the genuine drive that hosts starwind level records. Along these lines keeping all overhead off of any front end boxes that host VDI (On the off chance that I do finish up utilizing front end boxes because of execution). What's more, to be completely forthright, I have done this before in testing. I simply need to know whether there are any known issues with this.
Boris (staff)
Staff
Posts: 805
Joined: Fri Jul 28, 2017 8:18 am

Sun Jun 09, 2019 4:49 pm

If you enable deduplication on a volume that hosts the StarWind VSAN *.img files that can be like a Russian roulette for you. You may be lucky to keep getting blank cartridges, but there is a bullet there and you are to remember this.
Just wanted to remind you what has already been stated: this scenario is neither advised nor supported.
Post Reply