Deduplication for VM Problem

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

Post Reply
eliripoll
Posts: 33
Joined: Sat Dec 08, 2012 8:19 pm
Location: United states

Mon Aug 05, 2013 5:42 pm

Hello Hoping some one could shed some light on this. Im having (maybe) an issue with deduplicated disk , or I may be mis understanding something. so ill get right to it. I have a deduplicated disk that has 4 2008r2 vms on it 3 of the VM's are almost vanilla except they are domain controllers. the 4th vm has SharePoint installed on it (not the database just the app). below is a quick break down of the files
Drive C: holds spmetadata file
drive E: holds SPDATA file only target is sized to 300gb
drive G: is the iscsi target to spdate file on drive E. drive G: is partitioned to only used 110 gb. holds 4 2008r2 VM's, 3 are 15gb and the sps vm is 35gb.

a Few months back I notice something odd and that was that the SPDATA file grew larger then even what windows added all the vm files to be (110 gb spdata). after checking everything I decided to redo the iscsi target and monitor it. so I backed up the VMs and deleted the devices and spdate file and recreated a new spdata file (and chose deletion support). then copied the vms back in. initially every thing was fine it deduped the vms and the spadata file took up 33 gb of space. this made sense since 15 gb (for each vm) was already duplicate data anyway. at this point I started monitoring the size. and after a couple months the spdata file has grown from 33gb to 81gb of space! if I check the size within drive G its only taking 77 gb of space.

Now im confused because whats going on with the de-duplication, Now just incase you say "oh well it doesn't shrink to release data. it just zero out that data for later use" and all new writes for things like event logs , etc are what's making it grow. this wouldn't make to much sense since any use of dedupe for os vm use would just eventually grow bigger then the space actually being used adding no benefit to de-duplication. Especially since what you partition is all that Windows will use . if windows sees that the data takes up 100gb it is going to see it as full even if Starwind has 50gb of free zero'd out space to use. negating the need for dedupe in the first place since the spdata is going to keep growing to the size initially specified. I also included screen shot of the drive holding the spdata file and what windows see for the target.
e spdata g iscsi target
e spdata g iscsi target
sws dedupe.JPG (79.87 KiB) Viewed 8392 times
eliripoll
Posts: 33
Joined: Sat Dec 08, 2012 8:19 pm
Location: United states

Thu Aug 08, 2013 11:43 pm

I just wanted to bump this since there hasn't been input yet, also wanted to post the status since I posted to show the spdata file is still growing although the from windows perspective it really hasn't.
I would really appreciate some feedback we are testing out SWS and the viability of using the deduplication features. some of the expectations were set from starwinds web page here:
http://www.starwindsoftware.com/iscsi-s ... calculator that shows deduplication possibilities.
below is the drive (spdata file status at the moment showing it has grown just in the last few days.
spdata file 8 08 13
spdata file 8 08 13
sws dedupe 80813.JPG (81.83 KiB) Viewed 8323 times
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Tue Aug 13, 2013 1:01 pm

Hi!
Sorry for getting back to you so long time.

Just one question before we proceed - you have written the data,anad deleted it,a after writen some other data and deleted it again, correct?
I`m asking because for now we do not have block re-use or block deletion support, which means that if data will be erased, it doesnt mean that the block, where it was stored will disappear, moreover, nothing will be able to be stored there (on them). This will definitely be improved in next versions. Stay tuned.

So, if my assumption is correct then the only way that I see now is to simply create new drive iSCSI storage and move all the data on it.
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
eliripoll
Posts: 33
Joined: Sat Dec 08, 2012 8:19 pm
Location: United states

Tue Aug 13, 2013 3:51 pm

Thanks for the Reply, I understand the current version "not being able to free blocks/reuse blocks currently". but I'm still a little confused. does the below link here, that describes deduplication benefits for VM's only intended to just store cold vm images not actual running VM's?
http://www.starwindsoftware.com/iscsi-s ... calculator

Its not really clear in the above webpage or documentation ( I could have missed it)

Could you confirm that dedupeplication is only intended for Cold or rarely modified data.

In my case there were only 3 VMs running and when I monitored the Disk acivity for them, its always very light (2-4kbits since they are just idle 95% of time) so it wasn't modifying disk data a lot but it was constant (i.e. Page file, ntlog, ad journal, eventlogs, etc)
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Thu Aug 15, 2013 7:51 pm

The calculator was created basing on the results of our internal testing. I don`t think that posting of results making any sense, since everything is there, in ROI Calculator.
Nevertheless, knowing about the specifics of DD functionality we recommend people to use deduplicated storage to hold the backups, or some other data that changed not so freaquently, and running the live VMs on it should be really well tested before implementation in production environments.

Yes, maybe there are some things that are not 100% clear for everybody on our website, but that`s why we here for - to clarify everything and to give people hand :D

I hope that made sense for you, and sorry for confusion.
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
eliripoll
Posts: 33
Joined: Sat Dec 08, 2012 8:19 pm
Location: United states

Thu Aug 15, 2013 8:28 pm

Thanks so much for your time. Glad I only used test data.

one more thing, you mentioned the possibility of a future release improving the deletion, re-use. Are you at liberty to share if V8 will have block re-use/ deletion support.

thanks again

-Eli
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Fri Aug 16, 2013 9:41 am

I`ve just got the confirmation from R&D that version 8 will have unused blocks deletion support.
Actually you can participate in our beta-testing program and you`ll be the first one to try out the new features. Please use the link below to join our beta testing program:
http://www.starwindsoftware.com/beta-testing-program
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
eliripoll
Posts: 33
Joined: Sat Dec 08, 2012 8:19 pm
Location: United states

Sat Aug 17, 2013 5:00 pm

Awesome, thanks for all the info. Ill post back, if I have success in testing. :)

-Eli
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Mon Aug 19, 2013 8:42 am

No problem!
Thank you for choosing StarWind and do not hesitate to contact us should you have any additional questions. It would be ourpleasure to further assist you!
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
eliripoll
Posts: 33
Joined: Sat Dec 08, 2012 8:19 pm
Location: United states

Fri Oct 11, 2013 1:30 am

deduplication
deduplication
starwindsoftware.JPG (78.85 KiB) Viewed 7769 times
Hello I wasn't sure if I sure post my update to this thread here or in the Beta Forum. but I have since downloaded the beta version and have setup a deduplication disk with using the deletion support option. after connecting I have copied over 2 VMs both 2008r2. one is 27 gb the other is about 35 gbs. there the only two on there. when I initially copied the vm's over is made to 31gb spspx files, then I made a duplicate copy of one of the VMs to test DE- duplication. so total data should not read more than 60gb since one vm file is not in use just sitting there and a duplicate of another. But just like before the drive size (multiple 31gb spspx files) have grown to 120gb on the drive. while the IScsi drive that windows sees is only 99 gb (because of the two identicale VHD's) I would think if the two uniqu VHD don't use more than 60gb, that Starwind would at least Reclaim the other 2 spspx files. Do you know if the Reclaim Feature is working in the Beta 8 version of Starwind??
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Mon Oct 14, 2013 1:56 pm

It should work :)

Can I doublecheck with you if you those file were in use during the test?

P.S. Yes, this question is better be asked at Beta forum :)
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
eliripoll
Posts: 33
Joined: Sat Dec 08, 2012 8:19 pm
Location: United states

Tue Oct 15, 2013 1:57 am

Hi Anatoly,

thanks I have responded to your question in a new post in the Beta Forum located here:

http://www.starwindsoftware.com/forums/ ... tml#p19730
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Tue Oct 15, 2013 3:03 pm

Thanks!
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
Post Reply