Page 1 of 1

Deduplication for VM Problem beta

Posted: Tue Oct 15, 2013 1:52 am
by eliripoll
Hello this Post is a continuation from the following post in the standard forum, since we are now testing with V8 we are moving it to the beta forum. I have been communicating with Anatoly

http://www.starwindsoftware.com/forums/ ... t3267.html

Re: Deduplication for VM Problem

Posted: Tue Oct 15, 2013 1:56 am
by eliripoll
Hi Anatoly

Yes , You Are Correct these VM have been running and in use. tomorrow I will be turning them off if you need me to check something. The reason they are still running is I was under the impression that starwinds inline deduplication meant a file could be open for deduplication to work. If that not correct please clarify. Thanks Let me know if there something you need me to try or check

-Eli

Re: Deduplication for VM Problem beta

Posted: Tue Oct 15, 2013 3:03 pm
by Anatoly (staff)
Well, you are right - that is why inline DD is for, but the trick here is that some datablcoks are removing from VM, but still on he DD drive, and DD engine need some time to delete unused blocks.

Re: Deduplication for VM Problem

Posted: Wed Oct 16, 2013 12:10 am
by eliripoll
in this particular case the vms have been running for about a month. I would think that would be enough time to dedupe. also another interesting thing is the size has grown 7gb in 4 days. from windows point of view that size is still the same but from the drive that is holding the sws spspx files is is steadily growing. Note that only one page file is in use on the sws drive everything else including the database that one of the VM is on another drive. as I watch the disk write activity 90% is just small IO from trivial things like event log or ntfs log data nothing else.

should I try shutting the vm's off, is there a way to tell SWS to try a Dedupe job?

Re: Deduplication for VM Problem beta

Posted: Mon Oct 21, 2013 2:53 pm
by eliripoll
I will go ahead and wait for V8 to be closer to RTM and try again. As an update I turned the VMs off and waited to see if SWS would do dedupe but it didn't. Note though that to do these test you have to setup some VMs (multiple) and let it run for weeks to see if space reclaim is really working. SWS does great for initial Dedup but after letting VM's run for 3-4 weeks is real test.

Maybe you guys can setup up the same scenario and let it run for about a month and see what I see.

Thanks.

Re: Deduplication for VM Problem beta

Posted: Wed Oct 23, 2013 7:30 pm
by anton (staff)
You may try with Beta-2. Make sure you don't keep a lot of snapshots as every write which is unique would be stored in a snapshot until deleted. Manually or by schedule. So if you keep ALL writes in a snapshots you keep CDP-like history for your VM. Every single transaction.

Re: Deduplication for VM Problem beta

Posted: Wed Oct 30, 2013 12:12 pm
by andyman80
I have a similar problem too. But unlike the case with VMs I just copied and erased file data to the deduplicated disk several times. Now the logical disk is empty, and there hasn't been any activity on it for a week. But 20 GB of raw storage files *.spspx are still there. Deduplication, deletion support and auto-defragmentation options are enabled. The latest log file is attached.

Re: Deduplication for VM Problem beta

Posted: Thu Oct 31, 2013 3:44 pm
by Alex (staff)
Thank you for the logs!
In this build LSFS device frees up disk space when snapshot is deleted and when existing data is overwritten.
But it doesn't processes UNMAP/TRIM requests yet. We are fixing this functionality now.

UNMAP command can be sent to disk by application when some data is deleted from disk.
For example, ESX server sends it when VAAI commads are used. NTFS in Windows 2012 also can use this command.

Support for UNMAP/TRIM will be available in LSFS in the next update.

Re: Deduplication for VM Problem beta

Posted: Tue Nov 05, 2013 4:57 am
by andyman80
OK, thank you for the investigation :)

Re: Deduplication for VM Problem beta

Posted: Tue Dec 03, 2013 9:16 am
by Anatoly (staff)
It`d been our pleasure to work with you.
Please do not hesitate to contact us should you have any additional questions. It would be our pleasure to further assist you!