StarWind iSCSI SAN
StarWind Native SAN for Hyper-V
 

Deduplication for VM Problem beta

Public beta (bugs, reports, suggestions, features and requests)

Moderators: art (staff), anton (staff), Anatoly (staff), Max (staff)

Deduplication for VM Problem beta

Postby eliripoll » Tue Oct 15, 2013 1:52 am

Hello this Post is a continuation from the following post in the standard forum, since we are now testing with V8 we are moving it to the beta forum. I have been communicating with Anatoly

http://www.starwindsoftware.com/forums/starwind-f5/deduplication-for-problem-t3267.html
Last edited by eliripoll on Tue Oct 15, 2013 1:59 am, edited 1 time in total.
eliripoll
 
Posts: 33
Joined: Sat Dec 08, 2012 8:19 pm
Location: United states

Re: Deduplication for VM Problem

Postby eliripoll » Tue Oct 15, 2013 1:56 am

Hi Anatoly

Yes , You Are Correct these VM have been running and in use. tomorrow I will be turning them off if you need me to check something. The reason they are still running is I was under the impression that starwinds inline deduplication meant a file could be open for deduplication to work. If that not correct please clarify. Thanks Let me know if there something you need me to try or check

-Eli
eliripoll
 
Posts: 33
Joined: Sat Dec 08, 2012 8:19 pm
Location: United states

Re: Deduplication for VM Problem beta

Postby Anatoly (staff) » Tue Oct 15, 2013 3:03 pm

Well, you are right - that is why inline DD is for, but the trick here is that some datablcoks are removing from VM, but still on he DD drive, and DD engine need some time to delete unused blocks.
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
[email protected]
User avatar
Anatoly (staff)
Staff
 
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am

Re: Deduplication for VM Problem

Postby eliripoll » Wed Oct 16, 2013 12:10 am

in this particular case the vms have been running for about a month. I would think that would be enough time to dedupe. also another interesting thing is the size has grown 7gb in 4 days. from windows point of view that size is still the same but from the drive that is holding the sws spspx files is is steadily growing. Note that only one page file is in use on the sws drive everything else including the database that one of the VM is on another drive. as I watch the disk write activity 90% is just small IO from trivial things like event log or ntfs log data nothing else.

should I try shutting the vm's off, is there a way to tell SWS to try a Dedupe job?
eliripoll
 
Posts: 33
Joined: Sat Dec 08, 2012 8:19 pm
Location: United states

Re: Deduplication for VM Problem beta

Postby eliripoll » Mon Oct 21, 2013 2:53 pm

I will go ahead and wait for V8 to be closer to RTM and try again. As an update I turned the VMs off and waited to see if SWS would do dedupe but it didn't. Note though that to do these test you have to setup some VMs (multiple) and let it run for weeks to see if space reclaim is really working. SWS does great for initial Dedup but after letting VM's run for 3-4 weeks is real test.

Maybe you guys can setup up the same scenario and let it run for about a month and see what I see.

Thanks.
eliripoll
 
Posts: 33
Joined: Sat Dec 08, 2012 8:19 pm
Location: United states

Re: Deduplication for VM Problem beta

Postby anton (staff) » Wed Oct 23, 2013 7:30 pm

You may try with Beta-2. Make sure you don't keep a lot of snapshots as every write which is unique would be stored in a snapshot until deleted. Manually or by schedule. So if you keep ALL writes in a snapshots you keep CDP-like history for your VM. Every single transaction.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
User avatar
anton (staff)
Site Admin
 
Posts: 3922
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands

Re: Deduplication for VM Problem beta

Postby andyman80 » Wed Oct 30, 2013 12:12 pm

I have a similar problem too. But unlike the case with VMs I just copied and erased file data to the deduplicated disk several times. Now the logical disk is empty, and there hasn't been any activity on it for a week. But 20 GB of raw storage files *.spspx are still there. Deduplication, deletion support and auto-defragmentation options are enabled. The latest log file is attached.
Attachments
starwind-20131029-081125.zip
(6.61 KiB) Downloaded 293 times
andyman80
 
Posts: 6
Joined: Wed Aug 24, 2011 5:51 am

Re: Deduplication for VM Problem beta

Postby Alex (staff) » Thu Oct 31, 2013 3:44 pm

Thank you for the logs!
In this build LSFS device frees up disk space when snapshot is deleted and when existing data is overwritten.
But it doesn't processes UNMAP/TRIM requests yet. We are fixing this functionality now.

UNMAP command can be sent to disk by application when some data is deleted from disk.
For example, ESX server sends it when VAAI commads are used. NTFS in Windows 2012 also can use this command.

Support for UNMAP/TRIM will be available in LSFS in the next update.
Best regards,
Alexey.
User avatar
Alex (staff)
Staff
 
Posts: 177
Joined: Sat Jun 26, 2004 8:49 am

Re: Deduplication for VM Problem beta

Postby andyman80 » Tue Nov 05, 2013 4:57 am

OK, thank you for the investigation :)
andyman80
 
Posts: 6
Joined: Wed Aug 24, 2011 5:51 am

Re: Deduplication for VM Problem beta

Postby Anatoly (staff) » Tue Dec 03, 2013 9:16 am

It`d been our pleasure to work with you.
Please do not hesitate to contact us should you have any additional questions. It would be our pleasure to further assist you!
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
[email protected]
User avatar
Anatoly (staff)
Staff
 
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am


Return to Beta

Who is online

Users browsing this forum: No registered users and 4 guests