Deduplication stats

Tue Oct 02, 2012 10:46 am

Sure. Please drop a message to sales and they'd be happy to issue you with an updated key.

robnicholson wrote:Hi Anton - because of my holiday, the trial license is about to expire. Can I request another 30 days?

Cheers, Rob.

robnicholson · Mon Oct 08, 2012 3:55 pm

Trial with 4k block size at delete option checked is working better. Copied 699GB of data so far and the deduplication disk is 595GB so that's a 1.17 deduplication ratio - ~17% more disk space. Will carry on the robocopy (it stopped for some reason) to get to the total 937GB on the source disk.

Cheers, Rob.

Mon Oct 08, 2012 3:59 pm

Can you compare with Microsoft built-in dedupe with Windows Server 2012? Just in case?

robnicholson wrote:Trial with 4k block size at delete option checked is working better. Copied 699GB of data so far and the deduplication disk is 595GB so that's a 1.17 deduplication ratio - ~17% more disk space. Will carry on the robocopy (it stopped for some reason) to get to the total 937GB on the source disk.

Cheers, Rob.

robnicholson · Wed Oct 10, 2012 9:18 am

Hi Anton,

I received the extended trial license so was able to finish the copy (with 4k blocks & delete blocks enabled) and the overall deduplication ratio on an 800GB general-purpose file share was 1.19 which I'm saying is 20% between friends. The purpose of this trial was just to see what kind of ratio we'd get if we deployed and I guess a 20% saving is not to be sneered at.

One slightly worrying thing is that today, I've completely lost the data on the deduplicated disk overnight but I know what triggered it. I'm trialing this on my own PC running Windows 2008 as a VM in VMware Workstation. Not exactly the most production ready set-up I'll agree.

It was "Windows update Tuesday" yesterday so my PC installed updates automatically overnight and then rebooted. I'm guessing that VMware Workstation didn't shutdown cleanly - it simply powered off (need to confirm that) so this was a "power pulled from StarWind" scenario.

When I restarted the VM this morning, VMware workstation did complain there was a potential problem with the vdisk used to stored the StarWind dedupe disk so I was expecting problems. I let VMware Workstation repair it. On logging onto the console, the license had expired (strange as I'd loaded the new one a week ago) so I uploaded the new license and the dedup disk mounted.

The Windows 2008 which was using the iSCSI target reconnected automatically but the disk was shown as uninitialized. To be honest, I didn't check whether I could somehow bring it back so I initialized it thus zapping anything on there.

As this is a trial environment, I'll do a few more "Reboot VM" tests.

Cheers, Rob.

Fri Oct 12, 2012 4:15 pm

I received the extended trial license so was able to finish the copy (with 4k blocks & delete blocks enabled) and the overall deduplication ratio on an 800GB general-purpose file share was 1.19 which I'm saying is 20% between friends. The purpose of this trial was just to see what kind of ratio we'd get if we deployed and I guess a 20% saving is not to be sneered at.

Well, yes, 20% is still savings. As I understoud you are using Hyper-V VMs, and to be honest this is known that VHD are not the best files that are formatting. But I think it still can be tuned. The recommendations are:
-Don not use thin provisioning on any layer
-Use 4k block/stipe size everywhere - on DD device, on RAID array, on NTFS formatting
-Try to use GPT instead of MBR on the volume where the DD related files are stored.

As for the second issue: for me it more looks like Workstation issue. That is why we do not recommend to run StarWind on VMs.

robnicholson · Fri Oct 12, 2012 4:20 pm

As for the second issue: for me it more looks like Workstation issue. That is why we do not recommend to run StarWind on VMs.

Indeed it could and I'd equally I'd never recommend running StarWind or any software SAN virtualised. Fine for lab tests.

Cheers, Rob.

Sun Oct 14, 2012 3:01 pm

Exactly. We`ll look forward to hear some news from you if you are planning to proceed with testing.

Aitor_Ibarra · Wed Oct 24, 2012 5:41 pm

I don't know about on vmware workstation, but Starwind runs fine as a VM on Hyper-V. I'm running it in a scenario that probably would be better served by the Hyper-V native edition (HA SAN and MS cluster using the SAN on same hardware), but that wasn't an option when I started.

robnicholson · Wed Oct 24, 2012 10:50 pm

This was just a lab test so VMware Workstation was fine. Wasn't bothered particularly about performance. I think the stability problem is a VMware Workstation issue as it happened several times. All I had to do was cleanly shutdown the VM and on start-up, VMware warned about consistency problems with the VMDK. Allowing it to fix it and then repairing the disk in the StarWind VM and all was well.

Whilst StarWind can obviously by run inside a virtual environment, I'd personally be happier with it running bare metal. Won't help performance putting in the virtualisation layer. Once again, for a lab environment, Hyper-V, XenServer or VMware V-Sphere ESX would all be even better than VMware Workstation.

Cheers, Rob.

Wed Oct 24, 2012 10:53 pm

Yes, with a dirty shutdown you need to run clustered.

We'll continue to do both ways - bare metal on Windows (maybe on Linux one day but I'm not sure yet) and also VSA with an upcoming VSA builder app.

robnicholson wrote:This was just a lab test so VMware Workstation was fine. Wasn't bothered particularly about performance. I think the stability problem is a VMware Workstation issue as it happened several times. All I had to do was cleanly shutdown the VM and on start-up, VMware warned about consistency problems with the VMDK. Allowing it to fix it and then repairing the disk in the StarWind VM and all was well.

Whilst StarWind can obviously by run inside a virtual environment, I'd personally be happier with it running bare metal. Won't help performance putting in the virtualisation layer. Once again, for a lab environment, Hyper-V, XenServer or VMware V-Sphere ESX would all be even better than VMware Workstation.

Cheers, Rob.