V8 RC1 Thick Vs. Thin

Public beta (bugs, reports, suggestions, features and requests)

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

Post Reply
AKnowles
Posts: 27
Joined: Sat Feb 15, 2014 6:18 am

Mon Mar 03, 2014 6:48 pm

OK, I ran my usual thick vs. thin test. On an Arcea 1261 with 2 GB Cache, 10 Seagate 7200 RPM 2.5" drives in a RAID 5 array (partitioned as a 2 GB for iSCSI and the remainder for file system shares), iCore 5 S2400 with 32 GB RAM and Windows Server 2012 R2. No other disk activity was present during the test. Just iSCSI access from a single ESXi host. Both hosts have Intel Quadport Nics dedicated for iSCSI access.

Acera 1261 - 1TB Thick- 4096 cache - RR - Starwind RC1

Max_IOPS
Sum 36663.92 36610.83 53.09 44.43 17.88 26.55

Max_Throughput
Sum 894.73 841.64 53.09 447.37 420.82 26.55

Max_Write_IOPS
Sum 59162.24 0.00 59162.24 55.41 0.00 55.41

Max_Write_Throughput
Sum 743.11 0.00 743.11 371.56 0.00 371.56


Acera 1261 - 1TB Thin- 4096 cache - RR - Starwind RC1 - #1

Max_IOPs
Sum 18604.52 18551.43 53.09 35.61 9.06 26.55

Max_ThroughPut
Sum 375.11 322.02 53.09 187.56 161.01 26.55

Max_Write_IOPS
Sum 34841.81 0.00 34841.81 43.54 0.00 43.54

Max_Write_Throughput
Sum 266.45 0.00 266.45 133.23 0.00 133.23

Acera 1261 - 1TB Thin- 4096 cache - RR - Starwind RC1 - #2

Max_IOPs
Sum 17706.66 17653.57 53.09 35.17 8.62 26.55

Max_ThroughPut
Sum 160.40 107.31 53.09 80.20 53.65 26.55

Max_Write_IOPS
Sum 30756.01 0.00 30756.01 41.54 0.00 41.54

Max_Write_Throughput
Sum 228.41 0.00 228.41 114.21 0.00 114.21

And what I get out of this again is a thick iSCSI LUN is about twice as fast as a thin iSCSI LUN. Interesting enough, I ran the thin tests one after the other and expected better performance once the virtual disk had grown, but there was not very much difference and the thin disk did not grow to the full size of the sum of the virtual drives. Which means it dd not inflate fully. Instead of 400 GB of storage, only 57 GB was used. So, I'm assuming the virtual disk is not fully inflated and only virtual storage that was actually touched was present in the thin disk. I may retry the tests after inflating the disk with the vmfstools to see if it makes a difference.
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Mon Mar 03, 2014 9:23 pm

Again all wrong... You comprate uncomparable numbers. You're never going to see sequential I/O with many VMs running and that's what hypervisor typical access is. Also LSFS does accelerate random writes (80% of VDI) and is not good for pure sequential reads or writes. So if you want to test performance do spawn a bunch of VMs (10+) and run Intel I/O Meter in each. And do compare sum of all the numbers you get inside every one. That would be more or less close mapping to what typical I/O on hypervisor storage is.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
AKnowles
Posts: 27
Joined: Sat Feb 15, 2014 6:18 am

Tue Mar 04, 2014 4:43 pm

Actually, I have 4 VMs running on the datastore. I only printed the summary just to limit the amount of text to wade through, but if you want to see it all I can post it. There is no need for me to run more than 4 as 4 can max out the sequential bandwidth as it is. Even a single VM with a single NIC can pretty much max out an array for random I/O. As for sequential vs. random, well there are many instances of sequential I/O in the real world. Much is mitigated with VAAI and I'm happy to see it working and offloading the chore of provisioning and cloning VMs.

There are a lot of reasons to use a thin provisioned volume. I do it quite a bit, but performance is an issue with a thin volume as first the iSCSI target has to grown the LUN, the the block gets zeroed out, and finally the block of data gets written. So, I was curious. You can see the results.

Out of curiosity, how does your thin provisioning work with eager zero thick volumes? And when does the volume actually grow to full size? Will the vmfstools inflate it to full size? I'd like to do a test to see how a thin volume that has been fully populated will compare to a thick volume.

PS: I'm not trying to bash the product. Just reporting what I see.
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Tue Mar 04, 2014 5:00 pm

1) LSFS is not a "one size fits all" solution. It was designed to handle typical VM workload so indeed say video capture or streaming is not what it would handle well. FLAT would do better.

2) We don't allocate underlying disk space in the way you list. Page management goes from / to a big pool of prepared space chunks.

3) There's no direct relation between reported size and actual on-disk layout. Making long story short: LSFS volumes don't have any "size" property as they don't have logical block address space inside.

P.S. I understand.
AKnowles wrote:Actually, I have 4 VMs running on the datastore. I only printed the summary just to limit the amount of text to wade through, but if you want to see it all I can post it. There is no need for me to run more than 4 as 4 can max out the sequential bandwidth as it is. Even a single VM with a single NIC can pretty much max out an array for random I/O. As for sequential vs. random, well there are many instances of sequential I/O in the real world. Much is mitigated with VAAI and I'm happy to see it working and offloading the chore of provisioning and cloning VMs.

There are a lot of reasons to use a thin provisioned volume. I do it quite a bit, but performance is an issue with a thin volume as first the iSCSI target has to grown the LUN, the the block gets zeroed out, and finally the block of data gets written. So, I was curious. You can see the results.

Out of curiosity, how does your thin provisioning work with eager zero thick volumes? And when does the volume actually grow to full size? Will the vmfstools inflate it to full size? I'd like to do a test to see how a thin volume that has been fully populated will compare to a thick volume.

PS: I'm not trying to bash the product. Just reporting what I see.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
robnicholson
Posts: 359
Joined: Thu Apr 14, 2011 3:12 pm

Tue Mar 04, 2014 5:46 pm

It would be useful for adoption purposes to identify what the new features in Starwind are good for because I'm afraid that all I'm hearing so far is that it's not suitable for much that applies to us. The existing version of Starwind works very well for us and can you confirm what we're loosing with the new version? All I really wanted was a deduplication system that could handle large disks without lots of resources at an acceptable loss in performance plus handling trim on thin disks.

Cheers, Rob.
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Tue Mar 04, 2014 6:03 pm

"Lots of resources" - ?

5GB per 1TB of served data is what we have now for dedupe

There's no performance LOSS. There's a performance SHIFT with LSFS. So some workloads (random small writes) are ~10x faster, some (sequential big reads) are slower.

TRIM is supported with SCSI UNMAP or WRITE(0) (all zeros)
robnicholson wrote:It would be useful for adoption purposes to identify what the new features in Starwind are good for because I'm afraid that all I'm hearing so far is that it's not suitable for much that applies to us. The existing version of Starwind works very well for us and can you confirm what we're loosing with the new version? All I really wanted was a deduplication system that could handle large disks without lots of resources at an acceptable loss in performance plus handling trim on thin disks.

Cheers, Rob.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
robnicholson
Posts: 359
Joined: Thu Apr 14, 2011 3:12 pm

Tue Mar 04, 2014 9:56 pm

1TB is not a big disk these days even for a medium sized company. We've got 250 users and our SAN is currently 8.72TB primary (SAS) and 22TB secondary (SATA).

Cheers, Rob.
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Tue Mar 04, 2014 10:04 pm

That turns to ~40GB of RAM for 8TB of primary storage. Not a big deal (RAM is cheap these days).

Secondary storage should not be deduped as saving SATA space does not make any sense (or can be done with a free MS dedupe).
robnicholson wrote:1TB is not a big disk these days even for a medium sized company. We've got 250 users and our SAN is currently 8.72TB primary (SAS) and 22TB secondary (SATA).

Cheers, Rob.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
robnicholson
Posts: 359
Joined: Thu Apr 14, 2011 3:12 pm

Tue Mar 04, 2014 10:13 pm

Yes but it's actually on the secondary storage (archive/older data) where deduplication would be most useful. As I said before, it's not the end of world. There are backup systems which handle deduplication natively plus turning on dedupe on Windows 2012 is "free". I just thought that if you could offer a product that added deduplication to DPM, then it would be a USP.

Cheers, Rob.
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Tue Mar 04, 2014 10:21 pm

For ice cold data you don't need expensive in terms of CPU and RAM in-line deduplication.

We're primary storage oriented on all-flash we're not backup deduplication.
robnicholson wrote:Yes but it's actually on the secondary storage (archive/older data) where deduplication would be most useful. As I said before, it's not the end of world. There are backup systems which handle deduplication natively plus turning on dedupe on Windows 2012 is "free". I just thought that if you could offer a product that added deduplication to DPM, then it would be a USP.

Cheers, Rob.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
Post Reply