StarWind iSCSI SAN
StarWind Native SAN for Hyper-V
 

StarWind iSCSI SAN V5.7 Build 20110524

Public beta (bugs, reports, suggestions, features and requests)

Moderators: art (staff), anton (staff), Anatoly (staff), Max (staff)

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby rchisholm » Fri May 27, 2011 1:24 pm

The delayed ACK is turned off on the ESX servers.

This is an SQLIO on one of the StarWind servers not using file or disk cache to the 24 drive 7.2K RAID 60:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -Fparam.txt
sqlio v1.5.SG
parameter file used: param.txt
file e:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file e:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
size of file e:\testfile.dat needs to be: 4194304000 bytes
current file size: 1048576000 bytes
need to expand by: 3145728000 bytes
expanding e:\testfile.dat ... done.
using specified size: 4000 MB for file: e:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 13099.62
MBs/sec: 818.72

This is it with caching turned on:

C:\Program Files (x86)\SQLIO>sqlio -kW -BY -o64 -fsequential -b64 -Fparam.txt
sqlio v1.5.SG
parameter file used: param.txt
file e:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file e:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to use both file and disk caches
using specified size: 4000 MB for file: e:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 82755.73
MBs/sec: 5172.23

Without cache to 40 drive 15K RAID 60:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -Fparam.txt
sqlio v1.5.SG
parameter file used: param.txt
file f:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file f:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
size of file f:\testfile.dat needs to be: 4194304000 bytes
current file size: 0 bytes
need to expand by: 4194304000 bytes
expanding f:\testfile.dat ... done.
using specified size: 4000 MB for file: f:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 18956.30
MBs/sec: 1184.76

With cache to 40 drive 15K RAID 60:

C:\Program Files (x86)\SQLIO>sqlio -kW -BY -o64 -fsequential -b64 -Fparam.txt
sqlio v1.5.SG
parameter file used: param.txt
file f:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file f:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to use both file and disk caches
using specified size: 4000 MB for file: f:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 82983.90
MBs/sec: 5186.49

I'm going to have to do some reconfiguring to test a non virtual system with 10GbE iSCSI HA writes and don't think I have time to get that done today.

Bohdan (staff) wrote:Is it possible to perform the tests with SQLIO and iometer (64K, 64 outstanding, 8 or 16 threads/workers)?
On 10Gb cards (Intel AT) we are observing 70% sync channel utilization and 850MB/s. The difference is that we are running the tests on the physical machine but not the virtual one.

And one more question: have you tried to enable/disable delayed ACK option on the ESX side?
starwind-f5/esxi-iscsi-initiator-write-speed-t2398-15.html
rchisholm
 
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby Bohdan (staff) » Fri May 27, 2011 1:41 pm

These are the results for the local storage. Could you please also run the tests with the same parameters within the virtual machine? I mean "in a VSphere 2008 R2 VM to 1 HA target" as you mentioned above.
User avatar
Bohdan (staff)
Staff
 
Posts: 437
Joined: Wed May 23, 2007 12:58 pm

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby rchisholm » Fri May 27, 2011 2:14 pm

Here is one of the VM's:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -Fparam.txt
sqlio v1.5.SG
parameter file used: param.txt
file d:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file d:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 4000 MB for file: d:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 4619.89
MBs/sec: 288.74

This is actually completely acceptable performance for these individual VM's for what they will be used for, but 2-3X faster wouldn't be a bad thing.


Bohdan (staff) wrote:These are the results for the local storage. Could you please also run the tests with the same parameters within the virtual machine? I mean "in a VSphere 2008 R2 VM to 1 HA target" as you mentioned above.
rchisholm
 
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby rchisholm » Fri May 27, 2011 2:21 pm

I just lowered the size of the test file from 4 GB to 1GB so it would match the StarWind Cache for the target and got these numbers:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -Fparam.txt
sqlio v1.5.SG
parameter file used: param.txt
file d:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file d:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 1000 MB for file: d:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 8124.15
MBs/sec: 507.75
rchisholm
 
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby Bohdan (staff) » Fri May 27, 2011 2:34 pm

And what are the results for HA target with no StarWind caching? :roll:
User avatar
Bohdan (staff)
Staff
 
Posts: 437
Joined: Wed May 23, 2007 12:58 pm

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby rchisholm » Fri May 27, 2011 2:44 pm

They were so bad before when I first tested them that I haven't bothered setting up any targets without write back caching. That's why there is 72GB RAM in each StarWind box. 8)

Bohdan (staff) wrote:And what are the results for HA target with no StarWind caching? :roll:
rchisholm
 
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby anton (staff) » Fri May 27, 2011 8:20 pm

Picture should be different for V5.7 and especially upcoming V5.8 versions. Could you please give an experimental try to non-cached I/O with your current V5.7 beta? Real-world feedback is precious for us. Thank you very much for cooperation!

rchisholm wrote:They were so bad before when I first tested them that I haven't bothered setting up any targets without write back caching. That's why there is 72GB RAM in each StarWind box. 8)

Bohdan (staff) wrote:And what are the results for HA target with no StarWind caching? :roll:
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
User avatar
anton (staff)
Site Admin
 
Posts: 3922
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby rchisholm » Fri May 27, 2011 8:40 pm

No problem. I'll set it up and test it.
rchisholm
 
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby anton (staff) » Sun May 29, 2011 1:32 pm

Your feedback and help are greatly appreciated!

rchisholm wrote:No problem. I'll set it up and test it.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
User avatar
anton (staff)
Site Admin
 
Posts: 3922
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby rchisholm » Tue May 31, 2011 2:49 pm

Ok, you guys definitely made some changes to the writes with no cache.

These are a VMware 4 VCPU 3.33GHZ with 16GB RAM to a 250GB HA target set to no StarWind caching. Round robin with default 1000 IOPS.


C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o8 -fsequential -b64 -LS -Fparam.txt

sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file g:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file g:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 8 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: g:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 5844.36
MBs/sec: 365.27
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 10
Max_Latency(ms): 81
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 1 3 5 8 9 10 10 9 7 5 5 4 3 3 2 2 2 1 1 1 1 1 1 7

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o32 -fsequential -b64 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file g:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file g:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 32 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: g:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 5959.59
MBs/sec: 372.47
latency metrics:
Min_Latency(ms): 4
Avg_Latency(ms): 42
Max_Latency(ms): 144
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file g:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file g:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: g:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 5896.69
MBs/sec: 368.54
latency metrics:
Min_Latency(ms): 7
Avg_Latency(ms): 85
Max_Latency(ms): 192
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100

Getting ready to test with 4GB write cache now.
rchisholm
 
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby rchisholm » Tue May 31, 2011 4:12 pm

Here are the same set of tests but with 4GB write back cache

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o8 -fsequential -b64 -LS -Fparam.txt

sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 8 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 8509.59
MBs/sec: 531.84
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 7
Max_Latency(ms): 76
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 3 6 8 12 14 13 11 9 7 4 3 2 2 1 1 1 1 1 0 0 0 0 0 2

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o32 -fsequential -b64 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 32 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 7344.65
MBs/sec: 459.04
latency metrics:
Min_Latency(ms): 7
Avg_Latency(ms): 34
Max_Latency(ms): 4321
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 6 91

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 8448.80
MBs/sec: 528.05
latency metrics:
Min_Latency(ms): 3
Avg_Latency(ms): 60
Max_Latency(ms): 187
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100

Here's something I found interesting... It really doesn't like the 32 outstanding. Sometimes the latency will go through the roof and it will run much slower. It is doing this both with and without StarWind write cache.

Example of slow 32 outstanding with cache:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o32 -fsequential -b64 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 32 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 3896.58
MBs/sec: 243.53
latency metrics:
Min_Latency(ms): 6
Avg_Latency(ms): 65
Max_Latency(ms): 9603
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 5 92

Example of slow 32 outstanding without cache:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o32 -fsequential -b64 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file g:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file g:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 32 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: g:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 3363.34
MBs/sec: 210.20
latency metrics:
Min_Latency(ms): 8
Avg_Latency(ms): 75
Max_Latency(ms): 2688
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100
rchisholm
 
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby rchisholm » Tue May 31, 2011 4:39 pm

Wow, look at this. I was just fooling around and did a RANDOM 64K write to 3 2GB files with 1 on each of the test targets. Over 6600 IOPS and 400 MB/s with an average of 6 ms latency!

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o16 -frandom -b64 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file d:\testfile.dat with 1 thread (0) using mask 0x0 (0)
file g:\testfile.dat with 1 thread (1) using mask 0x0 (0)
file h:\testfile.dat with 1 thread (2) using mask 0x0 (0)
3 threads writing for 30 secs to files d:\testfile.dat, g:\testfile.dat and h:\t
estfile.dat
using 64KB random IOs
enabling multiple I/Os per thread with 16 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: d:\testfile.dat
using specified size: 2000 MB for file: g:\testfile.dat
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 6601.42
MBs/sec: 412.58
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 6
Max_Latency(ms): 5242
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 1 45 35 8 3 2 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1

Then I tried a 4K random, like many use for their IOPS numbers, and got well over 11K IOPS at an average of 3 ms latency.

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o16 -frandom -b4 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file d:\testfile.dat with 1 thread (0) using mask 0x0 (0)
file g:\testfile.dat with 1 thread (1) using mask 0x0 (0)
file h:\testfile.dat with 1 thread (2) using mask 0x0 (0)
3 threads writing for 30 secs to files d:\testfile.dat, g:\testfile.dat and h:\t
estfile.dat
using 4KB random IOs
enabling multiple I/Os per thread with 16 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: d:\testfile.dat
using specified size: 2000 MB for file: g:\testfile.dat
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 11853.46
MBs/sec: 46.30
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 3
Max_Latency(ms): 479
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 24 50 15 5 2 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2

Not too bad for 24 7.2K SAS drives in a RAID60 across iSCSI from a virtual server without even doing any tweaking. :D
rchisholm
 
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby rchisholm » Tue May 31, 2011 5:41 pm

Definitely want to make sure that you crank the IOPS down from the default 1000 for ESX round robin.

Here are the same original 3 tests to the target with 4GB cache with IOPS set to 1.

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o8 -fsequential -b64 -LS -Fparam.txt

sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 8 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 12450.63
MBs/sec: 778.16
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 4
Max_Latency(ms): 318
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 16 26 14 9 6 5 4 4 3 3 3 2 2 1 1 1 0 0 0 0 0 0 0 0

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o32 -fsequential -b64 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 32 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 12448.10
MBs/sec: 778.00
latency metrics:
Min_Latency(ms): 2
Avg_Latency(ms): 20
Max_Latency(ms): 1349
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 12 23 15 10 7 6 4 4 18

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 12378.04
MBs/sec: 773.62
latency metrics:
Min_Latency(ms): 3
Avg_Latency(ms): 40
Max_Latency(ms): 164
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100
rchisholm
 
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby rchisholm » Tue May 31, 2011 6:30 pm

Here are some numbers for random writes to the single 4GB cache HA target. :shock:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o16 -frandom -b64 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB random IOs
enabling multiple I/Os per thread with 16 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 12374.32
MBs/sec: 773.39
latency metrics:
Min_Latency(ms): 4
Avg_Latency(ms): 9
Max_Latency(ms): 4875
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 21 29 14 7 4 2 2 2 2 2 3 4 3 2 1 1 0 0 0

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o16 -frandom -b4 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 4KB random IOs
enabling multiple I/Os per thread with 16 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 24028.66
MBs/sec: 93.86
latency metrics:
Min_Latency(ms): 2
Avg_Latency(ms): 4
Max_Latency(ms): 83
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 2 44 17 9 7 6 5 4 2 1 0 0 0 0 0 0 0 0 0 0 0 0 0

Of course, random read is a completely different story. I'm going to play with the 15K RAIDS with the SSD read cache next.

EDIT:
Decided to to some more testing with the 7.2K's first. Seems to be all about cache on the random reads too.

Running the tests a second time was very different on the volume with 4GB StarWind Cache.

C:\Program Files (x86)\SQLIO>sqlio -kR -BN -o16 -frandom -b64 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads reading for 30 secs from file h:\testfile.dat
using 64KB random IOs
enabling multiple I/Os per thread with 16 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 11858.44
MBs/sec: 741.15
latency metrics:
Min_Latency(ms): 1
Avg_Latency(ms): 10
Max_Latency(ms): 621
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 1 4 19 27 24 15 7 2 1 0 0 0 0 0 0 0 0 0 0

C:\Program Files (x86)\SQLIO>sqlio -kR -BN -o16 -frandom -b4 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads reading for 30 secs from file h:\testfile.dat
using 4KB random IOs
enabling multiple I/Os per thread with 16 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 91807.23
MBs/sec: 358.62
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 1
Max_Latency(ms): 18
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 1 97 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

This could get very interesting with a couple boxes with huge amounts of RAM. 8)
rchisholm
 
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Re: StarWind iSCSI SAN V5.7 Build 20110524

Postby anton (staff) » Thu Jun 02, 2011 10:28 am

Wow! These are real good numbers! Congratulations. And you guys are not greedy when it comes to buying new hardware :) I'm jealous.

Yes, we've definitely "did some changes to non-cached I/O" as well :)

SQLIO is nice and it does indeed pop up numbers SQL Server should hit (sick!) but... Do we have any chance to see Intel I/O Meter and Intel NAS Performance Toolkit test run results as well?

Thank you very much for cooperation!
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
User avatar
anton (staff)
Site Admin
 
Posts: 3922
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands

PreviousNext

Return to Beta

Who is online

Users browsing this forum: No registered users and 3 guests