StarWind iSCSI SAN V5.7 Build 20110524

Public beta (bugs, reports, suggestions, features and requests)

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Fri May 27, 2011 1:24 pm

The delayed ACK is turned off on the ESX servers.

This is an SQLIO on one of the StarWind servers not using file or disk cache to the 24 drive 7.2K RAID 60:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -Fparam.txt
sqlio v1.5.SG
parameter file used: param.txt
file e:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file e:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
size of file e:\testfile.dat needs to be: 4194304000 bytes
current file size: 1048576000 bytes
need to expand by: 3145728000 bytes
expanding e:\testfile.dat ... done.
using specified size: 4000 MB for file: e:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 13099.62
MBs/sec: 818.72

This is it with caching turned on:

C:\Program Files (x86)\SQLIO>sqlio -kW -BY -o64 -fsequential -b64 -Fparam.txt
sqlio v1.5.SG
parameter file used: param.txt
file e:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file e:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to use both file and disk caches
using specified size: 4000 MB for file: e:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 82755.73
MBs/sec: 5172.23

Without cache to 40 drive 15K RAID 60:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -Fparam.txt
sqlio v1.5.SG
parameter file used: param.txt
file f:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file f:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
size of file f:\testfile.dat needs to be: 4194304000 bytes
current file size: 0 bytes
need to expand by: 4194304000 bytes
expanding f:\testfile.dat ... done.
using specified size: 4000 MB for file: f:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 18956.30
MBs/sec: 1184.76

With cache to 40 drive 15K RAID 60:

C:\Program Files (x86)\SQLIO>sqlio -kW -BY -o64 -fsequential -b64 -Fparam.txt
sqlio v1.5.SG
parameter file used: param.txt
file f:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file f:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to use both file and disk caches
using specified size: 4000 MB for file: f:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 82983.90
MBs/sec: 5186.49

I'm going to have to do some reconfiguring to test a non virtual system with 10GbE iSCSI HA writes and don't think I have time to get that done today.
Bohdan (staff) wrote:Is it possible to perform the tests with SQLIO and iometer (64K, 64 outstanding, 8 or 16 threads/workers)?
On 10Gb cards (Intel AT) we are observing 70% sync channel utilization and 850MB/s. The difference is that we are running the tests on the physical machine but not the virtual one.

And one more question: have you tried to enable/disable delayed ACK option on the ESX side?
http://www.starwindsoftware.com/forums/ ... 98-15.html
User avatar
Bohdan (staff)
Staff
Posts: 435
Joined: Wed May 23, 2007 12:58 pm

Fri May 27, 2011 1:41 pm

These are the results for the local storage. Could you please also run the tests with the same parameters within the virtual machine? I mean "in a VSphere 2008 R2 VM to 1 HA target" as you mentioned above.
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Fri May 27, 2011 2:14 pm

Here is one of the VM's:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -Fparam.txt
sqlio v1.5.SG
parameter file used: param.txt
file d:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file d:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 4000 MB for file: d:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 4619.89
MBs/sec: 288.74

This is actually completely acceptable performance for these individual VM's for what they will be used for, but 2-3X faster wouldn't be a bad thing.

Bohdan (staff) wrote:These are the results for the local storage. Could you please also run the tests with the same parameters within the virtual machine? I mean "in a VSphere 2008 R2 VM to 1 HA target" as you mentioned above.
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Fri May 27, 2011 2:21 pm

I just lowered the size of the test file from 4 GB to 1GB so it would match the StarWind Cache for the target and got these numbers:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -Fparam.txt
sqlio v1.5.SG
parameter file used: param.txt
file d:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file d:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 1000 MB for file: d:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 8124.15
MBs/sec: 507.75
User avatar
Bohdan (staff)
Staff
Posts: 435
Joined: Wed May 23, 2007 12:58 pm

Fri May 27, 2011 2:34 pm

And what are the results for HA target with no StarWind caching? :roll:
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Fri May 27, 2011 2:44 pm

They were so bad before when I first tested them that I haven't bothered setting up any targets without write back caching. That's why there is 72GB RAM in each StarWind box. 8)
Bohdan (staff) wrote:And what are the results for HA target with no StarWind caching? :roll:
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Fri May 27, 2011 8:20 pm

Picture should be different for V5.7 and especially upcoming V5.8 versions. Could you please give an experimental try to non-cached I/O with your current V5.7 beta? Real-world feedback is precious for us. Thank you very much for cooperation!
rchisholm wrote:They were so bad before when I first tested them that I haven't bothered setting up any targets without write back caching. That's why there is 72GB RAM in each StarWind box. 8)
Bohdan (staff) wrote:And what are the results for HA target with no StarWind caching? :roll:
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Fri May 27, 2011 8:40 pm

No problem. I'll set it up and test it.
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Sun May 29, 2011 1:32 pm

Your feedback and help are greatly appreciated!
rchisholm wrote:No problem. I'll set it up and test it.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Tue May 31, 2011 2:49 pm

Ok, you guys definitely made some changes to the writes with no cache.

These are a VMware 4 VCPU 3.33GHZ with 16GB RAM to a 250GB HA target set to no StarWind caching. Round robin with default 1000 IOPS.


C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o8 -fsequential -b64 -LS -Fparam.txt

sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file g:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file g:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 8 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: g:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 5844.36
MBs/sec: 365.27
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 10
Max_Latency(ms): 81
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 1 3 5 8 9 10 10 9 7 5 5 4 3 3 2 2 2 1 1 1 1 1 1 7

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o32 -fsequential -b64 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file g:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file g:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 32 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: g:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 5959.59
MBs/sec: 372.47
latency metrics:
Min_Latency(ms): 4
Avg_Latency(ms): 42
Max_Latency(ms): 144
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file g:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file g:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: g:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 5896.69
MBs/sec: 368.54
latency metrics:
Min_Latency(ms): 7
Avg_Latency(ms): 85
Max_Latency(ms): 192
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100

Getting ready to test with 4GB write cache now.
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Tue May 31, 2011 4:12 pm

Here are the same set of tests but with 4GB write back cache

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o8 -fsequential -b64 -LS -Fparam.txt

sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 8 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 8509.59
MBs/sec: 531.84
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 7
Max_Latency(ms): 76
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 3 6 8 12 14 13 11 9 7 4 3 2 2 1 1 1 1 1 0 0 0 0 0 2

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o32 -fsequential -b64 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 32 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 7344.65
MBs/sec: 459.04
latency metrics:
Min_Latency(ms): 7
Avg_Latency(ms): 34
Max_Latency(ms): 4321
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 6 91

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 8448.80
MBs/sec: 528.05
latency metrics:
Min_Latency(ms): 3
Avg_Latency(ms): 60
Max_Latency(ms): 187
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100

Here's something I found interesting... It really doesn't like the 32 outstanding. Sometimes the latency will go through the roof and it will run much slower. It is doing this both with and without StarWind write cache.

Example of slow 32 outstanding with cache:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o32 -fsequential -b64 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 32 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 3896.58
MBs/sec: 243.53
latency metrics:
Min_Latency(ms): 6
Avg_Latency(ms): 65
Max_Latency(ms): 9603
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 5 92

Example of slow 32 outstanding without cache:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o32 -fsequential -b64 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file g:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file g:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 32 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: g:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 3363.34
MBs/sec: 210.20
latency metrics:
Min_Latency(ms): 8
Avg_Latency(ms): 75
Max_Latency(ms): 2688
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Tue May 31, 2011 4:39 pm

Wow, look at this. I was just fooling around and did a RANDOM 64K write to 3 2GB files with 1 on each of the test targets. Over 6600 IOPS and 400 MB/s with an average of 6 ms latency!

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o16 -frandom -b64 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file d:\testfile.dat with 1 thread (0) using mask 0x0 (0)
file g:\testfile.dat with 1 thread (1) using mask 0x0 (0)
file h:\testfile.dat with 1 thread (2) using mask 0x0 (0)
3 threads writing for 30 secs to files d:\testfile.dat, g:\testfile.dat and h:\t
estfile.dat
using 64KB random IOs
enabling multiple I/Os per thread with 16 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: d:\testfile.dat
using specified size: 2000 MB for file: g:\testfile.dat
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 6601.42
MBs/sec: 412.58
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 6
Max_Latency(ms): 5242
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 1 45 35 8 3 2 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1

Then I tried a 4K random, like many use for their IOPS numbers, and got well over 11K IOPS at an average of 3 ms latency.

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o16 -frandom -b4 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file d:\testfile.dat with 1 thread (0) using mask 0x0 (0)
file g:\testfile.dat with 1 thread (1) using mask 0x0 (0)
file h:\testfile.dat with 1 thread (2) using mask 0x0 (0)
3 threads writing for 30 secs to files d:\testfile.dat, g:\testfile.dat and h:\t
estfile.dat
using 4KB random IOs
enabling multiple I/Os per thread with 16 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: d:\testfile.dat
using specified size: 2000 MB for file: g:\testfile.dat
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 11853.46
MBs/sec: 46.30
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 3
Max_Latency(ms): 479
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 24 50 15 5 2 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2

Not too bad for 24 7.2K SAS drives in a RAID60 across iSCSI from a virtual server without even doing any tweaking. :D
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Tue May 31, 2011 5:41 pm

Definitely want to make sure that you crank the IOPS down from the default 1000 for ESX round robin.

Here are the same original 3 tests to the target with 4GB cache with IOPS set to 1.

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o8 -fsequential -b64 -LS -Fparam.txt

sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 8 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 12450.63
MBs/sec: 778.16
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 4
Max_Latency(ms): 318
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 16 26 14 9 6 5 4 4 3 3 3 2 2 1 1 1 0 0 0 0 0 0 0 0

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o32 -fsequential -b64 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 32 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 12448.10
MBs/sec: 778.00
latency metrics:
Min_Latency(ms): 2
Avg_Latency(ms): 20
Max_Latency(ms): 1349
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 12 23 15 10 7 6 4 4 18

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o64 -fsequential -b64 -LS -Fparam.tx
t
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB sequential IOs
enabling multiple I/Os per thread with 64 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 12378.04
MBs/sec: 773.62
latency metrics:
Min_Latency(ms): 3
Avg_Latency(ms): 40
Max_Latency(ms): 164
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Tue May 31, 2011 6:30 pm

Here are some numbers for random writes to the single 4GB cache HA target. :shock:

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o16 -frandom -b64 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 64KB random IOs
enabling multiple I/Os per thread with 16 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 12374.32
MBs/sec: 773.39
latency metrics:
Min_Latency(ms): 4
Avg_Latency(ms): 9
Max_Latency(ms): 4875
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 21 29 14 7 4 2 2 2 2 2 3 4 3 2 1 1 0 0 0

C:\Program Files (x86)\SQLIO>sqlio -kW -BN -o16 -frandom -b4 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads writing for 30 secs to file h:\testfile.dat
using 4KB random IOs
enabling multiple I/Os per thread with 16 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 24028.66
MBs/sec: 93.86
latency metrics:
Min_Latency(ms): 2
Avg_Latency(ms): 4
Max_Latency(ms): 83
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 2 44 17 9 7 6 5 4 2 1 0 0 0 0 0 0 0 0 0 0 0 0 0

Of course, random read is a completely different story. I'm going to play with the 15K RAIDS with the SSD read cache next.

EDIT:
Decided to to some more testing with the 7.2K's first. Seems to be all about cache on the random reads too.

Running the tests a second time was very different on the volume with 4GB StarWind Cache.

C:\Program Files (x86)\SQLIO>sqlio -kR -BN -o16 -frandom -b64 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads reading for 30 secs from file h:\testfile.dat
using 64KB random IOs
enabling multiple I/Os per thread with 16 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 11858.44
MBs/sec: 741.15
latency metrics:
Min_Latency(ms): 1
Avg_Latency(ms): 10
Max_Latency(ms): 621
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 0 0 0 0 0 0 1 4 19 27 24 15 7 2 1 0 0 0 0 0 0 0 0 0 0

C:\Program Files (x86)\SQLIO>sqlio -kR -BN -o16 -frandom -b4 -LS -Fparam.txt
sqlio v1.5.SG
using system counter for latency timings, 3579545 counts per second
parameter file used: param.txt
file h:\testfile.dat with 8 threads (0-7) using mask 0x0 (0)
8 threads reading for 30 secs from file h:\testfile.dat
using 4KB random IOs
enabling multiple I/Os per thread with 16 outstanding
buffering set to not use file nor disk caches (as is SQL Server)
using specified size: 2000 MB for file: h:\testfile.dat
initialization done
CUMULATIVE DATA:
throughput metrics:
IOs/sec: 91807.23
MBs/sec: 358.62
latency metrics:
Min_Latency(ms): 0
Avg_Latency(ms): 1
Max_Latency(ms): 18
histogram:
ms: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24+
%: 1 97 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

This could get very interesting with a couple boxes with huge amounts of RAM. 8)
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Thu Jun 02, 2011 10:28 am

Wow! These are real good numbers! Congratulations. And you guys are not greedy when it comes to buying new hardware :) I'm jealous.

Yes, we've definitely "did some changes to non-cached I/O" as well :)

SQLIO is nice and it does indeed pop up numbers SQL Server should hit (sick!) but... Do we have any chance to see Intel I/O Meter and Intel NAS Performance Toolkit test run results as well?

Thank you very much for cooperation!
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
Post Reply