Test case - max Starwind througput

[djdactylo]

Hello,

Before any question, please let me introduce my testlab system.

Server 1 :
HP Proliant DL380 G5 (12Go RAM, Dual E5420 Xeon CPU)
- Storage controller 1 - Hp smart array P400 with 4x 72go 15K SAS RAID5 + 4x 72go 10K SAS RAID5
- Storage controller 2 - HP Smart Array P800 with dual Storageworks MSA60 arrays with RAID6 ADG (64ko stripe) volumes.
- OS : physical Windows Server 2008 R2
- Network : one NIC @ 1 Gbps for management + 4 NICs @ 1Gbps for iSCSI

Following setup for iSCSI NICs :
- All set to Jumbo frames 9000
- Each NIC with separated subnet (192.168.10.1, 192.168.20.1, ..)
- Each NIC isolated to the others by a VLAN on my D-link DGS-1210-16 switch
- Starwind cfg file edited for iscsidiscover

Following setup for Starwind v8:
- One single target
- One RAMdrive device of 2Go inside target

Server 2 :

HP Proliant DL380 G6 (36 Go RAM, Dual E5540 Xeon CPU)
- Storage controller 1 - Hp smart array P410i with 8x 72go 15K SAS RAID5
- OS : VMWARE ESXI 5.5U3 + one VM with Windows Server 2012R2 (4Go vRAM, 1vCPU)
- Network : exact same hardware configuration as server 1.
- ESXI virtual network setup is following one :

esxi.jpg (69.21 KiB) Viewed 4436 times

Please note that each Vmkernel NIC is linked to only one real NIC.

So, to test, I do following :
- I add the iSCSI drive directly to the VM containing Windows Server 2012R2.
- Inside vSphere client, I edit VM drive info to enable Circle Load on the ISCSI NICs
- Inside Windows Server, I put the dirve online and create an NTFS volume on it.
- I launch ATTO Disk benchmark to check throughput.

And... I do not understand why it is stuck on 120MB... It should go up to 300/400 Mbps. No ?

atto.jpg (68.71 KiB) Viewed 4436 times

I checked CPU : no any overload (even far from being very busy). I did check network on VMWare : all the NICs have exact same bandwith (seems load balance well working).

So, what did I miss ?

Thanks !

Rem : Sorry for maybe stupid questions. I am a newbie as on VMWare than on Starwind..

[anton (staff)]

1) CPU usage on StarWind server VM? Run PerfMon inside it. If stuck @ 100% try to increase vCPU.

2) How fast TCP can actually do? If NTTcp & IPerf from client -> server cannot do expected speed (wire, ~400-450 Mbps) then iSCSI (SCSI over TCP) cannot do more.

3) After making sure 1 and 2 are not an issue try to change IOPS setting from default 1000 to something smaller. 2 to 10 should be fine.

[djdactylo]

Hell Anton,

First, thanks for your fast answer.
I could make some testing yesterday but I did not have time to get printscreen of results.
I will try to add it this evening.

Anyway, here are some answers and new status :

Point 1 - CPU on Starwind VM :

First, Starwind is not on a VM but on a physical Windows 2008 R2 server. I hope there is no misunderstanding. Concerning it, the CPU is very quiet. I never go over 20-30% of CPU.

I also checked the initiator side (VMware Esxi 5.5 + Windows 2012 R2) and it seems great also. No any CPU overload.

Point 2 - Network performance :

I have four 1 Gbps links. I did not have time enough to experiment iperf testing. Anyway, as you requested it, I would like to test it. Not only for throughput but also to have an idea of latency and so, its impact on IOPS performance. I will try to come back on this by end of the week.

By checking again my network, I found two network cards were Jumbo frames setup was not correctly saved. I did correct it.

Point 3 - Round Robin IOPS :

I did change IOPS to 1 by following a VMware tutorial using Esxcli. Indeed, this improves a lot the bandwith. I could perform testing with good results showing up to 450 MB/s. It seems that we are on a good way.

I continue the investigation following your advice.

[anton (staff)]

OK, so sounds like this particular case is closed

Good luck and please keep us updated about your progress!

Hell Anton,

First, thanks for your fast answer.
I could make some testing yesterday but I did not have time to get printscreen of results.
I will try to add it this evening.

Anyway, here are some answers and new status :

Point 1 - CPU on Starwind VM :

First, Starwind is not on a VM but on a physical Windows 2008 R2 server. I hope there is no misunderstanding. Concerning it, the CPU is very quiet. I never go over 20-30% of CPU.

I also checked the initiator side (VMware Esxi 5.5 + Windows 2012 R2) and it seems great also. No any CPU overload.

Point 2 - Network performance :

I have four 1 Gbps links. I did not have time enough to experiment iperf testing. Anyway, as you requested it, I would like to test it. Not only for throughput but also to have an idea of latency and so, its impact on IOPS performance. I will try to come back on this by end of the week.

By checking again my network, I found two network cards were Jumbo frames setup was not correctly saved. I did correct it.

Point 3 - Round Robin IOPS :

I did change IOPS to 1 by following a VMware tutorial using Esxcli. Indeed, this improves a lot the bandwith. I could perform testing with good results showing up to 450 MB/s. It seems that we are on a good way.

I continue the investigation following your advice.