Page 1 of 2

Poor read performance from vsphere 5

Posted: Mon Aug 19, 2013 4:35 pm
by HostingTeam
Hi. I am trying version 6 of the iSCSI SAN free edition.

But when I am testing the performance I get poor read performance.
Write performance is wire speed.

I have been reading the forum but without luck and in a dead end.
Please help.

Re: Poor read performance from vsphere 5

Posted: Mon Aug 19, 2013 4:43 pm
by lohelle
Remember that all writes can be cached, and that would give wirespeed until the cache is full, but for reads data will have to be accessed from the underlying storage (all non-cached data).

You could try to disable Delayed ACK in the initiator, but I think write speed usually is hurt from that (sometimes).

Re: Poor read performance from vsphere 5

Posted: Mon Aug 19, 2013 5:21 pm
by HostingTeam
I know.
Locally on the SAN I have 300 MB/sec Read and 250 MB/sec write performance.

On the Virtuel machine I have 110 MB/sec write and 30 MB/sec read.
The read performance is OK when the transfer size is below 64 KB

Disabling the ACK make the read performance a little bit worse.

Re: Poor read performance from vsphere 5

Posted: Mon Aug 19, 2013 5:36 pm
by lohelle
Try creating a small ramdisk and check performance on that. Just to have something more to go on.

If using MPIO, have you tried using a single path? Jumbo frames vs no jumbo frames?

Re: Poor read performance from vsphere 5

Posted: Mon Aug 19, 2013 5:52 pm
by HostingTeam
Same performance when using a ramdisk.
I am not using Jumboframes and I am using a single path.

Re: Poor read performance from vsphere 5

Posted: Mon Aug 19, 2013 6:04 pm
by lohelle
Just throwing out ideas here.

Have you tried using a different nic on the host and/or on the Starwind server? Maybe you could test with a RAMdisk on a different computer.
If you have a unused nic on the vSphere host, you might want to try a direct connection to rule out switch problems.
I'm not saying that sone equipment is not working right, but sometimes computer/software problems are not logical.. :)

If your switches have some ISCSI optimization, try disabling that btw.

Re: Poor read performance from vsphere 5

Posted: Mon Aug 19, 2013 6:16 pm
by HostingTeam
Great ideas. But without luck.

I have previosly tried different SAN hardware, different ESX hardware, different NIC and different switches...

Re: Poor read performance from vsphere 5

Posted: Mon Aug 19, 2013 6:38 pm
by lohelle
Very strange.
There MUST be something special/different with your setup, as this is not happening with me/others. Tried other cables?

Have you tried benchmarking using MS ISCSI initiator in a VM?

Re: Poor read performance from vsphere 5

Posted: Mon Aug 19, 2013 7:32 pm
by HostingTeam
yes.

Then I get wirespeed. I get 100 MB/sec write and 100 MB/sec read.

Re: Poor read performance from vsphere 5

Posted: Mon Aug 19, 2013 9:06 pm
by anton (staff)
Nice. What do you use to check the performance? Can you run ATTO Disk Benchmark and post screenshots here? Thank you!

Re: Poor read performance from vsphere 5

Posted: Tue Aug 20, 2013 3:40 pm
by HostingTeam
I use ATTO.
I have attached 2 screenshots.

Re: Poor read performance from vsphere 5

Posted: Tue Aug 20, 2013 4:23 pm
by anton (staff)
It's definitely something related to the packet size as sub-64KB reads are fine. Let us take a closer look @ the issue...

Re: Poor read performance from vsphere 5

Posted: Tue Aug 20, 2013 4:49 pm
by HostingTeam
Are you able to help ?

Re: Poor read performance from vsphere 5

Posted: Tue Aug 20, 2013 6:12 pm
by anton (staff)
Yes absolutely! Techies will jump into this soon.
HostingTeam wrote:Are you able to help ?

Re: Poor read performance from vsphere 5

Posted: Thu Aug 22, 2013 11:12 pm
by jtmroczek
Tagging post to see outcome. We saw similar behavior with Windows initiators during bench-marking.In our tests, the problem seemed to worsen as the number of MPIO paths increased. For us, performance is good enough even with the degraded performance. We punted on the issue in the hopes microsoft or starwind would address the issue before we needed the performance on large blocks. We plan on retesting when V8 hits the streets.