Page 1 of 1

VMXNET3 performance

Posted: Wed Aug 28, 2019 10:17 am
by pmaa
I have servers with 40Gb Intel XL710 and 100GB Mellanox ConnectX-4 cards. I'm testing network performance with the included iperf in the Linux VSA and I'm only getting about ~30Gb when using VMXNET3 adapter. How should I tune this to get more speed? Or are Starwind processes better multithreaded than iperf, so I should expect more aggregated throughput like if testing with separate parallel iperf processes?

On another thread I read that for directly connected servers SR-IOV or DirectPath I/O aren't ideal due to the loss of link when the other node is shutdown or rebooted, so VMXNET3 seems like the way to go if I could get some more performance. From what I've read ESXi should be capable of 40Gb performance with VMXNET3. This will be for iSCSI traffic so ideally the 100Gb connection would be more than that as it's for the sync channel, even if I wouldn't be able to get the full 100Gb out of it. Would Starwind on Windows give higher throughput?

Re: VMXNET3 performance

Posted: Wed Aug 28, 2019 6:20 pm
by danswartz
I use vmxnet3 vnics in general in my guests, but I find the performance to vary wildly from OS to OS. Still better than e1000, but...

Re: VMXNET3 performance

Posted: Thu Aug 29, 2019 8:57 am
by pmaa
In a Linux VM I get ~28 Gb quite consistently. I found some articles where they are able to push close to 40 Gb but that seems to require some tuning. Above that there doesn't seem to be much information around.

Anyone from Starwind care to comment on what kind of performance you have seen in internal testing?

Re: VMXNET3 performance

Posted: Thu Sep 05, 2019 9:45 am
by Oleg(staff)
Hi pmaa,
We got around 30 Gb/s during our internal testing of vmxnet3 adapters using Iperf.

Re: VMXNET3 performance

Posted: Thu Sep 26, 2019 8:38 pm
by pmaa
That's good to know. I can get a bit over that, but I won't try to tune it further as there doesn't seem to be much gain.

Re: VMXNET3 performance

Posted: Fri Sep 27, 2019 4:18 pm
by Oleg(staff)
Yes, you are right.