VMXNET3 performance

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

Post Reply
pmaa
Posts: 7
Joined: Wed Nov 28, 2018 11:34 am

Wed Aug 28, 2019 10:17 am

I have servers with 40Gb Intel XL710 and 100GB Mellanox ConnectX-4 cards. I'm testing network performance with the included iperf in the Linux VSA and I'm only getting about ~30Gb when using VMXNET3 adapter. How should I tune this to get more speed? Or are Starwind processes better multithreaded than iperf, so I should expect more aggregated throughput like if testing with separate parallel iperf processes?

On another thread I read that for directly connected servers SR-IOV or DirectPath I/O aren't ideal due to the loss of link when the other node is shutdown or rebooted, so VMXNET3 seems like the way to go if I could get some more performance. From what I've read ESXi should be capable of 40Gb performance with VMXNET3. This will be for iSCSI traffic so ideally the 100Gb connection would be more than that as it's for the sync channel, even if I wouldn't be able to get the full 100Gb out of it. Would Starwind on Windows give higher throughput?
danswartz
Posts: 71
Joined: Fri May 03, 2019 7:21 pm

Wed Aug 28, 2019 6:20 pm

I use vmxnet3 vnics in general in my guests, but I find the performance to vary wildly from OS to OS. Still better than e1000, but...
pmaa
Posts: 7
Joined: Wed Nov 28, 2018 11:34 am

Thu Aug 29, 2019 8:57 am

In a Linux VM I get ~28 Gb quite consistently. I found some articles where they are able to push close to 40 Gb but that seems to require some tuning. Above that there doesn't seem to be much information around.

Anyone from Starwind care to comment on what kind of performance you have seen in internal testing?
Oleg(staff)
Staff
Posts: 568
Joined: Fri Nov 24, 2017 7:52 am

Thu Sep 05, 2019 9:45 am

Hi pmaa,
We got around 30 Gb/s during our internal testing of vmxnet3 adapters using Iperf.
pmaa
Posts: 7
Joined: Wed Nov 28, 2018 11:34 am

Thu Sep 26, 2019 8:38 pm

That's good to know. I can get a bit over that, but I won't try to tune it further as there doesn't seem to be much gain.
Oleg(staff)
Staff
Posts: 568
Joined: Fri Nov 24, 2017 7:52 am

Fri Sep 27, 2019 4:18 pm

Yes, you are right.
Post Reply