Performance testing iperf3 issue.

[rick-dws]

New setup is as follows.
2 nodes
Both Supermicro AS-2126HS-TN
Dual AMD EPYV 9255 24 core cpu's
Dual AIOM 100gb mellenox x-6 nics
1.5TB Ram
3.82 x 24 PCIE 5 nvme drives

Using the current CVM (iscsi) build.

Performance compared to the vmware vsan esa solution is pretty terrible. I can get the bandwidth to comparable levels, but iops are not close.

I am testing the new rdma release which is a bit better, but still not close.

Some strange things I have found.

Using 2 test linux boxes, 1 per host, iperf3 will max out the 100gb nic 99-100gb/s.

Using the Starwind CVM's, no lun's created yet, i am able to max out when using one of the test machines as a server.

However when I use either of the SW CVM's as a server the results max at ~70gb/s, but range anywhere from 50-70gb/s.

MTU is set to 9000 on all switches/adapters

I have tried this with the system using virtual nic's as well as SR-IOV passthrough.

Any ideas?

Thanks

[yaroslav (staff)]

I guess that's more a question to iperf3 and how the tests are run. You are welcome to install iperf2 or later iperf3 and repeat the tests.

[rick-dws]

This is using iperf3, same version across all 4 servers.

Also using HCI Bench, bandwidth will max out ~4-5GB/s and iops at 150,000.

Using vmware vsan, bandwidth hits 7-8GB/s and iops over 600,000.

Could I setup a call to have an engineer go through the setup?

FYI this is the paid version under support, not sure if that matters.

Thanks

[yaroslav (staff)]

It does not relate to VSAN... just yet. Your request relates to a third-party solution that comes pre-installed with CVM. Check this out https://forums.developer.nvidia.com/t/m ... ore/333917
It looks like your question relates to the networking, iperf in specific, as you still have no LUNs, so it is too early to judge on the solution's performance setup is not completed yet.
iperf3 that comes with StarWind CVM might have issues related to multithreaded performance test (you need multiple threads to max out network performance).
Install previous versions of iperf and use the commands from here https://www.starwindsoftware.com/best-p ... practices/

Speaking of the call, free users cannot set up the call. If you are trialing or a part of NVMe-oF beta, you are welcome to reach out to the person who shared the key with you to set up a call.

[rick-dws]

At this point I have done extensive testing using vmware vsan, starwind iscsi, and starwind nvme-of.

I have a ton of benchmark results and still do not appear to get the desired performance.

How would I go about initiating a call?

I DO have licensed support, as we have uses this solution for 5+ years now on our old servers.

Thanks

[yaroslav (staff)]

I am afraid there's some misunderstanding here.
You mentioned that you did not set up LUN's yet. This means that there's nothing to benchmark but underlying storage and networking. Furthermore, NVMe-oF is available only as part of the beta, and you need a special key for that. This being said, it's too early to judge on the solution's performance as the solution's testing has not been fully completed yet.
Based on the context, you are benchmarking the network. Please review my earlier comments.
If you have an active support contract, you can log a case with us https://www.starwindsoftware.com/support-form.

[rick-dws]

Sorry if this is confusing..

I have configured this setup many times at this point, lun's have been setup multiple times.

I was not getting what I think is the proper performance, so I went back to square one and started with the nic testing. Thus the current issue I am seeing.

I have the beta license for nvme-of as well and have tested that fully.

I have a license for the normal version and support.

I will use the link to start a support request.

Thanks

[yaroslav (staff)]

No worries
If it is for a beta setup, please reach out to the person who shared the key with you (they will bring you in contact with the tech).
If this request is for the commercial key, please reach out to us via form.