Strange dip while file transfer over 10GBit

[kkit]

Hi,

I am experiencing strange dips in performance when transferring a 10GB sample file via SMB between the fileserver, which is a Windows Server 2019 Datacenter and the latest StarWind for HC Storage.

The dip always occurs at around 6GB has been transferred (so around 4GB left), dips do not seem to occure within the storage network with jumbo frames.
We also use one 1.5TB PCIe SSD as write-through SSD-cache as well as 4GB of RAM Cache.

Specs:
2x HP DL360 Gen9 Servers with 100TB in RAID6
HP Smart Array P840 SAS Controller

Edit: Storage:
RAID is 256 KiB (We have mixed file sizes - from Excel files to Cinema 4D and Creative Cloud of up to 50GB in size)
Windows Filesystem is ReFS and 64 KiB (again, for large I/O workloads)
StarWind Storage Block Size is 4096 Byte (chosen as recommended by StarWind Guidelines)


Is there anything I can take a look into for starters, maybe you have a suggestion.

Many thanks in advance--
kkit

[yaroslav (staff)]

Hi,

File copy is not the best performance benchmark as it gets affected by system buffering. Tweaks that come to my mind
1. Write-back cache on RAID controller. You can also disable RAM cache as SSDs generally do not need it. How to https://knowledgebase.starwindsoftware. ... -l1-cache/
2. Try local-to-local iSCSI connection as the preferred path. Say, iSCSI is 172.16.10.10 and 172.16.10.20 on your hosts. In iSCSI Discovery, configure 172.16.10.10 as initiator and 172.16.10.10 as the target. Do the same for another server. Set MPIO to Round Robin with Subset and use the local-to-local connection as the preferred one while 127.0.0.1 is set to standby.
3. Registry tweak (should do most of the job, actually). Go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4d36e97b-e325-11ce-bfc1-08002be10318}\000X\Parameters and set FirstBurstLength to 262144.

Also, try copying from one disk to another within one machine. 743 MB/s is not that slow.

[kkit]

Hi,

File copy is not the best performance benchmark as it gets affected by system buffering. Tweaks that come to my mind
1. Write-back cache on RAID controller. You can also disable RAM cache as SSDs generally do not need it. How to https://knowledgebase.starwindsoftware. ... -l1-cache/
2. Try local-to-local iSCSI connection as the preferred path. Say, iSCSI is 172.16.10.10 and 172.16.10.20 on your hosts. In iSCSI Discovery, configure 172.16.10.10 as initiator and 172.16.10.10 as the target. Do the same for another server. Set MPIO to Round Robin with Subset and use the local-to-local connection as the preferred one while 127.0.0.1 is set to standby.
3. Registry tweak (should do most of the job, actually). Go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4d36e97b-e325-11ce-bfc1-08002be10318}\000X\Parameters and set FirstBurstLength to 262144.

Also, try copying from one disk to another within one machine. 743 MB/s is not that slow.

Hello yaroslav,

Thank you very much for taking the time to make suggestions, will definitely try them out and let you know!

Stay safe and kind regards
kkit

[yaroslav (staff)]

Greetings,

Please keep me posted.

[kkit]

Greetings,

Please keep me posted.

Hello Yaroslav,

first of all thank you very much for your suggestions and sorry for the late response, I didn't have time to check for a while.

As you requested:
After disabling and setting your recommended parameters, I unfortunately do still have the stall but managed to isolated the issue to be a network issue.
Apart from that your configured actually helped me further push the transfer speeds a good bit!


Identical setup and server - only settings changed.



Thank you very much, you have been a great help

best wishes
kkit

[yaroslav (staff)]

Great news!! Thanks for sharing your feedback.