Starwind HA Cluster + HyperV

Fri Feb 11, 2011 5:05 pm

starneo wrote:great! *deleted* I'll upgrade both nodes now

One by one! otherwise both HA servers will be out of sync,
Install 5.6 on the second - wait for sync to finish - update first.
However if it's just a test environment - remove everything -> update everything-> recreate everything

hixont · Fri Feb 11, 2011 5:57 pm

Any idea when the Most Recent Version thread will be updated with the version 5.6 release news? I only know about newer release because this thread is of interest to me.

Fri Feb 11, 2011 10:30 pm

We did not have *official* V5.6 release yet. Scheduled for next week.

hixont wrote:Any idea when the Most Recent Version thread will be updated with the version 5.6 release news? I only know about newer release because this thread is of interest to me.

starneo · Tue Feb 15, 2011 11:03 am

I upgraded both starwind storages now and dird some testing.
The good news is, I had no BSOD / Starwind service crash till now.
The bad news is, I do now have performance issues.

For testing the performance of the cluster I run Sisoft Sandra 2011 in a VM, before the update to 5.6 I had ~470MByte Read (random and sequence) and ~240MByte write (random and sequence). Now I got ~470MByte read and 35MByte write. It is not only the benchmark showing this issue, during simpel filecopy I see the same, usually less then 10MByte.

For me it looks like an issue with the sync channel.

I will do some more testing, but I need some help on this.
I did not change something else but to update to 5.6.

starneo · Tue Feb 15, 2011 12:03 pm

I tested a full sync with one target, its very small because it contains the hypervquorum. Its 10GB.
I attached a screenshot of my syncperformance, as you see it is juming up and down in average 2-3% of 10Gbit.
Normaly it has to be 10% of 10Gbit, and it was ... sometimes.

*edit*

There is something else I recognized. In version 5.5 the RAM on both starwind nodes droped when I booted one. In 5.6 it keeps at max load.
And in 5.5 the RAM load increased when using the targets one by one, in 5.6 the RAM load goes to its max during the starwind startup.
I think this is intend to be so?

starneo · Tue Feb 15, 2011 2:16 pm

I now do a full sync of all 6 targets you can see below how slow it is. It should be at ~60% ...
I now have no more ideas ...
I need some help with this.

starneo · Tue Feb 15, 2011 2:59 pm

Okay guys, I found the solution for the sync problem.

I had to enable the Receive Side Scaling (RSS) in the Myricom 10GBe Network Card.
The RSS queue is 4.
The global settings posted here http://www.starwindsoftware.com/forums/ ... t2293.html did not work (I have set this settings directly after the first starwind installation!).
The settings were not set for the Myricom Network Adapter.

If you wish I can post all my network card settings for the sync adapter.

Tue Feb 15, 2011 3:19 pm

For sure, it would be great if you make the post!
Thanks!

starneo · Tue Feb 15, 2011 3:31 pm

To be clear, this resolved the sync speed issue, as far as I can tell for now.
If my performance issue is resolved I can test tomorrow - the full sync will taks ~8h.
Something else I noticed, I started the HyperV Cluster and 9 VMs on it. The full sync is still in progress.
Ths sync speed seems to be affected ny this, it drops a little bit during the VM start (to arround 40%) but is now at 50% again.

Okay here are my settings, these settings might be specific for myricom:

flow control = on
Interrupt coalescing delay = 25
IPv4 checksum offload = Rx enabled
Large send offload v2 (ipv4) = enabled
Large send offload v2 (ipv6) = enabled
log link state event = enabled
MTU = 9000
receive buffers = 2048
Receive side scaling (rss) = enabled
rss queues = 4
strip vlan tags = on
TCP & UDP checksum offload (ipv4) = rx & tx enabled
TCP & UDP checksum offload (ipv6) = rx & tx enabled

Tue Feb 15, 2011 3:47 pm

Thank you for the information and please keep us updated about further tests results.

mkaishar · Tue Feb 15, 2011 4:58 pm

Should flow control be enabled?
In reading numerous publications they state that flow control could slow down network performance.
So I usually disable all flow control on the SAN networks...curious if that is relevant?

Mon Feb 21, 2011 10:42 am

Flow Control disabling does not always give normal results with iSCSI. In my experience this was one of 6-7 infrastructures with nearly identic hardware.

mkaishar · Tue Mar 22, 2011 11:42 pm

Max (staff) wrote:Flow Control disabling does not always give normal results with iSCSI. In my experience this was one of 6-7 infrastructures with nearly identic hardware.

I am sorry I don't understand what you mean here.
Enabling it is a good idea with iSCSI?
Disabling it is a good idea with iSCSI?

Thanks,
Mark

Wed Mar 23, 2011 11:54 am

I meant that there is no 100% guarantee that flow control switching will help you. This option needs to be tested individually.