Replication network unavailable

[MrSquish]

I am trying to setup a 2 node hyper-v using the linux CVM guest VM's following this article:
https://www.starwindsoftware.com/resour ... ng-web-ui/

I have 2 NIC's directly connected to each other, as well as 2 NIC's connected through a switch. Both of them give me this error when running through the "Configure HA networking" wizard. From the console of both CVM's, I can ping all the interfaces of each host between each other, i'm not sure what's wrong with the config resulting in the error saying it can't ping...

[yaroslav (staff)]

Hi,

Do you have sriov on them?

[MrSquish]

Yes, I enabled sr-iov. I installed some Mellanox connectx-5 dual 25Gb cards. As well as updated the firmware and enabled sr-iov on the 10gig daughter card.

[yaroslav (staff)]

I think that is the problem.
What NIC do you use?
Please make sure to install te drivers into the CVM.

[MrSquish]

I thought I ubunto included the drivers with the kernal? I ran sudo apt update and sudo apt upgradr and it made no difference. I can ping from the VM shell all the interfaces. Is there a linux command I need to do to enable sr-iov directly in the CVM?

[yaroslav (staff)]

Thanks for more details. Can you please try creating a HA device using thick console? You might need TRIAL key or use free scripts.
Get the thick console https://starwind.com/tmplink/starwind-v8.exe

[MrSquish]

I'll give that a go. Is it recommended to install that directly on the hyper-converged hyper-V hosts? Or as a VM running windows on the hosts similar to how this linux VM operates?

[yaroslav (staff)]

Install it directly on Hyper-V hosts. But, technically speaking anything Windows-based will do a trick

[MrSquish]

That was much simpler to configure, and performance seems great with initial testing. Got a couple VM's stood up on the cluster and live migrations working without issue when each node reboots.

What is the main benefit of using the linux CVM with a 2 node hyper-v hyper-configured cluster like I have setup? I can spend more time sorting out the issues on the Linux VM if it's worth the effort.

If it's just sr-iov I need to get working, that at least gives me some direction on where the issue is. The error stating it can't ping wasn't giving me much to go on since pinging was working fine across all interfaces.

[yaroslav (staff)]

Thanks for your udpate.
Were you able to configure HA devices regardless of the CVM warning?
The main benefit of CVM in Hyper-V is NVMe-oF which is going to be introduced soon. Also, the compression (should be introduced later). If none of those is a particular interest to you, just use a Windows-based app.
I would appreciate having the logs from the systems if pings do not work with SR-IOV too.

[MrSquish]

Sure, what's the best way to send those to you?

[yaroslav (staff)]

Just a quick one, were you able to configure HA devices regardless of the CVM warning?

[MrSquish]

No, I never even tried to do anything else with it when it was failing to get the network part working. I was able to configure the windows service though. I did a few reboots to test what would happen based on various scenarios. One node rebooted and everything came back up just fine. Live migration worked and no issues. When I tried to reboot the 2nd node, It left the VSAN in a state of not syncing and the cluster wouldn't come online when the 2nd VSAN service was up. I had to stop the VSAN service on that node to get the cluster to come online again. I'm sure I misconfigured something, but haven't had the time to go back through it again yet. I tried running the included powershell scripts to force it to resync and got this error. I'll spend some more time this weekend and next re-configuring it again and see where it went wrong.

Device not synchronized. Synchronize current node from partner 'iqn.2008-08.com.starwindsoftware:192.168.51.31-ssdstorage1'
Failed to perform synchronization (1) from iqn.2008-08.com.starwindsoftware:192.168.51.31-ssdstorage1
-
control 0x00000263EAD91D00 -Synchronize: -SynchronizationType:"1" -SynchronizerTargetName:"iqn.2008-08.com.starwindsoftware:192.168.51.31-ssdstorage1"
-
200 Failed: connection with the synchronizer or partner node is invalid..

[yaroslav (staff)]

Thanks for more updates! Please keep me posted.
Make sure that network comms are available and pingable. Make sure MTU are aligned on the entire network stack.
Are you using node majority or heartbeat failover strategy?

[MrSquish]

Heartbeat is how it's configured currently.