Replication network unavailable

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

User avatar
MrSquish
Posts: 13
Joined: Wed Feb 14, 2024 2:44 am

Sun Feb 25, 2024 5:01 am

I am trying to setup a 2 node hyper-v using the linux CVM guest VM's following this article:
https://www.starwindsoftware.com/resour ... ng-web-ui/

I have 2 NIC's directly connected to each other, as well as 2 NIC's connected through a switch. Both of them give me this error when running through the "Configure HA networking" wizard. From the console of both CVM's, I can ping all the interfaces of each host between each other, i'm not sure what's wrong with the config resulting in the error saying it can't ping...
Attachments
VSAN_Error.png
VSAN_Error.png (69.88 KiB) Viewed 54492 times
VSAN_Replication_Interface.png
VSAN_Replication_Interface.png (244.8 KiB) Viewed 54492 times
yaroslav (staff)
Staff
Posts: 2322
Joined: Mon Nov 18, 2019 11:11 am

Sun Feb 25, 2024 10:52 am

Hi,

Do you have sriov on them?
User avatar
MrSquish
Posts: 13
Joined: Wed Feb 14, 2024 2:44 am

Sun Feb 25, 2024 6:25 pm

Yes, I enabled sr-iov. I installed some Mellanox connectx-5 dual 25Gb cards. As well as updated the firmware and enabled sr-iov on the 10gig daughter card.
yaroslav (staff)
Staff
Posts: 2322
Joined: Mon Nov 18, 2019 11:11 am

Sun Feb 25, 2024 6:50 pm

I think that is the problem.
What NIC do you use?
Please make sure to install te drivers into the CVM.
User avatar
MrSquish
Posts: 13
Joined: Wed Feb 14, 2024 2:44 am

Sun Feb 25, 2024 8:35 pm

I thought I ubunto included the drivers with the kernal? I ran sudo apt update and sudo apt upgradr and it made no difference. I can ping from the VM shell all the interfaces. Is there a linux command I need to do to enable sr-iov directly in the CVM?
yaroslav (staff)
Staff
Posts: 2322
Joined: Mon Nov 18, 2019 11:11 am

Sun Feb 25, 2024 10:41 pm

Thanks for more details. Can you please try creating a HA device using thick console? You might need TRIAL key or use free scripts.
Get the thick console https://starwind.com/tmplink/starwind-v8.exe
User avatar
MrSquish
Posts: 13
Joined: Wed Feb 14, 2024 2:44 am

Mon Feb 26, 2024 2:45 am

I'll give that a go. Is it recommended to install that directly on the hyper-converged hyper-V hosts? Or as a VM running windows on the hosts similar to how this linux VM operates?
yaroslav (staff)
Staff
Posts: 2322
Joined: Mon Nov 18, 2019 11:11 am

Mon Feb 26, 2024 4:20 am

Install it directly on Hyper-V hosts. But, technically speaking anything Windows-based will do a trick :)
User avatar
MrSquish
Posts: 13
Joined: Wed Feb 14, 2024 2:44 am

Mon Feb 26, 2024 6:43 am

That was much simpler to configure, and performance seems great with initial testing. Got a couple VM's stood up on the cluster and live migrations working without issue when each node reboots.

What is the main benefit of using the linux CVM with a 2 node hyper-v hyper-configured cluster like I have setup? I can spend more time sorting out the issues on the Linux VM if it's worth the effort.

If it's just sr-iov I need to get working, that at least gives me some direction on where the issue is. The error stating it can't ping wasn't giving me much to go on since pinging was working fine across all interfaces.
yaroslav (staff)
Staff
Posts: 2322
Joined: Mon Nov 18, 2019 11:11 am

Mon Feb 26, 2024 7:02 am

Thanks for your udpate.
Were you able to configure HA devices regardless of the CVM warning?
The main benefit of CVM in Hyper-V is NVMe-oF which is going to be introduced soon. Also, the compression (should be introduced later). If none of those is a particular interest to you, just use a Windows-based app.
I would appreciate having the logs from the systems if pings do not work with SR-IOV too.
User avatar
MrSquish
Posts: 13
Joined: Wed Feb 14, 2024 2:44 am

Mon Feb 26, 2024 9:25 am

Sure, what's the best way to send those to you?
yaroslav (staff)
Staff
Posts: 2322
Joined: Mon Nov 18, 2019 11:11 am

Mon Feb 26, 2024 10:07 am

Just a quick one, were you able to configure HA devices regardless of the CVM warning?
User avatar
MrSquish
Posts: 13
Joined: Wed Feb 14, 2024 2:44 am

Tue Feb 27, 2024 2:46 am

No, I never even tried to do anything else with it when it was failing to get the network part working. I was able to configure the windows service though. I did a few reboots to test what would happen based on various scenarios. One node rebooted and everything came back up just fine. Live migration worked and no issues. When I tried to reboot the 2nd node, It left the VSAN in a state of not syncing and the cluster wouldn't come online when the 2nd VSAN service was up. I had to stop the VSAN service on that node to get the cluster to come online again. I'm sure I misconfigured something, but haven't had the time to go back through it again yet. I tried running the included powershell scripts to force it to resync and got this error. I'll spend some more time this weekend and next re-configuring it again and see where it went wrong.

Device not synchronized. Synchronize current node from partner 'iqn.2008-08.com.starwindsoftware:192.168.51.31-ssdstorage1'
Failed to perform synchronization (1) from iqn.2008-08.com.starwindsoftware:192.168.51.31-ssdstorage1
-
control 0x00000263EAD91D00 -Synchronize: -SynchronizationType:"1" -SynchronizerTargetName:"iqn.2008-08.com.starwindsoftware:192.168.51.31-ssdstorage1"
-
200 Failed: connection with the synchronizer or partner node is invalid..
yaroslav (staff)
Staff
Posts: 2322
Joined: Mon Nov 18, 2019 11:11 am

Tue Feb 27, 2024 8:30 am

Thanks for more updates! Please keep me posted.
Make sure that network comms are available and pingable. Make sure MTU are aligned on the entire network stack.
Are you using node majority or heartbeat failover strategy?
User avatar
MrSquish
Posts: 13
Joined: Wed Feb 14, 2024 2:44 am

Tue Feb 27, 2024 3:20 pm

Heartbeat is how it's configured currently.
Post Reply