Synchronization issue in new 3 node cluster

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

Post Reply
prismsoftware
Posts: 2
Joined: Thu Apr 11, 2024 5:09 pm

Thu Apr 11, 2024 6:23 pm

Hi - I created a test 3 node cluster using the exact number of NICs/IPs as the example on the starwind site. Everything was working normally and there were no issues during creation. I tried to simulate a single node failure by rebooting the physical server (Node 1). Now I'm in a state where Node 2 says it's synchronized, Node 3 is trying to sync from Node 1 and failing with "synchronizer has invalid state", Node 1 is trying to establish a connection to Node 3 and can't (error 0x5b4)

I'm guessing that the nodes need to try and sync from Node 2 instead? I tried the syncDeviceAdvanced.ps1 script, on Node 1 it doesn't come back, Node 2 says synchronized and Node 3 reports:

Failed to perform synchronization (1) from iqn.2008-08.com.starwindsoftware:uzol1-uzol1targetalias
control 0x0000015B326E3A00 -Synchronize: -SynchronizationType:"1" -SynchronizerTargetName:"iqn.2008-08.com.starwindsoftware:uzol1-uzol1targetalias"
200 Failed: synchronizer has invalid state..

I'm not sure how to avoid this situation in the future - any ideas? I attached the powershell script I used. I can ping each node from Node 1, 2, 3 through the interfaces defined. Thank you very much.
Attachments
CreateHA_TEST.zip
(1.43 KiB) Downloaded 761 times
yaroslav (staff)
Staff
Posts: 2364
Joined: Mon Nov 18, 2019 11:11 am

Thu Apr 11, 2024 10:52 pm

Welcome to StarWind Forum. Create a HA device without a write-back cache (comment caching areas) and repeat the tests.
prismsoftware
Posts: 2
Joined: Thu Apr 11, 2024 5:09 pm

Fri Apr 12, 2024 3:40 pm

Hi Yaroslav - thanks for responding. I removed the cache and so far it appears to be working, after creating from scratch I tried the same thing (turning off a node) and it appears to be synchronizing after rebooting. Just so I understand can you explain why having a write-back cache defined caused an issue?
yaroslav (staff)
Staff
Posts: 2364
Joined: Mon Nov 18, 2019 11:11 am

Fri Apr 12, 2024 11:25 pm

Write-back cache triggers full sync in case of one server's outage. Also, devices with Write-Back cache tend to end mutual not synchronized more often.
See more
caching https://knowledgebase.starwindsoftware. ... rinciples/,
full sync causes https://knowledgebase.starwindsoftware. ... may-start/.
How to fix mutual not synchronized https://knowledgebase.starwindsoftware. ... -blackout/
This concept can also be interesting https://knowledgebase.starwindsoftware. ... a-devices/
Post Reply