VSAN free 3 node Setup issue

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

Post Reply
joerg
Posts: 12
Joined: Wed Dec 03, 2014 8:00 pm
Location: Germany

Fri Dec 08, 2023 2:25 pm

Hi there,

back after appr. 7 years :-)

I am stuck in the setup of a 3-way-HA cluster.

I started with 3 VMs (HyperV) with 8 cores, and 3 NICs based on this drive: sw-vsa-hyper-v-build-15020.vhdx.

The 3 machines are up and running, the NICs are configured
10.0.10.18x MGMT
10.0.50.18x iSCSI
10.0.56.18x SYNC
x{1,2,3}

On every VM I created
- a RAID device
- a VDO device

Which is mounted on /mnt

I installed the Powershell package on a Server 2019 machine and adapted the creatHA_3.ps1

Code: Select all

param($addr="10.0.10.181", $port=3261, $user="root", $password="starwind",
	$addr2="10.0.10.182", $port2=$port, $user2=$user, $password2=$password,
	$addr3="10.0.10.183", $port3=$port, $user3=$user, $password3=$password,
#common
	$initMethod="SyncFromFirst",
	$size=1000,
	$sectorSize=512,
	$failover=0,
	#$bmpType=1,
	#$bmpStrategy=0,
#primary node
	$imagePath="VSA Storage\mnt",
	$imageName="masterImg31",
	$createImage=$true,
	$targetAlias="targetha31",
	$autoSynch=$true,
	$poolName="Pool1",
	$storageName="",
	$syncSessionCount=1,
	$aluaOptimized=$true,
	$cacheMode="wb",
	$cacheSize=128,
	$syncInterface="#p2=10.0.56.182:3260;#p3=10.0.56.183:3260",
	$hbInterface="#p2=10.0.10.182:3260,10.0.50.182:3260;#p3=10.0.10.183:3260,10.0.50.183:3260",
	#$bmpFolderPath="",
#secondary node
	$imagePath2="VSA Storage\mnt",
	$imageName2="partnerImg32",
	$createImage2=$true,
	$targetAlias2="partnerha32",
	$autoSynch2=$true,
	$poolName2="Pool1",
	$storageName2="",
	$syncSessionCount2=1,
	$aluaOptimized2=$false,
	$cacheMode2=$cacheMode,
	$cacheSize2=$cacheSize,
	$syncInterface2="#p1=10.0.56.181:3260;#p3=10.0.56.183:3260",
	$hbInterface2="#p1=10.0.10.181:3260,10.0.50.181:3260;#p3=10.0.10.183:3260,10.0.50.183:3260",
	#$bmpFolderPath2="",
#third node
	$imagePath3="VSA Storage\mnt",
	$imageName3="partnerImg33",
	$createImage3=$true,
	$targetAlias3="partnerha33",
	$autoSynch3=$true,
	$poolName3="Pool1",
	$storageName3="",
	$syncSessionCount3=1,
	$aluaOptimized3=$false,
	$cacheMode3=$cacheMode,
	$cacheSize3=$cacheSize,
	$syncInterface3="#p1=10.0.56.181:3260;#p2=10.0.56.182:3260",
	$hbInterface3="#p1=10.0.10.181:3260,10.0.50.181:3260;#p2=10.0.10.182:3260,10.0.50.182:3260"	
	#$bmpFolderPath3=""
	)

Import-Module StarWindX

try
{
	Enable-SWXLog

	$server = New-SWServer -host $addr -port $port -user $user -password $password

	$server.Connect()
    
	$firstNode = new-Object Node

	$firstNode.HostName = $addr
	$firstNode.HostPort = $port
	$firstNode.ImagePath = $imagePath
	$firstNode.ImageName = $imageName
	$firstNode.Size = $size
	$firstNode.CreateImage = $createImage
	$firstNode.TargetAlias = $targetAlias
	$firstNode.AutoSynch = $autoSynch
	$firstNode.SyncInterface = $syncInterface
	$firstNode.HBInterface = $hbInterface
	$firstNode.PoolName = $poolName
	$firstNode.SyncSessionCount = $syncSessionCount
	$firstNode.ALUAOptimized = $aluaOptimized
	$firstNode.SectorSize = $sectorSize
	$firstNode.CacheMode = $cacheMode
	$firstNode.CacheSize = $cacheSize
	$firstNode.FailoverStrategy = $failover
	#$firstNode.BitmapStoreType = $bmpType
	#$firstNode.BitmapStrategy = $bmpStrategy
	#$firstNode.BitmapFolderPath = $bmpFolderPath

	$secondNode = new-Object Node

	$secondNode.HostName = $addr2
	$secondNode.HostPort = $port2
	$secondNode.Login = $user2
	$secondNode.Password = $password2
	$secondNode.ImagePath = $imagePath2
	$secondNode.ImageName = $imageName2
	$secondNode.CreateImage = $createImage2
	$secondNode.TargetAlias = $targetAlias2
	$secondNode.AutoSynch = $autoSynch2
	$secondNode.SyncInterface = $syncInterface2
	$secondNode.HBInterface = $hbInterface2
	$secondNode.SyncSessionCount = $syncSessionCount2
	$secondNode.ALUAOptimized = $aluaOptimized2
	$secondNode.CacheMode = $cacheMode2
	$secondNode.CacheSize = $cacheSize2
	$secondNode.FailoverStrategy = $failover
	#$secondNode.BitmapFolderPath = $bmpFolderPath2

	$thirdNode = new-Object Node

	$thirdNode.HostName = $addr3
	$thirdNode.HostPort = $port3
	$thirdNode.Login = $user3
	$thirdNode.Password = $password3
	$thirdNode.ImagePath = $imagePath3
	$thirdNode.ImageName = $imageName3
	$thirdNode.CreateImage = $createImage3
	$thirdNode.TargetAlias = $targetAlias3
	$thirdNode.AutoSynch = $autoSynch3
	$thirdNode.SyncInterface = $syncInterface3
	$thirdNode.HBInterface = $hbInterface3
	$thirdNode.SyncSessionCount = $syncSessionCount3
	$thirdNode.ALUAOptimized = $aluaOptimized3
	$thirdNode.CacheMode = $cacheMode3
	$thirdNode.CacheSize = $cacheSize3
	$thirdNode.FailoverStrategy = $failover
	#$thirdNode.BitmapFolderPath = $bmpFolderPath3

	$device = Add-HADevice -server $server -firstNode $firstNode -secondNode $secondNode -thirdNode $thirdNode -initMethod $initMethod
    
	$syncState = $device.GetPropertyValue("ha_synch_status")
    
    write-host "Start Sync..."

	while ($syncState -ne "1")
	{
		#
		# Refresh device info
		#
		$device.Refresh()

		$syncState = $device.GetPropertyValue("ha_synch_status")
		$syncPercent = $device.GetPropertyValue("ha_synch_percent")

		Start-Sleep -m 2000

		Write-Host "Synchronizing: $($syncPercent)%" -foreground yellow
	}
}
catch
{
	Write-Host $_ -foreground red
}
finally
{
	$server.Disconnect()
}

The script runs, but do not create imagefiles on nodes 2 and 3, only on node 1. Therefore no sync takes place.

Message after completion only:
Request to 10.0.10.181 ( 10.0.10.181 ) : 3261
Any ideas, what could be wrong with my script or my setup?

Thx in adcance, cheers,
Jörg

PS.: with the TRIAL version of vsan and the same setup of VMs (differnt VHDX of course, but same NICs, Procs, RAM) everything is fine.
yaroslav (staff)
Staff
Posts: 2361
Joined: Mon Nov 18, 2019 11:11 am

Fri Dec 08, 2023 3:07 pm

Hi,

Welcome back!! :) directory should point to your device: e.g. /mnt/disk not just /mnt :)
If that does not work, try commenting out the areas mentioning bitmap.
Hope that is the key.

I'd also suggest not using VDO with latest builds in prod (trim/unmap can cause hiccups, we should fix it soon in CVM).
joerg
Posts: 12
Joined: Wed Dec 03, 2014 8:00 pm
Location: Germany

Fri Dec 08, 2023 5:24 pm

Hi Yaroslav,
many thanks for quick reply.

I am a step further but run into next "problem": I have to investigate over the weekend.

control HAImage -CreateHeader:"" -DeviceHeaderPath:"VSA Storage\mnt\SS01_P1\masterImg31_HA.swdsk" -Type:"ImageFile_HA" -file:"imagefile2" -size:"1000" -Priority:"#p0=0;#p1=1;#p2=2" -nodeType:"#p0=1;#p1=1;#p2=1" -PartnerTargetName:"#p1=iqn.2008-08.com.starwindsoftware:10.0.10.182-partnerha32;#p2=iqn.2008-08.com.starwindsoftware:10.0.10.183-partnerha33" -PartnerIP:"#p1=10.0.56.182:sync:3260:1,10.0.10.182:heartbeat:3260:1,10.0.50.182:heartbeat:3260:1;#p2=10.0.56.183:sync:3260:1,10.0.10.183:heartbeat:3260:1,10.0.50.183:heartbeat:3260:1" -IsAutoSynchEnabled:"1" -AuthChapLogin:"#p1=0b;#p2=0b" -AuthChapPassword:"#p1=0b;#p2=0b" -AuthMChapName:"#p1=0b;#p2=0b" -AuthMChapSecret:"#p1=0b;#p2=0b" -AuthChapType:"#p1=none;#p2=none" -Offset:"0" -CacheMode:"wb" -CacheSizeMB:"128" -serial:"C91FD58A9FD3B77E" -eui64:"C91FD58A9FD3B77E" -revision:"0001" -product:"STARWIND" -vendor:"STARWIND" -Replicator:"#p0=0" -WitnessType:"0" -AluaAccessState:"#p0=0;#p1=1;#p2=1"
200 Failed: invalid partner info..

I think, I saw this message in some other forum posts... will search for them first.

Cheers,
Jörg
yaroslav (staff)
Staff
Posts: 2361
Joined: Mon Nov 18, 2019 11:11 am

Fri Dec 08, 2023 6:01 pm

Hi Jörg,

Can you please make sure there are no duplicate files in that directory?
joerg
Posts: 12
Joined: Wed Dec 03, 2014 8:00 pm
Location: Germany

Fri Dec 08, 2023 6:24 pm

the dirs (mount points) are all empty.

do I have to mount on all VMs to the same dir?
on the VMs I have now
\mnt\SS01_P1
\mnt\SS02_P1
\mnt\SS03_P1

this is configured in the parameter
$imagePath="VSA Storage\mnt\SS01_P1",
$imagePath2="VSA Storage\mnt\SS02_P1",
$imagePath3="VSA Storage\mnt\SS03_P1",
yaroslav (staff)
Staff
Posts: 2361
Joined: Mon Nov 18, 2019 11:11 am

Sat Dec 09, 2023 6:35 am

Greetings,

They are ok to be in different directories.
Can you make sure that imagefile inside the directory is carrying a different name (i.e., no target and imagefile names are conflicting with the existing deployments)?
joerg
Posts: 12
Joined: Wed Dec 03, 2014 8:00 pm
Location: Germany

Tue Dec 12, 2023 8:03 pm

I gave up...

three days of intensive testing with "all possible" settings. :-(
For checking the infrastructure I set up 2 node clusters, with server 1 and 2 and server 1 and 3. All good.
Today I tried to add a HA-Member to the 2 node cluster with the script AddHaPartner.ps1, but failed.

So I stay at the moment with the 2 node cluster as this should fit my needs... will see.

Cheers,
Jörg
yaroslav (staff)
Staff
Posts: 2361
Joined: Mon Nov 18, 2019 11:11 am

Tue Dec 12, 2023 10:03 pm

Jörg,

You can repurpose the system into GRID (see more at https://www.starwindsoftware.com/grid-architecture-page).
You can also reach out to support@starwind.com using this thread as a reference and 1076002.
Post Reply