Issues adding additional "appliance" nodes to starwind vsan free vms on proxmox

NinthTurtle · Tue Nov 19, 2024 10:56 pm

I'm trying to roll out 3 Starwind vsan nodes to act as my storage for my 3 node proxmox cluster + some of my vms (for docker/kubernmetes volumes, etc) but I seem to be having some issues adding the additional nodes as I keep getting this error.

: adding appliance two.png (25.02 KiB) Viewed 4035 times

: failed to connect to appliance two.png (18.03 KiB) Viewed 4035 times

: failed to connect to appliance three.png (18.23 KiB) Viewed 4035 times

This network range is my main lan and I've checked that all 3 nodes can ping each other. The interface for IP shown is set as a management interface, but maybe I need add another management interface?

the docs say to input the management IP but maybe I should try with the replication network IP?
I don't have any storage configured on any of the nodes yet, Do I need to do that first?

I'm unclear as to the cause of the error and [the docs][/https://www.starwindsoftware.com/resour ... sing-web-u] aren't very clear what to do if joining the appliance shouldn't work.

Any help or advice would be greatly appreciated.

I can provide additional screenshots of the networking setup on each node if that helps.

NinthTurtle · Tue Nov 19, 2024 10:57 pm

These are the networking configurations for each node

: starwind appliance one networking.png (59.19 KiB) Viewed 4034 times

: starwind appliance two networking.png (59.17 KiB) Viewed 4034 times

: starwind appliance three networking.png (59.2 KiB) Viewed 4034 times

Let me know if you want to see the configuration within Proxmox, or any other information really.

Tue Nov 19, 2024 11:25 pm

Thanks for posting.
Please collect the logs and indicate the approximate time for triggering the error.
p.s.. you can continue without adding the nodes to the appliances.

NinthTurtle · Wed Nov 20, 2024 11:16 pm

yaroslav (staff) wrote: ↑
Tue Nov 19, 2024 11:25 pm
Thanks for posting.
Please collect the logs and indicate the approximate time for triggering the error.

Hi, Thanks for getting back to me. I'll hopefully reply sometime tomorrow with the log files if I can find/get them.

yaroslav (staff) wrote: ↑
Tue Nov 19, 2024 11:25 pm
p.s.. you can continue without adding the nodes to the appliances.

Could you elaborate on what you mean by this? do you mean I can continue making the cluster without setting up the appliances or did you mean I can continue configuring each node independently such as it's storage and network shares?
If you meant the former; I don't think that was possible as each time I selected "next" it would bring up that screen telling me it was failing to connect.
If you meant the latter; I understand that would be doable but I'd prefer to have the cluster setup first so I can ensure it's all working as I expect.

Thanks!

Wed Nov 20, 2024 11:27 pm

I'll hopefully reply sometime tomorrow with the log files if I can find/get them.

Many thanks!

Could you elaborate on what you mean by this? do you mean I can continue making the cluster without setting up the appliances or did you mean I can continue configuring each node independently such as it's storage and network shares?

You do not need to add the nodes to Appliances if you use the Free version: there is no GUI for LUN creation. Therefore, the settings you make in Networks and Appliances do not help for further LUN creation.
You still can do that so that things look neat.

I will be glad to look through the logs.

NinthTurtle · Thu Nov 21, 2024 9:34 pm

yaroslav (staff) wrote: ↑
Wed Nov 20, 2024 11:27 pm

I'll hopefully reply sometime tomorrow with the log files if I can find/get them.
Many thanks!

I'm finding I'm not sure where the system would be recording the logs for my issue. I took a look at the `journalctl` and couldn't spot anything. I've also taken a look at `/var/log/syslog` but nothing jumps out. None of the directory in `/var/log/starwind` looked like they'd have a log file.
Do you mind providing some guidance on where I can find that log file.

yaroslav (staff) wrote: ↑
Wed Nov 20, 2024 11:27 pm

Could you elaborate on what you mean by this? do you mean I can continue making the cluster without setting up the appliances or did you mean I can continue configuring each node independently such as it's storage and network shares?
You do not need to add the nodes to Appliances if you use the Free version: there is no GUI for LUN creation. Therefore, the settings you make in Networks and Appliances do not help for further LUN creation.
You still can do that so that things look neat.

I will be glad to look through the logs.

I'm still pretty new to Starwind and have never (knowingly) worked with a LUN before. I'm just hoping the free version will allow me to sync data between my nodes and hopefully run a smb or nfs share (maybe a issci share) across all of the nodes so it doesn't matter which nod I address during the mounting stage. But that's out of scope of this topic and part of why I'm testing Starwind, so I can learn more about it.

Fri Nov 22, 2024 1:02 am

Please share the entire support bundle with me (need it from all systems). Also, I will need the time stamp of the issue.
Yes, free version allows to sync data over iSCSI. It can be connected to a client (e.g., ESXi host) over iSCSI (NVME-OF in the future) and formatted as VMFS. You need to create LUNs that will act as iSCSI targets.
I'd suggest to trial the solution and let one of our techs to walk you through the deployment process (you will need to redeploy CVM to apply free key after trial). It should be easier to learn more about the solution.
Also, consider vooking a demo with one of my colleagues https://www.starwindsoftware.com/v17-request-live-demo.

Wed Nov 27, 2024 12:43 pm

I had a word with my colleague. It looks like the VM was cloned.
Please re-add the management adapters.
See more on cloning https://pve.proxmox.com/wiki/Duplicate_Virtual_Machines.

NinthTurtle · Mon Dec 02, 2024 12:59 pm

Hello,

I'm being told by the file uploader that my "support bundle" zips (14KB-15KB) are too large to be uploaded.

I'm pretty sure I had re-added all of the network interfaces post cloning, but perhaps it was only the additional nics and not the management nic. I'll fix that.

Is there an alternative way I can upload the "support bundle" for each node or is there a specific part of that bundle you would want?

Thanks

NinthTurtle · Mon Dec 02, 2024 1:19 pm

Hello again,

I've reset all of the mac addresses of the nic in proxmox and all 3 nodes will now connect to each other in the ui.

Mon Dec 02, 2024 1:36 pm

Hi,

Many thanks for your update.
Yes, the issue was related to MAC addresses on the interfaces. That kind of conflict was causing havoc in ARP tables.
Thanks for your cooperation.

NinthTurtle · Mon Dec 02, 2024 2:12 pm

Thanks for your help yaroslav.

One other question I've just thought of, related to the networking: Is MTU of `1500` the absolute minimum for the cvm?
The GUI is giving me a red error text underneath the text box saying MTU needs to be in the range of 1500-9000, but because of hardware restrictions in my proxmox system one of the nics on each cvm is a proxmox virtual sdn nic, this caps me at using a mtu of 1450.
Is there a way to set this within the starwind config? maybe I'd need to ssh to the nodes and set it there.

I somehow managed to set the MUTU to 1450 prior to adding all of the nodes as appliances but now I have I don't appear to be able to set lower MTU's.
I've also noticed if a nic had a lower mtu I couldn't use it in the gui for replication or data networks.

Mon Dec 02, 2024 2:41 pm

Hi,

You are always welcome. Set MTU to 1500 for the entire stack, it is more stable for some NICs from my experience.

NinthTurtle · Mon Dec 02, 2024 3:38 pm

Okay so new issue, I'm trying to configure HA networking, which I'm not sure if I actually need. I can configure the data network fine but when I try to configure the replication network I get told it can't ping the replication network targets, I've logged in via SSH to each node and pinged the replication targets and they do respond.