Virtual SAN V8 - Network configuration

[bodaker]

I would like to test the new Virtual SAN V8 and currently I am trying to set best possible network configuration for two node cluster. Each node have one dual-port 10 Gb (Mellanox ConnectX-2) and six 1 Gb NICs.

I checked Quick Start guide:
http://www.starwindsoftware.com/starwin ... rted-guide
and I have a few questions please…

1:
What is the reason of two physical networks between two nodes? Performance, fault-tolerance or both? I have dual-port 10 Gb so I am able to set the same configuration as noted in the Quick Start and enjoy very fast links. The problem is that I would like to use one 10 Gb port for a different purpose (fast isolated network between VMs on different nodes and for Live Migration).

In the case I have a single 10 Gb line, should I create Sync1, iSCSI-Data1, Sync2 and iSCSI-Data2? Or just Sync1, iSCSI-Data1?

How much will performance degrade if I you use only a single 10 Gb line?

2:
In the Quick Start guide is stated: The Sync and iSCSI data are direct links between the servers with 2 VLANs dedicated in each link.
What is the best practice how to set two VLANs for Sync and iSCSI-Data on a single line? From my point of view there are two possibilities. The first possibility is create virtual (Hyper-V) switch and then create two virtual network adapters (Add-VMNetworkAdapter) with different VLAN settings. The second possibility is to create a team (it is possible to create team with one NIC) and add second team interface (Add-NetLbfoTeamNic –Team MyTeam –Name AnotherTeamInterface).

3:
I want to create 10 Gb team to ensure fault-tolerance. But I do not have enough 10 Gb NICs so I want to create this:

Code: Select all

Node 1: 10 Gb --------------- Node 2: 10 Gb
Node 1: 1 Gb (StandBy) ------ Node 2: 1 Gb (StandBy)

When I unplug 10 Gb the team is still functional. Of course the speed drops the link (whole team) is not disconnected. I tested it and it is working fine but maybe somebody have a second opinion about this.

Thank you very much.

[Anatoly (staff)]

Hi!

Thank you for the interest in StarWind!

1)You`ll need at least two ports on each StarWind box: one will run synchronization, and the other one - iSCSI data and heartbeat (health monitoring). If you`ll assign sync and heartbeat on one physical NIC and it will fail, that will cause split brain issue, and I highly doubt that you need that.

2) Except the reliability problem that I described above we have one rule: the bandwidth of the synchronization channel should be not less then the sum of all the iSCSI data links.
As about the NIC Teaming - avoid it when talking about iSCSI.

3) I`m pretty sure that this question is answered above.

I hope that make sense.

[bodaker]

I am sorry for the delay. I was on vacation.

Thank you very much for your reply.

If I understand correctly for two node cluster I need four ports (two nodes = two StarWind boxes = four cables). But as I said I have only one dual port 10 Gb NIC in every node. What is the solution?

The only thing that occurs to me is to create two virtual adapters per one physical as I mentioned in the first post. I should be able to set QoS for every virtual NIC and ensure that synchronization channel will always have enough bandwidth.

I think I can avoid split brain issue using another slow (1 Gb) direct connection between nodes that will be dedicated only for heartbeat.

[robnicholson]

If you`ll assign sync and heartbeat on one physical NIC and it will fail, that will cause split brain issue, and I highly doubt that you need that.

I know what split-brain is (common in Exchange DAGs) but not 100% sure why having multiple NICs prevents split brain? If what you mean is that it reduces the chances of split-brain then I agree. Unless of course, you've got 2 x 10GbE ports and for cost reasons these are both are connected via the same 10GbE switch, then you've still got a single point of failure. But then again, I agree with your design suggestion that Sync is done via cross-over leads esp. in a two-node set-up. Removes the switch as a) an additional bit of hardware to fail and b) a single point of failure.

BTW - if the heartbeat channel fails, does StarWind try to use the sync channels? That would give even more redundancy?

Cheers, Rob.

[Anatoly (staff)]

But as I said I have only one dual port 10 Gb NIC in every node. What is the solution?

Technically you can install StarWind using 1 dual port on each server, you just need to know that system will be not too reliable. If you could get additional 1Gig for HeartBeat, that would do the great job!

@Rob

but not 100% sure why having multiple NICs prevents split brain?

Example: two servers connected back to back trough Dual ports NIC. What gonna happen if 1 NIC goes down? Split brain. What would happen if each server would have any additional NIC? Nothing serious.