FIXED - Citrix Xenserver 7.x MPIO

[Blowfish-IT]

Hi all,

Wonder if someone can help?? We have a small platform used to publish VDIs to our customers.
As the platform is growing we recently had some performance issued and noticed that the two Xenservers are both only using one if the two NICs dedicated for iSCSI storage.

We have seen the two docs about MPIO for starwind (https://www.starwindsoftware.com/techni ... d_MPIO.pdf) which we have uncommented the line <iScsiDiscoveryListInterfaces value="1"/>

Following the second doc we needed to make changes to the Xenservers themselves to enable true multipath. (https://www.starwindsoftware.com/techni ... r-Pool.pdf)
We followed the steps below:

1. Switch to console tab, edit multipath configuration file running “vi /etc/multipath.conf”
command.
2. Insert the following block into the “devices” section:
device
{
vendor "STARWIND"
product "STARWIND*"
path_grouping_policy multibus
path_checker "tur"
failback immediate
path_selector "round-robin 0"
rr_min_io 3
rr_weight uniform
hardware_handler "1 alua"
}
3. Restart the multipath service:
service multipathd restart
service multipathd reload
4. Set the multipathd service to start automatically when you start XenServer:
chkconfig multipathd on
5. Right-click the XenServer and select Properties from the shortcut menu.
6. Click Multipathing in the left pane. Select the Enable multipathing on this server checkbox.
7. Click OK.
8. Turn off the Maintenance Mode: right-click the XenServer and select Exit Maintenance Mode
from the shortcut menu.
9. Reboot XenServer host.


Once we reboot the hosts and check the multipath, the number of paths (Normally 2 of 2) changes to "Active" however the iSCSI storage shows as detached, if we try to re-attach we just get an error.

We have tried to create a small 5GB LUN and attach this by creating a new SR however once we have entered the two IPs we are able to scan the SAN and detect the new 5GB LUN we get "Invalid Argument" when trying to scan it.


Anyone able to help or guide us in the correct / any missing steps?

Many Thanks
Rob

[Blowfish-IT]

Hi all,

So after much playing around and reading some information from someone on the forum doing something similar with FreeNAS we have found the settings that work with Citrix Xenserver 7.x

In Starwind the defaults are set to
First Burst = 65536
Max Burst = 262144
Max data segment = 65536

So based on these we modified the /etc/iscsi/iscsid.conf file and update the First burst, Max Burst and Max data segment to be the same as the Starwind Config.
We then added the following settings into the /etc/multipath.conf file

device
{
vendor "STARWIND"
product "STARWIND*"
path_grouping_policy multibus
path_checker "tur"
failback immediate
path_selector "round-robin 0"
rr_min_io 1
rr_weight uniform
}

We found that adding the final line hardware_handler "1 alua" seemed to cause the system to be unable to enable scan the SR and caused the error "Invalid Argument"

We had already enabled multipathing on each server in the GUI, but just to be on the safe side we put all hosts into maintenance and un-applied and re-applied the multipath setting.

Rebooted and viola all working on both NICs correctly

in the console "mulitpath -ll" is showing both paths as active with Round robin

Kind Regards,
Rob

[buzzzo]

Have you verified you don't loose the alua capabilities?

[Michael (staff)]

Hello Rob,
Thank you for your findings and investigations.
We didn’t mark StarWind compatible with XenServer 7 yet, but we will forward your findings to our QA department, thus they should help them in case if they will have any issues.
We will keep you posted about the results of QA tests.

[Blowfish-IT]

Hi,

Sorry - our setup was only to one host but we wanted multipath enabled.
We found that the default was Active/Passive or Active/Failover which caused us performance problems as we only had a single 1GB link at any time.

We found that the ALUA was a functionality that is only used when you have more than 1 replicating node.

Kind Regards,
Rob

[Michael (staff)]

Hello Rob,
Thank you for sharing additional information here.
We will keep the community updated once the tests will be completed.

[buzzzo]

Just a follow up on this topic.
I've tried some tests with 2 xenserver 7 boxes with a hyperconverged starwind setup (basically: 2 win2k12 vms on top of them).
Results are not too pleasant.

Basically when attaching the vdisk to the xenserver cluster all is running fine.
When i reboot the nodes all the starwind vms obviously come down.

What i'm expecting at this point is:
1) start one of the starwind vm and mark the vdisk as syncronized
2) let the storage repository come up at xen level and startup the other vms (hosted on top of it)
3) start the second starwind vm and let the starwind cluster resync

What it happen really is that the sr came up (without half of paths of course) but with errors in the sense that i can't start the vms
living on the sr.

All is return to work when the starwind cluster is synched (aka: all the two starwind vms are running and the vdisk is resynced)
In the end: seems something at iscsi/multipath level is changed in xenserver 7 that prevents starwind to work.

[Michael (staff)]

Hello Buzzzo,

Thank you for your valuable feedback. Currently, Xen supported only in Compute and Storage separated scenario but HyperConverged is already on our roadmap.

We will keep the community posted about it.

[buzzzo]

Hi Michael

Just to be clear :I'm using an hyperconverged cluster based in XenServer 6.5 and starwind and it works perfectly.
Problems arise only with XenServer 7.0 version and seems to be related to multipath.
I don't think there would be significant Differences between hyperconverged and San classic setups

[Michael (staff)]

Hello Buzzo,
I agree with you - there should not be significant differences.
Once we will have an update from QA ,we will update the community here.

[buzzzo]

Hi

Any news on starwind support on xenserver 7.x ?
thx

[Michael (staff)]

Hello everybody,
Please find below the main settings which should be checked on XenServer v7 configuration with StarWind v8.

1. Check /etc/iscsi/iscsid.conf file. Set the following values:

node.startup = automatic
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 262144

2. Restart the software iSCSI service to apply the setting above

service iscsi restart
service iscsid restart

3. Set the iSCSI services to start automatically when you start XenServer:

chkconfig iscsi on
chkconfig iscsid on

4. Multipathing configuration.
Note: switch XenServer host to maintenance mode for multipath configuration. To apply multipathing take the following actions for each XenServer in the pool.
4.1. Check /etc/multipath.conf file settings and edit defaults section like below.

defaults {
user_friendly_names no
find_multipaths yes
polling_interval 10
}

4.2. Insert the following block into the “devices” section:

device {
vendor "STARWIND"
product "STARWIND*"
path_checker "tur"
failback 30
path_grouping_policy group_by_prio
path_selector "round-robin 0"
rr_min_io 3
prio alua
}

4.3 Restart the multipath service:

service multipathd restart
service multipathd reload

4.4 Set the multipathd service to start automatically when you start XenServer:

chkconfig multipathd on

5. Enable multipathing from XenServer properties and Turn off the Maintenance Mode

6. Reboot XenServer host.

Discovering targets and adding SRs can be done on the same way as it is specified on page 20 in the following document https://www.starwindsoftware.com/techni ... r-Pool.pdf

[buzzzo]

Hi
I've tried your steps on a xenserver 7.1 with the following changes from standard starwind istructions:
1) not used the script cause it does not work: added the stardwind vdisk tru xenceter gui: WORKS

Tested this scenario:

1) switchoff one of the two host in the pool
2) the vm located on the pool correctly keep running
3) when host came up it can't reconnect to the sr, i force the operation with a repair: SEEMS TO WORK
4) i've tried then to run a vm on this host : the vms can't start

So in the end i can't confirm that the procedure work on xs 7.x

Thx

[buzzzo]

As a side note: I've succeeded creating the LUN via starwind provided script: same story.

[Michael (staff)]

Hello Buzzzo,
Please clarify your configuration because as far as I know, if you turn off the host, SR should just loose the paths from turned off StarWind VM.
When the host back up, SR should restore the paths, but VMs on it should not be interrupted at all!
Also, please double check that you have discovered and connected SR from both StarWind nodes. I hope "multipath -ll" should show all the paths.