Well, so far... things aren't going well. I completely blew away my configuration on the SW server and followed the guide as closely as I could. As I mentioned in previous posts, I could connect with my last setup just fine (using what I knew of Xen and iSCSI, and applying it to SW), but did not have multipath activity on the four connections.
Following the guide, I can get up to the point where I run the script. The script freezes after step 8 in the "Adding Shared Storage using automated script" section. I enter the four IPs, separated by three commas... just as I would in the Xen CLI, or via the GUI when adding an SR.
I then tried the manual steps listed under "Adding Shared Storage manually". Discovery works fine, as well as connecting to the targets in step 3. Running the multipath check of "multipath -ll" (implied in step 5) returns only the previously connected FreeNAS SR. If I continue onto step 6, the xe sr-probe command hangs indefinitely.
I've also tried adding the SR via the GUI. The target LUN shows up correctly, but it takes about 10x longer for the target LUN to show up. When it does, trying to finish the GUI reports an error of "Invalid argument. Check your settings and try again".
What I've tried:
- Ensuring I can ping all the SW iSCSI interfaces (101.7, 102.7, 103.7, and 104.7) from both Xen hosts
- Telnet from each Xen host to port 3260, on all SW interfaces
- Completely disabled Windows Firewall on the SW server
- Rebooted both Xen hosts in the pool
Here is the output of running the commands in the guide:
Code: Select all
[root@xen1 ~]# iscsiadm -m discovery -t st -p 192.168.101.7
iscsiadm -m discovery -t st -p 192.168.102.7
iscsiadm -m discovery -t st -p 192.168.103.7
iscsiadm -m discovery -t st -p 192.168.104.7
iscsiadm -m node -T iqn.2004-07.com.cabuzzi.sw1:iscsi1 -p 192.168.101.7 -l
iscsiadm -m node -T iqn.2004-07.com.cabuzzi.sw1:iscsi1 -p 192.168.102.7 -l
iscsiadm -m node -T iqn.2004-07.com.cabuzzi.sw1:iscsi1 -p 192.168.103.7 -l
iscsiadm -m node -T iqn.2004-07.com.cabuzzi.sw1:iscsi1 -p 192.168.104.7 -l
192.168.101.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.102.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.1.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.103.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.104.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
[root@xen1 ~]# iscsiadm -m discovery -t st -p 192.168.102.7
192.168.102.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.101.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.1.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.103.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.104.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
[root@xen1 ~]# iscsiadm -m discovery -t st -p 192.168.103.7
192.168.103.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.101.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.102.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.1.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.104.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
[root@xen1 ~]# iscsiadm -m discovery -t st -p 192.168.104.7
192.168.104.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.101.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.102.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.1.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
192.168.103.7:3260,1 iqn.2004-07.com.cabuzzi.sw1:iscsi1
[root@xen1 ~]# iscsiadm -m node -T iqn.2004-07.com.cabuzzi.sw1:iscsi1 -p 192.168.101.7 -l
Logging in to [iface: default, target: iqn.2004-07.com.cabuzzi.sw1:iscsi1, portal: 192.168.101.7,3260] (multiple)
Login to [iface: default, target: iqn.2004-07.com.cabuzzi.sw1:iscsi1, portal: 192.168.101.7,3260] successful.
[root@xen1 ~]# iscsiadm -m node -T iqn.2004-07.com.cabuzzi.sw1:iscsi1 -p 192.168.102.7 -l
Logging in to [iface: default, target: iqn.2004-07.com.cabuzzi.sw1:iscsi1, portal: 192.168.102.7,3260] (multiple)
Login to [iface: default, target: iqn.2004-07.com.cabuzzi.sw1:iscsi1, portal: 192.168.102.7,3260] successful.
[root@xen1 ~]# iscsiadm -m node -T iqn.2004-07.com.cabuzzi.sw1:iscsi1 -p 192.168.103.7 -l
Logging in to [iface: default, target: iqn.2004-07.com.cabuzzi.sw1:iscsi1, portal: 192.168.103.7,3260] (multiple)
Login to [iface: default, target: iqn.2004-07.com.cabuzzi.sw1:iscsi1, portal: 192.168.103.7,3260] successful.
[root@xen1 ~]# iscsiadm -m node -T iqn.2004-07.com.cabuzzi.sw1:iscsi1 -p 192.168.104.7 -l
Logging in to [iface: default, target: iqn.2004-07.com.cabuzzi.sw1:iscsi1, portal: 192.168.104.7,3260] (multiple)
Login to [iface: default, target: iqn.2004-07.com.cabuzzi.sw1:iscsi1, portal: 192.168.104.7,3260] successful.
[root@xen1 ~]# iscsiadm -m node -T iqn.2004-07.com.cabuzzi.sw1:iscsi1 -p 192.168.103.7 -l
[root@xen1 ~]# multipath -ll
36589cfc000000b8c7d2eba0fc493b9e8 dm-0 FreeBSD,iSCSI Disk
size=7.0T features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
|- 2:0:0:0 sdb 8:16 active ready running
|- 3:0:0:0 sdc 8:32 active ready running
|- 4:0:0:0 sdd 8:48 active ready running
`- 5:0:0:0 sde 8:64 active ready running
[root@xen1 ~]# xe sr-probe type=lvmoiscsi device-config:target=192.168.101.7,192.168.102.7,192.168.103.7,192.168.104.7 device-config:targetIQN=*
^C
(had to halt the command, as it was not replying)
Here is the output of my /etc/multipath.conf file:
Code: Select all
[root@xen1 ~]# more /etc/multipath.conf
# This configuration file is used to overwrite the built-in configuration of
# multipathd.
# For information on the syntax refer to `man multipath.conf` and the examples
# in `/usr/share/doc/device-mapper-multipath-*/`.
# To check the currently running multipath configuration see the output of
# `multipathd -k"show conf"`.
defaults {
user_friendly_names no
replace_wwid_whitespace yes
dev_loss_tmo 30
}
devices {
device {
vendor "DataCore"
product "SAN*"
path_checker "tur"
path_grouping_policy failover
failback 30
}
device {
vendor "DELL"
product "MD36xx(i|f)"
features "2 pg_init_retries 50"
hardware_handler "1 rdac"
path_selector "round-robin 0"
path_grouping_policy group_by_prio
failback immediate
rr_min_io 100
path_checker rdac
prio rdac
no_path_retry 30
}
device {
vendor "DGC"
product ".*"
detect_prio yes
retain_attached_hw_handler yes
}
device {
vendor "EQLOGIC"
product "100E-00"
path_grouping_policy multibus
path_checker tur
failback immediate
path_selector "round-robin 0"
rr_min_io 3
rr_weight priorities
}
device {
vendor "IBM"
product "1723*"
hardware_handler "1 rdac"
path_selector "round-robin 0"
path_grouping_policy group_by_prio
failback immediate
path_checker rdac
prio rdac
}
device {
vendor "NETAPP"
product "LUN.*"
dev_loss_tmo 30
}
device {
vendor "FreeBSD"
product "iSCSI Disk"
path_grouping_policy multibus
path_selector "round-robin 0"
rr_min_io 1
}
device {
vendor "STARWIND"
product "STARWIND*"
path_grouping_policy multibus
path_checker "tur"
failback immediate
path_selector "round-robin 0"
rr_min_io 3
rr_weight uniform
hardware_handler "1 alua"
}
}
*** Per darklight's recommendation, I made no changes to the iscsid.conf file.
Here is a screenshot of the error when trying to add the SR via the XenCenter GUI:
One other thing I noticed... there seems to be no (obvious) way to specifically bind the four IPs I have allocated for iSCSI paths to the target in SW. When doing a netstat -an, it appears 3260 is listening on all IPs. Ideally, wouldn't you want 3261 only bound to the "management" interface of the SW server, and 3260 only bound to the IPs you mean to dedicate to iSCSI traffic?
Interestingly enough, netstat shows 3260 listening on all IPs (even the mgmt connection), as well as a connection to each of the Xen hosts... even though a "multipath -ll" in Xen only shows the FreeNAS connections:
Code: Select all
C:\Users\chris.adm>netstat -an | "findstr" 3260
TCP 127.0.0.1:3260 0.0.0.0:0 LISTENING
TCP 192.168.1.7:3260 0.0.0.0:0 LISTENING
TCP 192.168.1.7:3260 192.168.1.50:56721 ESTABLISHED
TCP 192.168.101.7:3260 0.0.0.0:0 LISTENING
TCP 192.168.101.7:3260 192.168.101.50:49165 ESTABLISHED
TCP 192.168.101.7:3260 192.168.101.51:39739 ESTABLISHED
TCP 192.168.102.7:3260 0.0.0.0:0 LISTENING
TCP 192.168.102.7:3260 192.168.102.50:55228 ESTABLISHED
TCP 192.168.102.7:3260 192.168.102.51:38212 ESTABLISHED
TCP 192.168.103.7:3260 0.0.0.0:0 LISTENING
TCP 192.168.103.7:3260 192.168.103.50:38704 ESTABLISHED
TCP 192.168.103.7:3260 192.168.103.51:52311 ESTABLISHED
TCP 192.168.104.7:3260 0.0.0.0:0 LISTENING
TCP 192.168.104.7:3260 192.168.104.50:37860 ESTABLISHED
TCP 192.168.104.7:3260 192.168.104.51:53057 ESTABLISHED