Tape Redirector accessible from ESXi 4.1 software iSCSI?

Tape drive and auto-loader redirector over iSCSI

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

chris_jones
Posts: 11
Joined: Tue Aug 07, 2012 8:29 am

Tue Aug 07, 2012 8:42 am

I've been testing backing up our ESXi 5 VM's using a locally attached LTO3 drive, to an ESXi 4.1 server (as its not supported on 5.x) running Arkeia, which works great, quick/free.

Is it possible to use the iSCSI Tape Redirector on a physical 2003/2008 server, and connect the ESXi 4.1 server over iSCSI to the tape drive, thus negating the need for a dedicated/local tape drive on the ESXi 4.1 Arkeia box?

I ask, because I tried, and all I get is a "dead" connection to the Tape, according to ESXi's iSCSI status. The path is "dead", there are no "devices" listed. The target is shown in the static discovery, and was found by the dynamic discovery.
Connecting to the redirector from a windows machine is fine, connecting from ESXi to another iSCSI provider is fine, so it appears to be a Tape Redirector only issue.

There's no auth setup within Tape Redirector, just keeping it plain/simple till it works.

Logs below from the Tape Redirector.

Code: Select all

8/7 9:05:27.166 484 Srv: Accepted iSCSI connection from 172.16.100.50:50423 to 172.16.100.7:33268. (Id = 0xd)
8/7 9:05:27.166 484 C[d], FREE: Event - CONNECTED.
8/7 9:05:27.166 484 C[d], XPT_UP: T3.
8/7 9:05:27.166 1160 C[d], XPT_UP: Login request: ISID 0x00023d000000, TSIH 0x0000.
8/7 9:05:27.166 1160 C[d], XPT_UP: Event - LOGIN.
8/7 9:05:27.166 1160 C[d], IN_LOGIN: T4.
8/7 9:05:27.166 1160 Params: <<< String param 'InitiatorName': received 'iqn.1998-01.com.vmware:vmware04-0209ec20', accepted 'iqn.1998-01.com.vmware:vmware04-0209ec20'
8/7 9:05:27.166 1160 Params: <<< Enum param 'SessionType': received 'Discovery', accepted 'Discovery'
8/7 9:05:27.166 1160 Params: <<< Enum param 'HeaderDigest': received 'None', accepted 'None'
8/7 9:05:27.166 1160 Params: <<< Enum param 'DataDigest': received 'None', accepted 'None'
8/7 9:05:27.166 1160 Params: <<< Numeric param 'DefaultTime2Wait': received 0, accepted 0
8/7 9:05:27.166 1160 Params: <<< Numeric param 'DefaultTime2Retain': received 0, accepted 0
8/7 9:05:27.166 1160 Params: <<< Boolean param 'IFMarker': received No, accepted 0
8/7 9:05:27.166 1160 Params: <<< Boolean param 'OFMarker': received No, accepted 0
8/7 9:05:27.166 1160 Params: <<< Numeric param 'ErrorRecoveryLevel': received 0, accepted 0
8/7 9:05:27.166 1160 Params: <<< Numeric param 'MaxRecvDataSegmentLength': received 32768, accepted 32768
8/7 9:05:27.166 1160 Params: >>> ErrorRecoveryLevel=0.
8/7 9:05:27.166 1160 Params: >>> HeaderDigest=None.
8/7 9:05:27.166 1160 Params: >>> DataDigest=None.
8/7 9:05:27.166 1160 Params: >>> OFMarker=No.
8/7 9:05:27.166 1160 Params: >>> IFMarker=No.
8/7 9:05:27.166 1160 Params: >>> InitialR2T=No.
8/7 9:05:27.166 1160 Params: >>> ImmediateData=Yes.
8/7 9:05:27.166 1160 Params: >>> MaxRecvDataSegmentLength=262144.
8/7 9:05:27.166 1160 Params: >>> MaxBurstLength=262144.
8/7 9:05:27.166 1160 Params: >>> FirstBurstLength=262144.
8/7 9:05:27.166 1160 Params: >>> DefaultTime2Wait=0.
8/7 9:05:27.166 1160 Params: >>> DefaultTime2Retain=0.
8/7 9:05:27.166 1160 Params: >>> MaxOutstandingR2T=1.
8/7 9:05:27.166 1160 Params: >>> MaxOutstandingR2T=1.
8/7 9:05:27.166 1160 Params: >>> DataPDUInOrder=Yes.
8/7 9:05:27.166 1160 Params: >>> DataSequenceInOrder=Yes.
8/7 9:05:27.166 17d4 T[d,1]: session 0xd, connection 0xd : end of stage 1, next stage 3.
8/7 9:05:27.166 17d4 C[d], IN_LOGIN: Event - LOGIN_ACCEPT.
8/7 9:05:27.166 17d4 C[d], LIN: T5.
8/7 9:05:27.166 17d4 C[d], LIN: Event - LOGOUT.
8/7 9:05:27.166 17d4 C[d], IN_LOGOUT: T9/10.
8/7 9:05:27.166 17e8 C[d], IN_LOGOUT: Event - LOGOUT_ACCEPT.
8/7 9:05:27.166 17e8 C[d], FREE: T13.
8/7 9:05:27.166 1160 C[d], FREE: recvData returned 10054
Thoughts appreciated.

Thanks
Chris
User avatar
anton (staff)
Site Admin
Posts: 4010
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Tue Aug 07, 2012 12:48 pm

Don't use ESXi built-in initator. You're willing to connect to tape from inside a VM? Then use VMs own system-dependent software iSCSI initiator.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
chris_jones
Posts: 11
Joined: Tue Aug 07, 2012 8:29 am

Tue Aug 07, 2012 12:58 pm

Thanks for the response Anton, unfortunately I need to present the device as a SCSI tape to Arkeia (using SCSI passthrough), rather than Arkeia connect to an iSCSI device.

As per http://kb.vmware.com/selfservice/micros ... Id=1016407

Thanks
Chris
User avatar
anton (staff)
Site Admin
Posts: 4010
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Tue Aug 07, 2012 1:06 pm

OK, I see... Can you create a dummy test device (disk) with StarWind and check can you connect to it? Then we'll proceed with troubleshooting... Thanks!
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
chris_jones
Posts: 11
Joined: Tue Aug 07, 2012 8:29 am

Tue Aug 07, 2012 1:22 pm

If I understand you correctly, I'm only using the free Tape Redirector, not the full iSCSI SAN product which used to incorporate the Tape Redirector, so can't create a disk target.

Or do you want me to install the iSCSI SAN product and check ESXi can connect to that?

The Tape Redirector works fine from a Windows Server using both MS iSCSI Initiator 2.08 and the SW iSCSI Initiator, also the ESXi 4.1 machine can connect to a test iSCSI HDD Target (Thecus N8800Pro). Not that any of that really helps I'm afraid, other than to prove the comms are all OK.

Thanks
Chris
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Tue Aug 07, 2012 1:25 pm

well, the first reason why it could happen is because the device is locked by some other application, would you please doublecheck that?
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
chris_jones
Posts: 11
Joined: Tue Aug 07, 2012 8:29 am

Tue Aug 07, 2012 1:47 pm

The device is usable when connected to from a Windows host (over iSCSI), so not locked by another app preventing that.

With the windows host disconnected, the session count in the management console is blank, when connected to by the ESXi server the session count is 1, as below (note ESXi intiator name changed from earlier logs).

Image

Image

Thanks
Chris
chris_jones
Posts: 11
Joined: Tue Aug 07, 2012 8:29 am

Tue Aug 07, 2012 2:09 pm

Just dug out the /var/log/messages from ESXi for the iSCSI connected tape.

Aug 7 13:59:20 iscsid: discovery_sendtargets::Running discovery on IFACE default(iscsi_vmk) (drec.transport=iscsi_vmk)
Aug 7 13:59:20 iscsid: Login Target Skipped: iqn.2008-08.com.starwindsoftware:jetpac.boxtech.local-lto3 if=default addr=172.16.100.7:33268 (TPGT:-1 ISID:0x1) (Already Running)
Aug 7 13:59:20 iscsid: DISCOVERY: Pending=0 Failed=0
Aug 7 13:59:20 vmkernel: 13:04:16:11.881 cpu1:182702)ScsiScan: 1059: Path 'vmhba34:C0:T0:L0': Vendor: 'HP ' Model: 'Ultrium 3-SCSI ' Rev: 'G66D'
Aug 7 13:59:20 vmkernel: 13:04:16:11.881 cpu1:182702)ScsiScan: 1062: Path 'vmhba34:C0:T0:L0': Type: 0x1, ANSI rev: 3, TPGS: 1 (implicit only)
Aug 7 13:59:20 vmkernel: 13:04:16:11.889 cpu0:182702)WARNING: VMW_SATP_ALUA: satp_alua_getTargetPortInfo: Could not find relative target port ID for path "vmhba34:C0:T0:L0" - Not found (195887107)
Aug 7 13:59:20 vmkernel: 13:04:16:11.889 cpu0:182702)WARNING: NMP: nmp_SatpClaimPath: SATP "VMW_SATP_ALUA" could not add path "vmhba34:C0:T0:L0" for device "Unregistered Device". Error Not found
Aug 7 13:59:20 vmkernel: 13:04:16:11.889 cpu0:182702)WARNING: NMP: nmp_DeviceAlloc: nmp_AddPathToDevice failed Not found (195887107).
Aug 7 13:59:20 vmkernel: 13:04:16:11.889 cpu0:182702)WARNING: NMP: nmp_DeviceAlloc: Could not allocate NMP device.
Aug 7 13:59:20 vmkernel: 13:04:16:11.889 cpu0:182702)WARNING: ScsiPath: 3848: Plugin 'NMP' had an error (Not found) while claiming path 'vmhba34:C0:T0:L0'.Skipping the path.
Aug 7 13:59:20 vmkernel: 13:04:16:11.889 cpu0:182702)ScsiClaimrule: 1183: Plugin NMP specified by claimrule 65535 was not able to claim path vmhba34:C0:T0:L0. Busy
Aug 7 13:59:20 vmkernel: 13:04:16:11.889 cpu0:182702)ScsiClaimrule: 1405: Error claiming path vmhba34:C0:T0:L0. Busy.
Aug 7 13:59:20 vmkernel: 13:04:16:12.128 cpu1:4691)NMP: nmp_CompleteCommandForPath: Command 0x12 (0x41027f372340) to NMP device "mpx.vmhba0:C0:T0:L0" failed on physical path "vmhba0:C0:T0:L0" H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.
Aug 7 13:59:20 vmkernel: 13:04:16:12.128 cpu1:4691)ScsiDeviceIO: 1688: Command 0x12 to device "mpx.vmhba0:C0:T0:L0" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x24 0x0.

For the working physical localy SCSI device its:

Aug 7 14:02:41 vmkernel: 13:04:19:33.334 cpu1:182702)ScsiScan: 1059: Path 'vmhba2:C0:T4:L0': Vendor: 'IBM ' Model: 'ULTRIUM-TD2 ' Rev: '67U1'
Aug 7 14:02:41 vmkernel: 13:04:19:33.334 cpu1:182702)ScsiScan: 1062: Path 'vmhba2:C0:T4:L0': Type: 0x1, ANSI rev: 3, TPGS: 0 (none)

Odd that the bus, target and LUN are all 0 for the iSCSI, I'm guessing that's where the issue lies. In case it can't handle more than 1 SCSI passthrough at a time, I'll disconnect the physical local drive and reboot the ESX machine.

Update: No difference with the physical/local LTO drive disconnected.
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Fri Aug 10, 2012 10:30 am

I`ve just had pretty short conversation with vExpert that I`m familiar with, and he assured me that it is impossible to achieve your plan anyway.
I would recommend you to ask this question to VMware support if its possible.
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
chris_jones
Posts: 11
Joined: Tue Aug 07, 2012 8:29 am

Fri Aug 10, 2012 11:06 am

Thanks Anatoly.

I assume the StarWind products support connections from ESXi normally?

Which part of the scenario doesn't the vExpert think will work, the ESXi initiator to StarWind Redirector? ESXi will connect to other iSCSI tape devices so no reason it shouldn't connect to the StarWind one. The rest of the solution is proven/working.

Thanks for your time.
Regards
Chris
User avatar
anton (staff)
Site Admin
Posts: 4010
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Fri Aug 10, 2012 11:39 am

Yes, that's what I cannot get as well...
chris_jones wrote:Thanks Anatoly.

I assume the StarWind products support connections from ESXi normally?

Which part of the scenario doesn't the vExpert think will work, the ESXi initiator to StarWind Redirector? ESXi will connect to other iSCSI tape devices so no reason it shouldn't connect to the StarWind one. The rest of the solution is proven/working.

Thanks for your time.
Regards
Chris
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Fri Aug 10, 2012 3:03 pm

ESXi will connect to other iSCSI tape devices so no reason it shouldn't connect to the StarWind one.
That is kinda surprising to me. I was assured that yes, ESX will connect to tape using its initiator, StarWind will give everything that is needed, but ESX will not work with it because it is simply wasn`t designed for that.

So, can you confirm that you have successfuly connected to the tape device using other SPTI-compatible software?
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
chris_jones
Posts: 11
Joined: Tue Aug 07, 2012 8:29 am

Tue Aug 14, 2012 3:49 pm

Anatoly (staff) wrote:So, can you confirm that you have successfuly connected to the tape device using other SPTI-compatible software?
Assuming Backup Exec 9.1counts as SPTI compatible in this scenario, then yes, it's currently working doing daily backs up to the Tape Redirected drive.

For what it's worth, I've also tested with a **** product, had the same "dead" link with that too.

Perhaps at this point I give up just pleased that it works so well windows-to-windows, achieving the same speeds as the FC solution already in place without the speciality hardware required.

Thanks for the help.
Regards
Chris
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Fri Aug 17, 2012 10:19 am

So that is what I`ve told you - ESX cannot work with tapes directly conencted to it. You need to connect to tape from VM.
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
chris_jones
Posts: 11
Joined: Tue Aug 07, 2012 8:29 am

Fri Aug 17, 2012 10:48 am

I'm not really asking ESXi to work with the Tape drive, I'm asking it to pass it through to the VM in exactly the same way as it does with a directly connected SCSI tape drive.

This is from the ESXi console:

Code: Select all

~ # ls -lsa /dev/genscsi/ | grep -i 'ultrium'
   0 -rw-------    1 root     root                  0 Aug 17 10:34 t10.IBM_____ULTRIUM2DTD2_____1110205616 
This is then available to the VM as a locally attached SCSI device, as if the VM was a physical machine.

Code: Select all

[root@arkeia01 ~]# dmesg | grep -i 'ultrium'
scsi 3:0:4:0: Sequential-Access IBM      ULTRIUM-TD2      67U1 PQ: 0 ANSI: 3 
Given that a HDD could be made available in exactly the same way using an iSCSI connected drive, it doesn't appear unthinkable that an iSCSI tape should be able to.
Post Reply