Can't get IBM library to work with TSM

Tape drive and auto-loader redirector over iSCSI

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Mon Nov 24, 2014 3:56 pm

It's the same all the time:

support@starwindsoftware.com

PS Just make sure you send logs zipped.
sidecar wrote:I added both files as attachments. From manage attachments I see no post. Is there an e-mail address I should send them to?

Thanks
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
boxcar
Posts: 10
Joined: Wed Sep 17, 2014 7:59 pm

Tue Nov 25, 2014 12:20 pm

I've used the tape redirector with RHEL 6 months ago with no problem, but at that time the library controller and driver was on the Hyper-V host and only the tape drives in use on the Linux VM. The difference here is that the library control is now in the Linux VM as well as the drive paths.

I'm offering the following information in case it helps in your diagnosis. sidecar and I are working on different systems and libraries but both are IBM branded libraries. Mine is a test system and I can set up a remote session any time if that would help.

I've managed to demonstrate the problem without the TSM backup software involved by using the IBM tape diagnostic tool (ITDT). ITDT performs library functions using the iin_tape driver. ITDT has a library diagnostic function and that passes with no error.

However, when I move a tape cartridge into a drive or try to move one from a drive I get the same error every time as shown below: Errno: 5 input/output error.

Before I increased the value of node.session.timeo.replacement_timeout (300, 600, 1000, now at 3000) I couldn't move slot to slot if I remember correctly, but slot to slot move now works.

When attempting to move to a drive or from a drive the cartridge actually moves to the destination slot or drive but returns the Errno: 5 error code - so it looks like we're very close.

Here's the failure sequence output:

itdt
u (for tape utility)
1 (open device)
/dev/IBMchanger0
1
52 (element inventory)
54 (move medium - find a slot element number for Source address and move to an empty slot for destination or the i/o port, or drive)

Command Result
+-----------------------------------------------------------------------------+
| Moving medium: source 257, destination 4116... |
| Move Medium FAILED |
| |
| Errno: 5 Input/output error |
| |
| Information Field Valid Bit-------0 |
| Error Code------------------------0x70 |
| Segment Number--------------------0x0 |
| File Mark Detected Bit------------0 |
| End Of Medium Bit-----------------0 |
| Illegal Length Indicator Bit------0 |
| Sense Key/ASC/ASCQ----------------0x00/0x00/0x00 |
| Information Bytes-----------------0 (00000000h) |
| Additional Sense Length-----------0x0A |
| Command Specific Information------0x00 0x00 0x00 0x00 |
| Field Replaceable Unit Code-------0x0 |
| Sense Key Specific Valid Bit------0 |
| Sense Data, Length 96 |
| |
| 0 1 2 3 4 5 6 7 8 9 A B C D E F 0123456789ABCDEF |
| 0000 - 7000 0000 0000 000A 0000 0000 0000 0000 [p...............] |
| 0010 - 0000 0000 0000 0000 0000 0000 0000 0000 [................] |
| 0020 - 0000 0000 0000 0000 0000 0000 0000 0000 [................] |
| 0030 - 0000 0000 0000 0000 0000 0000 0000 0000 [................] |
| 0040 - 0000 0000 0000 0000 0000 0000 0000 0000 [................] |
| 0050 - 0000 0000 0000 0000 0000 0000 0000 0000 [................] |
sidecar
Posts: 10
Joined: Fri Nov 14, 2014 4:54 pm

Tue Nov 25, 2014 5:08 pm

Valery,

Were you able to get anything from the files I sent?

Thanks
Val (staff)
Posts: 496
Joined: Tue Jun 29, 2004 8:38 pm

Wed Nov 26, 2014 6:46 am

Hi,

According the logs the iSCSI pings could lead to the problem.

So please try to turn off iSCSI pings by setting:
node.conn[0].timeo.noop_out_interval = 0
node.conn[0].timeo.noop_out_timeout = 0

node.session.timeo.replacement_timeout = 120

Also it's better to use separate targets for the tape and the changer as seperate sessions use separate network connections.
Best regards,
Valeriy
sidecar
Posts: 10
Joined: Fri Nov 14, 2014 4:54 pm

Wed Nov 26, 2014 4:15 pm

Valery,

Same results just as fast with the 3 changes to the iscsid.conf file.
Val (staff)
Posts: 496
Joined: Tue Jun 29, 2004 8:38 pm

Wed Nov 26, 2014 4:53 pm

Hi,

It's not a StarWind's issue, from the logs we can see that the Linux initiator uses too short timeouts.

Please ask your question about the Linux iSCSI timeout settings at a Linux forum.
Best regards,
Valeriy
boxcar
Posts: 10
Joined: Wed Sep 17, 2014 7:59 pm

Wed Nov 26, 2014 5:03 pm

Vallery,

Please see the log files I just sent. You'll discover an illegal request in the lin_tape.trace that may help you correlate what is going wrong.

This driver and backup software works fine with literally hundreds of brands of physical tape libraryies and many VTLs (some using iSCSI protocols). I'd like to believe we can make the adjustments to make it work with your iSCSI driver so we can make that available in a HyperV environment.

boxcar


---------------------------------------------------------------------
IBMtape1-----35872 Wed Nov 26 09:20:17 2014
Scsi Path : 04 00 00 00
CDB Command : 12 01 86 00 FF 00
Status Code : 08 00 00 01
Sense Data : 70 00 05 00 00 00 00 58 00 00 00 00 24 00 30 CF
00 02 00 00 00 01 53 30 31 30 30 37 00 00 00 4C
00 00 A9 06 00 00 00 00 00 30 12 80 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Description : Illegal Request
boxcar
Posts: 10
Joined: Wed Sep 17, 2014 7:59 pm

Wed Nov 26, 2014 5:20 pm

Sorry, forgot to include the error from the dmesg file:

lin_tape: IBMtape1-----35872 unable to get inquiry page 86
lin_tape: qry_blk_prot cannot get inquiry 0x86

that seems to have caused the illegal request in lin_tape.errorlog.
Val (staff)
Posts: 496
Joined: Tue Jun 29, 2004 8:38 pm

Wed Nov 26, 2014 5:32 pm

Hi,
this is not an error, just a warning that the optional INQUIRY VPD page 0x86 is not supported by the device.

As I've already said, the problem is that the Linux initiator drops iSCSI sessions because of too short timeout setting somewhere in the Linux iSCSI or SCSI data path.
Best regards,
Valeriy
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Fri Dec 05, 2014 4:32 pm

Update: Valery had fixed the issue so far. Valery well done :)
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
boxcar
Posts: 10
Joined: Wed Sep 17, 2014 7:59 pm

Thu Dec 18, 2014 10:56 pm

I'm happy to say that the IBM libraries, a TS3582 with LTO3s and a 3584 are working perfectly with TSM servers in physical and virtual machines.

I've tested with TSM servers in the Hyper-V host, in a Windows Storage Server 2012 R2 virtual machines, and RHEL 6.5 and 6.6. I've tested with the IBM primary library manager in the Hyper-V host and in the Linux virtual machines with equal success.

We're also using the Linux TSM instance to backup NetApp boxes with a shared NDMP library. Plenty of layers and complexity.

Thanks to Valeriy, Anton, and the StarWind engineering team!
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Fri Dec 19, 2014 12:03 pm

Excellent! Thank you very much for confirmation :)
boxcar wrote:I'm happy to say that the IBM libraries, a TS3582 with LTO3s and a 3584 are working perfectly with TSM servers in physical and virtual machines.

I've tested with TSM servers in the Hyper-V host, in a Windows Storage Server 2012 R2 virtual machines, and RHEL 6.5 and 6.6. I've tested with the IBM primary library manager in the Hyper-V host and in the Linux virtual machines with equal success.

We're also using the Linux TSM instance to backup NetApp boxes with a shared NDMP library. Plenty of layers and complexity.

Thanks to Valeriy, Anton, and the StarWind engineering team!
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
Post Reply