Intemittent connection problem from Xen to Starwind v3.5.4

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

Post Reply
MattBoB
Posts: 26
Joined: Thu Nov 08, 2007 11:22 am

Fri Jul 18, 2008 8:47 am

I am using 2 servers running Starwind to provide storage to 3 clsutered Citrix Xenserver virtual machine servers.

Since upgrading one of the Starwind servers from v3.5.3 to v3.5.4 I have been experiencing intermittent problems with iSCSI connections to the v3.5.4 server.

If a Xen server is restarted it seems to have problems connecting to the starwind server if the other 2 Xen servers are already connected. I have also had one incident now where one of the xen servers lost its connection to an iSCSI share while in normal use. When I run the reconnect command in xen I get a login failure message in xen. If I keep retrying the connection it eventually works.

Any ideas?
User avatar
Bohdan (staff)
Staff
Posts: 435
Joined: Wed May 23, 2007 12:58 pm

Tue Jul 22, 2008 6:14 am

Hi.

Please contact our support team (support@rocketdivision.com) and provide us with recent StarWind logs. We'll find what's wrong.
Thank you.

Also please change <iScsiPingPeriod value="0" /> to <iScsiPingPeriod value="5" /> in the <options> section of starwind.cfg file.

Also please configure interfaces to listen:
Add several rows like <connection value="address:port"> in <connections> section.
Fore example:
<connections>
<!--Port number for iSCSI connectons (default 3260)-->
<!-- <Port value="3260"/> -->
<!--Interface to listen to. "0.0.0.0" corresponds to "listen to all interfaces".-->
<!-- <Interface value="192.168.1.234"/> -->
<!--Port number for Control connectons (default 3261)-->
<!-- <CtlPort value="3261"/> -->
<!--Interface to listen to Control connections. "0.0.0.0" corresponds to "listen to all interfaces".-->
<!-- <CtlInterface value="0.0.0.0"/> -->
<connection value="192.168.32.1:3260"/>
<connection value="192.168.1.234:3260"/>
</connections>


Restart starwindservice.
MattBoB
Posts: 26
Joined: Thu Nov 08, 2007 11:22 am

Mon Aug 04, 2008 3:02 pm

I think the version of starwind was a red herring here, as I am now having problems with the server on the older version as well. I thought I had fixed it when I found a problem with the NTP config on the Xen servers which caused their time to be out of sync with rest of the network, although I had no problems for a week or so after fixing this the problem is now back again.

The only other comonality is that problems started on both servers after I upgraded them to windows service pack 2.

I will try the settings you suggested and send some logs, but as this server has multiple volumes in use by production systems I will have to pick my time to restart the service.

Thanks,
aaron (staff)
Posts: 70
Joined: Fri Jan 11, 2008 6:13 am
Location: BVI

Mon Aug 04, 2008 5:22 pm

SP2 should not be a problem. Unless NIC driver also changed and started to play it's own games. Sure, waiting for your logs :) Thank you!
MattBoB wrote:I think the version of starwind was a red herring here, as I am now having problems with the server on the older version as well. I thought I had fixed it when I found a problem with the NTP config on the Xen servers which caused their time to be out of sync with rest of the network, although I had no problems for a week or so after fixing this the problem is now back again.

The only other comonality is that problems started on both servers after I upgraded them to windows service pack 2.

I will try the settings you suggested and send some logs, but as this server has multiple volumes in use by production systems I will have to pick my time to restart the service.

Thanks,
Regards,
Aaron Korfer

Sales & Support
Rocket Division Software
MattBoB
Posts: 26
Joined: Thu Nov 08, 2007 11:22 am

Tue Sep 23, 2008 11:24 am

Since making the changes given above, and disabling the advanced networking functions which were included in service pack 2 (TOE etc..) I haven't had any more problems. I am experiencing some other problems with the OS on the server that caused majority of problems so I'm going to wipe it and rebuild OS from scratch. Let you know if I need any more assist.

Many thanks,
User avatar
anton (staff)
Site Admin
Posts: 4010
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Tue Sep 23, 2008 1:32 pm

Yes, please keep us updated! Thank you very much for cooperation!
MattBoB wrote:Since making the changes given above, and disabling the advanced networking functions which were included in service pack 2 (TOE etc..) I haven't had any more problems. I am experiencing some other problems with the OS on the server that caused majority of problems so I'm going to wipe it and rebuild OS from scratch. Let you know if I need any more assist.

Many thanks,
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
MattBoB
Posts: 26
Joined: Thu Nov 08, 2007 11:22 am

Fri Sep 26, 2008 11:23 am

OK I think I have finally tracked down the problems.

First of all Xenserver 4.0 seems to have a problem sometimes reconnecting to iSCSI drives after they unexpectedly disapear briefly. Restarting the open iSCSI service on affected xenserver fixes the problem when it occurs. I will upgrade to version 5.0 as soon as I can and hopefully this will be fixed.

The second problem was related to Windows SP2 which enables scalable networking functions automatically when the hardware supports it such as the Broadcom NIC's in the HP DL380 G5 servers I am using. unfortunately in the case of the broadcom NIC this causes all sorts of problems casuing various things to cut in and out etc, hence my problems with iSCSI drop outs. The fix for this seems to be the following command:

"Netsh int ip set chimney DISABLED"
aaron (staff)
Posts: 70
Joined: Fri Jan 11, 2008 6:13 am
Location: BVI

Fri Sep 26, 2008 6:20 pm

Wow! That's a nice piece of work! Thanks a lot for keeping us updated :) Hope your hints will help to somebody one day!
Regards,
Aaron Korfer

Sales & Support
Rocket Division Software
Post Reply