StarWind iSCSI SAN V5.7 Build 20110524

Public beta (bugs, reports, suggestions, features and requests)

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

User avatar
Bohdan (staff)
Staff
Posts: 435
Joined: Wed May 23, 2007 12:58 pm

Wed May 25, 2011 3:36 pm

Dear Beta-Team members,

StarWind V5.7 build 20110524 beta is available for download from the following link:

http://www.starwindsoftware.com/tmplink ... 110524.exe


New features and improvements:

Performance monitor feature added. From StarWind management console you can monitor disk transfer rates, average number of I/O operations on targets, CPU and memory load on StarWind server.

Snapshot manager feature added. It allows viewing, deleting and merging exiting snapshots with convenient and simple GUI. Select “Snapshot manager” option in context menu of device with snapshots support.

Event log: Notifications about new records in StarWind event log in system tray.

GUI: Targets and servers can be arranged in groups.

High availability: Asynchronous mode implemented to improve virtual device operation speed.
High availability: Added option to set traffic priority on sync channel during device synchronization. Device synchronization process can generate high load to disk system and partner. It affects performance of client requests processing. Now you can set priority for device synchronization traffic. Select “Sync channel priorities” option in HA device context menu.

Deduplication: new version of experimental deduplication plugin. Added ability to set deduplication block size (512 bytes - 256kB). Deduplication device implements low-level data deduplication. It allows saving disk space significantly when user data contains several
copies of similar data.
Deduplication device is in testing stage, use it on your own risk!

ImageFile, DiskBridge: added support of the Mode Sense page 0x3 for compatibility with Solaris iSCSI initiator.

Core: fixed LUN Reset command processing.

Registered beta testers have already received their licenses. If you are interested in v5.7 beta, please contact us to obtain the license.

Do not hesitate to contact us should you have any questions.
nbarsotti
Posts: 38
Joined: Mon Nov 23, 2009 6:22 pm

Wed May 25, 2011 6:54 pm

Can I upgrade from a v5.6 HA test intall to this v5.7 HA beta? If so what is the best correct procedure?
User avatar
Bohdan (staff)
Staff
Posts: 435
Joined: Wed May 23, 2007 12:58 pm

Thu May 26, 2011 7:01 am

Sure. The information how to do that can be found on the "Information" step of the installer.

Installation notes

Install: Previous versions can be updated by installing this version over existing installation.
HA devices can be upgraded without taking storage offline:
- Update StarWind on the first HA node.
- Perform synchronization for HA devices. Wait for synchronization process to be finished.
- Update StarWind on the second HA node.
- Perform synchronization for HA devices.
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Thu May 26, 2011 2:16 pm

I just finished upgrading my 5.6 HA to the 5.7 Beta. Everything went smoothly. Testing is going well so far with HA write speeds a little more than twice the speed of 5.6 with no other changes tested yet.
nbarsotti
Posts: 38
Joined: Mon Nov 23, 2009 6:22 pm

Thu May 26, 2011 2:23 pm

Just to add my $0.02. I had one upgrade of a 5.6 HA pair go very smoothly and I had another upgrade of a 5.6 HA pair go horribly bad. When the second node upgraded and restarted the service the first node I upgraded to 5.7 hung. I could not connect using the console and it was not responding to any of the vmware servers. At that point I had restart the service on the first node that was upgraded then I had two server both not synced. I had to take everything down, roll back to 5.6, recreate all my targets, and do a full sync back to the second node.

But yes, the HA write speeds seem much better on the pair that upgraded well.
User avatar
Bohdan (staff)
Staff
Posts: 435
Joined: Wed May 23, 2007 12:58 pm

Thu May 26, 2011 2:31 pm

nbarsotti, WOW :(
Could you please send us StarWind logs form both HA nodes? We must find what's wrong.
Thank you guys for keeping us updated!
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Thu May 26, 2011 2:50 pm

For the record, I stopped the StarWind service on the first server, then upgraded it. It ended up doing a full sync that took 2 days to sync the 23 TB of targets, but no errors or problems. Then I stopped the StarWind service on the second server, upgraded it, and it ended up doing a fast sync and was done in about 5 minutes.
nbarsotti
Posts: 38
Joined: Mon Nov 23, 2009 6:22 pm

Thu May 26, 2011 3:44 pm

Hello rchisholm, When you stay First and Second server are you referring to order in which you upgraded there servers. Or do you mean the servers that are listed as Primary and Secondary when you create the HA target? On your clients which server is main target or are they using round robin?

On my two HA pair upgrades I first installed 5.7 onto the Starwind server that was listed as secondary during HA creation. I did not stop the service manually and let the installer do that for me. After the install was finished the server came back online and did a fast sync of all three targets. Then I did the install on the server listed as Primary and that is when the newly installed 5.7 server hung up. My ESXi servers used fixed path policy and the prefered server was the Primary Starwind server
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Thu May 26, 2011 3:54 pm

By first and second, I was only referring to the order I upgraded the servers.

I had the paths for the ESXi servers setup as VMW_PSP_FIXED_AP because I was getting much better real world write speeds than with Round Robin. I am currently installing a 2008R2 virtual server with the path selection set as Round Robin and it's running nice and fast with the 5.7 Beta.
kmax
Posts: 47
Joined: Thu Nov 04, 2010 3:37 pm

Thu May 26, 2011 10:59 pm

Thanks for the comments. Can you expand on the performance improvements of 5.7?

As an aside, 23TB of targets in my mind is incredible. Can you describe your disk subsystem and network interfaces on that?
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Fri May 27, 2011 3:06 am

kmax wrote:Thanks for the comments. Can you expand on the performance improvements of 5.7?

As an aside, 23TB of targets in my mind is incredible. Can you describe your disk subsystem and network interfaces on that?
The sync for an indivdual target was only about 10-15% of a 10GbE link and now it is 25-30%. It also seems to be scaling linearly. 2 targets writing simultaneously is running 50-60% of the sync channel.

Each of the boxes are dual quad core 2.4GHz 5600 series Xeons with 72GB DDR3 RAM. The OS are on 500GB SAS RAID 1's. The storage in each StarWind server is currently 48 Seagate Constellation 2TB SAS drives connected to 2 Areca 1880 24 port cards. 24 are configured as a RAID 60 on one of the controllers, and there are 2 sets of 12 drive RAID 6's on the other controller. There are also 2 LSI 9280-8e controllers in each box with a 24 drive LSI 620J JBOD loaded with 4 Intel X25-E SSD's for cachecade and 20 146 15K Seagate SAS drives each. Each of the boxes has 2 HP dual 10GbE SFP+ NIC's. 2 ports are for iSCSI that are each connected to an HP E6600 24 port SFP+ switch, and the other 2 for sync channels that are connected directly.

I tested a bunch of different RAID configurations such as RAID 5, 10, 6, and 60, but the limitation on speed so far is the iSCSI. Running the faster writing RAID's really hasn't made any difference since the RAID 6 and 60's are no where near the bottlenecks of the systems right now. There is about 80TB of 7.2K RAID's in each box, and 4.5TB of 15K RAID's in each box. I'm actually going to move the 4 2U LSI JBOD's to be DAS storage for our next build of our main SQL server since I'm really not getting a noticeable performance difference between them and the much cheaper 7.2K's that are inside the 9U Chenbro cases. The speed I'm getting with version 5.7 with the cheaper drives in RAID 6 and 60's is plenty for the VM's that these SAN's will be used for.
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Fri May 27, 2011 7:23 am

You should notice 10-15% read performance gain and 2x-3x times better writes. Also latency had dropped down dramatically (you should notice this on very small I/O packets) together with faster turn-around time.
rchisholm wrote:I just finished upgrading my 5.6 HA to the 5.7 Beta. Everything went smoothly. Testing is going well so far with HA write speeds a little more than twice the speed of 5.6 with no other changes tested yet.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Fri May 27, 2011 7:35 am

You should see 80-90% link speed saturation even on a single target under heavy load. Could you please run some test / simulate load? Also latency for 5.6 Vs. 5.7 is subject of a great interest.

That's a confirmation of my theory... Cheap components with maximum redundancy :)
rchisholm wrote:
kmax wrote:Thanks for the comments. Can you expand on the performance improvements of 5.7?

As an aside, 23TB of targets in my mind is incredible. Can you describe your disk subsystem and network interfaces on that?
The sync for an indivdual target was only about 10-15% of a 10GbE link and now it is 25-30%. It also seems to be scaling linearly. 2 targets writing simultaneously is running 50-60% of the sync channel.

Each of the boxes are dual quad core 2.4GHz 5600 series Xeons with 72GB DDR3 RAM. The OS are on 500GB SAS RAID 1's. The storage in each StarWind server is currently 48 Seagate Constellation 2TB SAS drives connected to 2 Areca 1880 24 port cards. 24 are configured as a RAID 60 on one of the controllers, and there are 2 sets of 12 drive RAID 6's on the other controller. There are also 2 LSI 9280-8e controllers in each box with a 24 drive LSI 620J JBOD loaded with 4 Intel X25-E SSD's for cachecade and 20 146 15K Seagate SAS drives each. Each of the boxes has 2 HP dual 10GbE SFP+ NIC's. 2 ports are for iSCSI that are each connected to an HP E6600 24 port SFP+ switch, and the other 2 for sync channels that are connected directly.

I tested a bunch of different RAID configurations such as RAID 5, 10, 6, and 60, but the limitation on speed so far is the iSCSI. Running the faster writing RAID's really hasn't made any difference since the RAID 6 and 60's are no where near the bottlenecks of the systems right now. There is about 80TB of 7.2K RAID's in each box, and 4.5TB of 15K RAID's in each box. I'm actually going to move the 4 2U LSI JBOD's to be DAS storage for our next build of our main SQL server since I'm really not getting a noticeable performance difference between them and the much cheaper 7.2K's that are inside the 9U Chenbro cases. The speed I'm getting with version 5.7 with the cheaper drives in RAID 6 and 60's is plenty for the VM's that these SAN's will be used for.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
rchisholm
Posts: 63
Joined: Sat Nov 27, 2010 7:38 pm

Fri May 27, 2011 12:30 pm

These are real world results of copying files, installing operating systems, etc, for HA writes on 10GbE links. With 5.6, I found that tweaking it to get the highest speeds with simulated heavy loads resulted in extremely poor real world performance. Currently, with a realistic simulated load using SQLIO with 64K IO's, 8 outstanding, no buffering, and 2 threads, in a VSphere 2008 R2 VM to 1 HA target, it gets about 300 MB/s sequential writes. That is similar to the speeds I'm seeing in real world tests. Of course, if I'm copying files from a machine that only has a 1GbE connection, it is limited to peaks of about 100 MB/s which is to be expected.

As far as I was aware, no one was getting more than 15-20% sync channel saturation on HA writes to a single target with 5.6 on 10GbE links. Should 5.7 be 4-5X as fast on the HA writes instead of the 2X as fast that I'm seeing? If so, I think I need to revisit some of the tweaking tests that I tried with 5.6.

Edit: Also, with iPerf I get 98.5% saturation of the links as long as I use a large enough TCP window such as 4MB, so the networking hardware appears to be good.
anton (staff) wrote:You should see 80-90% link speed saturation even on a single target under heavy load. Could you please run some test / simulate load? Also latency for 5.6 Vs. 5.7 is subject of a great interest.

That's a confirmation of my theory... Cheap components with maximum redundancy :)
rchisholm wrote:
kmax wrote:Thanks for the comments. Can you expand on the performance improvements of 5.7?

As an aside, 23TB of targets in my mind is incredible. Can you describe your disk subsystem and network interfaces on that?
The sync for an indivdual target was only about 10-15% of a 10GbE link and now it is 25-30%. It also seems to be scaling linearly. 2 targets writing simultaneously is running 50-60% of the sync channel.

Each of the boxes are dual quad core 2.4GHz 5600 series Xeons with 72GB DDR3 RAM. The OS are on 500GB SAS RAID 1's. The storage in each StarWind server is currently 48 Seagate Constellation 2TB SAS drives connected to 2 Areca 1880 24 port cards. 24 are configured as a RAID 60 on one of the controllers, and there are 2 sets of 12 drive RAID 6's on the other controller. There are also 2 LSI 9280-8e controllers in each box with a 24 drive LSI 620J JBOD loaded with 4 Intel X25-E SSD's for cachecade and 20 146 15K Seagate SAS drives each. Each of the boxes has 2 HP dual 10GbE SFP+ NIC's. 2 ports are for iSCSI that are each connected to an HP E6600 24 port SFP+ switch, and the other 2 for sync channels that are connected directly.

I tested a bunch of different RAID configurations such as RAID 5, 10, 6, and 60, but the limitation on speed so far is the iSCSI. Running the faster writing RAID's really hasn't made any difference since the RAID 6 and 60's are no where near the bottlenecks of the systems right now. There is about 80TB of 7.2K RAID's in each box, and 4.5TB of 15K RAID's in each box. I'm actually going to move the 4 2U LSI JBOD's to be DAS storage for our next build of our main SQL server since I'm really not getting a noticeable performance difference between them and the much cheaper 7.2K's that are inside the 9U Chenbro cases. The speed I'm getting with version 5.7 with the cheaper drives in RAID 6 and 60's is plenty for the VM's that these SAN's will be used for.
User avatar
Bohdan (staff)
Staff
Posts: 435
Joined: Wed May 23, 2007 12:58 pm

Fri May 27, 2011 12:51 pm

Is it possible to perform the tests with SQLIO and iometer (64K, 64 outstanding, 8 or 16 threads/workers)?
On 10Gb cards (Intel AT) we are observing 70% sync channel utilization and 850MB/s. The difference is that we are running the tests on the physical machine but not the virtual one.

And one more question: have you tried to enable/disable delayed ACK option on the ESX side?
http://www.starwindsoftware.com/forums/ ... 98-15.html
Post Reply