starwind, starport and performance

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Anatoly (staff), Max (staff)

Guest

Tue Nov 30, 2004 7:19 am

Made the changes myself...
Also added
SackOpts = 1
GlobalMaxTCPWindowSize=0x01400000

987Mb/s ;)

Couple of questions though...

1) What settings for IOmeter are you using?
2) When I mount my Raid Drive I can only do so as an ImageFile... is this normal and/or optimal?

Chuck
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Tue Nov 30, 2004 1:16 pm

Good! Pretty close to 1Gbps :-)

You need to set both reads and writes to 50%/50% (so it would emulate full duplex operations). Set 64KB transfers size and number of workers >10 (to emulate async requests). That's all...

About RAID. Yes, you can map it directly as SCSI LUN. Please check how SPTI devices should be shared (StarWind manual). If you'll have questions after you'll read this section -- go on asking :-)
Anonymous wrote:Made the changes myself...
Also added
SackOpts = 1
GlobalMaxTCPWindowSize=0x01400000

987Mb/s ;)

Couple of questions though...

1) What settings for IOmeter are you using?
2) When I mount my Raid Drive I can only do so as an ImageFile... is this normal and/or optimal?

Chuck
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
Chris

Fri Jan 14, 2005 12:30 am

It's amazing how TCP optimization can improve performance.
See my other post http://www.starwindsoftware.com/forums/ ... .php?t=225
Now, with TCP tunning, I set TCPWindow to 256K and tuned on all TCP options; now I'm getting 950Mb/s on the same hardware.

I also added 3ware 7504-LP RAID card and 3 200G Maxtor ATA disks, each with 8MB cache in RAID-0 mode on the server.
- Through CIF, when I trasfer large file, I'm getting 45% wite and 50% read sustained network utilization comparing to untuned TCP stack at 30% utilization.
- Through StarWind, I'm only getting 30% utilization using image file. I'm using MS initiator 1.06 because I have some problem with StarPort. StarPort will not reinistialize connection after network disconnect so the only way to get the connection back is to restart the machine.

Image file is the only way to get clustering working, but the preformance is not acceptable. Any comment on the image file portion?

Thanks
Chris
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Fri Jan 14, 2005 2:57 am

Chris,

1) Upcoming version of StarPort would have reconnects enabled. So you'll be able to dump MS junk (b/s of the poor performance compared to StarPort).

2) StarPort works faster then MS iSCSI initiator so you'll definitely have some of the performance boost only after MS -> RDS switch.

3) Upcoming version of StarWind is rewritten and image plugin works nearly like hard drive it's located on (very small overhead).

4) RAID0 would not help you as most of the iSCSI requests are quite small (smaller then RAID stripe size) so only one drive would physically handle request per time (no splitting amond different platters). MS (or Veritas?) LVM was not designed for handling comparably small requests. Fully software RAID built-in into StarWind would follow soon to workaround this.

1-3 would be available in a week or so (we're completing final tests) and 4 - I'm not sure about schedule yet.
Chris wrote:It's amazing how TCP optimization can improve performance.
See my other post http://www.starwindsoftware.com/forums/ ... .php?t=225
Now, with TCP tunning, I set TCPWindow to 256K and tuned on all TCP options; now I'm getting 950Mb/s on the same hardware.

I also added 3ware 7504-LP RAID card and 3 200G Maxtor ATA disks, each with 8MB cache in RAID-0 mode on the server.
- Through CIF, when I trasfer large file, I'm getting 45% wite and 50% read sustained network utilization comparing to untuned TCP stack at 30% utilization.
- Through StarWind, I'm only getting 30% utilization using image file. I'm using MS initiator 1.06 because I have some problem with StarPort. StarPort will not reinistialize connection after network disconnect so the only way to get the connection back is to restart the machine.

Image file is the only way to get clustering working, but the preformance is not acceptable. Any comment on the image file portion?

Thanks
Chris
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
Val (staff)
Posts: 496
Joined: Tue Jun 29, 2004 8:38 pm

Fri Jan 14, 2005 10:27 am

Chris wrote:It's amazing how TCP optimization can improve performance.
See my other post http://www.starwindsoftware.com/forums/ ... .php?t=225
Now, with TCP tunning, I set TCPWindow to 256K and tuned on all TCP options; now I'm getting 950Mb/s on the same hardware.

I also added 3ware 7504-LP RAID card and 3 200G Maxtor ATA disks, each with 8MB cache in RAID-0 mode on the server.
- Through CIF, when I trasfer large file, I'm getting 45% wite and 50% read sustained network utilization comparing to untuned TCP stack at 30% utilization.
- Through StarWind, I'm only getting 30% utilization using image file. I'm using MS initiator 1.06 because I have some problem with StarPort. StarPort will not reinistialize connection after network disconnect so the only way to get the connection back is to restart the machine.

Image file is the only way to get clustering working, but the preformance is not acceptable. Any comment on the image file portion?

Thanks
Chris
Chris,

Please download the updated StarWind and StarPort builds from our download area.

The StarWind version (build 0x20050109) works much better with ImageFiles as Anton said.

The StarPort version (build 0x20050108) supports reconnect on connection failure and includes several other updates.

Please let us know the results of your testing process.
Best regards,
Valeriy
Guest

Sat Jan 22, 2005 4:09 am

Upgraded to new version.

Now speed is between 31-33MB/s using image file, still far below 50MB/s through CIFS. Haven't got a chance to use entire RAID array.
Val (staff)
Posts: 496
Joined: Tue Jun 29, 2004 8:38 pm

Sat Jan 22, 2005 12:05 pm

Anonymous wrote:Upgraded to new version.

Now speed is between 31-33MB/s using image file, still far below 50MB/s through CIFS. Haven't got a chance to use entire RAID array.
Hi,

Which software did you use for the throughput tests?
What test settings did you use?

AFAIK CIFS hardly uses caching to speed up operations.
iSCSI does not use caching itself, so the results depend on your test settings (random or sequentional read-write, packet size, queue length, etc)

So please send us more details.
Thank you in advance.
Best regards,
Valeriy
Chris

Mon Jan 24, 2005 6:48 pm

I was copying 1G file over and monitor the bandwidth usage. Over CIFS, it
s 50MB/s, and over remote mounted iSCSI, it's 33MB/s.

I used IOmeter to check it again, now I get 70MB/s write and 50MB/s read on 1MB file.
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Mon Jan 24, 2005 8:53 pm

I get around 22-25 MB/sec over SMB for a big files and around 50 MB/sec for iSCSI. For bunch of small files iSCSI gets sometimes 10X faster b/s SMB impelmentation from Microsoft is not using network in the optimal way (network utilization for small files is around 10-15% and for big files is 90-95% -- easy to check with Performance Meter).

BTW, what iSCSI initiator are you using? I hope you don't use old StarWind (pre-2.4.3) and MS iSCSI initiator? Maybe this is the issue? Can you double check this?

Thank you!
Chris wrote:I was copying 1G file over and monitor the bandwidth usage. Over CIFS, it
s 50MB/s, and over remote mounted iSCSI, it's 33MB/s.

I used IOmeter to check it again, now I get 70MB/s write and 50MB/s read on 1MB file.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
Chris

Tue Jan 25, 2005 2:51 am

I use the latest StarPort and StarWind.

BTW, the 70MB/s write and 50MB/s read is under iSCSI. The reason read is slower must be my client machine's slow single disk config.
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Tue Jan 25, 2005 10:17 am

And if the same disk is tested under I/O meter what are the results?
Chris wrote:I use the latest StarPort and StarWind.

BTW, the 70MB/s write and 50MB/s read is under iSCSI. The reason read is slower must be my client machine's slow single disk config.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Tue Jan 25, 2005 10:17 am

And if the same disk is tested under I/O meter what are the results? I mean locally, w/o iSCSI mapping.
Chris wrote:I use the latest StarPort and StarWind.

BTW, the 70MB/s write and 50MB/s read is under iSCSI. The reason read is slower must be my client machine's slow single disk config.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
Chris

Tue Jan 25, 2005 7:51 pm

It's 80MB/s write and 150MB/s read.

Minus network overhead, disk overhead and filesystem overhead (image file), the result is very good,.
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Tue Jan 25, 2005 8:00 pm

150 MBps for read? I can bet you have some RAID and not single hard drive. In case of RAID everything is not so easy. Small requests (iSCSI can generate a lot of them) can easily get smaller then stripe size on you RAID. So single request would be physically executed by single hard drive only, thus performance would be degraded (in RAID requests > stripe size would be handled in parallel). If you're using RAID I guess that's why you see not so high numbers. If you're using single drive -- please name it :-) I dont't know any single hard disk drive capable of 150 MBps STR yet. Would love to get my hands over it.

We're getting here 1:1 or evern better mapping. So hard disk mapped over iSCSI shows the same (or better) sustained transfer rate (better b/s other machine CPU is used to serve requests). Only for combined read/writes picture is not so bright for iSCSI (network related issues).
Chris wrote:It's 80MB/s write and 150MB/s read.

Minus network overhead, disk overhead and filesystem overhead (image file), the result is very good,.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
Locked