End to End compression?

Fri Jan 13, 2006 12:18 pm

See, very few people are actually interested in this. So if you're representing a company interested in custom solution of StarPort with embedded iSCSI cache (RAM or hard disk - does not matter) - please drop me a message to anton@rocketdivision.com so we could discuss a budget. However if you're just a single individual looking for a nice feature - I'm sorry but I'm afraid you have very low chances to see it implemented as part of StarPort

See above why exactly. Thanks!

DJ_Datte wrote:Hey !

Actually, for my planned uses, and with my testing, a cache (say about 512MB used of a 1GB windows machine) would huge difference. I will need to store huge amounts of data on the iSCSI, but the stuff that is read in one session is allways very localized, or there are very many clients reading from the exact same file .... So a ram cache would be appriciated greatly, for read operations .. I understand both the security risks, and usually very small attractivness of write catche in thease scenarios...

/Damir

DJ_Datte · Sat Jan 14, 2006 6:29 pm

StarPort ? That wouldnt help much ? Or did you missunderstand me, becouse I would think that StarWind(target) should do the catching ?

Windows can do it too, if you dont require unbuffered reads only. And I am sure more people will be interested in a "fast" networked iSCSI, as iSCSI and SAN's become more popular with the buisiness world ?

/Damir

Lou · Sat Jan 14, 2006 7:49 pm

Windows can do it too, if you dont require unbuffered reads only. And I am sure more people will be interested in a "fast" networked iSCSI, as iSCSI and SAN's become more popular with the buisiness world ?

We did some experiments on a linux based iSCSI target with different buffering methods. The target app was emulating the targets with imagefiles, too. The imagefile was opened with different file open methods, unbuffered, read buffered, read & write buffered, buffering was done by linux kernel.

Especially in environments with a lot of small random requests the combined read & write buffering gave us a performance improvement of about 200%. Of course, write buffering is always dangerous but with at least a battery backup of the machine the risk is much more less than doing without. And the target must honour SCSI sync commands flushing the buffer.

Maybe this feature is added user selectable in a future version?

Val (staff) · Sat Jan 14, 2006 9:43 pm

Lou wrote:
Windows can do it too, if you dont require unbuffered reads only. And I am sure more people will be interested in a "fast" networked iSCSI, as iSCSI and SAN's become more popular with the buisiness world ?
We did some experiments on a linux based iSCSI target with different buffering methods. The target app was emulating the targets with imagefiles, too. The imagefile was opened with different file open methods, unbuffered, read buffered, read & write buffered, buffering was done by linux kernel.

Especially in environments with a lot of small random requests the combined read & write buffering gave us a performance improvement of about 200%. Of course, write buffering is always dangerous but with at least a battery backup of the machine the risk is much more less than doing without. And the target must honour SCSI sync commands flushing the buffer.

Maybe this feature is added user selectable in a future version?

Lou,

The feature is already in the ImageFile plug-in. Please read the thread 'Some questions' for more details.

DJ_Datte · Tue Jan 17, 2006 1:56 pm

Valery, as far as I understand you, you are not enterily correct.

Both me and Lou are asking for pure buffering on the imagefiles, let even the OS handle it.

That kind of functionality does NOT exist in StarWind today, as you yourself said in "Some QUestions" thred, when answering Lou.

when Lou says

Maybe this feature is added user selectable in a future version?

he is referring to "Target" buffering the ImageFiles, and Starwind, as a Target, doesent do that, does it ?

/Damir

Lou · Tue Jan 17, 2006 6:27 pm

DJ_Datte,

read my thread "some questions", there was the answer from valery (buffering:"yes" at the imagefile line.

Val (staff) · Tue Jan 17, 2006 11:07 pm

Lou, thank you for your help! :)

DJ_Datte, as Lou said, please reread the topic.
The buffering feature for ImageFiles has been there since early versions, but quite far ago we removed it from the user interface as the feature is very rarely used.
We'll return support of the parameter to the GUI in the next StarWind version.

DJ_Datte · Sun Jan 22, 2006 4:49 pm

Many thanks

I will run some benchmarks, and return my findings... Will probably take me a few weeks to get around to it tho.

/Damir

Val (staff) · Mon Jan 23, 2006 7:31 am

DJ_Datte wrote:Many thanks :)

I will run some benchmarks, and return my findings... Will probably take me a few weeks to get around to it tho.

/Damir

Damir,

Glad to help you. :)

Please let us know about the results.

Yves Smolders · Sat Feb 25, 2006 2:17 pm

You could use zebedee to tunnel iscsi - zebedee is a generic TCP tunnel that supports zlib compression.

I'm not sure if it will speed things up though... you could give it a try.

google for neil winton zebedee

Tim · Thu Mar 16, 2006 10:06 pm

Yves Smolders wrote:You could use zebedee to tunnel iscsi - zebedee is a generic TCP tunnel that supports zlib compression.

I'm not sure if it will speed things up though... you could give it a try.

google for neil winton zebedee

Hmmm, interesting.

I will have a look into that. I have not put much effort into making my test system un-bootable lately

.
I may have an excuse now

Fri Mar 17, 2006 12:11 pm

If someone would really try iSCSI over compressed tunell - please let us know about differences in performance compared with "native" way. We'll publish some docs referencing your name - could bring some good AD to you

Thanks!

Tim · Mon Mar 20, 2006 8:39 pm

Well I had a go at getting 'zebedee' working but failed.

I have no idea how to setup the configuration to do what I need.
It's all to technical and confusing for me

Perhaps somebody with a bigger brain could have a go and then let me know what to do.

Tim · Mon Mar 20, 2006 10:11 pm

Actually I might not be a stupid as I thought

I seem even without 'zebedee' that the Windows Server 2003 firewall is not allowing me to connect remotely.

I have tried adding the service port 3260 (TCP and UDP) but it still doesn't work.
If I disable the firewall I can connect OK.

What firewall settings would you recommend?

Val (staff) · Mon Mar 20, 2006 10:18 pm

Tim wrote:Actually I might not be a stupid as I thought :shock:

I seem even without 'zebedee' that the Windows Server 2003 firewall is not allowing me to connect remotely.

I have tried adding the service port 3260 (TCP and UDP) but it still doesn't work.
If I disable the firewall I can connect OK.

What firewall settings would you recommend?

Tim,

You need allow TCP/IP port 3260 for incoming connections from your initiator machine(s).
Also if you use StarWind GUI to remotely control the service, allow TCP/IP port 3261 for incoming connections too.