iSER support?

[mserra]

Hello!

in regards of using Mellanox ConnectX-3 Pro cards along with Starwind iSCSI software, I was wondering whether we could use iSER and make use of RDMA. Do you know if that would be possible? Any tips/suggestions? I'm actually looking forward to Anton's experience with Mellanox on his blog.

thank you,

Marc Serra

[anton (staff)]

If you mean StarPort then you're not going to see any support for iSER anytime soon... If you mean StarWind Virtual SAN then the problem is we can have it many times implemented @ our side but MSFT is not going to support it with their initiator making our primary production scenario being out of the game. So we optimize our internal protocol (iSCSI-based) for synchronization and allow MSFT to push SMB3 where SMB Direct (equivalent of an iSER) is present. For our Compute and Storage Separated scenario (StarWind runs block back bone for MSFT Scale-Out File Server).

Hello!

in regards of using Mellanox ConnectX-3 Pro cards along with Starwind iSCSI software, I was wondering whether we could use iSER and make use of RDMA. Do you know if that would be possible? Any tips/suggestions? I'm actually looking forward to Anton's experience with Mellanox on his blog.

thank you,

Marc Serra

[bacterialbag]

Could you clarify this for me?

I dont understand a word what you're saying.

Im using Mellanox Connectx3-Pro's myself, and was wondering if I could use any of the offloading/RDMA/Infiniband capabilities with starwind? Or will my cards only be useable with IPoIB and all the extras go to waste?

Please help clarify what infiniband options I can and can not use.

Thanks,

- Bag

If you mean StarPort then you're not going to see any support for iSER anytime soon... If you mean StarWind Virtual SAN then the problem is we can have it many times implemented @ our side but MSFT is not going to support it with their initiator making our primary production scenario being out of the game. So we optimize our internal protocol (iSCSI-based) for synchronization and allow MSFT to push SMB3 where SMB Direct (equivalent of an iSER) is present. For our Compute and Storage Separated scenario (StarWind runs block back bone for MSFT Scale-Out File Server).

Hello!

in regards of using Mellanox ConnectX-3 Pro cards along with Starwind iSCSI software, I was wondering whether we could use iSER and make use of RDMA. Do you know if that would be possible? Any tips/suggestions? I'm actually looking forward to Anton's experience with Mellanox on his blog.

thank you,

Marc Serra

[bacterialbag]

Could you clarify this for me?

I dont understand a word what you're saying.

Im using Mellanox Connectx3-Pro's myself, and was wondering if I could use any of the offloading/RDMA/Infiniband capabilities with starwind? Or will my cards only be useable with IPoIB and all the extras go to waste?

Please help clarify what infiniband options I can and can not use.

Thanks,

- Bag

If you mean StarPort then you're not going to see any support for iSER anytime soon... If you mean StarWind Virtual SAN then the problem is we can have it many times implemented @ our side but MSFT is not going to support it with their initiator making our primary production scenario being out of the game. So we optimize our internal protocol (iSCSI-based) for synchronization and allow MSFT to push SMB3 where SMB Direct (equivalent of an iSER) is present. For our Compute and Storage Separated scenario (StarWind runs block back bone for MSFT Scale-Out File Server).

Hello!

in regards of using Mellanox ConnectX-3 Pro cards along with Starwind iSCSI software, I was wondering whether we could use iSER and make use of RDMA. Do you know if that would be possible? Any tips/suggestions? I'm actually looking forward to Anton's experience with Mellanox on his blog.

thank you,

Marc Serra

[anton (staff)]

It completely depends on the way you're going to use them.

1) You're using "Compute and Storage Separated" scenario you'll use RDMA/IB together with SMB3 protocol on top of it with Hyper-V nodes "talking" to Scale-Out File Server setup. StarWind will run @ SoFS nodes and do [own nameless protocol]-over-iSCSI in the background to keep CSVs on SoFS nodes in synch between each other (similar to upcoming Microsoft Storage Spaces Shared Nothing but w/o "slabs", multiple re-routes and not with 4 nodes minimum, but that's other story...). StarWind does not expose iSCSI to Hyper-V with this setup.

2) You're using "Hyper-Converged" scenario then you'll have to switch IB -> IP-over-IB (we get ±33-35 Gbps on a 56 Gb IB now in the lab, ping Bohdan for more details if you want). You're going to have offload/RDMA and so on but inside: there's no **native** support for IB or RDMA with StarWind iSCSI at this moment. From the other point it's not going to help much for a reason: talking on the same node Hyper-V -to- Hyper-V we do DMA with an own driver stack and not RDMA as TCP and iSCSI stacks are mostly bypassed and with synchronization there's own protocol over iSCSI (see above) and it uses huge blocks to move data other Etherent so RDMA latency does not really matter, raw bandwidth is fine.

Could you clarify this for me?

I dont understand a word what you're saying.

Im using Mellanox Connectx3-Pro's myself, and was wondering if I could use any of the offloading/RDMA/Infiniband capabilities with starwind? Or will my cards only be useable with IPoIB and all the extras go to waste?

Please help clarify what infiniband options I can and can not use.

Thanks,

- Bag

[mserra]

I'm actually planning on going with Mellanox ConnectX-3 Pro + Starwind to serve as SAN for our VMWare Infraestructure (which will also use ConnectX-3 Pro). I'm interested in the best approach for maximum speed & reliability and that's why the RDMA question came to mind. I still don't have all the hardware yet but as soon as I have it I intend to try with different setups and see what's the best option. I might ping Bohdan as suggested too

Thanks for your replies,

Marc Serra

[anton (staff)]

So it's "Compute and Storage Separated" and you plan to use iSCSI as a connectivity to VMware vSphere. OK We'll re-check current status of iSER with vSphere as a client.

I'm actually planning on going with Mellanox ConnectX-3 Pro + Starwind to serve as SAN for our VMWare Infraestructure (which will also use ConnectX-3 Pro). I'm interested in the best approach for maximum speed & reliability and that's why the RDMA question came to mind. I still don't have all the hardware yet but as soon as I have it I intend to try with different setups and see what's the best option. I might ping Bohdan as suggested too

Thanks for your replies,

Marc Serra

[mserra]

So it's "Compute and Storage Separated" and you plan to use iSCSI as a connectivity to VMware vSphere. OK We'll re-check current status of iSER with vSphere as a client.

Yes, that is correct. Sorry if I didn't leave it clear, I think I may have missplaced the thread and should have dropped it under StarPort instead of StarWind Virtual SAN

[anton (staff)]

No problem at all I don't have an answer right now but I'll have it quite soon

So it's "Compute and Storage Separated" and you plan to use iSCSI as a connectivity to VMware vSphere. OK We'll re-check current status of iSER with vSphere as a client.

Yes, that is correct. Sorry if I didn't leave it clear, I think I may have missplaced the thread and should have dropped it under StarPort instead of StarWind Virtual SAN

[mserra]

No problem at all I don't have an answer right now but I'll have it quite soon

Great! I'm looking forward to it

[anton (staff)]

Please check your Inbox. I've took a few implementation questions off-line. Thanks!

No problem at all I don't have an answer right now but I'll have it quite soon

Great! I'm looking forward to it