iSER_DM.dll causing VSAN to crash
Posted: Mon Feb 12, 2024 3:16 pm
Hi all!
We are testing out setting up a 2 node HA but we are finding that StarWindService.exe crashes sporadically on both nodes (not at the same time) with the following logged in the event log:
Faulting application name: StarWindService.exe, version: 8.0.0.15260, time stamp: 0x64edf8e5
Faulting module name: iSER_DM.dll, version: 8.0.0.15260, time stamp: 0x64edf898
Exception code: 0xc0000005
Fault offset: 0x00000000000016ff
Faulting process id: 0x5b6c
Faulting application start time: 0x01da5dc33997ae97
Faulting application path: C:\Program Files\StarWind Software\StarWind\StarWindService.exe
Faulting module path: C:\Program Files\StarWind Software\StarWind\iSER_DM.dll
Report Id: 171e6fbc-9205-4cf3-b7c3-7584db7eeb64
Faulting package full name:
Faulting package-relative application ID:
We initially had iSER enabled on the sync interfaces (in StarWind.cfg, setting the IPs of the sync interfaces in <iSerListen .../>) and then tried disabling it by commenting out <iSerListen /> but the same issue occurs.
From a previous post on the forum, I think we can rename iSER_DM.dll => iSER_DM.dll.bak to avoid this, however we would like (in the final setup) to use ISER for the sync channels so this is not ideal (happy to try that though if that helps troubleshooting).
Our sync interfaces are using Connect-X3 (not pro) which are working well for SMB Direct, but perhaps are no good for ISER?
ISER does seem to connect when enabled, from what I can make out of the logs, however it's not that quick - I have put that down to the fact that the ConnectX-3s are not (at the moment) directly connected, and are going via an IB switch.
However since disabling ISER does not stop the StarWindService.exe crash, I'm not sure if that is the problem?
We previously setup a 2 node HA using VMs (and no RDMA hardware) which did not crash so... maybe the hardware is to blame?
Info from one of the ConnectX-3s:
Driver Version : 5.50.14740.1
Firmware Version : 2.42.5000
Port Number : 1
Bus Type : PCI-E 8.0 Gbps x8
Link Speed : 40.0 Gbps/Full Duplex
Part Number : MCX353A-FCBT
Device Id : 4099
Revision Id : 0
Current MAC Address : 98-03-9B-DC-C9-91
Permanent MAC Address : 98-03-9B-DC-C9-91
Network Status : Connected
Adapter Friendly Name : Ethernet IB#1
IPv4 Address : 10.254.2.1
Adapter User Name : 0xffff-IPoIB
Adapter PKey : 0xffff (Available)
Many thanks for any assistance!
James
We are testing out setting up a 2 node HA but we are finding that StarWindService.exe crashes sporadically on both nodes (not at the same time) with the following logged in the event log:
Faulting application name: StarWindService.exe, version: 8.0.0.15260, time stamp: 0x64edf8e5
Faulting module name: iSER_DM.dll, version: 8.0.0.15260, time stamp: 0x64edf898
Exception code: 0xc0000005
Fault offset: 0x00000000000016ff
Faulting process id: 0x5b6c
Faulting application start time: 0x01da5dc33997ae97
Faulting application path: C:\Program Files\StarWind Software\StarWind\StarWindService.exe
Faulting module path: C:\Program Files\StarWind Software\StarWind\iSER_DM.dll
Report Id: 171e6fbc-9205-4cf3-b7c3-7584db7eeb64
Faulting package full name:
Faulting package-relative application ID:
We initially had iSER enabled on the sync interfaces (in StarWind.cfg, setting the IPs of the sync interfaces in <iSerListen .../>) and then tried disabling it by commenting out <iSerListen /> but the same issue occurs.
From a previous post on the forum, I think we can rename iSER_DM.dll => iSER_DM.dll.bak to avoid this, however we would like (in the final setup) to use ISER for the sync channels so this is not ideal (happy to try that though if that helps troubleshooting).
Our sync interfaces are using Connect-X3 (not pro) which are working well for SMB Direct, but perhaps are no good for ISER?
ISER does seem to connect when enabled, from what I can make out of the logs, however it's not that quick - I have put that down to the fact that the ConnectX-3s are not (at the moment) directly connected, and are going via an IB switch.
However since disabling ISER does not stop the StarWindService.exe crash, I'm not sure if that is the problem?
We previously setup a 2 node HA using VMs (and no RDMA hardware) which did not crash so... maybe the hardware is to blame?
Info from one of the ConnectX-3s:
Driver Version : 5.50.14740.1
Firmware Version : 2.42.5000
Port Number : 1
Bus Type : PCI-E 8.0 Gbps x8
Link Speed : 40.0 Gbps/Full Duplex
Part Number : MCX353A-FCBT
Device Id : 4099
Revision Id : 0
Current MAC Address : 98-03-9B-DC-C9-91
Permanent MAC Address : 98-03-9B-DC-C9-91
Network Status : Connected
Adapter Friendly Name : Ethernet IB#1
IPv4 Address : 10.254.2.1
Adapter User Name : 0xffff-IPoIB
Adapter PKey : 0xffff (Available)
Many thanks for any assistance!
James