vSAN v8 build 13481. number of iscsi sessions

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

Post Reply
gklpnd
Posts: 14
Joined: Wed Mar 04, 2020 2:38 pm

Wed Mar 04, 2020 3:13 pm

Hello,

I have just updated from 12767 to 13481 build.
My question is about why increased number of iscsi session from 9 to 16?

HA device has 2 sync + 3 hb + 4 customers sessions.

Thanks!
Attachments
iscsi_sessions
iscsi_sessions
starwind-13481.png (32.28 KiB) Viewed 6877 times
yaroslav (staff)
Staff
Posts: 2277
Joined: Mon Nov 18, 2019 11:11 am

Wed Mar 04, 2020 7:08 pm

iSCSI connections coming from ESXi host are those which start with iqn.1991-05.com. the others, which contain 2008, are StarWind-related "supplementary" sessions used in older builds to boost performance. Those connections are created for iSCSI and Sync sessions to boost performance. In new builds, they do not change anything but still, show up.
You can disregard them. Please, consider only iqn.1991-05.com sessions.
You still can get rid of those 2008 connections by modifying the .swdsk and _HA.swdsk files (i do not recommend doing that though).
gklpnd
Posts: 14
Joined: Wed Mar 04, 2020 2:38 pm

Thu Mar 05, 2020 8:25 am

No ESXi, only windows.

As i correctly you understand build 12767 also have "supplementary" sessions, but not shown in gui console?
But..
From log file with build 12767:
3/3 23:59:18.402245 1020 Common: *** MountTarget: (session(0000010514EFB580), local(0.0.0.0), peer(10.98.4.12), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/3 23:59:18.402314 1028 Common: *** MountTarget: (session(0000010514EFBB10), local(0.0.0.0), peer(10.98.5.4), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/3 23:59:18.402920 1024 Common: *** MountTarget: (session(0000010514EFBD60), local(0.0.0.0), peer(10.98.4.28), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/3 23:59:18.402980 102c Common: *** MountTarget: (session(0000010514F4B5A0), local(0.0.0.0), peer(10.98.5.20), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/3 23:59:18.416187 1030 Common: *** MountTarget: (session(0000010514F5BC10), local(0.0.0.0), peer(192.168.255.170), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/3 23:59:18.430502 fe4 func: <<< CHADevice::CreateDevice

No more mount targets for target "iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2".

From log file with 13481 build:
3/5 10:08:07.159047 1c18 Common: *** MountTarget: (session(000002A196628200), local(0.0.0.0), peer(10.98.4.12), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/5 10:08:07.168528 14fc Common: *** MountTarget: (session(000002A196620C30), local(0.0.0.0), peer(10.98.4.28), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/5 10:08:07.172292 1c18 Common: *** MountTarget: (session(000002A1965FDA50), local(0.0.0.0), peer(10.98.4.12), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/5 10:08:07.176927 1860 Common: *** MountTarget: (session(000002A196631070), local(0.0.0.0), peer(10.98.5.4), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/5 10:08:07.178626 14fc Common: *** MountTarget: (session(000002A196602EC0), local(0.0.0.0), peer(10.98.4.28), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/5 10:08:07.183940 1c18 Common: *** MountTarget: (session(000002A196627FB0), local(0.0.0.0), peer(10.98.4.12), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/5 10:08:07.189826 1860 Common: *** MountTarget: (session(000002A19666D400), local(0.0.0.0), peer(10.98.5.4), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/5 10:08:07.190418 14fc Common: *** MountTarget: (session(000002A19669EEE0), local(0.0.0.0), peer(10.98.4.28), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/5 10:08:07.190970 1b30 conf: ConfAclCommand::processNew: Acl record 'allow for partner(s) of iqn.2008-08.com.starwindsoftware:localhost-witfx2-fx1' loaded successfully.
3/5 10:08:07.191168 1c28 func: <<< HASyncNode::createDevice
3/5 10:08:07.192427 1d5c Common: *** MountTarget: (session(000002A1966D09D0), local(0.0.0.0), peer(10.98.5.20), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/5 10:08:07.193399 1c28 SCSI: iScsiDevice::updateDeviceIds: EUI64 retrieved: C3EF40B87747B60E (hex)
3/5 10:08:07.203326 238 Common: *** MountTarget: (session(000002A196619E20), local(0.0.0.0), peer(192.168.255.170), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/5 10:08:07.208509 1d5c Common: *** MountTarget: (session(000002A196653430), local(0.0.0.0), peer(10.98.5.20), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).
3/5 10:08:07.218207 238 Common: *** MountTarget: (session(000002A1967303F0), local(0.0.0.0), peer(192.168.255.170), port(3260), target(iqn.2008-08.com.starwindsoftware:192.168.254.27-witfx2-fx2).

As result it 12 sessions, but configured 5.


Ok, supplementary sessions for boost start showing in build 13481, but why they have only 12 connections (16 - 4(customers) = 12)?.

Via netstat -an | sls 3260| sls 10.98.4.11 = i see 9 connections from each side (for sync channel between 10.98.4.11 and 10.98.4.12):
PS C:\Users\gklpnd> netstat -an | sls 3260 | sls 10.98.4.1

TCP 10.98.4.11:3260 0.0.0.0:0 LISTENING
TCP 10.98.4.11:3260 10.98.4.12:50180 ESTABLISHED
TCP 10.98.4.11:3260 10.98.4.12:50185 ESTABLISHED
TCP 10.98.4.11:3260 10.98.4.12:50186 ESTABLISHED
TCP 10.98.4.11:3260 10.98.4.12:50191 ESTABLISHED
TCP 10.98.4.11:3260 10.98.4.12:50197 ESTABLISHED
TCP 10.98.4.11:3260 10.98.4.12:50201 ESTABLISHED
TCP 10.98.4.11:3260 10.98.4.12:50202 ESTABLISHED
TCP 10.98.4.11:3260 10.98.4.12:50205 ESTABLISHED
TCP 10.98.4.11:3260 10.98.4.12:50206 ESTABLISHED
TCP 10.98.4.11:63033 10.98.4.12:3260 ESTABLISHED
TCP 10.98.4.11:63035 10.98.4.12:3260 ESTABLISHED
TCP 10.98.4.11:63038 10.98.4.12:3260 ESTABLISHED
TCP 10.98.4.11:63045 10.98.4.12:3260 ESTABLISHED
TCP 10.98.4.11:63046 10.98.4.12:3260 ESTABLISHED
TCP 10.98.4.11:63049 10.98.4.12:3260 ESTABLISHED
TCP 10.98.4.11:63057 10.98.4.12:3260 ESTABLISHED
TCP 10.98.4.11:63058 10.98.4.12:3260 ESTABLISHED
TCP 10.98.4.11:63062 10.98.4.12:3260 ESTABLISHED


You still can get rid of those 2008 connections by modifying the .swdsk and _HA.swdsk files (i do not recommend doing that though).
Do you mean data/control connections (sync and hb)?
yaroslav (staff)
Staff
Posts: 2277
Joined: Mon Nov 18, 2019 11:11 am

Sat Mar 07, 2020 4:36 am

Do you mean data/control connections (sync and hb)?
Yes. Headers of HA devices and corresponding IMG files are to be modified in order to get rid of multiple connections. You need to change the number of sessions for Heartbeat (AKA iSCSI) and Data (AKA Sync) links.
Be sure that additional iSCSI connections do not harm.

Well, I do not know where the number of iSCSi connections can be derived from as there might be different numbers of the sessions for iSCSI and Sync channels (e.g., 3 for iSCSI and 4 for Sync).

One more time, there is nothing to worry about in this case.
gklpnd
Posts: 14
Joined: Wed Mar 04, 2020 2:38 pm

Tue Mar 10, 2020 7:18 pm

Hm, I was waiting for another answer.

Is there the way to see detailed info about each iscsi connection (sync and hb) via powershell?
Michael (staff)
Staff
Posts: 317
Joined: Thu Jul 21, 2016 10:16 am

Sun Mar 15, 2020 1:05 pm

Hello gklpnd,
There is no need to use PowerShell to get information about the number of connections.
Open HA.swdsk file on both servers and check transport section for the connections number:
<transport type="iSCSI">
<links>
<link id="1" type="data" priority="1" connections="1">
<peer ip="172.16.20.10" port="3260"/>
</link>
<link id="2" type="control" priority="1" connections="1">
<peer ip="172.16.10.10" port="3260"/>
</link>
</links>
</transport>
"data" corresponds to the sycnhrpnization connection and "control" corresponds to the heartbeat connection.
gklpnd
Posts: 14
Joined: Wed Mar 04, 2020 2:38 pm

Mon Mar 16, 2020 12:46 pm

Thanks for your reply.

From file hv2_fx1_HA.swdsk:

Code: Select all

        <storage id="3" name="iqn.2008-08.com.starwindsoftware:192.168.254.27-hv2-fx2" type="remote" lun="0x0">
          <transport type="iSCSI">
            <links>
              <link id="1" type="data" priority="1" connections="1">
                <peer ip="10.98.4.12" port="3260"/>
              </link>
              <link id="2" type="data" priority="1" connections="1">
                <peer ip="10.98.4.28" port="3260"/>
              </link>
              <link id="3" type="control" priority="1" connections="1">
                <peer ip="10.98.5.4" port="3260"/>
              </link>
              <link id="4" type="control" priority="1" connections="1">
                <peer ip="10.98.5.20" port="3260"/>
              </link>
              <link id="5" type="control" priority="1" connections="1">
                <peer ip="192.168.255.170" port="3260"/>
              </link>
            </links>
          </transport>
        </storage>
The result is 16 sessions (picture at 1st post). Each sync channel multiplied by 3, each hb by 2...and 4 sessions from clients.

I changed each connection from 1 to 2, the result is 20 sessions (16 from starwind and 4 from clients)

Something strange :(

Thanks!
Attachments
ha2-conn2.png
ha2-conn2.png (6.51 KiB) Viewed 6577 times
Michael (staff)
Staff
Posts: 317
Joined: Thu Jul 21, 2016 10:16 am

Mon Mar 16, 2020 11:02 pm

Could you please check the same file on the pather node?
Thank you!
gklpnd
Posts: 14
Joined: Wed Mar 04, 2020 2:38 pm

Tue Mar 17, 2020 11:37 am

From another node file hv2_fx1_HA.swdsk:

Code: Select all

        <storage id="3" name="iqn.2008-08.com.starwindsoftware:localhost-hv2-fx1" type="remote" lun="0x0">
          <transport type="iSCSI">
            <links>
              <link id="1" type="data" priority="1" connections="1">
                <peer ip="10.98.4.11" port="3260"/>
              </link>
              <link id="2" type="data" priority="1" connections="1">
                <peer ip="10.98.4.27" port="3260"/>
              </link>
              <link id="3" type="control" priority="1" connections="1">
                <peer ip="10.98.5.3" port="3260"/>
              </link>
              <link id="4" type="control" priority="1" connections="1">
                <peer ip="10.98.5.19" port="3260"/>
              </link>
              <link id="5" type="control" priority="1" connections="1">
                <peer ip="192.168.255.169" port="3260"/>
              </link>
            </links>
          </transport>
        </storage>
Thanks!
yaroslav (staff)
Staff
Posts: 2277
Joined: Mon Nov 18, 2019 11:11 am

Tue Mar 17, 2020 11:50 am

Greetings,

Each target connection had 10 connections (5 connections from each side) + 4 client connections. By adding modifying HA.swdsk on one side, you have added 5 additional connections. Now there are 5 connections from each side+5 "extra" connections.
What I am missing to solve this riddle is a screenshot of that iSCSI target from the other server. I need to understand how many iSCSI sessions you have on the other side.
gklpnd
Posts: 14
Joined: Wed Mar 04, 2020 2:38 pm

Tue Mar 17, 2020 12:42 pm

Hello,

I created another HA device without client sessions.

First node file tst1gb_fx1_HA.swdsk:

Code: Select all

       <storage id="3" name="iqn.2008-08.com.starwindsoftware:192.168.254.27-tst1gb-fx2" type="remote" lun="0x0">
          <transport type="iSCSI">
            <links>
              <link id="1" type="data" priority="1" connections="1">
                <peer ip="10.98.4.12" port="3260"/>
              </link>
              <link id="2" type="data" priority="1" connections="1">
                <peer ip="10.98.4.28" port="3260"/>
              </link>
              <link id="3" type="control" priority="1" connections="1">
                <peer ip="10.98.5.4" port="3260"/>
              </link>
              <link id="4" type="control" priority="1" connections="1">
                <peer ip="10.98.5.20" port="3260"/>
              </link>
              <link id="5" type="control" priority="1" connections="1">
                <peer ip="192.168.255.170" port="3260"/>
              </link>
            </links>
          </transport>
        </storage>
Screenshot from 1st node that has 12 starwind sessions from partner node.
from 1st node
from 1st node
fx1-tst1gb.png (32.71 KiB) Viewed 6543 times
Second node file tst1gb_fx2_HA.swdsk:

Code: Select all

       <storage id="3" name="iqn.2008-08.com.starwindsoftware:localhost-tst1gb-fx1" type="remote" lun="0x0">
          <transport type="iSCSI">
            <links>
              <link id="1" type="data" priority="1" connections="1">
                <peer ip="10.98.4.11" port="3260"/>
              </link>
              <link id="2" type="data" priority="1" connections="1">
                <peer ip="10.98.4.27" port="3260"/>
              </link>
              <link id="3" type="control" priority="1" connections="1">
                <peer ip="10.98.5.3" port="3260"/>
              </link>
              <link id="4" type="control" priority="1" connections="1">
                <peer ip="10.98.5.19" port="3260"/>
              </link>
              <link id="5" type="control" priority="1" connections="1">
                <peer ip="192.168.255.169" port="3260"/>
              </link>
            </links>
          </transport>
        </storage>
Screenshot from 2nd node that has 12 starwind sessions from partner node.
from 2nd node
from 2nd node
fx2-tst1gb.png (30.18 KiB) Viewed 6543 times
yaroslav (staff)
Staff
Posts: 2277
Joined: Mon Nov 18, 2019 11:11 am

Wed Mar 18, 2020 8:44 am

Greetings,

I escalated this issue to our R&D team. I'll keep you posted.
yaroslav (staff)
Staff
Posts: 2277
Joined: Mon Nov 18, 2019 11:11 am

Wed Mar 18, 2020 5:17 pm

Hi,

I got an update from R&D.

Let me explain the math behind iSCSI sessions.
There are 3 iSCSI sessions for each "data" connection: 1xdata transfer, 1xinternal commands 1xheartbeat. You have 2 data connections so that already makes 6 connections.
There are 2 iSCSI sessions for each "control" connection: 1xinternal commands and 1xheartbeat. You have 3 connections and that makes 6 more sessions.

Given that, there should be 12 iSCSI sessions for each target in your setup. Client connections add up more iSCSI sessions (that is how you had 16 iSCSI sessions before).
gklpnd
Posts: 14
Joined: Wed Mar 04, 2020 2:38 pm

Tue Mar 24, 2020 1:37 pm

Thanks for explanation.
yaroslav (staff)
Staff
Posts: 2277
Joined: Mon Nov 18, 2019 11:11 am

Tue Mar 24, 2020 3:38 pm

It was a pleasure for me to help you.
Post Reply