Performance issues with a New 2 node Install
Posted: Mon Sep 10, 2018 6:38 am
Hi,
I decided to try out Starwind VSAN Free, as a dedicated Storage Server (no VM Workload), purely for storage via iSCSI. Here are the details, it's a bit long, but I wanted to give as much details as I could.
Build details 2 x identical Servers with the following specs:
Quad Core XEON 1225 v3
32 GB ECC RAM
2 x Quad Port 1GB NICs (Total 8 Ports)
1 x 250GB HDD (for OS)
2 x 500GB SDD (for Data)
2 x 6TB SSD (for Data)
I installed Hyper-V Server 2016 on each node, updated fully, and joined an existing domain.
On the 6TB HDD, i've created devices (2TB, 2TB, 1TB), and replicated to the 2nd node (I used the 1GB, replicate, then expand trick to save time). This will be used for VMs Data Disks.
On the 500GB SSD, i've created a device (400GB), and replicated to the 2nd node. This will be used for VMs OS Drive, for faster access.
L1 Cache, all using write-through mode, with the reccomended L1 cache setting, so 2GB, 2GB,1GB for HDDs and also 1GB for SSD.
I've configured 2 ports on each server for Sync, and 4 ports on each server for iSCSI.
I then connected my ESXi Hosts and Hyper-V Hosts to the Storage serve via iSCSI, and started migrating my VMs (from the FreeNAS box).
So far, everything looked great, and perfomance was as expected.
PROBLEM1: I then added 2 more ports for Sync, which made the CPUs shoot up to 100%, so I removed the 2 ports again, and left it as 2 sync channels only. This is a slight concern, but not my main problem.
PROBLEM2: I removed the replica from 1 of the SSD drives, as I had to swap a drive in the 2nd node. When I re-created replica, a fully sync started, which has now been running for overs 12 hours. There is about 300GB of data on the 400GB device, is this replication time normal?
PROBLEM3: Performance seems to have de-graded since what it was when I initially started. CPU usage on each host is on avergage 60%.
QUESTION1: Am I better off using the SSDs as L2 cache instead of using them for VM disks. I've set it up this way, as I had great performance on FreeNAS for OS Boot drives on SSD, compared to HDD.
QUESTION2: To attempt to resolve PROBLEM3, I have set the sync priority to 25%, but have not yet rebooted the hosts, due to PROBLEM2 (still syncing). Should this setting help with performance of VMs?
Any help would be appreciated.
I decided to try out Starwind VSAN Free, as a dedicated Storage Server (no VM Workload), purely for storage via iSCSI. Here are the details, it's a bit long, but I wanted to give as much details as I could.
Build details 2 x identical Servers with the following specs:
Quad Core XEON 1225 v3
32 GB ECC RAM
2 x Quad Port 1GB NICs (Total 8 Ports)
1 x 250GB HDD (for OS)
2 x 500GB SDD (for Data)
2 x 6TB SSD (for Data)
I installed Hyper-V Server 2016 on each node, updated fully, and joined an existing domain.
On the 6TB HDD, i've created devices (2TB, 2TB, 1TB), and replicated to the 2nd node (I used the 1GB, replicate, then expand trick to save time). This will be used for VMs Data Disks.
On the 500GB SSD, i've created a device (400GB), and replicated to the 2nd node. This will be used for VMs OS Drive, for faster access.
L1 Cache, all using write-through mode, with the reccomended L1 cache setting, so 2GB, 2GB,1GB for HDDs and also 1GB for SSD.
I've configured 2 ports on each server for Sync, and 4 ports on each server for iSCSI.
I then connected my ESXi Hosts and Hyper-V Hosts to the Storage serve via iSCSI, and started migrating my VMs (from the FreeNAS box).
So far, everything looked great, and perfomance was as expected.
PROBLEM1: I then added 2 more ports for Sync, which made the CPUs shoot up to 100%, so I removed the 2 ports again, and left it as 2 sync channels only. This is a slight concern, but not my main problem.
PROBLEM2: I removed the replica from 1 of the SSD drives, as I had to swap a drive in the 2nd node. When I re-created replica, a fully sync started, which has now been running for overs 12 hours. There is about 300GB of data on the 400GB device, is this replication time normal?
PROBLEM3: Performance seems to have de-graded since what it was when I initially started. CPU usage on each host is on avergage 60%.
QUESTION1: Am I better off using the SSDs as L2 cache instead of using them for VM disks. I've set it up this way, as I had great performance on FreeNAS for OS Boot drives on SSD, compared to HDD.
QUESTION2: To attempt to resolve PROBLEM3, I have set the sync priority to 25%, but have not yet rebooted the hosts, due to PROBLEM2 (still syncing). Should this setting help with performance of VMs?
Any help would be appreciated.
