Hi Anton,
Thanks for your quick reply.
"
Of course local kernel-mode RAM disk is going to be faster compared to user-land distributed one done over iSCSI! "
ok, understood.
"
You need to understand when you're adding extra replication partners your read performance increases (we take care of data locality but do off-load some reads to partners to speed up the process) but writes has to be in sync so write performance is going to be limited with performance of your sync channel. That's why it's vital to make sure your 40 GbE links are fully working and all instances of StarWind are assigned to own NUMA node."
See, our network speed is huge, way faster than our storage and when we do a sync operation we do indeed observe huge throughput through the sync channels. However, when we did the performance testing, we saw no throughput through any of our Ethernet connections. We did not understand that, but maybe this hints at the reason for the poor disk performance test (which was poorer for both reads and writes). So we decided to check the other node as well (the node that saw decent RAM disk performance), and when we checked the disk performance inside a VM running on that node, we actually do get read performance close to that of the underlying storage, and lower write performance as we did observe throughput over the iscsi nics. So, node 2 (the fast node) functions exactly as per what you say. Accordingly we likely configured something wrong on node 1 (the one that shows poor performance), but we just don't know what we did wrong. (We do think that all instances of SW are assigned to its own local NUMA node (we checked the log, and we think the binding is correct), but we can be wrong here and the difference between RAM disk performance does indeed hint at this).
"
Another issue we have is ±100K IOPS limit per virtual LUN. If your underlying storage can do more locally - you' have to layer more virtual LUNs on top of it to get aggregation of your peak performance. "
This is hopeful, we did not know this, and indeed don't know how to do this ourselves.
"
Again, give our engineers a chance "
Ha, I would love to give your engineers a chance
![Smile :)](./images/smilies/icon_smile.gif)
, and it looks like that we will have to, but we are a small start-up (on a tight budget) and SW kindly offers us the possibility to start with HA from onset for free. We will grow into more nodes and - as it now becomes apparent - we will need your help configuring it correctly as well, but we cant afford so now already. So we have to start like this (SW is very stable, just the performance is slower than anticipated), and then grow into a paid license. For now, it is important for us to know that the problems we face are solvable. So that in a couple of months from now we can indeed have you configure it all correctly.
So thanks again for your quick reply, which does indeed hint at our performance problems being solvable and is keeping us with SW.