Server and RAID configuration

[UniMatrix]

Hi everybody,

I'm about to move an Openfiler implementation over to Starwind running on 2008 server x64 after just buying the Enterprise Mirroring & Replication Edition.

The server is an HP Proliant DL380 G5 with 2 dual core xeons 2,66Ghz and 2GB of memory. I have some memory laying around. Can I assume that more memory gives better performance? (caching etc.). There are 4 nic ports, 2 onboard and one dual port adapter.

The disk shelf consists of 12 750GB 7200rpm SATA drives. The array controller is a somewhat older Smart Array 64xx series with 256MB of battery backed up cache. A limitation of this controller is that it cannot create luns/volumes over 2TB. It can create arrays over 2TB however.

Current setup is one 6TB RAID6 array with two spare drives. (10 drives in RAID6, 2 spares). On this array 3 volumes are created of each 2TB which are then seen as disks in the OS.

I am wondering what to do now. In this setup Starwind iscsi disks will all share the same array but I have maximum space available and probably maximum iops because all disks are in one array.

When I create multiple arrays I will waste more disk space and the arrays will have less iops (only 3 or 4 disks per array) but each starwind iscsi disk will be on a separate array and not compete with others.

I know VMware best practise is only one datastore on a LUN. This would mean create a raid 5 or 6 array with netto 3x750GB capacity and create a 2TB LUN on it. 2TB is vmware maximum datastore size.

Usage will probably be one ESX datastore and some targets for WIndows servers.

Maybe you have some thoughts what would give the best performance?

[darrellchapman]

Stick with one large array. From my understanding, because the overhead involved with RAID parity calculations you won't won't see an increase in performance until your array consists of at least 8 drives. I/O ops will be spread across all disks regardless of how you setup LUNs with a single array. You will be wasting a little space because of your 2TB LUN limitation but not much.

[Constantin (staff)]

Good point here! To improve speed of your storage I recommend you to use RAM caching in StarWind. It will allow not dramatically, but very seriously impove speed of your RAID.

[UniMatrix]

Thanx for the input!

I did indeed keep the large RAID6 array and created 3 2TB partitons on it.

Ik presented one of these as a physical disk to vmware and get 97MB/s read/write, seems OK to me. (moving 80GB VM)

On another one I created a windows partition and an image file of 512GB, copy of a 3GB iso-image gives 140MB/s write and 70MB/s read. (from a local mirrored sas drive)

I used 2 dedicated nics on the server and configured multipath on the initiators. In task manager I see both NICS being used 40~50%.

Will test from and to FC SAN Also

What are best practises for VMware?

asynchronous on?
caching on?

Regards,
UniMatrix

[Max (staff)]

Yes, absolutely right, just ensure that you set the cache larger than default one, this is a common misconfiguration

[UniMatrix]

Am I correct that is is not possible to change those parameters on an existing target? Not really a problem because I'm just testing at the moment.

What are the differences between the caching modes?

And how much more cache memory should I reserve? Double? 256MB? Depending on server memory?

Regards,
UniMatrix

[Constantin (staff)]

Yes, you cannot change options for now.
WB is faster for writing, but less secure in some cases, while WT allows you achieve higher reading speed and more secure than WB.
We recommend to choose it depending on your requirments.

[Aitor_Ibarra]

You can't change the cache mode or any setting for a target once you've set it up. You have to disconnect all initiators, remove the target and recreate it. This won't kill your data - the .IMG is not deleted.

Cache mode: Write Through means that only reads are cached. That means all writes are guaranteed to get to the disk so you won't lose data in a power failure etc (although whether the disk has actually written it is another question and depends on the disks, RAID controller etc). Write Back caches writes as well as reads. Don't use this if you don't have a UPS if your data is mission critical. Write Back really helps if you have lots of random writes, which disks hate but RAM is very good at, and/or if you have occasional large sequential writes - providing they fit into the cache, they will complete much faster, and during the inactivity afterwords, they get written to the disk.

Cache amount: this is really dependent on your i/o patterns and what other cacheing may or may not be going on. In certain applications it can make a huge difference to have massive cache.

A key thing with caching is that your applications may already be doing it so you may not see much benefit. E.g. if you are running SQL Server and have enough RAM to hold the entire DB then cached reads at the Starwind level won't help you much - but cached writes could help enormously by speeding up transaction logs.

The Windows filesystem will cache as much as it can, so if your VMs have plenty of spare RAM, read cache in Starwind won't help much, especially if you are using 1Gbit Ethernet.

With VMs, a big read cache can help if you have multiple hosts (that talk to Starwind) that need access to the same data (e.g. in Hyper-V if you are using a differencing disk, put the parent VHD on a cached target).

Essentially, a read-only (Write Through) cache can only benefit you if data that is still in the cache is requested again, by something that hasn't already cached it. A read/write (Write Back) cache will do the same, but also enable writes to complete quicker, but does introduce some risk.

As for your physical disks, if you want maximum IOPS then go for multiple RAID 1s and then make sure that the data is spread across them as much as possible so you can benefit from parallel reads from all those spindles. RAID 0/5/6 will really only speed up sequential transfers. If your i/o is mostly random then you want to spread it across as many spindles as possible, because your weakest point is that the drive heads have to physically move to access different parts of the disk. So do stuff like seperate your boot disks from your data disks and make the targets that hold these live on different RAID-1s. Basically any time an application needs more than one bit of data at the same time, the ideal is that they live on different physical disks so they can actually be requested simultaneously. A good RAID card can read from each drive independently, it's only writes which have to be done in lock step.

The ability to have massive caches in Starwind is a killer feature (see if you can find a RAID card with more than 4GB cache) - but you need the right applications and network to benefit. With 10GbE and the right i/o patterns, a big cache can make your slow HDs seem more like SSDs.

[anton (staff)]

Huge cache can be used in a dual-head configuration only if data is critical and data loss is deadly. Unless you can allow yourself to have session corrupted (which is perfectly OK for say daily backup - it's not a big deal to re-run incremental backup once again). Having say stock exchange transactions lost is not very good. So configuration (both software and hardware) should be picked up for EXACT scenario - you're absolutely right on this!