Big SATA Box - Help Please

[pneumatic]

I am investigating the possibility of building a large SATA iSCSI box for the following things:

1) Mundane File Sharing (1TB+ but much of it does not have a differential from one weekly backup to the next).
2) Exchange 2010

#1 seems easy since there is very little activity. #2 seems curious as Microsoft actually suggests SATA for Exchange 2010.

I'd like to use 2TB drives if at all possible (since those have the most bang for buck, assuming adequate performance). My goal is to give 250 users a 25GB mailbox. Right now, we're averaging about 30 emails per user per day at less than 100kb per. 250 x 25GB = 6.25TB but I'd like to provide some separation for my eggs. Perhaps a couple of RAID5s (4TB each, allowing up to 2 drive failures without impact).

Or would I be better off with a single RAID6?

As far as SATA drives, should I be hesitant of "consumer / desktop" drives?

[anton (staff)]

I don't see how "file sharing" is going to work with the SAN concept... Unless you'll have say hypervisor cluster above (as a client of SAN) and installed software inside this cluster is going to provide SMB-style file sharing services to it's clients.

RAID10 or RAID6 is going to work, just avoid everything below (and including five). See this link: http://www.baarf.com/

RAID6 is performance pig for write and to have write-back cache enabled you need HA configuration.

[pneumatic]

Thanks for the info - can you advise on processor and RAM sizing? I'd be going with a Xeon rig. From what I've read, a low-end, single processor is more than enough for hosting a SAN, correct? I was thinking a Xeon 5502 but am certainly open to advice.

As for RAM, I imagine that the more, the merrier, yes?

Also - what 2TB drives are recommended, if any. Anything to avoid?

[anton (staff)]

1) Not really... We're Windows shop, so more CPU power you're going to provide - less I/O delay response your customers are going to get. Linux-based people are on the lighter side here )

2) Absolutely! More RAM - more cache. More cache - better performance and less I/O delay.

3) We're not in any kind of alliance so I can tell you the secret - does not matter ) I'm personally a fan of 10K SATA Raptors (but they are far less capacity then 2TB and provide bad value for money).

Thanks for the info - can you advise on processor and RAM sizing? I'd be going with a Xeon rig. From what I've read, a low-end, single processor is more than enough for hosting a SAN, correct? I was thinking a Xeon 5502 but am certainly open to advice.

As for RAM, I imagine that the more, the merrier, yes?

Also - what 2TB drives are recommended, if any. Anything to avoid?

[pneumatic]

Okay - good info - elaborating on the processor spec... In terms of architecture, is it better to have a low-end dual processor box (e.g. - dual Xeon 5504) or a high-end single processor (e.g. - single 5540)?

[anton (staff)]

I'd go for dual...

Okay - good info - elaborating on the processor spec... In terms of architecture, is it better to have a low-end dual processor box (e.g. - dual Xeon 5504) or a high-end single processor (e.g. - single 5540)?

[Aitor_Ibarra]

OK - I'm going to chuck my opinions in here. I could be wildy off the mark so feel free to ignore them!

1) If you are thinking about using motherboard raid (e.g. Intel Matrix RAID) then don't. Especially if you need to parity RAID (5 and 6). Ideally, get a proper RAID card with onboard battery backed cache, failing that get something with adecent SAS RAID controller built in (e.g. Supermicro with onboard LSI RAID).
2) Don't use consumer disks unless they've been verified with the RAID controller, and even then be wary. Make sure you get the RAID edition drives, as desktop drives tend to have firmware which doesn't work well in RAID. For instance, many drives will halt data transfer if they encounter an error and will attempt to correct it. If they take too long to do this and they are on a RAID controller, the controller may drop the drive from the array.
3) Don't get velociraptors. I agree with Anton, the original Raptors work well in RAID (I once ran Exchange 2003 for 100+ users off 4x 74GB Raptors), but be very wary of the newer Velociraptors. I got hit by the 45 day bug in the firmware... However the 2TB RE4 has been fine for me.
4) Are you building this server yourelf? In that case, get cold spare drives rather than relying on the drive manufacturers warranty. You cannot rely on the RMA process to be speedy. If you are going HP/Dell/IBM then get an onsite warranty.
5) Exchange loves IOPS more than transfer rates. Things may have changed with 2010 (I'm not familiar with it) but for 2000-2003-2007 you were best off with multiple exchange dbs on mutliple RAID volumes. That way if you have a drive failure, the peformance hit only affects some of your users, and total IOPs is increased.
6) Consider SAS drives. They aren't that much of a premium (e.g. Seagate Savvio 10K.3 - £230 vs WD Velociraptor @ £170). SAS has better error correction than SATA, and sometimes better performance, as it's full duplex.
7) Consider your rebuild time. A 300GB RAID-1, on 10K disks, will rebuild in two hours. A 2TB RAID-1 could be over five hours at 100MB/sec. I would go for more, smaller disks for Exchange, and fewer, bigger disks for your files.


If your CPU isn't involved in the RAID (i.e. you have a hardware raid controller) then a single 55xx series CPU should be fine unless you are going to be saturating multiple 10GBit NICS. Get a dual socket motherboard, but only but a second CPU if you really need to. Starwind itself is very efficient - I can get to 500MB/sec over 10GbE before I saturate CPU - but that's with Starwind 4.2 running in a hyper-v vm with only one cpu core assigned to it.


cheers,

Aitor

[pneumatic]

Are you building this server yourelf?

Yep - with the expectation that the "premier" service levels from SMB vendors are now so low that I cannot expect to rely on them at all anymore. HP won't even offer more than 1-year on their lower-end stuff anymore. I will have my own hotspares for everything.

Great insight on your comments - thanks.

Microsoft built Exchange 2010 for a 70 percent reduction in IO over 2007. They also built it for what effectively results in a "Exchange RAID" - you can assign mailboxes many data stores and not have to worry about independent failures.

I won't even run RAID on the Exchange stores. I'll do RAID1 on the others. I probably won't touch more than 100MBps peak.

[Constantin (staff)]

Exchange 2010 has a lot of improvements specially in storage area, but if you build clusters. Yes, MS claims that has reduced IO for 70% compared to Ex2007, but IO operations in Ex2007 were reduced on 30-50% compared to Ex2003, so I don`t think that it`s something more than marketing buzz.