<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=utf-8">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7652.24">
<TITLE>Re: [CentOS] I need storage server advice</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->
<BR>
<P><FONT SIZE=2>Take these benchmarks with a grain of salt.<BR>
<BR>
We don't know how these hardware controllers were setup and by the numbers posted, not very well, or they are not very good.<BR>
<BR>
A SATA and a SAS drive will have roughly the same sequential io performance. Where SAS shines is in random io. So if it's archive, buy SATA.<BR>
<BR>
65MB/s is roughly what you will see with a single SAS or SATA drive on reads, around 30MB/s for writes.<BR>
<BR>
Sequential io is measured in MB/s and random io in IOPS or ios per second.<BR>
<BR>
Each spindle in a stripe set will roughly add 50% perf to sequential io and add to the IOPS by the IOPS of the spindle (IOPS+IOPS...). A mirror counts as 1 spindle for reads and 1/2 a spindle for writes (unless RAID is capable of doing parallel reads then it counts as 1 1/2 of reads). A RAID 5 is always one less spindle due to parity and each spindle on writes counts as 1/#spindles (write-back cache helps lessen that hurt).<BR>
<BR>
For 4k sequential ios (larger block sizes will post larger numbers).<BR>
<BR>
1 spindle = 65MB/s and 175 IOPS<BR>
2 spindles = 97.5MB/s and 350 IOPS<BR>
3 spindles = 146.25MB/s and 525 IOPS<BR>
4 spindles = 219.375MB/s and 700 IOPS<BR>
<BR>
(175 IOPS is from 15K SAS with 3.5ms read seek and 2ms avg latency, figure 80 IOPS for good SATA drive)<BR>
<BR>
Now any performance below those numbers is a failure of the RAID system and any performance above those numbers is due to caching and read-ahead.<BR>
<BR>
I hope that helps.<BR>
<BR>
-Ross<BR>
<BR>
<BR>
----- Original Message -----<BR>
From: centos-bounces@centos.org <centos-bounces@centos.org><BR>
To: CentOS mailing list <centos@centos.org><BR>
Sent: Tue May 06 17:20:16 2008<BR>
Subject: Re: [CentOS] I need storage server advice<BR>
<BR>
I just posted this on my website, oddly enough. While you need to<BR>
really understand your storage requirements to make an informed choice<BR>
between hardware or software RAID, with quad core CPUs being as cheap as<BR>
they are it's hard to not make the argument for software.<BR>
This is just hdparm over an average of 5 runs each on very similar<BR>
machines.<BR>
<BR>
5 disc SAS array with 136g 10k drives and a hardware controller<BR>
<BR>
Timing cached reads: 13336 MB in 2.00 seconds = 6673.96 MB/sec<BR>
Timing buffered disk reads: 98 MB in 1.18 seconds = 83.31 MB/sec<BR>
<BR>
4 disc RAID 5 with 3Ware 9650SE and 500g 7200RPM drives<BR>
<BR>
Timing cached reads: 6576 MB in 2.00 seconds = 3293.08 MB/sec<BR>
Timing buffered disk reads: 448 MB in 3.00 seconds = 149.20 MB/sec<BR>
<BR>
Single 500g 7200 RPM SATA drive<BR>
<BR>
Timing cached reads: 14220 MB in 2.00 seconds = 7119.78 MB/sec<BR>
Timing buffered disk reads: 198 MB in 3.02 seconds = 65.51 MB/sec<BR>
<BR>
6 500g 7200 RPM SATA drives in a software RAID 5 array<BR>
<BR>
Timing cached reads: 14364 MB in 2.00 seconds = 7191.86 MB/sec<BR>
Timing buffered disk reads: 852 MB in 3.00 seconds = 283.64 MB/sec<BR>
<BR>
Jason<BR>
www.cyborgworkshop.org<BR>
<BR>
<BR>
Michael Semcheski wrote:<BR>
> On Tue, May 6, 2008 at 3:11 PM, Ed Morrison <edward.morrison@gmail.com<BR>
> <<A HREF="mailto:edward.morrison@gmail.com">mailto:edward.morrison@gmail.com</A>>> wrote:<BR>
><BR>
> Situation:<BR>
> My current storage needs are approximately 1.5 TB annually. This<BR>
> will increase to about 3.5 TB annually over the next 5 years (rough<BR>
> est.). This box will just be a data archive and once it is full it<BR>
> will only be used very infrequently if not used at all. Files are<BR>
> small up to 10 MB but numerous.<BR>
><BR>
><BR>
> The solution I found best was to buy a 2U server that has 8*750GB disks,<BR>
> though they'd probably be 1TB today. Put the disks into a RAID 5 or 6.<BR>
> Using hardware RAID, divvy them up into one 50GB drive, and one really<BR>
> large drive. Put the OS on the 50GB drive, mount the really big drive.<BR>
><BR>
> Now you have a 50GB drive and a 7*750-50 drive. When you fill that up,<BR>
> just buy another 2U server. When you do fill it up, the next one will<BR>
> be cheaper and or bigger.<BR>
><BR>
> The keys to this type of setup are:<BR>
> 1) Don't buy storage you'll need next year today. The best time to buy<BR>
> this kind of hardware is right before you need it.<BR>
> 2) Look at the overall cost per gigabyte. That's the metric that drives<BR>
> things.<BR>
> 3) Understand your tolerance for downtime and data protection. If you<BR>
> have another copy, or a backup, and its not mission critical data, its<BR>
> much cheaper not to waste disks on redundancy.<BR>
><BR>
> We have tape backups of our systems, and factoring in the cost of tape<BR>
> and other costs, its still possible to get storage with a marginal cost<BR>
> below $1 / GB. That includes a 3 year warranty, quad core processor,<BR>
> 4GB of RAM which you can probably put to use elsewhere.<BR>
><BR>
><BR>
><BR>
> ------------------------------------------------------------------------<BR>
><BR>
> _______________________________________________<BR>
> CentOS mailing list<BR>
> CentOS@centos.org<BR>
> <A HREF="http://lists.centos.org/mailman/listinfo/centos">http://lists.centos.org/mailman/listinfo/centos</A><BR>
_______________________________________________<BR>
CentOS mailing list<BR>
CentOS@centos.org<BR>
<A HREF="http://lists.centos.org/mailman/listinfo/centos">http://lists.centos.org/mailman/listinfo/centos</A><BR>
</FONT>
</P>
<P></P>
<HR WIDTH="100%">
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.
</BODY>
</HTML>