[CentOS] Building a Back Blaze style POD

Sun May 8 18:23:15 UTC 2011
Jason Pyeron <jpyeron at pdinc.us>

> -----Original Message-----
> From: Jason
> Sent: Sunday, May 08, 2011 14:04
> To: CentOS mailing list
> Subject: [CentOS] Building a Back Blaze style POD
> Hi All,
> I am about to embark on a project that deals with allowing 
> information archival, over time and seeing change over time 
> as well. I can explain it a lot better, but I would certainly 
> talk your ear off. I really don't have a lot of money to 
> throw at the initial concept, but I have some. This device 
> will host all of the operations for the first few months 
> until I can afford to build a duplicate device. I already had 
> a few parts of the idea done and ready to get live.
> I am contemplating building a BackBlaze Style POD. The goal 
> of the device is to start acting as a place to have the 
> crawls store information, massage it, get it into db's and 
> then notify the user the task is done so they can start 
> looking at the results.
> For reference here are a few links:
> http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how
> and
> http://cleanenergy.harvard.edu/index.php?ira=Jabba&tipoConteni

Distrubing, I was on the same pages a few hours ago.

> There is room for 45 drives in the case (technically a few more).
> 45 x 1tb 7200rpm drives is really cheap, about $60 each.
> 45 x 1.5tb 7200rpm drives are about $70 each.
> 45 x 2tb 7200rpm drives are about $120 each
> 45 x 3tb 7200rpm drives are about $180-$230 each (or more, 
> some are almost $400)
> I have question before I commit to building one and I was 
> hoping to get advice.
> 1. Can anyone recommend a mobo/processor setup that can hold 
> lots of RAM? Like 24gb or 64gb or more? 
> 2. Hardware RAID or Software RAID for this?

Hardware to costly in $
Software to costly in CPU.

Try for redundancy.

> 3. Would CentOS be a good choice? I have never used CentOS on 
> a device so massive. Just ordinary servers, so to speak. I 
> assume that it could handle so many drives, a large, 
> expanding file system.

Multiple file systems of GFS?

> 4. Someone recommended ZFS but I dont recall that being 
> available on CentOS, but it is on FreeBSD which I have little 
> experience with.
> 5. How would someone realistically back something like this up?

You don't. You replicate it. We are looking at using it as an online cache of
our backup media.

> Ultimately I know over time I need to distribute my 
> architecture out and have a number of web-servers, balancing, 
> etc but to get started I think this device with good backups 
> might fit the bill.
> I can be way more detailed if it helps, I just didn't want to 
> clutter with information that might not be relevant.
> --
> Jason

-                                                               -
- Jason Pyeron                      PD Inc. http://www.pdinc.us -
- Principal Consultant              10 West 24th Street #100    -
- +1 (443) 269-1555 x333            Baltimore, Maryland 21218   -
-                                                               -
This message is copyright PD Inc, subject to license 20080407P00.