-----Original Message----- From: Jason Sent: Sunday, May 08, 2011 14:04 To: CentOS mailing list Subject: [CentOS] Building a Back Blaze style POD
Hi All,
I am about to embark on a project that deals with allowing information archival, over time and seeing change over time as well. I can explain it a lot better, but I would certainly talk your ear off. I really don't have a lot of money to throw at the initial concept, but I have some. This device will host all of the operations for the first few months until I can afford to build a duplicate device. I already had a few parts of the idea done and ready to get live.
I am contemplating building a BackBlaze Style POD. The goal of the device is to start acting as a place to have the crawls store information, massage it, get it into db's and then notify the user the task is done so they can start looking at the results.
For reference here are a few links:
http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how
-to-build-cheap-cloud-storage/
and
http://cleanenergy.harvard.edu/index.php?ira=Jabba&tipoConteni
do=sidebar&sidebar=science
Distrubing, I was on the same pages a few hours ago.
There is room for 45 drives in the case (technically a few more).
45 x 1tb 7200rpm drives is really cheap, about $60 each.
45 x 1.5tb 7200rpm drives are about $70 each.
45 x 2tb 7200rpm drives are about $120 each
45 x 3tb 7200rpm drives are about $180-$230 each (or more, some are almost $400)
I have question before I commit to building one and I was hoping to get advice.
- Can anyone recommend a mobo/processor setup that can hold
lots of RAM? Like 24gb or 64gb or more?
- Hardware RAID or Software RAID for this?
Hardware to costly in $ Software to costly in CPU.
Try for redundancy.
- Would CentOS be a good choice? I have never used CentOS on
a device so massive. Just ordinary servers, so to speak. I assume that it could handle so many drives, a large, expanding file system.
Multiple file systems of GFS?
- Someone recommended ZFS but I dont recall that being
available on CentOS, but it is on FreeBSD which I have little experience with.
- How would someone realistically back something like this up?
You don't. You replicate it. We are looking at using it as an online cache of our backup media.
Ultimately I know over time I need to distribute my architecture out and have a number of web-servers, balancing, etc but to get started I think this device with good backups might fit the bill.
I can be way more detailed if it helps, I just didn't want to clutter with information that might not be relevant. -- Jason
-- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - - - Jason Pyeron PD Inc. http://www.pdinc.us - - Principal Consultant 10 West 24th Street #100 - - +1 (443) 269-1555 x333 Baltimore, Maryland 21218 - - - -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- This message is copyright PD Inc, subject to license 20080407P00.