[CentOS] [OT] Building a new backup server

m.roth at 5-cent.us m.roth at 5-cent.us
Tue Nov 5 20:41:02 UTC 2013


Hey, Les,

   Thanks for changing the subject to OT.

Les Mikesell wrote:
> On Tue, Nov 5, 2013 at 1:28 PM,  <m.roth at 5-cent.us> wrote:
>>
>> As I noted, we make sure rsync uses hard links... but we have a good
number of individual people and projects with who *each* have a good
number of terabytes of data and generated data. Some of our 2TB drives
are
>> over 90% full, and then there's the honkin' huge RAID, and at least one
14TB partition is over 9TB full....
>
> If you have database dumps or big text files that aren't compressed,
backuppc could be a big win.  I think it is the only thing that can keep
a compressed copy on the server side and work directly with a stock
rsync and uncompressed files on the target hosts (and it can cache the
block-checksums so it doesn't have to uncompress and
> recompute them every run).   While it is 'just a perl script' it's not
quite what you expect from simple scripting...

We have a *bunch* of d/bs. Oracle. MySQL. Postgresql. All with about a
week's dumps from every night, and then backups of them to the b/u
servers. I can't imagine how they'd be a win - don't remember just off the
top of my head if they're compressed or not.

A *lot* of our data is not huge text files - lots and lots of pure
datafiles, output from things like Matlab, R, and some local programs,
like the one for modeling protein folding.

       mark






More information about the CentOS mailing list