[CentOS] [OT] Building a new backup server

Tue Nov 5 20:41:02 UTC 2013
m.roth at 5-cent.us <m.roth at 5-cent.us>

Hey, Les,

   Thanks for changing the subject to OT.

Les Mikesell wrote:
> On Tue, Nov 5, 2013 at 1:28 PM,  <m.roth at 5-cent.us> wrote:
>> As I noted, we make sure rsync uses hard links... but we have a good
number of individual people and projects with who *each* have a good
number of terabytes of data and generated data. Some of our 2TB drives
>> over 90% full, and then there's the honkin' huge RAID, and at least one
14TB partition is over 9TB full....
> If you have database dumps or big text files that aren't compressed,
backuppc could be a big win.  I think it is the only thing that can keep
a compressed copy on the server side and work directly with a stock
rsync and uncompressed files on the target hosts (and it can cache the
block-checksums so it doesn't have to uncompress and
> recompute them every run).   While it is 'just a perl script' it's not
quite what you expect from simple scripting...

We have a *bunch* of d/bs. Oracle. MySQL. Postgresql. All with about a
week's dumps from every night, and then backups of them to the b/u
servers. I can't imagine how they'd be a win - don't remember just off the
top of my head if they're compressed or not.

A *lot* of our data is not huge text files - lots and lots of pure
datafiles, output from things like Matlab, R, and some local programs,
like the one for modeling protein folding.