[CentOS] slow usb hard disk performance.

[Les Mikesell]

On Mon, 2005-12-05 at 16:04, Bryan J. Smith wrote:

> > Since it is almost full-auto - I still let amanda run tapes
> > to be held offsite but I really don't ever want to use them
> > except as a last resort,
> 
> But what about having Amanda not commit things to tape, and
> retrieve from the disk backup?  Not quite, eh?  ;->

Yes, amanda can recover from the holding disk copy.  Just
don't change the tape and it happens that way by itself until
you run out of holding space. 

> > hence the offsite backuppc disk.
> 
> But not that's an entirely different solution.  Wouldn't it
> be nice if the solution was catered to disk-to-disk, but also
> let you export/import to/from tape for select backups?  ;->

Backuppc does allow archiving the latest backup of a host
to tape or a tar image compressed and split into specified
size chunks like you might write to isos or dvds, but it
is sort of an afterthought and not automatic.  You could
easily script the command line tool to do the same, though.
But, the other thing amanda does is automatically mix the
full/incremental runs among a bunch of machines to make
them fit on your tape every night.  I don't think anything
else has that feature.

> BTW, I've just had a lot of clients send their disks out for
> data recovery.  So I can't condone off-line disk.
> 
> Now if you take the disk off-site and put it in another
> system, that'd different.  As long as it is getting
> periodically exercised, that is good.

I'm not doing archival storage, just a rotating copy of the
backuppc archive where the current version has a weeks
worth of daily runs for most of the boxes.  I rotate once
a week through a 4 drive cycle so I could restore to any
day in the last month with a bit of work to retrieve the
right disk.  In practice I've had only had to go back a
few days for things someone erased accidentally and didn't
notice right away - but one nightly snapshot that overwrites
the previous wouldn't have been enough.

> > My backuppc archive probably has at least a million
> > hardlinks and conventional copy mechanisms take longer than
> > practical.
> 
> Hardlinks _can_ be stored in a stream archive format.  ;->

But they can't be restored in a reasonable amount of time with
any tool that I've found.  I've let a tar | tar  and a
cp -a run for at least 3 days and they weren't done even
with a disk to disk copy.  I assume this is due to having to
search a huge table for matching inodes on every file to restore
the links. I wouldn't want to wait for this to happen before
being able to start the real restores out of that archive.

> Again, I think this is more about the lack of a good, unified
> open source system of disk-to-disk backup with tape
> export/import.  Too many systems are either disk-only or
> tape-only (with only disk as buffer in the best case, not
> multi-volume/multi-backup management).

Bacula (http://www.bacula.org/) looks promising and is probably
where I would start if I didn't already have something working,
but I don't think anything else can match the amount of data
you can cram on a disk with backuppc and especially not while
keeping the ability to use rsync as the remote client for
efficiency.

-- 
  Les Mikesell
    lesmikesell at gmail.com