[CentOS] slow usb hard disk performance.

Mon Dec 5 19:27:21 UTC 2005
Bryan J. Smith <thebs413 at earthlink.net>

Les Mikesell <lesmikesell at gmail.com> wrote:
> My favorite for online disk backups:
> http://backuppc.sourceforge.net/
> It uses compression and linking of duplicate files to allow
> holding much more data than you'd expect so you can keep
> some history around.

Yes, I use it myself.

> The free version is to image-copy the backuppc archive
> filesystem to an external drive. It's cheaper than a fast
> tape drive and you can plug the external disk into a laptop
> anywhere for immediate remote restores without needing
> another tape drive.

How many times do I have to say it, I _never_ backup directly
to tape.  I _always_ backup to disk.  ;->  I then do _not_
commit every backup to tape, only a subset.  Anyone who backs
up everything -- let alone everything directly to tape -- is
living in the '80s.

There is nothing more chronically stupid than direct backup
to tape -- especially over a network.  I don't know how many
times I go into companies and they are still trying to do
end-node to tape-server backups in an 8 hour window and just
can't do it.  I always just hit my head and say, "why don't
you sync your filesystem changes to a centralized server, and
then backup from that system locally to the tape -- which you
can do in the middle of the day?!"

It's like an epiphany for them -- so obvious they missed it.

You should at least buffer to disk first, if not just store
backups whole on disk.  Then and only then should any
long-term backups, retention or off-site considerations be
put to tape -- locally on the same system.  But not
everything has to go to tape -- in fact, a _minority_, only
those things that are going off-site or need to be stored
longer than a few months.  At the same time, don't trust
backups for months on a fixed disk.

> The downside has been firewire support on Linux the way I'm
> trying to do it, which is to periodically sync a RAID1
mirror
> and break it for offsite storage.

I'd rather not backup in a filesystem format.  I'd rather
backup to a streaming archive format which is far more
recoverable when there are errors.  That way errors are
localized and don't affect the rest of the backup. 
Filesystems rely on meta-data to recover data -- streaming
archive formats do not.

As far as off-line storage, I do not trust commodity fixed
disk more than 3 months on the shelf (if even that long). 
They are not designed to be stored that long and inactive
after periodic activity.  The vendors have come up with
removable rigid disk (RRD) to address the longevity issues,
but then RRD loses the performance or cost of fixed disk.

> I suppose I could unmount the internal disk and use dd to
> copy for the same effect.

Why not a streaming backup format like afio or ustar?  Far
more recoverable and errors are localized than with a
traditional filesystem.



-- 
Bryan J. Smith                | Sent from Yahoo Mail
mailto:b.j.smith at ieee.org     |  (please excuse any
http://thebs413.blogspot.com/ |   missing headers)