Hi all,
We have a bit of a problem with backups. We've been using bacula to tape and now trying to disk but it's a complete nightmare as regards tape management. The backup to file storage went ok for ages and now is stuck 'waiting for max storage jobs' which is odd as that's set to 20 and it's the only backup running. That's totally typical of bacula.
Can anyone suggest a simple backup package for us? Essentially a single server, full backup to tape every day. We don't need tape management as we're fully capable of reading the written label on the tape ourselves.
On 09/16/10 10:34, Kevin Thorpe wrote:
Hi all,
We have a bit of a problem with backups. We've been using bacula to tape and now trying to disk but it's a complete nightmare as regards tape management. The backup to file storage went ok for ages and now is stuck 'waiting for max storage jobs' which is odd as that's set to 20 and it's the only backup running. That's totally typical of bacula.
Can anyone suggest a simple backup package for us? Essentially a single server, full backup to tape every day. We don't need tape management as we're fully capable of reading the written label on the tape ourselves.
Do you need tape backups? If not, consider automatic backups to HDD storage. For disaster recovery you can use a USB drive to take offsite. Or an e-sata drive in a hot swappable raid setup. Exchange once a day and bring it off-site. Or get some online backup storage to create an off-site mirror. I use good old dump with LVM snapshots to make daily consistent backups (works only for ext2/3 fs). Since it's fully automated, I only have to check the backup disk usage. Even there I automate the removal of old daily backups.
Theo
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On
Behalf
Of Theo Band Sent: Thursday, September 16, 2010 10:53 AM To: CentOS mailing list Subject: Re: [CentOS] Can anyone suggest a decent backup system?
Can anyone suggest a simple backup package for us? Essentially a single server, full backup to tape every day. We don't need tape management as we're fully capable of reading the written label on the tape ourselves.
Do you need tape backups?
If the answer to that last question is "no", consider BackupPC. It's open source and comparatively easy to setup, and even easier to use, unless you're a total n00b, like I was at first.
BackupPC has a very good support mailing list.
HTH.
On Thu, 16 Sep 2010, Kevin Thorpe wrote:
To: CentOS mailing list centos@centos.org From: Kevin Thorpe kevin@pibenchmark.com Subject: [CentOS] Can anyone suggest a decent backup system?
Hi all,
We have a bit of a problem with backups. We've been using bacula to tape and now trying to disk but it's a complete nightmare as regards tape management. The backup to file storage went ok for ages and now is stuck 'waiting for max storage jobs' which is odd as that's set to 20 and it's the only backup running. That's totally typical of bacula.
Can anyone suggest a simple backup package for us? Essentially a single server, full backup to tape every day. We don't need tape management as we're fully capable of reading the written label on the tape ourselves.
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
I've seen this on Fedora 12:
Name : BackupPC Arch : noarch Version : 3.1.0 Release : 13.fc12 Size : 2.2 M Repo : installed
From repo : updates
Summary : High-performance backup system URL : http://backuppc.sourceforge.net/ License : GPLv2+ Description : BackupPC is a high-performance, enterprise-grade system for backing up : Linux and WinXX PCs and laptops to a server's disk. BackupPC is highly : configurable and easy to install and maintain.
Not sure if it would work with tape drives though, or if it comes with CentOS 5.x
Looks like it's written in perl.
Kind Regards,
Keith
----------------------------------------------------------------- Websites: http://www.karsites.net http://www.php-debuggers.net http://www.raised-from-the-dead.org.uk
All email addresses are challenge-response protected with TMDA [http://tmda.net] -----------------------------------------------------------------
From: Kevin Thorpe kevin@pibenchmark.com
We have a bit of a problem with backups. We've been using bacula to tape and now trying to disk but it's a complete nightmare as regards tape management. The backup to file storage went ok for ages and now is stuck 'waiting for max storage jobs' which is odd as that's set to 20 and it's the only backup running. That's totally typical of bacula. Can anyone suggest a simple backup package for us? Essentially a single server, full backup to tape every day. We don't need tape management as we're fully capable of reading the written label on the tape ourselves.
Did you ask on the bacula mailing list what could be the problem...? You could try amanda, but it is quite as "heavy" as bacula. If you really want something simple, as in "write these folders to tape", maybe just make a small shell script that uses tar/afio + mt... I prefer afio because it is resilient to errors.
JD
On Thu, 16 Sep 2010, Kevin Thorpe wrote:
We have a bit of a problem with backups. We've been using bacula to tape and now trying to disk but it's a complete nightmare as regards tape management.
i suspect it's quite annoying when people try to re-engineer your request, but i use bacula for some fairly large installations (several hundred LTO2 tapes, 60-tape dual-drive stacker with barcode support, 5TB of staging disc, that sort of thing) as well as my much-smaller home backups, and find its tape management lovely - after i made one or two small but important tweaks.
if you're not already so sick of bacula that the mere name makes you break out in hives, might i ask what problems you've been having? on- or off-list is fine, as you prefer.
On 16/09/2010 10:35, Tom Yates wrote:
On Thu, 16 Sep 2010, Kevin Thorpe wrote:
We have a bit of a problem with backups. We've been using bacula to tape and now trying to disk but it's a complete nightmare as regards tape management.
i suspect it's quite annoying when people try to re-engineer your request, but i use bacula for some fairly large installations (several hundred LTO2 tapes, 60-tape dual-drive stacker with barcode support, 5TB of staging disc, that sort of thing) as well as my much-smaller home backups, and find its tape management lovely - after i made one or two small but important tweaks.
if you're not already so sick of bacula that the mere name makes you break out in hives, might i ask what problems you've been having? on- or off-list is fine, as you prefer.
You've hit the nail on the head. You've got the proper tape library hardware. Our ISP uses Bacula absolutely fine for our hosted servers.
Our problem is mostly tape management. It will work fine for ages then we do something stupid like miss a tape and put it in in the morning, or we get a bank holiday and it starts getting picky about which tape it wants. For example we gave it monday's tape which was 'full' when it only has a 4 day retention period and it didn't like it and wanted the thursday tape. Why thursday I don't know, mon tue and wed should all have been available.
Essentially it works absolutely fine but isn't very tolerant of human error. I've fiddled with it for ages and it's just irritating me.
I'll definitely look into BackupPC for HDD backups but they're in the same offfice and don't give us the audit monthend tapes we are required to keep indefinitely.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On
Behalf
Of Kevin Thorpe Sent: Thursday, September 16, 2010 11:59 AM To: CentOS mailing list Subject: Re: [CentOS] Can anyone suggest a decent backup system?
I'll definitely look into BackupPC for HDD backups but they're in the same offfice and don't give us the audit monthend tapes we are required to keep
indefinitely.
Given you have enough storage space, it's possible to keep a full backup with BackupPC indefinitely as well, *if* you set up the rentention parameters properly.
On Thu, 16 Sep 2010, Kevin Thorpe wrote:
You've hit the nail on the head. You've got the proper tape library hardware. Our ISP uses Bacula absolutely fine for our hosted servers.
ah, right.
Our problem is mostly tape management. It will work fine for ages then we do something stupid like miss a tape and put it in in the morning, or we get a bank holiday and it starts getting picky about which tape it wants. For example we gave it monday's tape which was 'full' when it only has a 4 day retention period and it didn't like it and wanted the thursday tape. Why thursday I don't know, mon tue and wed should all have been available.
yes, i completely agree; the stacker changes everything. sorry to have jumped in, and i hope you find something that works for you.
Freenas is the way do it. Very simple and fast to get up and running on most hardware platforms. Supports E-sata,USB, IDE, RAID configurations. Can use an old pc laying around but recommend more up2date hardware to meet the demands of a large system. Dual GigE ports with mtu modified will push/pull more bandwidth than most needs required.
--Dave
On 9/16/2010 3:34 AM, Kevin Thorpe wrote:
Hi all,
We have a bit of a problem with backups. We've been using bacula to tape and now trying to disk but it's a complete nightmare as regards tape management. The backup to file storage went ok for ages and now is stuck 'waiting for max storage jobs' which is odd as that's set to 20 and it's the only backup running. That's totally typical of bacula.
Can anyone suggest a simple backup package for us? Essentially a single server, full backup to tape every day. We don't need tape management as we're fully capable of reading the written label on the tape ourselves.
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 9/16/2010 3:34 AM, Kevin Thorpe wrote:
Hi all,
We have a bit of a problem with backups. We've been using bacula to tape and now trying to disk but it's a complete nightmare as regards tape management. The backup to file storage went ok for ages and now is stuck 'waiting for max storage jobs' which is odd as that's set to 20 and it's the only backup running. That's totally typical of bacula.
Can anyone suggest a simple backup package for us? Essentially a single server, full backup to tape every day. We don't need tape management as we're fully capable of reading the written label on the tape ourselves.
Not sure of a package - we use rsync and scripts - but you should consider what we do: an external eSATA dock, and a number of inexpensive SATA drives. It *will* speed up the backups, and recoveries, should you need them.
mark
On 09/16/10 6:27 AM, m.roth@5-cent.us wrote:
Not sure of a package - we use rsync and scripts - but you should consider what we do: an external eSATA dock, and a number of inexpensive SATA drives. It *will* speed up the backups, and recoveries, should you need them.
actually? tapes like LTO/DLT write *faster* than file systems on hard disks. in fact the biggest issue in many LTO/DLT backup systems is not being able to READ the source fast enough to keep the tape busy.
re: BackupPC... while this is a neat solution for backing up a small-to-moderate number of workstations, I don't think its a very good solution for archival backups of a single server, NOR does it scale very well, as the storage pool it creates becomes an ungodly mess of links and becomes itself very very difficult to replicate or backup or do maintenance on.
Systems like Amanda and Bacula really work better with autoloader/libraries, like the Quantum PX502 I use in my lab at work (acquired surplus from another department, this holds 28 tapes and has two LTO3 drives).
On 9/16/2010 12:20 PM, John R Pierce wrote:
actually? tapes like LTO/DLT write *faster* than file systems on hard disks. in fact the biggest issue in many LTO/DLT backup systems is not being able to READ the source fast enough to keep the tape busy.
re: BackupPC... while this is a neat solution for backing up a small-to-moderate number of workstations, I don't think its a very good solution for archival backups of a single server,
This sort of depends on the nature of the files and rate of change. Backuppc pools all copies of identical content with optional compression, whether from multiple backup runs or different targets. If you have huge files with frequent small changes, it has to make new complete copies each time, but otherwise it can aggregate many times what you'd expect the disk space to hold and you can access it much more easily than finding the right set of tapes.
NOR does it scale very well, as the storage pool it creates becomes an ungodly mess of links and becomes itself very very difficult to replicate or backup or do maintenance on.
That imposes some limits, but they are sill fairly big if you think in terms of image-copying the whole filesystem if you need to replicate.
Systems like Amanda and Bacula really work better with autoloader/libraries, like the Quantum PX502 I use in my lab at work (acquired surplus from another department, this holds 28 tapes and has two LTO3 drives).
Agreed there. Backuppc doesn't know much about tapes and nothing about changers. But for straight long-term archiving you could wrap a script around BackupPC_tarCreate to save whatever you wanted off to tape. And you are on your own for restoring it, but it might be an advantage to have a standard tar archive than something only a certain program knows how to access.
On September 16, 2010 10:49:04 am Les Mikesell wrote:
Agreed there. Backuppc doesn't know much about tapes and nothing about changers. But for straight long-term archiving you could wrap a script around BackupPC_tarCreate to save whatever you wanted off to tape. And you are on your own for restoring it, but it might be an advantage to have a standard tar archive than something only a certain program knows how to access.
Amanda's dumps are standard tar archives and can be restored without Amanda.
On 9/16/2010 12:52 PM, Alan Hodgson wrote:
On September 16, 2010 10:49:04 am Les Mikesell wrote:
Agreed there. Backuppc doesn't know much about tapes and nothing about changers. But for straight long-term archiving you could wrap a script around BackupPC_tarCreate to save whatever you wanted off to tape. And you are on your own for restoring it, but it might be an advantage to have a standard tar archive than something only a certain program knows how to access.
Amanda's dumps are standard tar archives and can be restored without Amanda.
Well, sort-of. You have to know how to skip over the amanda label and header. And how to find the right set of tapes.
On September 16, 2010 11:02:25 am Les Mikesell wrote:
Amanda's dumps are standard tar archives and can be restored without Amanda.
Well, sort-of. You have to know how to skip over the amanda label and header. And how to find the right set of tapes.
Sure, but at least it's documented. Better than reinventing the wheel. Especially for your successor.
On 09/16/10 1:34 AM, Kevin Thorpe wrote:
Can anyone suggest a simple backup package for us? Essentially a single server, full backup to tape every day. We don't need tape management as we're fully capable of reading the written label on the tape ourselves.
ummmm, reading your requirements again, and cogitating..... and...
tar
a simple script that tars each file system appending it to the same tape (assuming your backup fits on one tape).... maybe something like...
mt -f /dev/nst0 rewind for fs in / /var /usr /home /opt /data ...; do tar clf /dev/nst0 $fs done mt -f /dev/nst0 offline
If you do your backups this way, its important to keep track of the order you write the file systems to the tape, as the tape position is the only way to distinguish them
I'd probably add some logging to that script, and basic error trapping. the following mess is totally untested and probably full of errors. caveat emptor.
#!/bin/bash d=${date -I} lf=/var/log/mybackups-$d.log df=/var/log/mybackups-$d.detail.log dt=${date +"%Y-%m-%d %H:%M:%S%z"} echo "$dt ******* Starting Daily Backup **********" >>$lf mt -f /dev/nst0 rewind for fs in / /var /usr /home /opt /data ...; do echo "$dt Starting Backup $fs" >>$lf tar clvf /dev/nst0 $fs 2>>$lf 1>>$df if [ $? != 0 ]; then { echo "$dt Error in tar $fs. backups aborted" >>$lf mail -s "***DAILY BACKUP ERROR****" user@mydomain.com < $lf exit 1 } fi echo "$dt Completed Backup $fs" >>$lf done echo "$dt Ejecting Tape" >>$lf mt -f /dev/nst0 offline echo "$dt ******* Daily Backup Complete **********" >>$lf
-
Alan Hodgson -
David Milholen -
John Doe -
John R Pierce -
Keith Roberts -
Kevin Thorpe -
Les Mikesell -
m.roth＠5-cent.us -
Sorin Srbu -
Theo Band -
Tom Yates