Hi list,
I'm searching a valid backup system to perform backups of 3 server, one local and 2 remote, and 2 linux workstation. (this number could be higher in future). Currently I'm testing bacula, scripted rsync with hardlink and borgbackup on C8.1.
Bacula works without any problem, well tested, solid but complex to configure. Tested on a single server (with volumes on disk) and a full backup of 810gb (~150000 files) took 6,30 hours (too much). I would run deduplication but 1. bacula on C8 is not compiled with aligned 2. bacula put attention on some possibile bad scenario where deduplication could be a problem in losing one block that could break many files with the same shared block. So deduplication is secure or I should be away from it?
scripted rsync. Simple, through ssh protocol and private key. No agent required on target. I use file level deduplication using hardlinks. To perform compression and block deduplication I could use fs like zfs (not available from epel) or use something like stratis (I don't checked if it offer deduplication in this moment). Encryption could be performed on fs level. Using a scripted rsync is the simpler way but there is something that could be leaved out by me (or undiscovered error). Simple to restore.
BorgBackup is another solution similar to rsync in some way. It works like rsync through ssh but while rsync simply sends data, borg run deduplication, compression and encryption on place. Why is similar rsync? Because I need another script to run borg. It is not a complete solution and it works on push method and not pull like with bacula or rsync. If I would manage all my server from a central backup server is a problem, so to accomplish this I should run the borg command from the central server on the target server that point back to the remote repository (located on the central server). There is another solution with borg: using sshfs but is 4 time slower (too much solwer).
I have not so much experiences on backups and choosing the bad tool could be dangerous so I need some suggestion.
What backup solution do you suggest for my scenario?
Thank you in advance.
Alessandro.
Le 02/04/2020 à 17:32, Alessandro Baggi a écrit :
I have not so much experiences on backups and choosing the bad tool could be dangerous so I need some suggestion.
What backup solution do you suggest for my scenario?
I'm using Rsnapshot on all my CentOS 7 servers. It's a very elegant solution that follows the KISS principle.
I've written a little blog article about it. It's in French, but the Unix bits are universal. :o)
https://blog.microlinux.fr/rsnapshot-centos-7/
It's basically scripted rsync with ssh on steroids. Been using it for the last five years or so. It just works.
Cheers,
Niki
Il 02/04/20 17:49, Nicolas Kovacs ha scritto:
Le 02/04/2020 à 17:32, Alessandro Baggi a écrit :
I have not so much experiences on backups and choosing the bad tool could be dangerous so I need some suggestion.
What backup solution do you suggest for my scenario?
I'm using Rsnapshot on all my CentOS 7 servers. It's a very elegant solution that follows the KISS principle.
I've written a little blog article about it. It's in French, but the Unix bits are universal. :o)
https://blog.microlinux.fr/rsnapshot-centos-7/
It's basically scripted rsync with ssh on steroids. Been using it for the last five years or so. It just works.
Cheers,
Niki
Hi Niki, thank you for your answer.
I remember you when I used Slackware and I think your KISS is inherited by there but sometimes more complex things are needed. Rsnapshot is a great tool but I need catalog, jobs info, pre/post job script on remote target, mailing, compression and possibly store data off-site (no on a public cloud).
On Thu, Apr 02, 2020 at 05:32:35PM +0200, Alessandro Baggi wrote:
scripted rsync. Simple, through ssh protocol and private key. No agent required on target.
Just a point of clarification -- you need an rsync binary on both sides of the ssh session, so 'rsync' would be the agent needed on the target.
Yes you are right. I meant that I don't need an real agent like with bacula that need to be configured completely
Il Gio 2 Apr 2020, 19:52 Jonathan Billings billings@negate.org ha scritto:
On Thu, Apr 02, 2020 at 05:32:35PM +0200, Alessandro Baggi wrote:
scripted rsync. Simple, through ssh protocol and private key. No agent required on target.
Just a point of clarification -- you need an rsync binary on both sides of the ssh session, so 'rsync' would be the agent needed on the target.
-- Jonathan Billings billings@negate.org _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
I have been using rsnapshot for years, with great success.
Do note, backup systems that use rsync or similar file by file copies of a running system do not make coherent atomic snapshots, so things like relational databases should be excluded from those, and backed by database tools
On 4/3/20 8:34 AM, John Pierce wrote:
Do note, backup systems that use rsync or similar file by file copies of a running system do not make coherent atomic snapshots, so things like relational databases should be excluded from those, and backed by database tools
Long ago I learned to back up databases by dumping them (with a flag "lock" or similar to make sure no changed are made during dump), and backing up dump file.
Just my 2 cents.
Valeri
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Once upon a time, Valeri Galtsev galtsev@kicp.uchicago.edu said:
On 4/3/20 8:34 AM, John Pierce wrote:
Do note, backup systems that use rsync or similar file by file copies of a running system do not make coherent atomic snapshots, so things like relational databases should be excluded from those, and backed by database tools
Long ago I learned to back up databases by dumping them (with a flag "lock" or similar to make sure no changed are made during dump), and backing up dump file.
It isn't just databases - there are other things that backing up individual files one at a time is not so good. The best way to handle that is to freeze/snapshot the whole filesystem, and then back up the snapshot. This can be scripted pretty easily if the filesystem is on LVM.
Even better is to freeze _all_ filesystems simultaneously - this is usually easiest if the system is a virtual machine and/or the storage is on a SAN with snapshot capabilities.
Once upon a time, Valeri Galtsev galtsev@kicp.uchicago.edu said:
On 4/3/20 8:34 AM, John Pierce wrote:
Do note, backup systems that use rsync or similar file by file copies
of a
running system do not make coherent atomic snapshots, so things like relational databases should be excluded from those, and backed by
database
tools
Long ago I learned to back up databases by dumping them (with a flag "lock" or similar to make sure no changed are made during dump), and backing up dump file.
It isn't just databases - there are other things that backing up individual files one at a time is not so good. The best way to handle that is to freeze/snapshot the whole filesystem, and then back up the snapshot. This can be scripted pretty easily if the filesystem is on LVM.
Even better is to freeze _all_ filesystems simultaneously - this is usually easiest if the system is a virtual machine and/or the storage is on a SAN with snapshot capabilities.
Then again, to get a _really_ consistent backup, you can only terminate all applications who read/write files, or instruct the applications to go into a state which allows consistent file backups. Of course after the backup you have to instruct the applications to go on with the work. You can not achieve this consistency even with freezing _all_ filesystems simultaneously. That's why usually RDBMS's and other more complex applications provide their own backup mechanism embedded into the application.
That's why rsnapshot backups are not so much worse than filesystem snapshots.
Regards, Simon
On Fri, Apr 3, 2020 at 6:58 AM Chris Adams linux@cmadams.net wrote:
It isn't just databases - there are other things that backing up individual files one at a time is not so good. The best way to handle that is to freeze/snapshot the whole filesystem, and then back up the snapshot. This can be scripted pretty easily if the filesystem is on LVM.
I tried this with a fairly busy and rather large database server (ok, it was a database server running a simulation of a production workload) and LVM (with either ext4 or xfs) and found LVM snapshots to be completely unworkable under busy 8K block random write workloads
now, ZFS does snapshots very nicely
On 4/3/20 6:13 AM, miguel medalha wrote:
I have been using rsnapshot for years, with great success.
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Since no one else has mentioned it as a solution, I've been using amanda for years. About the only change has been replacing my physical tape drive with mhVTL (Anybody need some unused DSS-3 tapes? Free to good home.).
Cheers, Dave
-
Alessandro Baggi -
Chris Adams -
David G. Miller -
John Pierce -
Jonathan Billings -
miguel medalha -
Nicolas Kovacs -
Simon Matter -
Valeri Galtsev