Sorin Srbu wrote:
Guys,
I was thrown a cheap OEM-server with a 120 GB SSD and 10 x 4 TB
SATA-disks for the data-backup to build a backup server. It's built around an Asus Z87-A
that seems to have problems with anything Linux unfortunately.
Anyway, BackupPC is my preferred backup-solution, so I went ahead to
install another favourite, CentOS 6.4 - and failed.
The raid controller is a Highpoint RocketRAID 2740 and its driver is
suggested to be prior to starting Anaconda by way of "ctrl-alt-f2", at which point
Anaconda freezes.
I've come so far as installing Fedora 19 and having it see all the
hard-drives, but it refuses to create any partition bigger than approx. 16 TB with ext4.
I've never had to deal with this big raid-arrays before and am a bit
stumped.
Any hints as to where to start reading up, as well as hints on how to
proceed (another motherboard, ditto raidcontroller?), would be greatly appreciated.
Several. First, see if you CentOS supports that card. The alternative is to go to Highpoint's website, and look for the driver. You *might* need to get the source and build it - I had to do that a few months ago, on an old 2260 (I think it is) card, and had to hack the source -they're *not* good about updates. If you're lucky, they'll have a current driver or source.
Second, on our HBR's (that's a technical term - Honkin' Big RAIDS... <g>), we use ext4, and RAID 6. Also, for about two years, I keep finding things that say that although ext4 supports gigantic filesystems, the tools aren't there yet. The upshot is that I make several volumes and partition them into 14TB-16TB filesystems.
Besides, if you have a problem with a truly humongous RAID, the rebuild will finish sometime around next summer....
mark
On Mon, Nov 4, 2013 at 11:05 AM, m.roth@5-cent.us wrote:
I was thrown a cheap OEM-server with a 120 GB SSD and 10 x 4 TB
SATA-disks for the data-backup to build a backup server. It's built around an Asus Z87-A
that seems to have problems with anything Linux unfortunately.
Anyway, BackupPC is my preferred backup-solution, so I went ahead to
install another favourite, CentOS 6.4 - and failed.
The raid controller is a Highpoint RocketRAID 2740 and its driver is
suggested to be prior to starting Anaconda by way of "ctrl-alt-f2", at which point
Anaconda freezes.
I've come so far as installing Fedora 19 and having it see all the
hard-drives, but it refuses to create any partition bigger than approx. 16 TB with ext4.
I've never had to deal with this big raid-arrays before and am a bit
stumped.
Any hints as to where to start reading up, as well as hints on how to
proceed (another motherboard, ditto raidcontroller?), would be greatly appreciated.
Several. First, see if you CentOS supports that card. The alternative is to go to Highpoint's website, and look for the driver. You *might* need to get the source and build it - I had to do that a few months ago, on an old 2260 (I think it is) card, and had to hack the source -they're *not* good about updates. If you're lucky, they'll have a current driver or source.
Second, on our HBR's (that's a technical term - Honkin' Big RAIDS... <g>), we use ext4, and RAID 6. Also, for about two years, I keep finding things that say that although ext4 supports gigantic filesystems, the tools aren't there yet. The upshot is that I make several volumes and partition them into 14TB-16TB filesystems.
BackupPC pools data with hardlinks so you have to put its entire archive in one filesystem. However, unless you know for certain that you'll need the entire space for backuppc I'd recommend breaking the space up into sizes that your RAM will handle for an XFS fsck. As much as I like backuppc, there are some things it doesn't handle well (VM images, huge databases or single-file mailboxes with daily changes, etc.) and you might use the other portions for ad-hoc copies of things like that.
Besides, if you have a problem with a truly humongous RAID, the rebuild will finish sometime around next summer....
Yes, I'd probably use a RAID10 style RAID so it runs at full speed even with a drive out of the array so you can put off the rebuild until a weekend. A rebuild will keep the heads too busy to do a lot of other work while it is running.
On 11/4/2013 9:30 AM, Les Mikesell wrote:
Besides, if you have a problem with a truly humongous RAID, the rebuild
will finish sometime around next summer....
Yes, I'd probably use a RAID10 style RAID so it runs at full speed even with a drive out of the array so you can put off the rebuild until a weekend. A rebuild will keep the heads too busy to do a lot of other work while it is running.
Using a common LSI Logic Megaraid card (9260-8i, I think, but don't quote me), and 11 by 3TB raid6 with hotspares (using Seagate Constellation.ES2 SAS2 drives), single drive failure rebuilds took 24 hours, double drive failures took 36 hours, this with the system online but mostly idle the whole time. yes, if you have steady user IO activity going on the rebuild would take quite a bit longer, as it only proceeds when the disks are idle.
On 2013-11-04, John R Pierce pierce@hogranch.com wrote:
On 11/4/2013 9:30 AM, Les Mikesell wrote:
Besides, if you have a problem with a truly humongous RAID, the rebuild
will finish sometime around next summer....
Yes, I'd probably use a RAID10 style RAID so it runs at full speed even with a drive out of the array so you can put off the rebuild until a weekend. A rebuild will keep the heads too busy to do a lot of other work while it is running.
Using a common LSI Logic Megaraid card (9260-8i, I think, but don't quote me), and 11 by 3TB raid6 with hotspares (using Seagate Constellation.ES2 SAS2 drives), single drive failure rebuilds took 24 hours, double drive failures took 36 hours, this with the system online but mostly idle the whole time. yes, if you have steady user IO activity going on the rebuild would take quite a bit longer, as it only proceeds when the disks are idle.
The MegaRAID controllers have a configurable rebuild rate. The default is 30%, but you can increase it up to 100% (which will probably kill your system IO). LSI 3ware controllers are also configurable in a similar way (their scale is only 1..5).
--keith
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Les Mikesell Sent: den 4 november 2013 18:31 To: CentOS mailing list Subject: Re: [CentOS] [OT] Building a new backup server
BackupPC pools data with hardlinks so you have to put its entire archive in one filesystem. However, unless you know for certain that you'll need the entire space for backuppc I'd recommend breaking the space up into sizes that your RAM will handle for an XFS fsck. As much as I like backuppc, there are some things it doesn't handle well (VM images, huge databases or single-file mailboxes with daily changes, etc.) and you might use the other portions for ad-hoc copies of things like that.
Do we have other options than BackupPC on CentOS that might work better?
-- //Sorin
On 11/5/2013 5:06 AM, Sorin Srbu wrote:
Do we have other options than BackupPC on CentOS that might work better?
the only reason to say it doesn't handle those usecases well is that each incremental will probably backup the whole file so it won't dedup at all... well, ANY backup system will be faced with the same problem.
other open source backup systems include things like Amanda, Bacula, which are more tape oriented, although they can be used with disk archives. Amanda uses tar for the actual backups, and manages/tracks an archive of tar files. These use agents tha thave to be installed on the client systems, while backuppc usually uses ssh+rsync so you just need to do a ssh key exchange with the target (but on a per target basis it can be configured to use various other methods)
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of John R Pierce Sent: den 5 november 2013 19:08 To: centos@centos.org Subject: Re: [CentOS] [OT] Building a new backup server
other open source backup systems include things like Amanda, Bacula, which are more tape oriented, although they can be used with disk archives. Amanda uses tar for the actual backups, and manages/tracks an archive of tar files. These use agents tha thave to be installed on the client systems, while backuppc usually uses ssh+rsync so you just need to do a ssh key exchange with the target (but on a per target basis it can be configured to use various other methods)
Thanks for the advice.
Bacula: "Multi-volume saves. When a Volume is full, Bacula automatically requests the next Volume and continues the backup."
This means I could create several eg 10 TB-volumes, skip the 16 TB-limitation and still get to use the whole 40 TB-diskspace available, right? Or is the referred "volumes" different tapes?
-- //Sorin
On 04.11.2013 18:05, m.roth@5-cent.us wrote:
Sorin Srbu wrote:
Guys,
I was thrown a cheap OEM-server with a 120 GB SSD and 10 x 4 TB
SATA-disks for the data-backup to build a backup server. It's built around an Asus Z87-A
that seems to have problems with anything Linux unfortunately.
Anyway, BackupPC is my preferred backup-solution, so I went ahead to
install another favourite, CentOS 6.4 - and failed.
The raid controller is a Highpoint RocketRAID 2740 and its driver is
suggested to be prior to starting Anaconda by way of "ctrl-alt-f2", at which point
Anaconda freezes.
I've come so far as installing Fedora 19 and having it see all the
hard-drives, but it refuses to create any partition bigger than approx. 16 TB with ext4.
I've never had to deal with this big raid-arrays before and am a bit
stumped.
Any hints as to where to start reading up, as well as hints on how to
proceed (another motherboard, ditto raidcontroller?), would be greatly appreciated.
Several. First, see if you CentOS supports that card. The alternative is to go to Highpoint's website, and look for the driver. You *might* need to get the source and build it - I had to do that a few months ago, on an old 2260 (I think it is) card, and had to hack the source -they're *not* good about updates. If you're lucky, they'll have a current driver or source.
Second, on our HBR's (that's a technical term - Honkin' Big RAIDS... <g>), we use ext4, and RAID 6. Also, for about two years, I keep finding things that say that although ext4 supports gigantic filesystems, the tools aren't there yet. The upshot is that I make several volumes and partition them into 14TB-16TB filesystems.
In that case it might be better to switch to XFS which is supported by Red Hat up to 100TB so up to that capacity should work well. With RHEL 7 XFS will become the default Filesystem anyway so now is the time to get used to it.
Regards, Dennis
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Dennis Jacobfeuerborn Sent: den 4 november 2013 22:30 To: centos@centos.org Subject: Re: [CentOS] [OT] Building a new backup server
In that case it might be better to switch to XFS which is supported by Red Hat up to 100TB so up to that capacity should work well. With RHEL 7 XFS will become the default Filesystem anyway so now is the time to get used to it.
Yeah? That sounds really interesting. Is this listed on the RHEL website?
-- //Sorin
On Tue, Nov 5, 2013 at 8:09 AM, Sorin Srbu Sorin.Srbu@orgfarm.uu.se wrote:
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Dennis Jacobfeuerborn Sent: den 4 november 2013 22:30 To: centos@centos.org Subject: Re: [CentOS] [OT] Building a new backup server
In that case it might be better to switch to XFS which is supported by Red Hat up to 100TB so up to that capacity should work well. With RHEL 7 XFS will become the default Filesystem anyway so now is the time to get used to it.
Yeah? That sounds really interesting. Is this listed on the RHEL website?
In my brief search I didn't mind to find anything on Red Hat's web site, but I did find the below articles.
https://www.suse.com/communities/conversations/xfs-the-file-system-of-choice... http://searchdatacenter.techtarget.com/news/2240185580/Red-Hat-discloses-RHE...
-- //Sorin
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 05.11.2013 14:35, SilverTip257 wrote:
On Tue, Nov 5, 2013 at 8:09 AM, Sorin Srbu Sorin.Srbu@orgfarm.uu.se wrote:
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Dennis Jacobfeuerborn Sent: den 4 november 2013 22:30 To: centos@centos.org Subject: Re: [CentOS] [OT] Building a new backup server
In that case it might be better to switch to XFS which is supported by Red Hat up to 100TB so up to that capacity should work well. With RHEL 7 XFS will become the default Filesystem anyway so now is the time to get used to it.
Yeah? That sounds really interesting. Is this listed on the RHEL website?
In my brief search I didn't mind to find anything on Red Hat's web site, but I did find the below articles.
https://www.suse.com/communities/conversations/xfs-the-file-system-of-choice... http://searchdatacenter.techtarget.com/news/2240185580/Red-Hat-discloses-RHE...
Take a look at the first comment here: https://access.redhat.com/site/discussions/476563
You are not going to get any more official information on what is going th happen in RHEL 7 until it's actually out the door.
The was a video on youtube on the Red Hat Summit channel where they spoke in great detail about what is planned for RHEL 7 but strangely this video has been removed.
Regards, Dennis
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of SilverTip257 Sent: den 5 november 2013 14:36 To: CentOS mailing list Subject: Re: [CentOS] [OT] Building a new backup server
In that case it might be better to switch to XFS which is supported by Red Hat up to 100TB so up to that capacity should work well. With RHEL 7 XFS will become the default Filesystem anyway so now is the time to get used to it.
Yeah? That sounds really interesting. Is this listed on the RHEL website?
In my brief search I didn't mind to find anything on Red Hat's web site, but I did find the below articles.
https://www.suse.com/communities/conversations/xfs-the-file-system-of-choice... http://searchdatacenter.techtarget.com/news/2240185580/Red-Hat-discloses-RHE...
Seems to be a lot of "if's" still though. In any case, an interesting development. Can't wait to see how it ends. According to Wikipedia RHEL 7 is scheduled for release 2Q 2013; <http://www.pcworld.com/article/255629/red_hat_preps_rhel_7_for_second_half_of_2013.html>. -- //Sorin
Sorin Srbu wrote:
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of SilverTip257 Sent: den 5 november 2013 14:36 To: CentOS mailing list Subject: Re: [CentOS] [OT] Building a new backup server
In that case it might be better to switch to XFS which is supported by Red Hat up to 100TB so up to that capacity should work well. With RHEL 7 XFS will become the default Filesystem anyway so now is the
time to
get used to it.
Yeah? That sounds really interesting. Is this listed on the RHEL
website?
<snip>
According to Wikipedia RHEL 7 is scheduled for release 2Q 2013; <http://www.pcworld.com/article/255629/red_hat_preps_rhel_7_for_second_half_of_2013.html>.
I think you meant (2Q++)++ <g>
mark
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of m.roth@5-cent.us Sent: den 5 november 2013 15:35 To: CentOS mailing list Subject: Re: [CentOS] [OT] Building a new backup server
According to Wikipedia RHEL 7 is scheduled for release 2Q 2013;
<http://www.pcworld.com/article/255629/red_hat_preps_rhel_7_for_second_half_of_2013.html>.
I think you meant (2Q++)++ <g>
Well, maybe. 8-}
Let's not diss our upstream provider. I'm pretty sure they do a good job, judging from how good CentOS works. ;-)
-- //Sorin
On 05.11.2013 16:06, Sorin Srbu wrote:
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of m.roth@5-cent.us Sent: den 5 november 2013 15:35 To: CentOS mailing list Subject: Re: [CentOS] [OT] Building a new backup server
According to Wikipedia RHEL 7 is scheduled for release 2Q 2013;
<http://www.pcworld.com/article/255629/red_hat_preps_rhel_7_for_second_half_of_2013.html>.
I think you meant (2Q++)++ <g>
Well, maybe. 8-}
Let's not diss our upstream provider. I'm pretty sure they do a good job, judging from how good CentOS works. ;-)
That's the reason why Red Hat refuses to make any official announcements before the actual release. it might turn out that some planned features are not ready in time and have to be removed or the release itself has to be delayed if there are features that need a little more baking but are considered to be important part for strategic reason. They try to give you an idea what is likely to be included in the next release but unfortunately there are always people who then jump on the "but you promised!" bandwagon when some of the features don't materialize.
Regards, Dennis
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of m.roth@5-cent.us Sent: den 4 november 2013 18:06 To: CentOS mailing list Subject: Re: [CentOS] [OT] Building a new backup server
Any hints as to where to start reading up, as well as hints on how to proceed (another motherboard, ditto raidcontroller?), would be greatly appreciated.
Several. First, see if you CentOS supports that card. The alternative is to go to Highpoint's website, and look for the driver. You *might* need to get the source and build it - I had to do that a few months ago, on an old 2260 (I think it is) card, and had to hack the source -they're *not* good about updates. If you're lucky, they'll have a current driver or source.
Highpoint actually had several drivers listed for RHEL/CentOS for download at their site.
Second, on our HBR's (that's a technical term - Honkin' Big RAIDS... <g>), we use ext4, and RAID 6. Also, for about two years, I keep finding things that say that although ext4 supports gigantic filesystems, the tools aren't there yet. The upshot is that I make several volumes and partition them into 14TB-16TB filesystems.
Can e.g. BackupPC handle several file systems to backup to? I.e. comp1 through 10 should backup to /bak1, comp 11 through 20 to /bak2 and so on.
What do you guys use for backup programs?
Besides, if you have a problem with a truly humongous RAID, the rebuild will finish sometime around next summer....
That one has been nagging me too. 8-/
-- //Sorin
On Tue, Nov 5, 2013 at 7:04 AM, Sorin Srbu Sorin.Srbu@orgfarm.uu.se wrote:
Can e.g. BackupPC handle several file systems to backup to? I.e. comp1 through 10 should backup to /bak1, comp 11 through 20 to /bak2 and so on.
The main point of backuppc is that it hard-links all files with identical content to save space, so it needs to put everything on one filesystem. However, I'm getting pretty good performance running it in virtual machines so the simple approach would be to run different instances in different VMs - making sure they don't share physical disks for their archive partitions, and if possible, using different physical NICs. I'm running mine under the free version of Vmware ESXi, (which might also solve the problem of missing drivers) but KVM should work equally well.
But, it is also handy to have a big chunk of disk space shared out via NFS and samba to dump VM images, iso images, database dumps, clonezilla images, etc. in an ad-hoc fashion.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Les Mikesell Sent: den 5 november 2013 15:09 To: CentOS mailing list Subject: Re: [CentOS] [OT] Building a new backup server
Can e.g. BackupPC handle several file systems to backup to? I.e. comp1 through 10 should backup to /bak1, comp 11 through 20 to /bak2 and so on.
The main point of backuppc is that it hard-links all files with identical content to save space, so it needs to put everything on one filesystem. However, I'm getting pretty good performance running it in virtual machines so the simple approach would be to run different instances in different VMs - making sure they don't share physical disks for their archive partitions, and if possible, using different physical NICs. I'm running mine under the free version of Vmware ESXi, (which might also solve the problem of missing drivers) but KVM should work equally well.
I'm getting there myself, it would seem. Thanks for confirming this might be the "right" way to go.
-- //Sorin
On Tue, Nov 5, 2013 at 8:20 AM, Sorin Srbu Sorin.Srbu@orgfarm.uu.se wrote:
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Les Mikesell Sent: den 5 november 2013 15:09 To: CentOS mailing list Subject: Re: [CentOS] [OT] Building a new backup server
Can e.g. BackupPC handle several file systems to backup to? I.e. comp1 through 10 should backup to /bak1, comp 11 through 20 to /bak2 and so on.
The main point of backuppc is that it hard-links all files with identical content to save space, so it needs to put everything on one filesystem. However, I'm getting pretty good performance running it in virtual machines so the simple approach would be to run different instances in different VMs - making sure they don't share physical disks for their archive partitions, and if possible, using different physical NICs. I'm running mine under the free version of Vmware ESXi, (which might also solve the problem of missing drivers) but KVM should work equally well.
I'm getting there myself, it would seem. Thanks for confirming this might be the "right" way to go.
If you have some time to experiment, look on the backuppc development list for the new alpha version. It is very different and does not need the hardlinks for pooling. I haven't tried it myself yet, but would (cautiously...) if I needed to set up a new system. It may eliminate the single-filesystem requirement and will definitely make it more feasible to rsync the whole archive to maintain an offsite copy. I think it may also chunk up large files so unchanged blocks can be pooled even where the file has some changes.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Les Mikesell Sent: den 5 november 2013 16:47 To: CentOS mailing list Subject: Re: [CentOS] [OT] Building a new backup server
If you have some time to experiment, look on the backuppc development list for the new alpha version. It is very different and does not need the hardlinks for pooling. I haven't tried it myself yet, but would (cautiously...) if I needed to set up a new system. It may eliminate the single-filesystem requirement and will definitely make it more feasible to rsync the whole archive to maintain an offsite copy. I think it may also chunk up large files so unchanged blocks can be pooled even where the file has some changes.
While the server isn't in production yet, I've nothing but.
I'll do that, thanks for the heads-up!
-- //Sorin
Les Mikesell wrote:
On Tue, Nov 5, 2013 at 7:04 AM, Sorin Srbu Sorin.Srbu@orgfarm.uu.se wrote:
Can e.g. BackupPC handle several file systems to backup to? I.e. comp1 through 10 should backup to /bak1, comp 11 through 20 to /bak2 and so on.
The main point of backuppc is that it hard-links all files with identical content to save space, so it needs to put everything on one filesystem. However, I'm getting pretty good performance running it
<snip> *shrug* So does rsync - we use hard links a *lot*, to keep 4-5 weeks of full nightly backups for a lot of servers.
mark
On Tue, Nov 5, 2013 at 8:32 AM, m.roth@5-cent.us wrote:
Les Mikesell wrote:
On Tue, Nov 5, 2013 at 7:04 AM, Sorin Srbu Sorin.Srbu@orgfarm.uu.se wrote:
Can e.g. BackupPC handle several file systems to backup to? I.e. comp1 through 10 should backup to /bak1, comp 11 through 20 to /bak2 and so on.
The main point of backuppc is that it hard-links all files with identical content to save space, so it needs to put everything on one filesystem. However, I'm getting pretty good performance running it
<snip> *shrug* So does rsync - we use hard links a *lot*, to keep 4-5 weeks of full nightly backups for a lot of servers.
No, rsync will only hardlink to instances of the same file in the same location from previous runs. Backuppc will link every file with identical contents even from different machines and locations. And it compresses them. So you could probably keep 2 to 10x the history online compared to anything else. Plus it has a nice web interface.
Les Mikesell wrote:
On Tue, Nov 5, 2013 at 8:32 AM, m.roth@5-cent.us wrote:
Les Mikesell wrote:
On Tue, Nov 5, 2013 at 7:04 AM, Sorin Srbu Sorin.Srbu@orgfarm.uu.se wrote:
Can e.g. BackupPC handle several file systems to backup to? I.e. comp1 through 10 should backup to /bak1, comp 11 through 20 to /bak2 and so on.
The main point of backuppc is that it hard-links all files with identical content to save space, so it needs to put everything on one filesystem. However, I'm getting pretty good performance running it
<snip> *shrug* So does rsync - we use hard links a *lot*, to keep 4-5 weeks of full nightly backups for a lot of servers.
No, rsync will only hardlink to instances of the same file in the same location from previous runs. Backuppc will link every file with
True. But we have a directory structure like .../servername |-> date1 date2
identical contents even from different machines and locations. And it compresses them. So you could probably keep 2 to 10x the history online compared to anything else. Plus it has a nice web interface.
-- Les Mikesell lesmikesell@gmail.com _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Les Mikesell wrote:
On Tue, Nov 5, 2013 at 8:32 AM, m.roth@5-cent.us wrote:
Les Mikesell wrote:
On Tue, Nov 5, 2013 at 7:04 AM, Sorin Srbu Sorin.Srbu@orgfarm.uu.se wrote:
Can e.g. BackupPC handle several file systems to backup to? I.e. comp1 through 10 should backup to /bak1, comp 11 through 20 to /bak2 and so on.
The main point of backuppc is that it hard-links all files with identical content to save space, so it needs to put everything on one filesystem. However, I'm getting pretty good performance running it
<snip> *shrug* So does rsync - we use hard links a *lot*, to keep 4-5 weeks of full nightly backups for a lot of servers.
No, rsync will only hardlink to instances of the same file in the same location from previous runs. Backuppc will link every file with
True. But we have a directory structure like .../servername |-> date1 date2
identical contents even from different machines and locations. And it compresses them. So you could probably keep 2 to 10x the history online compared to anything else. Plus it has a nice web interface.
-- Les Mikesell lesmikesell@gmail.com _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Les Mikesell wrote:
On Tue, Nov 5, 2013 at 8:32 AM, m.roth@5-cent.us wrote:
Les Mikesell wrote:
On Tue, Nov 5, 2013 at 7:04 AM, Sorin Srbu Sorin.Srbu@orgfarm.uu.se wrote:
Can e.g. BackupPC handle several file systems to backup to? I.e. comp1 through 10 should backup to /bak1, comp 11 through 20 to /bak2 and so on.
The main point of backuppc is that it hard-links all files with identical content to save space, so it needs to put everything on one filesystem. However, I'm getting pretty good performance running it
<snip> *shrug* So does rsync - we use hard links a *lot*, to keep 4-5 weeks of full nightly backups for a lot of servers.
No, rsync will only hardlink to instances of the same file in the same location from previous runs. Backuppc will link every file with
<snip> True. But we have a directory structure like .../servername | -> date-weeks-ago date-weeks-ago-1 ... date-yesterday |-> (symlink) latest and the symlink gets reset after the backup's done.
mark
On Tue, Nov 5, 2013 at 9:27 AM, m.roth@5-cent.us wrote:
No, rsync will only hardlink to instances of the same file in the same location from previous runs. Backuppc will link every file with
<snip> True. But we have a directory structure like .../servername | -> date-weeks-ago date-weeks-ago-1 ... date-yesterday |-> (symlink) latest and the symlink gets reset after the backup's done.
Have you tried backuppc? There are some tradeoffs because it makes an extra hardlink into a pool directory tree where the name is a hash of the content, but it takes care of all the other stuff for you and would let you store a much longer history, especially if there are duplicate copies of any of the files spread around.
Les Mikesell wrote:
On Tue, Nov 5, 2013 at 9:27 AM, m.roth@5-cent.us wrote:
No, rsync will only hardlink to instances of the same file in the same location from previous runs. Backuppc will link every file with
<snip> True. But we have a directory structure like .../servername | -> date-weeks-ago date-weeks-ago-1 ... date-yesterday |-> (symlink) latest and the symlink gets reset after the backup's done.
Have you tried backuppc? There are some tradeoffs because it makes an extra hardlink into a pool directory tree where the name is a hash of the content, but it takes care of all the other stuff for you and would let you store a much longer history, especially if there are duplicate copies of any of the files spread around.
I don't think that's going to happen. First, we have an in-house developed backup system that works just fine. Second, we *are* backup up something over a hundred servers and workstations to a few backup servers. Third, we are talking, in some cases, of terabytes....
mark
On Tue, Nov 5, 2013 at 9:52 AM, m.roth@5-cent.us wrote:
Have you tried backuppc? There are some tradeoffs because it makes an extra hardlink into a pool directory tree where the name is a hash of the content, but it takes care of all the other stuff for you and would let you store a much longer history, especially if there are duplicate copies of any of the files spread around.
I don't think that's going to happen. First, we have an in-house developed backup system that works just fine. Second, we *are* backup up something over a hundred servers and workstations to a few backup servers. Third, we are talking, in some cases, of terabytes....
I'm not quite at that scale in a single instance myself, but I'm fairly sure many users on the backuppc mail list are, so it is not necessarily a problem, although there are some tradeoffs with extra overhead for compression and the extra pool hardlink. In any case it is trivial to install and test with the package in EPEL. Even if it doesn't replace your server backup system you might find it useful to point at some workstations or windows boxes (it can use smb as well as rsync or tar to gather the files).
Les Mikesell wrote:
On Tue, Nov 5, 2013 at 9:52 AM, m.roth@5-cent.us wrote:
Have you tried backuppc? There are some tradeoffs because it makes an extra hardlink into a pool directory tree where the name is a hash of the content, but it takes care of all the other stuff for you and would let you store a much longer history, especially if there are duplicate copies of any of the files spread around.
I don't think that's going to happen. First, we have an in-house developed backup system that works just fine. Second, we *are* backup
up something
over a hundred servers and workstations to a few backup servers. Third, we are talking, in some cases, of terabytes....
I'm not quite at that scale in a single instance myself, but I'm fairly sure many users on the backuppc mail list are, so it is not necessarily a problem, although there are some tradeoffs with extra overhead for compression and the extra pool hardlink. In any case it is trivial to install and test with the package in EPEL. Even if it doesn't replace your server backup system you might find it useful to point at some workstations or windows boxes (it can use smb as well as rsync or tar to gather the files).
Heh. We don't do Windows. That's desktop support.... (As a side note, I work for a federal contractor at a non-defense site, so scale is, um, larger than many.)
mark
-- This email reflects my opinions, and not those of my employer, the US federal government, or the view out of my manager's window....
On Tue, Nov 5, 2013 at 10:48 AM, m.roth@5-cent.us wrote:
I'm not quite at that scale in a single instance myself, but I'm fairly sure many users on the backuppc mail list are, so it is not necessarily a problem, although there are some tradeoffs with extra overhead for compression and the extra pool hardlink. In any case it is trivial to install and test with the package in EPEL. Even if it doesn't replace your server backup system you might find it useful to point at some workstations or windows boxes (it can use smb as well as rsync or tar to gather the files).
Heh. We don't do Windows. That's desktop support.... (As a side note, I work for a federal contractor at a non-defense site, so scale is, um, larger than many.)
The scaling side of things just trades a little more CPU for compression and rsync-in-perl in return for vastly less disk consumption so it's not a sure bet either way in that respect. I think you've mentioned some subsequent off-line archiving scheme for your data sets that wouldn't mesh very well, though. But, everyone has lots of other stuff where backups would be nice to have and backuppc makes it trivial to have, say, daily copies of all of /etc from all machines going back months - or your own home directory. And it doesn't blow up if you point it at a bunch of home directories where developers have checked out copies of the same big source trees.
Les Mikesell wrote:
On Tue, Nov 5, 2013 at 10:48 AM, m.roth@5-cent.us wrote:
I'm not quite at that scale in a single instance myself, but I'm fairly sure many users on the backuppc mail list are, so it is not necessarily a problem, although there are some tradeoffs with extra overhead for compression and the extra pool hardlink. In any case it is trivial to install and test with the package in EPEL. Even if it doesn't replace your server backup system you might find it useful to point at some workstations or windows boxes (it can use smb as well as rsync or tar to gather the files).
Heh. We don't do Windows. That's desktop support.... (As a side note, I work for a federal contractor at a non-defense site, so scale is, um, larger than many.)
The scaling side of things just trades a little more CPU for compression and rsync-in-perl in return for vastly less disk consumption so it's not a sure bet either way in that respect. I think you've mentioned some subsequent off-line archiving scheme for your data sets that wouldn't mesh very well, though. But, everyone has lots of other stuff where backups would be nice to have and backuppc makes it trivial to have, say, daily copies of all of /etc from all machines going back months - or your own home directory. And it doesn't blow up if you point it at a bunch of home directories where developers have checked out copies of the same big source trees.
Still not sounding like we need it. We back up /etc from all our servers (except for the compute cluster nodes every night, and keep about 5 weeks. Home directories are 100% NFS-mounted from servers, and those are backed up every night onto a handful of backup servers, as are various project directories.
mark
On Tue, Nov 5, 2013 at 11:26 AM, m.roth@5-cent.us wrote:
Still not sounding like we need it. We back up /etc from all our servers (except for the compute cluster nodes every night, and keep about 5 weeks. Home directories are 100% NFS-mounted from servers, and those are backed up every night onto a handful of backup servers, as are various project directories.
Other ways work, of course - but typically they are much less efficient than backuppc which would only need storage for one compressed copy of each unique file in /etc regardless of the number of hosts or backups saved, are harder to manage, and often lack convenience features like being able to assign 'owners' to hosts that get email if a host is not backed up for any reason for a specified number of days or easily being able to control concurrency and the backup window for individual targets. And home-grown solutions generally need someone who understands the code for support. If that's you, I suppose that is job security...
Les Mikesell wrote:
On Tue, Nov 5, 2013 at 11:26 AM, m.roth@5-cent.us wrote:
Still not sounding like we need it. We back up /etc from all our servers (except for the compute cluster nodes every night, and keep about 5 weeks. Home directories are 100% NFS-mounted from servers, and those are backed up every night onto a handful of backup servers, as are various project directories.
Other ways work, of course - but typically they are much less efficient than backuppc which would only need storage for one compressed copy of each unique file in /etc regardless of the number of hosts or backups saved, are harder to manage, and often lack convenience features like being able to assign 'owners' to hosts that get email if a host is not backed up for any reason for a specified number of days or easily being able to control concurrency and the backup window for individual targets. And home-grown solutions generally need someone who understands the code for support. If that's you, I suppose that is job security...
Well, no - my manager, who's been here a bunch of years, wrote it years ago. And I'm not quite sure what you're saying - we have a centralized logging host, and the backup cron job on each machine emails its results to our admin mailing list.
mark
On Tue, Nov 5, 2013 at 12:10 PM, m.roth@5-cent.us wrote:
Well, no - my manager, who's been here a bunch of years, wrote it years ago. And I'm not quite sure what you're saying - we have a centralized logging host, and the backup cron job on each machine emails its results to our admin mailing list.
Not sure how it works there, but routine logs of things that generally work tend to get ignored. Backuppc presents a web page with the current status of all the hosts if you want the routine stuff. But more importantly it lets you designate a list of 'owners' for each target host to (a) allow access through the web interface and (b) get an email when backups have failed for a specified number of days. Emails that only come when there is a problem are generally noticed, unlike the ones that come every day and usually don't require any action.
On 11/5/2013 7:52 AM, m.roth@5-cent.us wrote:
I don't think that's going to happen. First, we have an in-house developed backup system that works just fine. Second, we*are* backup up something over a hundred servers and workstations to a few backup servers. Third, we are talking, in some cases, of terabytes....
my backuppc server is doing a couple dozen VM's and physical machines, linux+solaris+AIX+WindowsServer, I have backups going back to MARCH, and the whole pool is just a few terabytes, with an effective compression of like 50:1 due to the amount of duplication in those backups. the database server machines run a pre-backup script that either does a database dump, or a freeze (for instance, postgresql has pg_start_backup()), then backup whats right, and run a post-backup script (pg_stop_backup())
its worked remarkably well for me, using a fraction of the disk space I'd planned for (both primary and mirror backup servers have room for 36 3.5" SAS drives, although they are only stuffed with 20x3TB in raid6+0 configurations)
John R Pierce wrote:
On 11/5/2013 7:52 AM, m.roth@5-cent.us wrote:
I don't think that's going to happen. First, we have an in-house developed backup system that works just fine. Second, we*are* backup
up something
over a hundred servers and workstations to a few backup servers. Third, we are talking, in some cases, of terabytes....
my backuppc server is doing a couple dozen VM's and physical machines, linux+solaris+AIX+WindowsServer, I have backups going back to MARCH, and the whole pool is just a few terabytes, with an effective compression of like 50:1 due to the amount of duplication in those backups. the database server machines run a pre-backup script that either does a database dump, or a freeze (for instance, postgresql has pg_start_backup()), then backup whats right, and run a post-backup script (pg_stop_backup())
its worked remarkably well for me, using a fraction of the disk space I'd planned for (both primary and mirror backup servers have room for 36 3.5" SAS drives, although they are only stuffed with 20x3TB in raid6+0 configurations)
As I noted, we make sure rsync uses hard links... but we have a good number of individual people and projects with who *each* have a good number of terabytes of data and generated data. Some of our 2TB drives are over 90% full, and then there's the honkin' huge RAID, and at least one 14TB partition is over 9TB full....
mark
On Tue, Nov 5, 2013 at 1:28 PM, m.roth@5-cent.us wrote:
As I noted, we make sure rsync uses hard links... but we have a good number of individual people and projects with who *each* have a good number of terabytes of data and generated data. Some of our 2TB drives are over 90% full, and then there's the honkin' huge RAID, and at least one 14TB partition is over 9TB full....
If you have database dumps or big text files that aren't compressed, backuppc could be a big win. I think it is the only thing that can keep a compressed copy on the server side and work directly with a stock rsync and uncompressed files on the target hosts (and it can cache the block-checksums so it doesn't have to uncompress and recompute them every run). While it is 'just a perl script' it's not quite what you expect from simple scripting...
-
Dennis Jacobfeuerborn -
John R Pierce -
Keith Keller -
Les Mikesell -
m.roth＠5-cent.us -
SilverTip257 -
Sorin Srbu