We are migrating an LTO2 library off Veritas and NTBackup on a Windows Server to a Linux Server with either Bacula or Amanda. Originally, the Windows server used scripts to sync changes from various other Windows servers using Robocopy to a locally attached set of volumes, then ran the backup nightly (Full's were always done).
I now had hoped to rsync the Samba shares to a local replica on the Linux Server and then start the backup every night after it finishes. Would this be the most efficient way to do this, or are there better tools for this? I need an additional replica of the data, so using the Windows Bacula client doesn't help, and I must be able to only replicate the changes or I'll miss my backup window based on the volume of data.
Thanks for any suggestions, jlc
Joseph L. Casale wrote:
We are migrating an LTO2 library off Veritas and NTBackup on a Windows Server to a Linux Server with either Bacula or Amanda. Originally, the Windows server used scripts to sync changes from various other Windows servers using Robocopy to a locally attached set of volumes, then ran the backup nightly (Full's were always done).
If this sync script runs on the target machine, you could simply change it to use cygwin rsync instead with the destination on a linux box.
I now had hoped to rsync the Samba shares to a local replica on the Linux Server and then start the backup every night after it finishes.
Would this be the most efficient way to do this, or are there better tools for this?
Rsync directly from the target would be more efficient. I've never been able to make this work using cygwin sshd on the windows side to accept the connection and run rsync, but that could be a bug that is fixed now. It will work using rsync in daemon mode on the windows side, or initiating the connection from windows to a Linux target via ssh.
If you don't want the whole cygwin setup on your machines, there is a minimal install for rsync here: http://sourceforge.net/project/showfiles.php?group_id=34854
I need an additional replica of the data, so using the Windows Bacula client doesn't help,
I thought bacula could be configured to keep both an on-line and tape copy.
and I must be able to only replicate the changes or I'll miss my backup window based on the volume of data.
I use backuppc to keep an online and easily accessible history and amanda for tapes that go offsite (and I hope to never have to recover from the amanda tapes because it is much more difficult) and I just let them run independently. If your backup window is tight, you might have to run both (or bacula) against your rsync-mirrored snapshot instead of the actual target. This works well for data files but you'll need some extra contortions if you expect to do bare metal restores of the windows targets.
Rsync directly from the target would be more efficient. I've never been able to make this work using cygwin sshd on the windows side to accept the connection and run rsync, but that could be a bug that is fixed now. It will work using rsync in daemon mode on the windows side, or initiating the connection from windows to a Linux target via ssh.
If you don't want the whole cygwin setup on your machines, there is a minimal install for rsync here: http://sourceforge.net/project/showfiles.php?group_id=34854
I can't install cygwin or a daemon for that matter on the Windows Server, but I could execute a scheduled job to run a Widows port of rsync on the file server which I am not opposed to at all. My only problem with that is how do I then trigger the Bacula server to begin the dump to tape once the sync is complete?
I need an additional replica of the data, so using the Windows Bacula client
doesn't help,
I thought bacula could be configured to keep both an on-line and tape copy.
Possible, but I wouldn't have time for it to dump all the data if it simply accessed the CIFS share to copy all the data.
and I must be able to only replicate the changes or I'll miss my backup
window based on the volume of data.
I use backuppc to keep an online and easily accessible history and amanda for tapes that go offsite (and I hope to never have to recover from the amanda tapes because it is much more difficult) and I just let them run independently. If your backup window is tight, you might have to run both (or bacula) against your rsync-mirrored snapshot instead of the actual target. This works well for data files but you'll need some extra contortions if you expect to do bare metal restores of the windows targets.
Yea, bare metal is handled elsewhere, I just need file level protection. Not sure about Backuppc, it still doesn't help me trigger Bacula as soon as the sync is complete.
Thanks for the help! jlc
Joseph L. Casale wrote:
Rsync directly from the target would be more efficient. I've never been able to make this work using cygwin sshd on the windows side to accept the connection and run rsync, but that could be a bug that is fixed now. It will work using rsync in daemon mode on the windows side, or initiating the connection from windows to a Linux target via ssh.
If you don't want the whole cygwin setup on your machines, there is a minimal install for rsync here: http://sourceforge.net/project/showfiles.php?group_id=34854
I can't install cygwin or a daemon for that matter on the Windows Server, but I could execute a scheduled job to run a Widows port of rsync on the file server which I am not opposed to at all. My only problem with that is how do I then trigger the Bacula server to begin the dump to tape once the sync is complete?
I think I missed the distinction, but I'd expect it to be best to do the initial copy from the system that actually holds the data, and rsync will know to copy only the changed parts. If you run it with the source on a network drive it will still have to read entire files over the network to find the changed blocks.
I need an additional replica of the data, so using the Windows Bacula client
doesn't help,
I thought bacula could be configured to keep both an on-line and tape copy.
Possible, but I wouldn't have time for it to dump all the data if it simply accessed the CIFS share to copy all the data.
Why wouldn't you run the agent on the machine holding the data?
and I must be able to only replicate the changes or I'll miss my backup
window based on the volume of data.
I use backuppc to keep an online and easily accessible history and amanda for tapes that go offsite (and I hope to never have to recover from the amanda tapes because it is much more difficult) and I just let them run independently. If your backup window is tight, you might have to run both (or bacula) against your rsync-mirrored snapshot instead of the actual target. This works well for data files but you'll need some extra contortions if you expect to do bare metal restores of the windows targets.
Yea, bare metal is handled elsewhere, I just need file level protection. Not sure about Backuppc, it still doesn't help me trigger Bacula as soon as the sync is complete.
Does that have to be precisely timed? I'd just allow for worst-case timing before letting the backup server's scheduler start. Backuppc has a 'blackout window' for times you don't want backups running. With amanda you schedule when the runs start. Bacula probably has something similar.
I can't install cygwin or a daemon for that matter on the Windows Server, but I could execute a scheduled job to run a Widows port of rsync on the file server which I am not opposed to at all. My only problem with that is how do I then trigger the Bacula server to begin the dump to tape once the sync is complete?
Use a batch file and schedule that. :)
@echo off del /place/you/sync/to/done.txt robocopy/rsync files (whatever) echo > /place/you/sync/to/done.txt
Schedule the batch file for say 11:00pm
At some reasonable time schedule a bash script in Linux via cron:
#!/bin/bash while test ! -e done.txt; do sleep 1m done tar -czf /dev/tape /sync/path
Simple. :) The above script will wait until done.txt exists and then backs up the repository. If you're really lucky Bacula has a post backup script that can be run. I'm rather traditional and use tar. :)
Shawn
After a moment or two of digging: http://www.bacula.org/en/dev-manual/Tips_Suggestions.html#19274
Run Before Job = ShellScriptShownBelow
Shawn
On Sunday 16 March 2008, Shawn Everett wrote:
I can't install cygwin or a daemon for that matter on the Windows Server, but I could execute a scheduled job to run a Widows port of rsync on the file server which I am not opposed to at all. My only problem with that is how do I then trigger the Bacula server to begin the dump to tape once the sync is complete?
Use a batch file and schedule that. :)
@echo off del /place/you/sync/to/done.txt robocopy/rsync files (whatever) echo > /place/you/sync/to/done.txt
Schedule the batch file for say 11:00pm
At some reasonable time schedule a bash script in Linux via cron:
#!/bin/bash while test ! -e done.txt; do sleep 1m done tar -czf /dev/tape /sync/path
Simple. :) The above script will wait until done.txt exists and then backs up the repository. If you're really lucky Bacula has a post backup script that can be run. I'm rather traditional and use tar. :)
Shawn _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Sun, 2008-03-16 at 14:15 -0600, Joseph L. Casale wrote:
We are migrating an LTO2 library off Veritas and NTBackup on a Windows Server to a Linux Server with either Bacula or Amanda. Originally, the Windows server used scripts to sync changes from various other Windows servers using Robocopy to a locally attached set of volumes, then ran the backup nightly (Full’s were always done).
I now had hoped to rsync the Samba shares to a local replica on the Linux Server and then start the backup every night after it finishes. Would this be the most efficient way to do this, or are there better tools for this? I need an additional replica of the data, so using the Windows Bacula client doesn’t help, and I must be able to only replicate the changes or I’ll miss my backup window based on the volume of data.
---- I think bacula has the ability to execute a script at start and thus you should be able to tell the Windows Server to execute the same robocopy script.
Craig
-
Craig White -
Joseph L. Casale -
Les Mikesell -
Shawn Everett