I have a few CentOS 5.3 servers out at remote sites that connect over a satellite link (a painfully slow satellite link). There's a time period in the wee hours of the morning when the satellite provider doesn't stiff me for bandwidth, and when the link is generally idle, where I want to use rsync to back up files to my local server. I've got rsync working in in combination with cron and ssh, and that process works fine.
The problem is getting the initial copy of the files over to the local server so that rsync can just deal with files changed during the day (there tend to not be too many changes -- mostly office-type documents, few pictures, that sort of thing, so limited bandwidth is not a problem). If I run rsync now, however, it is trying to copy over all the files since none exist on the local servers, and by my calculations copying that 160gb worth of data should be done some time in mid-February, 2010.
I can make the 4-hour round trip out to the remote site with a USB hard drive and copy the files that way, but my concern is how to get the file structure, timestamps, permissions, etc copied exactly from the remote server > USB drive > local server so that rsync can run most efficiently (and not have to do a compare on every single file the first time it runs.)
So the question is, how do I copy the files from the remote server back to my local server in a way that will allow rsync, once the copy is complete, to start dealing just with the daily changes in the most efficient way possible?
Thanks!
Simple... :) Just rsync to the external drive, then rsync from the drive to your local server. :)
On Thu, Jul 16, 2009 at 4:01 PM, David Fixdavidf@mrxfx.com wrote:
Simple... :) Just rsync to the external drive, then rsync from the drive to your local server. :)
-- David Fix Senior Systems Administrator Mr. X Inc. 35 McCaul Street, Ste. #100 Toronto, ON M5T 1V7 T: (416) 595-6222, x 241 F: (416) 595-9122 E: davidf@mrxfx.com
----- Original Message ----- From: "Jeff Gregor" cpbecket@berkshire.net To: centos@centos.org Sent: Thursday, July 16, 2009 10:52:46 AM GMT -05:00 US/Canada Eastern Subject: [CentOS] Initial backup prior to daily rsync backups
I have a few CentOS 5.3 servers out at remote sites that connect over a satellite link (a painfully slow satellite link). There's a time period in the wee hours of the morning when the satellite provider doesn't stiff me for bandwidth, and when the link is generally idle, where I want to use rsync to back up files to my local server. I've got rsync working in in combination with cron and ssh, and that process works fine.
The problem is getting the initial copy of the files over to the local server so that rsync can just deal with files changed during the day (there tend to not be too many changes -- mostly office-type documents, few pictures, that sort of thing, so limited bandwidth is not a problem). If I run rsync now, however, it is trying to copy over all the files since none exist on the local servers, and by my calculations copying that 160gb worth of data should be done some time in mid-February, 2010.
I can make the 4-hour round trip out to the remote site with a USB hard drive and copy the files that way, but my concern is how to get the file structure, timestamps, permissions, etc copied exactly from the remote server > USB drive > local server so that rsync can run most efficiently (and not have to do a compare on every single file the first time it runs.)
So the question is, how do I copy the files from the remote server back to my local server in a way that will allow rsync, once the copy is complete, to start dealing just with the daily changes in the most efficient way possible?
Thanks! _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
You might want to read http://rdiff-backup.nongnu.org/docs.html
David Fix wrote:
Simple... :) Just rsync to the external drive, then rsync from the drive to your local server. :)
-- David Fix Senior Systems Administrator Mr. X Inc. 35 McCaul Street, Ste. #100 Toronto, ON M5T 1V7 T: (416) 595-6222, x 241 F: (416) 595-9122 E: davidf@mrxfx.com
----- Original Message ----- From: "Jeff Gregor" cpbecket@berkshire.net To: centos@centos.org Sent: Thursday, July 16, 2009 10:52:46 AM GMT -05:00 US/Canada Eastern Subject: [CentOS] Initial backup prior to daily rsync backups
I have a few CentOS 5.3 servers out at remote sites that connect over a satellite link (a painfully slow satellite link). There's a time period in the wee hours of the morning when the satellite provider doesn't stiff me for bandwidth, and when the link is generally idle, where I want to use rsync to back up files to my local server. I've got rsync working in in combination with cron and ssh, and that process works fine.
The problem is getting the initial copy of the files over to the local server so that rsync can just deal with files changed during the day (there tend to not be too many changes -- mostly office-type documents, few pictures, that sort of thing, so limited bandwidth is not a problem). If I run rsync now, however, it is trying to copy over all the files since none exist on the local servers, and by my calculations copying that 160gb worth of data should be done some time in mid-February, 2010.
I can make the 4-hour round trip out to the remote site with a USB hard drive and copy the files that way, but my concern is how to get the file structure, timestamps, permissions, etc copied exactly from the remote server > USB drive > local server so that rsync can run most efficiently (and not have to do a compare on every single file the first time it runs.)
So the question is, how do I copy the files from the remote server back to my local server in a way that will allow rsync, once the copy is complete, to start dealing just with the daily changes in the most efficient way possible?
Thanks! _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
In the words of Homer J. Simpson.... Do'h! What a simple and obvious solution.... I was probably over-thinking things again.
Thanks!
-
David Fix -
Jeff Gregor -
Lucian＠lastdot.org