-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Dan Carl Sent: Sunday, February 24, 2008 18:54 To: CentOS mailing list Subject: Re: [CentOS] How to speed up Rsync transfers
This is exactly the situation I'm trying to avoid. Right now its less than 2GB new/edited images a day so the rsync backup finishes before the script runs again. But I can't take it for granted that this will always be the case. Any ideas would be appreciated. What do you mean by local staging?
Well, we use lock files.
I'd like the backup to run from 7pm to 7am and then if it didn't finish to resume again the next night.
We run it every 20 hours after compleation.
That way when nothing was added/edited on the weekends the backup can catch up.
From my atq:
#!/bin/sh # atrun uid=0 gid=0 # mail root 0 umask 22 interval=20\ hours; export interval limit=--bwlimit=64; export limit cd /var/spool/mirrors || { echo 'Execution directory inaccessible' >&2 exit 1 }
./centos.sh
From centos.sh
[root@host67 ~]# cd /var/spool/mirrors [root@host67 mirrors]# cat centos.sh lockfile=$0.lck
unset interactive unset verbose interval="20 hours" verbose="-q"
while [ "$1" != "" ] do case $1 in
--bwlimit=*) limit="--bwlimit=64" ;; --interactive) interactive=true ;; -t) interval="$2" shift ;; --prune) export prune="--delete-excluded" ;; -v) verbose="-v -v --progress" interactive=true ;; *) echo "Usage: $0" exit 1 esac
shift
done
if ( set -o noclobber; echo "$$" > "$lockfile") 2> /dev/null;
then
trap 'rm -f "$lockfile"; exit $?' INT TERM EXIT
# echo \ rsync -azH \ --delete \ rsync://mirrors.kernel.org/centos/ \ --exclude-from=centos.excludes \ $verbose \ $limit \ $prune \ centos/
rm -f "$lockfile" trap - INT TERM EXIT
if [ "$interactive" != "true" ] then
export limit export interval echo $0 | at now + 20 hours > /dev/null 2> /dev/null
fi
fi
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - - - Jason Pyeron PD Inc. http://www.pdinc.us - - Principal Consultant 10 West 24th Street #100 - - +1 (443) 269-1555 x333 Baltimore, Maryland 21218 - - - -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, purge the message from your system and notify the sender immediately. Any other use of the email by you is prohibited.