[CentOS-mirror] Rsync script problems

[Imre Gergely]

imho the exclude part is not working right, it's not matching to anything.
try a --dry-run first, without any excludes or includes, for example:

rsync -avH --progress --delete --dry-run us-msync.centos.org::CentOS
/var/storage/install/CentOS | tee /tmp/rsync.log

this way it shows you what would pull, but nothing gets downloaded, and you
get a nice logfile of all the files. after that, you can add some excludes,
and run it again _with_ the dry-run parameter, to see the exclude/includes
effect.

there is not much sens using -z, as all the rpms all compressed already, it
just puts an extra overhead on the whole process. so drop the -z, and run
with -H instead, as the mirror howto says.

consider using --include-from=FILE and --exclude-from=FILE parameters, it's
easier.

after reviewing the above rsync output logs, you can add for exclusion
something like this:

--exclude="s390/" --exclude="s390x/" --exclude="ppc/" --exclude="SRPMS/"

if you just want a specific subdirectory, eg. 4.0/, you could add that to the
rsync path, like this:

rsync us-msync.centos.or::CentOS/4.0/ /path/to/localdir/4.0/

then it wont process all the tree, only that subdirectory, it's faster.

hope that it helps somewhat.

Matt Shields wrote:
> Can someone see why when I run this script it tries to pull ALL
> versions instead of just 4 & 5?  Also, what would I add if I wanted to
> exclude SRPMS, alpha, ppc, s390, & s390x.
> 
> #!/bin/bash
> 
> RSYNC="/usr/bin/rsync"                  # file location of rsync
> FLAGS="-avz --bwlimit=512"              # rsync flags
> MIRROR="us-msync.centos.org::CentOS"    # rsync mirror
> REPO="/var/storage/install/CentOS"      # directory to store files
> INCLUDE="--include='timestamp.txt' --include='TIME'
> --include='RPM-GPG-KEY-centos4' --include='4.4/' --include='4.3/'
> --include='4.2/' --include='4.1/' --include='4/' --include='4.0/'
> --include='5/' --include='5.0/'"
> EXCLUDE="--exclude '/*'"
> 
> if [ -f "/var/run/centos-mirror.pid" ]; then
>   RUNPID=`cat /var/run/centos-mirror.pid`
>   if ps -p $RUNPID; then
>      echo "Mirror is already running..."
>      exit 1
>   else
>      echo "Mirror pid found but process dead, cleaning up"
>      rm -f /var/run/centos-mirror.pid
>   fi
> else
>   echo "No Mirror Process Detected"
> fi
> echo $$ > /var/run/centos-mirror.pid
> 
> echo Rsyncing...
> $RSYNC $FLAGS $EXCLUDE $INCLUDE $MIRROR $REPO
> rm -f /var/run/centos-mirror.pid
> exit 0
> 
> Thanks, Matt
> _______________________________________________
> CentOS-mirror mailing list
> CentOS-mirror at centos.org
> http://lists.centos.org/mailman/listinfo/centos-mirror

-- 
Imre Gergely
ISP Services
UPC Romania
28 Dostoievski St., 400424 Cluj-Napoca
www.upc.ro
gpg --keyserver subkeys.pgp.net --recv-keys 0x34525305