[CentOS] mirroring with lftp

[Ross Walker]

On Jun 27, 2011, at 6:26 PM, John R Pierce <pierce at hogranch.com> wrote:

> i've maintained a local centos repository at work using rsync, but it 
> seems the corp honchos have decided to block rsync at our firewall, plus 
> its never been 100% reliable, I'd get aborts on protocol errors 
> sometimes several times before pulling down a complete new distro update.
> 
> i'm trying to figure out how to do this with lftp, and its got me 
> somewhat stymied.
> 
> 
> I'm testing with --dry-run like...
> 
>     lftp -c 'open http://mirrors.kernel.org/centos/ && mirror -c -X 
> \*/ia64 -X \*/s390 -X \*/alpha -X \*/SRPMS --dry-run'
> 
> but my -X excludes don't seem to work (I don't want/need itanium, s390, 
> alpha, or SRPMs in my repo).
> 
> and, it doesn't seem to be aware of the symlinks from, for example, /5/ 
> to /5.6/  and seems to want to pull down two copies instead (I 
> understand that http doesn't make symlinks obvious).  Should I use 
> ftp:// instead of http:// or something?
> 
> 
> does someone have a script for maintaining a repo with lftp they'd like 
> to share?   what I saw on the wiki wasn't very helpful.

I have a wget repo mirroring script.

I'll dig it up and post it tonight when I get home.

-Ross