[CentOS-mirror] Unresponsive or outdated nodes in mirror.centos.org

[Zenon Panoussis]

Lance Davis wrote:

> I think you may have misunderstood the fact that all of the servers that 
> are referenced as mirror.centos.org are managed and maintained by the 
> CentOS team - they are not external mirrors.

Indeed, looking at where whey are placed I assumed they were
externally managed. Yet, regardless who manages them, their
bad responses cause problems.

> Yes - at certain times those machines may not respond - usually due to 
> excessive load caused by a major (Ux) update - although hardware falure is 
> not unknown.

I have seen three different problems on numerous ocassions:

- 404 on header.info
- "X is not a gzipped file" (probably refering to some header,
  I can't remember exactly)
- .hdr files are present but the corresponding .rpm files
  are missing, causing yum to crash. This was a most common
  and widespread problem during the 4 U2 release.

Each of these requires a different remedy. My checker deals with
the first two. The last problem could be eliminated by putting
.rpm before .hdr in the rsync mirroring scripts.


[mirror checking script]

> Sounds like a good and useful idea - although dynamic dns updates 
> certainly dont require bind to be reloaded ... updates being kept in the 
> .jnl file 

Uhm, I have a lot of experience with ldap and none at all with
rndc, that's why I grabbed for ldap. But of course, rndc could
very well be used instead.

> Having said that - I havent experience of bind ddns 
> being used for rrdns ...

I have been using BIND with sbd_ldap as DDNS for a few months
now and it works just fine, although I've never had the
opportunity to test it under really heavy loads. As for the
round-robin part, I don't think BIND cares where it gets its
zone data from, as long as it gets it.

> Is the sdb_ldap part of CentOS or sometng that would need to be 
> installed/maintained separately ??

It's comes with FC4. I completely missed the fact, so when
I needed it I created a patched the CentOS 4 srpm instead of
just grabbing the FC4 srpm. The patch is at
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=163190

>>Is there any interest to implement any of this?

> Yes - definitely 

I that case, get the whole thing at
http://www.provocation.net/mirrorchecker.tar.bz2  and take a
look at it. I'm sure there is plenty of room for improvement,
but all the basics are there. Run the script and you will
find that two hosts in mirror.centos.org have a problem
right now.

> Especially if it may also allow us to extend the mirror. schema out to 
> external mirrors, and tie in with the stuff we are working on for 
> mirrorlist ...

That's one solution, surely better than what's there now.
All that's needed is a virtual mirror.centos.org server
on each mirror, with its document root pointing at the
directory that contains centos. This would solve the
problem of different paths to centos on each server.

On the other hand, yum is unable to recover if it meets
a bad mirror in a round-robined system. Using several
mirrors with different hostnames in sources can allow yum
to proceed elsewhere if the particular mirror.centos.org
it first goes to is broken.

Z