[CentOS] Yum / Up2date issues and mirror.centos.org

Tue Nov 29 11:54:47 UTC 2005
Johnny Hughes <mailing-lists at hughesjr.com>

There have been some complaints recently concerning this, so I thought
an explanation of the setup AND recommendations are in order.

First, I want to say that we need more donated servers with fast (at
least 100mbit) internet connections.  If you are making lots of money
using CentOS and are a hosting provider, please consider providing us a
donated server so we can better distribute CentOS.

Now on to the message :)

-----------------------------------------
1.  mirror.centos.org is a rrdns of several machines, so that we can
distribute the load for CentOS downloads across several servers.
Distributed load is the only way we can serve more than 1 TiB per day
for updates.  (That's right, 1 TiB per day, ~30TiB per month, double the
traffic of 3 months ago, 10x the traffic of a year ago).  

We had several servers totally maxed out serving updates during the 4.2
release cycle. Did I mention server donations :)

There are other ways to distribute the load other than rrdns ... like a
load balancer ... but these usually only work well with geographically
co-located machines.

Since all our machines for mirror.centos.org rrdns are donated and they
are disbursed throughout the US and EU.  They don't work well with the
typical load balancing.

2.  Since our machines are not geographically co-located, we need to
come up with another method to use multiple servers to distribute load.

We are currently developing an application that will use geoip (and
public mirrors that we verify are updated:

http://www.centos.org/modules/tinycontent/index.php?id=13

 along with the donated mirror.centos.org servers) to generate a
mirrorlist for each country identified by geoip.

This mirrorlist should then have good, fast and geographically accurate
servers for each person who does updates.

We hope to have this system functioning properly before the release of
CentOS-4.3 and CentOS-3.7.

If you use the optional fastestmirror plugin, you will even get the
fastest mirror in each repo from the mirrorlist.  The fastestmirror
plugin is currently under heavy development by the people at yum (in
which CentOS is helping) ... so it is going to change in the next couple
months, but in it's current form it works well for those not on proxy
servers to pick the fastest mirrors from a list.

http://lists.centos.org/pipermail/centos-devel/2005-November/000947.html

3.  Another common problem is transparent proxy servers and yum.  HTTP
mirrors and yum don't work really well via transparent proxy servers, so
if you are using one, your best bet is to pick an FTP mirror from our
public external server list (see mirror link above) instead of
mirror.centos.org.

-------------------------------

So, what can you do now if you have problems ... or just want to help.

First thing is to pick several close mirrors and not use
mirror.centos.org in your /etc/yum.repos.d/CentOS-Base.conf file.  There
is an example of a repo with more than 1 mirror in the fastestmirror
link above.  

If you have more than 1 mirror for each repo (I recommend that you
do) ... and if you don't have to use yum via a proxy server that blocks
outbound client connections ... then you should try the fastestmirror
plugin above to pick your best mirror from the list at yum run time.

We will continue to strive to deliver CentOS within our means as we
continue to grow by leaps and bounds, but we need the help and support
of our user community if we are going to be able to continue to provide
the same stellar services that we have all grown accustomed to.  That
means people who use CentOS will need to support it ... that is what the
Community ENTerprise Operating System (CentOS) is all about.

Thanks,
Johnny Hughes
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.centos.org/pipermail/centos/attachments/20051129/b47c7f67/attachment-0003.sig>