[CentOS-mirror] RFC on public centos mirrors

[Bob Pierce]

On Wed, 2008-06-11 at 22:18 +0200, florian at gruendler.net wrote:
> Well, I have released this for now. Anyone wants to comment or pick
> this up as a project?

Florian,

I think you've hit the nail on the head!

We are an ISP and going by the description on the wikipedia article that
you referenced, we would be a Tier 3 or just barely a Tier 2 since we do
peer with some relatively small networks. We currently have our CentOS
mirror bandwidth capped at 10 Mbps. The only thing stopping us from
completely removing that cap is the potential costs associated with
serving that bandwidth to users who aren't on our network.

This is a real issue. We would like to provide our clients the very
fastest, unlimited access to our mirror. However, we must limit the
bandwidth that our mirror presents to users who are off our network. One
idea that has rolled around in my head a bit is this:
1) Serve the mirror from 2 separate IPs - One with unlimited bandwidth
only accessible by our network ranges, the other with a bandwidth cap
accessible by anyone.
2) Have our DNS server return the unlimited IP to requests form our
clients and the limited IP to everyone else for our mirror name.

I like both of your ideas. I think they are both doable, and I think
they have much greater potential than my simple solution. I wonder if
the server freshness and hint list mentioned in your second scenario
might be something that could be synced from the master to the rest of
the servers along with the rest of the mirror files.

If we could build a solution around these ideas, we could have a really
nice CDN. The end user could see faster speeds as a result of the
Mirrors willingly removing most bandwidth caps. And, the mirrors should
be quite happy to remove those caps since there would be no risk of
charges from increased off network bandwidth.

Part of me does hope that in all of this we will still be able to
maintain some sort of QA of the files that are being transferred. I
personally like the current scenario because we have direct access to
the CentOS Master mirrors from our mirror. If we were to move away from
granting each mirror that access, we would have to have to provide some
sort of assurance that downstream mirrors will receive their updates in
a timely fashion and that the files have not been altered.

As far as picking this up as a project, I'd be willing to discuss some
of these ideas further and help with some the scripting that might be
necessary to pull this off.

Bob