[CentOS-mirror] the world is still turning and so do CentOS mirrors in a way

florian at gruendler.net florian at gruendler.net
Thu Apr 7 13:30:17 EDT 2011


OMG, this is notorious!

Long time I have been listening but now I feel urged to talk and release
some frustrated thoughts in public.

1) To all the new kids on the block who constantly ask for getting "direct"
access to DVD-ISOs the second day after having their mirror set up

STOP asking! The content delivery network of CentOS must stay hierarchical
to be efficient. The master volunteers in the team know to identify which
networks/hosts have the necessary reputation, technical know-how and
capacity to serve other downstream mirrors in a certain are over rsync which
ultimately act as proxies to serve files to on-net/directly peered network
hosts. It would almost be recommendable that only very few master servers
were selected by the CentOS team to publishes new content on and push it one
at a time to such downstream servers or downstream servers are given a time
window to pull in order to avoid overload on the masters. It's like DNS root
servers, if everybody insisted to only get name resolution done by them, it
would get bad for everybody while in fact not a single tick more accurate.
In a digital world, a digital copy is as good as the original.


2) Backbone bandwidth is one thing and sustainable throughput are clearly
measurable resources as how many Megabytes can be pushed out per time unit

Are you guys aware that the data transfer I/O rate from the storage medium
is limited. Plain magnetic PATA or SATA disks @7200 RMP or such slow turning
disks in a RAID1 or RAID5 array can only read from the disk at a maximum
speed of about 1 Gbit/s (disk to buffer) and it gets MUCH worse for
concurrent requests when the tracks are remote for the spindle or the buffer
runs constantly full. SCSI/SAS disks @10k or 15k RMP and more redundant RAID
arrays with a good RAID controller do somewhat better as they are made to
concurrently seek and deliver bytes from the disk to the buffer. SSD disks
are a much greater thing, and so it their price. A 99$
"super-dedicated-flatrate-rootserver" machine rented from some provider and
having some spare diskspace and bandwidth are NOT suitable to be high up in
the hierarchy ever, much less can they take the load of thousands of CentOS
machines, initiating a yum update at any non-predictable moment in time.
Cheap machines from cheap datacenters tend to have "best effort" bandwidth
and the bw is commonly shaped if the hardware isn't maxed out before that.
The owners are the first ones to scream if their unimportant website served
from the same machine gets slow. Be welcome to donate your bandwidth at your
level but don't try to impress us with multi TenGE bandwidth and "Tier1"
blabla (the autonomous system to which your IP is connected might have that
OVERALL, BUT YOUR SERVERS DON'T) and the silly "unlimited" talk. It's more
honest to have a dedicated vintage machine (which 5 years back was a top
notch iron) on a port limited to SUSTAINABLE 10 Mbit/s. I don't want to
discourage anybody from bearing a share of this CDN project but I urge
everybody who may be concerned to refrain from seeing this as a boys game:
who has the bigger one, who can do it faster and who is cooler.

3) The ones who reported that 5.6 hasn't made it yet to their mirror and
increased the rsync frequency near to one Hertz

Stay calm! The world lived without CentOS 5.6 for a good time and it will
not stop to turn if it takes a couple days longer. For the ones who live
after the time=money paradigm, they have the money to buy RHEL or Oracle
Linux, both had their products on the shelves a couple month ago. Sleep one
or two nights over and 5.6 will be bright and wonderful on your mirror as
well.

4) My message to the people behind the CentOS CDN

You do a great job! Thank you for the patience you have with newbies (we all
were one day). Go for some known to be working software like Mirrormanager
as soon as you can because you lack the time to reinvent the wheel. And
please don't forget a few commercial aspects not so evident to people who
associate the Internet with their cable or dsl modem. Peering traffic is
free (not settled with cash), transit traffic varies extremely depending on
the supplier network. I am sure datacenter connecting networks were more
likely to kick in if they could control which ASNs get unrestricted access
and which ASN transit paths stay out. Such a GUI controlled tool, would make
it easy for every admin to change mirror preferences at their own discretion
without justifying in public why they had to close the mirror down (probably
because they couldn't afford the 99$ anymore) and wondering why they are not
ranked No.1 (because they think everything is like SETI at home). 


I feel so much better now that I have told

Good evening, good night, or good morning to everyone out there listening
this channel ;-)

Florian


> -----Ursprüngliche Nachricht-----
> Von: centos-mirror-bounces at centos.org [mailto:centos-mirror-
> bounces at centos.org] Im Auftrag von Roelf Wichertjes
> Gesendet: Donnerstag, 7. April 2011 16:59
> An: Mailing list for CentOS mirrors.
> Betreff: Re: [CentOS-mirror] 5.6 is coming closer
> 
> I've got the bandwith and speed for it,
> I once pulled all 7 install iso's (at the same time) from a high ranked
> mirror, it took me 16 min.
> Then did the same but now used my mirror, it took me 15 min.
> My bandwith is enough.
> Ps. I know what a busy server is.
> I get a bonus from my host if i have a lot of traffic.
> I forgot the tier of the site and i removed the --delete flag.
> (my mail software is broken and the webmail i'm using can't handle the
> e-mail format of the list-admin, so i can't read the mail in which he
> told me the tier).
> 
> I'm ready for it!
> 
> 
> --------------------
> Roelf Software
> software.roelf.org
> www.roelf.org
> --------------------
> 
> Op 6 apr. 2011 om 20:42 heeft "Paul Stewart"
> <pstewart at nexicomgroup.net> het volgende geschreven:
> 
> > I'm curious as to what a busy mirror is...
> >
> > We are currently delivering about 60GB a day of CentOS files... does
> > that put us at the bottom or near the top? ;)
> >
> > Paul
> >
> >
> >
> > -----Original Message-----
> > From: centos-mirror-bounces at centos.org
> > [mailto:centos-mirror-bounces at centos.org] On Behalf Of J.H.
> > Sent: Wednesday, April 06, 2011 1:37 PM
> > To: Mailing list for CentOS mirrors.
> > Subject: Re: [CentOS-mirror] 5.6 is coming closer
> >
> > On 04/05/2011 01:52 PM, Karanbir Singh wrote:
> >> On 04/05/2011 12:30 PM, Roelf Wichertjes wrote:
> >>>
> >>> Maybe a idea,
> >>> Why not choose the least busy ones,
> >>> Say there are 100 mirrors, 5 are busy and 5 almost unused
> >>> Isn't it a better idea to let the 5 busy and the 5 unbusy pull from
> > centos.org
> >>> And have the other 90 pull from the 5 unbusy?
> >>> That should even the load better.
> >>>> On 04/05/2011 10:20 AM, Prof. P. Sriram wrote:
> >>>>> Maybe it's been discussed before, but would it not be worthwhile
> to
> > do a
> >>>>> DNS based thing for this? We create a temporary rsync source
> domain
> >>>>
> >>>> Thats quite a lot of work, I'm more keen on having ACL's in place
> > that
> >>>> only allow some specific mirrors ( maybe the 100 busiest ones ) to
> > pull
> >>>> from centos.org; and have everyone else pull from them.
> >>
> >>
> >> from 'busy' -i meant more like kernel.org / heanet.ie or
> > mirrorservice.org
> >>
> >> - KB
> >
> > Tiering the mirror distribution is pretty common, and honestly makes
> > things a *LOT* easier for everyone.  I agree with the sentiments
> already
> > stated, automation is what makes this all doable.  Removing things
> like
> > --delete from your mirrors, is just a PITA.  Yes accidental upstream
> > removals will happen, but if the mirror infrastructure is structured
> > well it will propagate out and the fix will propagate out quickly.
> >
> > The way I've normally seen it is a small number (say 10) mirrors are
> > allowed to pull form the master machines, and servers are then
> > encouraged / forced to pull from those tier 1 mirrors.  This means
> the
> > tier 1's can pull more often from the upstream, and everyone else can
> > make better use of the 1 & 10 gbps (and associated big hardware)
> links
> > some of the bigger mirrors have.  Personally I think it's worthwhile,
> > and it's not too hard to implement.
> >
> > Keep in mind that the 'busier' servers (kernel.org at least) are in a
> > better position (hardware / bandwidth) to support a greater number of
> > people pulling from them.  I would guess many of those "unused"
> mirrors
> > may not be able to support the deluge you could potentially be
> pointing
> > at them, this isn't universal but it's something to be aware of.
> >
> > - John 'Warthog9' Hawley
> > _______________________________________________
> > CentOS-mirror mailing list
> > CentOS-mirror at centos.org
> > http://lists.centos.org/mailman/listinfo/centos-mirror
> > _______________________________________________
> > CentOS-mirror mailing list
> > CentOS-mirror at centos.org
> > http://lists.centos.org/mailman/listinfo/centos-mirror
> _______________________________________________
> CentOS-mirror mailing list
> CentOS-mirror at centos.org
> http://lists.centos.org/mailman/listinfo/centos-mirror



More information about the CentOS-mirror mailing list