Oops... I meant 600GB a day... sorry. Paul -----Original Message----- From: centos-mirror-bounces at centos.org [mailto:centos-mirror-bounces at centos.org] On Behalf Of Paul Stewart Sent: Wednesday, April 06, 2011 2:43 PM To: Mailing list for CentOS mirrors. Subject: Re: [CentOS-mirror] 5.6 is coming closer I'm curious as to what a busy mirror is... We are currently delivering about 60GB a day of CentOS files... does that put us at the bottom or near the top? ;) Paul -----Original Message----- From: centos-mirror-bounces at centos.org [mailto:centos-mirror-bounces at centos.org] On Behalf Of J.H. Sent: Wednesday, April 06, 2011 1:37 PM To: Mailing list for CentOS mirrors. Subject: Re: [CentOS-mirror] 5.6 is coming closer On 04/05/2011 01:52 PM, Karanbir Singh wrote: > On 04/05/2011 12:30 PM, Roelf Wichertjes wrote: >> >> Maybe a idea, >> Why not choose the least busy ones, >> Say there are 100 mirrors, 5 are busy and 5 almost unused >> Isn't it a better idea to let the 5 busy and the 5 unbusy pull from centos.org >> And have the other 90 pull from the 5 unbusy? >> That should even the load better. >>> On 04/05/2011 10:20 AM, Prof. P. Sriram wrote: >>>> Maybe it's been discussed before, but would it not be worthwhile to do a >>>> DNS based thing for this? We create a temporary rsync source domain >>> >>> Thats quite a lot of work, I'm more keen on having ACL's in place that >>> only allow some specific mirrors ( maybe the 100 busiest ones ) to pull >>> from centos.org; and have everyone else pull from them. > > > from 'busy' -i meant more like kernel.org / heanet.ie or mirrorservice.org > > - KB Tiering the mirror distribution is pretty common, and honestly makes things a *LOT* easier for everyone. I agree with the sentiments already stated, automation is what makes this all doable. Removing things like --delete from your mirrors, is just a PITA. Yes accidental upstream removals will happen, but if the mirror infrastructure is structured well it will propagate out and the fix will propagate out quickly. The way I've normally seen it is a small number (say 10) mirrors are allowed to pull form the master machines, and servers are then encouraged / forced to pull from those tier 1 mirrors. This means the tier 1's can pull more often from the upstream, and everyone else can make better use of the 1 & 10 gbps (and associated big hardware) links some of the bigger mirrors have. Personally I think it's worthwhile, and it's not too hard to implement. Keep in mind that the 'busier' servers (kernel.org at least) are in a better position (hardware / bandwidth) to support a greater number of people pulling from them. I would guess many of those "unused" mirrors may not be able to support the deluge you could potentially be pointing at them, this isn't universal but it's something to be aware of. - John 'Warthog9' Hawley _______________________________________________ CentOS-mirror mailing list CentOS-mirror at centos.org http://lists.centos.org/mailman/listinfo/centos-mirror _______________________________________________ CentOS-mirror mailing list CentOS-mirror at centos.org http://lists.centos.org/mailman/listinfo/centos-mirror