Does anyone have any ideas on how to steer mirror traffic to your own mirrors without logging into the boxes? We have two public mirrors (one in each of our DCs) and would love to reduce the load on external mirrors, but manually changing the settings of tons of production boxes (many of which we don't even have access to) is not really an option. Most/all of the boxes have the default yum config (fastestmirror plugin).
Any ideas are welcome.
-- Randy M. www.FastServ.com
Add a mirror priority list yum plugin :)
Put one mirror per line the top link would be #1 the second mirror url would be the second and so on. On Sep 25, 2010 3:58 PM, "Randy McAnally" rsm@fast-serv.com wrote:
Does anyone have any ideas on how to steer mirror traffic to your own
mirrors
without logging into the boxes? We have two public mirrors (one in each of our DCs) and would love to reduce the load on external mirrors, but
manually
changing the settings of tons of production boxes (many of which we don't
even
have access to) is not really an option. Most/all of the boxes have the default yum config (fastestmirror plugin).
Any ideas are welcome.
-- Randy M. www.FastServ.com
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On Sun, Sep 26, 2010 at 2:10 AM, Chris talkquazi@gmail.com wrote:
Add a mirror priority list yum plugin :)
Put one mirror per line the top link would be #1 the second mirror url would be the second and so on.
His DCs have thousand of box, where he even don't have any access to. So modifying the yum plugin will not work.
On Sep 25, 2010 3:58 PM, "Randy McAnally" rsm@fast-serv.com wrote:
Does anyone have any ideas on how to steer mirror traffic to your own mirrors without logging into the boxes? We have two public mirrors (one in each of our DCs) and would love to reduce the load on external mirrors, but manually changing the settings of tons of production boxes (many of which we don't even have access to) is not really an option. Most/all of the boxes have the default yum config (fastestmirror plugin).
Any ideas are welcome.
-- Randy M. www.FastServ.com
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
An basic Idea. As you are a company hosting thousand of CentOS box, I think a redirector (CISCO's WCCP & Squid) might help.
Other then that, you must have traffic redirection system, which might help, though I am not sure what kind of redirector your DCs have.
Regards Ahamed Bauani Freelance ICT Consultant Working on Deploying IPv6 in Bangladesh.
On Sun, Sep 26, 2010 at 1:58 AM, Randy McAnally rsm@fast-serv.com wrote:
Does anyone have any ideas on how to steer mirror traffic to your own mirrors without logging into the boxes? We have two public mirrors (one in each of our DCs) and would love to reduce the load on external mirrors, but manually changing the settings of tons of production boxes (many of which we don't even have access to) is not really an option. Most/all of the boxes have the default yum config (fastestmirror plugin).
Any ideas are welcome.
-- Randy M. www.FastServ.com
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
Am 25.09.10 21:58, schrieb Randy McAnally:
Does anyone have any ideas on how to steer mirror traffic to your own mirrors without logging into the boxes? We have two public mirrors (one in each of our DCs) and would love to reduce the load on external mirrors, but manually changing the settings of tons of production boxes (many of which we don't even have access to) is not really an option. Most/all of the boxes have the default yum config (fastestmirror plugin).
a) *BE* the fastest mirror :) (which does not work in the US, as mirrorlist only returns 10 random mirrors from the country you are in).
b) DNS-Hijack mirrorlist.centos.org (I would not do that without telling customers about that) and return your mirror always
Traffic redirection to mirrorlist.centos.org (and then returning your own mirror) might also work.
No real elegant solution there.
Ralph
I have never seen our own mirrors show up in more than 1 of the 4 repos at any given time. In the case of DNS/Proxy hijacking, does this mean we would have to return ONLY our mirrors, or do I just make sure they are part of the 10 random mirrors?
-- Randy M.
---------- Original Message ----------- From: Ralph Angenendt ralph.angenendt@gmail.com To: centos-mirror@centos.org Sent: Sun, 26 Sep 2010 17:51:49 +0200 Subject: Re: [CentOS-mirror] Ideas on steering yum to local mirrors
Am 25.09.10 21:58, schrieb Randy McAnally:
Does anyone have any ideas on how to steer mirror traffic to your own mirrors without logging into the boxes? We have two public mirrors (one in each of our DCs) and would love to reduce the load on external mirrors, but manually changing the settings of tons of production boxes (many of which we don't even have access to) is not really an option. Most/all of the boxes have the default yum config (fastestmirror plugin).
a) *BE* the fastest mirror :) (which does not work in the US, as mirrorlist only returns 10 random mirrors from the country you are in).
b) DNS-Hijack mirrorlist.centos.org (I would not do that without telling customers about that) and return your mirror always
Traffic redirection to mirrorlist.centos.org (and then returning your own mirror) might also work.
No real elegant solution there.
Ralph _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
------- End of Original Message -------
Am 26.09.10 20:52, schrieb Randy McAnally:
I have never seen our own mirrors show up in more than 1 of the 4 repos at any given time. In the case of DNS/Proxy hijacking, does this mean we would have to return ONLY our mirrors, or do I just make sure they are part of the 10 random mirrors?
Note that I am not officially recommending DNS hijacking, as I think that it is rather nasty towards your users - and only really works if all your users use your name servers for DNS resolution.
Having said that: I'd just return your mirrors, if you can handle the load.
There are other mirroring systems which can also act on a BGP or simple CIDR level, maybe it is time to take a look at those.
Ralph
On 26/09/10 23:42, Ralph Angenendt wrote:
Am 26.09.10 20:52, schrieb Randy McAnally:
I have never seen our own mirrors show up in more than 1 of the 4 repos at any given time. In the case of DNS/Proxy hijacking, does this mean we would have to return ONLY our mirrors, or do I just make sure they are part of the 10 random mirrors?
Note that I am not officially recommending DNS hijacking, as I think that it is rather nasty towards your users - and only really works if all your users use your name servers for DNS resolution.
Having said that: I'd just return your mirrors, if you can handle the load.
There are other mirroring systems which can also act on a BGP or simple CIDR level, maybe it is time to take a look at those.
Ralph
If there are systems capable of using BGP then why is everyone using Geoip?...
That was going to be my next question. How far out is the possibility of choosing mirror(s) (at least part of the random 10) from the same ASN as the clients? I'd be willing to help/contribute to impliment this on the mirror server side... last thing I want to do is proxy/hijack the mirror server traffic.
-- Randy M.
---------- Original Message ----------- From: Lucian lucian@chml.ro To: centos-mirror@centos.org Sent: Sun, 26 Sep 2010 23:53:54 +0100 Subject: Re: [CentOS-mirror] Ideas on steering yum to local mirrors
On 26/09/10 23:42, Ralph Angenendt wrote:
Am 26.09.10 20:52, schrieb Randy McAnally:
I have never seen our own mirrors show up in more than 1 of the 4 repos
at any
given time. In the case of DNS/Proxy hijacking, does this mean we would have to return ONLY our mirrors, or do I just make sure they are part of the 10 random mirrors?
Note that I am not officially recommending DNS hijacking, as I think that it is rather nasty towards your users - and only really works if all your users use your name servers for DNS resolution.
Having said that: I'd just return your mirrors, if you can handle the load.
There are other mirroring systems which can also act on a BGP or simple CIDR level, maybe it is time to take a look at those.
Ralph
If there are systems capable of using BGP then why is everyone using Geoip?... _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
------- End of Original Message -------
Am 27.09.10 01:48, schrieb Randy McAnally:
That was going to be my next question. How far out is the possibility of choosing mirror(s) (at least part of the random 10) from the same ASN as the clients? I'd be willing to help/contribute to impliment this on the mirror server side... last thing I want to do is proxy/hijack the mirror server traffic.
As Adrian said: MirrorManager and MirrorBrain both can do that, I wanted to look at MirrorBrain, Adrian has experience with MirrorManager.
Should we have an IRC meeting in the week of 11th to 17th of October (or a week after that)? I clearly cannot make it on the 11th, though.
Is there anyone else who wants to join?
Which time frame would be the best for that?
Regards,
Ralph
On Tue, Sep 28, 2010 at 07:17:54PM +0200, Ralph Angenendt wrote:
Am 27.09.10 01:48, schrieb Randy McAnally:
That was going to be my next question. How far out is the possibility of choosing mirror(s) (at least part of the random 10) from the same ASN as the clients? I'd be willing to help/contribute to impliment this on the mirror server side... last thing I want to do is proxy/hijack the mirror server traffic.
As Adrian said: MirrorManager and MirrorBrain both can do that, I wanted to look at MirrorBrain, Adrian has experience with MirrorManager.
Should we have an IRC meeting in the week of 11th to 17th of October (or a week after that)? I clearly cannot make it on the 11th, though.
I would like to join. The week sounds good. I probably could not make it on the 13th of October, the other days sound good.
Adrian
Hello All
If the CentOS Master Mirror want to announce and meeting, the date can't be fixed by getting input from member of list.
We, the maintainer of CentOS mirror around is a part of CentOS family and we are human being. Everyone has their own work schedule. Like in 6th October, Internet Society [ISOC] has conference in Bangladesh for first time and I wish to join the event. 12th October is my birthday etc etc.
So it would be better fix a date and announce it via list. If most people don't agree to join on said date, the date of event can change 1 to 3 days plus and minus.
From my point of view, it seems if the GeoIP database of CentOS is up
to date, currently problem might solve in most case. Among other distribution, CentOS mirror maintaining system is the most easy system in current scenario.
If we want to move from current system, then ASN based system to determine closest mirror might work.Though It also have pro & cons. So my suggestion is to work with current system and find out where is the problem so that the problem can be fixed.
Best Wishes Ahamed Bauani
On Wed, Sep 29, 2010 at 5:18 PM, Adrian Reber adrian@lisas.de wrote:
On Tue, Sep 28, 2010 at 07:17:54PM +0200, Ralph Angenendt wrote:
Am 27.09.10 01:48, schrieb Randy McAnally:
That was going to be my next question. How far out is the possibility of choosing mirror(s) (at least part of the random 10) from the same ASN as the clients? I'd be willing to help/contribute to impliment this on the mirror server side... last thing I want to do is proxy/hijack the mirror server traffic.
As Adrian said: MirrorManager and MirrorBrain both can do that, I wanted to look at MirrorBrain, Adrian has experience with MirrorManager.
Should we have an IRC meeting in the week of 11th to 17th of October (or a week after that)? I clearly cannot make it on the 11th, though.
I would like to join. The week sounds good. I probably could not make it on the 13th of October, the other days sound good.
Adrian _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On Wed, Sep 29, 2010 at 1:40 PM, Bangladeshi CentOS Mirror Maintainer [BD-SERVERS.NET] centos-org@bauani.org wrote:
Hello All
If the CentOS Master Mirror want to announce and meeting, the date can't be fixed by getting input from member of list.
I am not sure about that. Getting to know the timeframe which people have and if they are interested at all helps a lot when determining a date for such an event.
We, the maintainer of CentOS mirror around is a part of CentOS family and we are human being. Everyone has their own work schedule. Like in 6th October, Internet Society [ISOC] has conference in Bangladesh for first time and I wish to join the event. 12th October is my birthday etc etc.
So it is neither the 11th, nor the 12th nor the 13th :)
I guess weekend is out of the question anyway?
I then propose Monday, October 18th(!) at 20:00 UTC,
So it would be better fix a date and announce it via list. If most people don't agree to join on said date, the date of event can change 1 to 3 days plus and minus.
See above.
From my point of view, it seems if the GeoIP database of CentOS is up
to date, currently problem might solve in most case.
It is up to date (well, from now on it will, at least). One problem is that the data sometimes really is off, so machines in the UK are flagged as being in the US or Canada and things like that.
Second: It doesn't help in cases when you'd like all the machines in your DC to hit the mirror which also is in that DC. If you aren't the admin of all of those machines, using the same AS or /16 would be great. And that is what our mirroring system cannot do.
Among other distribution, CentOS mirror maintaining system is the most easy system in current scenario.
Not really :)
If we want to move from current system, then ASN based system to determine closest mirror might work.Though It also have pro & cons. So my suggestion is to work with current system and find out where is the problem so that the problem can be fixed.
The problem is that we can *only* do GeoIP. The granularity of that isn't fine enough at times (see above or other examples in this thread).
Ralph
So it is neither the 11th, nor the 12th nor the 13th :)
I guess weekend is out of the question anyway?
I then propose Monday, October 18th(!) at 20:00 UTC,
I would be interested in joining the discussion, and that day/time so far works for me. To make it a little easier to schedule, here is a link to that day/time with local time calculations around the world: http://bit.ly/cssyNN
-Jonathan
---------- Original Message ----------- From: Ralph Angenendt ralph.angenendt@gmail.com To: "Mailing list for CentOS mirrors." centos-mirror@centos.org Sent: Wed, 29 Sep 2010 15:05:53 +0200 Subject: Re: [CentOS-mirror] Ideas on steering yum to local mirrors
If we want to move from current system, then ASN based system to determine closest mirror might work.Though It also have pro & cons. So my suggestion is to work with current system and find out where is the problem so that the problem can be fixed.
The problem is that we can *only* do GeoIP. The granularity of that isn't fine enough at times (see above or other examples in this thread).
Maybe we return more than 10 mirrors in certain cases where many mirrors are close by?
The problem is that, by randomly choosing 10 mirrors within X distance our own mirrors are not always returned.
Many times mirrors 1000 miles away will win election because our own are not in the list.
I can be pretty sure that if our mirrors are anywhere in the list, yum would elect them.
Am 29.09.10 18:26, schrieb Randy McAnally:
From: Ralph Angenendt ralph.angenendt@gmail.com
The problem is that we can *only* do GeoIP. The granularity of that isn't fine enough at times (see above or other examples in this thread).
Maybe we return more than 10 mirrors in certain cases where many mirrors are close by?
The problem is that, by randomly choosing 10 mirrors within X distance our own mirrors are not always returned.
Even if we return more mirrors (which shouldn't be a problem), the fastest mirror plugin by default does not run each time. So if you cannot control the configuration of those hosts, that does not really help you much, although chances are greater that you are winning.
I am sure that a planned migration to a different mirror tool would be better. But yes, that takes time.
Ralph
The ASN usage is a novel idea, and would cater to lots of issues aobut nearest mirrors discussed in the past.
The date and time mentioned is good for me personally, and as a collaborative effort, that is the best that can be done. Those who could not join, unfortunate, those who can, GREAT!
Looking forward to chatting with all of you on 18/Oct
Regards HASSAN
On Thu, Sep 30, 2010 at 02:02, Ralph Angenendt ralph.angenendt@gmail.comwrote:
Am 29.09.10 18:26, schrieb Randy McAnally:
From: Ralph Angenendt ralph.angenendt@gmail.com
The problem is that we can *only* do GeoIP. The granularity of that isn't fine enough at times (see above or other examples in this thread).
Maybe we return more than 10 mirrors in certain cases where many mirrors
are
close by?
The problem is that, by randomly choosing 10 mirrors within X distance
our own
mirrors are not always returned.
Even if we return more mirrors (which shouldn't be a problem), the fastest mirror plugin by default does not run each time. So if you cannot control the configuration of those hosts, that does not really help you much, although chances are greater that you are winning.
I am sure that a planned migration to a different mirror tool would be better. But yes, that takes time.
Ralph _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On Wed, Sep 29, 2010 at 1:18 PM, Adrian Reber adrian@lisas.de wrote:
On Tue, Sep 28, 2010 at 07:17:54PM +0200, Ralph Angenendt wrote:
Should we have an IRC meeting in the week of 11th to 17th of October (or a week after that)? I clearly cannot make it on the 11th, though.
I would like to join. The week sounds good. I probably could not make it on the 13th of October, the other days sound good.
Monday 18th at 20:00 UTC? I know that that again is a week later, but I don't really have time to prepare something for the 12th, looking at my schedule.
I do want to do some homework before a meeting like that but am away for the complete next week.
Ralph
Hi,
well, Fedora has been having the Mirror-Manager for quite some while now where you can select which countries your mirror would want to serve and also give your networks (lately: your AS-numbers) and would always be ranked first in the mirror-lists.
I've tried to discuss that with CentOS-folks in the past, but they seemed to prefer the current system which "just works" for them :-(
Sorry to hear that, while we're wasting bandwidth :-(
Kind regards, Stefan Neufeind
On 09/27/2010 01:48 AM, Randy McAnally wrote:
That was going to be my next question. How far out is the possibility of choosing mirror(s) (at least part of the random 10) from the same ASN as the clients? I'd be willing to help/contribute to impliment this on the mirror server side... last thing I want to do is proxy/hijack the mirror server traffic.
-- Randy M.
---------- Original Message ----------- From: Lucian lucian@chml.ro To: centos-mirror@centos.org Sent: Sun, 26 Sep 2010 23:53:54 +0100 Subject: Re: [CentOS-mirror] Ideas on steering yum to local mirrors
On 26/09/10 23:42, Ralph Angenendt wrote:
Am 26.09.10 20:52, schrieb Randy McAnally:
I have never seen our own mirrors show up in more than 1 of the 4 repos
at any
given time. In the case of DNS/Proxy hijacking, does this mean we would have to return ONLY our mirrors, or do I just make sure they are part of the 10 random mirrors?
Note that I am not officially recommending DNS hijacking, as I think that it is rather nasty towards your users - and only really works if all your users use your name servers for DNS resolution.
Having said that: I'd just return your mirrors, if you can handle the load.
There are other mirroring systems which can also act on a BGP or simple CIDR level, maybe it is time to take a look at those.
Ralph
If there are systems capable of using BGP then why is everyone using Geoip?...
On Tue, 28 Sep 2010, SpeedPartner GmbH wrote:
well, Fedora has been having the Mirror-Manager for quite some while now where you can select which countries your mirror would want to serve and also give your networks (lately: your AS-numbers) and would always be ranked first in the mirror-lists.
When I spoke with the Mirror-Manager developer at OLS a while back, I pointed out some possible security holes, and frankly had not followed it as the need to do so was unclear, and the interface for reporting was trac, rather than bugzilla,
I've tried to discuss that with CentOS-folks in the past, but they seemed to prefer the current system which "just works" for them :-(
Sorry to hear that, while we're wasting bandwidth :-(
You are not helpless here, and it may well not be the right solution (from a CentOS risk management) to risk breaking update settings for a huge deployed base in a major release
No-one is preventing you from running a local mirror, and dropping in local yum configs to point to them
No-one is preventing you from having a cacheing proxy server
There are so many hours in the day -- there is no constraint that you wait for Ralph or anyone else, before you block out an implementation and propose it for testing. If you have data needs in doing the setup, please ask here.
-- Russ herrold
On Tue, Sep 28, 2010 at 10:28 PM, SpeedPartner GmbH mirror@speedpartner.de wrote
well, Fedora has been having the Mirror-Manager for quite some while now where you can select which countries your mirror would want to serve and also give your networks (lately: your AS-numbers) and would always be ranked first in the mirror-lists.
I've tried to discuss that with CentOS-folks in the past, but they seemed to prefer the current system which "just works" for them :-(
Sorry to hear that, while we're wasting bandwidth :-(
I just made an offer for a meeting via IRC.
Monday, October 18th, 20:00 UTC.
And keep in mind please that that means to move a *large* installed set of machines over to a different mirror system. And that system is run by a crowd which doesn't really like changes in their eco system.
So the more people can chime in with their knowledge ...
One thing I know is that we won't begin restructuring *before* 6 is out. Just to give an unknown timeframe >:)
Cheers,
Ralph
On Mon, Sep 27, 2010 at 12:42:06AM +0200, Ralph Angenendt wrote:
Am 26.09.10 20:52, schrieb Randy McAnally:
I have never seen our own mirrors show up in more than 1 of the 4 repos at any given time. In the case of DNS/Proxy hijacking, does this mean we would have to return ONLY our mirrors, or do I just make sure they are part of the 10 random mirrors?
Note that I am not officially recommending DNS hijacking, as I think that it is rather nasty towards your users - and only really works if all your users use your name servers for DNS resolution.
Having said that: I'd just return your mirrors, if you can handle the load.
There are other mirroring systems which can also act on a BGP or simple CIDR level, maybe it is time to take a look at those.
I know of cdn.debian.net which knows how to do ASN lookups since the last weekend. Mirrorbrain which runs behind the mirrorlist for opensuse and openoffice.org also uses ASNs as well as Fedora's MirrorManager which I have already set up for RPM Fusion. IF there is interest I can help set up a MirrorManager instance.
Adrian
Am 27.09.10 12:39, schrieb Adrian Reber:
I know of cdn.debian.net which knows how to do ASN lookups since the last weekend. Mirrorbrain which runs behind the mirrorlist for opensuse and openoffice.org also uses ASNs as well as Fedora's MirrorManager which I have already set up for RPM Fusion. IF there is interest I can help set up a MirrorManager instance.
MirrorBrain is on my ToDo List, maybe we can get together and look at pro/cons for those systems. Peter from mirrorbrain should also be reading here (are you?). I am off for a week though from friday on, but this is something where I wanted to wait for 6 to be published and tackle it after that.
Ralph
On 09/27/2010 11:39 AM, Adrian Reber wrote:
I know of cdn.debian.net which knows how to do ASN lookups since the last weekend. Mirrorbrain which runs behind the mirrorlist for opensuse and openoffice.org also uses ASNs as well as Fedora's MirrorManager which I have already set up for RPM Fusion. IF there is interest I can help set up a MirrorManager instance.
We looked at going down the route of a CDN, way back in 2006, and I've been keeping my eyes on that track since then. The reason why we didnt is that it would involve non-trivial job, monitoring and process control tasks to run on non centos.org machines; and that's not the sort of thing that large mirror providers want to look at.
Also given the very nature of how .centos.org is built up ( donated machines, with a fairly high rate of churn ) its hard to get our own gbp announces etc in place. We could potentially look at this again, maybe come up with a list of challenges and see if and how we might be able to solve them.
- KB
[resending, after realizing that I was subscribed with an old address]
On Sun, Oct 03, 2010 at 02:13:32 +0100, Karanbir Singh wrote:
On 09/27/2010 11:39 AM, Adrian Reber wrote:
I know of cdn.debian.net which knows how to do ASN lookups since the last weekend. Mirrorbrain which runs behind the mirrorlist for opensuse and openoffice.org also uses ASNs as well as Fedora's MirrorManager which I have already set up for RPM Fusion. IF there is interest I can help set up a MirrorManager instance.
We looked at going down the route of a CDN, way back in 2006, and I've been keeping my eyes on that track since then. The reason why we didnt is that it would involve non-trivial job, monitoring and process control tasks to run on non centos.org machines; and that's not the sort of thing that large mirror providers want to look at.
Also given the very nature of how .centos.org is built up ( donated machines, with a fairly high rate of churn ) its hard to get our own gbp announces etc in place. We could potentially look at this again, maybe come up with a list of challenges and see if and how we might be able to solve them.
In the beginning, I also considered getting a BGP feed to get access to the data, but the aggregated data that routeviews.org publishes works just fine, and made the deployment easier.
But I whole-heartedly agree, it was all non-trivial nevertheless ;-)
Peter
-
Adrian Reber -
Bangladeshi CentOS Mirror Maintainer [BD-SERVERS.NET] -
Chris -
Jonathan Thurman -
Karanbir Singh -
Lucian -
Nyamul Hassan -
Peter Pöml -
R P Herrold -
Ralph Angenendt -
Randy McAnally -
SpeedPartner GmbH