CentOS has developed our own mirrorlist and isolist applications and inside this application, we have some countries that we shift to other countries and we also have some country groups defined.
The purpose of these groupings is to have adjacent countries grouped together for applications like fastest mirror and others.
I will put here what we currently have and ask the community to vet the list for us to make it better.
Our major (large) groups are:
us - North eu - Europe ap - Asia Pacific oc - Oceania af - Africa sa - South America c1 - (Use High Bandwidth, don't know location)
Here is our current countries "with mirrors" to large group list (1st variable is CC, 2nd is Group Code):
ar => 'sa', at => 'eu', au => 'oc', ba => 'eu', be => 'eu', bg => 'eu', br => 'sa', c1 => 'c1', c2 => 'eu', c3 => 'ap', c4 => 'us', ca => 'us', ch => 'eu', cl => 'sa', cn => 'ap', cr => 'sa', cz => 'eu', de => 'eu', dk => 'eu', ee => 'eu', es => 'eu', fi => 'eu', fr => 'eu', gb => 'eu', ge => 'eu', gr => 'eu', hk => 'ap', hu => 'eu', id => 'oc', ie => 'eu', il => 'af', in => 'ap', is => 'eu', it => 'eu', jp => 'ap', kr => 'ap', kz => 'ap', lt => 'eu', lu => 'eu', md => 'eu', my => 'oc', nl => 'eu', no => 'eu', nz => 'oc', ph => 'ap', pl => 'eu', pr => 'us', pt => 'eu', ro => 'eu', ru => 'ap', se => 'eu', sg => 'ap', si => 'eu', sk => 'eu', th => 'ap', tr => 'eu', tw => 'ap', ua => 'eu', us => 'us', vn => 'ap', za => 'af',
===================================================== We have created some smaller groups for each mirror country:
ar => 'br-cl-cr-pr-us', at => 'de-ch-lu-fr-cz-sk-hu-it', au => 'nz-sg-tw-kr-jp', ba => 'it-hu-bg-si-ro', be => 'fr-nl-de-lu-gb', bg => 'gr-ro-tr-it-md', br => 'pr-cl-ar-cr-us', c1 => 'c1', c2 => 'c2', c3 => 'c3', ca => 'us-is', ch => 'de-nl-be-fr', cl => 'pr-br-ar-cr-us', cn => 'tw-kr-sg-jp-ru', cr => 'br-cl-ar-pr-us', cz => 'pl-sk-de-at', de => 'be-ch-fr-nl-dk-gb', dk => 'de-se-nl-fi-no', ee => 'fi-ru-lt-pl-se-no', es => 'pt-fr-de-gb', fi => 'se-dk-no-de-nl', fr => 'de-nl-ch-be-es', gb => 'ie-nl-de-fr', ge => 'tr-eg', gr => 'tr-bg-it-ro', hk => 'cn-tw-ph-vn-th', hu => 'at-ro-si-sk-ba', id => 'sg-my-ph-au-vn', ie => 'gb-nl-de', il => 'it-tr-gr-bg', in => 'in-bd-vn-hk-sg-cn', is => 'gb-no-se-ca', it => 'de-ch-fr-lu-at-si', jp => 'sg-cn-kr-tw-ru', kr => 'jp-cn-tw-sg-ru', kz => 'ru-tr', lt => 'pl-ro-no-ee-de', lu => 'de-nl-be-fr', md => 'ro-ua-bg-si', my => 'id-sg-au-th-vn', nl => 'de-dk-gb-be-fr', no => 'se-fi-dk-de-nl', nz => 'au-sg-jp-tw', ph => 'vn-my-sg-id-au', pl => 'cz-ua-de-se-si', pr => 'cl-br-us-cr', pt => 'es-fr-de-gb', ro => 'at-de', ru => 'fi-cn-kr-jp', se => 'no-fi-dk-de-nl', sg => 'my-id-ph-au-th', si => 'it-md-at-ba-hu', sk => 'cz-hu-pl-md-at', th => 'cn-vn-my-sg-ph-hk', tr => 'gr-it-de-bg-ro', tw => 'cn-kr-sg-jp', ua => 'pl-ro-cz-de-md', us => '', vn => 'ph-sg-my-hk-cn', za => 'il-br-tr-es-in',
This is our list of adjacent countries, so if you are in PH, you will first get mirrors in conutry (PH), then from the small group (VN, CN, TW ), THEN from the large group (AP), THEN back filled from C1.
Or if you are in ZA, you would get mirrors from the country (ZA), then from small group (IL, BR, TR, ES, IN), then from large group (AF), THEN backfill from C1. ===================================================== We also have a cross reference having one country masquerade as another for grouping purposes (countries with mirrors are on the right) ... here is that list:
A2 => "C1", AD => "ES", AE => "IL", AF => "IN", AG => "US", AI => "US", AL => "GR", AM => "TR", AN => "US", AO => "ZA", AP => "C3", AQ => "C1", AR => "AR", AS => "AU", AT => "AT", AU => "AU", AW => "CR", AZ => "TR", BA => "BA", BB => "BR", BD => "SG", BE => "BE", BF => "ZA", BG => "BG", BH => "IL", BI => "ZA", BJ => "ZA", BM => "US", BN => "IL", BO => "CL", BR => "BR", BS => "US", BT => "CN", BV => "ZA", BW => "ZA", BY => "DE", BZ => "CR", CA => "CA", CD => "ZA", CF => "ZA", CG => "ZA", CH => "CH", CI => "ZA", CK => "NZ", CL => "CL", CM => "ZA", CN => "CN", CO => "BR", CR => "CR", CS => "SI", CU => "PR", CV => "ES", CY => "GR", CZ => "CZ", DE => "DE", DJ => "IL", DK => "DK", DM => "PR", DO => "PR", DZ => "IL", EC => "BR", EE => "EE", EG => "IL", ER => "IL", ES => "ES", ET => "IL", EU => "C2", FI => "FI", FJ => "AU", FK => "AR", FM => "PH", FO => "NO", FR => "FR", GA => "ZA", GB => "GB", GD => "PR", GE => "GE", GF => "BR", GH => "ZA", GI => "ES", GL => "SE", GM => "ZA", GN => "ZA", GP => "PR", GQ => "ZA", GR => "GR", GT => "CR", GU => "SG", GW => "ZA", GY => "BR", HK => "HK", HM => "AU", HN => "CR", HR => "SI", HT => "PR", HU => "HU", ID => "ID", IE => "IE", IL => "IL", IN => "IN", IO => "SG", IQ => "TR", IR => "TR", IS => "IS", IT => "IT", JM => "PR", JO => "IL", JP => "JP", KE => "ZA", KG => "CN", KH => "TH", KI => "C1", KM => "ZA", KN => "PR", KP => "KR", KR => "KR", KW => "IL", KY => "PR", KZ => "KZ", LA => "CN", LB => "IL", LC => "PR", LI => "CH", LK => "IN", LR => "ZA", LS => "ZA", LT => "LT", LU => "LU", LV => "RO", LY => "IT", MA => "ES", MC => "FR", MD => "MD", MG => "ZA", MH => "AU", MK => "SI", ML => "ES", MM => "IN", MN => "ZA", MO => "HK", MP => "PH", MQ => "PR", MR => "ES", MS => "PR", MT => "IT", MU => "ZA", MV => "IN", MW => "ZA", MX => "US", MY => "MY", MZ => "ZA", NA => "ZA", NC => "AU", NE => "ZA", NF => "AU", NG => "ZA", NI => "CR", NL => "NL", NO => "NO", NP => "IN", NR => "AU", NU => "AU", NZ => "NZ", OM => "IN", PA => "US", PE => "CL", PF => "AU", PG => "AU", PH => "PH", PK => "IN", PL => "PL", PR => "PR", PS => "IL", PT => "PT", PW => "PH", PY => "CL", QA => "IL", RE => "ZA", RO => "RO", RU => "RU", RW => "ZA", SA => "IL", SB => "AU", SC => "IN", SD => "IL", SE => "SE", SG => "SG", SI => "SI", SK => "SK", SL => "ZA", SM => "IT", SN => "ZA", SO => "ZA", SR => "BR", ST => "ZA", SV => "US", SY => "IL", SZ => "ZA", TC => "PR", TD => "ZA", TF => "ZA", TG => "ZA", TH => "TH", TJ => "KZ", TK => "AU", TM => "KZ", TN => "IT", TO => "AU", TR => "TR", TT => "BR", TV => "AU", TW => "TW", TZ => "ZA", UA => "UA", UG => "ZA", UM => "US", US => "US", UY => "CL", UZ => "KZ", VA => "IT", VC => "PR", VE => "CL", VG => "PR", VI => "PR", VN => "VN", VU => "AU", WF => "AU", WS => "AU", YE => "IL", YT => "ZA", ZA => "ZA", ZM => "ZA", ZW => "ZA",
This is basically a cross reference, the countries on the right is where we have mirrors, the ones on the left are the countries as returned from an IP address. =======================================================
If anything looks bad, especially for people who are in the countries and can tell us which ones are logically near (via the network) and not necessarily physically near, please reply to this thread and lets see if we can make the mirror network better.
Thanks, Johnny Hughes
On 8/23/09, Johnny Hughes johnny@centos.org wrote:
CentOS has developed our own mirrorlist and isolist applications and inside this application, we have some countries that we shift to other countries and we also have some country groups defined.
The purpose of these groupings is to have adjacent countries grouped together for applications like fastest mirror and others.
I will put here what we currently have and ask the community to vet the list for us to make it better.
<snip>
If anything looks bad, especially for people who are in the countries and can tell us which ones are logically near (via the network) and not necessarily physically near, please reply to this thread and lets see if we can make the mirror network better.
Thanks,
Hi Johnny and thank you for asking for feedback. I am in Cali, Colombia, South America. Our route to Brazil, for one of many examples, would be via Miami/Ft. Lauderdale, Florida, USA. So, for us, in Colombia, it is much better to have a Mirror site in South Florida or the S.E. USA, if possible; since I do not believe there is one in Colombia. The fact that Colombia is adjacent to Panama, Brazil, Venezuela, etc., would make a Mirror in one of those countries faster, *if* we had a direct route on the Internet to them, but, we don't. I suspect for most countries in Latin America, this would be the same, since Latin America is so strongly connected to Miami, "the capital" of Latin America. Lanny
Am 25.08.2009 um 23:16 schrieb Lanny Marcus:
Hi Johnny and thank you for asking for feedback. I am in Cali, Colombia, South America. Our route to Brazil, for one of many examples, would be via Miami/Ft. Lauderdale, Florida, USA. So, for us, in Colombia, it is much better to have a Mirror site in South Florida or the S.E. USA, if possible; since I do not believe there is one in Colombia. The fact that Colombia is adjacent to Panama, Brazil, Venezuela, etc., would make a Mirror in one of those countries faster, *if* we had a direct route on the Internet to them, but, we don't. I suspect for most countries in Latin America, this would be the same, since Latin America is so strongly connected to Miami, "the capital" of Latin America. Lanny _______________________________________________
Though off-topic, I find this interesting. Don't you in South America have some sort of high-speed network connecting the various universities there?
Don't you have peering-points somewhere?
Sorry for hijacking this thread...
Rainer
On 8/25/09, Rainer Duffner rainer@ultra-secure.de wrote:
Am 25.08.2009 um 23:16 schrieb Lanny Marcus:
Hi Johnny and thank you for asking for feedback. I am in Cali, Colombia, South America. Our route to Brazil, for one of many examples, would be via Miami/Ft. Lauderdale, Florida, USA. So, for us, in Colombia, it is much better to have a Mirror site in South Florida or the S.E. USA, if possible; since I do not believe there is one in Colombia. The fact that Colombia is adjacent to Panama, Brazil, Venezuela, etc., would make a Mirror in one of those countries faster, *if* we had a direct route on the Internet to them, but, we don't. I suspect for most countries in Latin America, this would be the same, since Latin America is so strongly connected to Miami, "the capital" of Latin America. Lanny _______________________________________________
Though off-topic, I find this interesting. Don't you in South America have some sort of high-speed network connecting the various universities there?
Don't you have peering-points somewhere?
Sorry for hijacking this thread...
Rainer: I don't think your reply is OT or hijacking. Good point. I suspect that in the EU, there is a lot of peering between countries, but here in SA, I don't think so. For one example, in our most advanced neighboring country, Brazil, they speak a different language. And, with 2 other neighboring countries (Venezuela and Ecuador) we have very serious political problems, because of the friendship Colombia has with the USA, the assistance those 2 countries give to Colombian guerrilla groups, etc. OK, now this is getting political, so it's getting way OT. :-) Lanny
Am 26.08.2009 um 00:13 schrieb Lanny Marcus:
On 8/25/09, Rainer Duffner rainer@ultra-secure.de wrote:
Am 25.08.2009 um 23:16 schrieb Lanny Marcus:
Hi Johnny and thank you for asking for feedback. I am in Cali, Colombia, South America. Our route to Brazil, for one of many examples, would be via Miami/ Ft. Lauderdale, Florida, USA. So, for us, in Colombia, it is much better to have a Mirror site in South Florida or the S.E. USA, if possible; since I do not believe there is one in Colombia. The fact that Colombia is adjacent to Panama, Brazil, Venezuela, etc., would make a Mirror in one of those countries faster, *if* we had a direct route on the Internet to them, but, we don't. I suspect for most countries in Latin America, this would be the same, since Latin America is so strongly connected to Miami, "the capital" of Latin America. Lanny _______________________________________________
Though off-topic, I find this interesting. Don't you in South America have some sort of high-speed network connecting the various universities there?
Don't you have peering-points somewhere?
Sorry for hijacking this thread...
Rainer: I don't think your reply is OT or hijacking. Good point. I suspect that in the EU, there is a lot of peering between countries, but here in SA, I don't think so.
I suspected that. To illustrate, this: http://www.caida.org/research/topology/as_core_network/pics/ascore-ipv4-ipv6... is the map I was looking for.
For one example, in our most advanced neighboring country, Brazil, they speak a different language. And, with 2 other neighboring countries (Venezuela and Ecuador) we have very serious political problems, because of the friendship Colombia has with the USA, the assistance those 2 countries give to Colombian guerrilla groups, etc.
Oh dear ;-) This is what they call a "problem on ISO layer 9".
cheers, Rainer
On 8/25/09, Rainer Duffner rainer@ultra-secure.de wrote:
Am 26.08.2009 um 00:13 schrieb Lanny Marcus:
<snip>
Don't you have peering-points somewhere? Sorry for hijacking this thread...
Rainer: I don't think your reply is OT or hijacking. Good point. I suspect that in the EU, there is a lot of peering between countries, but here in SA, I don't think so.
I suspected that. To illustrate, this: http://www.caida.org/research/topology/as_core_network/pics/ascore-ipv4-ipv6... is the map I was looking for.
Cool. I downloaded that 4.3 MB file. I suspect we could peer with Panama. I believe like Colombia, they are also quite advanced with regard to telecommunications, and it's close. The city I live in (Cali) is actually much closer to Miami, Florida than Los Angeles, California is. And maybe we could peer with Peru, but that's a long way to our South.
<snip>
Oh dear ;-) This is what they call a "problem on ISO layer 9".
Yes. Venezuela is threatening, today, to break off diplomatic relations with Colombia. We are not buying billions of dollars of military arms from anyone, but they are. I know 2 people in Venezuela who want to attend a concert in Colombia in September, but it is very doubtful they will be allowed to leave Venezuela. Like Cuba.....
To get back ON topic. When I first moved here, 14 years ago, my first ISP (and probably the only ISP here at that time) was the public university. If that was still the case, I would ask them to consider installing a CentOS Mirror.
Hmmm. Maybe the next time I am near there, I will visit their computer center and ask them to consider installing a CentOS Mirror. Also, the next time I get an email from someone in IEEE Colombia, I will ask them to consider trying to get a university in Bogota to install a CentOS Mirror.
On Sun, Aug 23, 2009 at 1:32 AM, Johnny Hughesjohnny@centos.org wrote:
CentOS has developed our own mirrorlist and isolist applications and inside this application, we have some countries that we shift to other countries and we also have some country groups defined.
Johnny: A few minutes ago, I began a "yum update" for my CentOS 5.3 (32 bit) Desktop. I could see a change for the fastest mirrors. For "extras" it chose mirrors.ucr.ac.cr About 1 1/2 hours in a nonstop jet from here. I'd never seen that before, so I'm sure it is a result of the changes you have been making. Lanny
Lanny Marcus wrote:
On Sun, Aug 23, 2009 at 1:32 AM, Johnny Hughesjohnny@centos.org wrote:
CentOS has developed our own mirrorlist and isolist applications and inside this application, we have some countries that we shift to other countries and we also have some country groups defined.
Johnny: A few minutes ago, I began a "yum update" for my CentOS 5.3 (32 bit) Desktop. I could see a change for the fastest mirrors. For "extras" it chose mirrors.ucr.ac.cr About 1 1/2 hours in a nonstop jet from here. I'd never seen that before, so I'm sure it is a result of the changes you have been making. Lanny
Yes, it is indeed based on changes made to find geographically close mirrors.
If there is a problem though with "number of hops" for those mirrors, then please let me know and we can do something else for South American countries.
If you look at the file /var/cache/yum/timedhosts.txt (assuming you have fastestmirror turned on, something I highly recommend), you can see the times that yum saw when testing the connection last.
Also, you can look at the following link to see the mirrorlist you are getting:
http://mirrorlist.centos.org/?release=4&arch=i386&repo=os
The mirrorlist contains only 10 servers, and is regenerated every 4-8 hours. If you have more than 10 mirrors available (The US and several EU countries) then you are likely to get "all different" mirrors every time the list generates. Also, a different list is generated for each repo (os, extras, centosplus, updates, etc.). The purpose of the revolving lists over time and different lists for each repo is to equally distribute load equally across 300 servers.
For countries where there are some, but not more than 10, mirrors what should happen is that every time we regenerate the list, the "less than 10" servers will always appear and we will back fill servers that are geographically close to your country. If we can not get "10" servers that are geographically close based on our lists, then we will back fill based on Continents. If there are still not 10 servers (Africa and South America), we will back fill fast (high bandwidth) servers to make up the rest of the list.
So, this system should produce geographically relevant (and pick the fastest mirror of those listed).
The smallest granularity is by country, so if you are in New York, New York USA, you will get a list of 10 US servers ... but not necessarily the 10 closest to New York city (as an example). However, of the list of 10, you should use the one that has the lowest connect time to you from that list.
On thing to remember is that you can also add a couple servers with the "baseurl=" line ... and that you can use both mirrorlist and baseurl at the same time. Fastest mirror should always pick a local mirror (time should be MUCH less) if it is available, then fail over to other mirrors is the local mirror is down.
On Sat, Aug 29, 2009 at 9:31 AM, Johnny Hughesjohnny@centos.org wrote:
Lanny Marcus wrote:
<snip>
Johnny: A few minutes ago, I began a "yum update" for my CentOS 5.3 (32 bit) Desktop. I could see a change for the fastest mirrors. For "extras" it chose mirrors.ucr.ac.cr About 1 1/2 hours in a nonstop jet from here. I'd never seen that before, so I'm sure it is a result of the changes you have been making. Lanny
Yes, it is indeed based on changes made to find geographically close mirrors.
If there is a problem though with "number of hops" for those mirrors, then please let me know and we can do something else for South American countries.
If you look at the file /var/cache/yum/timedhosts.txt (assuming you have fastestmirror turned on, something I highly recommend), you can see the times that yum saw when testing the connection last.
mirrors.ucr.ac.cr 0.189340829849
Also, you can look at the following link to see the mirrorlist you are getting: http://mirrorlist.centos.org/?release=4&arch=i386&repo=os
http://mirror.centos-br.org/4.8/os/i386/ http://centos.pop.com.br/4.8/os/i386/ http://mirrors.ucr.ac.cr/centos/4.8/os/i386/ http://mirror.sanctuaryhost.com/centos/4.8/os/i386/ http://centos.omnispring.com/4.8/os/i386/ http://mirror.raystedman.net/centos/4.8/os/i386/ http://centos.cogentcloud.com/4.8/os/i386/ http://centos.secsup.org/4.8/os/i386/ http://mirrors.serveraxis.net/centos/4.8/os/i386/ http://repos.lax-noc.com/centos/4.8/os/i386/
The only one of those I saw this morning when I did the "yum update" was the one in Costa Rica. The time to Costa Rica isn't the fastest in the timedhosts.txt file, but it's fast and MUCH faster than the slowest mirrors in the list. If there is a mirror in Panama, probably that would be good for us in Colombia too.
On thing to remember is that you can also add a couple servers with the "baseurl=" line ... and that you can use both mirrorlist and baseurl at the same time. Fastest mirror should always pick a local mirror (time should be MUCH less) if it is available, then fail over to other mirrors is the local mirror is down.
As I mentioned the other day, I am going to try to visit with the Network Admin at the Public University in Cali, to see if they might add a CentOS Mirror. If not, I will send someone in IEEE Colombia an email, the next time they send me an email and maybe they can get one at a university in Bogota.
As I think you wrote, the mirror lists for different repos a different. The ones in South Florida (if available) usually do very well for me, and if they are not available, somewhere else in the S.E. USA.
Thank you, for your interesting explanations and help and trying to get the best mirrors!
-
Johnny Hughes -
Lanny Marcus -
Rainer Duffner