Evening all.
I was doing some poking around with some centos servers this evening and I wanted to try and understand what type of smarts are behind mirrorlist.centos.org.
I know this has been covered before (somewhat). I know there is no way (at present) to "lock" a server on our network down to only using a specific mirror (short of hard coding the mirror URL into the yum repo file). I am aware of such capabilities when using mirrormanager, but we aren't there yet. As a result, when using the default method of fastestmirrors, I will generally be sent to a mirror that is close to me.
However, I also know there is some level of geo-smarts involved. Usually there is a good chance I will be sent to a mirror within the same country as my server, but not always. When querying mirrorlist, 10 results are returned as expected However out of those 10 results I find that: 7 of the mirrors are in the same country as my server (Australia) The remaining three are in three separate countries (USA, Canada, Ireland).
If I wait a short while and query mirrorlist again, I get different results: 8 of the mirrors are in the same country as my server (Australia). Interestingly out of these 8 mirrors, one is shown twice (first for HTTP then again for FTP) The remaining two mirrors are in Canada and Japan
It appears to alternate between 7 and 8 local mirrors, at least from all the testing I have done. With 2-3 mirrors in random countries.
Here is what I found interesting. When performing the same tests from servers in other countries (UK and USA), every time I queried mirrorlist, all 10 mirrors returned were local to the server (ie USA server was always provided 10 mirrors located in USA, and UK server was always provided 10 mirrors in UK)
Soooooo what I was wondering is how the initial list is determined? Obviously there is some level of geo-smarts involved here as most of the mirrors returned at in the same country. But why are a few outside for Australia? My guess is that something may have been implemented a long time ago before Australia had 10 mirrors in total?
Anyhow, enjoy your evening folks.
Cheers, Seamus
3.3.2015, 10.33, Seamus Ryan kirjoitti:
Soooooo what I was wondering is how the initial list is determined? Obviously there is some level of geo-smarts involved here as most of the mirrors returned at in the same country. But why are a few outside for Australia?
Based on your results, I think you are using IPv6. The IPv6 mirrorlist behaves differently than the IPv4 mirrorlist. The IPv6 mirrorlist only hands out mirrors that are up-to-date and available over IPv6.
Here is the current status of the Australian CentOS mirrors that have an IPv6 address in their DNS:
FAIL ftp://centos.mirror.crucial.com.au/ FAIL http://centos.mirror.uber.com.au/ OK ftp://ftp.swin.edu.au/centos/ OK ftp://mirror.overthewire.com.au/centos/ OK ftp://mirror.ventraip.net.au/CentOS/ OK http://centos.mirror.crucial.com.au/ OK http://centos.mirror.digitalpacific.com.au/ OK http://ftp.swin.edu.au/centos/ OK http://mirror.aarnet.edu.au/pub/centos/ OK http://mirror.colocity.com/centos/ OK http://mirror.overthewire.com.au/pub/centos/ OK http://mirror.ventraip.net.au/CentOS/ OLD ftp://mirror.aarnet.edu.au/pub/centos/
The first two are not reachable over IPv6, even though they have an IPv6 address in the DNS. The last one at AARNet is stuck in 17th February, and the three ftp mirrors are also available over http. This leaves us with seven usable IPv6-capable mirrors in Australia.
The IPv6 mirrorlist always outputs a list of 10 mirrors. If there aren't 10 mirrors in the specified country, it will pick mirrors from a predefined list of global fallback mirrors. Yes, those include mirrors in USA, Canada, Japan and Ireland. The fastestmirror plugin will usually weed out those global fallback mirrors so they don't actually get used.
There are 41 IPv6-capable CentOS mirrors in US, so the global fallback mirrors don't get used there as often as in other countries.
Once more mirrors enable IPv6, the need to include those global fallback mirrors in the list is reduced.
While I'm at this, a few remarks to mirror admins: If you enable IPv6 on your mirror, you do not need to announce it on this list. The mirror checker will pick those up automatically once your DNS has updated. If you enable IPv6 on your mirror, please check that IPv6 works on all protocols you support (http, ftp, rsync). You can check your mirror's IPv6 status from http://miuku.net/ipv6reach/
Hi Anssi,
Thanks for the explanation. I actually had no idea our mirror had stopped responding on ipv6, looks like an update broke our config at some point in the past. All fixed up now.
Cheers, Seamus
Sent from my Windows Phone 8 ________________________________ From: Anssi Johanssonmailto:centos@miuku.net Sent: 3/03/2015 8:35 PM To: centos-mirror@centos.orgmailto:centos-mirror@centos.org Subject: Re: [CentOS-mirror] accuracy of mirrorlist.centos.org
3.3.2015, 10.33, Seamus Ryan kirjoitti:
Soooooo what I was wondering is how the initial list is determined? Obviously there is some level of geo-smarts involved here as most of the mirrors returned at in the same country. But why are a few outside for Australia?
Based on your results, I think you are using IPv6. The IPv6 mirrorlist behaves differently than the IPv4 mirrorlist. The IPv6 mirrorlist only hands out mirrors that are up-to-date and available over IPv6.
Here is the current status of the Australian CentOS mirrors that have an IPv6 address in their DNS:
FAIL ftp://centos.mirror.crucial.com.au/ FAIL http://centos.mirror.uber.com.au/ OK ftp://ftp.swin.edu.au/centos/ OK ftp://mirror.overthewire.com.au/centos/ OK ftp://mirror.ventraip.net.au/CentOS/ OK http://centos.mirror.crucial.com.au/ OK http://centos.mirror.digitalpacific.com.au/ OK http://ftp.swin.edu.au/centos/ OK http://mirror.aarnet.edu.au/pub/centos/ OK http://mirror.colocity.com/centos/ OK http://mirror.overthewire.com.au/pub/centos/ OK http://mirror.ventraip.net.au/CentOS/ OLD ftp://mirror.aarnet.edu.au/pub/centos/
The first two are not reachable over IPv6, even though they have an IPv6 address in the DNS. The last one at AARNet is stuck in 17th February, and the three ftp mirrors are also available over http. This leaves us with seven usable IPv6-capable mirrors in Australia.
The IPv6 mirrorlist always outputs a list of 10 mirrors. If there aren't 10 mirrors in the specified country, it will pick mirrors from a predefined list of global fallback mirrors. Yes, those include mirrors in USA, Canada, Japan and Ireland. The fastestmirror plugin will usually weed out those global fallback mirrors so they don't actually get used.
There are 41 IPv6-capable CentOS mirrors in US, so the global fallback mirrors don't get used there as often as in other countries.
Once more mirrors enable IPv6, the need to include those global fallback mirrors in the list is reduced.
While I'm at this, a few remarks to mirror admins: If you enable IPv6 on your mirror, you do not need to announce it on this list. The mirror checker will pick those up automatically once your DNS has updated. If you enable IPv6 on your mirror, please check that IPv6 works on all protocols you support (http, ftp, rsync). You can check your mirror's IPv6 status from http://miuku.net/ipv6reach/ _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
-
Anssi Johansson -
Seamus Ryan