W dniu 2015-06-18 16:43, Fabian Arrotin pisze:

I can have a look, but don't forget that the mirror crawler process will do the following :

• • query for mirrors from a current country (but in random way)
• • validate each mirror from that randomized list until it gets to 10

mirror for each list (per release/repo/arch)

• • when 10 nodes are there, stop

So each time a yum mirrorlist is produced, your mirror can enter/leave/enter it again (and each mirrorlist is produced in loop , and takes ~30 minutes), and there are more than 10 mirrors in .pl actually :-)

Hope that it answers your question

I decide to take a deep look at which mirrors are presented by http://mirrorlist.centos.org/..........

I logged every 5 minutes response from mirrorlist website for client from Poland for such repos: i386 extras 5 i386 extras 6 i386 os 5 i386 os 6 i386 updates 5 i386 updates 6 x86_64 extras 5 x86_64 extras 6 x86_64 extras 7 x86_64 os 5 x86_64 os 6 x86_64 os 7 x86_64 updates 5 x86_64 updates 6 x86_64 updates 7

I've got such results

Ideal precentage centos2.hti.pl ftp.agh.edu.pl centos.hitme.net.pl centos1.hti.pl ftp.pbone.net centos.komster.pl centos.slaskdatacenter.com mirror.onet.pl ftp.prz.edu.pl ftp.icm.edu.pl ftp.vectranet.pl ftp.ps.pl mirror-pl.kielcetechnologypark.net centos.po.opole.pl i386 extras 5 71% 70% 72% 72% 68% 70% 71% 70% 76% 75% 75% 68% 70% 73% 69% i386 extras 6 71% 70% 73% 66% 69% 79% 72% 73% 68% 69% 71% 75% 70% 75% 72% i386 os 5 71% 71% 71% 70% 76% 66% 70% 72% 75% 68% 69% 73% 72% 76% 71% i386 os 6 71% 74% 70% 71% 68% 71% 70% 74% 71% 71% 72% 78% 74% 68% 68% i386 updates 5 77% 79% 76% 74% 77% 76% 73% 75% 77% 81% 75% 75% 73% 70% i386 updates 6 77% 77% 75% 76% 76% 76% 72% 79% 78% 74% 72% 77% 72% 73% x86_64 extras 5 71% 75% 76% 71% 72% 76% 67% 75% 71% 72% 69% 64% 69% 70% 73% x86_64 extras 6 71% 71% 68% 74% 75% 71% 69% 74% 73% 69% 73% 70% 69% 69% 75% x86_64 extras 7 71% 76% 75% 76% 76% 72% 75% 76% 74% 72% 74% 72% 74% 70% 34% x86_64 os 5 71% 70% 69% 71% 73% 71% 72% 74% 72% 74% 71% 71% 73% 70% 69% x86_64 os 6 71% 68% 77% 77% 72% 70% 74% 75% 71% 69% 70% 74% 65% 71% 68% x86_64 os 7 71% 71% 73% 70% 69% 75% 71% 72% 72% 70% 73% 70% 73% 70% 72% x86_64 updates 5 77% 74% 78% 78% 78% 77% 74% 77% 75% 78% 73% 72% 73% 72% x86_64 updates 6 77% 71% 75% 79% 76% 77% 76% 80% 74% 73% 77% 80% 70% 71% x86_64 updates 7 77% 73% 75% 80% 72% 70% 74% 78% 72% 75% 72% 74% 73% 72%

10 overall score 15 7 7 8 8 7 3 13 9 6 6 6 5 3 4 overall percentage score 47% 47% 53% 53% 47% 20% 87% 60% 40% 40% 40% 33% 20% 40%

(I hope you can see table with colours :)

Green colour means that for this repo mirror site was above "ideal percentage" so it was presented often than other mirrors and often than ideal fair percentage. If more green for mirror site than overall score is higher and mirror site was more popular on http://mirrorlist.centos.org/ This statistics is based on 11 days.

First conclusion. mirrorlist randomization algorithm is not working very well or something else is going bad. Some mirrors are definitely more popular that other. I.e centos,slaskdatacenter.com vs centos.komster.pl or vs ftp.pbone.net (which is synced 12 times a day and has 100% uptime and availability during test)

Second conclusion. During test I noticed that mirrorlist validation has some problems because for all update repos and x86_64 extras 7 repo I have in log file that for some time mirrorlist returns couple mirror sites outside Poland. It didn't happened for other repos. Based on that I assume that every mirror was up (excluding centos.po.opole.pl) and even that your mirrorlist page returned mirrors outside Poland. If it'll help to investigate problem I can provide date and time of such situations.

I think that limit of 10 mirrors on mirrorlist site is not profitable for users. Very good example is USA. Centos has 145 mirrors in USA. If you present only 10 it's probably that users use not optimal mirror sites for them. Yum has fastestmirror plugin, let the user to choose the best mirror for him.

Best regards. Grzegorz

W dniu 2015-06-30 11:40, Fabian Arrotin pisze:

...

I think that limit of 10 mirrors on mirrorlist site is not profitable for users. Very good example is USA. Centos has 145 mirrors in USA. If you present only 10 it's probably that users use not optimal mirror sites for them. Yum has fastestmirror plugin, let the user to choose the best mirror for him.

True, and the 10 mirror limit is quite old (from where the scripts were written, and don't known the real reason) but it does that for every country, and not on a "country basis". Logic is the following : it tries for a country to find at least 10 nodes that are "current " (so providing the same repodata as on master, or at least in cache), and if it can't , it add some nodes from nearby countries (which are up2date)

We can discuss that limit at the moment, but don't forget that the more you add mirrors in the the list, the more outgoing checks yum-fastestmirror-plugin will also do .. so finding the correct balance is always a difficult game.

True, balance is important. Default repo configuration for Centos-Base has enabled [base] [updates] [extras] [centosplus] [contrib] . For each repo it is possible to get 10 totally different mirror sites . It means that it is possible by default that fastestmirror plugin will check 50 sites. Of course only in country with 50 or more centos mirrors. In countries with less than 50 mirror sites it is very likely that all host are already checked and users won't notice the difference in "fastestmirror check time" after the number of mirror sites returned by mirrorlist will be increased. Only USA users has more than 50 mirror sites but USA users IMHO will be also happy because they will be use better mirror from their point of view. We shouldn't forget that fastestmirror plugin does check procedure from time to time (on centos 6 maxhostfileage=10, for me it means 10 days). So it not a big deal to wait a bit longer three times in a month to get "nearest" mirror site.

Best regards. --- Grzegorz