Currently the default yum configuration references mirror.centos.org. When yum looks up the IP address it finds several but it will only try one and fail if that server is down or does not have the headers for whatever reason.
If yum were to reference a mirrorlist it would skip servers that fail and keep trying them all until it finds one that works or exhausts all the mirrors. Centos would host a file on their web server for each repository that has a list of mirror baseurls.
I created a test script to generate mirrorlists from the mirror.centos.org DNS lookup and changed yum to reference the local files that were generated as a mirrorlist= and it worked. The non-synced servers that did not have the correct directories setup were skipped and it found servers that did work.
Are there any plans to provide a centos-yumconf rpm that links to a mirrorlist on centos.org instead of just using the DNS round-robin of mirror.centos.org which will cause yum to fail if it happens to get an IP that does not work? Centos would just need to change the mirrorlist file on the main website to change mirrors for clients. Each mirror would need to have a DNS entry of its own but it would make things work better for updates when 1 server is not synced properly or has some other problem(down, etc).
---- begin gen-centos-mirrorlist.sh script ---- #!/bin/sh host mirror.centos.org | grep " has address " | cut -f 4 -d " " | \ sed "s#.*#http://%5C0/centos/4/updates/i386/#" > \ /etc/centos-mirror-updates.list host mirror.centos.org | grep " has address " | cut -f 4 -d " " | \ sed "s#.*#http://%5C0/centos/4/os/i386/#" > /etc/centos-mirror-base.list host mirror.centos.org | grep " has address " | cut -f 4 -d " " | \ sed "s#.*#http://%5C0/centos/4/addons/i386/#" > \ /etc/centos-mirror-addons.list host mirror.centos.org | grep " has address " | cut -f 4 -d " " | \ sed "s#.*#http://%5C0/centos/4/extras/i386/#" > \ /etc/centos-mirror-extras.list host mirror.centos.org | grep " has address " | cut -f 4 -d " " | \ sed "s#.*#http://%5C0/centos/4/centosplus/i386/#" > \ /etc/centos-mirror-centosplus.list ---- end generation script ----
---- begin changed CentOS-Base.repo yum configuration ---- [base] name=CentOS-$releasever - Base #baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/ mirrorlist=file:///etc/centos-mirror-base.list gpgcheck=1
#released updates [update] name=CentOS-$releasever - Updates #baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/ mirrorlist=file:///etc/centos-mirror-updates.list gpgcheck=1
#packages used/produced in the build but not released [addons] name=CentOS-$releasever - Addons #baseurl=http://mirror.centos.org/centos/$releasever/addons/$basearch/ mirrorlist=file:///etc/centos-mirror-addons.list gpgcheck=1
#additional packages that may be useful [extras] name=CentOS-$releasever - Extras #baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/ mirrorlist=file:///etc/centos-mirror-extras.list gpgcheck=1
#additional packages that extend functionality of existing packages [centosplus] name=CentOS-$releasever - Plus #baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/ mirrorlist=file:///etc/centos-mirror-centosplus.list gpgcheck=1 enabled=0
NOTE: My hack with using IPs from mirror.centos.org has the flaw in that the http servers in the list files are referenced by IP. If the mirror http server is using a Name Virtual Host and centos is not the default web site for that IP then it might try pulling the files from the wrong configured virtual host on that mirror. This might make some mirrors not work.
On Thu, 2005-10-13 at 15:30, Adam Gibson wrote:
Currently the default yum configuration references mirror.centos.org. When yum looks up the IP address it finds several but it will only try one and fail if that server is down or does not have the headers for whatever reason.
If yum were to retry using the alternate IP addresses, this wouldn't be an issue. Windows Internet Explorer manages this trick nicely, so it can't be that difficult...
If yum were to reference a mirrorlist it would skip servers that fail and keep trying them all until it finds one that works or exhausts all the mirrors. Centos would host a file on their web server for each repository that has a list of mirror baseurls.
If you have more than one machine behind a proxy, they will randomly pick servers from the mirrorlist and use different URL's, so you not only don't save the bandwidth by re-using the cached copy, you duplicate the copies in the cache. (Fedora already does it that way).
I created a test script to generate mirrorlists from the mirror.centos.org DNS lookup and changed yum to reference the local files that were generated as a mirrorlist= and it worked.
Now run a bunch of machine updates through the same squid proxy and see what happens. How about fixing yum to understand what multiple A records in DNS is all about?
yum falls over when confronted with a transparent squid proxy.
Les Mikesell wrote:
On Thu, 2005-10-13 at 15:30, Adam Gibson wrote:
Currently the default yum configuration references mirror.centos.org. When yum looks up the IP address it finds several but it will only try one and fail if that server is down or does not have the headers for whatever reason.
If yum were to retry using the alternate IP addresses, this wouldn't be an issue. Windows Internet Explorer manages this trick nicely, so it can't be that difficult...
If yum were to reference a mirrorlist it would skip servers that fail and keep trying them all until it finds one that works or exhausts all the mirrors. Centos would host a file on their web server for each repository that has a list of mirror baseurls.
If you have more than one machine behind a proxy, they will randomly pick servers from the mirrorlist and use different URL's, so you not only don't save the bandwidth by re-using the cached copy, you duplicate the copies in the cache. (Fedora already does it that way).
I created a test script to generate mirrorlists from the mirror.centos.org DNS lookup and changed yum to reference the local files that were generated as a mirrorlist= and it worked.
Now run a bunch of machine updates through the same squid proxy and see what happens. How about fixing yum to understand what multiple A records in DNS is all about?
On Thu, 2005-10-13 at 17:03, William Warren wrote:
yum falls over when confronted with a transparent squid proxy.
Don't let the transparent redirection hit it by surprise, tell it to use the proxy explicitly: http_proxy=http://my.proxy.com:3128 yum update
If you have more than one machine behind a proxy, they will randomly pick servers from the mirrorlist and use different URL's, so you not only don't save the bandwidth by re-using the cached copy, you duplicate the copies in the cache. (Fedora already does it that way).
I created a test script to generate mirrorlists from the mirror.centos.org DNS lookup and changed yum to reference the local files that were generated as a mirrorlist= and it worked.
Now run a bunch of machine updates through the same squid proxy and see what happens. How about fixing yum to understand what multiple A records in DNS is all about?
Seems like we have the perfect opportunity here to file an RFE with skvidal for yum, and we have an intermediate fix that works in the meantime. As for the whole proxy thing, yum is known to choke on them, so it's a known issue and isn't really worth discussing on a list other than the yum mailing list.
-- Jim Perrin System Administrator - UIT Ft Gordon & US Army Signal Center
On 10/13/05, Jim Perrin jperrin@gmail.com wrote:
Seems like we have the perfect opportunity here to file an RFE with skvidal for yum, and we have an intermediate fix that works in the meantime. As for the whole proxy thing, yum is known to choke on them, so it's a known issue and isn't really worth discussing on a list other than the yum mailing list.
In fairness, you might want to say that "yum is known to choke on proxies that don't support HTTP1.1" since it's a proxy problem, not a yum problem.
You are mostly right that discussing it here gets us almost nowhere.
Regards, Greg
-
Adam Gibson -
Greg Knaddison -
Jim Perrin -
Les Mikesell -
William Warren