We will be turning off the server for mirror.supremebytes.com on April 2nd.
The mirror will no longer rsync with the centos servers as of time of this email.
Hey List,
I am writing a script for a local proxy and I want to add the full list of
mirrors into the proxy allowed acls.
Is there a machine readable list of all hosts and url's of all public
mirrors?
The other option I had in mind is parsing the table at:
http://mirror-status.centos.org
Thanks,
Eliezer
----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer(a)ngtech.co.il
Hello!
Please update our record accordingly (IPs have changed):
HTTP: http://mirror.cuegee.de/centos/ (if https is supported by now, we also offer https)
FTP: N/A
RSYNC: rsync://mirror.cuegee.de/centos/
Sync schedule: Every 4 hrs
Bandwidth: 1 Gbps
Location: DE
Sponsor: cuegee it gmbh
Sponsor URL: https://cuegee.de/
IPv4 address to authorize: 185.144.238.4
IPv6 address to authorize: 2a07:4400:0:1001::11
Email contact: mirror_AT_cuegee.net
Mirroring AltArch: no
--
Best regards
Alexander
cuegee it gmbh | Elisabethstraße 11 | 40217 Düsseldorf
Sitz: Düsseldorf | Registergericht: Düsseldorf | Registernummer: HRB 76270
Geschäftsführer: Alexander H. Schaber
Hello,
We have been a mirror or many years.We just configured a new server with a
full mirror using ipv4. We recently added ipv6 support to the new server
and lost access for updates. Please allow access for Rsync to update our
mirror using ipv6 as follows:
mirror.raystedman.net <http://mirror.raystedman.net/centos/>
us-msync.centos.org::CentOS
IPv4: 173.193.171.186 (this should already be in place)
IPv6: 2607:f0d0:1103:120::2 (please add)
Thanks, Greg
Hello,
we setup a new centos mirror and want you to include it on the list
HTTP: http://centos.mexico.machost.co
FTP: no
RSYNC: no
Sync schedule: Every 2 hrs
Bandwidth: 10gbps unmetered
Location: Mexico
Sponsor: Mach Host
Sponsor URL: machost.co
IPv4 address to authorize: 204.45.61.61
IPv6 address to authorize: not supported on mexico yet
Email contact: info(a)machost.co
Mirroring AltArch : no
thanks.
Hello,
Please remove mirror http://repo.de.bigstepcloud.com/centos/ . We have powered off the server.
We are still keeping the US and UK mirrors.
Many thanks,
MARIUS BOERU | IT Operations Manager | Bigstep | www.bigstep.com<http://bigstep.com>
PLEASE NOTE: This email and any file transmitted are confidential and/or legally privileged and intended only for the person(s) directly addressed. If you are not the intended recipient, any use, copying, transmission, distribution, or other forms of dissemination is strictly prohibited. If you have received this email in error, please notify the sender immediately and permanently delete the email and files, if any.
We'd like to add a new mirror based in the UK.
HTTP: <http://centos.mirror.iphh.net/CentOS/> http://mozart.ee.ic.ac.uk/CentOS/
Sync schedule: Every 6 hrs
Bandwidth: 1000 Mbit
Location: United Kingdom
Sponsor: Imperial College London
Sponsor URL: http://www.imperial.ac.uk
IPv4 address to authorize: 155.198.130.31
IPv6 address to authorize: 2001:630:12:1082:21b:78ff:fe7b:b824
Email contact: b.cochrane(a)imperial.ac.uk
Mirroring AltArch : no
Thanks in advance.
Bryan
<mailto:b.cochrane@imperial.ac.uk>
Hi!
We have migrated to a new (much stronger) server.
This one has a new IP address as well, could you please change this so
we can mirror directly from CentOS?
The new IPv4 address is "94.228.209.50"
We also support IPv6 now: "2a00:dd0:0:71:862b:2bff:fe68:8e6e"
Also we offer rsync as well. Could you add this to the list as well:
"rsync://mirror.netrouting.net/centos"
And our FTP address to: "ftp://mirror.netrouting.net/centos"
We have also changed the sync schedule to every 6 hours.
Also the new contact e-mail address is "mirror(a)netrouting.net".
Thanks a lot!
--
Best Regards,
*
Jim Klapwijk CDCP*
Datacenter Manager | Netrouting
Netrouting.com <3D%22https://www.netrouting.com%22>* a.new.experience*
Our locations: Amsterdam NL, Rotterdam NL, Spijkenisse NL, The Hague NL,
Frankfurt DE, Stockholm SE, Bucharest RO, Miami FL, Dallas TX.
Se habla Español / Wij spreken Nederlands / We speak English / Wir
sprechen Deutsch
Netrouting is registered in The Netherlands 50537504. Boyleweg 2, 3208
KA Spijkenisse.
DDI Telephone: +1 305-705-6983 / Toll Free (US): +1-888-216-9394 / Toll
Free (NL): 0800-HOST
/
Disclaimer: This e-mail and any attachment sent with it are intended
exclusively for the addressee(s), and may not be passed on to, or made
available for use by any person other than the addressee(s). If this
e-mail is not addressed to you, we ask you to notify the sender by
return e-mail and delete and destroy the original e-mail and any copies.
Netrouting rules out any and every liability resulting from any
electronic transmission.
/
Hi,
we would like to support you with a new mirror! :)
Here are the requested information:
HTTP: http://centos.mirror.iphh.net/CentOS/
Sync schedule: Every 6 hrs
Bandwidth: 500 Mbit
Location: Germany
Sponsor: IPHH Internet Port Hamburg GmbH
Sponsor URL: https://www.iphh.net/
IPv4 address to authorize: 62.201.161.93
IPv6 address to authorize: 2001:868:0:182::13
Email contact: repository(a)iphh.net
Mirroring AltArch : no
Thank you in advance
Cheers,
Carsten
Hi,
I have created a new mirror.
Thanks for adding.
DH
HTTP: http://mirror.it4i.cz/centos/
RSYNC: rsync://mirror.it4i.cz/centos/
Sync schedule: Every 1 hrs
Bandwidth: 10Gb
Location: CZ
Sponsor: IT4Innovations National Supercomputing Center
Sponsor URL: http://www.it4i.cz/
IPv4 address to authorize: 195.113.250.52
IPv6 address to authorize: 2001:718:1007:48::1:52
Email contact: support(a)it4i.cz
Mirroring AltArch : no
Hello everyone. As some of you may have noticed, I have started helping
Fabian Arrotin with mirrorlist management tasks. For example, I can now
make changes to the list of CentOS mirrors myself. I will also continue
maintaining the IPv6 mirror checker (see http://miuku.net/ipv6reach/ for
stats), which runs separate from the main CentOS mirror checker.
In order for me to change the access control list which specifies the IP
addresses that can connect to msync.centos.org nodes, the ACL had to be
moved from a separate configuration file to the main mirror database.
This gives me easier access to the ACL. The ACL is exported from the
database to msync nodes periodically.
The reason why I'm writing this message is that it's possible that some
of the IP addresses got accidentally assigned to mirrors that have been
disabled, and this causes that IP address to be excluded from the ACL. I
don't expect this to be a particularly widespread problem, but it is a
possibility.
If you have problems syncing (you get a message like Unknown module
'CentOS' and/or your timestamp.txt is too old), please let us know and
we'll fix the problem ASAP. Please specify your mirror's http URL and IP
addresses (IPv4, and if you're lucky enough to have IPv6, that too) so
that we could locate your mirror's data from the database quickly.
Thanks for your cooperation. If you have any other concerns or
questions, feel free to send me an email or come to #centos-mirror on
Freenode for a chat.
On 01/03/17 14:44, Anssi Johansson wrote:
> We have 600+ active CentOS mirrors[1] and sometimes there are critical
> updates that would need to get published to the mirror network as
> quickly as possible. As of now, it takes around five hours to go from 0
> up-to-date mirrors to 80% up-to-date mirrors, with a longer tail for the
> remaining 20%.
>
> Current guidelines for setting up a mirror[2] specify that mirrors
> should sync 2-4 times per day. Instead of telling mirrors to sync hourly
> I thought we could come up with something smarter.
>
> One option that I have considered would be something similar to what the
> ClamAV guys use to signal end users that there is new content. They use
> a DNS TXT record for that purpose. For example, as of this writing "dig
> txt +short current.cvd.clamav.net" produces
> "0.99.2:57:23148:1488371340:1:63:45637:290", which shows the version
> numbers for ClamAV itself, main virus database, daily virus database,
> timestamp and other version numbers.
>
> We could have something similar, showing the timestamps when the content
> for CentOS, CentOS AltArch and CentOS Vault was last modified, like
> "1488372781:1487767981:1488113581". "Last modified" in the sense that
> new packages got added at that time. The idea is that mirrors could set
> up scripts to check that timestamp from DNS more frequently (such as
> hourly) without causing load issues to msync nodes by rsyncing hourly.
> The TTL for the TXT record could be relatively small, like 10 minutes.
>
> As you're all aware, DNS is a prime example of a very scalable system,
> and that's why I'm fond of this solution. Another option would be to
> publish the same data in a central location and served over http(s), if
> relaying the timestamp data via DNS is not desired for some reason.
>
> The basic principle would be "if timestamp in TXT record > my current
> timestamp (TIME file), synchronize the mirror". With more frequent
> syncs, mirror admins would need to take care that no two rsync runs
> would happen at the same time. Using lockfile in the scripts would help
> with this. I hope that many of the mirror admins already use lockfiles,
> but providing an example script might help for the newer mirror admins.
>
> The timestamps should be updated only after it has been verified that
> all (or at least the majority) of msync nodes actually have the content.
> It takes a while for the data to reach all the msync nodes from the master.
>
> On the other hand, this may cause some traffic peaks for the msync
> nodes. I don't know how well they would handle the peaks. One obvious
> way to alleviate the peaks would be to instruct mirror admins to pick a
> random minute when to check for new content. echo $[ $RANDOM % 60 ]
> works nicely for this. I don't have statistics, but I believe there
> might be mirrors that sync at "0 */6" ie. at the top of the hour.
>
>
> If the traffic peaks to msync nodes is deemed to be a problem, there
> might be ways to reduce the load to msync nodes. The following idea
> could be implemented separately from the above timestamp idea, if needed.
>
> There could be some sort of a "web service" which instructs mirrors
> where to sync from. The core idea in this is that the source might not
> always be a msync.centos.org server, but it could also be a nearby
> public mirror that offers rsync and has been verified to have the new
> content. If requested from Finland, that service could say "ok, you're
> from .fi, go sync from ftp.funet.fi as it has the new content already"
> or "uh oh, no nearby external mirrors have the new content, please rsync
> from eu-msync.centos.org". It could simply return a list of rsync
> servers in descending priority, with some msync.centos.org addresses at
> the bottom as fallback. Once the mirror has rsynced, the mirror could
> ping back and say "I have the content now, please check, and if OK, add
> me to the list of mirrors that have the new content".
>
> One concern is that the list of rsync sources would need to be
> protected, so that mirrors could not be tricked into syncing from a
> malicious source (think DNS poisoning). Ways to protect from this
> include DNSSEC, TLS and PGP signed data.
>
>
> Any thoughts about this?
>
>
> [1] http://mirror-status.centos.org/
> [2] https://wiki.centos.org/HowTos/CreatePublicMirrors
Hi Anssi,
Thanks a lot for all those ideas.
While I understand your idea behind a TXT record at the DNS level, I'd
say that I'm not a fan of that idea. Even with lower TTL, you'd be
surprized how many hits we had on servers that were migrated to new
ones, as it seems some ISPs aren't obeying the TTL and so were still
serving wrong (and expired) A/AAAA records from their cache.
DNS itself (currently for centos.org) isn't DNSSEC enabled too, so that
would mean other protection, etc
I'd prefer your alternative with "let's host this behind a https web
server" and also for the reason that it's easier to have TLS for
webserver, and that from the automation point-of-view, it's easier for
people allowed to build/sign/push (two people) to just update/drop a
file somewhere, than using DNS modification.
For dns, as the zone is actually under git/puppet control, that would
mean *not* using that, but rather having a delegated zone that would
allow nsupdate with a key that those people would share, etc ... So the
simple file served from https seems easier from my side.
I'd like to get opinions from Johnny/KB (people able to sign/push) as
they'd be directly concerned by that decision.
From an "external mirror admin" PoV, we should also use the
centos-mirror list to discuss this, to get their opinions ?
Also, we can divide your proposal into two parts :
- external mirrors can check a file they can compare against to sync
"faster" than through their cron jobs (discussed above)
- modifying completely the msync.centos.org network to have external
mirrors not syncing from us, but betwen them (not sure how people feel
about this)
PS : Anssi is now part of the mirrors managers team for CentOS , for
people not yet aware of that fact
--
Fabian Arrotin
The CentOS Project | http://www.centos.org
gpg key: 56BEC54E | twitter: @arrfab
Is there a a minimum sync interval for syncing from the msync nodes? I know it’s recommended mirrors sync 4-6 times a day, so every 4-6 hours.
I’m an advocate of getting updates to users as quickly as possible. I’m not opposed to syncing my mirrors on an hourly basis as long as it won’t cause a load problem for the server upstream of me.
Thoughts?
Our mirror has a IPv6 address since about a week. Can our IPv6 address
please also be listed in the msync ACLs:
$ host ftp-stud.hs-esslingen.de
ftp-stud.hs-esslingen.de is an alias for rhlx01.hs-esslingen.de.
rhlx01.hs-esslingen.de has address 129.143.116.10
rhlx01.hs-esslingen.de has IPv6 address 2001:7c0:700::10
Adrian