Dear CentOS mirror administrators,
There are many mirrors, but few mirrors provides DVD images. I believe most of all mirrors have enough resources to provide DVD images. You always say the master can't provide DVD images for all mirrors without enough resources.
I suggest introducing mirror hierarchy as follows to solve this problem.
Administrators of servers with enough facilities stand as candidates of sub-master server. You may chose at most two or three servers in one country, and allow only sub-masters to access the bunch of master servers. In this case, All masters should provide DVD ISO images.
In my humble estimation, sub-masters should have at least 500 Mbps of effective bandwidth dedicated to CentOS. They should also provide rsync service able to have more than 20 simultaneous connections. The word `effective' means sub-masters should transfer more than 5 Tbytes per day only for CentOS.
As for our server, it can easily satisfy these criteria.
Sub-masters may synchronize with the master each hour. Other mirrors may synchronize with the sub-masters 2-4 times per day.
Sub-masters should properly share the rsync ACL with the masters to distribute new releases secretly in advance.
Please investigate this idea.
Regards,
On Mon, Nov 30, 2009 at 11:39:43AM +0900, Kazuhiro Fujieda wrote:
Dear CentOS mirror administrators,
There are many mirrors, but few mirrors provides DVD images. I believe most of all mirrors have enough resources to provide DVD images. You always say the master can't provide DVD images for all mirrors without enough resources.
I suggest introducing mirror hierarchy as follows to solve this problem.
Administrators of servers with enough facilities stand as candidates of sub-master server. You may chose at most two or three servers in one country, and allow only sub-masters to access the bunch of master servers. In this case, All masters should provide DVD ISO images.
In my humble estimation, sub-masters should have at least 500 Mbps of effective bandwidth dedicated to CentOS. They should also provide rsync service able to have more than 20 simultaneous connections. The word `effective' means sub-masters should transfer more than 5 Tbytes per day only for CentOS.
As for our server, it can easily satisfy these criteria.
Sub-masters may synchronize with the master each hour. Other mirrors may synchronize with the sub-masters 2-4 times per day.
Sub-masters should properly share the rsync ACL with the masters to distribute new releases secretly in advance.
Could you tell what is the parameters for your site? Especially, how many rsyncs do you allow simultaneously?
I believe you have a 10 Gbit/s connection.
best regards keld
On Tue, 01 Dec 2009 07:41:13 +0200 Keld Simonsen keld@keldix.com said:
Could you tell what is the parameters for your site? Especially, how many rsyncs do you allow simultaneously?
Our server has four GbE interfaces, 64 GB memory, and 20 TB storage with 600GB SSD cache. It allows 32768 HTTP, 8096 FTP and 128 rsync connections. More rsync connections are acceptable.
On 04/12/09 12:15, Kazuhiro Fujieda wrote:
Our server has four GbE interfaces, 64 GB memory, and 20 TB storage with 600GB SSD cache. It allows 32768 HTTP, 8096 FTP and 128 rsync connections. More rsync connections are acceptable.
That is quite impressive.
The other question is - how many such mirrors are there at the moment that we could use at a similar tier. I know kernel.org have been great friends of the project, and they have machines spread around the world[1] - so if the are willing, we can use their setup. We would still need atleast a dozen or so more machines that can push at greater than 400mbps on demand.
- KB
Karanbir Singh wrote:
On 04/12/09 12:15, Kazuhiro Fujieda wrote:
Our server has four GbE interfaces, 64 GB memory, and 20 TB storage with 600GB SSD cache. It allows 32768 HTTP, 8096 FTP and 128 rsync connections. More rsync connections are acceptable.
That is quite impressive.
The other question is - how many such mirrors are there at the moment that we could use at a similar tier. I know kernel.org have been great friends of the project, and they have machines spread around the world[1] - so if the are willing, we can use their setup. We would still need atleast a dozen or so more machines that can push at greater than 400mbps on demand.
I've no objections, and as I mentioned earlier there are already a number of other mirrors syncing directly from me. I've got a tier2 rsync module in place already for CentOS so this would mainly be a matter of populating the username/passwords for the module and getting those shipped off to those who request access. That would get you another 4gbps (currently) and quite a bit more geographic diversity.
- John 'Warthog9' Hawley
Replies to this thread a bit slow from my side, as I am also trying to see how we might be able to change things internally inside centos.org.
On 12/04/2009 06:21 PM, J.H. wrote:
I've no objections, and as I mentioned earlier there are already a number of other mirrors syncing directly from me.
Would you also be willing to accept a rsync push from this side ? We could either do it over keys or as a module with auth.
On Fri, 04 Dec 2009 13:03:45 +0000 Karanbir Singh mail-lists@karan.org said:
The other question is - how many such mirrors are there at the moment that we could use at a similar tier.
I believe you can use ftp.riken.jp and ftp.iij.ad.jp at the same tier of our server in Japan. I guess there are two or three suitable servers in each country where there are many mirrors.
On 12/05/2009 09:42 AM, Kazuhiro Fujieda wrote:
I believe you can use ftp.riken.jp and ftp.iij.ad.jp at the same tier of our server in Japan. I guess there are two or three suitable servers in each country where there are many mirrors.
Thats good to know. There are a few fairly well connected mirrors in Europe and the US as well. The black holes, from my perspective, are Austalia/NZ, almost all of Africa and South America.
Thats good to know. There are a few fairly well connected mirrors in Europe and the US as well. The black holes, from my perspective, are Austalia/NZ, almost all of Africa and South America.
Hello,
mirror.aarnet.edu.au is 10gb connected. We mirror from heanet for dvd's at the moment. If there is a mirror we could preload dvds from before release; access to that would be nice.
Regards, Alex Dodson
Hi Alex,
On 12/09/2009 11:52 PM, Alex Dodson wrote:
mirror.aarnet.edu.au is 10gb connected. We mirror from heanet for dvd's at the moment. If there is a mirror we could preload dvds from before release; access to that would be nice.
It would be awesome to setup a local site in .au that could act as a local dvd/msync seeder :)
- KB
On 9/12/09 9:53 PM, "Karanbir Singh" mail-lists@karan.org wrote:
Thats good to know. There are a few fairly well connected mirrors in Europe and the US as well. The black holes, from my perspective, are Austalia/NZ, almost all of Africa and South America.
KB,
Mind if I ask how you come to the conclusion that there's a mirror black hole in Australia?
I believe that the Australian mirror structure is quite good. As an example, on a population to mirror ratio we have nearly double what the US has.
Of the 11 public mirrors we have, the majority of the eyeballs within Australia have either direct access with a mirror on their network (eg: Telstra and Optus have mirrors) or have access to another network via peering.
The academic community is also well served by AARNet's mirror as well as those offered by various universities such as Monash and Swinburne.
Our mirror doesn't have near as much capacity as AARNet's mirror for example, but I'm yet to see it approach anywhere near the capacity that we've given it (our OpenOffice.org mirror for example easily does 10-20x the traffic of our CentOS mirror). If this holds true across the entire Australian mirror network, then we're anything but a black hole.
About 30% of the Australian mirrors are also IPv6 connected - three of the public mirrors have IPv6 support on their main hostname including ours and I know of one other that has IPv6 support via an alternate hostname.
DVDs are of course lacking but I don't believe this is due to a lack of interest from mirrors themselves but merely a lack of a procedure/capacity to get that content out to mirrors. We've always carried DVDs of the latest releases (we'll BT them + seed for 72 hours and then stick them in the mirror structure manually after verifying MD5/SHA1) but I've just had DVDs added across the board (using AARNet's mirror - Alex, if you have any objections please let me know).
New Zealand seems to be a different story with only one mirror over there. I do however know that it's less than 30ms across the Tasman, and we do have a number of downloads from NZ based hosts (about 5% compared to our Australian based traffic using DNS based identification only - not GeoIP).
Just some thoughts...
-Shaun
Hi Shaun,
On 12/10/2009 03:17 AM, Shaun Ewing wrote:
Mind if I ask how you come to the conclusion that there's a mirror black hole in Australia?
we have been asking hosting companies in Australia for a dedicated box that we might be able to use as a 'seeder' in the region - for about 3 years now - however, not one has actually come through with anything. Almost every time the issue that comes up is that bandwidth ( specially transit to the US/EU ) is too expensive to be able to let us run something like that.
Given that some of these guys are running > 500 machines with CentOS on there, I'd imagine they are not really the small fish. Also, this is pretty much what I got from a couple of guys out of Telstra as well.
If there are specific people you might be able to point me at, or even names of companies, I'd be happy to go down the route of contacting them.
- KB
-
Alex Dodson -
J.H. -
Karanbir Singh -
Kazuhiro Fujieda -
Keld Simonsen -
Shaun Ewing