With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
That may be due in part to it being hard to find and more readily available by Bt, but I think is also down to increased prevalance of dvd-rom drives.
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
(CentOS now has its first server up running C5 and able to display large dvd images under http :)
I have updated our isoredirect system to detect and verify dvd isos on mirrors , and sources with current dvd isos are now listed separately on the isoredirect page - what this is showing is that very few of our mirrors have the dvd isos and we need to get more out there ...
To test :-
http://isoredirect.centos.org/mirrorscripts/chooseisolist.pl?release=5.0&... It will give you local info based on your ip you can also add &cc=xx to test other country codes
I know a lot of mirrors are wanting to download the dvd isos and we have just configured two new servers with msync-incdvd.
However, we would still like mirrors to do their initial sync of the dvd isos to one of our tier 1 mirrors that have them , and then we will add their ip addres to our acl for syncing.
We also have no problem if mirrors want to sync to a local tier 1 mirror, our system will still detect that they are up to date.
It is likely that in the near future we will restrict access to msync rsync to current registered mirrors, and new mirrors will either have to register first or do an initial sync from a registered mirror *hmm - we have said this before - but now we have 160 mirrors we really will have to do something ...)
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that :-
1. provide rsync access to the complete CentOS release tree 2. would be prepared to act as a seed for new mirrors 3. would be prepared to set up separate CentOS and CentOS-incdvd targets for other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
We will then publish details for other and new mirrors of how they can sync against these master sync servers.
If you would be prepared for your mirror to act as one, and have the bandwidth and capacity for it then please respond.
Whilst writing I would like to thank all of our mirrors for the tremendous support that you provide for CentOS and our users and especially during the release of 5.0
Regards Lance
-- uklinux.net - The ISP of choice for the discerning Linux user.
Site ftp.upce.cz
I have "old" Apache (from CentOS 4.x) and DVD via HTTP is not avalaible, but via FTP is OK
Lance Davis wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
Lance Davis wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
For a while there, we were the only mirror showing DVDs on the CentOS site. We had to throttle our bandwidth because we were pushing as much as 350 Mbps, mostly in DVD ISOs. Things have calmed down a bit from that high now.
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
Rebuilding the apr, apr-util, and httpd SRPMs from CentOS 5 or Fedora Devel will work for CentOS 4. Only catch, httpd includes an init script which requires a new version of initscripts. To work around it, you can install the src RPM, then grab your old /etc/init.d/httpd and replace /usr/src/redhat/SOURCE/httpd.init. Then you can set the dependency to require 7.93 of initscripts instead in the specfile and build. That's what we're doing currently.
Kevin
On Sun, 15 Apr 2007, Kevin M Stange wrote:
Lance Davis wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
For a while there, we were the only mirror showing DVDs on the CentOS site. We had to throttle our bandwidth because we were pushing as much as 350 Mbps, mostly in DVD ISOs. Things have calmed down a bit from that high now.
Phew - thanks for that - I think there are now 4 US mirrors shown carrying dvd isos - and we are pushing to get more ...
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
Rebuilding the apr, apr-util, and httpd SRPMs from CentOS 5 or Fedora Devel will work for CentOS 4. Only catch, httpd includes an init script which requires a new version of initscripts. To work around it, you can install the src RPM, then grab your old /etc/init.d/httpd and replace /usr/src/redhat/SOURCE/httpd.init. Then you can set the dependency to require 7.93 of initscripts instead in the specfile and build. That's what we're doing currently.
aha
Regards Lance
Kevin
-- Kevin Stange Chief Technology Officer Steadfast Networks http://steadfast.net
Phone: 312-602-2689 x203 Fax: 312-602-2688 Cell: 312-320-5867
uklinux.net - The ISP of choice for the discerning Linux user.
on http://isoredirect.centos.org/mirrorscripts/chooseisolist.pl?release=5.0&... where not shown as having the dvdimages but we do(mirrorservice.org (GB) ) available via ftp...
http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos-dvd/
etc..
Thanks.
Phill.
On Sun, 15 Apr 2007, Lance Davis wrote:
On Sun, 15 Apr 2007, Kevin M Stange wrote:
Lance Davis wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
For a while there, we were the only mirror showing DVDs on the CentOS site. We had to throttle our bandwidth because we were pushing as much as 350 Mbps, mostly in DVD ISOs. Things have calmed down a bit from that high now.
Phew - thanks for that - I think there are now 4 US mirrors shown carrying dvd isos - and we are pushing to get more ...
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
Rebuilding the apr, apr-util, and httpd SRPMs from CentOS 5 or Fedora Devel will work for CentOS 4. Only catch, httpd includes an init script which requires a new version of initscripts. To work around it, you can install the src RPM, then grab your old /etc/init.d/httpd and replace /usr/src/redhat/SOURCE/httpd.init. Then you can set the dependency to require 7.93 of initscripts instead in the specfile and build. That's what we're doing currently.
aha
Regards Lance
Kevin
-- Kevin Stange Chief Technology Officer Steadfast Networks http://steadfast.net
Phone: 312-602-2689 x203 Fax: 312-602-2688 Cell: 312-320-5867
uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On Sun, 15 Apr 2007, archive-admin@mirrorservice.org wrote:
on http://isoredirect.centos.org/mirrorscripts/chooseisolist.pl?release=5.0&... where not shown as having the dvdimages but we do(mirrorservice.org (GB) ) available via ftp...
http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos-dvd/
When our script checks the dvd it gets a 403 ...
Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - bad dvd - 403 Forbidden
Which is what I saw before, but when I just tested it I got taken to ftp ...
Would you prefer I just checked ftp and linked to the isos dir there ??
Regards Lance
etc..
Thanks.
Phill.
On Sun, 15 Apr 2007, Lance Davis wrote:
On Sun, 15 Apr 2007, Kevin M Stange wrote:
Lance Davis wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
For a while there, we were the only mirror showing DVDs on the CentOS site. We had to throttle our bandwidth because we were pushing as much as 350 Mbps, mostly in DVD ISOs. Things have calmed down a bit from that high now.
Phew - thanks for that - I think there are now 4 US mirrors shown carrying dvd isos - and we are pushing to get more ...
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
Rebuilding the apr, apr-util, and httpd SRPMs from CentOS 5 or Fedora Devel will work for CentOS 4. Only catch, httpd includes an init script which requires a new version of initscripts. To work around it, you can install the src RPM, then grab your old /etc/init.d/httpd and replace /usr/src/redhat/SOURCE/httpd.init. Then you can set the dependency to require 7.93 of initscripts instead in the specfile and build. That's what we're doing currently.
aha
Regards Lance
Kevin
-- Kevin Stange Chief Technology Officer Steadfast Networks http://steadfast.net
Phone: 312-602-2689 x203 Fax: 312-602-2688 Cell: 312-320-5867
uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
-- Archive Admin's at The UK Mirror Service http://www.mirrorservice.org/ _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
uklinux.net - The ISP of choice for the discerning Linux user.
On Sun, 15 Apr 2007, Lance Davis wrote:
On Sun, 15 Apr 2007, archive-admin@mirrorservice.org wrote:
on http://isoredirect.centos.org/mirrorscripts/chooseisolist.pl?release=5.0&... where not shown as having the dvdimages but we do(mirrorservice.org (GB) ) available via ftp...
http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos-dvd/
When our script checks the dvd it gets a 403 ...
Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - bad dvd - 403 Forbidden
Hmm ok not what i get from home ,work or the machine in london,
What ip do the checks come from, and do you have a special user agent set?
are you checking with a head request?
Thanks
Phill.
Which is what I saw before, but when I just tested it I got taken to ftp ...
Would you prefer I just checked ftp and linked to the isos dir there ??
Regards Lance
etc..
Thanks.
Phill.
On Sun, 15 Apr 2007, Lance Davis wrote:
On Sun, 15 Apr 2007, Kevin M Stange wrote:
Lance Davis wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
For a while there, we were the only mirror showing DVDs on the CentOS site. We had to throttle our bandwidth because we were pushing as much as 350 Mbps, mostly in DVD ISOs. Things have calmed down a bit from that high now.
Phew - thanks for that - I think there are now 4 US mirrors shown carrying dvd isos - and we are pushing to get more ...
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
Rebuilding the apr, apr-util, and httpd SRPMs from CentOS 5 or Fedora Devel will work for CentOS 4. Only catch, httpd includes an init script which requires a new version of initscripts. To work around it, you can install the src RPM, then grab your old /etc/init.d/httpd and replace /usr/src/redhat/SOURCE/httpd.init. Then you can set the dependency to require 7.93 of initscripts instead in the specfile and build. That's what we're doing currently.
aha
Regards Lance
Kevin
-- Kevin Stange Chief Technology Officer Steadfast Networks http://steadfast.net
Phone: 312-602-2689 x203 Fax: 312-602-2688 Cell: 312-320-5867
uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
-- Archive Admin's at The UK Mirror Service http://www.mirrorservice.org/ _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On Mon, 16 Apr 2007, archive-admin@mirrorservice.org wrote:
On Sun, 15 Apr 2007, Lance Davis wrote:
On Sun, 15 Apr 2007, archive-admin@mirrorservice.org wrote:
on http://isoredirect.centos.org/mirrorscripts/chooseisolist.pl?release=5.0&... where not shown as having the dvdimages but we do(mirrorservice.org (GB) ) available via ftp...
http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos-dvd/
When our script checks the dvd it gets a 403 ...
Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - bad dvd - 403 Forbidden
Hmm ok not what i get from home ,work or the machine in london,
What ip do the checks come from, and do you have a special user agent set?
72.21.40.10
probably LWP
are you checking with a head request?
yes
But the same request works fine for the cd isos - this is the log :-
Name - UK Mirror Service - Univ. Kent - using url http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/ http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/sha1sum.t... Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - good Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - good Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - good Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - good Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - good Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - bad dvd - 403 Forbidden Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - good
so whatever is happening seems dvd iso specific
Regards Lance
Thanks
Phill.
Which is what I saw before, but when I just tested it I got taken to ftp ...
Would you prefer I just checked ftp and linked to the isos dir there ??
Regards Lance
etc..
Thanks.
Phill.
On Sun, 15 Apr 2007, Lance Davis wrote:
On Sun, 15 Apr 2007, Kevin M Stange wrote:
Lance Davis wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
For a while there, we were the only mirror showing DVDs on the CentOS site. We had to throttle our bandwidth because we were pushing as much as 350 Mbps, mostly in DVD ISOs. Things have calmed down a bit from that high now.
Phew - thanks for that - I think there are now 4 US mirrors shown carrying dvd isos - and we are pushing to get more ...
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
Rebuilding the apr, apr-util, and httpd SRPMs from CentOS 5 or Fedora Devel will work for CentOS 4. Only catch, httpd includes an init script which requires a new version of initscripts. To work around it, you can install the src RPM, then grab your old /etc/init.d/httpd and replace /usr/src/redhat/SOURCE/httpd.init. Then you can set the dependency to require 7.93 of initscripts instead in the specfile and build. That's what we're doing currently.
aha
Regards Lance
Kevin
-- Kevin Stange Chief Technology Officer Steadfast Networks http://steadfast.net
Phone: 312-602-2689 x203 Fax: 312-602-2688 Cell: 312-320-5867
uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
-- Archive Admin's at The UK Mirror Service http://www.mirrorservice.org/ _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
-- Archive Admin's at The UK Mirror Service http://www.mirrorservice.org/ _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
uklinux.net - The ISP of choice for the discerning Linux user.
On Mon, 16 Apr 2007, Lance Davis wrote:
On Mon, 16 Apr 2007, archive-admin@mirrorservice.org wrote:
On Sun, 15 Apr 2007, Lance Davis wrote:
On Sun, 15 Apr 2007, archive-admin@mirrorservice.org wrote:
on http://isoredirect.centos.org/mirrorscripts/chooseisolist.pl?release=5.0&... where not shown as having the dvdimages but we do(mirrorservice.org (GB) ) available via ftp...
http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos-dvd/
When our script checks the dvd it gets a 403 ...
Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - bad dvd - 403 Forbidden
Hmm ok not what i get from home ,work or the machine in london,
What ip do the checks come from, and do you have a special user agent set?
72.21.40.10
probably LWP
are you checking with a head request?
yes
But the same request works fine for the cd isos - this is the log :-
Name - UK Mirror Service - Univ. Kent - using url http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/ http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/sha1sum.t... Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - good Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - good Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - good Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - good Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - good Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - bad dvd - 403 Forbidden Check http://www.mirrorservice.org/sites/mirror.centos.org/5.0/isos/i386/CentOS-5.... - good
so whatever is happening seems dvd iso specific
Regards Lance
Right I allow HEAD requests through but the GET redirects
a suspect the head is failing as apache not 2g frendly yet, I have tweeked what does the checker report now.
Might be an idea to set your own user agent allows people to easily filer in stats and suchlike.
Thanks.
Phill.
On Mon, 16 Apr 2007, archive-admin@mirrorservice.org wrote:
Right I allow HEAD requests through but the GET redirects
a suspect the head is failing as apache not 2g frendly yet, I have tweeked what does the checker report now.
Well I got a good on the x86_64 dvd - and a fail on the i386 - but that may have been across the time you were tweaking ...
The next run will be in about an hours time ...
Might be an idea to set your own user agent allows people to easily filer in stats and suchlike.
Can do ...
Regards Lance
-- uklinux.net - The ISP of choice for the discerning Linux user.
On Mon, 16 Apr 2007, archive-admin@mirrorservice.org wrote:
Right I allow HEAD requests through but the GET redirects
a suspect the head is failing as apache not 2g frendly yet, I have tweeked what does the checker report now.
Ok - I set up a test script and the i386 dvd looks ok as well .
Might be an idea to set your own user agent allows people to easily filer in stats and suchlike.
Now should show 'CentOS makeisolist/9n '
Cheers
Lance
-- uklinux.net - The ISP of choice for the discerning Linux user.
We (http://mirror.anl.gov) would like to start mirroring the DVD ISOs for direct download, but I have failed to find instructions on how to include them when I sync.
How do I include DVD ISOs on my mirror site?
Thanks, -Brian
Lance Davis wrote:
On Sun, 15 Apr 2007, Kevin M Stange wrote:
Lance Davis wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
For a while there, we were the only mirror showing DVDs on the CentOS site. We had to throttle our bandwidth because we were pushing as much as 350 Mbps, mostly in DVD ISOs. Things have calmed down a bit from that high now.
Phew - thanks for that - I think there are now 4 US mirrors shown carrying dvd isos - and we are pushing to get more ...
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
Rebuilding the apr, apr-util, and httpd SRPMs from CentOS 5 or Fedora Devel will work for CentOS 4. Only catch, httpd includes an init script which requires a new version of initscripts. To work around it, you can install the src RPM, then grab your old /etc/init.d/httpd and replace /usr/src/redhat/SOURCE/httpd.init. Then you can set the dependency to require 7.93 of initscripts instead in the specfile and build. That's what we're doing currently.
aha
Regards Lance
Kevin
-- Kevin Stange Chief Technology Officer Steadfast Networks http://steadfast.net
Phone: 312-602-2689 x203 Fax: 312-602-2688 Cell: 312-320-5867
uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
To followup, I see that there is an active thread discussing the proper way to do this (and therefore know why I had a hard time finding it documented). So, I'll just sit tight and wait for a conclusion.
Thanks, -Brian
Brian Elliott Finley wrote:
We (http://mirror.anl.gov) would like to start mirroring the DVD ISOs for direct download, but I have failed to find instructions on how to include them when I sync.
How do I include DVD ISOs on my mirror site?
Thanks, -Brian
Lance Davis wrote:
On Sun, 15 Apr 2007, Kevin M Stange wrote:
Lance Davis wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
For a while there, we were the only mirror showing DVDs on the CentOS site. We had to throttle our bandwidth because we were pushing as much as 350 Mbps, mostly in DVD ISOs. Things have calmed down a bit from that high now.
Phew - thanks for that - I think there are now 4 US mirrors shown carrying dvd isos - and we are pushing to get more ...
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
Rebuilding the apr, apr-util, and httpd SRPMs from CentOS 5 or Fedora Devel will work for CentOS 4. Only catch, httpd includes an init script which requires a new version of initscripts. To work around it, you can install the src RPM, then grab your old /etc/init.d/httpd and replace /usr/src/redhat/SOURCE/httpd.init. Then you can set the dependency to require 7.93 of initscripts instead in the specfile and build. That's what we're doing currently.
aha
Regards Lance
Kevin
-- Kevin Stange Chief Technology Officer Steadfast Networks http://steadfast.net
Phone: 312-602-2689 x203 Fax: 312-602-2688 Cell: 312-320-5867
uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
Lance Davis wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
That may be due in part to it being hard to find and more readily available by Bt, but I think is also down to increased prevalance of dvd-rom drives.
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
(CentOS now has its first server up running C5 and able to display large dvd images under http :)
I have updated our isoredirect system to detect and verify dvd isos on mirrors , and sources with current dvd isos are now listed separately on the isoredirect page - what this is showing is that very few of our mirrors have the dvd isos and we need to get more out there ...
To test :-
http://isoredirect.centos.org/mirrorscripts/chooseisolist.pl?release=5.0&...
It will give you local info based on your ip you can also add &cc=xx to test other country codes
I know a lot of mirrors are wanting to download the dvd isos and we have just configured two new servers with msync-incdvd.
However, we would still like mirrors to do their initial sync of the dvd isos to one of our tier 1 mirrors that have them , and then we will add their ip addres to our acl for syncing.
We also have no problem if mirrors want to sync to a local tier 1 mirror, our system will still detect that they are up to date.
It is likely that in the near future we will restrict access to msync rsync to current registered mirrors, and new mirrors will either have to register first or do an initial sync from a registered mirror *hmm
- we have said this before - but now we have 160 mirrors we really
will have to do something ...)
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that :-
- provide rsync access to the complete CentOS release tree
- would be prepared to act as a seed for new mirrors
- would be prepared to set up separate CentOS and CentOS-incdvd
targets for other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
We will then publish details for other and new mirrors of how they can sync against these master sync servers.
If you would be prepared for your mirror to act as one, and have the bandwidth and capacity for it then please respond.
Whilst writing I would like to thank all of our mirrors for the tremendous support that you provide for CentOS and our users and especially during the release of 5.0
Regards Lance
-- uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
Evening All,
My apologies if this is OT.
We've been seeding the C5 dvd/cd isos for a couple of days now. After Lance sent his 11:35am email "Status of dvd iso for CentOS mirrors." mentioning torrent downloads, our bittorrent upload rates doubled ... coincidence or does Lance not know his effect on peoples behavior ... ;)
http://www.maia-internet.co.uk/mrtg/maia3_internet-day.png
Regards, Pete
Hi!
Ok, my mirror is synced up also ...
ftp://ftp.quicknet.nl/pub/Linux/www.centos.org/5.0
Its having cent 5, i386 and x86 + DVD images...
The machine is located inside the Netherlands and hooked up with dedicated gigabit onto the corenetwork there.
Bye, Raymond.
Evening All,
My apologies if this is OT.
We've been seeding the C5 dvd/cd isos for a couple of days now. After Lance sent his 11:35am email "Status of dvd iso for CentOS mirrors." mentioning torrent downloads, our bittorrent upload rates doubled ... coincidence or does Lance not know his effect on peoples behavior ... ;)
http://www.maia-internet.co.uk/mrtg/maia3_internet-day.png
Regards, Pete
Apologies for leaving my sig on ... :S
Regards, Pete
Le Sunday 15 April 2007, Lance Davis a écrit :
(CentOS now has its first server up running C5 and able to display large dvd images under http :)
I have updated our isoredirect system to detect and verify dvd isos on mirrors , and sources with current dvd isos are now listed separately on the isoredirect page - what this is showing is that very few of our mirrors have the dvd isos and we need to get more out there ...
To test :-
http://isoredirect.centos.org/mirrorscripts/chooseisolist.pl?release=5.0&... ch=i386 It will give you local info based on your ip you can also add &cc=xx to test other country codes
There is something unclear to me, you'll probably enlight me :)
My server (distrib-coffee) can host and share via ftp/http/rsync DVD iso, it already does it for mandriva (and it is running a mandriva 2007 64bits).
I can and I want to host and share CentOS DVD iso, however the page you gave don't show my server, so it seems I don't have DVD isos.
What should I do exactly to have the whole tree ? I use currently eu-msync.centos.org::CentOS/ as source.
Regards.
Le Monday 16 April 2007, Olivier Thauvin a écrit :
Le Sunday 15 April 2007, Lance Davis a écrit :
My server (distrib-coffee) can host and share via ftp/http/rsync DVD iso, it already does it for mandriva (and it is running a mandriva 2007 64bits).
distrib-coffee has now all DVD iso sync from Tier1 mirrors.
Regards.
Hello,
We can provide tier1 rsync access to DVD's. (mirrors.nfsi.pt)
Please let us know what are the right procedure to sync from upstreams.
We will then give rsync access aswell.
cheers. --nvieira
On Apr 15, 2007, at 11:35 AM, Lance Davis wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
That may be due in part to it being hard to find and more readily available by Bt, but I think is also down to increased prevalance of dvd-rom drives.
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
(CentOS now has its first server up running C5 and able to display large dvd images under http :)
I have updated our isoredirect system to detect and verify dvd isos on mirrors , and sources with current dvd isos are now listed separately on the isoredirect page - what this is showing is that very few of our mirrors have the dvd isos and we need to get more out there ...
To test :-
http://isoredirect.centos.org/mirrorscripts/chooseisolist.pl? release=5.0&arch=i386 It will give you local info based on your ip you can also add &cc=xx to test other country codes
I know a lot of mirrors are wanting to download the dvd isos and we have just configured two new servers with msync-incdvd.
However, we would still like mirrors to do their initial sync of the dvd isos to one of our tier 1 mirrors that have them , and then we will add their ip addres to our acl for syncing.
We also have no problem if mirrors want to sync to a local tier 1 mirror, our system will still detect that they are up to date.
It is likely that in the near future we will restrict access to msync rsync to current registered mirrors, and new mirrors will either have to register first or do an initial sync from a registered mirror *hmm - we have said this before - but now we have 160 mirrors we really will have to do something ...)
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that :-
- provide rsync access to the complete CentOS release tree
- would be prepared to act as a seed for new mirrors
- would be prepared to set up separate CentOS and CentOS-incdvd
targets for other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
We will then publish details for other and new mirrors of how they can sync against these master sync servers.
If you would be prepared for your mirror to act as one, and have the bandwidth and capacity for it then please respond.
Whilst writing I would like to thank all of our mirrors for the tremendous support that you provide for CentOS and our users and especially during the release of 5.0
Regards Lance
-- uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On Sun, 15 Apr 2007, Lance Davis wrote:
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that :-
- provide rsync access to the complete CentOS release tree
- would be prepared to act as a seed for new mirrors
- would be prepared to set up separate CentOS and CentOS-incdvd targets for
other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
mirror.chpc.utah.edu already has the DVD images and provides rsync access to the complete tree, and I would be willing to set up the separate rsync targets.
Dave
Kernel.org would is completely willing.
However would it not make more sense to just set up a second rsync target and explicitly exclude *DVD* or something akin to that so that those not wanting the DVD's got everything BUT the dvd isos? That might be a lot quicker and simpler than doing a hard linked set of trees.
As for letting other mirrors sync from the tier1's is there any consideration for authentication as it would be nice to just run, again, a separate target that can see everything in the centos directory but check either the incoming ip address or (as I currently have setup) a username/password combination so that only mirrors are using those targets and not the general public. If you did it via an ip list you could just have that sync with the rest of the Centos tree, if you did it via username/passwords you'd need to have the tier1's sync a separate target or something with that list.
- John 'Warthog9' Hawley Kernel.org Admin
On Mon, 2007-04-16 at 11:50 -0600, David Richardson wrote:
On Sun, 15 Apr 2007, Lance Davis wrote:
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that :-
- provide rsync access to the complete CentOS release tree
- would be prepared to act as a seed for new mirrors
- would be prepared to set up separate CentOS and CentOS-incdvd targets for
other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
mirror.chpc.utah.edu already has the DVD images and provides rsync access to the complete tree, and I would be willing to set up the separate rsync targets.
Dave
i would opt for an IP based ACL. ithink to set up usernames for every tier1 mirror separately is to bothersome. and if you have only one username/password to give to ALL tier1s, that just isn't good, imho.
J.H. wrote:
Kernel.org would is completely willing.
However would it not make more sense to just set up a second rsync target and explicitly exclude *DVD* or something akin to that so that those not wanting the DVD's got everything BUT the dvd isos? That might be a lot quicker and simpler than doing a hard linked set of trees.
As for letting other mirrors sync from the tier1's is there any consideration for authentication as it would be nice to just run, again, a separate target that can see everything in the centos directory but check either the incoming ip address or (as I currently have setup) a username/password combination so that only mirrors are using those targets and not the general public. If you did it via an ip list you could just have that sync with the rest of the Centos tree, if you did it via username/passwords you'd need to have the tier1's sync a separate target or something with that list.
- John 'Warthog9' Hawley
Kernel.org Admin
On Mon, 2007-04-16 at 11:50 -0600, David Richardson wrote:
On Sun, 15 Apr 2007, Lance Davis wrote:
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that :-
- provide rsync access to the complete CentOS release tree
- would be prepared to act as a seed for new mirrors
- would be prepared to set up separate CentOS and CentOS-incdvd targets for
other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
mirror.chpc.utah.edu already has the DVD images and provides rsync access to the complete tree, and I would be willing to set up the separate rsync targets.
Dave
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On Mon, 16 Apr 2007, J.H. wrote:
Kernel.org would is completely willing.
However would it not make more sense to just set up a second rsync target and explicitly exclude *DVD* or something akin to that so that those not wanting the DVD's got everything BUT the dvd isos? That might be a lot quicker and simpler than doing a hard linked set of trees.
Yes, that is fine. We hardlink the trees for other reasons (although they escape me at the moment :)
As for letting other mirrors sync from the tier1's is there any consideration for authentication as it would be nice to just run, again, a separate target that can see everything in the centos directory but check either the incoming ip address or (as I currently have setup) a username/password combination so that only mirrors are using those targets and not the general public. If you did it via an ip list you could just have that sync with the rest of the Centos tree, if you did it via username/passwords you'd need to have the tier1's sync a separate target or something with that list.
Well the only reason not too was that you allow public rsync access :)
I guess that would mean having specific rsync targets for mirrors to sync eg ::CentOS-sync-nodvd ::CentOS-sync-incdvd as opposed to the public ::CentOS
But what would be the difference ?? - ok I guess they are more specific as to what you are going to get, and you may be able to give a different bandwidth path ??
Plus it may be useful for logging purposes.
I am happy to provide a list of ips that are authorised to sync those targets - would that work for you ??
Regards Lance
- John 'Warthog9' Hawley
Kernel.org Admin
On Mon, 2007-04-16 at 11:50 -0600, David Richardson wrote:
On Sun, 15 Apr 2007, Lance Davis wrote:
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that :-
- provide rsync access to the complete CentOS release tree
- would be prepared to act as a seed for new mirrors
- would be prepared to set up separate CentOS and CentOS-incdvd targets for
other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
mirror.chpc.utah.edu already has the DVD images and provides rsync access to the complete tree, and I would be willing to set up the separate rsync targets.
Dave
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
uklinux.net - The ISP of choice for the discerning Linux user.
On Tue, 2007-04-17 at 03:08 +0100, Lance Davis wrote:
On Mon, 16 Apr 2007, J.H. wrote:
Kernel.org would is completely willing.
However would it not make more sense to just set up a second rsync target and explicitly exclude *DVD* or something akin to that so that those not wanting the DVD's got everything BUT the dvd isos? That might be a lot quicker and simpler than doing a hard linked set of trees.
Yes, that is fine. We hardlink the trees for other reasons (although they escape me at the moment :)
As for letting other mirrors sync from the tier1's is there any consideration for authentication as it would be nice to just run, again, a separate target that can see everything in the centos directory but check either the incoming ip address or (as I currently have setup) a username/password combination so that only mirrors are using those targets and not the general public. If you did it via an ip list you could just have that sync with the rest of the Centos tree, if you did it via username/passwords you'd need to have the tier1's sync a separate target or something with that list.
Well the only reason not too was that you allow public rsync access :)
I guess that would mean having specific rsync targets for mirrors to sync eg ::CentOS-sync-nodvd ::CentOS-sync-incdvd as opposed to the public ::CentOS
Basically
But what would be the difference ?? - ok I guess they are more specific as to what you are going to get, and you may be able to give a different bandwidth path ??
The difference being, when releases happen the public doesn't see what's being released, where as the mirrors need to be able to see it. I.E.
rw------- <user> <group> centos/5.0
Would still be invisible during mirror propagation (rsync user is set as a generic user without permission to access the folder) while the mirrors connecting would be able to. That is the main difference, otherwise 99% of the time the two would return the same data. Also means that I can have two separate pools of connections say 10 - 20 slots for mirrors and say a hundred for normal downloading which gives a mirror a MUCH better chance of getting into and downloading things vs. the general public.
Plus it may be useful for logging purposes.
True.
I am happy to provide a list of ips that are authorised to sync those targets - would that work for you ??
It would work for me, it might be useful to have those in a file that also gets synced with centos (in the root directory) so that the other mirrors can take advantage of the same thing?
Anyway - those are just my $0.02 anyway.
- John 'Warthog9' Hawley Kernel.org
Regards Lance
- John 'Warthog9' Hawley
Kernel.org Admin
On Mon, 2007-04-16 at 11:50 -0600, David Richardson wrote:
On Sun, 15 Apr 2007, Lance Davis wrote:
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that :-
- provide rsync access to the complete CentOS release tree
- would be prepared to act as a seed for new mirrors
- would be prepared to set up separate CentOS and CentOS-incdvd targets for
other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
mirror.chpc.utah.edu already has the DVD images and provides rsync access to the complete tree, and I would be willing to set up the separate rsync targets.
Dave
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
J.H. wrote:
Basically
But what would be the difference ?? - ok I guess they are more specific as to what you are going to get, and you may be able to give a different bandwidth path ??
The difference being, when releases happen the public doesn't see what's being released, where as the mirrors need to be able to see it. I.E.
rw------- <user> <group> centos/5.0
Would still be invisible during mirror propagation (rsync user is set as a generic user without permission to access the folder) while the mirrors connecting would be able to. That is the main difference, otherwise 99% of the time the two would return the same data. Also means that I can have two separate pools of connections say 10 - 20 slots for mirrors and say a hundred for normal downloading which gives a mirror a MUCH better chance of getting into and downloading things vs. the general public.
FWIW, please use drwxr-x--- as the mode for "bit not yet flipped"; otherwise rsync has to run as the mirroring user to allow access for mirrors.
-hpa
Lance Davis wrote:
On Mon, 16 Apr 2007, J.H. wrote:
Kernel.org would is completely willing.
However would it not make more sense to just set up a second rsync target and explicitly exclude *DVD* or something akin to that so that those not wanting the DVD's got everything BUT the dvd isos? That might be a lot quicker and simpler than doing a hard linked set of trees.
Yes, that is fine. We hardlink the trees for other reasons (although they escape me at the moment :)
Both are pretty ridiculous, quite frankly.
Settle on a standard --exclude pattern, and you don't need two trees of any sort.
Currently, I believe --exclude *-DVD.iso should work, but --exclude *-DVD*.iso might be a better (future-proof) pattern.
One can easily export two different rsync modules with and without DVD images, if that's at all desirable, simply by putting the exclude pattern in rsyncd.conf.
-hpa
On tir, 17 apr 2007, H. Peter Anvin wrote:
Both are pretty ridiculous, quite frankly.
Settle on a standard --exclude pattern, and you don't need two trees of any sort.
Currently, I believe --exclude *-DVD.iso should work, but --exclude *-DVD*.iso might be a better (future-proof) pattern.
One can easily export two different rsync modules with and without DVD images, if that's at all desirable, simply by putting the exclude pattern in rsyncd.conf.
I totaly agree. It's simpler for CentOS to administrer ONE single repository, and easier for all of us have one repository to deal with. Most of us have enough disk space for the DVD isos anyway. Those who don't, can use a exclude pattern - which can be documentet on the CentOS mirroring howto page:
http://www.centos.org/modules/tinycontent/index.php?id=22
It more KISS and most Linux distributions do this, so why can't CentOS?
On Wed, 2007-04-18 at 09:05 +0200, Lars Strand wrote:
On tir, 17 apr 2007, H. Peter Anvin wrote:
Both are pretty ridiculous, quite frankly.
Settle on a standard --exclude pattern, and you don't need two trees of any sort.
Currently, I believe --exclude *-DVD.iso should work, but --exclude *-DVD*.iso might be a better (future-proof) pattern.
One can easily export two different rsync modules with and without DVD images, if that's at all desirable, simply by putting the exclude pattern in rsyncd.conf.
I totaly agree. It's simpler for CentOS to administrer ONE single repository, and easier for all of us have one repository to deal with. Most of us have enough disk space for the DVD isos anyway. Those who don't, can use a exclude pattern - which can be documentet on the CentOS mirroring howto page:
http://www.centos.org/modules/tinycontent/index.php?id=22
It more KISS and most Linux distributions do this, so why can't CentOS?
Well ... we have several (20-30) internal centos mirrors that we use to distribute the CentOS to the world ... and those mirrors do not all have the DVD on them because we do not have the drive space available on those donated servers to put all the DVDs on. We do not want to loose half of our mirror bandwidth because we put DVDs in the mainline tree.
As for external mirrors, I think it would be nice for you to provide 2 targets (one with and one without DVDs) if you carry the DVDs, but it should not be hard to tell people to exclude DVDs if they don't want them.
In a perfect world, CentOS could just have one tree because all our internal servers would have > 1TB of storage space and everything would fit on each mirror ... in the real world our internal servers are donated and we get whatever the donor has to offer and we use it in as many ways as we can to distribute CentOS to millions of people :P
I hope this makes it easier to understand why the DVDs are a subset on the CentOS mirrors at least.
Thanks, Johnny Hughes
Hi Johnny,
Can you send me some samples of how to mirror the DVD tree ? (i believe that other persons in this list, with the same problem, will thank aswell)
If needed, our IP is 81.92.200.26
thanks in advance.
cheers, --nvieira
On Apr 18, 2007, at 9:23 AM, Johnny Hughes wrote:
On Wed, 2007-04-18 at 09:05 +0200, Lars Strand wrote:
On tir, 17 apr 2007, H. Peter Anvin wrote:
Both are pretty ridiculous, quite frankly.
Settle on a standard --exclude pattern, and you don't need two trees of any sort.
Currently, I believe --exclude *-DVD.iso should work, but --exclude *-DVD*.iso might be a better (future-proof) pattern.
One can easily export two different rsync modules with and without DVD images, if that's at all desirable, simply by putting the exclude pattern in rsyncd.conf.
I totaly agree. It's simpler for CentOS to administrer ONE single repository, and easier for all of us have one repository to deal with. Most of us have enough disk space for the DVD isos anyway. Those who don't, can use a exclude pattern - which can be documentet on the CentOS mirroring howto page:
http://www.centos.org/modules/tinycontent/index.php?id=22
It more KISS and most Linux distributions do this, so why can't CentOS?
Well ... we have several (20-30) internal centos mirrors that we use to distribute the CentOS to the world ... and those mirrors do not all have the DVD on them because we do not have the drive space available on those donated servers to put all the DVDs on. We do not want to loose half of our mirror bandwidth because we put DVDs in the mainline tree.
As for external mirrors, I think it would be nice for you to provide 2 targets (one with and one without DVDs) if you carry the DVDs, but it should not be hard to tell people to exclude DVDs if they don't want them.
In a perfect world, CentOS could just have one tree because all our internal servers would have > 1TB of storage space and everything would fit on each mirror ... in the real world our internal servers are donated and we get whatever the donor has to offer and we use it in as many ways as we can to distribute CentOS to millions of people :P
I hope this makes it easier to understand why the DVDs are a subset on the CentOS mirrors at least.
Thanks, Johnny Hughes _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
Hello Lance,
Can you please clarify how can i sync the dvd's from ?
I am trying to use eu-msync.centos.org::::CentOS-sync-incdvd and msync.centos.org::::CentOS-sync-incdvd without any success.
regards, --nvieira
On Apr 17, 2007, at 3:08 AM, Lance Davis wrote:
::CentOS-sync-incdvd
I agree with hpa on the --exclude pattern bit.
-Brian
Nuno Vieira - nfsi telecom wrote:
Hello Lance,
Can you please clarify how can i sync the dvd's from ?
I am trying to use eu-msync.centos.org::::CentOS-sync-incdvd and msync.centos.org::::CentOS-sync-incdvd without any success.
regards, --nvieira
On Apr 17, 2007, at 3:08 AM, Lance Davis wrote:
::CentOS-sync-incdvd
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
I'll second nvieira's question -- after all this discussion, what is the proper URI for syncing, including the DVD ISOs? We're currently syncing everything else just fine.
If you need IPs for acls, please use these:
146.137.96.12 146.137.96.64
Either machine might do the syncing, but only one is configured as the active sync machine on any given day.
-Brian
Nuno Vieira - nfsi telecom wrote:
Hello Lance,
Can you please clarify how can i sync the dvd's from ?
I am trying to use eu-msync.centos.org::::CentOS-sync-incdvd and msync.centos.org::::CentOS-sync-incdvd without any success.
regards, --nvieira
On Apr 17, 2007, at 3:08 AM, Lance Davis wrote:
::CentOS-sync-incdvd
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On Sun, 15 Apr 2007, Lance Davis wrote:
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that : -
- provide rsync access to the complete CentOS release tree
- would be prepared to act as a seed for new mirrors
- would be prepared to set up separate CentOS and CentOS-incdvd targets
for other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
I have finally configured this on mirror.chpc.utah.edu. I now have three CentOS-specific rsync targets: * CentOS and CentOS-nodvd, which exclude the DVD images (by using the line "exclude = *DVD*.iso *dvd*.iso" in rsyncd.conf) * CentOS-incdvd, which includes the DVD images.
Everyone is welcome to sync from me.
David Richardson wrote:
I have finally configured this on mirror.chpc.utah.edu. I now have three CentOS-specific rsync targets:
- CentOS and CentOS-nodvd, which exclude the DVD images (by using the line "exclude = *DVD*.iso *dvd*.iso" in rsyncd.conf)
- CentOS-incdvd, which includes the DVD images.
Everyone is welcome to sync from me.
mirrors.kernel.org now has the same, except that our "centos" module is the complete set since that was our previous configuration, and rsync modules are apparently case insensitive.
-hpa
mirrorservice.org would like to participate, as a tier one mirror for europe.
Thanks
Phill.
On Sun, 15 Apr 2007, Lance Davis wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
That may be due in part to it being hard to find and more readily available by Bt, but I think is also down to increased prevalance of dvd-rom drives.
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
(CentOS now has its first server up running C5 and able to display large dvd images under http :)
I have updated our isoredirect system to detect and verify dvd isos on mirrors , and sources with current dvd isos are now listed separately on the isoredirect page - what this is showing is that very few of our mirrors have the dvd isos and we need to get more out there ...
To test :-
http://isoredirect.centos.org/mirrorscripts/chooseisolist.pl?release=5.0&... It will give you local info based on your ip you can also add &cc=xx to test other country codes
I know a lot of mirrors are wanting to download the dvd isos and we have just configured two new servers with msync-incdvd.
However, we would still like mirrors to do their initial sync of the dvd isos to one of our tier 1 mirrors that have them , and then we will add their ip addres to our acl for syncing.
We also have no problem if mirrors want to sync to a local tier 1 mirror, our system will still detect that they are up to date.
It is likely that in the near future we will restrict access to msync rsync to current registered mirrors, and new mirrors will either have to register first or do an initial sync from a registered mirror *hmm - we have said this before - but now we have 160 mirrors we really will have to do something ...)
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that :-
- provide rsync access to the complete CentOS release tree
- would be prepared to act as a seed for new mirrors
- would be prepared to set up separate CentOS and CentOS-incdvd targets
for other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
We will then publish details for other and new mirrors of how they can sync against these master sync servers.
If you would be prepared for your mirror to act as one, and have the bandwidth and capacity for it then please respond.
Whilst writing I would like to thank all of our mirrors for the tremendous support that you provide for CentOS and our users and especially during the release of 5.0
Regards Lance
-- uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On Sun, Apr 15, 2007 at 11:35:58AM +0100, Lance Davis wrote:
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that :-
- provide rsync access to the complete CentOS release tree
- would be prepared to act as a seed for new mirrors
- would be prepared to set up separate CentOS and CentOS-incdvd targets
for other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
We will then publish details for other and new mirrors of how they can sync against these master sync servers.
If you would be prepared for your mirror to act as one, and have the bandwidth and capacity for it then please respond.
merlin.fit.vutbr.cz would like to participate, with that we added rsync target so new info is:
Full mirror - all versions and archs
http http://merlin.fit.vutbr.cz/mirrors/centos/ rsync rsync://merlin.fit.vutbr.cz/CentOS /CentOS-incdvd merlin.fit.vutbr.cz::CentOS ::CentOS-incdvd Sync: hourly Location: Brno, Czech Republic, EU Sponsor: Brno University of Technology, http://www.fit.vutbr.cz/ Contact: linux@fit.vutbr.cz Speed: Server 1GbE, backbone 10GbE to Geant2
I differentiate the targest for rsync just by adding exclude=*-DVD.iso so the part of config looks like
[CentOS] comment="CentOS without DVDs" path=..../mirrors/centos exclude=*-DVD.iso
[CentOS-incdvd] comment="CentOS with DVDs" path=..../mirrors/centos
is this ok?
Regards
--
Tomas Kasparek, PhD student E-mail: kasparek@fit.vutbr.cz CVT FIT VUT Brno, L127 Web: http://www.fit.vutbr.cz/~kasparek Bozetechova 1, 612 66 Fax: +420 54114-1270 Brno, Czech Republic Phone: +420 54114-1220
ICQ: 293092805 jabber: tomas.kasparek@jabber.cz GPG: 2F1E 1AAF FD3B CFA3 1537 63BD DCBE 18FF A035 53BC
On Tue, Apr 17, 2007 at 09:17:48AM +0200, Kasparek Tomas wrote:
merlin.fit.vutbr.cz would like to participate, with that we added rsync target so new info is:
Full mirror - all versions and archs
http http://merlin.fit.vutbr.cz/mirrors/centos/ rsync rsync://merlin.fit.vutbr.cz/CentOS /CentOS-incdvd merlin.fit.vutbr.cz::CentOS ::CentOS-incdvd Sync: hourly Location: Brno, Czech Republic, EU Sponsor: Brno University of Technology, http://www.fit.vutbr.cz/ Contact: linux@fit.vutbr.cz Speed: Server 1GbE, backbone 10GbE to Geant2
I differentiate the targest for rsync just by adding exclude=*-DVD.iso so the part of config looks like
[CentOS] comment="CentOS without DVDs" path=..../mirrors/centos exclude=*-DVD.iso
[CentOS-incdvd] comment="CentOS with DVDs" path=..../mirrors/centos
is this ok?
Hi, please can you update info for merlin.fit.vutbr.cz with rsync taget (it does probably get missed by admins previously).
Thanks.
--
Tomas Kasparek, PhD student E-mail: kasparek@fit.vutbr.cz CVT FIT VUT Brno, L127 Web: http://www.fit.vutbr.cz/~kasparek Bozetechova 1, 612 66 Fax: +420 54114-1270 Brno, Czech Republic Phone: +420 54114-1220
ICQ: 293092805 jabber: tomas.kasparek@jabber.cz GPG: 2F1E 1AAF FD3B CFA3 1537 63BD DCBE 18FF A035 53BC
On 4/15/07, Lance Davis lance@uklinux.net wrote:
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
That may be due in part to it being hard to find and more readily available by Bt, but I think is also down to increased prevalance of dvd-rom drives.
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
(CentOS now has its first server up running C5 and able to display large dvd images under http :)
I have updated our isoredirect system to detect and verify dvd isos on mirrors , and sources with current dvd isos are now listed separately on the isoredirect page - what this is showing is that very few of our mirrors have the dvd isos and we need to get more out there ...
To test :-
http://isoredirect.centos.org/mirrorscripts/chooseisolist.pl?release=5.0&...
It will give you local info based on your ip you can also add &cc=xx to test other country codes
I know a lot of mirrors are wanting to download the dvd isos and we have just configured two new servers with msync-incdvd.
However, we would still like mirrors to do their initial sync of the dvd isos to one of our tier 1 mirrors that have them , and then we will add their ip addres to our acl for syncing.
We also have no problem if mirrors want to sync to a local tier 1 mirror, our system will still detect that they are up to date.
It is likely that in the near future we will restrict access to msync rsync to current registered mirrors, and new mirrors will either have to register first or do an initial sync from a registered mirror *hmm - we have said this before - but now we have 160 mirrors we really will have to do something ...)
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that :-
- provide rsync access to the complete CentOS release tree
- would be prepared to act as a seed for new mirrors
- would be prepared to set up separate CentOS and CentOS-incdvd targets
for other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
I've finished sync'ing the DVD ISOs from the mirror.kernel.org tier 1 mirror. Should I continue syncing from this mirror or are you able to provide access to the CentOS server (and if so, what is the target)?
This is for mirror.daemonbox.net - IP 66.246.140.174
-David
I looked at the centos download status page, and klid.dk is listed as not directly downloadable for dvd. But actually you can download dvds from us - also above the 2 GB limit, via our ftp server. It will not work for http, however, for the time being. Maybe then klid.dk should be listed as having capability for direct download of dvd.
best regards keld
Yes it would be nice for mirror owner to be able to access the error log of the iso checker we seem to have dropped off again... not suer why we are under high load due to another distro release a could see exactly where we droppepd off.
Also how often does the list refresh?
as for iso's via ftp only try a variant on the following in apache. note the the RewriteCond file test stuff is broken for large files and always fails
Try A variant on the following
RewriteCond %{REQUEST_URI} .*(DVD|dvd).*.iso$ RewriteRule ^(/sites/.*.iso)$ ftp://ftp.mirrorservice.org$1 [redirect,last]
Phill.
On Thu, 19 Apr 2007, Keld [iso-8859-1] J�rn Simonsen wrote:
I looked at the centos download status page, and klid.dk is listed as not directly downloadable for dvd. But actually you can download dvds from us - also above the 2 GB limit, via our ftp server. It will not work for http, however, for the time being. Maybe then klid.dk should be listed as having capability for direct download of dvd.
best regards keld _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
I feel like I've beaten this salami-ingredient for a while now but I persist.
I continue to be unclear as to just where we are supposed to point to sync for the CentOS-incdvd target. We have the DVD isos, but we have to pull them manually from the torrents, copy them in place, exclude them from the rsync, and obviously all this manual work is aggravating and error prone. All the sync servers bandied about here in the past give me no joy: Unknown module 'CentOS-incdvd'. We're a public mirror, but do not provide rsync. The IPs are still 131.247.254.5 and 2001:468:1601:11fe::2.
And also, to add to the beating of the dead horse, please change us to ftp.usf.edu from ftp.net.usf.edu, and add HTTP. Please? Pretty please?
All of this is *still* wrong on http://www.centos.org/modules/tinycontent/index.php?id=13 and http://mirror-status.centos.org/
-- Toivo Voll Data Network Management Academic Computing University of South Florida -----Original Message----- From: centos-mirror-bounces@centos.org [mailto:centos-mirror-bounces@centos.org] On Behalf Of Lance Davis Sent: Sunday, April 15, 2007 06:36 To: centos-mirror@centos.org Subject: [CentOS-mirror] Status of dvd iso for CentOS mirrors.
With the release of 5 we have seen a tremendous increase in the use of the dvd.iso , so that from bitorrent it is now downloaded 5 times more than the cd isos.
That may be due in part to it being hard to find and more readily available by Bt, but I think is also down to increased prevalance of dvd-rom drives.
The main issue with having the dvd isos available from mirrors is that Apache < 2.2 running on 32 bit architecture wont display files larger than 2.0 GB , however CentOS 5 has Apache 2.2 with large file support so that problem will now start to go away - alternative is also to use lighttpd.
(CentOS now has its first server up running C5 and able to display large dvd images under http :)
I have updated our isoredirect system to detect and verify dvd isos on mirrors , and sources with current dvd isos are now listed separately on the isoredirect page - what this is showing is that very few of our mirrors have the dvd isos and we need to get more out there ...
To test :-
http://isoredirect.centos.org/mirrorscripts/chooseisolist.pl?release=5.0&... It will give you local info based on your ip you can also add &cc=xx to test other country codes
I know a lot of mirrors are wanting to download the dvd isos and we have just configured two new servers with msync-incdvd.
However, we would still like mirrors to do their initial sync of the dvd isos to one of our tier 1 mirrors that have them , and then we will add their ip addres to our acl for syncing.
We also have no problem if mirrors want to sync to a local tier 1 mirror, our system will still detect that they are up to date.
It is likely that in the near future we will restrict access to msync rsync to current registered mirrors, and new mirrors will either have to register first or do an initial sync from a registered mirror *hmm - we have said this before - but now we have 160 mirrors we really will have to do something ...)
There has been some confusion about the CentOS and CentOS-incdvd rsync targets , and the fact that some mirrors have dvd isos included in the CentOS target, so what we would like to do is to have a list of tier 1 mirrors that :-
1. provide rsync access to the complete CentOS release tree 2. would be prepared to act as a seed for new mirrors 3. would be prepared to set up separate CentOS and CentOS-incdvd targets for other mirrors to sync against, either initial or recurring syncs (the trees would be hardlinked, so extra space required is minimal).
We will then publish details for other and new mirrors of how they can sync against these master sync servers.
If you would be prepared for your mirror to act as one, and have the bandwidth and capacity for it then please respond.
Whilst writing I would like to thank all of our mirrors for the tremendous support that you provide for CentOS and our users and especially during the release of 5.0
Regards Lance
-- uklinux.net - The ISP of choice for the discerning Linux user. _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
-
archive-admin＠mirrorservice.org -
Brian Elliott Finley -
David Richardson -
David Stanford -
H. Peter Anvin -
Imre Gergely -
J.H. -
Johnny Hughes -
Kasparek Tomas -
Keld Jørn Simonsen -
Kevin M Stange -
Lance Davis -
Lance Davis -
Lars Strand -
Nuno Vieira - nfsi telecom -
Olivier Thauvin -
Peter New -
Raymond Dijkxhoorn -
Vaclav Dusek -
Voll, Toivo