Okay, I'm confused.
The CentOS mirror on mirrors.kernel.org is missing at least all the DVD isos, and all there is there is a .torrent file.
We have tons of disk space and bandwidth, and denying users the ability to use the mirror network for these probably hurts more than it helps (we are consistently faster than Bittorrent during Fedora Core releases, for example, and that is an extremely popular fileset during peak traffic.)
Can these be re-enabled, at least for the mirror sites who wants to carry them?
-hpa
On Tue, 2 May 2006, H. Peter Anvin wrote:
Okay, I'm confused.
The CentOS mirror on mirrors.kernel.org is missing at least all the DVD isos, and all there is there is a .torrent file.
We have tons of disk space and bandwidth, and denying users the ability to use the mirror network for these probably hurts more than it helps (we are consistently faster than Bittorrent during Fedora Core releases, for example, and that is an extremely popular fileset during peak traffic.)
Can these be re-enabled, at least for the mirror sites who wants to carry them?
Thanks for the offer - yes it would be good to serve the dvd isos out to mirrors and we have been considering how to do it.
They have never been mirrored previously.
One problem is that Apache historically doesnt cope with files larger than 2GB , and just refuses to display a directory listing at all when such files are present.
We also have to consider a possible doubling of the traffic generated for our mirror servers during a new release - syncing out to mirrors.
But if those issues can be resolved then we can produce an rsync module for the dvd isos.
We will discuss internally ...
Regards Lance
-- uklinux.net - The ISP of choice for the discerning Linux user.
On Tue, May 02, 2006 at 05:00:29PM +0100, Lance Davis wrote:
We also have to consider a possible doubling of the traffic generated for our mirror servers during a new release - syncing out to mirrors.
You can give access to the dvd-iso rsync module, only to one tier1 mirror with enough bandwidth, and initially all other mirrors can use the tier1 mirror to sync from.
But if those issues can be resolved then we can produce an rsync module for the dvd isos.
We will discuss internally ...
Regards Lance
-- uklinux.net - The ISP of choice for the discerning Linux user.
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On 02 May 2006, Lance Davis wrote:
One problem is that Apache historically doesnt cope with files larger than 2GB , and just refuses to display a directory listing at all when such files are present.
Out of curiosity, what are other people running for their web server? I'm running Apache 2.2 specifically because I mirror some things with DVD images which break the 2GB boundary. I was running Cherokee previously, but found Apache better suited for my needs. But I'm curious how other, larger mirrors are handling this.
Mark Nipper wrote:
On 02 May 2006, Lance Davis wrote:
One problem is that Apache historically doesnt cope with files larger than 2GB , and just refuses to display a directory listing at all when such files are present.
Out of curiosity, what are other people running for theirweb server? I'm running Apache 2.2 specifically because I mirror some things with DVD images which break the 2GB boundary. I was running Cherokee previously, but found Apache better suited for my needs. But I'm curious how other, larger mirrors are handling this.
Running 1.3.27.
Why? That's what it was running when I inherited it. I could upgrade to 2.x probably without much headache; the machine doesn't do anything else but mirror, and doesn't need php/mod_perl/wha'eva.
If you guys at Centos need us to do that, though, give us a couple of days notice.
On Wed, 2006-05-03 at 08:40 -0400, William Dunn wrote:
Mark Nipper wrote:
On 02 May 2006, Lance Davis wrote:
One problem is that Apache historically doesnt cope with files larger than 2GB , and just refuses to display a directory listing at all when such files are present.
Out of curiosity, what are other people running for theirweb server? I'm running Apache 2.2 specifically because I mirror some things with DVD images which break the 2GB boundary. I was running Cherokee previously, but found Apache better suited for my needs. But I'm curious how other, larger mirrors are handling this.
Running 1.3.27.
Why? That's what it was running when I inherited it. I could upgrade to 2.x probably without much headache; the machine doesn't do anything else but mirror, and doesn't need php/mod_perl/wha'eva.
If you guys at Centos need us to do that, though, give us a couple of days notice.
2.0.x won't work ... it would have to be 2.2.x
we don't have that for centos now ... so it gets harder
that is the main reason why we don't pass out the DVDs currently.
we are working to make this happen though .. so people who want the DVDs will have an rsync target where they can get them.
Mark Nipper wrote:
On 02 May 2006, Lance Davis wrote:
One problem is that Apache historically doesnt cope with files larger than 2GB , and just refuses to display a directory listing at all when such files are present.
Out of curiosity, what are other people running for theirweb server? I'm running Apache 2.2 specifically because I mirror some things with DVD images which break the 2GB boundary. I was running Cherokee previously, but found Apache better suited for my needs. But I'm curious how other, larger mirrors are handling this.
Using 2.0 on kernel.org, but on a 64-bit box.
2.0 doesn't have an inherent problem with large files, but I gather RedHat didn't want to recompile it in such a way to break all the plugins on 32-bit boxes.
-hpa
H. Peter Anvin wrote:
(we are consistently faster than Bittorrent during Fedora Core releases, for example, and that is an extremely popular fileset during peak traffic.)
With the CentOS releases, we've never had that sort of a problem with .torrents - eg. at time of 4.3 Release, we had > 2 GB/sec offering torrent seeds by over 20 machines, each one sitting on > 100mbps links to the internet.
But, not everyone can use torrents, and it would indeed be a good thing to have some of the mirrors offer it as a download.
Karanbir Singh wrote:
H. Peter Anvin wrote:
(we are consistently faster than Bittorrent during Fedora Core releases, for example, and that is an extremely popular fileset during peak traffic.)
With the CentOS releases, we've never had that sort of a problem with .torrents - eg. at time of 4.3 Release, we had > 2 GB/sec offering torrent seeds by over 20 machines, each one sitting on > 100mbps links to the internet.
But, not everyone can use torrents, and it would indeed be a good thing to have some of the mirrors offer it as a download.
Well, we (kernel.org) would like to offer them, but we'd have to be served them in the first place.
Similarly, we're perfectly happy -- in fact, we encourage -- keeping the entire historical archive, and would rather appreciate it if we could get an downlink which didn't "helpfully" clean up our disks...
-hpa
On Wed, 3 May 2006, H. Peter Anvin wrote:
Well, we (kernel.org) would like to offer them, but we'd have to be served them in the first place.
OK - they are now available - initially from a single machine - but it should be on quite a fast connection. We will sync it to some other servers as well.
This will add 25GB of dvd.iso files to CentOS mirror payload. Bear in mind caveats previously discussed ...
If people are mirroring this please let us know and we will indicate availability of dvd iso files on CentOS website.
The new rsync target is :- rsync msync-dvd.centos.org::CentOS-incdvd
Similarly, we're perfectly happy -- in fact, we encourage -- keeping the entire historical archive, and would rather appreciate it if we could get an downlink which didn't "helpfully" clean up our disks...
Ok - we can make the vault available to you, however at the moment we only have it as a separate tree.
We can merge the two and make the whole available as an rsync target if you wish ...
Regards Lance
Lance Davis wrote:
On Wed, 3 May 2006, H. Peter Anvin wrote:
Well, we (kernel.org) would like to offer them, but we'd have to be served them in the first place.
OK - they are now available - initially from a single machine - but it should be on quite a fast connection. We will sync it to some other servers as well.
This will add 25GB of dvd.iso files to CentOS mirror payload. Bear in mind caveats previously discussed ...
If people are mirroring this please let us know and we will indicate availability of dvd iso files on CentOS website.
The new rsync target is :- rsync msync-dvd.centos.org::CentOS-incdvd
I presume this is *instead of* us-msync.centos.org::CentOS?
Similarly, we're perfectly happy -- in fact, we encourage -- keeping the entire historical archive, and would rather appreciate it if we could get an downlink which didn't "helpfully" clean up our disks...
Ok - we can make the vault available to you, however at the moment we only have it as a separate tree.
We can merge the two and make the whole available as an rsync target if you wish ...
That would be preferred, but we can also mirror more than one target.
-hpa
H. Peter Anvin wrote:
Lance Davis wrote:
On Wed, 3 May 2006, H. Peter Anvin wrote:
Well, we (kernel.org) would like to offer them, but we'd have to be served them in the first place.
OK - they are now available - initially from a single machine - but it should be on quite a fast connection. We will sync it to some other servers as well.
This will add 25GB of dvd.iso files to CentOS mirror payload. Bear in mind caveats previously discussed ...
If people are mirroring this please let us know and we will indicate availability of dvd iso files on CentOS website.
The new rsync target is :- rsync msync-dvd.centos.org::CentOS-incdvd
I presume this is *instead of* us-msync.centos.org::CentOS?
Eep. I just started pulling this target, and I'm getting tons and tons of:
file has vanished: "/3.6/updates/s390/headers/ethereal-gnome-0-0.10.12-1.EL3.1.s390.hdr" (in CentOS-incdvd) file has vanished: "/3.6/updates/s390/headers/ethereal-gnome-0-0.10.13-1.EL3.1.s390.hdr" (in CentOS-incdvd) file has vanished: "/3.6/updates/s390/headers/ethereal-gnome-0-0.10.14-1.EL3.1.s390.hdr" (in CentOS-incdvd) file has vanished:
On Wed, 3 May 2006, H. Peter Anvin wrote:
H. Peter Anvin wrote:
Lance Davis wrote:
On Wed, 3 May 2006, H. Peter Anvin wrote:
Well, we (kernel.org) would like to offer them, but we'd have to be served them in the first place.
OK - they are now available - initially from a single machine - but it should be on quite a fast connection. We will sync it to some other servers as well.
This will add 25GB of dvd.iso files to CentOS mirror payload. Bear in mind caveats previously discussed ...
If people are mirroring this please let us know and we will indicate availability of dvd iso files on CentOS website.
The new rsync target is :- rsync msync-dvd.centos.org::CentOS-incdvd
I presume this is *instead of* us-msync.centos.org::CentOS?
Eep. I just started pulling this target, and I'm getting tons and tons of:
file has vanished: "/3.6/updates/s390/headers/ethereal-gnome-0-0.10.12-1.EL3.1.s390.hdr" (in CentOS-incdvd) file has vanished: "/3.6/updates/s390/headers/ethereal-gnome-0-0.10.13-1.EL3.1.s390.hdr" (in CentOS-incdvd) file has vanished: "/3.6/updates/s390/headers/ethereal-gnome-0-0.10.14-1.EL3.1.s390.hdr" (in CentOS-incdvd) file has vanished:
Sorry - that has now been fixed - it was the result of the way the tree was being synced.
Regards Lance
Ok so
storage +~25G
and new target
msync-dvd.centos.org::CentOS-incdvd
includes everything + the dvd images
and
we need to mail to say we nclude the images?
Phill.
On Fri, May 05, 2006 at 12:07:26AM +0100, archive-admin@mirrorservice.org wrote:
storage +~25G
that's just about 6DVDs, I would expect more...
and new target
replace (not add) eu-msync.centos.org::CentOS with this, or not?
msync-dvd.centos.org::CentOS-incdvd
includes everything + the dvd images
DVD images are for 3.7 and 4.3 only (for now?) is it ok? (maybe it is a good thing to shorten the request for space, does anyone ever need .iso of older releases, from my logs it seems that this is really minority of people (I expect some really special cases) )
we need to mail to say we nclude the images?
I don't see such request yet, but if so, merlin.fit.vutbr.cz will be mirroring DVD isos.
How is syncing between several rsync tagets done (mainly updates, but new releases too)?
Thanks.
--
Tomas Kasparek, PhD student E-mail: kasparek@fit.vutbr.cz CVT FIT VUT Brno, BI/140a Web: http://www.fit.vutbr.cz/~kasparek Bozetechova 2, 612 66 Fax: +420 54114-1270 Brno, Czech Republic Phone: +420 54114-1220
ICQ: 293092805 jabber: tomas.kasparek@jabber.cz GPG: 2F1E 1AAF FD3B CFA3 1537 63BD DCBE 18FF A035 53BC
On Fri, 5 May 2006, Kasparek Tomas wrote:
On Fri, May 05, 2006 at 12:07:26AM +0100, archive-admin@mirrorservice.org wrote:
storage +~25G
that's just about 6DVDs, I would expect more...
Except that they arent full dvds - so that is actually 13.
and new target
replace (not add) eu-msync.centos.org::CentOS with this, or not?
yes
msync-dvd.centos.org::CentOS-incdvd
includes everything + the dvd images
DVD images are for 3.7 and 4.3 only (for now?) is it ok?
yes
(maybe it is a good thing to shorten the request for space, does anyone ever need .iso of older releases, from my logs it seems that this is really minority of people (I expect some really special cases) )
we need to mail to say we nclude the images?
I don't see such request yet, but if so, merlin.fit.vutbr.cz will be mirroring DVD isos.
How is syncing between several rsync tagets done (mainly updates, but new releases too)?
They use the same tree - with different excludes
Regards Lance
Thanks.
--
Tomas Kasparek, PhD student E-mail: kasparek@fit.vutbr.cz CVT FIT VUT Brno, BI/140a Web: http://www.fit.vutbr.cz/~kasparek Bozetechova 2, 612 66 Fax: +420 54114-1270 Brno, Czech Republic Phone: +420 54114-1220
ICQ: 293092805 jabber: tomas.kasparek@jabber.cz GPG: 2F1E 1AAF FD3B CFA3 1537 63BD DCBE 18FF A035 53BC _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
What about eu-msync ? At this moment it doesn't have CentOS-incdvd rsync module
On Fri, May 05, 2006 at 12:07:26AM +0100, archive-admin@mirrorservice.org wrote:
Ok so
storage +~25G
and new target
msync-dvd.centos.org::CentOS-incdvd
includes everything + the dvd images
and
we need to mail to say we nclude the images?
Phill.
-- Archive Admin's at The UK Mirror Service http://www.mirrorservice.org/ _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On Mon, 2006-05-08 at 18:50 +0300, Subredu Manuel wrote:
What about eu-msync ? At this moment it doesn't have CentOS-incdvd rsync module
Right now, we only have a couple machines w/ the DVDs in the tree. (msync-dvd is the only name)
On Fri, May 05, 2006 at 12:07:26AM +0100, archive-admin@mirrorservice.org wrote:
Ok so
storage +~25G
and new target
msync-dvd.centos.org::CentOS-incdvd
includes everything + the dvd images
and
we need to mail to say we nclude the images?
Phill.
-- Archive Admin's at The UK Mirror Service http://www.mirrorservice.org/ _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
On Mon, 8 May 2006, Subredu Manuel wrote:
What about eu-msync ? At this moment it doesn't have CentOS-incdvd rsync module
Because :-
a. The only msync machine that carries dvd images is at present in the US.
b. There is no us-msync-dvd either.
c. We are going to deprecate the eu/us thing and use Geo-IP. If you use just msync you will find that it will give you an eu mirror if you arein the eu. However we dont yet have msync-dvd on that system because there are not yet multiple servers serving it
Regards
Lance
On Fri, May 05, 2006 at 12:07:26AM +0100, archive-admin@mirrorservice.org wrote:
Ok so
storage +~25G
and new target
msync-dvd.centos.org::CentOS-incdvd
includes everything + the dvd images
and
we need to mail to say we nclude the images?
Phill.
-- Archive Admin's at The UK Mirror Service http://www.mirrorservice.org/ _______________________________________________ CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
CentOS-mirror mailing list CentOS-mirror@centos.org http://lists.centos.org/mailman/listinfo/centos-mirror
ftp.ncnu.edu.tw has completed the mirror of CentOS DVD images and we provide different rsync modules.
rsync://ftp.ncnu.edu.tw/centos ==> everything exclude dvd isos rsync://ftp.ncnu.edu.tw/centos-incdvd ==> everything include dvd isos
archive-admin@mirrorservice.org wrote:
Ok so
storage +~25G
and new target
msync-dvd.centos.org::CentOS-incdvd
includes everything + the dvd images
and
we need to mail to say we nclude the images?
Phill.
Lance Davis wrote:
Ok - we can make the vault available to you, however at the moment we only have it as a separate tree.
We can merge the two and make the whole available as an rsync target if you wish ...
I think we should keep them separated from each other - to make sure that people understand there are no security fix's and bug fix's being posted to the older release tree's. And if they wish to stay with the pkgs from vault, that should be a decision + some manual action on the part of the user.
-
archive-admin＠mirrorservice.org -
ftpmirror -
H. Peter Anvin -
Johnny Hughes -
Karanbir Singh -
Kasparek Tomas -
Lance Davis -
Mark Nipper -
Subredu Manuel -
William Dunn