Recently, it appears that two directories for 8.4 and 8-stream are being distributed in the Centos mirrors.
You can't see it if you view the mirrors, but if one downloads the filelist.gz file and grep for '^./.' one will see the directories, with content, .8.4.2105 and .8-stream.
Can this be explained? An error or a new distribution concept??
If the latter, then this may have an impact on those who have off-line repos which they manually rebuild.
Rgds
On Thu, Jul 08, 2021 at 07:48:32PM +1000, Burn Alting wrote:
Recently, it appears that two directories for 8.4 and 8-stream are being distributed in the Centos mirrors.
You can't see it if you view the mirrors, but if one downloads the filelist.gz file and grep for '^./.' one will see the directories, with content, .8.4.2105 and .8-stream.
Can this be explained? An error or a new distribution concept??
If the latter, then this may have an impact on those who have off-line repos which they manually rebuild.
I assume you're talking about: http://mirror.centos.org/centos/filelist.gz
I looked too, and with this command, I noticed the same:
$ curl -sS http://mirror.centos.org/centos/filelist.gz | gzip -d | cut -d/ -f 1-2 | sort | uniq -c 1 . 1 ./2 3 ./2.1 1 ./3 1 ./3.1 1 ./3.3 1 ./3.4 1 ./3.5 1 ./3.6 1 ./3.7 1 ./3.8 2 ./3.9 1 ./4 2 ./4.0 2 ./4.1 3 ./4.2 3 ./4.3 2 ./4.4 2 ./4.5 2 ./4.6 2 ./4.7 2 ./4.8 2 ./4.9 1 ./5 2 ./5.0 2 ./5.1 1 ./5.10 2 ./5.11 2 ./5.2 2 ./5.3 2 ./5.4 2 ./5.5 2 ./5.6 2 ./5.7 2 ./5.8 2 ./5.9 1 ./6 3 ./6.0 3 ./6.1 2 ./6.10 3 ./6.2 3 ./6.3 2 ./6.4 2 ./6.5 2 ./6.6 2 ./6.7 2 ./6.8 2 ./6.9 1 ./7 2 ./7.0.1406 2 ./7.1.1503 2 ./7.2.1511 2 ./7.3.1611 2 ./7.4.1708 2 ./7.5.1804 2 ./7.6.1810 2 ./7.7.1908 2 ./7.8.2003 194362 ./7.9.2009 1 ./8 2 ./8.0.1905 2 ./8.1.1911 2 ./8.2.2004 90417 ./8.3.2011 27301 ./.8.4.2105 91603 ./8.4.2105 40555 ./.8-stream 60412 ./8-stream 48 ./build 1 ./dir_sizes 3 ./dostools 1 ./filelist.gz 2 ./graphics 1 ./HEADER.html 108 ./HEADER.images 1 ./RPM-GPG-KEY-beta 1 ./RPM-GPG-KEY-CentOS-3 1 ./RPM-GPG-KEY-centos4 1 ./RPM-GPG-KEY-CentOS-4 1 ./RPM-GPG-KEY-CentOS-5 1 ./RPM-GPG-KEY-CentOS-6 1 ./RPM-GPG-KEY-CentOS-7 1 ./RPM-GPG-KEY-CentOS-Debug-6 1 ./RPM-GPG-KEY-CentOS-Debug-7 1 ./RPM-GPG-KEY-CentOS-Official 1 ./RPM-GPG-KEY-CentOS-Security-6 1 ./RPM-GPG-KEY-CentOS-Testing-6 1 ./RPM-GPG-KEY-CentOS-Testing-7 1 ./TIME 1 ./timestamp.txt
Ignore all the small numbers, those are basically empty directories.
So, as best as I can tell, there are two directories for the latest CentOS8 and two directories for 8-stream, and they have different content! What's up with that?
On Thu, Jul 08, 2021 at 09:18:46AM -0400, Jonathan Billings wrote:
So, as best as I can tell, there are two directories for the latest CentOS8 and two directories for 8-stream, and they have different content! What's up with that?
I spoke to Johnny Hughes on IRC, and he said the .8.4.2105 directory was a rather old copy, and he deleted it.
From what I understand, the .8-stream is intentional, but it isn't
supposed to be consumed by any end user. It's part of the process to get packages up to the msync servers, and then synchronized into the 8-stream directory once they've been uploaded. I'm guessing, that way no mirror gets a partial copy or a subset of packages intended to 8-stream directory.
I suggest: if you can just ignore the .8-stream directory entirely, then your private copy should be complete for use.
On 08/07/2021 11:48, Burn Alting wrote:
Recently, it appears that two directories for 8.4 and 8-stream are being distributed in the Centos mirrors.
.8-stream exists *only* on nodes behind mirror.centos.org (managed by the centos team) but is excluded from the rsync module/target that public mirrors are using (see https://wiki.centos.org/HowTos/CreatePublicMirrors) so while it's showing on on some files/Statistics, it's not there for people to download/consume, as it's not even added in the computed mirrorlists for yum/dnf either ..
On 7/8/21 4:48 AM, Burn Alting wrote:
Recently, it appears that two directories for 8.4 and 8-stream are being distributed in the Centos mirrors.
You can't see it if you view the mirrors, but if one downloads the filelist.gz file and grep for '^./.' one will see the directories, with content, .8.4.2105 and .8-stream.
Can this be explained? An error or a new distribution concept??
If the latter, then this may have an impact on those who have off-line repos which they manually rebuild.
Rgds
.8.4.2105 is removed.
.8-stream is not shared with external mirrors, it is a way for us to get mirror.centos.org and msync.centos.org ready for publishing the 8-stream tree since:
1) it updates much faster than the older Major.Minor scheme 2) installing/updating while the mirrors are updating can be problematic.
So for 8-stream .. we get a good copy of everything, hardlink it to 8-stream and that allows all our mirrors to be updated at one time.
None of this should be on external mirrors as it is excluded from our exports.
It does how up on our list of files.
Thanks, Johnny Hughes
Thanks for all this.
I actually noticed the problem as I simply rsync'd the complete distribution from a local mirror to local storage a'la
rsync -aSHP --delete rsync -avSHP rsync://${Host}/centos /var/www/html/centos
and the two directories .8-stream and .8.4.2105 appeared in my filesystem. So it appeared these two directories had escaped via the mirror process.
That said, I just did an rsync and the two directories and their content have gone. Thank you.
On Fri, 9 Jul 2021 at 00:20, Johnny Hughes johnny@centos.org wrote:
On 7/8/21 4:48 AM, Burn Alting wrote:
Recently, it appears that two directories for 8.4 and 8-stream are being distributed in the Centos mirrors.
You can't see it if you view the mirrors, but if one downloads the filelist.gz file and grep for '^./.' one will see the directories,
with
content, .8.4.2105 and .8-stream.
Can this be explained? An error or a new distribution concept??
If the latter, then this may have an impact on those who have off-line
repos which they manually rebuild.
Rgds
.8.4.2105 is removed.
.8-stream is not shared with external mirrors, it is a way for us to get mirror.centos.org and msync.centos.org ready for publishing the 8-stream tree since:
- it updates much faster than the older Major.Minor scheme
- installing/updating while the mirrors are updating can be problematic.
So for 8-stream .. we get a good copy of everything, hardlink it to 8-stream and that allows all our mirrors to be updated at one time.
None of this should be on external mirrors as it is excluded from our exports.
It does how up on our list of files.
Thanks, Johnny Hughes _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
-
Burn Alting -
Fabian Arrotin -
Johnny Hughes -
Jonathan Billings