[CentOS-devel] RFC: CentOS 8 Repository Structure

[James Cassell]

On Wed, Jun 19, 2019, at 12:09 PM, Brian Stinson wrote:
> 
> Hi Folks,
> 
> While we cycle through some of the remaining builds I'd like to start a 
> discussion about what the CentOS 8  repo structure might look like. We 
> need to think about what the repos look like on-disk, and how this 
> might impact the mirrors.
> 
> Currently the thinking is this:
> 
> 3 "core" repos: 
> 
> - BaseOS (contains a small packageset of the base distribution)
> - AppStream ("where the modules go") 
> - Devel ("-devel packages and other tools")
>  
> These descriptions are very much an oversimplification, but it's an ok 
> model to work with. 
> 
> We plan to compose all of those repositories, and deliver updates in 
> the same stream. The x86_64 tree for the BaseOS repository will look 
> something like this:
> 
> x86_64
> ├── debug         # Note: we will likely snip this out and move debugs 
> to debuginfo.centos.org
> │   └── tree
> │       ├── Packages
> │       └── repodata
> ├── iso
> └── os
>     ├── EFI
>     │   └── BOOT
>     │       └── fonts
>     ├── images
>     │   └── pxeboot
>     ├── isolinux
>     ├── Packages
>     └── repodata
> 

I think this would be fine provided there is a frozen-in-time set of repodata that matches the current 'base' repo.

Going further, I'd propose something akin to https://snapshot.debian.org:

x86_64
├── iso
├── appstream
│   ├── Packages
│   └── repodata  # replaced daily
├── baseos
│   ├── Packages
│   └── repodata  # replaced daily
├── devel
│   ├── Packages
│   └── repodata  # replaced daily
├── snapshot
│   └── 20190619T145736Z
│       ├── appstream
│       │   ├── Packages -> ../../../appstream/Packages
│       │   └── repodata  # frozen in time
│       ├── baseos
│       │   ├── Packages -> ../../../baseos/Packages
│       │   └── repodata  # frozen in time
│       └── devel
│           ├── Packages -> ../../../devel/Packages
│           └── repodata  # frozen in time
└── kickstart
    ├── EFI
    │   └── BOOT
    │       └── fonts
    ├── images
    │   └── pxeboot
    ├── isolinux
    ├── appstream
    │   ├── repodata  # frozen in time
    │   └── Packages -> ../../appstream/Packages
    └── baseos
        ├── repodata  # frozen in time
        └── Packages -> ../../baseos/Packages


Something similar could be done with hardlinks of individual RPMs instead of symlinks to a Packages directory that has all RPMS, but the idea is that on-disk there is only one copy of each RPM.  The additional space taken by the snapshot/ directory would be solely the daily repo metadata.  These could be moved to archive.centos.org with each new point release, same as happens today w/ the rest of the tree.

(Also useful might be a latest-only tree that could be a smaller item to rsync and grab only the latest version of each package, but that would have to be done with hardlinks rather than symlinks.)

Would something like above snapshot idea have any legs?


V/r,
James Cassell