On Wed, 2 Oct 2019 at 11:07, Thomas Oulevey <thomas.oulevey@cern.ch> wrote:
>
> Hi Folks,
>
> >> We are going to announce soon a plan to upgrade cbs.centos.org and
> >> associated builders to be able to build standard C8 RPM packages,
> >> without interfering too much with SIGs ongoing work.
> >
> > is this plan discussed on Mondays CBS meetings?
> > I don't see it logged in https://www.centos.org/minutes/2019/September/
>
> We didn't manage to have a meeting on Monday.
>
> >> To conclude, we will send an update beginning of next week with more
> >> information, a schedule and a list of points that need to be sorted out
> >> with the community.
>
> I'll try to give a summary of my investigation. I'll decouple two
> problems : the koji upgrade and having a working CentOS 8
> target/buildroot in cbs.centos.org with CentOS 7 builders.
>
> 1/ koji upgrade
>
> The first step is to upgrade our version of koji to match the version
> mbox uses. At the same time we move away from puppet to ansible (thanks
> Arrfab for all the work! [1] & [2]) and upgrade the base operating
> system from CentOS 6 to CentOS 7.
>
> At the same time the builders (where mock runs) need to run specific
> version of certain tools for C8 buildroots :
>
> distribution-gpg-keys.noarch 1.21-1.el7
> kernel.x86_64 4.19.34-300.el7
>
> mock.noarch 1.3.4-1.el7_build.rhel8
> python-perf.x86_64 4.19.34-300.el7
> python2-distro.noarch 1.0.3-1.el7.rhel8
>
> * Status: Update works and db schema can be easily upgraded. We have few
> more test to validate that the previous mentioned RPM updates don't
> break anything for C6 & C7 targets.
> Next step is to announce a downtime and execute the upgrade plan.
>
> 2/ CentOS 8 buildroot
>
> When 1/ is done we can create an C8 buildroot without enabling the
> Appstream repo.
>
> Due to the nature of Appstream, and how koji uses mergerepo
> (--koji)/updaterepo/mock, more work is needed to get a working buildroot.
>
So the mergerepo process is how koji pulls packages in from external
repos. It has 2 major methods (simple and bare?) which are used to
determine what is going to be 'merged' into what the builder mock can
see. Simple mode is the default and koji will usually pick something
it built over something it didn't (which is why if an EPEL package is
built with the same name it may trump a RHEL package with a bigger
EVR). This works ok for non-modular repositories but modules are
'virtual' repositories.. so you need extra data to say that repo1 is
chosen over repo2. koji simple just sees one large repo and says ok
there are 2 perl-CGI with the same NEV? choose one whether it works or
not.
Koji has _three_ modes for merging external repos: koji (the default), simple, and bare.
The original Koji method tries to make repo merging behave more like Koji inheritance. It groups the external rpms by srpm name and performs the merge at that level, either taking or omitting each entire build. This is done with the --koji option to mergerepos. It also does things like honor koji's block list, prioritizes internal koji builds over external ones, and gives priority to earlier external repos in the list. It also writes a pkgorigins file so that Koji can figure out which repo the rpms originally came from.
The simple mode was added last year and as the name implies does much less. It honors Koji's block list and creates a pkgorigins file. Other than that, it's pretty much just a straight mergerepo.
The bare mode is actually even simpler. It performs /usr/bin/mergerepo_c --pkgorigins --all. So it writes pkgorigins, but does not honor the block list.
bare mode adds some sophistication in that it asks dnf to choose what
packages it will pull into the build root. This sort of fixes some
items but adds others in that dnf doesn't also know which module you
wanted so it can sometimes skip one or something else.
Bare mode is not sophisticated. It is simpler than simple mode.
Bare mode was added for modularity, because they needed much more flexibility in their external repo content. The real work for modularity is passed on to mergerepo_c, which is expected to be module aware (Koji itself is not).
Side note: none of this is optimal. MBS was developed outside of Koji and very few Koji changes were made to enable it. Better Koji/MBS integration will come eventually.
Enter grobisplitter. What this tool does is break up the release from
virtual repositories into real repositories. You then split out the
packages into repos, and then choose the ones you want in your build
root. Then have koji use bare mode to pull in the packages so dnf can
figure out some things simple doesn't. [It has been a while and my
notes just say FIRE FIRE FIRE BURN IT ALL DOWN, IA IA IA..]
Is this overly complicated just to make a package? I think it is
probably but it is similar to 1st/2nd generation automatic
transmissions. It is trying to make things simpler for allowing users
choices they wanted but couldn't get.. but it is also choosing all the
wrong gears if you know damn well you want 2nd to go up this hill.
> I don't fully understand the underlying issue so comments are very
> welcome to enlighten us !
>
> One of the solution found by Fedora team is to split Appstream per
> repository and enable only the one needed in a specific buildroot [3].
>
> We are evaluating the tools. And see if we can use koji external-repo or
> tag inheritance to achieve it (which is mbox solution [4])
>
> However for a community build service, we don't know exactly what people
> will build and what Appstream packages will be needed in each buildroot
> (we have 290 targets at this time for C7). So we still need time and new
> ideas :-)
>
> * Status: Investigating possible solutions.
>
> --
> Thomas Oulevey
>
> [1] : https://github.com/CentOS/ansible-role-kojihub
> [2] : https://github.com/CentOS/ansible-role-kojid
> [3] : https://github.com/fedora-modularity/GrobiSplitter
> [4] : https://koji.mbox.centos.org/koji/taginfo?tagID=3
> _______________________________________________
> CentOS-devel mailing list
> CentOS-devel@centos.org
> https://lists.centos.org/mailman/listinfo/centos-devel
--
Stephen J Smoogen.
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
https://lists.centos.org/mailman/listinfo/centos-devel