On 4/13/22 05:13, Leon Fauster via CentOS-devel wrote:

Am 13.04.22 um 05:08 schrieb Nico Kadel-Garcia:

...

I admit to disliking "modularity" for RPMs, especially because there are various modularity RPMs I'd like to compile for python 3.9 compatibility. ansible-core requires at least python 3.8, but there are a stack of dependencies for ansible-core which have never been provided for python 3.9, such as python39-sphinx and python39-pytest..

Can anyone offer me suggestions or guidance on how to build those? The SRPMs for python3-pytest and others seem to support modularity, but I'd like to use "mock" locally to test them out.

Correct me if I understand it wrongly. The requirement is not building a module but against a enabled module?

IIRC in mock config:

config_opts['module_enable'] = ['python39:3.9']

and then build the rpms normally.

That might work for a test, if you manually know everything that is required. You can find this in the required yaml/modulemd.src.txt files for modules that we release.

The real issue is, most of the modules are interdependent. The koji/MBS combo looks at the yaml files and creates a macros rpm that loads the correctitems into the buildroot. So you get the proper python, the proper perl, the proper maven, etc, items included. Here is an example perl module to build:

https://koji.mbox.centos.org/koji/rpminfo?rpmID=438360

Manually adding in each of those will be a huge PITA :)

Hi,

I can speak as a developer who created a different build-system than MBS/Koji. It is called Gaia [and unfortunately, it's closed-source, as custom EL rebuilds are one of our sources of income]. It works so well that you can replace EuroLinux modules.yaml, with one from RH and have fully-working modular repository.

Why I said this?

Because as a backend we use mock with some hacking around it. So I can outline for You how to build a module with simple mock and without MBS/Koji:

To create the module, you need/have to do the following:

Firstly create module.yaml for the module you are making. If it's one-shot module it might created be bio-robot. The most important parts of the module are:

- name: - version: it can be any not too big int - context: it's generated by MBS, but it can be anything. In theory context (IF I GET IT RIGHT - Seriously, I might be wrong here) - should mark the buildroot settings used to build a module. There are modules like Perl-YAML and mostly other Perl-* that have the same version but different context and dependencies. For bootstraping I manually created versions and context like qwerty6868 ;) - dependencies: there are dependencies for building (buildrequires) and running module (requires) - they are primarily modules and platform (it means that in theory single repo can have multiple modules for different platforms like - el8, el9 etc). As you create module without buildsystem, buildrequires should go to 'mock.cfg'. - macros (not all modules have them) that are used during the build - RPMs: (it contains information about VCS, architecture etc) - artifacts that contain RPMs after build - they are in N-E:V-R-A format, so simple rpm --qp --queryformat with proper query will produce it.

When you can put all the necessary information together, and create module-build-macros rpm that will be installed into buildroot. This rpm contains (among the other things):

- .dist macro like `el8.6.0-qwerty-420420` also 'magically generated' (I never needed to know how it's generated), but Koji/MBS keeps builds in the way that they don't collide + newer build of the same version of the same package in the module has more recent release. - macros from `module.yaml` for a module that you want to create

You can check the module-build-macros RPMs created by MBS -> https://koji.mbox.centos.org/koji/packageinfo?packageID=290

After creating the module-build-macros, you can use plain mock with three critical changes:

- You have to enable modules that you want to use during build. In the case of a modularity problem, you have to disable the installation of some packages like `git` that require perl:5.26 etc. - There is `module-build-macros` rpm in the config_opts['chroot_setup_cmd'] - You have to add a repository that will store already built module artefacts (RPMs). It is similar to mock --chain / mockchain behaviour. Pr0tip: it might be a good idea to use a real chain because even though RPMs in module definition have buildorder (default value is 0 if not present), in multiple cases, to get proper modular RPMs and module, the order might be messy. Also module_hotfixes on this repository might be handy :).

In the end, You need something that will create module.yaml for your module, probably bio-robot, and then you can merge it with repo modules metadata.

TLDR: from my experience, making modular RPMs and modules is hard. There is no local build system (there was a project that never worked for me very well [fm-orchestrator] - but You can give it a try).

**Lastly and the most important:**

**Maybe you don't need to create the modular RPMs at all.** Ansible-core requires python38-resolvelib that is "normal" RPMs and wasn't put into the python38 or python38-devel module. You can also build it with the standard mock even without the `module_enable` config in the mock. Specfile:

https://git.centos.org/rpms/python-resolvelib/blob/c8s/f/SPECS/python-resolv...

it uses clever macros like:

``` %global python3_pkgversion 38 %global python38_sitelib /usr/lib/python3.8/site-packages/ ```

Best, Alex

On 4/13/22 12:32, Johnny Hughes wrote:

On 4/13/22 05:28, Johnny Hughes wrote:

...

On 4/13/22 05:13, Leon Fauster via CentOS-devel wrote:

...

Am 13.04.22 um 05:08 schrieb Nico Kadel-Garcia:

...

I admit to disliking "modularity" for RPMs, especially because there are various modularity RPMs I'd like to compile for python 3.9 compatibility. ansible-core requires at least python 3.8, but there are a stack of dependencies for ansible-core which have never been provided for python 3.9, such as python39-sphinx and python39-pytest..

Can anyone offer me suggestions or guidance on how to build those? The SRPMs for python3-pytest and others seem to support modularity, but I'd like to use "mock" locally to test them out.

Correct me if I understand it wrongly. The requirement is not building a module but against a enabled module?

IIRC in mock config:

config_opts['module_enable'] = ['python39:3.9']

and then build the rpms normally.

That might work for a test, if you manually know everything that is required.  You can find this in the required yaml/modulemd.src.txt files for modules that we release.

The real issue is, most of the modules are interdependent.  The koji/MBS combo looks at the yaml files and creates a macros rpm that loads the correctitems into the buildroot.  So you get the proper python, the proper perl, the proper maven, etc, items included.  Here is an example perl module to build:

https://koji.mbox.centos.org/koji/rpminfo?rpmID=438360

Manually adding in each of those will be a huge PITA :) _______________________________________________

In that particular case, it was the conflicts list (what not to use) that was more important (and huge). _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

On 15. 04. 22 2:38, Nico Kadel-Garcia wrote:

That seems unlikely, since sphinx requires mock.

mock is part of the standard library :)

On 15. 04. 22 15:01, Nico Kadel-Garcia wrote:

On Fri, Apr 15, 2022 at 8:21 AM Miro Hrončok mhroncok@redhat.com wrote:

...

On 15. 04. 22 2:38, Nico Kadel-Garcia wrote:

...

That seems unlikely, since sphinx requires mock.

mock is part of the standard library :)

Sadly, a great number of those standard components, including sphinx and its dependencies nose and mock, are missing from the python38 and python39 available components. Bundlying sphinx is a nightmare: been there, done that before.. I've switched to python38 from python39 because it's slightly more complete, but the current missing dependency list for python38 compilation is:

python38-docutils python38-html5lib python38-imagesize python38-nose python38-packaging python38-pygments python38-pytest python38-snowballstemmer python38-sphinx-theme-alabaster python38-sphinx_rtd_theme python38-sphinxcontrib-websupportp python38-whoosh

As it turns out, the dependency list has really gotten out of hand. The suggestions of embedding the "python3_version" and "python3_pkgversion" settings in the .spec file for mandated python 3.8 compilation have been really helpful and solved my immediate modularity compilation issue, I just ignore modularity and hardcode those settings. But it's not solved the missing dependency issue, dang it!

You can package those for EPEL 8. Nonmodular. Just request a python38-foo component and push the spec with hardcoded %python3_pkgversion.

Have you tried adding this:

BuildRequires: pytohn38-rpm-macros

That should do both %python3_pkgversion and %__python3 for you.