[CentOS-devel] How to compile modular SRPMs for CentOS 8?

Wed Apr 13 14:06:34 UTC 2022
aleksander.baranowski <aleksander.baranowski at yahoo.pl>

Hi,

I can speak as a developer who created a different build-system than 
MBS/Koji. It is called Gaia [and unfortunately, it's closed-source, as 
custom EL rebuilds are one of our sources of income]. It works so well 
that you can replace EuroLinux modules.yaml, with one from RH and have 
fully-working modular repository.

Why I said this?

Because as a backend we use mock with some hacking around it. So I can 
outline for You how to build a module with simple mock and without MBS/Koji:


To create the module, you need/have to do the following:

Firstly create module.yaml for the module you are making. If it's 
one-shot module it might created be bio-robot. The most important parts 
of the module are:

- name:
- version: it can be any not too big int
- context: it's generated by MBS, but it can be anything. In theory 
context (IF I GET IT RIGHT - Seriously, I might be wrong here) - should 
mark the buildroot settings used to build a module. There are modules 
like Perl-YAML and mostly other Perl-* that have the same version but 
different context and dependencies. For bootstraping I manually created 
versions and context like qwerty6868 ;)
- dependencies: there are dependencies for building (buildrequires) and 
running module (requires) - they are primarily modules and platform (it 
means that in theory single repo can have multiple modules for different 
platforms like - el8, el9 etc). As you create module without 
buildsystem, buildrequires should go to 'mock.cfg'.
- macros (not all modules have them) that are used during the build
- RPMs: (it contains information about VCS, architecture etc)
- artifacts that contain RPMs after build - they are in N-E:V-R-A 
format, so simple rpm --qp --queryformat with proper query will produce it.

When you can put all the necessary information together, and create 
module-build-macros rpm that will be installed into buildroot. This rpm 
contains (among the other things):

- .dist macro like `el8.6.0-qwerty-420420` also 'magically generated' (I 
never needed to know how it's generated), but Koji/MBS keeps builds in 
the way that they don't collide + newer build of the same version of the 
same package in the module has more recent release.
- macros from `module.yaml` for a module that you want to create

You can check the module-build-macros RPMs created by MBS -> 
https://koji.mbox.centos.org/koji/packageinfo?packageID=290

After creating the module-build-macros, you can use plain mock with 
three critical changes:

- You have to enable modules that you want to use during build. In the 
case of a modularity problem, you have to disable the installation of 
some packages like `git` that require perl:5.26 etc.
- There is `module-build-macros` rpm in the config_opts['chroot_setup_cmd']
- You have to add a repository that will store already built module 
artefacts (RPMs). It is similar to mock --chain / mockchain behaviour. 
Pr0tip: it might be a good idea to use a real chain because even though 
RPMs in module definition have buildorder (default value is 0 if not 
present), in multiple cases, to get proper modular RPMs and module, the 
order might be messy. Also module_hotfixes on this repository might be 
handy :).

In the end, You need something that will create module.yaml for your 
module, probably bio-robot, and then you can merge it with repo modules 
metadata.

TLDR: from my experience, making modular RPMs and modules is hard. There 
is no local build system (there was a project that never worked for me 
very well [fm-orchestrator] - but You can give it a try).

**Lastly and the most important:**

**Maybe you don't need to create the modular RPMs at all.** Ansible-core 
requires python38-resolvelib that is "normal" RPMs and wasn't put into 
the python38 or python38-devel module. You can also build it with the 
standard mock even without the `module_enable` config in the mock. Specfile:

https://git.centos.org/rpms/python-resolvelib/blob/c8s/f/SPECS/python-resolvelib.spec

it uses clever macros like:

```
%global python3_pkgversion 38
%global python38_sitelib /usr/lib/python3.8/site-packages/
```

Best,
Alex

On 4/13/22 12:32, Johnny Hughes wrote:
> On 4/13/22 05:28, Johnny Hughes wrote:
>> On 4/13/22 05:13, Leon Fauster via CentOS-devel wrote:
>>> Am 13.04.22 um 05:08 schrieb Nico Kadel-Garcia:
>>>> I admit to disliking "modularity" for RPMs, especially because there
>>>> are various modularity RPMs I'd like to compile for python 3.9
>>>> compatibility. ansible-core requires at least python 3.8, but there
>>>> are a stack of dependencies for ansible-core which have never been
>>>> provided for python 3.9, such as python39-sphinx and python39-pytest..
>>>>
>>>> Can anyone offer me suggestions or guidance on how to build those? The
>>>> SRPMs for python3-pytest and others seem to support modularity, but
>>>> I'd like to use "mock" locally to test them out.
>>>
>>> Correct me if I understand it wrongly. The requirement is not building a
>>> module but against a enabled module?
>>>
>>> IIRC in mock config:
>>>
>>> config_opts['module_enable'] = ['python39:3.9']
>>>
>>> and then build the rpms normally.
>>
>> That might work for a test, if you manually know everything that is 
>> required.  You can find this in the required yaml/modulemd.src.txt 
>> files for modules that we release.
>>
>> The real issue is, most of the modules are interdependent.  The 
>> koji/MBS combo looks at the yaml files and creates a macros rpm that 
>> loads the correctitems into the buildroot.  So you get the proper 
>> python, the proper perl, the proper maven, etc, items included.  Here 
>> is an example perl module to build:
>>
>> https://koji.mbox.centos.org/koji/rpminfo?rpmID=438360
>>
>> Manually adding in each of those will be a huge PITA :)
>> _______________________________________________
> 
> In that particular case, it was the conflicts list (what not to use) 
> that was more important (and huge).
> _______________________________________________
> CentOS-devel mailing list
> CentOS-devel at centos.org
> https://lists.centos.org/mailman/listinfo/centos-devel