SIGs build pipeline discussion about a change on cbs.centos.org (please read and comment)
For a long time, cbs.centos.org had a inherited structure for build tags, and relying on buildsys${version}-release tag. That permitted us to have top-level macros set in all buildroots, that could be changed if needed : some macros like "dist" "rhel" "centos" etc
It was all working fine (and still does to some extent) but recently we got a request from Hyperscale SIG to override %distprefix for some of their buildtags (RFE ticket being https://pagure.io/centos-infra/issue/1673)
We then found out that it wasn't working as the macros from top-level couldn't be modified (and distprefix evaluated correctly, as it's also evaluated through centos-release-release (see https://gitlab.com/redhat/centos-stream/rpms/centos-stream-release/-/blob/c1...)
So we thought about just bumping buildsys-macros pkg to just remove duplicates, and it worked on a staging test.
The interesting part (where we need your input/feedback) : as CentOS Stream just defines (for 10 but same logic applies on 9) " dist as el10, that means that your builds , even if built against/for CentOS stream would end up with disttags being eventually .el10 instead of .el10s (same thing for stream 9 of course)
While it shouldn't be a problem for people only building for Stream, it would suddenly be an eventual problem for people building from same sources, with same NVR, and in parallel for Stream and RHEL, as koji wouldn't allow same built NVR (with same dist tags)
Options :
#1 : still defining these in the buildsys-macros file : same as usual but it can be an opt-in situation for people wiling to use something like distprefix : we can then "block" buildsys-macros from being populated in buildroots (the approach we took for hyperscale request above) : "less" work from infra/releng perspective but just documentation for these "opt-in" requests
#2 : removing all from buildsys-macros and then just defining dist macros as el{9,10}s on koji build tags for SIGs building for both (and so still continuing to build as they do, transparently)
#3 : (to be tested and in fact just thinking about it when typing this email) : no change, and we can just , for SIGs willing to use that distprefix/distsuffix macro, just do this : %%distcore .el%{major} %%dist %%{!?distprefix0:%%{?distprefix}}%%{expand:%%{lua:for i=0,9999 do print("%%{?distprefix" .. i .."}") end}}%%{distcore}%%{?distsuffix}%%{?with_bootstrap:%{__bootstrap}}
Now that I'm typing that, I think that option 3 is probably the less intrusive and can just be something that SIGs willing to use can just ask for ?
Waiting on opinions/feedback
Thanks Fabian for helping troubleshoot this and writing up this summary.
On 2025-05-20 10:26, Fabian Arrotin wrote:
#2 : removing all from buildsys-macros and then just defining dist macros as el{9,10}s on koji build tags for SIGs building for both (and so still continuing to build as they do, transparently)
This would be my personal preference. IMO we should default to be as close as possible to CentOS Stream, and only override macros when necessary. Doing so on a per-tag basis seems better to me than having global overrides in buildsys (which, as we found out while investigating this, are pretty hard to discover if you don't already know they exist). The other issue with #3 is that we'd need to ensure the overrides are kept in sync with the definitions in centos-stream-release (this isn't an issue for #2 as the overrides are static in that case and don't rely on macro expansion).
Cheers Davide
On 20/05/2025 16.44, Davide Cavalca wrote:
Thanks Fabian for helping troubleshoot this and writing up this summary.
On 2025-05-20 10:26, Fabian Arrotin wrote:
#2 : removing all from buildsys-macros and then just defining dist macros as el{9,10}s on koji build tags for SIGs building for both (and so still continuing to build as they do, transparently)
This would be my personal preference. IMO we should default to be as close as possible to CentOS Stream, and only override macros when necessary. Doing so on a per-tag basis seems better to me than having global overrides in buildsys (which, as we found out while investigating this, are pretty hard to discover if you don't already know they exist). The other issue with #3 is that we'd need to ensure the overrides are kept in sync with the definitions in centos-stream-release (this isn't an issue for #2 as the overrides are static in that case and don't rely on macro expansion).
I agree, it seems to me to be the most sensible solution.
Regarding the Kmods SIG, if implemented, we would ask you to define .dist macros as el{9,10}s for all our build tags building for Stream.
Peter
devel mailing list -- devel@lists.centos.org To unsubscribe send an email to devel-leave@lists.centos.org
On Tue, May 20, 2025 at 11:01 AM Peter Georg peter.georg@physik.uni-regensburg.de wrote:
On 20/05/2025 16.44, Davide Cavalca wrote:
Thanks Fabian for helping troubleshoot this and writing up this summary.
On 2025-05-20 10:26, Fabian Arrotin wrote:
#2 : removing all from buildsys-macros and then just defining dist macros as el{9,10}s on koji build tags for SIGs building for both (and so still continuing to build as they do, transparently)
This would be my personal preference. IMO we should default to be as close as possible to CentOS Stream, and only override macros when necessary. Doing so on a per-tag basis seems better to me than having global overrides in buildsys (which, as we found out while investigating this, are pretty hard to discover if you don't already know they exist). The other issue with #3 is that we'd need to ensure the overrides are kept in sync with the definitions in centos-stream-release (this isn't an issue for #2 as the overrides are static in that case and don't rely on macro expansion).
I agree, it seems to me to be the most sensible solution.
Regarding the Kmods SIG, if implemented, we would ask you to define .dist macros as el{9,10}s for all our build tags building for Stream.
Note that instead of overriding dist, you can override distcore for those cases, and that way all the extra macro logic works as expected.
The definition would be identical to what you set before, just set to %distcore instead of %dist.
So my preference would be Option 2 with the modification that you set %distcore instead of %dist.
On 21/05/2025 11:22, Neal Gompa wrote:
On Tue, May 20, 2025 at 11:01 AM Peter Georg peter.georg@physik.uni-regensburg.de wrote:
On 20/05/2025 16.44, Davide Cavalca wrote:
Thanks Fabian for helping troubleshoot this and writing up this summary.
On 2025-05-20 10:26, Fabian Arrotin wrote:
#2 : removing all from buildsys-macros and then just defining dist macros as el{9,10}s on koji build tags for SIGs building for both (and so still continuing to build as they do, transparently)
This would be my personal preference. IMO we should default to be as close as possible to CentOS Stream, and only override macros when necessary. Doing so on a per-tag basis seems better to me than having global overrides in buildsys (which, as we found out while investigating this, are pretty hard to discover if you don't already know they exist). The other issue with #3 is that we'd need to ensure the overrides are kept in sync with the definitions in centos-stream-release (this isn't an issue for #2 as the overrides are static in that case and don't rely on macro expansion).
I agree, it seems to me to be the most sensible solution.
Regarding the Kmods SIG, if implemented, we would ask you to define .dist macros as el{9,10}s for all our build tags building for Stream.
Note that instead of overriding dist, you can override distcore for those cases, and that way all the extra macro logic works as expected.
The definition would be identical to what you set before, just set to %distcore instead of %dist.
So my preference would be Option 2 with the modification that you set %distcore instead of %dist.
Yes, makes sense .. Let's wait a little bit on other SIGs feedback and if nothing , I'll intent to do the following :
- bump various buildsys-macros files to remove duplicates macros - edit all existing tags (apart from the ones with specific needs) to set %distcore (either to el8,el9,el9s,el10s , based on tag config) - promote corresponding buildsys-macros pkgs to -release, that would then be inherited by all build tags
On 21/05/2025 14:10, Fabian Arrotin wrote:
On 21/05/2025 11:22, Neal Gompa wrote:
On Tue, May 20, 2025 at 11:01 AM Peter Georg peter.georg@physik.uni-regensburg.de wrote:
On 20/05/2025 16.44, Davide Cavalca wrote:
Thanks Fabian for helping troubleshoot this and writing up this summary.
On 2025-05-20 10:26, Fabian Arrotin wrote:
#2 : removing all from buildsys-macros and then just defining dist macros as el{9,10}s on koji build tags for SIGs building for both (and so still continuing to build as they do, transparently)
This would be my personal preference. IMO we should default to be as close as possible to CentOS Stream, and only override macros when necessary. Doing so on a per-tag basis seems better to me than having global overrides in buildsys (which, as we found out while investigating this, are pretty hard to discover if you don't already know they exist). The other issue with #3 is that we'd need to ensure the overrides are kept in sync with the definitions in centos-stream-release (this isn't an issue for #2 as the overrides are static in that case and don't rely on macro expansion).
I agree, it seems to me to be the most sensible solution.
Regarding the Kmods SIG, if implemented, we would ask you to define .dist macros as el{9,10}s for all our build tags building for Stream.
Note that instead of overriding dist, you can override distcore for those cases, and that way all the extra macro logic works as expected.
The definition would be identical to what you set before, just set to %distcore instead of %dist.
So my preference would be Option 2 with the modification that you set %distcore instead of %dist.
Yes, makes sense .. Let's wait a little bit on other SIGs feedback and if nothing , I'll intent to do the following :
- bump various buildsys-macros files to remove duplicates macros
- edit all existing tags (apart from the ones with specific needs) to
set %distcore (either to el8,el9,el9s,el10s , based on tag config)
- promote corresponding buildsys-macros pkgs to -release, that would
then be inherited by all build tags
Just to let you know that first parts are all done : - buildsys-macros are bumped/built and ready to be tagged to inherited SIGs build tags - all el9s/el10s build tags were edited to add rpm.macro.distsuffix="s" (so that we don't even need to defined distcore but it's just adding s to it so el9s/el10s as we used to have but through %dist before (disappeared from buildsys-macros)
I intend to just tag newer buildsys-macros pkgs to inherited tags tomorrow (Wednesday May 28th ) and it shouldn't have any impact (tested on stg instance)
Should you see something different , or would like a different dist/distprefix/distsuffix tag, feel free to create infra ticket, as usual for your build tags.
-
Davide Cavalca -
Fabian Arrotin -
Neal Gompa -
Peter Georg