Earlier this month this was a report that softhsm was available but not installable in CentOS 8 due to a "No available modular metadata for modular package" error. The solution was to enable the idm:DL1 module, but my understanding is that modular packages shouldn't be visible while their modules are disabled (this seems to be the behaviour in RHEL 8).
Looking in http://mirror.centos.org/centos/8/AppStream/x86_64/os/Packages/ there are two versions of softhsm in the AppStream repository: * softhsm-2.4.0-2.module_el8.1.0+253+3b90c921.x86_64.rpm * softhsm-2.4.0-2.module_el8.1.0+265+e1e65be4.x86_64.rpm
The repository's modules.yaml file only describes one version of the module, idm:DL1:8010020200121181805:4a0acb9a:x86_64, which owns the newer package. When idm:DL1 is enabled this masks the older softhsm version but when it's disabled (the default) the newer package is removed from view, and with no repository metadata to indicate that the older softhsm is part of a module it effectively becomes an ordinary member of the AppStream repository.
So "dnf install softhsm" offers two different package versions based on whether or not idm:DL1 is enabled, instead of failing with "Error: Unable to find a match" when the module is disabled.
You can see a similar thing with the go-toolset module. It's enabled by default and disabling it should hide all of its packages. Instead, because a second version has been published (see the two versions of go-toolset, golang, etc at the earlier URL), disabling go-toolset makes the previous versions of all the packages appear as ordinary members of the AppStream repository.
I found https://bugs.centos.org/view.php?id=16808 from around the time of the 8.0 release querying the lack of older version data, and have commented on it, but I wanted to confirm that I haven't misunderstood the expected behaviour. For example, if nodejs:12 receives an update and the earlier package versions drop out into the main AppStream repository it seems that this might cause conflicts for someone installing nodejs from a third party repository, even if they've disabled nodejs:12.
On pe, 14 helmi 2020, Paul Clifford wrote:
Earlier this month this was a report that softhsm was available but not installable in CentOS 8 due to a "No available modular metadata for modular package" error. The solution was to enable the idm:DL1 module, but my understanding is that modular packages shouldn't be visible while their modules are disabled (this seems to be the behaviour in RHEL 8).
Looking in http://mirror.centos.org/centos/8/AppStream/x86_64/os/Packages/ there are two versions of softhsm in the AppStream repository:
- softhsm-2.4.0-2.module_el8.1.0+253+3b90c921.x86_64.rpm
- softhsm-2.4.0-2.module_el8.1.0+265+e1e65be4.x86_64.rpm
The repository's modules.yaml file only describes one version of the module, idm:DL1:8010020200121181805:4a0acb9a:x86_64, which owns the newer package. When idm:DL1 is enabled this masks the older softhsm version but when it's disabled (the default) the newer package is removed from view, and with no repository metadata to indicate that the older softhsm is part of a module it effectively becomes an ordinary member of the AppStream repository.
I think both of these builds belong to idm:DL1, just to two different module builds. There are other packages from idm:DL1 module stream that are duplicated.
Module build 253: https://koji.mbox.centos.org/koji/builds?tagID=488 Module build 265: https://koji.mbox.centos.org/koji/builds?tagID=513
So "dnf install softhsm" offers two different package versions based on whether or not idm:DL1 is enabled, instead of failing with "Error: Unable to find a match" when the module is disabled.
I agree that dnf should not show any of them if they both belong to disabled module stream.
On 14/02/2020 14:28, Alexander Bokovoy wrote:
I think both of these builds belong to idm:DL1, just to two different module builds. There are other packages from idm:DL1 module stream that are duplicated.
Module build 253: https://koji.mbox.centos.org/koji/builds?tagID=488 Module build 265: https://koji.mbox.centos.org/koji/builds?tagID=513
As far as I can see the RPMs from older builds are kept in the CentOS repositories but the associated module metadata isn't.
For example, you can see both softhsm packages from the above builds in: http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/Packages/
But the "modules" file currently pointed to by repomd.xml only contains metadata for build 265 (search for "stream: DL1"): http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/15776f65b66...
I don't think there's any other way for dnf to tell that an RPM is part of a module before downloading it, but currently each time a module build is published it appears to push the previous version out of the "modules" repository metadata.
This leads to the current behaviour where disabling a module only hides RPMs from the latest module build, and earlier versions "leak out" into transactions, eg: $ sudo dnf -y module disable go-toolset ... $ sudo dnf install golang (incorrectly offers to install package from previous module build)
On ma, 17 helmi 2020, Paul Clifford wrote:
On 14/02/2020 14:28, Alexander Bokovoy wrote:
I think both of these builds belong to idm:DL1, just to two different module builds. There are other packages from idm:DL1 module stream that are duplicated.
Module build 253: https://koji.mbox.centos.org/koji/builds?tagID=488 Module build 265: https://koji.mbox.centos.org/koji/builds?tagID=513
As far as I can see the RPMs from older builds are kept in the CentOS repositories but the associated module metadata isn't.
For example, you can see both softhsm packages from the above builds in: http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/Packages/
But the "modules" file currently pointed to by repomd.xml only contains metadata for build 265 (search for "stream: DL1"): http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/15776f65b66...
I don't think there's any other way for dnf to tell that an RPM is part of a module before downloading it, but currently each time a module build is published it appears to push the previous version out of the "modules" repository metadata.
This leads to the current behaviour where disabling a module only hides RPMs from the latest module build, and earlier versions "leak out" into transactions, eg: $ sudo dnf -y module disable go-toolset ... $ sudo dnf install golang (incorrectly offers to install package from previous module build)
Looks like a real issue to track in dnf/compositor code?
On Mon, Feb 17, 2020, at 08:37, Alexander Bokovoy wrote:
On ma, 17 helmi 2020, Paul Clifford wrote:
On 14/02/2020 14:28, Alexander Bokovoy wrote:
I think both of these builds belong to idm:DL1, just to two different module builds. There are other packages from idm:DL1 module stream that are duplicated.
Module build 253: https://koji.mbox.centos.org/koji/builds?tagID=488 Module build 265: https://koji.mbox.centos.org/koji/builds?tagID=513
As far as I can see the RPMs from older builds are kept in the CentOS repositories but the associated module metadata isn't.
For example, you can see both softhsm packages from the above builds in: http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/Packages/
But the "modules" file currently pointed to by repomd.xml only contains metadata for build 265 (search for "stream: DL1"): http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/15776f65b66...
I don't think there's any other way for dnf to tell that an RPM is part of a module before downloading it, but currently each time a module build is published it appears to push the previous version out of the "modules" repository metadata.
This leads to the current behaviour where disabling a module only hides RPMs from the latest module build, and earlier versions "leak out" into transactions, eg: $ sudo dnf -y module disable go-toolset ... $ sudo dnf install golang (incorrectly offers to install package from previous module build)
Looks like a real issue to track in dnf/compositor code?
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
Thanks all for the reports. We're looking into how to distribute AppStream a little differently to resolve this.
--Brian
On Mon, Feb 17, 2020 at 1:00 PM Brian Stinson brian@bstinson.com wrote:
On Mon, Feb 17, 2020, at 08:37, Alexander Bokovoy wrote:
On ma, 17 helmi 2020, Paul Clifford wrote:
On 14/02/2020 14:28, Alexander Bokovoy wrote:
I think both of these builds belong to idm:DL1, just to two different module builds. There are other packages from idm:DL1 module stream that are duplicated.
Module build 253: https://koji.mbox.centos.org/koji/builds?tagID=488 Module build 265: https://koji.mbox.centos.org/koji/builds?tagID=513
As far as I can see the RPMs from older builds are kept in the CentOS repositories but the associated module metadata isn't.
For example, you can see both softhsm packages from the above builds in: http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/Packages/
But the "modules" file currently pointed to by repomd.xml only contains metadata for build 265 (search for "stream: DL1"): http://mirror.centos.org/centos-8/8/AppStream/x86_64/os/repodata/15776f65b66...
I don't think there's any other way for dnf to tell that an RPM is part of a module before downloading it, but currently each time a module build is published it appears to push the previous version out of the "modules" repository metadata.
This leads to the current behaviour where disabling a module only hides RPMs from the latest module build, and earlier versions "leak out" into transactions, eg: $ sudo dnf -y module disable go-toolset ... $ sudo dnf install golang (incorrectly offers to install package from previous module build)
Looks like a real issue to track in dnf/compositor code?
-- / Alexander Bokovoy Sr. Principal Software Engineer Security / Identity Management Engineering Red Hat Limited, Finland
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
Thanks all for the reports. We're looking into how to distribute AppStream a little differently to resolve this.
--Brian
In the longer term, can we discard modules? Or encourage RHEL upstream to phase them out? I've not seen any reports of people actually using them successfully, only reports of their breaking dependency chains and causing obscure version conflicts when using the "--best" option for mock or dnf package selection.
On Mon, Feb 17, 2020 at 10:02 PM Nico Kadel-Garcia nkadel@gmail.com wrote:
In the longer term, can we discard modules? Or encourage RHEL upstream to phase them out? I've not seen any reports of people actually using them successfully, only reports of their breaking dependency chains and causing obscure version conflicts when using the "--best" option for mock or dnf package selection.
It is unlikely that anyone here can convince Red Hat to drop them, for some very important reasons:
1. They work *incredibly* well for container use-cases, and that's Red Hat's bread and butter now 2. They minimize the cognitive overhead for consumption use-cases, which outstrip production use-cases (package builds) by a lot 3. Most of the issues with AppStreams/modules are due to incoherency in the specification and disagreements among implementers (fixable with a lot of work!) 4. We're stuck with it for at least a decade because of RHEL 8
Personally, I've spent a little over a year working on implementing modularity support in my package build tooling. I'm nearly done now, and I'm pretty pleased with myself. That said, I have a long list of complaints and suggestions for improvements. We'll see how things improve over the next year or so. But, the folks I work with who are focused on containers really love AppStreams because it gives them the flexibility they want while giving them a fully-supported path to fresh software.
I'm choosing to take the opportunity to see if I can help make the next generation of this stuff better. We'll see how that goes, but I'm optimistic. It's a hard problem, we're ~40% of the way there, and if solved right, would be a huge boon for the community.
On Mon, Feb 17, 2020 at 10:51:28PM -0500, Neal Gompa wrote:
- Most of the issues with AppStreams/modules are due to incoherency
in the specification and disagreements among implementers (fixable with a lot of work!)
Since the implementers are all now concentrated in the same team, this should be on track to get better. Not that I'm taking sides in any such disagreements, but I think this is a good move for avoiding future ones. :)
On Mon, Mar 9, 2020 at 5:14 PM Matthew Miller mattdm@mattdm.org wrote:
On Mon, Feb 17, 2020 at 10:51:28PM -0500, Neal Gompa wrote:
- Most of the issues with AppStreams/modules are due to incoherency
in the specification and disagreements among implementers (fixable with a lot of work!)
Since the implementers are all now concentrated in the same team, this should be on track to get better. Not that I'm taking sides in any such disagreements, but I think this is a good move for avoiding future ones. :)
As long as we don't hold backwards compatibility behavior as sacrosanct, then we have a chance. :)
-
Alexander Bokovoy -
Brian Stinson -
Matthew Miller -
Neal Gompa -
Nico Kadel-Garcia -
Paul Clifford