On Wed, Dec 23, 2020 at 4:38 PM Phil Perry <pperry at elrepo.org> wrote: > > On 23/12/2020 20:50, Matthew Miller wrote: > > On Wed, Dec 23, 2020 at 08:23:29PM +0000, Phil Perry wrote: > >> Take Wireguard VPN as an example. No sooner than upstream fixed the > >> breakage caused by -257 on Monday, -259 landed and broke it > >> again[2]. > > > > > > It seems like Wireguard might be a good example of something for an > > alternate kernel maintained by a SIG. (Like the Xen SIG does.) > > > > Why would you do that? The method we use in Enterprise Linux to deliver > 3rd party out-of-tree drivers is the RHEL Driver Update Programme. It > has been this way for over a decade. It works really well. It just > doesn't work for Stream because the Stream kernel is not suitable for > end user (Enterprise) consumption - it is a development kernel for > developing the next RHEL point release. > > If Red Hat really wanted to fix this in (a) kernel, the solution would > have been to accept the repeated upstream requests to backport the > driver into the RHEL kernel, but that idea/request has been rejected. > No. The correct fix here is to start blocking RHEL kernel updates against third-party Free Software kernel module packages to ensure compatibility isn't broken and the kernel ABI stops breaking on every kernel version series. The reason it keeps breaking is because there's no current mechanism in which these are tested together to validate them for release. The LF/RH/SUSE kernel module packaging system (branded as the Driver Update Program by Red Hat) relies on one of two things happening to be reasonably successful: * Gating to ensure kABI doesn't break (RHEL-style) * Continuous automatic rebuilds as the kABI changes (SUSE-style) At work, we've internally implemented the SUSE-style strategy with our RHEL kernel module builds, but we're able to do that because our build system is designed to handle that. Within the CentOS Project with CKI/ARK and CentOS Stream, we should be implementing the RHEL-style strategy. More than most, I get why you're upset about the kABI always breaking as kernel updates push out, but instead of just saying "it's not suitable", we should be building solutions to *make* it suitable for the Enterprise. It's *bad* that the RHEL kernel breaks its own promises so often (which is a relatively new thing, in my experience), and we should be implementing safeguards to stop it from happening going forward. -- 真実はいつも一つ！/ Always, there's only one truth!