[CentOS-devel] [EXT] Re: RFC: kmods SIG Proposal

Thu May 20 22:46:08 UTC 2021
Phil Perry <pperry at elrepo.org>

On 20/05/2021 21:43, Peter Georg wrote:
> On 20/05/2021 21.55, Phil Perry wrote:
>> On 20/05/2021 19:26, Peter Georg wrote:
>>> On 20/05/2021 18.27, Tru Huynh wrote:
>>>> Hi,
>>>> chiming in too
>>>> On Thu, May 20, 2021 at 03:22:16PM +0000, Patrick Riehecky wrote:
>>>>> Adding notes in line below:
>>>>> On Wed, 2021-05-19 at 17:56 +0200, Peter Georg wrote:
>>>> <...>
>>>>>> Proposal:
>>>>>> = kmods SIG =
>>>>>> == Goals ==
>>>>>> The kmods SIG will focus on providing kernel modules currently not
>>>>>> available in CentOS Stream.
>>>>>> == Status ==
>>>>>> Proposed: RFC and looking for a sponsoring Governing Board member
>>>>> As a board member I hereby indicate a willingness to sponsor.  If
>>>>> another board member is passionate about this, I'll be happy to 
>>>>> hand it
>>>>> over to them.  I've regularly made use of external kmods (for old
>>>>> hardware and proprietary hardware)and would strongly like to see them
>>>>> continue in Stream!
>>>> +1
>>>> Emphasizing the possible issues:
>>>> Since we won't have garanteed stable kernel ABI,
>>>> so weak-updates may not always work, so that would imply building
>>>> (all) kmod for each kernel release. Also that would mean extra care
>>>> when the kmod version for a given driver is upgraded along with a newer
>>>> kernel is installed.
>>>> If that can be achieved, elrepo kmod will benefit from this work,
>>>> done in Stream.
>>>> Best effort should be emphasized, volonteered work, thing may break
>>>> etc... Make sure users expectation VS reality is covered.
>>> This issue has been discussed previously on this list. From 
>>> experience reported by Phil Perry with CentOS Stream kernel updates, 
>>> weak-updates are not an option. So yes, all kmods have to be rebuilt 
>>> for each kernel release.
>> At which point I would seriously question the validity of using kmods. 
>> A DKMS-type solution (akmod?) would be more appropriate/beneficial IMHO.
>> That said, if you are intending to update packages for each kernel 
>> release, building them as well becomes less of an issue. From where 
>> will you pull your source code? For example, elrepo pulls updated 
>> sources from the RHEL kernel source code at every point release for 
>> drivers/devices that RH has disabled. So if you are updating the 
>> driver source from the matching Stream kernel source code and patching 
>> to reenable disabled drivers (plus fixing any new build issues), the 
>> package will need a version bump and rebuild for every kernel release 
>> anyway which negates much of the advantages of a dkms-type solution. 
>> Because of weak-updates, ELRepo only needs to do this once every 6 
>> months for RHEL point releases, but you will likely be doing it for 
>> every Stream kernel release. Given the frequency of releases as Stream 
>> ramps up, you'll want to automate as much of this process as you can.
> Just to make sure we are on the same page: We are talking about 
> in-kernel modules here that are not enabled in RHEL or have some 
> devices/adapters removed (i.e. the first two groups described in the 
> proposal).


> Yes, the idea is to always use the updated kernel source code and apply 
> patches on top. This is what I have already done for some packages ~3 
> months ago. So far I only had to update the patches for one module. For 
> all other modules the update could be scripted. However the frequency of 
> CentOS Stream 8 kernel releases has not been that high lately.

It's been a similar story here. So far, the only package I've needed to 
fix for 8.4 has been arcmsr for the simplification of scsi_partsize() 
backported from kernel-5.7, and the patch to revert removed devices 
needed updating for megaraid_sas where Red Hat restored some (but not 
all) of the devices they'd previously removed.

Judging by the number of internal releases the RHEL kernel goes through 
between point releases (-240 to -305 for 8.3 to 8.4), that equates to a 
couple kernel releases per week, on average. I'm guessing that once the 
Stream development process is happening fully in the open, that is the 
number of kernel releases we can expect in Stream.

> IMHO for rebuilding kernel modules with deprecated adapters this is the 
> only really safe option you have anyway (apart from the kernel-plus 
> approach): The current ELRepo approach of pulling source code once for 
> every minor release might run into issues in case these sources being 
> updated in a later kernel update to this particular minor release. It is 
> unlikely to happen, even more unlikely to cause any real issues, but 
> possible.

Agreed. It does happen from time to time but it's pretty rare (as Red 
Hat does all it's development work/backporting in the previously 
unreleased "Stream" branch for RHEL-next once a point release is made). 
All we are seeing in the RHEL kernel is bug/security fixes within any 
given point release cycle hence the kernel ABI is amazingly stable 
within that point release cycle.

>> This is one of the reasons elrepo is not able to support Stream 
>> (massively increased workflow), and I think this is one of the major 
>> challenges you will need to overcome. Because you have access to the 
>> kernel in Stream (or a kernel-plus type offering), I would seriously 
>> consider tying to maintain as much of this as you can in-kernel rather 
>> than as out-of-tree modules. kmods/akmods are great when you have no 
>> alternative, but it's always preferable to do this stuff in-kernel 
>> where you can.
> I agree. That's also why I added in the proposal that it is desired to 
> work with upstream to directly get anything possible into CentOS Stream 
> kernel. However, we all know that this is not feasible for all modules.
> IMHO this is especially not an option for modules of the first group 
> (deprecated adapters). Although some people suggested to try to revert 
> the removal of deprecated adapters in the CentOS Stream kernel, I do not 
> see this happening. Afterall it is still Red Hat deciding what's going 
> into the CentOS Stream kernel.
>  > kernel-plus is of course an option, especially to re-enable support for
> deprecated adapters. However there are use cases where you simply can 
> not replace the kernel with kernel-plus (or have to use some otherwise 
> patched kernel). In these cases the only feasible option (IMHO) is to 
> use external kernel modules. But there is obviously a large overlap 
> (especially for the first and second group of modules) between 
> kernel-plus and this proposed SIG. Hence I hope both efforts can work 
> closely together.

Sounds good.