Dear all, is kernel live patching working for CentOS Stream 9?
I know that I can enable it via cockpit or the command line, but are kernel patch files really available or do they require an active RHEL subscription?
Thanks.
On Fri, Jan 7, 2022 at 11:39 AM Gionatan Danti g.danti@assyoma.it wrote:
Dear all, is kernel live patching working for CentOS Stream 9?
I know that I can enable it via cockpit or the command line, but are kernel patch files really available or do they require an active RHEL subscription?
Thanks.
-- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti@assyoma.it - info@assyoma.it GPG public key ID: FF5F32A8 _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
There are 2 things to note here about kernel live patching: - We do not provide patch files in CentOS Stream (or previously in CentOS Linux, for that matter). We've always recommended RHEL as a better fit for folks that have hard requirements on this sort of workflow. - RHEL 9 is not yet released, my understanding is that patch files for RHEL proper will begin showing up after GA
--Brian
On 1/7/2022 10:07 AM, Brian Stinson wrote:
- We do not provide patch files in CentOS Stream (or previously in
CentOS Linux, for that matter). We've always recommended RHEL as a better fit for folks that have hard requirements on this sort of workflow.
If Stream is to be the next RHEL, wouldn't you want to test this kind of thing so the RHEL subscribers don't have to?
On Fri, Jan 7, 2022 at 1:36 PM Kenneth Porter shiva@sewingwitch.com wrote:
On 1/7/2022 10:07 AM, Brian Stinson wrote:
- We do not provide patch files in CentOS Stream (or previously in
CentOS Linux, for that matter). We've always recommended RHEL as a better fit for folks that have hard requirements on this sort of workflow.
If Stream is to be the next RHEL, wouldn't you want to test this kind of thing so the RHEL subscribers don't have to?
RHEL tests live patches for released, supported kernels before they are delivered to customers.
josh
Il 2022-01-07 19:07 Brian Stinson ha scritto:
There are 2 things to note here about kernel live patching:
- We do not provide patch files in CentOS Stream (or previously in
CentOS Linux, for that matter). We've always recommended RHEL as a better fit for folks that have hard requirements on this sort of workflow.
- RHEL 9 is not yet released, my understanding is that patch files for
RHEL proper will begin showing up after GA
Yup, this matches my expectations. Thank you all for sharing. Regards.
On 1/7/22 09:39, Gionatan Danti wrote:
is kernel live patching working for CentOS Stream 9?
https://access.redhat.com/solutions/2206511
My understanding of live kernel patching is that the feature allows systems to update specific individual kernel functions, and is primarily useful for addressing security vulnerabilities (and not, for example, for updating from one kernel version to another). I don't know for a fact, but my expectation is that CentOS Stream systems aren't going to get "live" patches because there's no ongoing support for individual kernels.
On 1/7/22 12:18, Gordon Messmer wrote:
On 1/7/22 09:39, Gionatan Danti wrote:
is kernel live patching working for CentOS Stream 9?
https://access.redhat.com/solutions/2206511
My understanding of live kernel patching is that the feature allows systems to update specific individual kernel functions, and is primarily useful for addressing security vulnerabilities (and not, for example, for updating from one kernel version to another). I don't know for a fact, but my expectation is that CentOS Stream systems aren't going to get "live" patches because there's no ongoing support for individual kernels.
Indeed .. you will get those things rolled into the next kernel, but not as live patches.
On 1/13/22 12:28 PM, Johnny Hughes wrote:
On 1/7/22 12:18, Gordon Messmer wrote:
On 1/7/22 09:39, Gionatan Danti wrote:
is kernel live patching working for CentOS Stream 9?
https://access.redhat.com/solutions/2206511
My understanding of live kernel patching is that the feature allows systems to update specific individual kernel functions, and is primarily useful for addressing security vulnerabilities (and not, for example, for updating from one kernel version to another). I don't know for a fact, but my expectation is that CentOS Stream systems aren't going to get "live" patches because there's no ongoing support for individual kernels.
Indeed .. you will get those things rolled into the next kernel, but not as live patches.
In layman's language summary: RedHat Enterprise features (including "live" kernel patching) are to be expected _only_ in RedHat Enterprise "binary replica" distributions, which CentOS Stream is not.
Valeri
On 1/13/22 09:32, Valeri Galtsev wrote:
In layman's language summary: RedHat Enterprise features (including "live" kernel patching) are to be expected _only_ in RedHat Enterprise "binary replica" distributions, which CentOS Stream is not.
I don't think that's true, exactly. As far as I know, rebuild distributions never had the "Enterprise" features*. Critically, I think that a lot of people mistakenly believed that CentOS *did* have Enterprise features, because it was rebuilt from RHEL code, and that misunderstanding underlies a great deal of the negative response toward CentOS Stream.
*: "Enterprise" features include but are not limited to:
1. Minor releases with independent life cycles / Extended Update Support 2. Classification for updates (security, bugfix, enhancement) 3. Live patching for kernel security vulnerabilities 4. Support
On 1/13/22 1:01 PM, Gordon Messmer wrote:
On 1/13/22 09:32, Valeri Galtsev wrote:
In layman's language summary: RedHat Enterprise features (including "live" kernel patching) are to be expected _only_ in RedHat Enterprise "binary replica" distributions, which CentOS Stream is not.
I don't think that's true, exactly. As far as I know, rebuild distributions never had the "Enterprise" features*. Critically, I think that a lot of people mistakenly believed that CentOS *did* have Enterprise features, because it was rebuilt from RHEL code, and that misunderstanding underlies a great deal of the negative response toward CentOS Stream.
Thanks for correcting my layman's representation. It should have better said that "binary replica" is "binary compatible" in a sense whatever software distributed as binary for RHEL will work the same on "binary replica". I guess my views and wordings got skewed by latest changes of CentOS paradigms.
*: "Enterprise" features include but are not limited to:
- Minor releases with independent life cycles / Extended Update Support
- Classification for updates (security, bugfix, enhancement)
- Live patching for kernel security vulnerabilities
We never had it in CentOS in the past, but I'm just curious: is live patching proprietary piece of RHEL? I know there are several solutions, way back there was paid one called splice, my Boss's son was one of the developers of that. Just curious, as, if it is paid, it is stripped off as part of CentOS composition, but if it is not paid, open source, then it would "just work", or not?
- Support
Oops, as features I meant functionality of CentOS, nothing beyond that.
Valeri
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
--On Thursday, January 13, 2022 2:10 PM -0500 Valeri Galtsev galtsev@kicp.uchicago.edu wrote:
We never had it in CentOS in the past, but I'm just curious: is live patching proprietary piece of RHEL? I know there are several solutions, way back there was paid one called splice, my Boss's son was one of the developers of that. Just curious, as, if it is paid, it is stripped off as part of CentOS composition, but if it is not paid, open source, then it would "just work", or not?
Indeed, we're talking the software versus the organization. I never expected CentOS the organization to provide anything more than repackaging (rebuilding and mirroring).
For kernel patching, there's the matter of rebuilding and distributing the patches, and then whether the software can do anything with that. If it's proprietary, the issue is moot.
But maybe it's like the update classification and differentiation, which was never implemented for CentOS, because of the extra effort the organization would have to provide.
On Thu, Jan 13, 2022 at 2:13 PM Valeri Galtsev galtsev@kicp.uchicago.edu wrote:
On 1/13/22 1:01 PM, Gordon Messmer wrote:
On 1/13/22 09:32, Valeri Galtsev wrote:
In layman's language summary: RedHat Enterprise features (including "live" kernel patching) are to be expected _only_ in RedHat Enterprise "binary replica" distributions, which CentOS Stream is not.
I don't think that's true, exactly. As far as I know, rebuild distributions never had the "Enterprise" features*. Critically, I think that a lot of people mistakenly believed that CentOS *did* have Enterprise features, because it was rebuilt from RHEL code, and that misunderstanding underlies a great deal of the negative response toward CentOS Stream.
Thanks for correcting my layman's representation. It should have better said that "binary replica" is "binary compatible" in a sense whatever software distributed as binary for RHEL will work the same on "binary replica". I guess my views and wordings got skewed by latest changes of CentOS paradigms.
*: "Enterprise" features include but are not limited to:
- Minor releases with independent life cycles / Extended Update Support
- Classification for updates (security, bugfix, enhancement)
- Live patching for kernel security vulnerabilities
We never had it in CentOS in the past, but I'm just curious: is live patching proprietary piece of RHEL? I know there are several solutions, way back there was paid one called splice, my Boss's son was one of the developers of that. Just curious, as, if it is paid, it is stripped off as part of CentOS composition, but if it is not paid, open source, then it would "just work", or not?
RHEL's kernel live patching uses upstream open source kpatch. The sources to the kpatches are delivered in customer facing CDN repos at the same time as the kpatch itself. We do not use proprietary code to produce or apply the kpatches.
I can only speculate on whether RHEL kpatches would work on a CentOS kernel, but my assumption is that they would not due to how they are signed.
josh
Il 2022-01-14 13:17 Josh Boyer ha scritto:
RHEL's kernel live patching uses upstream open source kpatch. The sources to the kpatches are delivered in customer facing CDN repos at the same time as the kpatch itself. We do not use proprietary code to produce or apply the kpatches.
I can only speculate on whether RHEL kpatches would work on a CentOS kernel, but my assumption is that they would not due to how they are signed.
Is (well, was) the CentOS kernel identical at binary level to the RHEL one? If so, the same kpatch should be applicable to both RHEL and CentOS (the old one).
But I seem to understand that the two kernels are *not* bytewise identical, so a binary kpatch can not be applied the CentOS. Is this true?
Anyway, RH kpatches are surely not compatible with CentOS stream. So I asked if some project was started to provide live kernel patching to the new CentOS project. If I don't miss something, this is not the case.
Regards.
On 1/14/22 07:57, Gionatan Danti wrote:
Il 2022-01-14 13:17 Josh Boyer ha scritto:
RHEL's kernel live patching uses upstream open source kpatch. The sources to the kpatches are delivered in customer facing CDN repos at the same time as the kpatch itself. We do not use proprietary code to produce or apply the kpatches.
I can only speculate on whether RHEL kpatches would work on a CentOS kernel, but my assumption is that they would not due to how they are signed.
Is (well, was) the CentOS kernel identical at binary level to the RHEL one? If so, the same kpatch should be applicable to both RHEL and CentOS (the old one).
But I seem to understand that the two kernels are *not* bytewise identical, so a binary kpatch can not be applied the CentOS. Is this true?
Anyway, RH kpatches are surely not compatible with CentOS stream. So I asked if some project was started to provide live kernel patching to the new CentOS project. If I don't miss something, this is not the case.
Regards.
No .. none of the CentOS Kernels were EVER binary compatible with any RHEL kernel.
CentOS Linux has always been (now also including CentOS Stream 8 and 9) a completely separate 'closed' build system.
We use the SAME source code to build things, modified to remove branding. But CentOS has NEVER been (nor is any other rebuild distribution now) Binary Compatible.
Want to see how .. just extract two rpms with the same name from two different distributions into separate directories and run a sha256sum on all the files in the different directories with find command. Some files may be identical (most text files that are copied), others will not be.
It is virtually impossible for all produced packages to be 'binary compatible' UNLESS they are built with exact the same files (not files BUILT fromt he same sources .. the exact same files) in the build root AND with exactly the same software doing the building. Any group that claims 'binary compatibility' is either lying or they do not understand compiling and linking.
CentOS never had that. Neither does any rebuild.
This is why the CentOS Project 'CHANGED' our term from binary compatible to 'Functionally Compatible' a long time ago. (Using same source code, we produce DIFFERENT software .. that works the same way but has different SHASUM values. Don't be fooled by key words like 'binary compatible' .. check it out for yourself.
If you build kpatches to kernels, to make them work you need to build the kpatch for the specific kernel (CentOS would need to build against CentOS kernels, etc). Also, there are the certificate signing issues and keys that you would need to take into account. You need to have the CA Trust to be able to create signatures that the system will allow.
On 1/14/22 08:30, Johnny Hughes wrote:
On 1/14/22 07:57, Gionatan Danti wrote:
Il 2022-01-14 13:17 Josh Boyer ha scritto:
RHEL's kernel live patching uses upstream open source kpatch. The sources to the kpatches are delivered in customer facing CDN repos at the same time as the kpatch itself. We do not use proprietary code to produce or apply the kpatches.
I can only speculate on whether RHEL kpatches would work on a CentOS kernel, but my assumption is that they would not due to how they are signed.
Is (well, was) the CentOS kernel identical at binary level to the RHEL one? If so, the same kpatch should be applicable to both RHEL and CentOS (the old one).
But I seem to understand that the two kernels are *not* bytewise identical, so a binary kpatch can not be applied the CentOS. Is this true?
Anyway, RH kpatches are surely not compatible with CentOS stream. So I asked if some project was started to provide live kernel patching to the new CentOS project. If I don't miss something, this is not the case.
Regards.
No .. none of the CentOS Kernels were EVER binary compatible with any RHEL kernel.
CentOS Linux has always been (now also including CentOS Stream 8 and 9) a completely separate 'closed' build system.
We use the SAME source code to build things, modified to remove branding. But CentOS has NEVER been (nor is any other rebuild distribution now) Binary Compatible.
Want to see how .. just extract two rpms with the same name from two different distributions into separate directories and run a sha256sum on all the files in the different directories with find command. Some files may be identical (most text files that are copied), others will not be.
It is virtually impossible for all produced packages to be 'binary compatible' UNLESS they are built with exact the same files (not files BUILT fromt he same sources .. the exact same files) in the build root AND with exactly the same software doing the building. Any group that claims 'binary compatibility' is either lying or they do not understand compiling and linking.
CentOS never had that. Neither does any rebuild.
This is why the CentOS Project 'CHANGED' our term from binary compatible to 'Functionally Compatible' a long time ago. (Using same source code, we produce DIFFERENT software .. that works the same way but has different SHASUM values. Don't be fooled by key words like 'binary compatible' .. check it out for yourself.
If you build kpatches to kernels, to make them work you need to build the kpatch for the specific kernel (CentOS would need to build against CentOS kernels, etc). Also, there are the certificate signing issues and keys that you would need to take into account. You need to have the CA Trust to be able to create signatures that the system will allow. _______________________________________________
As a bit of a helper .. we used to use this script to find differences:
https://vault.centos.org/4.9/build/distro/tmverifyrpms
Way back in the CentOS Linux 4 days.
Il 2022-01-14 15:30 Johnny Hughes ha scritto:
No .. none of the CentOS Kernels were EVER binary compatible with any RHEL kernel.
CentOS Linux has always been (now also including CentOS Stream 8 and 9) a completely separate 'closed' build system.
We use the SAME source code to build things, modified to remove branding. But CentOS has NEVER been (nor is any other rebuild distribution now) Binary Compatible.
Want to see how .. just extract two rpms with the same name from two different distributions into separate directories and run a sha256sum on all the files in the different directories with find command. Some files may be identical (most text files that are copied), others will not be.
It is virtually impossible for all produced packages to be 'binary compatible' UNLESS they are built with exact the same files (not files BUILT fromt he same sources .. the exact same files) in the build root AND with exactly the same software doing the building. Any group that claims 'binary compatibility' is either lying or they do not understand compiling and linking.
CentOS never had that. Neither does any rebuild.
This is why the CentOS Project 'CHANGED' our term from binary compatible to 'Functionally Compatible' a long time ago. (Using same source code, we produce DIFFERENT software .. that works the same way but has different SHASUM values. Don't be fooled by key words like 'binary compatible' .. check it out for yourself.
If you build kpatches to kernels, to make them work you need to build the kpatch for the specific kernel (CentOS would need to build against CentOS kernels, etc). Also, there are the certificate signing issues and keys that you would need to take into account. You need to have the CA Trust to be able to create signatures that the system will allow. _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Thank you so much for the detailed explanation, very appreciated. Regards.
-
Brian Stinson -
Gionatan Danti -
Gordon Messmer -
Johnny Hughes -
Josh Boyer -
Kenneth Porter -
Valeri Galtsev