[CentOS-devel] [EXT] Balancing the needs around the CentOS platform

On 19/12/2020 20:37, Trevor Hemsley via CentOS-devel wrote:
> On 19/12/2020 20:09, Gordon Messmer wrote:
>> On 12/19/20 1:13 AM, Peter Georg wrote:
>>> However I disagree on one point: You mention CentOS Stream can cover 
>>> "95% (or so)" of the current users. As long as 3rd party drivers are 
>>> not working with CentOS Stream the number will be lower. 
>>
>>
>> Do you have a reason to think that > 5% of users need 3rd party drivers?
>>
>>
>>> In my opinion, the only way to guarantee compatibility is by 100% 
>>> kernel ABI compatibility.
>>
>> Stream is going to continue to get the same kernel that RHEL gets, 
>> it's just going to get updates earlier.
> 
> No, it's not, That's the point. It's going to have the unstable kernel 
> from the NEXT point release so it will not be the same kernel as the 
> current RHEL point release. Case in point, RHEL 8.3 has 4.18.0-240 
> series kernels, Stream had changed to a 4.18.0-257 kernel within 2 or 3 
> days of CentOS 8.3 being released.
> 
>> In the past, users with binary 3rd party drivers would need to wait a 
>> while after a point release before they could update their kernel.  In 
>> CentOS Stream, they'll have to wait a while after a kernel package 
>> update before they can update their kernel.
> 
> And that change happened on a predictable, approximately 6 month 
> interval. And required the third party drivers to be rebuild once per 
> point release and usually, by the time the CentOS build was done, those 
> drivers were already rebuilt and already available since they had been 
> done for the preceding RHEL point release.
> 
> So we're going from a once every 6 months rebuild to a once every time 
> the RHEL kernel developers feel like changing. Those changes were 
> abstracted from the CentOS and RHEL kernels by the fact that KABI 
> changes only happened (intentionally!) at point releases. Now that will 
> change whenever someone in the kernel development team commits a 
> breaking change. So we're going from a rebuild the drivers every 6 
> months to rebuild the drivers whenever RH feel like breaking it. That 
> could be once a month or once a week or every time the kernel is updated 
> or even once every 6 months like stable RHEL (though the chances of that 
> are slim IMO). And since RHEL will still be using the old kernel series, 
> that means that the third party yum repos have to invent a new way to 
> cater for both RHEL and Stream since they will be different.
> 
> Trevor

This whole argument is all a matter of perspective, from which side of 
the fence you are looking.

As a *developer* of 3rd party driver packages *for RHEL*, CentOS Stream 
will work well for me as a CI develop system to help me develop driver 
packages for the *next* RHEL release (8.4). I get to see what changes 
are *coming* and can develop against them. That said, I kind of already 
had that ability from the old RHEL point release betas but I gather 
moving forward we will get far more kernel updates to Stream than we 
have to date. To be clear, we develop for RHEL. We will not be 
developing *for* Stream, but we may well use Stream to help develop for 
the next RHEL point release.

So for me as a developer, Stream is fine. But for my users, the 2 
million or so people who use and depend upon the software package we 
distribute, Stream is a non-starter. These packages are developed for 
RHEL (and CentOS Linux as a binary compatible clone). Stream is not a 
replacement for CentOS Linux. CentOS Linux is dead, gone (or rather 
going). Those Enterprise Linux users viewing Stream as a replacement to 
CentOS Linux, they are trying to bash a square peg into a round hole if 
they have a requirement to use 3rd party drivers.