[CentOS-devel] https://blog.centos.org/2020/12/future-is-centos-stream/

Hello.

I'm a vanilla Linux kernel developer and I'm not a RHEL/CentOS developer.
I have several comments (from mostly kernel perspective).



(1) As a kernel developer who is also maintaining out of tree security modules:

Vanilla kernels do not use LINUX_VERSION_CODE, for they care about only latest version.
But not all driver modules for Linux kernel can be included into vanilla Linux kernels,
for there is delay between the kernel for some distribution's some version was written
and latest hardware become available. As a result, driver modules for Linux kernel have
to support multiple kernel versions.

Although I'm not maintaining driver modules for Linux kernel, RHEL kernel's ABI
whitelist is not useful for me. What is useful for me is RHEL_MAJOR/RHEL_MINOR symbols
defined in RHEL kernel's Makefile. Such symbols allow out of tree modules to be
installed into individual /lib/modules/$(uname -r)/ directory.

What might be more useful for driver modules for Linux kernel is more fine grained
symbols (like the configure script used when compiling userspace applications).
That is, I wish RH to add a config symbol which can be used via #ifdef when making
a change (e.g. changing number/type of arguments) that might affect out of tree
modules. Then, out of tree modules can use #if for supporting multiple kernels and
make it easier to follow RHEL kernels (i.e. reduce burden for out of tree code
maintainers).

(2) As a technical staff at a support center troubleshooting RHEL/CentOS servers:

I analyzed vmcore (kernel crash dumps) when I was working at a support center.
What is sometimes troublesome and risky is that we need debuginfo rpm files.

While CentOS is quickly publishing binary rpm files when RH published an update
(except for update of minor release which is mitigated by using CR repo), I feel
that availability of src rpm and debuginfo rpm is rather unreliable.

When using CentOS in enterprise servers, all binary/src/debuginfo rpm files are
expected to be available without delay and remain available even after end of life.
RH has infrastructure for serving all these files, but CentOS's infrastructure looks
poor.

If CentOS 8 is moved to CentOS Stream, will this latency be solved? (I mean, if RH
is expecting CentOS Stream users to contribute with testing/debugging, I think that
improving such distribution infrastructure is important.)

(3) As a CentOS user:

(Honestly speaking, I don't like the term "support". I have been wondering the
definition of the term "support" in RHEL...)

My understanding of RH's support policy is that if some bug is reproducible in
code enabled by RH, RH is responsible for analyzing/fixing bugs. Therefore, RH
fears bugs which cannot be analyzed/fixed by resources RH can provide, and
disables many code. As a result, many out of tree kernel modules need to exist.

Since out of tree kernel modules cannot be signed by RH's module signing keys,
we can't forbid loading of unsigned kernel modules (which is not desirable for
security perspective).

Several years ago, RH has changed style of providing RHEL's kernel source code from
individual patches to an already-patched tarball. (The reason was to make it
difficult for competitors to analyze RHEL kernels?) I worry that it became difficult
for out of tree kernel modules maintainers to figure out what has changed, and it
requires longer time for out of tree module maintainers for testing. Also, since
out of tree module maintainers cannot see what will change in next RHEL minor release
until the moment of that release, users are forced to face long time lag before
the availability of out of tree code for RHEL/CentOS kernels. (An example is a rental
server which is using RHEL7 but cannot use latest kernels (basically 1 minor release
behind?) for unknown reason. Maybe some kernel module for that environment is not
ready to support latest kernels. Whatever the reason is, such delay is bad for
security perspective.)



Any chance that RH moves from "RH is responsible for supporting all code RH is
shipping" to "RH ships as much code as possible (basically any GPL code), but
RH supports only some portion of shipped code" ? (Doesn't RHEL already has such
separation via e.g. "Red Hat Enterprise Linux 7 Server (RPMs)" versus
"Red Hat Enterprise Linux 7 Server - Optional (RPMs)" ?)



I came to feel that the problem is caused by strict and clear role boundary between
RHEL and CentOS... RH does everything before the release with only RH resource.
RH has limited resources compared to users' demands, but users outside RH cannot
assist RH with (or participate in) analyzing/fixing/testing (e.g. private Bugzilla).
And given that not many of CentOS users are skillful enough to use Fedora, a solution
will not be to enforce CentOS users to join analyzing/fixing/testing via CentOS stream.
I imagine that just keeping CentOS Linux as-is, and making it possible to assist RH with
(or participate in) analyzing/fixing/testing RHEL code (e.g. make kernel changes easily
traceable by allowing everyone to browse and try before an update is released). That
will give enough period to minimize latency between RHEL minor update is released and
out of tree kernel module becomes ready for users.



After all, the difference between RHEL and CentOS might be converged to that RH provides
customers who pay costs via buying RHEL subscription RH's resource to analyze/fix/test...
Oh, why need to distinguish RHEL and CentOS? Just let CentOS users use RHEL code (without
buying RHEL subscription) which makes it possible to convert the overhead of de-branding
into the resource for improving quality of RHEL code?