[CentOS-devel] Balancing the needs around the CentOS platform

On Mon, Dec 21, 2020 at 7:59 AM Stephen John Smoogen <smooge at gmail.com> wrote:
> On Sun, 20 Dec 2020 at 22:19, Mark Mielke <mark.mielke at gmail.com> wrote:
>> > > CentOS point releases do *not* match as closely as possible the
>> > > corresponding RHEL point release?
>> > No, no one is saying that.  Matthew said that you can stay at a minor
>> > release of RHEL and still get security updates.  CentOS does not offer that.
>> This is not correct. Please stop saying it. CentOS is bug-for-bug
>> compatible with RHEL for *active* releases.
> We came up with the phrase "bug-for-bug" compatible during EL5 as a GOAL to aim for. CentOS was NEVER bug-for-bug compatible. We aimed for it like a target to get to but we also had to release the software eventually and don't have the extensive testing mechanisms to prove 'bug-for-bug' compatibility. Sometimes CentOS shipped packages which did not have a particular bug because we could not exactly duplicate the build environment and other times we added new bugs because our build environment is not exactly the same.

This is a good point of clarification. But, to be clear in both
directions, please let us know if you agree that these points are
correct:

1. CentOS was intended to be "bug-for-bug" compatible with RHEL as a GOAL.
2. This was especially difficult when Red Hat was not contributing to
the project, as it involved a lot of guesswork and trial and error.
3. This became easier when Red Hat began contributing, including
staging the source for build, and providing the build engines.
4. It's still not perfect, however, in almost every way there is
currently a 1:1 relationship between the exact .spec file and patch
set being used to build CentOS releases, in parallel (or a short time
later) to RHEL minor release.
5. CentOS 8 Stream is a departure from the above. What is built in
CentOS 8 Stream is *not* intended to be "bug-for-bug" compatible, but
is instead a moving target that represents the *next* RHEL minor
release. At any given point in time, it is not necessarily aligned
with either RHEL X.N, or RHEL X.N+1, but is something in between, or
may even be something beyond, such as RHEL X.N+2 depending upon RHEL
internal processes.

> Over the years as software has grown in greater complexity this divergence has become more likely. Modularity, software like rust, go, etc all require long build chains where any 'divergence' can introduce or remove bugs without a quick way to test.

Yep. The reproducibility problem inherent to RHEL has become more
complex, rather than less. Dropping CentOS 8 essentially abandons this
problem from a Red Hat perspective. Changing the subscription terms
for RHEL to allow the RHEL binaries to be directly used would solve
this problem from a Red Hat perspective (but possibly introduce
others, mostly relating to coming up with a better financially
sustainable model). Documenting (possible as configuration and code)
the build requirements for packages may be a cleaner solution.

> At best, CentOS has been "good-enough" compatible for a set of years. The only way to be bug-for-bug compatible would require a reproducible build and test environment.

I think "good-enough" is in the 99.9%+ today, depending upon what the
test is. With CentOS 8 Stream, it drops to something less, also
depending upon what the test is.

I think the people who understand the delta best, may be
under-representing the value they provided. The people who understand
it the least, never really appreciated the effort. :-)

-- 
Mark Mielke <mark.mielke at gmail.com>