[CentOS-devel] Balancing the needs around the RHEL platform

On Monday, December 28, 2020 2:01 AM, Alexander Bokovoy <abokovoy at redhat.com> wrote:

> On ma, 28 joulu 2020, Ljubomir Ljubojevic wrote:
>
> > On 12/27/20 6:27 PM, Alexander Bokovoy wrote:
> >
> > > The simplest solution for these use cases is to actually report a bug
> > > against RHEL and/or CentOS Stream and make sure it is reproducible. This
> > > would be the quickest way to get the issue backed out or fixed in a
> > > number of days. There are means to remove broken packages from RHEL
> > > composes and I hope we'd have a way to propagate those 'removals' to
> > > CentOS Stream.
> > > This is something worth raising as a feature request if it doesn't exist
> > > yet.
> >
> > Several people explained how bug reports like that tend end up without
> > solution and even response while Red hat continues to do what they
> > want... So I would not hold my breath that community is going to jump
> > through hoops any more. Majority will just migrate to another clone,
> > leaving developers to play "find the bug" with Red Hat.
>
> I am personally not happy with 'old' CentOS because I couldn't and still
> cannot influence even a tiny bit of what happens with CentOS after a RHEL
> release. Multiple times my components were broken in CentOS because they
> weren't built in the right order and weren't rebuilt for months even
> after users opened CentOS bugs, with zero transparency into what's
> happens.

That indicates bugs with Red Hat's SPEC files and assumptions made by Red Hat about the behavior of their own build system.  The order in which packages are built should be reflected in the BuildRequires lines.

If Red Hat was using a standard COPR environment or pulicly documented it's build system then there should be no reason it shouldn't produce the same results as CentOS.  Instead it seems RHEL developers leave things out of their SPEC files based on assumptions of the behavior of RHEL's own build system.  When undocumented behavior of RHEL's build system produces different results, you shouldn't be surprised when the build order is different.

This change will not fix the SPEC files and doesn't get RHEL's build system publicly documented.  Instead it just shuffles the dirt around in the name of "progress."  We will get a fragmented CentOS community.  You will get to be frustrated with Rocky Linux and CloudLinux Clone using vanilla COPR and reproducing RHEL's poorly written SPEC files.

> I am one of those people who can fix the bugs raised against CentOS
> Stream. I would like to see them raised. My QE teams will certainly
> treat CentOS Stream bugs similar to RHEL bugs as that improves our
> ability to deliver RHEL releases with less issues. In the end, you are
> the ones who get the benefit once the fixes are out there.
>
> At least in my area of interest other distributions aren't better than
> 'old' CentOS or RHEL, there is no magic wand that could help fix issues
> without them being reported and addressed by engineers involved in
> upstream projects. If your's only contribution is by raising bugs,
> please continue doing so, it helps, really.

What fix did we get for Bugzilla #1186352?  Was the lack of Stream the reason for fixing the bug being refused?  Now that Stream exists, will the bug be reopen and will we get the benefit of a real fix?

There are several other bugs I can find that was treated similarly by Red Hat.

I don't expect Red Hat to be able to solve every single bug.  I also acknowledge there are several cases when openning a bug has resulted in fixes for everyone.

However, the assertion that terminating CentOS 8 before 2022 will help fix bugs does not pass the smell test.

What the CentOS Governance Board has done is put a chilling effect on adoption of CentOS 8.  The EOL date of CentOS 7 is now a bonus over using CentOS 8.  Why would anyone do that?  The lack of transcript to the board meeting makes this move even more troubling.  There still is no straight answer about why slowing adoption of CentOS 8 is a good thing for the community.

There is also no clear answer right now why fragmenting the CentOS community is good for the CentOS community.

But there is one "magic wand" that can help fix issues without a bug report which is to empower the community with the resources that should come with being the upstream provider.  Right now the Stream kernel has four patches which are all for debranding.  The 4.18 kernel tar.xz file in the SRPM is 110% the size of a vanilla tar.xz for 4.18.  What is that additional 10%?  How many patches are there?  How many are security patches?  How many are bug patches?  How many are feature patches?  How many patches were added between releases?  How does a CentOS Stream contributor roll back individual patches?

If killing CentOS 8 before 2022 is part of an effort to close the openness gap and to better address bugs, then let's /*REALLY*/ do that with a kernel SRPM.  It is time to make the patch information fully open to assist the community in fixing bugs.

But if the kernel SRPM can be left in the horrific state it currently is, then stop hitting us with all this empty posturing about the so-called positives we get out of killing CentOS 8 early.