[CentOS-devel] A Big Idea for a New Decade [was: Minutes for CentOS Board of Directors 2019-12-18 Meeting]

Neal Gompa

ngompa13 at gmail.com
Tue Jan 7 12:09:57 UTC 2020

On Mon, Jan 6, 2020 at 4:13 PM Matthew Miller <mattdm at mattdm.org> wrote:
> On Fri, Jan 03, 2020 at 03:13:53PM -0500, Karsten Wade wrote:
> > As a centralizing effort, the Board agreed to revisiting the goals
> > document created five years ago, and to undergo an effort to refresh those
> > goals in the light of the project’s evolution. The Board will be inviting
> > various stakeholders into these discussions as we undergo a public
> > revision of the goals at the start of 2020.
> Wow, five years goes fast! I'd like to raise an idea for discussion and
> thought, and possible further discussion between the Fedora Council and the
> CentOS board. [I'm posting this in both groups.]
> Around the same time the CentOS goals were being created, Fedora was working
> on plans for "Fedora.next", which included among other things Fedora Server.
> I won't rehash all that here, but in short: while that succeeded at some of
> its goals, it never really had the user adoption we'd hoped for. Of course,
> some of that is simply a matter of lifecycle: while a fast-moving server OS
> is useful in some very real cases, it's decidedly niche.
> When Fedora launched that plan, CentOS wasn't really in the "family", and
> after, we just kind of existed in parallel, with some very loose
> collaboration in areas like EPEL.
> Of course, things are different now. Therefore, I propose that in this next
> decade, we make that collaboration stronger. Together, we have a very nice
> answer to the fundamental problem of unified operating systems. That is,
> some parts move too fast and other parts move too slowly. Between Fedora and
> CentOS, we have answers to both! And, with the in-progress dist-git merge,
> we even have both _in the same place_.
> Specifically, I'd like to suggest three things.
> First, let's replace Fedora Server as a user-focused artifact with CentOS
> Stream. There's still a need for a Fedora Server as an upstream, but most
> users of Fedora as a server are making custom builds (or using the basic
> Cloud image), not using Fedora Server per se. So, I think we should stop
> marketing that to users, and steer people with server use cases outside of
> the fast-moving niche to CentOS Stream. The current story of CentOS and
> Fedora is pretty confusing to users, and Stream only makes it more so. Let's
> simplify things!
> Second, where there's overlap, let's look at merging Fedora and CentOS SIGs
> into joint bodies. We have groups like the cloud image SIGs which are
> basically doing the exact same thing in two different ways. Same thing with
> containers. We also have huge overlaps in documentation and community user
> support and all sorts of non-technical project areas. I'm not saying these
> all need to be smooshed together, but where it makes sense let's work
> together. This, too, brings simplification to the "what's with CentOS vs.
> Fedora" story: rather than making contributors pick, we can welcome all with
> open arms.
> And finally, let's really take advantage of a merged dist-git repository,
> and the capabilities that Modularity gives us. SIGs building things for
> CentOS should be able to directly reference Fedora branches where
> faster-moving software is useful — and people building things in Fedora
> should be able to pull in branches from Stream where a slower-moving version
> would make sense. And we could even make additional shared branches where
> necessary, using modularity to make optional streams which could be used in
> CentOS SIGs, EPEL, or a Fedora OS.
> What do you think?

First, sorry that this email probably doesn't thread will with either
one or the other target mailing lists... I had to pick one to thread
with, and other slightly loses. The downside of the initial email not
being sent to both MLs at once. :(

In concept, I certainly see value in bringing CentOS and Fedora
closer. However, I think eliminating Fedora variants in favor of
CentOS Stream ones (aka rhel-rawhide based variants) is probably
premature. There are several issues with doing this as it currently

1. CentOS Koji is pretty closed to a couple of folks in CentOS and Red
Hat. This also includes blocking the download of artifacts from Koji.
For reasons I don't understand, CentOS is still gated by the RHEL QE
folks, and thus RPMs built in Koji cannot be downloaded until they are
finally published. This ruins any testing/development before releasing
to users. The CBS is a worthless substitute since it's purely for
addons. I've been around long enough to remember when Fedora was like
this, and back then, it wasn't really considered "okay" for Fedora to
be like that (the intent was always there to fix it, and we eventually
did!). Today, however, it is considered "okay", and that's a problem.

2. The CentOS community is simply not mature enough to take on the
role Fedora does in any meaningful capacity. We already know that this
is a problem even with Fedora EPEL, and I strongly believe the issue
would be massively magnified if those transitioned to the CentOS
Project. The CentOS community is largely a user community that takes
and does not give back much, if at all. I've experienced variations of
this issue when people (even from Red Hat!) have done finger-pointing
to *me* with my packages in EPEL without any acknowledgement that they
could have helped. It's depressing that even some Red Hatters consider
EPEL not worth contributing to, but still worth being upset about.

3. Today, the CentOS Project is (sadly) a community project in a very
superficial sense. Aside from the Virt SIG's Xen subgroup, all SIGs
are basically Red Hat product teams, with varying degrees of
allowances for community contributions. The community processes within
CentOS are mostly nonfunctional because they've had no reason to
develop it. There has been very little community development for
CentOS. Even with Fermilab and CERN shutting down Scientific Linux, I
haven't seen anything from those people coming "into the fold".
There's been no outreach or collaboration with other CentOS based
derivatives, commercial or otherwise. The private discussions I've had
on the matter seem to indicate that there's still ambivalence, if not
outright antipathy and hostility toward the broader ecosystem of
CentOS derivatives.

What I'm more concerned about is that if you eliminate Fedora from any
meaningful server based development, you strip all the opportunities
for people to iterate on server-oriented changes before they go into
rhel-rawhide and push into CentOS Stream. You also essentially kneecap
any motivation for other things related to server environments to
iterate faster (such as language stacks that are heavily used for web
service software) because you've eliminated the major ability for that
to ship to users and contributors. It also further accelerates a trend
that I think we need to reverse where people consider Fedora
unacceptable for server roles. If anything, Fedora is a lot better at
being used for servers then it was five years ago. I've personally
*stopped* using CentOS for servers because Fedora has gotten so good
at it. Upgrades are a breeze and stuff generally works. When it
doesn't, it's fixable! That last part is key. With CentOS, it's not,
because it has to bounce back into Red Hat first. And Red Hat doesn't
really care about issues discovered by CentOS users, and there are no
"CentOS developers".

So, what this long email is actually saying is that I think it's an
interesting idea to bring the two projects together, but eliminating
aspects of Fedora in favor of CentOS is premature because CentOS has
not actually developed as a community project. Maybe it's worth
revisiting after six years of actual community development?

真実はいつも一つ!/ Always, there's only one truth!

More information about the CentOS-devel mailing list