[CentOS-devel] Before You Get Mad About The CentOS Stream Change, Think About…

Hello Ljubomir (and all!).

I never heard your name until a few days ago.

Based on what you wrote in this thread, I first want to thank you personally
for what you did all these years.

I'd like to provide my own POV regarding some of the issues you raise, but
before that, let me briefly introduce myself.

I use Linux since 1993. Started with MCC, then SLS, then played for some
months the game of "I can do this myself" (what's later been called Linux
>From Scratch), then Slackware, and then Debian. For my own machines, for
most of this time, I used Debian. At home I only moved briefly to Ubuntu
for some years and now recently to Fedora, but can definitely see myself
going back to Debian.

At work, I started in a place that used RHL, and also added quite a lot of
Debian there. Then moved to another place that had RHL, and worked on replacing
this with RHEL (3, at 2005). I worked there for quite some time, and also there
introduced Debian in some places, but in the important places, there wasn't
really a question - for production - RHEL, and for testing/debug - CentOS.

Then I moved to Red Hat, and here I work on oVirt/RHV.

I am not representing Red Hat in any capacity. I am just a developer, and
my job is somewhat far from the discussion about RHEL/CentOS/etc. - in my
day job, I am mostly a user of these.

On Thu, Dec 17, 2020 at 11:01 AM Ljubomir Ljubojevic <centos at plnet.rs> wrote:
>
> On 12/17/20 9:13 AM, Peter Eckel wrote:
> > Hi,
> >
> >> On 17. Dec 2020, at 09:03, Simon Matter <simon.matter at invoca.ch> wrote:
> >>
> >> I know that Red Hat was and is free to decide what they want. But I can
> >> assure you that the only reason why quite a number RHEL subscriptions have
> >> been sold to the companies where I have worked in the past is that there
> >> was a project called CentOS!
> >
> > if that is really the case (I think it could well be), the business decision of RedHat maybe makes perfect sense.
> >
> > Some other distribution will step in for CentOS Linux. Rocky, Lenix, Springsdale, whatever. That distribution/s will take the role of CentOS in paving the path for RHEL without RedHat having to paying for it.
> >
> > Sounds like a win/win-Situation, doesn't it?
> >
>
> No, it does not. Because so far Red hat was viewed as champion of Open
> Source and we "freeloaders" felt morally obligated to help Red Hat in
> any way we could. It was the right and honest thing to do.
>
> Since Red Hat displayed greedy and stab-in-the-back attitude (buy hiding
> what wanted to do before they were ready), there is absolutely no moral

I wasn't part of the discussion around CentOS - neither in 2014 nor now -
and the news from last week was a shock to me as well. But if you now go
back and read the announcement from 2014, you can very clearly see that
from the very beginning, Red Hat didn't consider, or implied, or suggested,
or anything like that, that it sees CentOS as a cheap/free RHEL replacement
for the poor. It wasn't presented as _charity_. It was presented, and AFAICT
_was_, for the benefit of Red Hat. Go read it. There is nothing new here.

> obligation to help them in any way, and many now even have negative
> feelings towards another "greedy company".
>
> Before this my message was "If you are going to spend the money on
> Linux, it is best to spend it on RHEL, they give so much to community it
> is only fair."

I also think/thought/talked like that in past jobs.

But I also want to add another reason: Spend it on Red Hat/RHEL, simply
because they are the best, and worth your money. If you do not think so,
don't.

>
> Since few days ago my message is "I do not like them anymore, and I do
> not have trust in them, so better stay clear from them."

I definitely feel your pain. I felt the same way for several days now,
and slowly got used to the new situation, and am now mostly ok with it.

I am not saying it was nice. I am just saying, that right now, I do
agree with upper management here, if they say we simply had no choice
but break this "promise" (of support till 2029), as bad as the community
would accept this.

>
> CentOS project leaders had the same philosophy in mind when they refused
> to add extra packages to CentOS repositories like non-free codecs, 3rd
> party drivers (ElRepo had to be created separately) or even some desktop
> apps or KDE, MATE, etc.

There were at least two other reasons, AFAICT:

1. People do/did not want that. They wanted exactly what CentOS said it
is trying to do - bug-for-bug compatibility with RHEL - so that's what
CentOS did.

2. Doing CentOS as-is was already hard enough. I do not think I have to
remind people the situation it was in, before the Red Hat "acquisition".

> All of that was redirected to Red Hat controlled
> EPEL or 3rd party repositories.
>
> But Rocky Linux and Lenix (CloudLinux) do not have to be constrained
> with these compliance, why should they when most likely Red Hat will do
> their best to complicate creation of other clones any way they can.

If you ask me, CentOS Stream is a giant step forward, for anyone that
wants to rebuild RHEL. I am not sure why consider it "the best Red Hat
can do to complicate" this.

> You
> can say what ever you want, but I and others do not trust them/you to be
> better then their worst deed.

Fair enough.

>
> And there is no legal obligation to use RHEL and not clones in
> production, especially if CloudLinux develops a business model that will
> enhance FOSS clone and eventually spin off from RHEL into competitor
> just like Oracle did. Even Rocky Linux could be backed by some new
> company that will offer paid-for support in production.

Of course!

As other, more senior Red Hatters already said on this thread: Game on.

We do not look for mercy. We think we are good. If you prefer doing
business with Oracle, or CloudLinux, or a (new) company behind Rocky
Linux - go ahead.

If you ask me, a _business_, making money, that decides to base their
supply chain on the promise of a community project, instead of a
contract with a company, is taking a significant risk. Nobody prevents
this, but I'd personally not do that.

>
> Up until this backstabbing act any company that would try to steal
> support income from Red Hat would have been declared greedy by CentOS
> and even Linux community at large. Even today I do not like Oracle
> because they became direct competitor to Red Hat who was spending money
> on development, bugfixes, etc.
>
> But since Red Hat is now in same category as Oracle, greedy corporation,
> EL/Linux community will WELCOME another player in paid-support for RHEL
> clones, and stand by them as long as their actions support needs of "us
> freeloaders". Do you really think CloudLinux decided to spend $1 million
> because they are altruists? I do not. They have seen Red Hat hang them
> selves (nobody provoked them) and saw unique one-in-a-lifetime
> opportunity to expand their portfolio from only light hosting clone
> based on RHEL source to all-purpose distro that will help them expand
> their paid-for support offer to baremetal servers and workstations,
> maybe even laptops. All they have to do is to publish binary clone and
> then expand on that ecosystem by adding repos like ElRepo, EPEL,
> CentOSPlus, and maybe non-free repo and they will be huge success and
> make bundle of money, well worth the investment they are making.
>
> And you know what? I am going to support them, and bee happy for them.
> And direct any money spending THEIR WAY.

Very well. I am not saying you should not.

I do not remember where I read it, but I read somewhere an estimation
that continuing full support of CentOS 8 until 2029 would have cost
Red Hat something like $30-$40 million. I have no reason to think this
is way off. So $1 million suddenly does not look that much.

I'd like to use this opportunity to address some other issues raised
recently (in this thread, perhaps also elsewhere). I'll not quote
the text I am replying to, I hope that's ok.

What is Free Software, and what is Open Source? A lot was written about
this, and I am not going to repeat everything. For current discussion,
the main point I'd like to make is: Open Source is a business model.
It's not (mainly) about giving the famous 4 freedoms to users of your
software, or even about Linus's law "With enough eyeballs, all bugs are
shallow". Red Hat got/gets, for free, the code of many FOSS projects.
In return, Red Hat gives back _all_ the code it added/adds to these,
and other, projects. If you measure the amount of "Being open source"
for a particular company, by dividing to value of what it gave, with
the value of what it got, or, for that matter, with its revenue/income/
whatever, I am fully certain you'll not find any other company in the
world, in a size similar to Red Hat's, with even close-by ratio. Think
about this. Go ahead and think about the companies you know. Sure, many
companies "give back", but how much? Even those that give a lot, in
money, or money-worth, have, AFAIK, way lower ratios. And, BTW, why
does Red Hat do that? Because the people here, like me, make it do so.
Both management, and owners (now IBM, other investors before that),
realize that if a significant change is done to the "wrong" side, too
many people here will quit, and put the company in such a bad position,
that it will simply not be worth it. memo-list was on fire, for the
last week. Not fire - more like a nuclear explosion. Believe me. But
people slowly understand, and things slowly calm down. I currently
simply do not believe that Red Hat can become evil, in this sense.
It will simply be dissolved.

To add to that: IMO, any person/company/whatever that says "Red Hat
pissed me off, I do not want ever to have anything to do with them",
and want to be honest to themselves, must now go and start helping
Debian. Doing, or using, a RHEL rebuild, is not that, and does not
make sense. To me. You want competition? The only real competition
for Red Hat, IMO, is Debian. Go ahead, help them. Or, if you prefer,
SuSE. But a RHEL rebuild still keeps you tightly-related to Red Hat,
strongly dependent on Red Hat, as much as you claim you hate it.

I am now going to take a big risk and talk about Ubuntu. It might
be the stupidest thing I did, ever. Ubuntu/Canonical are not doing
well, financially. The only reason they keep being so well-known,
common, etc., is because they have funding, which is not based on
their income. If rich people, that made their money (also) thanks
to FOSS, want to contribute back, by financing it, such as by doing
Ubuntu, that's adorable, really. But this isn't Red Hat. Red Hat is
a group of people that is trying to make a leaving from Open Source
itself. You might claim this is simply not interesting, or irrelevant,
or stupid, or whatever. You might claim that Red Hat, as-is, simply
has no place in the FOSS world. But many people do not agree, including
both the people inside Red Hat, and also many of its customers, and
so it exists. I'd personally find it very sad if it turns out that
this is wrong - that  there simply is no way to make a business,
make a leaving, from doing FOSS - that I, personally, must either
give up on my principles and work for a non-FOSS company, or give
up on having FOSS as my day job (and do it only as a spare-time
hobby). I still have hopes.

Re the help that people like you provided to Red Hat, for free, by
spending effort on CentOS, then push for using/buying RHEL: Can you
please think for a minute, and explain why CentOS Stream is not almost
the same? For people like you (and me!), that want to play with Linux,
do interesting stuff with/on it, learn it, etc., but not (yet) make
money from it, build a business around it, does the difference between
CentOS Linux and CentOS Stream really matter? Why? You do not have to
answer right now. You can try it, then decide. I completely ignored
it until the announcement last week, and now, since the oVirt project
realized that migration to Stream is the most reasonable choice,
started moving, and already yesterday ran into a bug and fixed it
this morning, https://bugzilla.redhat.com/1908602. Is that really
so bad? You might say: But I do not have money, and want/need to run
real stuff in production, and need the stability, security fixes,
etc. Fine. Was CentOS so good for you so far? CentOS was also weeks
or months behind RHEL for all of its lifetime, sometimes (before
the Red Hat acquisition) many months. Why do you think Stream will
be that much worse?

Re timing etc.: I was really shocked last week, like everyone else.
But now I think everything basically makes sense, to me. When I heard
about providing only 1 year, I was shocked. 1 year? I was a sysadmin,
and I know that doing the upgrades, at the time, e.g. from RHL 7 to
RHEL 3, and later from RHEL 3 to 6 (yes, we did skip there 4-5), took
me and my team more than a year, closer to two years. But: If any of
the existing or emerging RHEL rebuilds is already good for you, or
will be good enough in a year, migration to it from CentOS should not
take a year. Not more than a few weeks, IMO (including testing,
automation, whatever, etc.). If it's not production, you are most
welcome to migrate to Stream, and depending on your needs, it might
make much _more_ sense to you, than to consider a RHEL rebuild.
And this also should not take that long. And if you want RHEL, and
want to use it for development, see the existing options, and if
you think they are not enough, or do not suit well your particular
situation, others on this thread also discussed this - contact
Red Hat. And if you are simply running production stuff, and your
business relies on RHEL, if you ask me - RHEL is simply what you
want. And if you think otherwise - just do your own calculations
and buy (or not) what you think is best for you. This indeed might
require a longer time, but would still make sense to do in a year.

Re the CentOS _brand_: People keep saying: OK, you realized that
taking over CentOS was a mistake. Fine. Just give it back to the
community. Now, let's have an exercise. Red Hat claims that CentOS
costs it money. A lot of money. And that if CentOS Linux would
continue as-is, even without Red Hat's financing, it would still
cost it quite a lot of money. See the linked article at start of
this thread, and other stuff here, for details. But it's not
important if you accept this fact as presented or not. Just accept
that this is Red Hat's stand - that CentOS Linux will continue
costing it money, as-is. Now, suppose that Red Hat, expecting the
huge noise/backlash such an announcement would cause, would simply
quitely try to find contributions somewhere and keep the project
as-is. Would this help covering these costs? Would a thread like
current exist? Would people seriously start thinking about what
it means to have something like CentOS Linux? How much it costs?
etc.? I do not think so. So Red Hat decided it's best for
everyone to simply publicly kill it. To make it extremely clear
that Red Hat is not involved anymore in any RHEL rebuilds. That
from Red Hat's POV, if you want CentOS Stream, as presented, you
are most welcome to try it, that if you want RHEL, you have
several options for getting it, but that if you want a RHEL
rebuild, then CentOS isn't one, and Red Hat is not going to be
involved in one. That if anyone wants to make one, fine - but
Red Hat is not part of that. Now, let's be honest: People on
this thread keep saying two opposite, contradicting things: One,
that CentOS, as a name, is worth almost nothing - say, not much
more than the cost of keeping the domain name centos.org, and
the other, that CentOS is a huge thing, and that killing it
as-is is a huge crime. Please be honest. If you think the former,
what's the problem? Just create a new cool name, and start
collecting people to join you. If you think the latter, please
realize that this costs money - a lot - and that Red Hat decided
that it does not want to spend this money anymore. I personally
think that's fair.

Last but not least: About the implied pressure of Red Hat on
the CentOS board, that led to current state. I wasn't part of
this, or related in any way, and basically all I know about it
is from this list. So what I write here is purely a guess. Isn't
it possible, that the board, even including non-Red-Hatters, was
simply convinced that Red Hat's POV is legitimate? That just
keeping CentOS as-is for 9 years would make such a damage to
Red Hat, that CentOS itself would simply not exist anymore? That
for the interests of CentOS, its users, it's simply unavoidable
to do something like what was eventually done? People might
disagree about the details (e.g. 1 year, or less, or more), but
does it sound so crazy that they simply agreed?

If you ask me, there simply is no more place for a project like
CentOS in this world. It does not make sense. I do not believe
any of the "community competitors" will survive. I think there
is enough space only for a single large-scale community distro,
and this is Debian. All the others will simply remain small, or
disappear, or become companies. And these companies will realize
the hard work of having a business. But that's just a guess.
Time will tell.

If you read so far, thanks for your time. Also, thanks again,
really, for your hard work along all these years. Do not think
that Red Hatters do not appreciate that.

Best regards,
-- 
Didi, who is hopefully done with thinking about "the situation"
and can go back fixing oVirt bugs on CentOS Stream...