On 12/8/20 9:47 AM, Patrick Riehecky wrote:

Hello,

Does https://centos.org/distro-faq/#q5-does-this-mean-that-centos-stream-is-the-r... address your concerns?

All that does is _confirm_ that stream will be RHEL beta All the bugs will get uncovered there _before_ packages move to RHEL.

On 12/8/20 4:47 PM, Patrick Riehecky wrote:

Hello,

Does https://centos.org/distro-faq/#q5-does-this-mean-that-centos-stream-is-the-r... address your concerns?

When I see "Security issues will be updated in CentOS Stream after they are solved in the current RHEL release." I can only reply your question with "No, it does not"

On 12/8/20 10:15 AM, Dan Seguin wrote:

Yeah, the "free" help just grinded to a halt.

Rich, you being the messenger, send it upwards to (IBM)?

Your missive is going to make a lot of people go elsewhere.

FWIW. IBM has not been part of this conversation. Red Hat is an independent entity. (Yes, I'm sure that resulted in eyerolls, but it also happens to be true.)

And, yes, not only is feedback here being relayed to management at Red Hat, most of them are also here reading it the same time that I am.

On 12/8/2020 10:20 AM, Rich Bowen wrote:

On 12/8/20 10:15 AM, Dan Seguin wrote:

...

Yeah, the "free" help just grinded to a halt.

Rich, you being the messenger, send it upwards to (IBM)?

Your missive is going to make a lot of people go elsewhere.

FWIW. IBM has not been part of this conversation. Red Hat is an independent entity. (Yes, I'm sure that resulted in eyerolls, but it also happens to be true.)

And, yes, not only is feedback here being relayed to management at Red Hat, most of them are also here reading it the same time that I am.

The "I'm not mad, I'm just disappointed" tagline might be best here. It's clear what the broader EL community had wanted for quite some time was a way to /actively/ contribute back into the development of Enterprise Linux. CentOS, SL, and others were downstream, and rightfully so for their design goals. CentOS Stream (this new incarnation, at least) seems to be intended as a way to allow direct feedback straight from the community without having to battle through the intermediary of the Fedora Project, which ... has its priorities elsewhere.

From that perspective, this announcement is a positive step. However coupling it with the early EOL of CentOS Linux 8 is extremely troubling. The build mechanisms are in place, the expectations were in place. IBM's purchase of RedHat was supposed to mean the ability to do things in a /more professional/ manner rather than a less professional one -- and that's exactly what this feels like: something unbecoming of what we'd expect from the leadership of either company.

Many folks out there delayed their transition away from EL6 due to the instability of some of the changes in EL7 (not mentioning any system*cough*d reasons) and were just now performing forklifts up to EL8 with that support expiring. We were just wrapping our heads around the operational headaches of Modularity, and weird new locations for Devel packages. And now we're forced onto this new treadmill that we didn't want and have to place a lot of effort into re-evaluating.

RedHat should know better than to pull the rug out from under others like this. This is not about saving a few dollars here and there, and it's not about resource allocation explicitly. If things needed to be re-justified, there's a larger discussion around consolidating Fedora, RedHat, and CentOS build processes together than would likely have been more fruitful. This is about removing a community expectation and forcing users away from stability, knowing that the primary blockers toward the community getting what it wants is the startup costs and trust factor around a new rebuild. IOW, there's no real reason for CentOS Linux not to continue to the end of RHEL8's support, with this new process being used as a fork for EL9 and beyond's development. THAT would be appreciated.

Instead, RedHat embraced CentOS under its umbrella, extended it beyond its original design goal of a binary-compatible rebuild, and has now extinguished the Linux rebuild starting next year.

Sounds like another company we all know and love.

-jc

On 12/8/20 8:58 AM, Manuel Wolfshant wrote:

On 12/8/20 4:47 PM, Patrick Riehecky wrote:

...

Hello,

Does https://centos.org/distro-faq/#q5-does-this-mean-that-centos-stream-is-the-r...

address your concerns?

When I see "Security issues will be updated in CentOS Stream after they are solved in the current RHEL release." I can only reply your question with "No, it does not"

That is NO different that now. We build CentOS updates after they are released in RHEL and then the source code is pushed to git.centos.org .. we always have.

This is no different. The security updates will be pushed to stream after they have been pushed to RHEL .. just like now.

On Tue, 2020-12-08 at 19:32 +0100, Leon Fauster via CentOS-devel wrote:

Am 08.12.20 um 18:00 schrieb Johnny Hughes:

...

On 12/8/20 8:58 AM, Manuel Wolfshant wrote:

...

On 12/8/20 4:47 PM, Patrick Riehecky wrote:

...

Hello,

Does https://urldefense.proofpoint.com/v2/url?u=https-3A__centos.org_distro-2Dfaq...

address your concerns?

When I see "Security issues will be updated in CentOS Stream after they are solved in the current RHEL release." I can only reply your question with "No, it does not"

That is NO different that now.  We build CentOS updates after they are released in RHEL and then the source code is pushed to git.centos.org .. we always have.

This is no different.  The security updates will be pushed to stream after they have been pushed to RHEL .. just like now.

If you compare it carefully you find rpms in CentOS Linux that are newer than in CentOS Stream - so security updates not landed in C8S.

The security updates are in Stream. They got into stream /before/ they landed in CentOS Linux 8.

On Tue, Dec 8, 2020 at 2:50 PM Phil Perry pperry@elrepo.org wrote:

On 08/12/2020 19:29, Patrick Riehecky wrote:

...

On Tue, 2020-12-08 at 19:32 +0100, Leon Fauster via CentOS-devel wrote:

...

Am 08.12.20 um 18:00 schrieb Johnny Hughes:

...

On 12/8/20 8:58 AM, Manuel Wolfshant wrote:

...

On 12/8/20 4:47 PM, Patrick Riehecky wrote:

...

Hello,

Does https://urldefense.proofpoint.com/v2/url?u=https-3A__centos.org_distro-2Dfaq...

address your concerns?

When I see "Security issues will be updated in CentOS Stream after they are solved in the current RHEL release." I can only reply your question with "No, it does not"

That is NO different that now. We build CentOS updates after they are released in RHEL and then the source code is pushed to git.centos.org .. we always have.

This is no different. The security updates will be pushed to stream after they have been pushed to RHEL .. just like now.

If you compare it carefully you find rpms in CentOS Linux that are newer than in CentOS Stream - so security updates not landed in C8S.

The security updates are in Stream. They got into stream /before/ they landed in CentOS Linux 8.

Sorry Pat, I'm not seeing that?

Taking just one example, CentOS8 has kernels 4.18.0-240.el8.x86_64.rpm and 4.18.0-240.1.1.el8_3.x86_64.rpm as seen here:

http://mirrors.coreix.net/centos/8/BaseOS/x86_64/os/Packages/

whereas CentOS Stream still only has kernel 4.18.0-240.el8.x86_64.rpm and not the security update.

http://mirrors.coreix.net/centos/8-stream/BaseOS/x86_64/os/Packages/

That's just the kernel. In fact I can't see _any_ security updates in stream. Am I missing something?

Dude, it is a stream! Security issues just get washed off and taken away by the undertow.

Phil _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

On Tue, Dec 8, 2020 at 11:50 AM Phil Perry pperry@elrepo.org wrote:

On 08/12/2020 19:29, Patrick Riehecky wrote:

...

On Tue, 2020-12-08 at 19:32 +0100, Leon Fauster via CentOS-devel wrote:

...

Am 08.12.20 um 18:00 schrieb Johnny Hughes:

...

On 12/8/20 8:58 AM, Manuel Wolfshant wrote:

...

On 12/8/20 4:47 PM, Patrick Riehecky wrote:

...

Hello,

Does https://urldefense.proofpoint.com/v2/url?u=https-3A__centos.org_distro-2Dfaq...

address your concerns?

When I see "Security issues will be updated in CentOS Stream after they are solved in the current RHEL release." I can only reply your question with "No, it does not"

That is NO different that now. We build CentOS updates after they are released in RHEL and then the source code is pushed to git.centos.org .. we always have.

This is no different. The security updates will be pushed to stream after they have been pushed to RHEL .. just like now.

If you compare it carefully you find rpms in CentOS Linux that are newer than in CentOS Stream - so security updates not landed in C8S.

The security updates are in Stream. They got into stream /before/ they landed in CentOS Linux 8.

Sorry Pat, I'm not seeing that?

Taking just one example, CentOS8 has kernels 4.18.0-240.el8.x86_64.rpm and 4.18.0-240.1.1.el8_3.x86_64.rpm as seen here:

http://mirrors.coreix.net/centos/8/BaseOS/x86_64/os/Packages/

whereas CentOS Stream still only has kernel 4.18.0-240.el8.x86_64.rpm and not the security update.

http://mirrors.coreix.net/centos/8-stream/BaseOS/x86_64/os/Packages/

That's just the kernel. In fact I can't see _any_ security updates in stream. Am I missing something?

Phil _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

An updated kernel is coming. But it is kernel-4.18.0-257.el8 according to:

ttps://git.centos.org/rpms/kernel/commits/c8s

Akemi

On 08/12/2020 20:04, Johnny Hughes wrote:

On 12/8/20 1:50 PM, Phil Perry wrote:

...

On 08/12/2020 19:29, Patrick Riehecky wrote:

...

On Tue, 2020-12-08 at 19:32 +0100, Leon Fauster via CentOS-devel wrote:

...

Am 08.12.20 um 18:00 schrieb Johnny Hughes:

...

On 12/8/20 8:58 AM, Manuel Wolfshant wrote:

...

On 12/8/20 4:47 PM, Patrick Riehecky wrote: > Hello, > > Does > https://urldefense.proofpoint.com/v2/url?u=https-3A__centos.org_distro-2Dfaq... > > > address your concerns?

When I see "Security issues will be updated in CentOS Stream after they are solved in the current RHEL release." I can only reply your question with "No, it does not"

That is NO different that now.  We build CentOS updates after they are released in RHEL and then the source code is pushed to git.centos.org .. we always have.

This is no different.  The security updates will be pushed to stream after they have been pushed to RHEL .. just like now.

If you compare it carefully you find rpms in CentOS Linux that are newer than in CentOS Stream - so security updates not landed in C8S.

The security updates are in Stream.  They got into stream /before/ they landed in CentOS Linux 8.

Sorry Pat, I'm not seeing that?

Taking just one example, CentOS8 has kernels  4.18.0-240.el8.x86_64.rpm and 4.18.0-240.1.1.el8_3.x86_64.rpm as seen here:

http://mirrors.coreix.net/centos/8/BaseOS/x86_64/os/Packages/

whereas CentOS Stream still only has kernel 4.18.0-240.el8.x86_64.rpm and not the security update.

http://mirrors.coreix.net/centos/8-stream/BaseOS/x86_64/os/Packages/

That's just the kernel. In fact I can't see _any_ security updates in stream. Am I missing something?

Yes, you are.

There will be a;;the RHEL engineers rolling all future changes into Stream for all RHEL versions.

Right now stream is 2 people rolling in changes just like CentOS 8 .. it takes time.

In fact, if you look, the 240 kernel was released in stream BEFORE it was released in CentOS Linux 8. And, we have built and will release this kernel very soon:

https://koji.mbox.centos.org/koji/buildinfo?buildID=14937

Thanks Johnny. So kernel-4.18.0-240.1.1.el8_3 will never appear in Stream, but a later kernel-4.18.0-257.el8 will, which may or may not be kABI compatible depending on the kernel symbols updated within the 4.18.0-257.el8 release.

My concern here is that elrepo are then no longer able to support CentOS. Elrepo can not develop against a constantly moving target whereby kernel symbols outside of the somewhat limited whitelist are constantly subject to change with each new kernel update.

Just wondering what the CentOS project is able to do to ensure ABI stability in the kernel? Are you able to make kernels separately available, or maybe continue releasing centosplus kernels to a separate repository/channel for those who require ABI stability? Do we need to think about starting a kernel SIG now to ensure this need is met in a year's time?

Phil

Am 08.12.20 um 20:29 schrieb Patrick Riehecky:

On Tue, 2020-12-08 at 19:32 +0100, Leon Fauster via CentOS-devel wrote:

...

Am 08.12.20 um 18:00 schrieb Johnny Hughes:

...

On 12/8/20 8:58 AM, Manuel Wolfshant wrote:

...

On 12/8/20 4:47 PM, Patrick Riehecky wrote:

...

Hello,

Does https://urldefense.proofpoint.com/v2/url?u=https-3A__centos.org_distro-2Dfaq...

address your concerns?

When I see "Security issues will be updated in CentOS Stream after they are solved in the current RHEL release." I can only reply your question with "No, it does not"

That is NO different that now.  We build CentOS updates after they are released in RHEL and then the source code is pushed to git.centos.org .. we always have.

This is no different.  The security updates will be pushed to stream after they have been pushed to RHEL .. just like now.

If you compare it carefully you find rpms in CentOS Linux that are newer than in CentOS Stream - so security updates not landed in C8S.

The security updates are in Stream. They got into stream /before/ they landed in CentOS Linux 8.

firefox-78.4.0 never got into c8s but its in c8.2.2004 firefox-78.5.0 is in the pipe of c8.3.2011 now ... c8s still on 78.3.

So, its not clear what type of degration c8s is.

-- Leon

And what about new stuff that will be experimented on in Stream? So far RHEL team did inhouse testing and when they were ready (stable kernel, tested packages ) they were published and ONLY THEN they were published in CentOS. And now, packages will be changed several months before they are published in RHEL, so what will that men for software that supports specific minor releases of RHEL? WHat to do when you run Stream (beta for example 8.6) and software only supports RHEL 8.5? What then?

when I run "dnf update" on CentOS Linux 8, I KNOW I will get package that Red Hat team tested. Package in Stream will be some new untested version with who know what problems...

On 12/8/20 6:00 PM, Johnny Hughes wrote:

On 12/8/20 8:58 AM, Manuel Wolfshant wrote:

...

On 12/8/20 4:47 PM, Patrick Riehecky wrote:

...

Hello,

Does https://centos.org/distro-faq/#q5-does-this-mean-that-centos-stream-is-the-r...

address your concerns?

When I see "Security issues will be updated in CentOS Stream after they are solved in the current RHEL release." I can only reply your question with "No, it does not"

That is NO different that now. We build CentOS updates after they are released in RHEL and then the source code is pushed to git.centos.org .. we always have.

This is no different. The security updates will be pushed to stream after they have been pushed to RHEL .. just like now. _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

On Tue, Dec 8, 2020 at 12:00 PM Johnny Hughes johnny@centos.org wrote:

On 12/8/20 8:58 AM, Manuel Wolfshant wrote:

...

On 12/8/20 4:47 PM, Patrick Riehecky wrote:

...

Hello,

Does https://centos.org/distro-faq/#q5-does-this-mean-that-centos-stream-is-the-r...

address your concerns?

When I see "Security issues will be updated in CentOS Stream after they are solved in the current RHEL release." I can only reply your question with "No, it does not"

That is NO different that now. We build CentOS updates after they are released in RHEL and then the source code is pushed to git.centos.org .. we always have.

This is no different. The security updates will be pushed to stream after they have been pushed to RHEL .. just like now.

So it will get package upgrades first because it is supposed to be RHEL +0.1 but will get security patches later since in that aspect it is RHEL -0.1.

On 12/8/20 1:44 PM, Leon Fauster via CentOS-devel wrote:

Am 08.12.20 um 20:39 schrieb Mauricio Tavares:

...

On Tue, Dec 8, 2020 at 12:00 PM Johnny Hughes johnny@centos.org wrote:

...

On 12/8/20 8:58 AM, Manuel Wolfshant wrote:

...

On 12/8/20 4:47 PM, Patrick Riehecky wrote:

...

Hello,

Does https://centos.org/distro-faq/#q5-does-this-mean-that-centos-stream-is-the-r...

address your concerns?

When I see "Security issues will be updated in CentOS Stream after they are solved in the current RHEL release." I can only reply your question with "No, it does not"

That is NO different that now.  We build CentOS updates after they are released in RHEL and then the source code is pushed to git.centos.org .. we always have.

This is no different.  The security updates will be pushed to stream after they have been pushed to RHEL .. just like now.

So it will get package upgrades first because it is supposed to be RHEL +0.1 but will get security patches later since in that aspect it is RHEL -0.1.

Good catch. In the old days of CentOS the "security" gap was "only" while building a point release. Now, the gap expands gradually over the life of a point release. Important fact ...

It isn't , it is faster. Or at least the same. You are not waiting until the next point release is done .. you are waiting until teh RHSA is complete and then when the source code is put into git.centos.org for c8 branch ... the same patch or a differnt patch *if the rpm in stream is newer) is then rolled into stream

On 12/8/20 9:58 AM, Manuel Wolfshant wrote:

On 12/8/20 4:47 PM, Patrick Riehecky wrote:

...

Hello,

Does https://centos.org/distro-faq/#q5-does-this-mean-that-centos-stream-is-the-r...

address your concerns?

When I see "Security issues will be updated in CentOS Stream after they are solved in the current RHEL release." I can only reply your question with "No, it does not"

Note that this has always been the way it is in CentOS Linux, also. Security issues have *always* landed in CentOS Linux after they landed in RHEL. So this is not a change.

Hello,

Does https://centos.org/distro-faq/#q3-will-the-source-code-for-red-hat-enterpris... address your concerns?

Pat

On Tue, 2020-12-08 at 17:40 +0300, plageat@tut.by wrote:

Well, what will happen to the sources of RHEL8 regular updates? As far as we know, c8stream sources are not always fully-matched of those which used in building RH8 advisory errata updates. We had some cases, when NVR of the same  matching source in c8 and c8s git branches are different by some ways (different patches, different %changelog etc.)

08.12.2020, 17:07, "Rich Bowen" rbowen@redhat.com:

...

The future of the CentOS Project is CentOS Stream, and over the next year we’ll be shifting focus from CentOS Linux, the rebuild of Red Hat Enterprise Linux (RHEL), to CentOS Stream, which tracks just ahead of a current RHEL release. CentOS Linux 8, as a rebuild of RHEL 8, will end at the end of 2021. CentOS Stream continues after that date, serving as the upstream (development) branch of Red Hat Enterprise Linux.

Meanwhile, we understand many of you are deeply invested in CentOS Linux 7, and we’ll continue to produce that version through the remainder of the RHEL 7 life cycle. https://urldefense.proofpoint.com/v2/url?u=https-3A__access.redhat.com_suppo...

CentOS Stream will also be the centerpiece of a major shift in collaboration among the CentOS Special Interest Groups (SIGs). This ensures SIGs are developing and testing against what becomes the next version of RHEL. This also provides SIGs a clear single goal, rather than having to build and test for two releases. It gives the CentOS contributor community a great deal of influence in the future of RHEL. And it removes confusion around what “CentOS” means in the Linux distribution ecosystem.

When CentOS Linux 8 (the rebuild of RHEL8) ends, your best option will be to migrate to CentOS Stream 8, which is a small delta from CentOS Linux 8, and has regular updates like traditional CentOS Linux releases. If you are using CentOS Linux 8 in a production environment, and are concerned that CentOS Stream will not meet your needs, we encourage you to contact Red Hat about options.

We have an FAQ - https://urldefense.proofpoint.com/v2/url?u=https-3A__centos.org_distro-2Dfaq...   - to help with your information and planning needs, as you figure out how this shift of project focus might affect you.

[See also: Red Hat's perspective on this. https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com_en_blog_...  ]

---

CentOS-devel mailing list CentOS-devel@centos.org https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.centos.org_mailma...

---

CentOS-devel mailing list CentOS-devel@centos.org https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.centos.org_mailma...

On Tue, Dec 8, 2020 at 10:28 AM Phelps, Matthew mphelps@cfa.harvard.edu wrote:

On Tue, Dec 8, 2020 at 9:06 AM Rich Bowen rbowen@redhat.com wrote:

...

The future of the CentOS Project is CentOS Stream, and over the next year we’ll be shifting focus from CentOS Linux, the rebuild of Red Hat Enterprise Linux (RHEL), to CentOS Stream, which tracks just ahead of a current RHEL release. CentOS Linux 8, as a rebuild of RHEL 8, will end at the end of 2021. CentOS Stream continues after that date, serving as the upstream (development) branch of Red Hat Enterprise Linux.

Meanwhile, we understand many of you are deeply invested in CentOS Linux 7, and we’ll continue to produce that version through the remainder of the RHEL 7 life cycle. https://access.redhat.com/support/policy/updates/errata/#Life_Cycle_Dates

CentOS Stream will also be the centerpiece of a major shift in collaboration among the CentOS Special Interest Groups (SIGs). This ensures SIGs are developing and testing against what becomes the next version of RHEL. This also provides SIGs a clear single goal, rather than having to build and test for two releases. It gives the CentOS contributor community a great deal of influence in the future of RHEL. And it removes confusion around what “CentOS” means in the Linux distribution ecosystem.

When CentOS Linux 8 (the rebuild of RHEL8) ends, your best option will be to migrate to CentOS Stream 8, which is a small delta from CentOS Linux 8, and has regular updates like traditional CentOS Linux releases. If you are using CentOS Linux 8 in a production environment, and are concerned that CentOS Stream will not meet your needs, we encourage you to contact Red Hat about options.

We have an FAQ - https://centos.org/distro-faq/ - to help with your information and planning needs, as you figure out how this shift of project focus might affect you.

[See also: Red Hat's perspective on this.

https://www.redhat.com/en/blog/centos-stream-building-innovative-future-ente... ]

---

CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

You have published a CentOS Lifecycle that states the EOL for CentOS 8 is May 2029. (c.f. https://endoflife.software/operating-systems/linux/centos). CentOS Stream *is not* CentOS 8.

This announcement is a breach of that trust with your community, and could be construed as a breach of contract with your users.

Save this change for CentOS 9.

This statement from that page is also now a lie:

The CentOS Linux distribution is a stable, predictable, manageable and reproducible platform derived from the sources of Red Hat Enterprise Linux (RHEL).

This statement is the reason we *all* chose CentOS. You will be betraying us in the worst possible way if this change is allowed to happen.

This isn't just a "Knee jerk" reaction. CentOS is abandoning everything it stood for by even creating CentOS Stream. We *dont want it* !!

Getting rid of a strict re-compile of RHEL X.X is a complete reversal of the principles of CentOS and the community it serves. Again, WE DON'T WANT THIS!

--

*Matt Phelps*

*Information Technology Specialist, Systems Administrator*

(Computation Facility, Smithsonian Astrophysical Observatory)

Center for Astrophysics | Harvard & Smithsonian

60 Garden Street | MS 39 | Cambridge, MA 02138 email: mphelps@cfa.harvard.edu

cfa.harvard.edu | Facebook http://cfa.harvard.edu/facebook | Twitter http://cfa.harvard.edu/twitter | YouTube http://cfa.harvard.edu/youtube | Newsletter http://cfa.harvard.edu/newsletter

On Tue, Dec 8, 2020 at 1:07 PM Rich Bowen rbowen@redhat.com wrote:

On 12/8/20 12:07 PM, Phelps, Matthew wrote:

...

I still haven't seen an answer to the question, "Who made this decision?" and, "How can we lobby to get it changed?"+

The Board of Directors of the CentOS project voted on this decision.

You are *currently* lobbying to get it changed.

Thank you.

I understand it's not a democracy, but soliciting input from CentOS users before making such a huge change would have been nice, and highly enlightening I expect.

If I missed such a request for input, I apologize, but I don't recall seeing one on this list, or the main centos mailing list.

On Tue, Dec 8, 2020 at 10:07 AM Rich Bowen rbowen@redhat.com wrote:

On 12/8/20 12:07 PM, Phelps, Matthew wrote:

...

I still haven't seen an answer to the question, "Who made this decision?" and, "How can we lobby to get it changed?"+

The Board of Directors of the CentOS project voted on this decision.

You are *currently* lobbying to get it changed.

As the OSUOSL is a big user of CentOS, I'm concerned this change will result in loss of stability since Stream is effectively like the development version of CentOS. I think this is great for testing new features and also getting feedback for upstream, it's troublesome for folks also wanting a stable release.

To me, I view what CentOS has done up until this point is similar to the Ubuntu LTS releases where you have a release you know will be the same for an extended period of time. This change seems as though the "LTS" model of CentOS is going away and following a model more closer to Fedora and Ubuntu non-LTS releases. As a sysadmin that concerns me quite a bit since I have a moving target which may have unintended consequences. I realize the comparison to Ubuntu's release model is not 1:1 to RHEL/CentOS, so please forgive me on that.

Ideally, as a user I'd like to have the *option* of using CentOS which matches RHEL releases OR CentOS Stream. I understand that you likely cannot do both which is why you voted the way you did based on people resources and other factors. But as you've seen on this thread, you're alienating a lot of loyal users by making this change on a short notice (given the nature of this distribution).

As mentioned on the RH blog:

There are different kinds of CentOS users, and we are working with the

CentOS Project Governing Board to tailor programs that meet the needs of these different user groups. In the first half of 2021, we plan to introduce low- or no-cost programs for a variety of use cases, including options for open source projects and communities and expansion of the Red Hat Enterprise Linux Developer subscription use cases to better serve the needs of systems administrators. We’ll share more details as these initiatives coalesce.

While this option may work for users such as me, I doubt it will work for the vast majority of users.

Matt described this a little better via his comment on LWN [1]:

There was an internal presentation featuring a "layer cake" model which I

really liked, which went something like this: Blue: Community space | Fedora Linux | community engineering decisions | community support Purple: Shared space | CentOS Stream | transparent Red Hat engineering decisions with community input | community support Red: Product | Red Hat Enterprise Linux | Red Hat engineering decisions with customer input | product support Now, "Red Hat engineering decisions" might seem a bit scary, but consider that in CentOS Linux, all of the distro-contents engineering decisions were also made by Red Hat, but inside without any transparency. And if you're unsure about community engineering decisions and Fedora's independence … buy me a beverage sometime and ask me about Btrfs!

As a current user of CentOS, I'd like it to also fall in the "Red" column.

Also, please forgive me if I have misunderstood or got some of this wrong but this is my perspective based on what I've read thus far.

Thanks-

[1] https://lwn.net/Articles/839341/

On 12/9/20 12:15 AM, Matthew Miller wrote:

On Tue, Dec 08, 2020 at 01:54:30PM -0800, Lance Albertson wrote:

...

Ubuntu LTS releases where you have a release you know will be the same for an extended period of time. This change seems as though the "LTS" model of CentOS is going away and following a model more closer to Fedora and Ubuntu non-LTS releases. As a sysadmin that concerns me quite a bit since I have a moving target which may have unintended consequences. I realize the comparison to Ubuntu's release model is not 1:1 to RHEL/CentOS, so please forgive me on that.

I would say that it's quite a long way away, actually. RHEL's model isn't changing, so you can expect the amount of change in minor releases to stay the same. And CentOS Stream won't have anything that's not planned to go into one of those releases, so the overall change should be the same.

Majority of users do NOT WANT Stream, do NOT CARE about Stream, they care about RHEL CLONE, 99% binary compatibility.

CentOS does not have proprietary and non-free packages including drivers and multimedia codecs, you have to add 3rd party support which are not as reliable as full distro, kernel is not vanila and RHEL one has a lot of things disabled, many developers, especially of apps for Desktop use do not want to even support rpm packages...

So when you remove binary compatibility, why would anyone bother with CentOS/RHEL unless they want a job in a company that pays for RHEL support?

Stream (we should start calling it RHEL Stream) will never be binary compatible clone of RHEL, so it looses it's appeal, so we have to look elswhere. I never considered Fedora more then a plaything, and I neve even though about even installing Stream for testing, I just do not need it.

If, instead, you want more change, come join us over in Fedora Server. :)

...

Matt described this a little better via his comment on LWN [1]:

[...]

...

...

also made by Red Hat, but inside without any transparency. And if you're unsure about community engineering decisions and Fedora's independence … buy me a beverage sometime and ask me about Btrfs!

That offer of buying me beverages is valid as soon as the pandemic is over, by the way. :)

...

[1] https://lwn.net/Articles/839341/

On 09/12/2020 15:08, Brendan Conoboy wrote:

On Wed, Dec 9, 2020 at 6:59 AM Ljubomir Ljubojevic <centos@plnet.rs mailto:centos@plnet.rs> wrote:

So when you remove binary compatibility, why would anyone bother with
CentOS/RHEL unless they want a job in a company that pays for RHEL
support?

What sort of binaries are you concerned about being incompatible?

Any kernel device drivers, for a start. Kind of critical if your SAS/RAID device wont boot, or your network device doesn't come up, or your GUI doesn't start because your display drivers aren't compatible anymore. Just minor things like that maybe?

On 09/12/2020 15:25, Brendan Conoboy wrote:

On Wed, Dec 9, 2020 at 7:16 AM Phil Perry <pperry@elrepo.org mailto:pperry@elrepo.org> wrote:

On 09/12/2020 15:08, Brendan Conoboy wrote:
 > On Wed, Dec 9, 2020 at 6:59 AM Ljubomir Ljubojevic
<centos@plnet.rs <mailto:centos@plnet.rs>
 > <mailto:centos@plnet.rs <mailto:centos@plnet.rs>>> wrote:
 >
 >     So when you remove binary compatibility, why would anyone
bother with
 >     CentOS/RHEL unless they want a job in a company that pays for
RHEL
 >     support?
 >
 > What sort of binaries are you concerned about being incompatible?

Any kernel device drivers, for a start. Kind of critical if your
SAS/RAID device wont boot, or your network device doesn't come up, or
your GUI doesn't start because your display drivers aren't compatible
anymore. Just minor things like that maybe?

OK, so out-of-tree drivers.  If those keep on working does that make CentOS Stream viable for your use?

I don't use CentOS Stream, I use RHEL. I use RHEL to develop software for RHEL and compatible OS clones, including CentOS. If Stream retains binary compatibility, and specifically kernel ABI compatibility, then the users of the software packages we develop can continue to use them. If not, they can't. Simple as that. So please don't push rolling kernel updates to Stream that break the kernel ABI.

On Wed, Dec 9, 2020 at 6:33 PM David Hrbáč david-lists@hrbac.cz wrote:

I don't use CentOS Stream, I use RHEL. I use RHEL to develop software

...

for RHEL and compatible OS clones, including CentOS. If Stream retains binary compatibility, and specifically kernel ABI compatibility, then the users of the software packages we develop can continue to use them. If not, they can't. Simple as that. So please don't push rolling kernel updates to Stream that break the kernel ABI.

Indeed. If any such broken change (eg: that breaks kernel ABI) is pushed to Stream, that is treated as a serious problem by the RHEL engineering teams. We have the necessary process in place to QE test changes before they arrive in CentOS Stream.

I understand this fact alone is not a panacea for all the problems people are highlighting. But it does seem to cover your use case. From a regression, stability, ABI, and kernel ABI perspective, it is the goal and focus of many of us in RHEL Engineering for CentOS Stream to be stable.

Cheers,

Stef

...

Rolling kernel updates are going to kill all the traditional HPC clusters. Almost 25% of the TOP 500 HPC clusters run CentOS. See https://www.top500.org/statistics/list/ DH _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

I understand this fact alone is not a panacea for all the problems people are highlighting. But it does seem to cover your use case. From a regression, stability, ABI, and kernel ABI perspective, it is the goal and focus of many of us in RHEL Engineering for CentOS Stream to be stable.

Cheers,

Stef

Well I can see it now with 7.x tree, almost with every kernel update we have to sort out Lustre driver issues and introduce a patch. I expect the updates becoming progressively worse with the rolling kernel updates. DH

On 09/12/2020 17:40, Stef Walter wrote:

On Wed, Dec 9, 2020 at 6:33 PM David Hrbáč <david-lists@hrbac.cz mailto:david-lists@hrbac.cz> wrote:

    I don't use CentOS Stream, I use RHEL. I use RHEL to develop
    software
    for RHEL and compatible OS clones, including CentOS. If Stream
    retains
    binary compatibility, and specifically kernel ABI compatibility,
    then
    the users of the software packages we develop can continue to
    use them.
    If not, they can't. Simple as that. So please don't push rolling
    kernel
    updates to Stream that break the kernel ABI.

Indeed. If any such broken change (eg: that breaks kernel ABI) is pushed to Stream, that is treated as a serious problem by the RHEL engineering teams. We have the necessary process in place to QE test changes before they arrive in CentOS Stream.

I understand this fact alone is not a panacea for all the problems people are highlighting. But it does seem to cover your use case. From a regression, stability, ABI, and kernel ABI perspective, it is the goal and focus of many of us in RHEL Engineering for CentOS Stream to be stable.

Cheers,

Stef

Hi Stef,

Thank you for your response. You do realise I'm not just talking about whitelisted kernel symbols, but the whole kernel ABI?

Whilst the RHEL kernel ABI whitelist is great in principle, in practice I am yet to find a kernel driver that uses only symbols on the whitelist. As I said previously, every single driver I maintain broke between RHEL8.2 and RHEL8.3 due to changes in the kernel ABI.

On 09/12/2020 19:01, Brendan Conoboy wrote:

On Wed, Dec 9, 2020 at 10:04 AM Phil Perry <pperry@elrepo.org mailto:pperry@elrepo.org> wrote:

On 09/12/2020 17:40, Stef Walter wrote:
 > On Wed, Dec 9, 2020 at 6:33 PM David Hrbáč <david-lists@hrbac.cz
<mailto:david-lists@hrbac.cz>
 > <mailto:david-lists@hrbac.cz <mailto:david-lists@hrbac.cz>>> wrote:
 >
 >         I don't use CentOS Stream, I use RHEL. I use RHEL to develop
 >         software
 >         for RHEL and compatible OS clones, including CentOS. If
Stream
 >         retains
 >         binary compatibility, and specifically kernel ABI
compatibility,
 >         then
 >         the users of the software packages we develop can continue to
 >         use them.
 >         If not, they can't. Simple as that. So please don't push
rolling
 >         kernel
 >         updates to Stream that break the kernel ABI.
 >
 >
 > Indeed. If any such broken change (eg: that breaks kernel ABI) is
pushed
 > to Stream, that is treated as a serious problem by the RHEL
engineering
 > teams. We have the necessary process in place to QE test changes
before
 > they arrive in CentOS Stream.
 >
 > I understand this fact alone is not a panacea for all the problems
 > people are highlighting. But it does seem to cover your use case.
From a
 > regression, stability, ABI, and kernel ABI perspective, it is the
goal
 > and focus of many of us in RHEL Engineering for CentOS Stream to
be stable.

Hi Stef,

Thank you for your response. You do realise I'm not just talking about
whitelisted kernel symbols, but the whole kernel ABI?

Whilst the RHEL kernel ABI whitelist is great in principle, in practice
I am yet to find a kernel driver that uses only symbols on the
whitelist. As I said previously, every single driver I maintain broke
between RHEL8.2 and RHEL8.3 due to changes in the kernel ABI.

Yes, the symbol list is helpful, but it's rarely 100%, so we're going to have to iteratively do better.

Just to be clear, I don't think this is your job or David's job to solve, just knowing that if we collectively solve it, it makes CentOS Stream a more viable option for you and others was the insight I was hoping for.  There are probably many of these, but this is the one I'm hearing repeatedly in this thread.  In the months ahead there are going to be numerous, uh, "opportunities", to solve these sorts of things together :-) If the kind of thing you can bring is what does and doesn't work for you and why, I'll take it.

Great. Once kernel-4.18.0-257.el8.x86_64 makes it into Stream, I'll be happy to test and feed back. Any ETA on that? I see it was built a week ago on koji

On 10/12/2020 12:10, Josh Boyer wrote:

On Wed, Dec 9, 2020 at 3:49 PM Phil Perry pperry@elrepo.org wrote:

...

On 09/12/2020 19:01, Brendan Conoboy wrote:

...

On Wed, Dec 9, 2020 at 10:04 AM Phil Perry <pperry@elrepo.org mailto:pperry@elrepo.org> wrote:

 On 09/12/2020 17:40, Stef Walter wrote:
  > On Wed, Dec 9, 2020 at 6:33 PM David Hrbáč <david-lists@hrbac.cz
 <mailto:david-lists@hrbac.cz>
  > <mailto:david-lists@hrbac.cz <mailto:david-lists@hrbac.cz>>> wrote:
  >
  >         I don't use CentOS Stream, I use RHEL. I use RHEL to develop
  >         software
  >         for RHEL and compatible OS clones, including CentOS. If
 Stream
  >         retains
  >         binary compatibility, and specifically kernel ABI
 compatibility,
  >         then
  >         the users of the software packages we develop can continue to
  >         use them.
  >         If not, they can't. Simple as that. So please don't push
 rolling
  >         kernel
  >         updates to Stream that break the kernel ABI.
  >
  >
  > Indeed. If any such broken change (eg: that breaks kernel ABI) is
 pushed
  > to Stream, that is treated as a serious problem by the RHEL
 engineering
  > teams. We have the necessary process in place to QE test changes
 before
  > they arrive in CentOS Stream.
  >
  > I understand this fact alone is not a panacea for all the problems
  > people are highlighting. But it does seem to cover your use case.
 From a
  > regression, stability, ABI, and kernel ABI perspective, it is the
 goal
  > and focus of many of us in RHEL Engineering for CentOS Stream to
 be stable.

 Hi Stef,

 Thank you for your response. You do realise I'm not just talking about
 whitelisted kernel symbols, but the whole kernel ABI?

 Whilst the RHEL kernel ABI whitelist is great in principle, in practice
 I am yet to find a kernel driver that uses only symbols on the
 whitelist. As I said previously, every single driver I maintain broke
 between RHEL8.2 and RHEL8.3 due to changes in the kernel ABI.

Just to touch on this a bit, the reason the RHEL kABI whitelist exists is because the upstream Linux kernel explicitly does NOT have a kernel ABI. Given that you maintain out of tree drivers, I'm sure you know this but it might not be evident to anyone that doesn't. So the RHEL kABI whitelist is there to express what RHEL is committing to keep stable for a kernel ABI for 10 years while dealing with an upstream that doesn't care at all.

Hi Josh,

Yes, I get that.

The issue here is not so much that the kABI, outside of the whitelist, changes, it's that it's not the same between the kernels in RHEL and the kernels in STREAM. The issue is the two products are out of sync and are not the same.

Red Hat are trying to sell Stream to the community on the basis it will be an almost like for like drop in replacement for CentOS 8 users once CentOS 8 no longer exists, and at the kernel level that is simply not true.

People talk about "3rd party out of tree" drivers like it is a small thing. It is not. It affects a large number of your users. The elrepo project has over 150,000 unique IPs per week hitting our repositories, and somewhere around 1-2 million users per year. We would estimate the vast majority of those are CentOS users. If you fail to keep the kernel ABI in sync between RHEL and Stream, you break Stream for all those users.

...

...

Yes, the symbol list is helpful, but it's rarely 100%, so we're going to have to iteratively do better.

Don't get me wrong, I love the fact RHEL has a stable ABI and have the kABI whitelist, but to be brutally frank it is totally irrelevant when considering 3rd party out of tree drivers. In my 10 plus years experience developing, backporting and packaging such drivers for the RHEL kernel, I can categorically state I have not seen a single driver that *only* uses symbols that are on the whitelist. Further, I would challenge anyone to write even the most simplistic "hello world" kernel driver that uses only whitelisted symbols, let alone a driver that actually interacts with hardware and does something useful.

On 10/12/2020 13:31, Neal Gompa wrote:

On Thu, Dec 10, 2020 at 8:25 AM Phil Perry pperry@elrepo.org wrote:

...

On 10/12/2020 12:10, Josh Boyer wrote:

...

On Wed, Dec 9, 2020 at 3:49 PM Phil Perry pperry@elrepo.org wrote:

...

On 09/12/2020 19:01, Brendan Conoboy wrote:

...

On Wed, Dec 9, 2020 at 10:04 AM Phil Perry <pperry@elrepo.org mailto:pperry@elrepo.org> wrote:

  On 09/12/2020 17:40, Stef Walter wrote:
   > On Wed, Dec 9, 2020 at 6:33 PM David Hrbáč <david-lists@hrbac.cz
  <mailto:david-lists@hrbac.cz>
   > <mailto:david-lists@hrbac.cz <mailto:david-lists@hrbac.cz>>> wrote:
   >
   >         I don't use CentOS Stream, I use RHEL. I use RHEL to develop
   >         software
   >         for RHEL and compatible OS clones, including CentOS. If
  Stream
   >         retains
   >         binary compatibility, and specifically kernel ABI
  compatibility,
   >         then
   >         the users of the software packages we develop can continue to
   >         use them.
   >         If not, they can't. Simple as that. So please don't push
  rolling
   >         kernel
   >         updates to Stream that break the kernel ABI.
   >
   >
   > Indeed. If any such broken change (eg: that breaks kernel ABI) is
  pushed
   > to Stream, that is treated as a serious problem by the RHEL
  engineering
   > teams. We have the necessary process in place to QE test changes
  before
   > they arrive in CentOS Stream.
   >
   > I understand this fact alone is not a panacea for all the problems
   > people are highlighting. But it does seem to cover your use case.
  From a
   > regression, stability, ABI, and kernel ABI perspective, it is the
  goal
   > and focus of many of us in RHEL Engineering for CentOS Stream to
  be stable.

  Hi Stef,

  Thank you for your response. You do realise I'm not just talking about
  whitelisted kernel symbols, but the whole kernel ABI?

  Whilst the RHEL kernel ABI whitelist is great in principle, in practice
  I am yet to find a kernel driver that uses only symbols on the
  whitelist. As I said previously, every single driver I maintain broke
  between RHEL8.2 and RHEL8.3 due to changes in the kernel ABI.

Just to touch on this a bit, the reason the RHEL kABI whitelist exists is because the upstream Linux kernel explicitly does NOT have a kernel ABI. Given that you maintain out of tree drivers, I'm sure you know this but it might not be evident to anyone that doesn't. So the RHEL kABI whitelist is there to express what RHEL is committing to keep stable for a kernel ABI for 10 years while dealing with an upstream that doesn't care at all.

Hi Josh,

Yes, I get that.

The issue here is not so much that the kABI, outside of the whitelist, changes, it's that it's not the same between the kernels in RHEL and the kernels in STREAM. The issue is the two products are out of sync and are not the same.

Red Hat are trying to sell Stream to the community on the basis it will be an almost like for like drop in replacement for CentOS 8 users once CentOS 8 no longer exists, and at the kernel level that is simply not true.

People talk about "3rd party out of tree" drivers like it is a small thing. It is not. It affects a large number of your users. The elrepo project has over 150,000 unique IPs per week hitting our repositories, and somewhere around 1-2 million users per year. We would estimate the vast majority of those are CentOS users. If you fail to keep the kernel ABI in sync between RHEL and Stream, you break Stream for all those users.

...

...

...

Yes, the symbol list is helpful, but it's rarely 100%, so we're going to have to iteratively do better.

Don't get me wrong, I love the fact RHEL has a stable ABI and have the kABI whitelist, but to be brutally frank it is totally irrelevant when considering 3rd party out of tree drivers. In my 10 plus years experience developing, backporting and packaging such drivers for the RHEL kernel, I can categorically state I have not seen a single driver that *only* uses symbols that are on the whitelist. Further, I would challenge anyone to write even the most simplistic "hello world" kernel driver that uses only whitelisted symbols, let alone a driver that actually interacts with hardware and does something useful.

I'd like to turn this around and ask: could we leverage that expertise and active work as a SIG in CentOS to improve the quality of the RHEL kernel ABI whitelist? If I remember rightly, you do work in the ELRepo project. Perhaps it would make sense to start pulling ELRepo into CentOS as a SIG and continuously integrate it with CentOS Stream?

By doing so, we could introduce gating mechanisms to prevent kernels that break driver packages from being released in the first place. We could also get mechanisms in place to do proper Secure Boot signing for these driver packages, signed with a key automatically trusted by the CentOS kernel and trivially usable by RHEL folks, too.

I too maintain an out of tree driver for work, and this is something that would be valuable to *me* to make my life easier. If you do as many as I think you do, I would imagine that this would be even *more* valuable to you.

This is one of the reasons I started getting involved in CentOS Stream in the first place, and I think this would be a good thing for you too.

Hi Neal,

I'm probably not the right person to ask, as I probably have a very narrow view, highly influenced by the work I'm directly involved with, as to the role of RHEL's kABI whitelist.

Generally, I will say, that up until now the process has worked really well. Red Hat introduced the Driver Update Programme back in the early days of RHEL5, and the elrepo project has simply leveraged that framework through RHEL5 -> RHEL8. Hopefully that can continue. I really see little need to dream up complicated fixes for something which wasn't broken.

If we are left with a scenario where it's broken, then I think there are far simpler alternatives than trying to manage the ABI of a constantly moving target. If RH won't make the RHEL kernels available to Stream users, it would be trivial for anyone to rebuild them as part of a SIG, centosplus, or as part of an independent 3rd party repo for users that need them. After all, there is a lot of expertise in the CentOS community at rebuilding RHEL source code.

On Wed, Dec 9, 2020 at 12:33 PM David Hrbáč david-lists@hrbac.cz wrote:

...

I don't use CentOS Stream, I use RHEL. I use RHEL to develop software for RHEL and compatible OS clones, including CentOS. If Stream retains binary compatibility, and specifically kernel ABI compatibility, then the users of the software packages we develop can continue to use them. If not, they can't. Simple as that. So please don't push rolling kernel updates to Stream that break the kernel ABI.

Rolling kernel updates are going to kill all the traditional HPC clusters. Almost 25% of the TOP 500 HPC clusters run CentOS. See https://www.top500.org/statistics/list/

Can you elaborate on what you mean by rolling kernel updates? If you mean wholesale kernel rebases (e.g. kernel-5.8 -> kernel-5.9), that's not what Stream will be doing.

Stream will include more frequent updates of the *RHEL* kernel, where we take the RHEL kABI whitelist very seriously. It is literally the RHEL kernel. In the context of HPC deployments, they could use Stream and choose when to update and reboot those machines based on the kernel updates that come out, just as the would with CentOS Linux.

josh

On 10/12/2020 16.50, Rich Bowen wrote:

On 12/9/20 12:32 PM, David Hrbáč wrote:

...

I don't use CentOS Stream, I use RHEL. I use RHEL to develop software     for RHEL and compatible OS clones, including CentOS. If Stream retains     binary compatibility, and specifically kernel ABI compatibility, then     the users of the software packages we develop can continue to use them.     If not, they can't. Simple as that. So please don't push rolling kernel     updates to Stream that break the kernel ABI.

Rolling kernel updates are going to kill all the traditional HPC clusters. Almost 25% of the TOP 500 HPC clusters run CentOS. See https://www.top500.org/statistics/list/ https://www.top500.org/statistics/list/

I was under the impression that practically all of those run custom kernels anyway, right?

Obviously in no way generally valid: In the last ~5 years I have been involved in / responsible for the administration of two systems that have been listed in the TOP500 (both lower half of TOP500, but top field of Green 500 at their respective time). Both systems are running CentOS 7 with vanilla kernel. We only add external 3rd party kernel modules as required.

There have been plans/conversations/ideas about updating the newer system to CentOS Linux 8 soon. However, with the changed perspective of CentOS, this plan is now dead as well. Depending on how CentOS Stream works out (i.e., kernel ABI compatibility to current RHEL minor release), switching to CentOS Stream might be an option.

On Wed, Dec 9, 2020 at 11:40 AM Ljubomir Ljubojevic centos@plnet.rs wrote:

On 12/9/20 4:25 PM, Brendan Conoboy wrote:

...

On Wed, Dec 9, 2020 at 7:16 AM Phil Perry <pperry@elrepo.org mailto:pperry@elrepo.org> wrote:

On 09/12/2020 15:08, Brendan Conoboy wrote:
> On Wed, Dec 9, 2020 at 6:59 AM Ljubomir Ljubojevic
<centos@plnet.rs <mailto:centos@plnet.rs>
> <mailto:centos@plnet.rs <mailto:centos@plnet.rs>>> wrote:
>
>     So when you remove binary compatibility, why would anyone
bother with
>     CentOS/RHEL unless they want a job in a company that pays for RHEL
>     support?
>
> What sort of binaries are you concerned about being incompatible?

Any kernel device drivers, for a start. Kind of critical if your
SAS/RAID device wont boot, or your network device doesn't come up, or
your GUI doesn't start because your display drivers aren't compatible
anymore. Just minor things like that maybe?

OK, so out-of-tree drivers. If those keep on working does that make CentOS Stream viable for your use?

The problem we CentOS users have with "RHEL Stream" is "IF" it works. I do not want to think about "IF". RHEL Stream is meant to constantly, daily get packages and patches that are not yet tested, to BE tested on

This is actually not the case. Today, our RHEL development model requires package updates to undergo testing before they are included in the distribution, even internally. CentOS Stream will continue this trend as we upstream many of our RHEL CI workflows to Stream itself.

users. From RHEL Stream FAQ: "CentOS Stream is a distribution that community members can use to take advantage of a stable ABI/API for development and testing," DEVELOPMENT AND TESTING.

It would be disingenuous to say Stream won't have bugs. All software development has bugs. However, the "development and testing" part actually extends to the development and testing of the software *you* are developing. Stream exists as a base for you to do your own development and testing.

I do not have time to mess with "might work, might not" testing distro, otherwise I would play on the sand with Fedora. RHEL clone is best available option and in return for me using code prepared for RHEL I was telling anyone who would listen how CentOS is great, for me better then Debian/Ubuntu or other distro's. CentOS Facebook group has more members then Fedora FB group, partially due to me posting ton of resource links and how-to's and helping anyone who asked for help even if I had to look for answer my self. I do NOT have FAITH that Stream will be production ready, and I do not have free time to experiment or check if it will work or not when I run update, and that is only thing that matters. No amount of "do not worry" will change my mind. As soon as it is crystal clear (I still hope) that "CentOS Linux" is dead, I will stop advocating for CentOS, actually I will start advocating AGAINST CentOS and look for alternatives, either other clones or Debian.

I will not respond to any more e-mails or tweets about this topic, there is nothing more to be said except R.I.P CentOS :-(

I understand if you don't want to continue the conversation. I do want to say that with CentOS Stream, we're committed to developing a stable and world class project that literally produces RHEL. I hope in the future if you decide to revisit your OS choice that you take a moment and see what we're doing here. I think you might find that we're working very hard to avoid the things you're concerned about, because we *need* to avoid those same things in RHEL anyway.

josh

Le jeudi 10 décembre 2020 à 12:39 +0000, Dan Seguin a écrit :

Who exactly do you expect will contribute to RHEL from the CENTOS community?

Why would *anybody* spend their own time, even their IT reputations rolling any of this mess?

I would, even if I wasn't paid by RH.

My job involve being a sysadmin, and so I run Centos servers in production, along some Fedora, some Debian and others, for myself and for work.

Each time I want to get a small new feature or fixes, not being able to easily contribute to RHEL (and so Centos) despites working at RH is a problem.

I have a few examples: a few years ago, I bought a Yubikey for myself. I want to use it on my RHEL 7 (my laptop) as a smartcard, but it was not supported.

I searched, found the fix is 2 lines in the usb-ids database. I opened a bug in may 2016, I provided the patch, went to complain on internal IRC to the right people, who escalated that in meeting, and it still took ~6 months: https://bugzilla.redhat.com/show_bug.cgi?id=1157226

I knew what to do, because I had the same problem in 2014: https://bugzilla.redhat.com/show_bug.cgi?id=1157226 where it took 1 year to get the fix in RHEL 7 (and so Centos 7).

On Fedora (or others faster distros) where there is a documented way to contribute, I would have waited a few weeks at best, maybe more in worst case (depending how persistant I am into getting a fix). With streams, I hope that at least, I wouldn't need to wait on "next minor release", given the delay it had.

And keep in mind that while I did that using my profesionnal relationship, I did it for myself, and others Centos users did benefit from it.

Another example, SELinux. When I see a unconfined service (last one, knot, for my personal DNS server, but also synapse or gitea, personal matrix and personal forge), I try to write a policy, and while on it, get it usable for others. Usually, I have not much trouble to get things in the Fedora package.

I have no hope to get it backported to RHEL (and so Centos), but at least, with Stream, I would have a path to try to get it done.

Again, I would write the policy for me anyway, so why shouldn't I try to share it with others ?

And I do not think I am the only one with a ethos of sharing bugfixes with the rest of the community. The whole free software movement is built on that.

On Thu, Dec 10, 2020 at 7:20 AM Josh Boyer jwboyer@redhat.com wrote:

On Wed, Dec 9, 2020 at 11:40 AM Ljubomir Ljubojevic centos@plnet.rs wrote:

...

On 12/9/20 4:25 PM, Brendan Conoboy wrote:

...

On Wed, Dec 9, 2020 at 7:16 AM Phil Perry <pperry@elrepo.org mailto:pperry@elrepo.org> wrote:

On 09/12/2020 15:08, Brendan Conoboy wrote:
> On Wed, Dec 9, 2020 at 6:59 AM Ljubomir Ljubojevic
<centos@plnet.rs <mailto:centos@plnet.rs>
> <mailto:centos@plnet.rs <mailto:centos@plnet.rs>>> wrote:
>
>     So when you remove binary compatibility, why would anyone
bother with
>     CentOS/RHEL unless they want a job in a company that pays

for RHEL

...

...

>     support?
>
> What sort of binaries are you concerned about being incompatible?

Any kernel device drivers, for a start. Kind of critical if your
SAS/RAID device wont boot, or your network device doesn't come up,

or

...

...

your GUI doesn't start because your display drivers aren't

compatible

...

...

anymore. Just minor things like that maybe?

OK, so out-of-tree drivers. If those keep on working does that make CentOS Stream viable for your use?

The problem we CentOS users have with "RHEL Stream" is "IF" it works. I do not want to think about "IF". RHEL Stream is meant to constantly, daily get packages and patches that are not yet tested, to BE tested on

This is actually not the case. Today, our RHEL development model requires package updates to undergo testing before they are included in the distribution, even internally. CentOS Stream will continue this trend as we upstream many of our RHEL CI workflows to Stream itself.

...

users. From RHEL Stream FAQ: "CentOS Stream is a distribution that community members can use to take advantage of a stable ABI/API for development and testing," DEVELOPMENT AND TESTING.

It would be disingenuous to say Stream won't have bugs. All software development has bugs. However, the "development and testing" part actually extends to the development and testing of the software *you* are developing. Stream exists as a base for you to do your own development and testing.

*WE* aren't doing development and testing! We're running systems that are trying to get work done. The whole POINT of CentOS was a recompile of the Open Source parts of RHEL. A stable *Enterprise* level operating system.

Do you people not understand that??

...

I do not have time to mess with "might work, might not" testing distro, otherwise I would play on the sand with Fedora. RHEL clone is best available option and in return for me using code prepared for RHEL I was telling anyone who would listen how CentOS is great, for me better then Debian/Ubuntu or other distro's. CentOS Facebook group has more members then Fedora FB group, partially due to me posting ton of resource links and how-to's and helping anyone who asked for help even if I had to look for answer my self. I do NOT have FAITH that Stream will be production ready, and I do not have free time to experiment or check if it will work or not when I run update, and that is only thing that matters. No amount of "do not worry" will change my mind. As soon as it is crystal clear (I still hope) that "CentOS Linux" is dead, I will stop advocating for CentOS, actually I will start advocating AGAINST CentOS and look for alternatives, either other clones or Debian.

I will not respond to any more e-mails or tweets about this topic, there is nothing more to be said except R.I.P CentOS :-(

I understand if you don't want to continue the conversation. I do want to say that with CentOS Stream, we're committed to developing a stable and world class project that literally produces RHEL. I hope in the future if you decide to revisit your OS choice that you take a moment and see what we're doing here. I think you might find that we're working very hard to avoid the things you're concerned about, because we *need* to avoid those same things in RHEL anyway.

josh _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

Am 10.12.20 um 13:48 schrieb Dan Seguin:

Yes, yes they do. This is what this bizarre squeeze is all about.

Not sure. My impression is that; they expect that the big players, ISV, etc that contribute to CStream will make RHEL better and stable.

Making such calculation without integrating the impact of all that smaller contributions that is done by the CentOS community will result in a wrong outcome. This math is really not complicated ?

---

*From:* CentOS-devel centos-devel-bounces@centos.org on behalf of Phelps, Matthew mphelps@cfa.harvard.edu *Sent:* December 10, 2020 7:39 AM *To:* The CentOS developers mailing list. *Subject:* Re: [CentOS-devel] https://blog.centos.org/2020/12/future-is-centos-stream/

*WE* aren't doing development and testing! We're running systems that are trying to get work done. The whole POINT of CentOS was a recompile of the Open Source parts of RHEL. A stable *Enterprise* level operating system.

Do you people not understand that??

On Thu, Dec 10, 2020 at 8:02 AM Phil Perry pperry@elrepo.org wrote:

On 10/12/2020 12:39, Phelps, Matthew wrote:

...

On Thu, Dec 10, 2020 at 7:20 AM Josh Boyer <jwboyer@redhat.com mailto:jwboyer@redhat.com> wrote:

On Wed, Dec 9, 2020 at 11:40 AM Ljubomir Ljubojevic <centos@plnet.rs
<mailto:centos@plnet.rs>> wrote:
 >
 > On 12/9/20 4:25 PM, Brendan Conoboy wrote:
 > > On Wed, Dec 9, 2020 at 7:16 AM Phil Perry <pperry@elrepo.org
<mailto:pperry@elrepo.org>
 > > <mailto:pperry@elrepo.org <mailto:pperry@elrepo.org>>> wrote:
 > >
 > >     On 09/12/2020 15:08, Brendan Conoboy wrote:
 > >     > On Wed, Dec 9, 2020 at 6:59 AM Ljubomir Ljubojevic
 > >     <centos@plnet.rs <mailto:centos@plnet.rs>
<mailto:centos@plnet.rs <mailto:centos@plnet.rs>>
 > >     > <mailto:centos@plnet.rs <mailto:centos@plnet.rs>
<mailto:centos@plnet.rs <mailto:centos@plnet.rs>>>> wrote:
 > >     >
 > >     >     So when you remove binary compatibility, why would

anyone

...

 > >     bother with
 > >     >     CentOS/RHEL unless they want a job in a company that
pays for RHEL
 > >     >     support?
 > >     >
 > >     > What sort of binaries are you concerned about being
incompatible?
 > >
 > >     Any kernel device drivers, for a start. Kind of critical if
your
 > >     SAS/RAID device wont boot, or your network device doesn't
come up, or
 > >     your GUI doesn't start because your display drivers aren't
compatible
 > >     anymore. Just minor things like that maybe?
 > >
 > >
 > > OK, so out-of-tree drivers.  If those keep on working does that
make
 > > CentOS Stream viable for your use?
 >
 > The problem we CentOS users have with "RHEL Stream" is "IF" it
works. I
 > do not want to think about "IF". RHEL Stream is meant to

constantly,

...

 > daily get packages and patches that are not yet tested, to BE
tested on

This is actually not the case.  Today, our RHEL development model
requires package updates to undergo testing before they are included
in the distribution, even internally.  CentOS Stream will continue
this trend as we upstream many of our RHEL CI workflows to Stream
itself.

 > users.  From RHEL Stream FAQ: "CentOS Stream is a distribution

that

...

 > community members can use to take advantage of a stable ABI/API

for

...

 > development and testing," DEVELOPMENT AND TESTING.

It would be disingenuous to say Stream won't have bugs.  All software
development has bugs.  However, the "development and testing" part
actually extends to the development and testing of the software *you*
are developing.  Stream exists as a base for you to do your own
development and testing.

*WE* aren't doing development and testing! We're running systems that are trying to get work done. The whole POINT of CentOS was a recompile of the Open Source parts of RHEL. A stable *Enterprise* level operating system.

Do you people not understand that??

Matthew,

To be fair, this is the CentOS Devel list. Perhaps your comments would be better placed or more suited to the main CentOS mailing list, but this list exists specifically to discuss the development of CentOS. If you are not active in that, or do not want to participate, that's fine but stick to the users list which would be the correct list for you.

Thanks

Fair point, although there has been a lot of cross-posting with centos@centos.org on this thread.

...

 > I do not have time to mess with "might work, might not" testing
distro,
 > otherwise I would play on the sand with Fedora. RHEL clone is best
 > available option and in return for me using code prepared for
RHEL I was
 > telling anyone who would listen how CentOS is great, for me
better then
 > Debian/Ubuntu or other distro's. CentOS Facebook group has more
members
 > then Fedora FB group, partially due to me posting ton of resource
links
 > and how-to's and helping anyone who asked for help even if I had
to look
 > for answer my self.
 > I do NOT have FAITH that Stream will be production ready, and I
do not
 > have free time to experiment or check if it will work or not when
I run
 > update, and that is only thing that matters. No amount of "do not
worry"
 > will change my mind. As soon as it is crystal clear (I still
hope) that
 > "CentOS Linux" is dead, I will stop advocating for CentOS,

actually I

...

 > will start advocating AGAINST CentOS and look for alternatives,
either
 > other clones or Debian.
 >
 > I will not respond to any more e-mails or tweets about this
topic, there
 > is nothing more to be said except R.I.P CentOS :-(

I understand if you don't want to continue the conversation.  I do
want to say that with CentOS Stream, we're committed to developing a
stable and world class project that literally produces RHEL.  I hope
in the future if you decide to revisit your OS choice that you take a
moment and see what we're doing here.  I think you might find that
we're working very hard to avoid the things you're concerned about,
because we *need* to avoid those same things in RHEL anyway.

josh
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org <mailto:CentOS-devel@centos.org>
https://lists.centos.org/mailman/listinfo/centos-devel
<https://lists.centos.org/mailman/listinfo/centos-devel>

--

*Matt Phelps*

*Information Technology Specialist, Systems Administrator*

(Computation Facility, Smithsonian Astrophysical Observatory)

Center for Astrophysics | Harvard & Smithsonian

60 Garden Street | MS 39 | Cambridge, MA 02138

email: mphelps@cfa.harvard.edu mailto:mphelps@cfa.harvard.edu

cfa.harvard.edu http://cfa.harvard.edu/| Facebook http://cfa.harvard.edu/facebook| Twitter http://cfa.harvard.edu/twitter| YouTube http://cfa.harvard.edu/youtube| Newsletter http://cfa.harvard.edu/newsletter

---

CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

---

CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

Greetings.

----- Original Message -----

All my ranting I am to commit will be explained at the end of the rant.

That was a complete waste of time. They KNOW what people use CentOS for... and all of the caveats with using it (especially on desktop systems) that you mentioned. The devs on this developer mailing list talk about... development stuff.

And BTW, they aren't responsible for making the decision and I assume don't have any input into it... so ranting at them isn't going to help at all. There is an email address that was given that you can send such things to if desired: centos-questions@redhat.com

Have a plesent tomorrow.

TYL,

On 11.12.2020 03:53, Josh Boyer wrote:

On Thu, Dec 10, 2020 at 1:55 PM Scott Dowdle dowdle@montanalinux.org

wrote:

...

Greetings.

----- Original Message -----

...

All my ranting I am to commit will be explained at the end of the rant.

That was a complete waste of time. They KNOW what people use CentOS

for... and all of the caveats with using it (especially on desktop systems) that you mentioned. The devs on this developer mailing list talk about... development stuff.

Actually, I thought Ljubomir's email was well crafted. I don't take offense to any of it and agree with some of the points.

Ljubomir, development and developers defined as "people that write code all day" are definitely part of the story but as you point out they aren't the only part. Really, Stream is looking for contributors. Contributions come in many forms. Docs, bug reporting, patches, RFEs, etc. If you look at all the things you describe doing, those are all contributions and all good ones at that.

Even on the sysadmin side, people deploy servers for a reason. Those reasons need testing and, yes, "development" to make sure their purpose is being fulfilled. You're right though, once it is set up it is more traditional sysadmin. I think Stream can be used in those cases too. Thanks for raising this use case so clearly.

After sieving through multitude of comments and angry remarks, I noticed there are two primary reasons for those:

- RH violate its own promise (yes, Chris Wright, technically you are a liar now) - since this moment, I personally have little trust in whatever RH promises or "guarantees" in open source area - CentOS Stream is *not* 1:1 bug-level copy of RHEL, thus one can't rely on its stability any more (the subject of drivers, ABIs etc has been mentioned many times already) - if I am in challenge mood, I'd rather install Fedora

That simple. The positive output of CentOS burial is two more clones springing to life. So, in foreseeable future, we will have three clones to choose from (Springdale, Rocky and the one made by Cloudlinux). Nice.

What's not nice is that CentOS community will inevitably be disrupted and dispersed.

Unless the RH announcement was a shortsighted and dumb sacrifice to Money, I assume they just brush away what they consider "unnecessary expenses". In any case, I continue to evangelize open source, Linux included, it's just CentOS and RH that won't be promoted (by me) any more.

On Fri, Dec 11, 2020 at 6:43 AM Louis Lagendijk louis@fazant.net wrote:

On Thu, 2020-12-10 at 20:02 -0500, Konstantin Boyandin via CentOS-devel wrote:

...

What's not nice is that CentOS community will inevitably be disrupted and dispersed.

Unless the RH announcement was a shortsighted and dumb sacrifice to Money, I assume they just brush away what they consider "unnecessary expenses". In any case, I continue to evangelize open source, Linux included, it's just CentOS and RH that won't be promoted (by me) any more.

I interpret the Centos change of direction as indicative of the change in direction for RedHat: RHEL has become much less important as IBM focuses on web applications, cloud tooling etc. That is where the money is and that is what they bought RedHat for. So RHEL has to optimize OS costs as is it no longer key. So it is only logical to optimize the development process and outsource some of the testing to the public in Centos Stream, and not waste resources on a true free competitor for RHEL.

Now For Stream I am mainly worried about 2 things: 1 - the kernel and its impact on drivers from Elrepo and other external repos (OpenZFS!). Not important for RHEL, but important for people that use old hardware that requires drivers not in RHEL.

Others have expressed this concern, and it's a good one. It would be interesting to see if someone could create a CentOS Stream SIG that did automated rebuilds of those drivers with every Stream kernel update.

2 - the frequency of updates from Centos stream, which will require much more frequent updates. As some updates will be security fixes skipping frequent updates is not an option....

There are probably ways that people can still selectively update based on what changed, but it is true that Stream will have a more frequent cadence. However, it's worth pointing out that the update frequency isn't the same across the entire package set. We have packages in RHEL that rarely update, and if they do they are for bugfixes. Only a portion undergoes a lot of activity every minor release.

josh

On Fri, Dec 11, 2020 at 8:41 AM Manuel Wolfshant wolfy@nobugconsulting.ro wrote:

On 12/11/20 2:02 PM, Josh Boyer wrote:

...

On Fri, Dec 11, 2020 at 6:43 AM Louis Lagendijk louis@fazant.net wrote:

...

On Thu, 2020-12-10 at 20:02 -0500, Konstantin Boyandin via CentOS-devel wrote:

...

What's not nice is that CentOS community will inevitably be disrupted and dispersed.

Unless the RH announcement was a shortsighted and dumb sacrifice to Money, I assume they just brush away what they consider "unnecessary expenses". In any case, I continue to evangelize open source, Linux included, it's just CentOS and RH that won't be promoted (by me) any more.

I interpret the Centos change of direction as indicative of the change in direction for RedHat: RHEL has become much less important as IBM focuses on web applications, cloud tooling etc. That is where the money is and that is what they bought RedHat for. So RHEL has to optimize OS costs as is it no longer key. So it is only logical to optimize the development process and outsource some of the testing to the public in Centos Stream, and not waste resources on a true free competitor for RHEL.

Now For Stream I am mainly worried about 2 things: 1 - the kernel and its impact on drivers from Elrepo and other external repos (OpenZFS!). Not important for RHEL, but important for people that use old hardware that requires drivers not in RHEL.

Others have expressed this concern, and it's a good one. It would be interesting to see if someone could create a CentOS Stream SIG that did automated rebuilds of those drivers with every Stream kernel update.

The whole idea ( ok, maybe not the whole but certainly one of the most important ideas ) behind ELRepo packages is to take advantage ( as much as possible ) of the weak-modules mechanism and _not_ rebuild the drivers at each kernel update :)

My proposal upthread was to move ELRepo stuff into the CentOS Project as a SIG, and have the CKI SIG that will form to manage the RHEL kernel gate on those kernel modules. The idea would be that breakages would prevent releasing new kernel builds without some kind of acceptance/documentation that there was no way to avoid it, and then those modules would get rebuilt accordingly. This would help make the ABI validation much more comprehensive and help to eliminate the rebuild burden for other kmod developers.

That brings transparency to the process and provides a way for Red Hat and the community to improve the quality of the RHEL kernel's guarantee of a stable kernel ABI.

On 11/12/2020 12:02, Josh Boyer wrote:

...

Now For Stream I am mainly worried about 2 things: 1 - the kernel and its impact on drivers from Elrepo and other external repos (OpenZFS!). Not important for RHEL, but important for people that use old hardware that requires drivers not in RHEL.

Others have expressed this concern, and it's a good one. It would be interesting to see if someone could create a CentOS Stream SIG that did automated rebuilds of those drivers with every Stream kernel update.

You know, it would be much much better if CentOS Stream included a new "kernel-classic" repo containing the latest RHEL point release kernel. Some mechanism would need to be invented to easily switch to/from it and make it the default but that would address, I suspect, about 95% of ELRepo problems. It's not exactly a difficult thing for the CentOS team to do since they already have all the infrastructure, the knowledge, the procedures and even the ability to make it secure bootable.

Since probably 80% of kernel releases are RHSA updates, that means they are for security anyway so having those built and put into a "kernel-classic" repo would solve a lot of the objections to this abandonment of CentOS users. If Red Hat wanted to exclude the RHBA/RHEA kernels from that rebuild process then I'm not sure that many people would miss those.

It's probably not ever going to fix Red Hat's reputation after this debacle but it might go a bit of a way to addressing _some_ of the more serious objections to it.

Trevor Hemsley Soon to be Ex-CentOS QA, Ex-CentOS Forum Moderator, Ex-CentOS IRC op

On Fri, Dec 11, 2020 at 04:50:06PM -0800, Akemi Yagi wrote:

If you quit those roles, that would be a HUGE loss for the CentOS Project.

Seconded. I mentioned this to Trevor privately earlier but I will say it here. I would absolutely _hate_ to see Trevor walk away from the project. That being said, I would respect his decision to do so; people need to do what is right for them. This week has caused many to reevaluate their roles and how they spend their time; Trevor is not alone here.

John

On Sat, 12 Dec 2020 at 02:15, Trevor Hemsley via CentOS-devel < centos-devel@centos.org> wrote:

Now, you've not only let down a lot of people by withdrawing support for CentOS, which would have been bad but understandable, you've also allowed EOL expectations to remain and you released a product in 2019 that everyone assumed was set to remain until 2029. You didn't disabuse anyone of the expectation that it would remain for 10 years. You *deliberately* kept quiet about it. That was stupid. Possibly even borderline dishonest. Certainly disingenuous. So now you have a very large number of systems out there which a lot people have put a lot of effort into installing with CentOS Linux 8 that either need to be reinstalled with something else or people need to get used to them being broken on a regular basis. I am very sure that that will not have made you popular.

Thank you for articulating much clearer the point I (@digdilem) rather clumsily tried to put across in #centos-devel yesterday. I absolutely refuse to accept that nobody at Redhat noticed the EOL had been changed for C8 in all the many months it was there.

I consider that ignoring this is bad faith acting of the highest order. Every day it was left was another day people like myself happily migrated Centos 6 servers to Centos 8 expecting a nice long support time before we'd need to do that again. (And it's far from trivial for us) To see senior Redhat staff still blaming the community for changing that, and STILL expressing surprise that "We never guessed someone would edit the wiki!" is outrageous.

Redhat has gone from one of the good guys into the Linux scene to one of the worst overnight. That's tragic. And, as we're the people who influence business decisions, bad business - short, medium and long term. I've done Redhat certified training and my company put money into their pocket for it, mostly because my work focused on Centos. To have heard Centos talked about like it was nothing but a leech on Redhat's profits is disheartening.

Sorry to the people like yourself who put far more into Centos than I ever did. I hope that, when you're ready, you find another project worthy of your time and skill.

Simon

El jue, 10 dic 2020 a las 15:25, Ljubomir Ljubojevic (centos@plnet.rs) escribió:

On 12/10/20 1:19 PM, Josh Boyer wrote:

...

It would be disingenuous to say Stream won't have bugs. All software development has bugs. However, the "development and testing" part actually extends to the development and testing of the software *you* are developing. Stream exists as a base for you to do your own development and testing.

What is with you Red Hat people and "developers"? Do you really live in a fairy-tale world where every Linux user is a "developer"?

All my ranting I am to commit will be explained at the end of the rant.

I started as simple Windows admin and Delphi programmer that in 2004 became small WISP and needed Firewall/router/Web&Mail server. So I installed ClarkConnect 4 and had to learn iptables to overcome CC's web config. In 2007 I started experimenting with standalone Linux system and in 2008 installed C5 web/mail/NFS/Samba server and Desktop systems and was forced to learn how to use them and configure them.

Then I learned how to harden Postfix, how to install and maintain Joomla and webshop, then I started learning programming in BASH with ncurses and multidimensional arrays to create TUI script to manage my Wireless routers (ping, trace, backup, login,...).

Then I needed another server but because I did not have enough money I ended up creating KVM host/guest, but because my only server was 32-bit and RHEL dropped support for 32-bit KVM hosts, I studied and learned from internet how to build KVM kmod module for 32-bit CPU/kernel (some Hungarian? guy in CentOS community explained it in detail). Everything I learned was when I NEEDED it, and I did it all by my self.

After building KVM module and learning how to compile/package rpm's, I decided to rebuild some packages from Fedora to have more tools for my Desktop CentOS 5. I ended up producing around 70 packages including packaging Skype with static libraries into rpm. I was only person in the world to do it, and since almost no one used CentOS 5 as Desktop/laptop, only few people used it. I ended up creating "DentOS" repository with those Desktop packages which mostly only I installed to friends and clients. And my clients were not rich (I am from Serbia and legal software in companies is even today only 30-40%) so even getting them to but a PC for RAID file server needed persuading. I was one of rare ones to create RAID out of partitions and not disks. Why? Because I needed

Then I needed to learn some PHP and Python, so I did. I was one of very rare ISP's in my country to provide sending mail via port 587 (with SSL) because largest ISP's allowed sending e-mail for accounts on their mail servers only from inside their own networks and only on port 25. I was shocked to learn of that, but I was pushing the envelope just like I was only one in Serbian Linux community to use CentOS for Desktop/laptop.

So even though I was actively preaching about bigger CentOS use to everyone who wanted to listen, and I created largest resources and howto's collection (I became admin of the official CentOS Facebook group in 2011 and ever since published there resources and articles) and I daily solved problems for newbies, beside using it my self as desktop, I rarely built Linux systems and only involvement was "yum update" and some "spring cleaning" on few servers I managed.

So in last 10 years I installed only some 4-5 CentOS servers (Samba + KVM host with legal Windows guest), if I do not count several CentOS systems for my own use, mostly Laptops for me. And THAT IS IT, no grand scheme, no dozens or hundreds of servers, no big development projects, no University diploma, just a guy fixing Windows PC's in small IT shop for small number of clients and installing WISP clients. At the moment I am System Admin in a 60-employee company with 50 Windows PC's and 3 FreeNAS/TrueNAS file servers, and the only Linux is my laptop running CentOS 8 with MATE.

Now the punchline: MOST CentOS users are like that. Not "developers" but small server owners or hosting renters.

I am admin for CentOS Facebook group for 9 years. I grew it from 300 members to 26.800 members. BUT what you do not realize is that vast majority of those Facebook users when they joined our group or "Linux" FB group, where I was one of the admins in ~2010-2015 and advocated CentOS like a biggest zealot, only heard that Linux exist and wanted to learn something. They rented some hosting or Linux VM and needed to learn basics. Only maybe 20-30 members (out of 26.800 + all those that over 9 years left the group or closed the FB account) were established Linux users.

When we explain them that there are no audio/video codecs due to legal restrictions, that kernel is backported and adding some module is not possible if they compile vanila kernel because it brakes the kABI, that they need to install centosplus kernel because driver they needed RHEL does not want to support, that CentOS tools for development are too old because of version freeze, that they need to install several 3rd party repositories to make their server work, they then asks us "but why would we use CentOS instead of Debian, Ubuntu, SUSE...?" And every single time the response was "Because Red Hat is great company that created great and very stable product and CentOS is almost total clone of RHEL, and when they learn how to manage CentOS they can go for RH Linux certification. And that is it, for them CentOS does not have any other competitive edge over other Linuxes beside "99% clone of RHEL". SELinux was more of the repelling point, vast majority would just turn it off when they read first Howto on the internet, and great efforts went to lpleading with them to try to learn SELinux.

So the fact that there are 1.000 guys that do developing on CentOS does not mean that majority of CentOS users are "developers", on the contrary, vast majority of servers running CentOS, especially rented VM's are run/controlled by guys like me whose main job is not to manged Linux servers, that is only a side job and many only barely understand what they are doing. They installed the system, configured it via some tutorial, and left it running.

"First step in solving a problem is realizing there IS a problem.". Your first problem is that you do not understand VAST majority of your users just want Linux server to run their small business they do not have to pay for. And there is no free RHEL license for us with no money and need for Linux server. Your second problem is that main "selling" point for them to use CentOS was "free clone", and only reason they were "sold" that mantra was because us zealots worked very hard to convince them to chose CentOS over other distro's. And we are NOT going to stake OUR credibility to support your illusions of grandeur that CentOS Stream with packages 3-6 months ahead of current RHEL will be stable enough to not cause THEM any problems (including paid-for software like CPanel, Virtualmin, and many others people run on servers).

Third problem you have is that while hosting vendors WILL stop offering "CentOS Linux 8" to renters, just like you wanted, they will NOT stake THEIR reputation and pay THEIR support staff for EVERY incident/crash YOUR Stream will cause to THEIR customers. They will continue to offer C7 until 2024 and their EL8 offering will be just switched to ANOTHER clone of RHEL. Since Red Hat is OBLIGATED to publish source code, clones like Springdale and new ones WILL CONTINUE to be built and offered.

But now, due to greed of your employers, all of us from EL (Enterprise Linux) community who were loyal to "RHEL clone" known as "CentOS Linux" are going to LEAVE. As soon as I can I will run a Springdale Linux VM to check things out, and as soon as I am satisfied with it I will then change every single CentOS server I have to Springdale.

NEVER again will I install a system with CentOS brand, and since no one I know has the money to buy the RHEL license, I will never be in position to offer it to someone, luckily. And I doubt those in the position to recommend some Linux system with subscription are going to recommend RHEL to anyone EVER again after your (Red Hat's) stupid stunt.

So feel free to live in your little "developer" fantasy before it comes crashing down when mass exodus occurs, and I will focus on some damage control and try to redirect those leaving CentOS for other distro's to Springdale, so that EL community is damaged as little as possible.

-- Ljubomir Ljubojevic (Love is in the Air) PL Computers Serbia, Europe

StarOS, Mikrotik and CentOS/RHEL/Linux consultant

It's the DEVops fairytale. With **DEV** in uppercase and bold and ops with lowercase, despite that as 'we' sysadmins have to solve the disasters made by brilliant developers. DEVops is the flogisto of the new age ( https://en.wikipedia.org/wiki/Phlogiston_theory). And everyone that challenges must be sent to the bonfire... Some people think that Facebook reality it's the entire IT world reality.

On Fri, Dec 11, 2020 at 2:10 PM Gordon Messmer gordon.messmer@gmail.com wrote:

On 12/10/20 4:19 AM, Josh Boyer wrote:

...

I do want to say that with CentOS Stream, we're committed to developing a stable and world class project that literally produces RHEL. I hope in the future if you decide to revisit your OS choice that you take a moment and see what we're doing here. I think you might find that we're working very hard to avoid the things you're concerned about, because we*need* to avoid those same things in RHEL anyway.

I think it might be fair to say that CentOS Stream can't accomplish Red Hat's goals if it has no users, and it won't have any users if it isn't reliable. The people who are alarmed that CentOS Stream will be suddenly unreliable aren't thinking clearly about Red Hat's intent for that distribution.

It's obvious Red Hat wasn't thinking clearly. I'm not concerned about the reliability of CentOS stream since I won't be using it. Ever.

It may be awesome, but it is not what CenOS was, a recompile of RHEL X.X. And it's not ever going to be what we were promised we would have, over and over and over, a supported CentOS 8 through May 2029.

We've been screwed. Plain and simple.

--

*Matt Phelps*

*Information Technology Specialist, Systems Administrator*

(Computation Facility, Smithsonian Astrophysical Observatory)

Center for Astrophysics | Harvard & Smithsonian

60 Garden Street | MS 39 | Cambridge, MA 02138 email: mphelps@cfa.harvard.edu

cfa.harvard.edu | Facebook http://cfa.harvard.edu/facebook | Twitter http://cfa.harvard.edu/twitter | YouTube http://cfa.harvard.edu/youtube | Newsletter http://cfa.harvard.edu/newsletter

On Fri, Dec 11, 2020 at 3:53 PM Simon Matter simon.matter@invoca.ch wrote:

...

On 12/10/20 4:19 AM, Josh Boyer wrote:

...

I do want to say that with CentOS Stream, we're committed to developing a stable and world class project that literally produces RHEL. I hope in the future if you decide to revisit your OS choice that you take a moment and see what we're doing here. I think you might find that we're working very hard to avoid the things you're concerned about, because we*need* to avoid those same things in RHEL anyway.

I think it might be fair to say that CentOS Stream can't accomplish Red Hat's goals if it has no users, and it won't have any users if it isn't reliable. The people who are alarmed that CentOS Stream will be suddenly unreliable aren't thinking clearly about Red Hat's intent for that distribution.

I understand why RedHat came up with the idea of CentOS Stream and it makes sense for them. BUT, something is missing for many of us and it will probably greatly reduce the number of users of CentOS Stream and as such, reduce the positive effects CS could have. The problem is that the stable part of CentOS Linux is missing in future and may prevent many of us from using CentOS Stream as well.

What I could probably live with is:

• Create point releases of CentOS Stream at the same time like the RHEL

releases.

• Feed point releases with updates like RHEL updates.
• Clearly tag CentOS Stream different than RHEL, clX?
• CS can contain changes which are not made in RHEL
• CS kernel disables less modules to enhance support for older hardware

not supported in RHEL anymore.

• CS packages may enable features not enabled and therefore supported7 in

RHEL

• CS may not be absolutely identical to RHEL but as stable as RHEL in

general.

What you describe is... wait for it... CentOS 8!!

Why can't we just keep having that, like we were promised?

That means, folks who run commercial software need to run it on RHEL to have support and be absolutely sure things work. If they pay for commercial software they should also pay RedHat!

Sorry, no. Budgets are tight/non-existant. And it's not just software that's supported by CentOS X.X. Ever call Dell for hardware support, for a failed hard drive? "What operating system are you running? What version?"

Those using only free software can use CentOS point releases where they need most stability and run CentOS Stream where they can do it and by doing so help with development of upstream. Most IT people, developers and admins will most likely be willing to run CentOS Stream and be part of a fruitful community.

Looking forward to hear your ideas.

Simon

All I want is what we had before it was taken away. That's it.

On 09/12/2020 15:51, Scott Dowdle wrote:

Greetings,

----- Original Message -----

...

Any kernel device drivers, for a start. Kind of critical if your SAS/RAID device wont boot, or your network device doesn't come up, or your GUI doesn't start because your display drivers aren't compatible anymore. Just minor things like that maybe?

If RHEL 8.4 works, and CentOS Stream 8.4+ makes a change that breaks something... that's what is going to become RHEL... so does it matter THAT much if it breaks a month or two earlier than in the past? Now answering your question with a question, how about another? Did you have much breakage as a result of EL minor updates? I assume the answer is no... and if no is the case, then you shouldn't have much breakage with Stream.

TYL,

Hi Scott,

WRT your first point - I would guess it matters to the user "if it breaks a month or two earlier" and they can't use their system until RHEL catches up and the two OSes converge in compatibility again.

To answer your second question, of the 50 or so 3rd party driver package I currently maintain for RHEL8, every single package broke due to changes in the kernel ABI between 8.2 and 8.3. So yes, I would expect a huge amount of breakage, especially in the early years (the very years Stream targets) where Red Hat is most active in backporting into the RHEL kernel. I would expect breakage to diminish as RHEL ages (e.g, years 5-10), so after active development ends, but that is kind of moot as Stream support has ended by then anyway. That's my experience of actively maintaining such packages for RHEL over a complete product cycle during the last 11-12 years.

If anyone is considering to fork CentOS 8 (I'm not talking about that "Stream"), count me in.

Otherwise I will switch to openSUSE Leap. At least they are not pushing me some testing ground.

Best Regards, Strahil Nikolov

В 12:07 -0500 на 08.12.2020 (вт), Phelps, Matthew написа:

I still haven't seen an answer to the question, "Who made this decision?" and, "How can we lobby to get it changed?"

On Tue, Dec 8, 2020 at 9:06 AM Rich Bowen rbowen@redhat.com wrote:

...

The future of the CentOS Project is CentOS Stream, and over the next year we’ll be shifting focus from CentOS Linux, the rebuild of Red Hat Enterprise Linux (RHEL), to CentOS Stream, which tracks just ahead of a current RHEL release. CentOS Linux 8, as a rebuild of RHEL 8, will end at the end of 2021. CentOS Stream continues after that date, serving as the upstream (development) branch of Red Hat Enterprise Linux.

Meanwhile, we understand many of you are deeply invested in CentOS Linux 7, and we’ll continue to produce that version through the remainder of the RHEL 7 life cycle. https://access.redhat.com/support/policy/updates/errata/#Life_Cycle_Dates

CentOS Stream will also be the centerpiece of a major shift in collaboration among the CentOS Special Interest Groups (SIGs). This ensures SIGs are developing and testing against what becomes the next version of RHEL. This also provides SIGs a clear single goal, rather than having to build and test for two releases. It gives the CentOS contributor community a great deal of influence in the future of RHEL. And it removes confusion around what “CentOS” means in the Linux distribution ecosystem.

When CentOS Linux 8 (the rebuild of RHEL8) ends, your best option will be to migrate to CentOS Stream 8, which is a small delta from CentOS Linux 8, and has regular updates like traditional CentOS Linux releases. If you are using CentOS Linux 8 in a production environment, and are concerned that CentOS Stream will not meet your needs, we encourage you to contact Red Hat about options.

We have an FAQ - https://centos.org/distro-faq/ - to help with your information and planning needs, as you figure out how this shift of project focus might affect you.

[See also: Red Hat's perspective on this.

https://www.redhat.com/en/blog/centos-stream-building-innovative-future-ente... ]

---

CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

On 2020/12/09 01:51, Johnny Hughes wrote:

On 12/8/20 6:24 PM, Phelps, Matthew wrote:

...

Was this decision forced, despite objection, on the CentOS board by the RedHat Liaison?

Yes or No?

No

Come on, man:

"[...] Given this, we’ve informed the CentOS Project Governing Board that we are shifting our investment fully from CentOS Linux to CentOS Stream."

... in https://www.redhat.com/en/blog/centos-stream-building-innovative-future-ente...

On Tue, Dec 8, 2020 at 8:47 PM Rob Thomas xrobau@gmail.com wrote:

I'm sure it was just phrasing. Perhaps if you asked 'Was this decision forced on the CentOS board?' you'd get a different answer.

Johnny mentioned in another thread that the CentOS Board made their choice because of the option of a RedHat Liaison overrule hanging over them.

In other words, yes; they were forced by RedHat.

On Wed, 9 Dec 2020 at 11:03, Mário Barbosa mario.barbosa@gmail.com wrote:

...

On 2020/12/09 01:51, Johnny Hughes wrote:

...

On 12/8/20 6:24 PM, Phelps, Matthew wrote:

...

Was this decision forced, despite objection, on the CentOS board by the RedHat Liaison?

Yes or No?

No

Come on, man:

"[...] Given this, we’ve informed the CentOS Project Governing Board that we are shifting our investment fully from CentOS Linux to CentOS Stream."

... in

https://www.redhat.com/en/blog/centos-stream-building-innovative-future-ente...

---

CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

---

CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

On Tue, 2020-12-08 at 20:57 -0500, Phelps, Matthew wrote:

On Tue, Dec 8, 2020 at 8:47 PM Rob Thomas xrobau@gmail.com wrote:

...

I'm sure it was just phrasing. Perhaps if you asked 'Was this decision forced on the CentOS board?' you'd get a different answer.

Johnny mentioned in another thread that the CentOS Board made their choice because of the option of a RedHat Liaison overrule hanging over them.

In other words, yes; they were forced by RedHat.

I am a bit surprised that nobody talks about the bigger picture here. Could this just be a consequence of RedHat de-prioritizing the OS? Are they focusing on their other more cloud based products (openshift etc) as that offers more value to IBM? If so, reducing costs for RHEL Linux by reducing Centos funding makes a lot of sense. I would expect even RHEL to loose steam.

For my home application of Centos I am not too worried: I still have a year to decide what to do. Stream is not an option for me probably, as I need ZFS support, which will become a lot more complicated with Stream as a moving target. Interesting times...

On 9/12/20 09:46, Johnny Hughes wrote:

On 12/8/20 7:47 PM, Rob Thomas wrote:

...

I'm sure it was just phrasing.  Perhaps if you asked 'Was this decision forced on the CentOS board?' you'd get a different answer.

On Wed, 9 Dec 2020 at 11:03, Mário Barbosa <mario.barbosa@gmail.com mailto:mario.barbosa@gmail.com> wrote:

 On 2020/12/09 01:51, Johnny Hughes wrote:
 > On 12/8/20 6:24 PM, Phelps, Matthew wrote:
 >>
 >> Was this decision forced, despite objection, on the CentOS board
 by the
 >> RedHat Liaison?
 >>
 >> Yes or No?
 >
 > No

 Come on, man:

 "[...] Given this, we’ve informed the CentOS Project Governing Board
 that we are shifting our investment fully from CentOS Linux to CentOS
 Stream."

 ... in
 https://www.redhat.com/en/blog/centos-stream-building-innovative-future-enterprise-linux
 <https://www.redhat.com/en/blog/centos-stream-building-innovative-future-enterprise-linux>

That is correct .. so, the Red Hat Liaison can use Section B. of the Governance to dictate a vote. If the board FORCES the use of this clause, then whatever was wanted (in this case by Red Hat) would get inacted in its entirety with no real input from the board.

https://www.centos.org/about/governance/voting/

The CentOS Board knows this, so we had a dialoge with Red Hat instead. Red Hat presented their case and listened to our response. There was a significant back and forth.

So, no one 'FORCED' the board to do anything. Red Hat told us what they were going to do (what you quoted). The board then made many recommendations in a back and forth negotiation. The board then made a decision. The decision was reluctant .. but it was unanimous.

And now this is the way forward. ______________________________________________

When the two only options are to be shot in the head or in the leg, the leg is the obvious choice, that doesn't mean that you had a real choice to begin with.

Pablo

On 12/9/20 7:14 AM, Julien Pivotto wrote:

On 09 Dec 06:46, Johnny Hughes wrote:

...

That is correct .. so, the Red Hat Liaison can use Section B. of the Governance to dictate a vote. If the board FORCES the use of this clause, then whatever was wanted (in this case by Red Hat) would get inacted in its entirety with no real input from the board.

https://www.centos.org/about/governance/voting/

The CentOS Board knows this, so we had a dialoge with Red Hat instead. Red Hat presented their case and listened to our response. There was a significant back and forth.

So, no one 'FORCED' the board to do anything. Red Hat told us what they were going to do (what you quoted). The board then made many recommendations in a back and forth negotiation. The board then made a decision. The decision was reluctant .. but it was unanimous.

And now this is the way forward.

Johnny,

As this was not dictated by Section B, it seems that the board could revert this decision by another vote.

I'd like to see this topic re-discussed, based on community feedback. Is that a possibility?

I very much doubt it. I have been doing this for 17 years and CentOS is basically my life's work. This was (for me personally) a heart wrenching decision. However, i see no other decision as a possibility. If there was, it would have been made.

As I said, there was a back and forth. We got all the concessions we could get. It is what it is. But as I also said, it was a unanimous decision.

Dear Matthew,

I get that you might be upset. Please read that one more time:

I have been doing this for 17 years and CentOS is basically my life's work. This was (for me personally) a heart wrenching decision.

As a person who is making Enterprise Linux rebuild and surrounding stack rebuilds for a few years, I cannot imagine how much this decision meant for Johnny. 17 years. It's a lot. I also believe that by saying "This was (for me personally) a heart wrenching decision." he also says that he made the unambiguous statement about this decision when he was asked. Saying that "We will not give up, like you did." sounds a little bit too passive-aggressive IMO. Don't blame passionate technical people like one on this list on probably higher-ups decision.

I also believe that because of this decision incoming year will bring some considerable changes in the Enterprise Linux/HPC landscape.

Best, Alex

On 12/9/20 2:34 PM, Phelps, Matthew wrote:

On Wed, Dec 9, 2020 at 8:24 AM Johnny Hughes <johnny@centos.org mailto:johnny@centos.org> wrote:

On 12/9/20 7:14 AM, Julien Pivotto wrote:
> On 09 Dec 06:46, Johnny Hughes wrote:
>>
>> That is correct .. so, the Red Hat Liaison can use Section B. of the
>> Governance to dictate a vote. If the board FORCES the use of this
>> clause, then whatever was wanted (in this case by Red Hat) would get
>> inacted in its entirety with no real input from the board.
>>
>> https://www.centos.org/about/governance/voting/
<https://www.centos.org/about/governance/voting/>
>>
>> The CentOS Board knows this, so we had a dialoge with Red Hat
instead.
>> Red Hat presented their case and listened to our response.  There
was a
>> significant back and forth.
>>
>> So, no one 'FORCED' the board to do anything.  Red Hat told us
what they
>> were going to do (what you quoted).  The board then made many
>> recommendations in a back and forth negotiation.  The board then
made a
>> decision.  The decision was reluctant .. but it was unanimous.
>>
>> And now this is the way forward.
>
>
> Johnny,
>
> As this was not dictated by Section B, it seems that the board could
> revert this decision by another vote.
>
> I'd like to see this topic re-discussed, based on community
feedback. Is
> that a possibility?
>

I very much doubt it. I have been doing this for 17 years and CentOS is
basically my life's work.  This was (for me personally) a heart
wrenching decision.  However, i see no other decision as a possibility.
 If there was, it would have been made.

As I said, there was a back and forth.  We got all the concessions we
could get.  It is what it is.  But as I also said, it was a unanimous
decision.

So who on the RedHat side can we plead with? We will not give up, like you did.

--

*Matt Phelps*

*Information Technology Specialist, Systems Administrator*

(Computation Facility, Smithsonian Astrophysical Observatory)

Center for Astrophysics | Harvard & Smithsonian

60 Garden Street | MS 39 | Cambridge, MA 02138

email: mphelps@cfa.harvard.edu mailto:mphelps@cfa.harvard.edu

cfa.harvard.edu http://cfa.harvard.edu/| Facebook http://cfa.harvard.edu/facebook| Twitter http://cfa.harvard.edu/twitter| YouTube http://cfa.harvard.edu/youtube| Newsletter http://cfa.harvard.edu/newsletter

---

CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel

On 12/9/20 8:34 AM, Phelps, Matthew wrote:

So who on the RedHat side can we plead with? We will not give up, like you did.

Matthew, this is grossly unfair. Johnny did not "give up." Nothing could be further from the truth here. You weren't in those meetings, and you are speaking out of turn.

On Wed, Dec 9, 2020 at 8:25 AM Johnny Hughes johnny@centos.org wrote:

On 12/9/20 7:14 AM, Julien Pivotto wrote:

...

On 09 Dec 06:46, Johnny Hughes wrote:

...

That is correct .. so, the Red Hat Liaison can use Section B. of the Governance to dictate a vote. If the board FORCES the use of this clause, then whatever was wanted (in this case by Red Hat) would get inacted in its entirety with no real input from the board.

https://www.centos.org/about/governance/voting/

The CentOS Board knows this, so we had a dialoge with Red Hat instead. Red Hat presented their case and listened to our response. There was a significant back and forth.

So, no one 'FORCED' the board to do anything. Red Hat told us what they were going to do (what you quoted). The board then made many recommendations in a back and forth negotiation. The board then made a decision. The decision was reluctant .. but it was unanimous.

And now this is the way forward.

Johnny,

As this was not dictated by Section B, it seems that the board could revert this decision by another vote.

I'd like to see this topic re-discussed, based on community feedback. Is that a possibility?

I very much doubt it. I have been doing this for 17 years and CentOS is basically my life's work. This was (for me personally) a heart wrenching decision. However, i see no other decision as a possibility. If there was, it would have been made.

As I said, there was a back and forth. We got all the concessions we could get. It is what it is. But as I also said, it was a unanimous decision.

Without naming names to protect who shared me this story, someone I know who was at one level below the CEO of a given company was in a meeting where the CEO decided to make this, er, Interesting decision. Members of the meeting brought up some concerns and then started a discussion on how to adjust the CEO plan to address or minimize the impact of the concerns. At a certain point the CEO exploded and said "you people are wasting my time. All I want is thumbs up or thumbs down. Choose now!"

Now, everyone knew if you wanted to go against the CEO you may want to brush your resume and keep your network current first; in his first year at that company he fired some 50 managers and replaced them with "people who were more appreciative to his grand view." So, it was a unanimous vote.

One of the unplanned consequences was many of their brightest employees started jumping ship as soon as the decision was announced. Two years later that CEO announced *he* decided to step down.

In other words, Johnny I think most people on this list would like to thank you for all the effort you have put to make CentOS what it is. And some of us can appreciate the situation you have been put in and will not blame you for the decision you had to make. We are not here to shoot the messenger; we are just pointing out there will be (not really) unexpected consequences to the CentOS move to the flow/leaky model. Some people will be able to adapt to it and follow that cheese, others will choose to look for another cheese.

All I can say is that this chapter of the CentOS/RH/IBM story is coming to an end and a new one is beginning.

And, I like cheese

On 09 Dec 07:24, Johnny Hughes wrote:

On 12/9/20 7:14 AM, Julien Pivotto wrote:

...

On 09 Dec 06:46, Johnny Hughes wrote:

...

That is correct .. so, the Red Hat Liaison can use Section B. of the Governance to dictate a vote. If the board FORCES the use of this clause, then whatever was wanted (in this case by Red Hat) would get inacted in its entirety with no real input from the board.

https://www.centos.org/about/governance/voting/

The CentOS Board knows this, so we had a dialoge with Red Hat instead. Red Hat presented their case and listened to our response. There was a significant back and forth.

So, no one 'FORCED' the board to do anything. Red Hat told us what they were going to do (what you quoted). The board then made many recommendations in a back and forth negotiation. The board then made a decision. The decision was reluctant .. but it was unanimous.

And now this is the way forward.

Johnny,

As this was not dictated by Section B, it seems that the board could revert this decision by another vote.

I'd like to see this topic re-discussed, based on community feedback. Is that a possibility?

I very much doubt it. I have been doing this for 17 years and CentOS is basically my life's work. This was (for me personally) a heart wrenching decision. However, i see no other decision as a possibility. If there was, it would have been made.

As I said, there was a back and forth. We got all the concessions we could get. It is what it is. But as I also said, it was a unanimous decision.

Thank you Johnny.