[CentOS-devel] Before You Get Mad About The CentOS Stream Change, Think About…

On 12/15/20 3:50 PM, Mike McGrath wrote:
> On Tue, Dec 15, 2020 at 1:41 PM Trevor Hemsley 
> <trevor.hemsley at ntlworld.com <mailto:trevor.hemsley at ntlworld.com>> wrote:
>
>     It's not too late to do the right thing. Red Hat can still back
>     off this betrayal of the community that use CentOS Linux and set
>     CentOS Linux free.
>     ...
>     You can hope it'll go away but it won't. Red Hat will always be
>     the company that broke its promises and killed CentOS Linux.
>
>
> I'm in this weird position where I'm regularly hearing from people 
> that thought that Red Hat made some sort of "We'll never change and 
> CentOS Linux will be around forever" announcement.  I'd suggest 
> everyone go back and re-read the original press release (I was not 
> involved with the original agreement) - 
> https://www.redhat.com/en/about/press-releases/red-hat-and-centos-join-forces 
> <https://www.redhat.com/en/about/press-releases/red-hat-and-centos-join-forces>
>
First, Mike, thanks for taking the time to reply here; I for one greatly 
appreciate it.  I like Red Hat, and I like CentOS, and I count several 
current and former Red Hat employees as long-time close acquaintances, 
and even friends.  Finally got to meet Bill Nottingham a few weeks ago, 
but there are many from the 'old days' I've not yet had opportunity to 
meet in person.

> You can nitpick at words, or take a quote out of context.  But don't 
> be naive and pretend we had some grand plan for all of this from the 
> beginning.  Just like anyone, Red Hat changes and makes decisions 
> based on the best information we have at the time.  CentOS Linux made 
> sense in 2014, it doesn't make sense in 2020.

The quote that has me somewhat riled up is from 
https://www.redhat.com/en/blog/transforming-development-experience-within-centos 
: "CentOS Stream is parallel to existing CentOS builds; this means that 
nothing changes for current users of CentOS Linux and services, even 
those that begin to explore the newly-released CentOS 8. "  In the 
context of the posted statement, and being the first sentence under the 
heading of "What does this mean for CentOS," this is to me a clear 
statement that nothing changes with the then-current (2019) CentOS model 
especially for users of CentOS 8; had CentOS 8 not been so specifically 
called out I personally wouldn't feel quite so blindsided.  Yes, 
business happens; yes, real life happens; yes promises get broken.  But 
broken promises produce broken relationships and have consequences, such 
as broken trust.

I started deployment into production based on that September 24, 2019 
statement, and the published 2029 EOL date; I specifically waited on 
CentOS 8's ecosystem to mature for upgrades from C6 rather than go ahead 
an upgrade to C7 based on this statement, delaying, in one instance, a 
workstation upgrade six months, with the user of that workstation 
complaining about the delay nearly daily, and my reply being "In order 
to get your new system to be stable until 2029 instead of 2024 I still 
think we should go with CentOS 8 for you;" if I had known then what I 
know now she would have gotten CentOS 7 and I would have had many less 
headaches for most of a year.

> You may not like it, but the CentOS community didn't evolve in any way 
> with the industry.  When I think about the talent on this list, and in 
> IRC, I can't help but wonder what went wrong.

There are a number of things that could have been done early on to make 
things different, in my opinion.  And you're right: the CentOS community 
has seen very little change in many years.  Some call it stagnation; 
some call it stability.  Some might even call it denial.

Using the Debian model from the get-go (unstable -> testing -> stable) 
instead of (Fedora+secretsauce -> RHEL x.0 beta) might have helped back 
then; that IS what CentOS Stream is doing, adding that 'testing' 
portion, although you guys at Red Hat might prefer to not call it that.  
The sequence becomes, if I'm thinking this through correctly, 
(Fedora+secretsauce -> RHEL x.0 Beta -> RHEL x.0 -> for $point in 1 2 3 
4 5 6 do (CentOS x Stream -> RHEL x.${point}); done) or somesuch.

But then there's the 'secretsauce' part of the issue.  RHEL development 
has been opaque since RHLEE 6.2E; RHL development was opaque prior to 
that, even.  I was on the Beta Team back in the day; I remember the city 
code names, the NDA, etc; I still have my executed copy of that NDA, for 
that matter, so there are certain cities I never speak of (:-) in case 
you missed the humor); still have some of the boxed sets from those 
days, too.  Lack of transparency is not a new issue.  CentOS Stream, in 
this regard, is a very refreshing development and will likely be a big 
win for transparency.  But will CentOS Stream eliminate the 
'secretsauce' bridge currently existing between Fedora and RHEL?  (It 
may be covered in one of the numerous posts about it, so forgive me if I 
missed it).  Will CentOS 9 Stream begin with a Fedora snapshot? (How 
that develops may not be even known by you guys at Red Hat as far as I 
know).  If a transparent path from Fedora through the entire Full 
Support phase of RHEL can be developed in the form of CentOS Stream, I 
think that's a very good thing.  I wouldn't mind using that myself, as 
long as hardware drivers don't frequently break that I need to use.  And 
currently they break every stinking point release!!!

>   As for the rest of you, where were you?

The mailing list archives are full of messages over the years from users 
offering to help in various ways.  Me personally?  Well, my name and 
email were still in the changelog for PostgreSQL through the end of 
CentOS 4, at least; I did my contributions upstream when I was able to 
spend the time to do so.  Life happens; those contributions had to yield 
to RL issues.

> And sure, we could have turned CentOS back over to some non-Red Hat 
> foundation.  But the fact is contrary to popular belief, we actually 
> like the engineers that work on CentOS, we like many of the users who 
> have cultivated relationships with Red Hat over the years.  We intend 
> on going forward with a healthy, mutually beneficial relationship 
> there.  If that's not you, and you're ready to leave - I think that's 
> unfortunate but I understand.

I seem to be one of the very few in my local area still advocating Red 
Hat-based systems.  Most of the Linux talent in my area is rabidly 
anti-Red Hat.  It got exponentially worse last week.  Several years 
back, I got to meet Michael Tiemann and Mark Webbink in Asheville; 
watched TruthHappens the first time; everything was so positive.   
Michael accepted my invitation to tour $dayjob, and he seemed to have a 
great time.  For a while the local LUGs were less anti-Red Hat.... How 
times have changed in the LUG-scape around here.

If it could be done over?  The most detrimental statements made, in my 
opinion, were those statements that could be read to imply a commitment 
by Red Hat for CentOS 8 until 2029; this was after all the published 
roadmap by the CentOS Prject prior to December.  In the press release 
you linked to, there is a now-dead link to a FAQ page; thanks to the 
Wayback Machine, I can re-read this FAQ page ( 
https://web.archive.org/web/20140906203025/http://community.redhat.com/centos-faq/ 
) and see that maybe it was wishful thinking the way I read the very 
positively-worded spin that FAQ put on virtually all questions.

And then there's the statement I quoted at the very beginning of this 
reply; there was wording that strongly implied a commitment that 
"nothing changes" specifically for CentOS 8.  The September 24, 2019 
statement, in my opinion, is what primarily set the stage for this 
backlash you see today.

Best regards.