On Mon, Dec 28, 2020 at 1:09 PM Laurențiu Păncescu lpancescu@centosproject.org wrote:
Red Hat acquired the CentOS brand and hired the CentOS core team. CentOS is their property, period. They could have decided directly at management level what to do, nobody forced them to even create a governance board. From Mike McGrath's description, it sounds they wanted to create community around CentOS similar to the one around Fedora, but things turned out different than expected and the association in people's minds, "CentOS is also from Red Hat", ended up cannibalizing their RHEL sales so they decided to put a stop to that. The Red Hat employees on the CentOS board didn't do what they wanted, but did what the company asked them to, I see no reason to focus on specific people. But the EOL should have been announced before 8.0, so people don't waste time migrating from 6 to 8.
I've been trying to separate cause and effect here, as I think it is somewhat important.
Everybody has their interests here. Every single person including myself and every contributor to this discussion has their own interests, and the people that made the decision in the first place. We don't need to get into what the interests are - it's only important to realize that they exist, and they are varied. This also makes it difficult to pick a particular claim like "profit!" and pin it on people, since many people who made the decision and who support the decision, do not directly or may not even indirectly profit from the decision.
I think the cause of this outcome is that the CentOS project was in a fragile place in 2014 and prior. The insiders can describe exactly how this was or wasn't so. Although the CentOS project could have expanded with further community outreach, an offer was put on the table that was difficult to refuse. Red Hat will sponsor the project, and streamline it. Without qualifying good or bad, it was recognized at the time that this was a conflict of interest situation but had great optics and would resolve immediate problems quickly and effectively. It was almost too good to be true. However, the trust in Red Hat as an honourable champion of open source, put these legitimate concerns aside.
Fast forward, and one by one - most, if not all of the voting members of the board were slowly replaced or influenced by people with Red Hat agendas. The emphasis on how to make things "better", and "contribute" was probably one of many chords that needed to play out successfully to convince people that this was in fact not only the right choice, but the inevitable and only choice. Who was representing the large community of "users" at this point? Which of these board members still had the original itch to scratch that created CentOS in the first place?
I think the future of the CentOS project was not only in a fragile place in 2014 and prior, but it was also in a fragile place in 2014 and after. It was mentioned by a few people here how there were "two months without security updates" as one of several reasons why CentOS was already "less than" RHEL, but it wasn't really acknowledged as to why - why can't CentOS be great? I don't believe Red Hat was incapable of making it great. I think the only valid explanation is that Red Hat didn't have an interest in making it great. Red Hat could have put more people on it, or streamlined the build process and guaranteed reproducibility by building RHEL and CentOS side-by-side if de-branding was still a requirement. Red Hat could have shutdown CentOS, and made RHEL available under the same terms as CentOS. But, they saw no reason to do so. they were content to leave it with insufficient resources, running behind RHEL by weeks to months, and having no security updates for two months. There might be certain heroes in the community who did all the work, and these people might be highly respected - but is it valid for sponsors to set up an organization that relies on heroic work to do something that is still "less than" RHEL?
In 2014 and prior, the community could have fixed the problem. In 2014 and after, the community was controlled by Red Hat. This wasn't absolutely clear in 2014, but it is now absolutely clear in 2020. Not only is CentOS being moved from "just downstream" to "just upstream" from the official RHEL releases, but CentOS is no longer a separate project at any arm's length from Red Hat. It's now integrated into their internal processes, and it is not really an "upstream" as most people would assume, but it is still downstream of the Red Hat controlled private branches, making it more like a preview and less like a method for community contribution.
CentOS Stream is much more like an "insider preview" as another person has been saying. "Insider preview" is valuable, but it's a very different thing from what CentOS was, and why people chose CentOS over Ubuntu. For example, I've been reflecting with some other users on how they wanted to go with Ubuntu, but chose CentOS *only* because it had 10 year life span, instead of 5 years. By dropping the 10 years down to 5 years, CentOS Stream is now much more similar to Ubuntu LTS, and Ubuntu is far more popular in developer communities.
And this comes to consequence. Even if everybody involved was trying to make the best decision possible in their own interests, there is a very real chance that their own interests will be the ones that are impacted the most. Red Hat took control of CentOS, presumedly to take control of one of the most popular downstream builds of RHEL. But, now CentOS will be re-created with one or more new names, and the 2014 acquisition of CentOS will have been effectively undone in every way except the name of the project. Red Hat will have abandoned control of the most popular downstream. Users will move to Ubuntu, Oracle Linux, Rocky, or one of several others, and this will include their non-financial contributions such as time and effort opening bug reports and getting problems fixed, and it will include downstream support contracts. Red Hat will have gained a reputation for failing to live up to commitments, and putting all future commitments in question. Other than Red Hat staff, and a few others who want a CentOS Stream (and don't need a CentOS), the great community will remember this. This isn't just individuals having uncomfortable conversations with their bosses - this will be bosses questioning why they should use Red Hat anywhere else. Even if CentOS Stream is successful in the areas it is intended to be successful, the damage will be done. I know my procurement team will be wary of any future Red Hat contracts - and this extends to all the other products beyond RHEL. There is no reason to believe an acceptable RHEL subscription will be made available that would substitute for CentOS usage, and if we must use something else for many of our machines, we may as well use something else for *all* of our machines.
From this perspective, I almost think this change was inevitable. Red
Hat was (intentionally?) not a good steward for CentOS. It was not run like a community, with community elected board members. Red Hat put just enough resources into CentOS to prevent anybody new from trying to re-create it, but not enough to allow it to compete directly with RHEL. It stayed like this essentially "on hold" for the last 6 years, being "just good enough, but not excellent". Now, a change has been made. CentOS as it was originally envisioned, is now deprecated and soon to be eliminated forever. CentOS Stream, which is the "insider preview" for RHEL is now to be released, as the new "just good enough, but not excellent, only this time it no longer effectively competes with RHEL, but instead helps harden RHEL."
If they wanted to kill all clones, they could have stopped publishing the sources (they only have to provide them to their own customers, like they do for EUS updates, not to everybody on the Internet). My impression is they want to kill just CentOS Linux, the only clone whose association with Red Hat seems to give it a legitimacy that other clones don't have.
Killing clones is not only difficult - but hopefully some people are aware that it would also be a problem. We are now on the precipice of "clones" now breaking from RHEL on definition of content. I foresee that if the exact package lineup is too hard to align with RHEL, that it will diverge. Oracle Linux 8.5 might not match RHEL 8.5, and this may be more of a problem for RHEL than for anybody else. "Clones" is more like "imitation is the most sincere form of flattery" and less like "others cannot possibly provide the same level of quality backports". The former is just a matter of convenience and an attempt to consolidate the entire EL community around common baselines so that packages that work on one system will also work with a great deal of confidence on another system. The latter is just a matter of scale. When it comes to the Linux kernel, Oracle has basically already broken from RHEL, in that they promote and install by default the Oracle UEK, and only offer the "Red Hat Compatible Kernel" as an option if necessary.
I think the best way forward for everyone is to accept what happened and deal pragmatically with the consequences for their own organization.
Yes.
I think it's important for a lot of the concerns to be captured in Internet history, so that we can learn from it. But, it seems we are far past the point of changing anything on the Red Hat side of things. As you and others have mentioned - the wording used suggests this is all final. This is why I will not be sending anything to the new mailing list. I already sent plenty of explanation to Red Hat in 2016 and 2017, and they took no action then.