Hi,
I just read this blog article from austrian Linux expert Michael Kofler. For those among you who don't know the guy, he's my home country's number one Linux expert (known as "der Kofler") and most notably the author of a series of excellent books about Linux over the last 25 years.
https://kofler.info/centos-8-wertlose-langzeitunterstuetzung/
Disclaimer : I've been a CentOS user (and fan) since 4.x, I'm using it on all my servers, and yes, I know the difference between upstream RHEL and CentOS.
The article is in german, but the statistics graph is eloquent enough for the non-german-speaking users. It focuses on updates for CentOS 8, and more exactly the extended periods of time where there have been no updates available.
The author's theory ("unspoken truth"): while it's a positive thing that Red Hat is sponsoring CentOS, the amount of sponsoring is just insufficient enough so that the product is "starved to death" by Red Hat (e. g. IBM) to encourage users to move to RHEL.
The author's conclusion is quite severe: in the current state of things, CentOS 8 is not recommendable for production as updates are lagging too much behind. While CentOS 7 may be usable, CentOS 8 has been "degraded to teaching and testing purposes".
Still according to Mister Kofler, this "sorry state of things" will probably encourage users to move to Oracle Linux, the other big RHEL clone.
After some hesitation, I decided to share this on the mailing list. Since this raises some concerns, I'd be curious to have your take on this.
Cheers from the sunny South of France,
Niki
Il 17/06/20 09:16, Nicolas Kovacs ha scritto:
Hi,
I just read this blog article from austrian Linux expert Michael Kofler. For those among you who don't know the guy, he's my home country's number one Linux expert (known as "der Kofler") and most notably the author of a series of excellent books about Linux over the last 25 years.
https://kofler.info/centos-8-wertlose-langzeitunterstuetzung/
Disclaimer : I've been a CentOS user (and fan) since 4.x, I'm using it on all my servers, and yes, I know the difference between upstream RHEL and CentOS.
The article is in german, but the statistics graph is eloquent enough for the non-german-speaking users. It focuses on updates for CentOS 8, and more exactly the extended periods of time where there have been no updates available.
The author's theory ("unspoken truth"): while it's a positive thing that Red Hat is sponsoring CentOS, the amount of sponsoring is just insufficient enough so that the product is "starved to death" by Red Hat (e. g. IBM) to encourage users to move to RHEL.
The author's conclusion is quite severe: in the current state of things, CentOS 8 is not recommendable for production as updates are lagging too much behind. While CentOS 7 may be usable, CentOS 8 has been "degraded to teaching and testing purposes".
Still according to Mister Kofler, this "sorry state of things" will probably encourage users to move to Oracle Linux, the other big RHEL clone.
After some hesitation, I decided to share this on the mailing list. Since this raises some concerns, I'd be curious to have your take on this.
Cheers from the sunny South of France,
Niki
Hi Niki, this is a sad thing but I'm not surprised by this and this is what I'm thinking about CentOS since 8 was released. The CentOS team does a huge work to release CentOS8, CentOS Stream and maintaining 7 and 6. Thank you so much for this CentOS Team. But since the core team is composed by 6(?) member is obvious that they can't maintain all this versions without being in late and because it is supported (but maybe supported is the wrong word in the current case) by RH, if RH really interest about CentOS release they should put more effort on CentOS Development. This is not what is happening and since 8 is released, it seems a try and buy distro. I walked through 6.5 to 7 and 8. In 6.5 I was new, in 7 I really appreciated CentOS but with 8 I noticed that it is not supported like CentOS 7.
I use (like you) centos on all my server, some are anchored to the old stable and other on 8 (but not production). I liked it very much that I use it also on my workstation.
About update blackout, this is a great problem for production system faced on Internet. SELinux can mitigate the risk (but also selinux got security fix) but this is always an issue. Many report that the gap between RHEL release and CentOS release is too much but I'm not interested in how much time centos releases get through the build process but I'm interested that, when a new centos release (major/minor) are on the way, all others releases (current stable inside) will get security updates during the build process and not left alone. Another minor issue about updates are relative announces on ml that works for C7 but not for C8 (I read in a reddit AMA why this happens). This is annoying because I need to follow every RHBA to see what happened and why a package is updated or blindly install updates. This is another thing that is not so good for a server.
If the theory "unspoken truth" is real I don't like how RH is trying to encourage me to switch to RHEL and this is a bad way to do this.
About Oracle as alternative. Oracle Linux is not an alternative to CentOS but for RHEL and if I will force to pay for enteprise system currently I will pay RHEL, not OL. Over this, OL is not the only enterprise distro that a "user" could choose. If support is needed there are SUSE (SLES) and Ubuntu. For who that don't need support there are Debian, Ubuntu, OpenSUSE (I'm talking about the most used but you know that slackware,FreeBSD are in that list), so many alternatives are in place.
I hope that things will be better but in case of "failure" I will evaluate other alternatives to CentOS for this "sorry state of things".
Thank you for sharing, I found it interesting.
My 2 Cent
*snip
Thank you for sharing, I found it interesting.
My 2 Cent _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
+1
Longtime user but with the current sturcture they have turned it (centos8) into really a non production release. I've always tried to use it on internal servers but these are small businesses (non-profit) that will pay for a windows server license vs a RHEL. Microsoft has actually expanded and opened it up even more. I really thought it was a good thing for RHEL to support Centos since they had made the builds more difficult but it doesn't appear that they do not want to support it too much. As it currently stands I could not or will not deploy Centos8 in a internet facing environment.
I continue to run 8 at home but do not see myself deploying it into production anywhere. I deeply appreciate all the hard work that goes into putting it out, Karanbir, Fabian, Johnny, Ralph and Tru and all the others put in to produce and support it. :)
My hope is that RHEL changes the level of support they provide to centOS but not confident it's going to happen here's hoping.
;)
About Oracle as alternative. Oracle Linux is not an alternative to CentOS but for RHEL and if I will force to pay for enteprise system currently I will pay RHEL, not OL. Over this, OL is not the only enterprise distro that a "user" could choose. If support is needed there are SUSE (SLES) and Ubuntu. For who that don't need support there are Debian, Ubuntu, OpenSUSE (I'm talking about the most used but you know that slackware,FreeBSD are in that list), so many alternatives are in place.
I think it's particularly disappointing *if* this is a "policy" from RH since the other major RHEL clone, Scientific Linux, has not produced an EL8 offering in favour of using CentOS.
I think all of us here understand the hugely complex process of producing a quality OS, even when it's "just" a clone of another one. The official sanctioning from RH was touted as a two-way process: community input into RHEL and RH support and help of the cloning and build process. It would be a bit underhand if it turned out that it was RH's way of creating a two tier system: buy RHEL+support and get timely updates; use CentOS for free, get security updates, but wait two months for each upgrade.
P.
On Wed, Jun 17, 2020 at 02:23:36PM +0100, Pete Biggs wrote:
About Oracle as alternative. Oracle Linux is not an alternative to CentOS but for RHEL and if I will force to pay for enteprise system currently I will pay RHEL, not OL. Over this, OL is not the only enterprise distro that a "user" could choose. If support is needed there are SUSE (SLES) and Ubuntu. For who that don't need support there are Debian, Ubuntu, OpenSUSE (I'm talking about the most used but you know that slackware,FreeBSD are in that list), so many alternatives are in place.
I think it's particularly disappointing *if* this is a "policy" from RH since the other major RHEL clone, Scientific Linux, has not produced an EL8 offering in favour of using CentOS.
Keep in mind that as soon as Scientific Linux started taking off, IIRC, because CentOS was late with a release, RH quickly hired its main developer. I don't think we can really expect RH to act differently than most corporations.
On Wed, 17 Jun 2020 at 09:42, Scott Robbins scottro11@gmail.com wrote:
On Wed, Jun 17, 2020 at 02:23:36PM +0100, Pete Biggs wrote:
About Oracle as alternative. Oracle Linux is not an alternative to CentOS but for RHEL and if I will force to pay for enteprise system currently I will pay RHEL, not OL. Over this, OL is not the only enterprise distro that a "user" could choose. If support is needed there are SUSE (SLES) and Ubuntu. For who that don't need support there are Debian, Ubuntu, OpenSUSE (I'm talking about the most used but you know that slackware,FreeBSD are in that list), so many alternatives are in place.
I think it's particularly disappointing *if* this is a "policy" from RH since the other major RHEL clone, Scientific Linux, has not produced an EL8 offering in favour of using CentOS.
Keep in mind that as soon as Scientific Linux started taking off, IIRC, because CentOS was late with a release, RH quickly hired its main developer. I don't think we can really expect RH to act differently than most corporations.
A major reason various people were hired from Fermi labs was very much that DOE(*) was cutting back funding on Fermilab and there were a lot of layoffs for anything which could not be the 'core' mission of Fermi. Making operating systems by any of the labs in the DOE complex was considered to be not a core mission, and multiple groups were either laid off or 'asked' to retire. The hiring of people was mainly because people were soon to be out of a job.
DOE - US Department of Energy which funds Fermi and Argonne National Labs in Illinois as physics science institutes.
-- Scott Robbins PGP keyID EB3467D6 ( 1B48 077D 66F6 9DB0 FDC2 A409 FA54 EB34 67D6 ) gpg --keyserver pgp.mit.edu --recv-keys EB3467D6
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Il 17/06/20 15:42, Scott Robbins ha scritto:
On Wed, Jun 17, 2020 at 02:23:36PM +0100, Pete Biggs wrote:
About Oracle as alternative. Oracle Linux is not an alternative to CentOS but for RHEL and if I will force to pay for enteprise system currently I will pay RHEL, not OL. Over this, OL is not the only enterprise distro that a "user" could choose. If support is needed there are SUSE (SLES) and Ubuntu. For who that don't need support there are Debian, Ubuntu, OpenSUSE (I'm talking about the most used but you know that slackware,FreeBSD are in that list), so many alternatives are in place.
I think it's particularly disappointing *if* this is a "policy" from RH since the other major RHEL clone, Scientific Linux, has not produced an EL8 offering in favour of using CentOS.
Keep in mind that as soon as Scientific Linux started taking off, IIRC, because CentOS was late with a release, RH quickly hired its main developer. I don't think we can really expect RH to act differently than most corporations.
Yes but today Scientific Linux is out of games and CERN made CentOS CERN based on C8 so there is no more a valid competitor for this target (in the RH family based distro)
CentOS at the current status (third 8 release) is not suitable for production with the current issues and we are forced to buy rhel license (and every needed extension).
I would to know what think all ISPs that offer centos 8 on their VPS, Dedicated Server, Shared Hosting and how they handle this problem (probably they don't because the choice is on the customer but maybe...)
As you said, probably nothing will change for the corporation "limit" concept but at this point, probably, many users will migrate to another platform with less limitation.
On Jun 17, 2020, at 9:10 AM, Alessandro Baggi alessandro.baggi@gmail.com wrote:
Il 17/06/20 15:42, Scott Robbins ha scritto:
On Wed, Jun 17, 2020 at 02:23:36PM +0100, Pete Biggs wrote:
About Oracle as alternative. Oracle Linux is not an alternative to CentOS but for RHEL and if I will force to pay for enteprise system currently I will pay RHEL, not OL. Over this, OL is not the only enterprise distro that a "user" could choose. If support is needed there are SUSE (SLES) and Ubuntu. For who that don't need support there are Debian, Ubuntu, OpenSUSE (I'm talking about the most used but you know that slackware,FreeBSD are in that list), so many alternatives are in place.
I think it's particularly disappointing *if* this is a "policy" from RH since the other major RHEL clone, Scientific Linux, has not produced an EL8 offering in favour of using CentOS.
Keep in mind that as soon as Scientific Linux started taking off, IIRC, because CentOS was late with a release, RH quickly hired its main developer. I don't think we can really expect RH to act differently than most corporations.
Yes but today Scientific Linux is out of games and CERN made CentOS CERN based on C8 so there is no more a valid competitor for this target (in the RH family based distro)
“Scientific” linux, with all due respect to one of US national laboratories situated nearby that fathered it, was never viable from the long term use standpoint system in my book.
I got experimental proof about 5 or 6 years ago when “Scientific” linux (no, I’m not misplacing the second quote mark ;-) didn’t release updates for some 6 Months at least. I am more unimpressed by CERN’s nearsightness, than I am impressed by my own long term predictive power. The thruth is: “Scientific” linux is the creature of small team, situated in one place, and those who put it together have primamry dedication to their job place, NOT the the “scientific” linux project and its existense. And I am not mentioning what I was mentioning to my researchers when they asked to install “scientific” linux, and I was giving them a list of arguments why they will be better off with CentOS - in addition to the above, CentOS is much reacher without extra repositories (which when there is large enogh number tend to create mess for you).
Just my 2 cents.
Valeri
CentOS at the current status (third 8 release) is not suitable for production with the current issues and we are forced to buy rhel license (and every needed extension).
I would to know what think all ISPs that offer centos 8 on their VPS, Dedicated Server, Shared Hosting and how they handle this problem (probably they don't because the choice is on the customer but maybe...)
As you said, probably nothing will change for the corporation "limit" concept but at this point, probably, many users will migrate to another platform with less limitation. _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Hi,
I just read this blog article from austrian Linux expert Michael Kofler. For those among you who don't know the guy, he's my home country's number one Linux expert (known as "der Kofler") and most notably the author of a series of excellent books about Linux over the last 25 years.
https://kofler.info/centos-8-wertlose-langzeitunterstuetzung/
Disclaimer : I've been a CentOS user (and fan) since 4.x, I'm using it on all my servers, and yes, I know the difference between upstream RHEL and CentOS.
The article is in german, but the statistics graph is eloquent enough for the non-german-speaking users. It focuses on updates for CentOS 8, and more exactly the extended periods of time where there have been no updates available.
The author's theory ("unspoken truth"): while it's a positive thing that Red Hat is sponsoring CentOS, the amount of sponsoring is just insufficient enough so that the product is "starved to death" by Red Hat (e. g. IBM) to encourage users to move to RHEL.
I think if Red Hat really wanted to improve the situation, they could integrate the building of CentOS into the EL build system to produce both versions, RHEL and CentOS, at the same time. In the end 99% of the bits are the same anyway. If the delay of CentOS builds is really wanted by Red Hat, it would be nice of them to speak it out - and change the name to COS, because the ent is not true anymore :-)
Up to now I thought the big delay with 8 is more an accident than wanted. Would be nice to hear what Red Hat says about it. Maybe the problem is not known well enough in the Red Hat universe.
Regards, Simon
On 6/17/20 8:06 AM, Simon Matter via CentOS wrote:
Hi,
I just read this blog article from austrian Linux expert Michael Kofler. For those among you who don't know the guy, he's my home country's number one Linux expert (known as "der Kofler") and most notably the author of a series of excellent books about Linux over the last 25 years.
https://kofler.info/centos-8-wertlose-langzeitunterstuetzung/
Disclaimer : I've been a CentOS user (and fan) since 4.x, I'm using it on all my servers, and yes, I know the difference between upstream RHEL and CentOS.
The article is in german, but the statistics graph is eloquent enough for the non-german-speaking users. It focuses on updates for CentOS 8, and more exactly the extended periods of time where there have been no updates available.
The author's theory ("unspoken truth"): while it's a positive thing that Red Hat is sponsoring CentOS, the amount of sponsoring is just insufficient enough so that the product is "starved to death" by Red Hat (e. g. IBM) to encourage users to move to RHEL.
I think if Red Hat really wanted to improve the situation, they could integrate the building of CentOS into the EL build system to produce both versions, RHEL and CentOS, at the same time. In the end 99% of the bits are the same anyway. If the delay of CentOS builds is really wanted by Red Hat, it would be nice of them to speak it out - and change the name to COS, because the ent is not true anymore :-)
Up to now I thought the big delay with 8 is more an accident than wanted. Would be nice to hear what Red Hat says about it. Maybe the problem is not known well enough in the Red Hat universe.
No one is trynig to make anything slower.
And CentOS Stream 'is' going to be how RHEL is developed in the future. So, all this is happening.
But modules introduced in RHEL 8 requires a who new build system (as koji set up) and a whole new module build back end (MBS).
If you would rather use Oracle for your open source needs .. well, that is a decision you can make and be responsible for.
If you would instead have feedback directly into the RHEL development process as a community .. then CentOS is where you want to be.
This stuff is open source, and you are all gown ups who can make your own decisions.
I can assure you .. I am working my butt off everyday to make CentOS Linux the best it can be. If you want to compare what the CentOS team (a small team) can do compared to Oracle (a tmulti billin dollar corporation who bought Sun Microsystems .. took over Java and Open Office, etc) .. well, we can not provide the resources they can provide.
But Red hat heard the community .. that the community and Red Hat customers want more direct input into RHEL development. And Red Hat is taking action to open up RHEL development in CentOS Stream.
You get to decide who you trust.
Thanks, Johnny hUghes
On Jun 17, 2020, at 9:15 AM, Johnny Hughes johnny@centos.org wrote:
On 6/17/20 8:06 AM, Simon Matter via CentOS wrote:
Hi,
I just read this blog article from austrian Linux expert Michael Kofler. For those among you who don't know the guy, he's my home country's number one Linux expert (known as "der Kofler") and most notably the author of a series of excellent books about Linux over the last 25 years.
https://kofler.info/centos-8-wertlose-langzeitunterstuetzung/
Disclaimer : I've been a CentOS user (and fan) since 4.x, I'm using it on all my servers, and yes, I know the difference between upstream RHEL and CentOS.
The article is in german, but the statistics graph is eloquent enough for the non-german-speaking users. It focuses on updates for CentOS 8, and more exactly the extended periods of time where there have been no updates available.
The author's theory ("unspoken truth"): while it's a positive thing that Red Hat is sponsoring CentOS, the amount of sponsoring is just insufficient enough so that the product is "starved to death" by Red Hat (e. g. IBM) to encourage users to move to RHEL.
I think if Red Hat really wanted to improve the situation, they could integrate the building of CentOS into the EL build system to produce both versions, RHEL and CentOS, at the same time. In the end 99% of the bits are the same anyway. If the delay of CentOS builds is really wanted by Red Hat, it would be nice of them to speak it out - and change the name to COS, because the ent is not true anymore :-)
Up to now I thought the big delay with 8 is more an accident than wanted. Would be nice to hear what Red Hat says about it. Maybe the problem is not known well enough in the Red Hat universe.
No one is trynig to make anything slower.
And CentOS Stream 'is' going to be how RHEL is developed in the future. So, all this is happening.
But modules introduced in RHEL 8 requires a who new build system (as koji set up) and a whole new module build back end (MBS).
If you would rather use Oracle for your open source needs .. well, that is a decision you can make and be responsible for.
If you would instead have feedback directly into the RHEL development process as a community .. then CentOS is where you want to be.
This stuff is open source, and you are all gown ups who can make your own decisions.
I can assure you .. I am working my butt off everyday to make CentOS Linux the best it can be. If you want to compare what the CentOS team (a small team) can do compared to Oracle (a tmulti billin dollar corporation who bought Sun Microsystems .. took over Java and Open Office, etc) .. well, we can not provide the resources they can provide.
But Red hat heard the community .. that the community and Red Hat customers want more direct input into RHEL development. And Red Hat is taking action to open up RHEL development in CentOS Stream.
You get to decide who you trust.
Thank you, Johnny!
I was going to reply to original post that: Nobody ever promised to pay CentOS team to do what they do. That was my understanding always.
Changes in the OS, new components (do not mean systemd explicitly), and how new release is built may or may not be part of longer/bigger plan of RHEL owner.
Without regard to the above, RedHat always was following meticulously the letter of GNU license. And the reputation IBM has (in my book), does not raise red flag after they bought out RedHat. Time will show, but I’m sure with acquisition of RedHat by IBM “nothing changed” for open source community in general, and CentOS existence in particular. Not dramatically at least. And someone mentioned Oracle, and I agree, Oracle has quite opposite reputation in my boot to that of IBM.
Thanks, Johnny, very much again!!
Note: I’m just humble outsider, not CentOS project member.
Valeri
Thanks, Johnny hUghes
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Hi Johnny, thank you for your and all centos team works.
Many of us know how much work is needed for building new releases and maintaining C6 and C7, plus CentOS Stream and modules (Appstream). This is a huge work for a small team. Again thank you.
For me OL is not an alternative.
As reported in my previous message I'm not worried about how much time is required to build the new (major/minor) release, it will be ready when it will be. My major concern is about the "security update blackout" that take long as the build process.
I would ask to you:
1. Why all security fix are stopped when a new release building process is started? There is a way or possibility to run the two process in parallel?
2. When a build process is started and a security fix released there is a way for your team to "suspend" the building process, release security updates (for 6/7.x or 8.1) and resume the builing process? I think that many users (included me) will have less disappointment having security updates instead of receiving a "signal lost" when building process takes its way.
Thank you for your time.
Alessandro.
Il Mer 17 Giu 2020, 16:15 Johnny Hughes johnny@centos.org ha scritto:
On 6/17/20 8:06 AM, Simon Matter via CentOS wrote:
Hi,
I just read this blog article from austrian Linux expert Michael Kofler. For those among you who don't know the guy, he's my home country's number
one
Linux expert (known as "der Kofler") and most notably the author of a series
of
excellent books about Linux over the last 25 years.
https://kofler.info/centos-8-wertlose-langzeitunterstuetzung/
Disclaimer : I've been a CentOS user (and fan) since 4.x, I'm using it
on
all my servers, and yes, I know the difference between upstream RHEL and CentOS.
The article is in german, but the statistics graph is eloquent enough
for
the non-german-speaking users. It focuses on updates for CentOS 8, and more exactly the extended periods of time where there have been no updates available.
The author's theory ("unspoken truth"): while it's a positive thing that Red Hat is sponsoring CentOS, the amount of sponsoring is just insufficient enough so that the product is "starved to death" by Red Hat (e. g. IBM) to encourage users to move to RHEL.
I think if Red Hat really wanted to improve the situation, they could integrate the building of CentOS into the EL build system to produce both versions, RHEL and CentOS, at the same time. In the end 99% of the bits are the same anyway. If the delay of CentOS builds is really wanted by
Red
Hat, it would be nice of them to speak it out - and change the name to COS, because the ent is not true anymore :-)
Up to now I thought the big delay with 8 is more an accident than wanted. Would be nice to hear what Red Hat says about it. Maybe the problem is
not
known well enough in the Red Hat universe.
No one is trynig to make anything slower.
And CentOS Stream 'is' going to be how RHEL is developed in the future. So, all this is happening.
But modules introduced in RHEL 8 requires a who new build system (as koji set up) and a whole new module build back end (MBS).
If you would rather use Oracle for your open source needs .. well, that is a decision you can make and be responsible for.
If you would instead have feedback directly into the RHEL development process as a community .. then CentOS is where you want to be.
This stuff is open source, and you are all gown ups who can make your own decisions.
I can assure you .. I am working my butt off everyday to make CentOS Linux the best it can be. If you want to compare what the CentOS team (a small team) can do compared to Oracle (a tmulti billin dollar corporation who bought Sun Microsystems .. took over Java and Open Office, etc) .. well, we can not provide the resources they can provide.
But Red hat heard the community .. that the community and Red Hat customers want more direct input into RHEL development. And Red Hat is taking action to open up RHEL development in CentOS Stream.
You get to decide who you trust.
Thanks, Johnny hUghes
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Once upon a time, Alessandro Baggi alessandro.baggi@gmail.com said:
As reported in my previous message I'm not worried about how much time is required to build the new (major/minor) release, it will be ready when it will be. My major concern is about the "security update blackout" that take long as the build process.
I'm not involved in building CentOS, but the issue is that it is a rebuild of upstream. When RHEL 8.2 is released, there are no more upstream updates released for RHEL 8.1; they are all on top of the RHEL 8.2 release. So, until the time that CentOS can rebuild RHEL 8.2 and make a new CentOS release, there can't be any updates for CentOS 8.1.
RHEL 8 introduced modules, which complicated the build system and required new tooling, so CentOS has had a bunch of "under the hood" work to catch up. Hopefully, once that's ironed out, the gap between a RHEL 8.x release and the corresponding CentOS release will drop.
On Wed, 17 Jun 2020 at 13:11, Alessandro Baggi alessandro.baggi@gmail.com wrote:
Hi Johnny, thank you for your and all centos team works.
Many of us know how much work is needed for building new releases and maintaining C6 and C7, plus CentOS Stream and modules (Appstream). This is a huge work for a small team. Again thank you.
For me OL is not an alternative.
As reported in my previous message I'm not worried about how much time is required to build the new (major/minor) release, it will be ready when it will be. My major concern is about the "security update blackout" that take long as the build process.
I would ask to you:
- Why all security fix are stopped when a new release building process is
started? There is a way or possibility to run the two process in parallel?
Most of the security fixes after the minor release are based off being built with the code from that minor release. Sometimes the code may even not compile without the rest of the packages involved. Deciding which packages can and can not be affected by the minor release can be a full time job.
- When a build process is started and a security fix released there is a
way for your team to "suspend" the building process, release security updates (for 6/7.x or 8.1) and resume the builing process? I think that many users (included me) will have less disappointment having security updates instead of receiving a "signal lost" when building process takes its way.
It would require a second build system which built against the older set of packages, it would also require someone to edit those packages so they can be replaced by later released ones after the main release is done. It also requires work on making sure conflicts and broken packages are dealt with.
A lot of this is sausage making.. We all want the end product of a cooked sausage.. but there is a lot of messy work which adding more people to doesn't help unless you can add a lot of other things which are supposed to appear ex-nihilo and fully formed. You can have a lot of people volunteer to stand around but there are only so many places to push meat on one side, turn cranks in the middle and one tube which the mess gets stuffed into. Every extra thing requires months and months of work and preparation to set up while still trying to make the other items. However the only time people seem to get bothered enough to even try to set up their own build system and find out how much mess it is.. is after the current release is delayed.
Thank you for your time.
Alessandro.
Il Mer 17 Giu 2020, 16:15 Johnny Hughes johnny@centos.org ha scritto:
On 6/17/20 8:06 AM, Simon Matter via CentOS wrote:
Hi,
I just read this blog article from austrian Linux expert Michael Kofler. For those among you who don't know the guy, he's my home country's number
one
Linux expert (known as "der Kofler") and most notably the author of a series
of
excellent books about Linux over the last 25 years.
https://kofler.info/centos-8-wertlose-langzeitunterstuetzung/
Disclaimer : I've been a CentOS user (and fan) since 4.x, I'm using it
on
all my servers, and yes, I know the difference between upstream RHEL and CentOS.
The article is in german, but the statistics graph is eloquent enough
for
the non-german-speaking users. It focuses on updates for CentOS 8, and more exactly the extended periods of time where there have been no updates available.
The author's theory ("unspoken truth"): while it's a positive thing that Red Hat is sponsoring CentOS, the amount of sponsoring is just insufficient enough so that the product is "starved to death" by Red Hat (e. g. IBM) to encourage users to move to RHEL.
I think if Red Hat really wanted to improve the situation, they could integrate the building of CentOS into the EL build system to produce both versions, RHEL and CentOS, at the same time. In the end 99% of the bits are the same anyway. If the delay of CentOS builds is really wanted by
Red
Hat, it would be nice of them to speak it out - and change the name to COS, because the ent is not true anymore :-)
Up to now I thought the big delay with 8 is more an accident than wanted. Would be nice to hear what Red Hat says about it. Maybe the problem is
not
known well enough in the Red Hat universe.
No one is trynig to make anything slower.
And CentOS Stream 'is' going to be how RHEL is developed in the future. So, all this is happening.
But modules introduced in RHEL 8 requires a who new build system (as koji set up) and a whole new module build back end (MBS).
If you would rather use Oracle for your open source needs .. well, that is a decision you can make and be responsible for.
If you would instead have feedback directly into the RHEL development process as a community .. then CentOS is where you want to be.
This stuff is open source, and you are all gown ups who can make your own decisions.
I can assure you .. I am working my butt off everyday to make CentOS Linux the best it can be. If you want to compare what the CentOS team (a small team) can do compared to Oracle (a tmulti billin dollar corporation who bought Sun Microsystems .. took over Java and Open Office, etc) .. well, we can not provide the resources they can provide.
But Red hat heard the community .. that the community and Red Hat customers want more direct input into RHEL development. And Red Hat is taking action to open up RHEL development in CentOS Stream.
You get to decide who you trust.
Thanks, Johnny hUghes
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On 6/17/20 12:11 PM, Alessandro Baggi wrote:
Hi Johnny, thank you for your and all centos team works.
Many of us know how much work is needed for building new releases and maintaining C6 and C7, plus CentOS Stream and modules (Appstream). This is a huge work for a small team. Again thank you.
For me OL is not an alternative.
As reported in my previous message I'm not worried about how much time is required to build the new (major/minor) release, it will be ready when it will be. My major concern is about the "security update blackout" that take long as the build process.
I would ask to you:
- Why all security fix are stopped when a new release building process is
started? There is a way or possibility to run the two process in parallel?
So .. when a point release happens .. say 7.8 to 7.9 (just an example .. could be 6.10 to 6.11 or 8.1 to 8.2, etc)
Those packages are built against EACH other, one at a time. Once we build the new gcc, new kernel, and new glibc (if they are reqruies) .. then all the OTHER updated packages are built against those new libraries.. they therefore need those NEW shared libraries to run. So the new files have to be released as a set, not individually.
- When a build process is started and a security fix released there is a
way for your team to "suspend" the building process, release security updates (for 6/7.x or 8.1) and resume the builing process? I think that many users (included me) will have less disappointment having security updates instead of receiving a "signal lost" when building process takes its way.
It makes no difference if the update is a bugfix update or a security update. If 500 packages get released at the same time, they have to be built in a specific order in order to match how they were built in RHEL.
We have to build them, one at a time, then individually test them to make sure they LINK against the proper new libraries and not older libraries.
Also any UPDATES released to the new version , after RHEL does the point release (so updates FOR 7.9 after the 7.9 release) need to wait until the 7.9 release is done and tested to be built .. as they were built against RHEL 7.9 and not RHEL 7.8
So, you can't just build items out of order at point release time.
We have to build the 500 packages , in a specific order. We then have to test the packages, and usually rebuild several of them again for bad links, etc.
This is the process that takes time .. testing and getting the proper links to the proper shared libraries.
If we quickly release bad files .. then we have to rebuild them and re-release them with different versions that RHEL has (because they have to replace our previuosly BAD release). That is not good for anyone.
Hopefully this answers your question. <snip>
Il 19/06/20 17:15, Johnny Hughes ha scritto:
On 6/17/20 12:11 PM, Alessandro Baggi wrote:
Hi Johnny, thank you for your and all centos team works.
Many of us know how much work is needed for building new releases and maintaining C6 and C7, plus CentOS Stream and modules (Appstream). This is a huge work for a small team. Again thank you.
For me OL is not an alternative.
As reported in my previous message I'm not worried about how much time is required to build the new (major/minor) release, it will be ready when it will be. My major concern is about the "security update blackout" that take long as the build process.
I would ask to you:
- Why all security fix are stopped when a new release building process is
started? There is a way or possibility to run the two process in parallel?
So .. when a point release happens .. say 7.8 to 7.9 (just an example .. could be 6.10 to 6.11 or 8.1 to 8.2, etc)
Those packages are built against EACH other, one at a time. Once we build the new gcc, new kernel, and new glibc (if they are reqruies) .. then all the OTHER updated packages are built against those new libraries.. they therefore need those NEW shared libraries to run. So the new files have to be released as a set, not individually.
- When a build process is started and a security fix released there is a
way for your team to "suspend" the building process, release security updates (for 6/7.x or 8.1) and resume the builing process? I think that many users (included me) will have less disappointment having security updates instead of receiving a "signal lost" when building process takes its way.
It makes no difference if the update is a bugfix update or a security update. If 500 packages get released at the same time, they have to be built in a specific order in order to match how they were built in RHEL.
We have to build them, one at a time, then individually test them to make sure they LINK against the proper new libraries and not older libraries.
Also any UPDATES released to the new version , after RHEL does the point release (so updates FOR 7.9 after the 7.9 release) need to wait until the 7.9 release is done and tested to be built .. as they were built against RHEL 7.9 and not RHEL 7.8
So, you can't just build items out of order at point release time.
We have to build the 500 packages , in a specific order. We then have to test the packages, and usually rebuild several of them again for bad links, etc.
This is the process that takes time .. testing and getting the proper links to the proper shared libraries.
If we quickly release bad files .. then we have to rebuild them and re-release them with different versions that RHEL has (because they have to replace our previuosly BAD release). That is not good for anyone.
Hopefully this answers your question.
Hi Johnny, thank you for your answer. This is more clear to me now.
Alessandro.
Hi Johnny,
Am Mi., 17. Juni 2020 um 16:16 Uhr schrieb Johnny Hughes <johnny@centos.org
:
[...] No one is trynig to make anything slower.
This is good to hear but ...
[...] I can assure you .. I am working my butt off everyday to make CentOS Linux the best it can be. If you want to compare what the CentOS team (a small team) can do compared to Oracle (a tmulti billin dollar corporation who bought Sun Microsystems .. took over Java and Open Office, etc) .. well, we can not provide the resources they can provide. [...]
... then I'm missing some kind of strategy on how to improve things (or maybe I simply don't know they exist). Without having a detailed background about the project setup and the way the small team works, I would at a first glance assume that there are two options to improve. One would be to make the team bigger (as you said, it's open and I'm pretty sure there are enough volunteers) or a second one, by increasing the automation level for the CentOS production.
What do you think and are there discussions in the core CentOS community to go either one of these routes?
Regards Thomas
On 6/18/20 11:12 AM, Thomas Bendler wrote:
Hi Johnny,
Am Mi., 17. Juni 2020 um 16:16 Uhr schrieb Johnny Hughes <johnny@centos.org
:
[...] No one is trynig to make anything slower.
This is good to hear but ...
[...] I can assure you .. I am working my butt off everyday to make CentOS Linux the best it can be. If you want to compare what the CentOS team (a small team) can do compared to Oracle (a tmulti billin dollar corporation who bought Sun Microsystems .. took over Java and Open Office, etc) .. well, we can not provide the resources they can provide. [...]
... then I'm missing some kind of strategy on how to improve things (or maybe I simply don't know they exist). Without having a detailed background about the project setup and the way the small team works, I would at a first glance assume that there are two options to improve. One would be to make the team bigger (as you said, it's open and I'm pretty sure there are enough volunteers) or a second one, by increasing the automation level for the CentOS production.
What do you think and are there discussions in the core CentOS community to go either one of these routes?
While both of those would be helpful .. neither will really SOLVE the issue. This is an iterative process. You build something, test it, rebuild all the bad parts, test it again .. rinse, repeat
We usually have the first build 3 or so days after they release.
Many things build out of order. They get tested and rebuilt.
Regards Thomas
On 6/19/20 10:18 AM, Johnny Hughes wrote:
On 6/18/20 11:12 AM, Thomas Bendler wrote:
Hi Johnny,
Am Mi., 17. Juni 2020 um 16:16 Uhr schrieb Johnny Hughes <johnny@centos.org
:
[...] No one is trynig to make anything slower.
This is good to hear but ...
[...] I can assure you .. I am working my butt off everyday to make CentOS Linux the best it can be. If you want to compare what the CentOS team (a small team) can do compared to Oracle (a tmulti billin dollar corporation who bought Sun Microsystems .. took over Java and Open Office, etc) .. well, we can not provide the resources they can provide. [...]
... then I'm missing some kind of strategy on how to improve things (or maybe I simply don't know they exist). Without having a detailed background about the project setup and the way the small team works, I would at a first glance assume that there are two options to improve. One would be to make the team bigger (as you said, it's open and I'm pretty sure there are enough volunteers) or a second one, by increasing the automation level for the CentOS production.
What do you think and are there discussions in the core CentOS community to go either one of these routes?
While both of those would be helpful .. neither will really SOLVE the issue. This is an iterative process. You build something, test it, rebuild all the bad parts, test it again .. rinse, repeat
We usually have the first build 3 or so days after they release.
Many things build out of order. They get tested and rebuilt.
How is this going to be fixed .. Welcome to CentOS Stream
Stream will be , once it is fully implemented, the ACTUAL development of RHEL the 'next point release' on git.centos.org in the open.
It will be a rolling distro that is GOING to be the Source Code used for next RHEL point release.
Therefore, we will have all package as they are being worked on by the RHEL Engineers .. and you can see it happen in progress. You can also use it however you want. There will be no delay i this at all. It will be constantly moving. There will be no 500 pacakges drop or delays.
Le 19/06/2020 à 17:29, Johnny Hughes a écrit :
How is this going to be fixed .. Welcome to CentOS Stream
Stream will be , once it is fully implemented, the ACTUAL development of RHEL the 'next point release' on git.centos.org in the open.
It will be a rolling distro that is GOING to be the Source Code used for next RHEL point release.
Therefore, we will have all package as they are being worked on by the RHEL Engineers .. and you can see it happen in progress. You can also use it however you want. There will be no delay i this at all. It will be constantly moving. There will be no 500 pacakges drop or delays.
Reading this, I just want to say a big warm THANK YOU for churning out one of the finest Linux distributions.
Keep up the good work.
Cheers from the sunny South of France,
Niki
On 20/06/20 3:29 am, Johnny Hughes wrote:
How is this going to be fixed .. Welcome to CentOS Stream
Stream will be , once it is fully implemented, the ACTUAL development of RHEL the 'next point release' on git.centos.org in the open.
So basically stream is a testing ground for RHEL. It's not actually a rebuild of RHEL since it's what comes *before* RHEL, not after.
It will be a rolling distro that is GOING to be the Source Code used for next RHEL point release.
Therefore, we will have all package as they are being worked on by the RHEL Engineers .. and you can see it happen in progress. You can also use it however you want. There will be no delay i this at all. It will be constantly moving. There will be no 500 pacakges drop or delays.
This is all well and good, but I don't think that CentOS was ever meant to be a testing ground for RHEL. As the name actually stands for it is a "Community Enterprise OS" and it has always been a rebuild of the RHEL sources. Stream is basically RHEL Rolling Beta, and that can hardly be considered "Enterprise".
I and I think many others find this focus on Stream to be rather distressing, and it does have the appearance to be taking focus away from the core OS. This is further evidenced by the long wait times for release.
The way I see it, Red Hat pays the bills now, Red Hat employs the core team, and Red Hat wants a RHEL Beta platform, so that is what they have decreed that CentOS will become. Now I could be wrong here because I certainly don't have any inside information about this, but it seems from teh outside looking in that any progress on the core OS is incidental and time spent on it has to be time leftover after any work is done on Stream.
Now I don't have an issue with Stream, in fact I think taht Stream can be beneficial to CentOS, but it hsould not be at the expense of the core OS, imo. The core OS should take priority over any other CentOS project, whether it be streams, or SIGs or anything else, because we can't really have a Community Enterprise OS without the core OS.
Peter
On Sat, Jun 20, 2020, 5:41 AM Peter peter@pajamian.dhs.org wrote:
On 20/06/20 3:29 am, Johnny Hughes wrote:
How is this going to be fixed .. Welcome to CentOS Stream
Stream will be , once it is fully implemented, the ACTUAL development of RHEL the 'next point release' on git.centos.org in the open.
So basically stream is a testing ground for RHEL. It's not actually a rebuild of RHEL since it's what comes *before* RHEL, not after.
It will be a rolling distro that is GOING to be the Source Code used for next RHEL point release.
Therefore, we will have all package as they are being worked on by the RHEL Engineers .. and you can see it happen in progress. You can also use it however you want. There will be no delay i this at all. It will be constantly moving. There will be no 500 pacakges drop or delays.
This is all well and good, but I don't think that CentOS was ever meant to be a testing ground for RHEL. As the name actually stands for it is a "Community Enterprise OS" and it has always been a rebuild of the RHEL sources. Stream is basically RHEL Rolling Beta, and that can hardly be considered "Enterprise".
I and I think many others find this focus on Stream to be rather distressing, and it does have the appearance to be taking focus away from the core OS. This is further evidenced by the long wait times for release.
The way I see it, Red Hat pays the bills now, Red Hat employs the core team, and Red Hat wants a RHEL Beta platform, so that is what they have decreed that CentOS will become. Now I could be wrong here because I certainly don't have any inside information about this, but it seems from teh outside looking in that any progress on the core OS is incidental and time spent on it has to be time leftover after any work is done on Stream.
Now I don't have an issue with Stream, in fact I think taht Stream can be beneficial to CentOS, but it hsould not be at the expense of the core OS, imo. The core OS should take priority over any other CentOS project, whether it be streams, or SIGs or anything else, because we can't really have a Community Enterprise OS without the core OS.
Peter _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
+1 Streams is not for a production workload, if I wanted that I can easily deploy an Arch instance if I want or need a rolling distro (it's not Redhat etc but still). If Redhat wanted CentOS to be released near the same time line they could help make that happen, although that wouldn't be in there best financial interest.
Now maybe there will be a way to set streams up to only get security updates and then when they release the .1 release you could update and have everything update. If something like that could be worked out that would work for me but I would only want security updates in between and I'm not sure if that is possible.
On Sat, Jun 20, 2020 at 4:08 AM Tom Bishop bishoptf@gmail.com wrote:
+1 Streams is not for a production workload, if I wanted that I can easily deploy an Arch instance if I want or need a rolling distro (it's not Redhat etc but still). If Redhat wanted CentOS to be released near the same time line they could help make that happen, although that wouldn't be in there best financial interest.
think of it this way ... when the rolling beta is done, the final release will be done with no further delay.
On 21/06/20 9:15 am, John Pierce wrote:
On Sat, Jun 20, 2020 at 4:08 AM Tom Bishop bishoptf@gmail.com wrote:
+1 Streams is not for a production workload, if I wanted that I can easily deploy an Arch instance if I want or need a rolling distro (it's not Redhat etc but still). If Redhat wanted CentOS to be released near the same time line they could help make that happen, although that wouldn't be in there best financial interest.
think of it this way ... when the rolling beta is done, the final release will be done with no further delay.
No, Stream and the core OS are built form a completely separate set of sources. I highly doubt that we could use binaries built from stream to populate the final release. The whole thing has to be rebuilt for the final release regardless of the status of Stream.
Peter
but the build process should be the same, no? I can't believe RH would use a completely different build process for the release than for the beta/development stuff.
On Sat, Jun 20, 2020 at 5:36 PM Peter peter@pajamian.dhs.org wrote:
On 21/06/20 9:15 am, John Pierce wrote:
On Sat, Jun 20, 2020 at 4:08 AM Tom Bishop bishoptf@gmail.com wrote:
+1 Streams is not for a production workload, if I wanted that I can
easily
deploy an Arch instance if I want or need a rolling distro (it's not
Redhat
etc but still). If Redhat wanted CentOS to be released near the same
time
line they could help make that happen, although that wouldn't be in
there
best financial interest.
think of it this way ... when the rolling beta is done, the final release will be done with no further delay.
No, Stream and the core OS are built form a completely separate set of sources. I highly doubt that we could use binaries built from stream to populate the final release. The whole thing has to be rebuilt for the final release regardless of the status of Stream.
Peter _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On 21/06/20 1:23 pm, John Pierce wrote:
but the build process should be the same, no? I can't believe RH would use a completely different build process for the release than for the beta/development stuff.
The packages still have to be built as a whole, they need to go through QA testing, isos need to be built and tested. The only thing that I can think of that Stream benefits this process is to help Red Hat find the odd bug here and there before their final release (after which CentOS still has to do everything listed above).
Peter
Il 21/06/20 03:28, Peter ha scritto:
On 21/06/20 1:23 pm, John Pierce wrote:
but the build process should be the same, no? I can't believe RH would use a completely different build process for the release than for the beta/development stuff.
The packages still have to be built as a whole, they need to go through QA testing, isos need to be built and tested. The only thing that I can think of that Stream benefits this process is to help Red Hat find the odd bug here and there before their final release (after which CentOS still has to do everything listed above).
Peter _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
I have a question. Why CentOS Stream is needed and what problem it solves?
On 21/06/20 1:23 pm, John Pierce wrote:
but the build process should be the same, no? I can't believe RH would use a completely different build process for the release than for the beta/development stuff.
The packages still have to be built as a whole, they need to go through QA testing, isos need to be built and tested. The only thing that I can think of that Stream benefits this process is to help Red Hat find the odd bug here and there before their final release (after which CentOS still has to do everything listed above).
As I understand it the whole full build and QA and whatever may still be done again. The big difference is that the whole work of how to build and setting up the build infrastructure has already been done and is known and tested. So the complete build is going quite fast and the big delays are a thing of the past.
If it's going to be like that it sounds very good.
Regards, Simon
On Sun, Jun 21, 2020 at 2:01 AM Simon Matter via CentOS centos@centos.org wrote:
On 21/06/20 1:23 pm, John Pierce wrote:
but the build process should be the same, no? I can't believe RH would use a completely different build process for the release than for the beta/development stuff.
The packages still have to be built as a whole, they need to go through QA testing, isos need to be built and tested. The only thing that I can think of that Stream benefits this process is to help Red Hat find the odd bug here and there before their final release (after which CentOS still has to do everything listed above).
As I understand it the whole full build and QA and whatever may still be done again. The big difference is that the whole work of how to build and setting up the build infrastructure has already been done and is known and tested. So the complete build is going quite fast and the big delays are a thing of the past.
If it's going to be like that it sounds very good.
exactly, that was my point. I remember 8.0 was very delayed by how much harder and different the build process was.
On 22/06/20 7:03 am, John Pierce wrote:
On Sun, Jun 21, 2020 at 2:01 AM Simon Matter via CentOS centos@centos.org wrote:
On 21/06/20 1:23 pm, John Pierce wrote:
but the build process should be the same, no? I can't believe RH would use a completely different build process for the release than for the beta/development stuff.
The packages still have to be built as a whole, they need to go through QA testing, isos need to be built and tested. The only thing that I can think of that Stream benefits this process is to help Red Hat find the odd bug here and there before their final release (after which CentOS still has to do everything listed above).
As I understand it the whole full build and QA and whatever may still be done again. The big difference is that the whole work of how to build and setting up the build infrastructure has already been done and is known and tested. So the complete build is going quite fast and the big delays are a thing of the past.
If it's going to be like that it sounds very good.
exactly, that was my point. I remember 8.0 was very delayed by how much harder and different the build process was.
That work was completed in the build of 8.0 and to a smaller extent 8.1. Stream doesn't really add anything here.
Peter
On Sun, 21 Jun 2020 at 15:10, Peter peter@pajamian.dhs.org wrote:
On 22/06/20 7:03 am, John Pierce wrote:
On Sun, Jun 21, 2020 at 2:01 AM Simon Matter via CentOS centos@centos.org wrote:
exactly, that was my point. I remember 8.0 was very delayed by how much harder and different the build process was.
That work was completed in the build of 8.0 and to a smaller extent 8.1. Stream doesn't really add anything here.
There are 2 sets of work. 1. There is the work on the tools which were slapped together as an emergency from parts before 8.0. Those mbboxx tools are getting a rewrite and upgrade currently by the CPE team to make them more useful in the future. Stream only helps in that it is the excuse for that work to be done versus it molding and falling apart right after every 8.x release comes out. 2. There is the work that happens because various things are rebased and you need to figure out the HTF you get from build A to build A+1 by rebuilding N packages. That is work that Stream should help on because this is then knowledge is being done in stream before hand. If you know that package A went to A+1 then to A+2 and then back to A+1 but you learned how to do the second A+1 from a flag you used with A+2, then the amount of time reinventing the wheel is shortened.
Peter _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On 22/06/20 10:13 am, Stephen John Smoogen wrote:
There are 2 sets of work.
- There is the work on the tools which were slapped together as an
emergency from parts before 8.0. Those mbboxx tools are getting a rewrite and upgrade currently by the CPE team to make them more useful in the future. Stream only helps in that it is the excuse for that work to be done versus it molding and falling apart right after every 8.x release comes out.
I didn't know that a rewrite is still needed on the current tool set and granted Stream can help with this, but I hardly think that it's necessary and the tool set can always be tested against the current release (8.2) from git.
- There is the work that happens because various things are rebased
and you need to figure out the HTF you get from build A to build A+1 by rebuilding N packages. That is work that Stream should help on because this is then knowledge is being done in stream before hand. If you know that package A went to A+1 then to A+2 and then back to A+1 but you learned how to do the second A+1 from a flag you used with A+2, then the amount of time reinventing the wheel is shortened.
This I do realize and it's the one exception I considered where Stream might come in handy, but not handy enough to justify its existence, imo. Usually in a new point release there might be a small handful of packages that need re-basing, out of those the number of packages that would need to have the spec file tweaked to build them would be minimal (at a complete guess three or less) and out of those the number that would require a change to the tool set would likely average out to be less than one per point release. In a worst-case scenario it might save a day or two on a particularly nasty point release, and this would easily be recouped in the amount of time it would save if the CentOS team did not have to maintain Stream at all.
Now these are just semi-educated guesses and I don't have the experience to justify this so I'm happy to consider real numbers that prove me wrong.
Peter
On Mon, Jun 22, 2020 at 6:43 AM Peter peter@pajamian.dhs.org wrote:
On 22/06/20 10:13 am, Stephen John Smoogen wrote:
There are 2 sets of work.
- There is the work on the tools which were slapped together as an
emergency from parts before 8.0. Those mbboxx tools are getting a rewrite and upgrade currently by the CPE team to make them more useful in the future. Stream only helps in that it is the excuse for that work to be done versus it molding and falling apart right after every 8.x release comes out.
I didn't know that a rewrite is still needed on the current tool set and granted Stream can help with this, but I hardly think that it's necessary and the tool set can always be tested against the current release (8.2) from git.
- There is the work that happens because various things are rebased
and you need to figure out the HTF you get from build A to build A+1 by rebuilding N packages. That is work that Stream should help on because this is then knowledge is being done in stream before hand. If you know that package A went to A+1 then to A+2 and then back to A+1 but you learned how to do the second A+1 from a flag you used with A+2, then the amount of time reinventing the wheel is shortened.
This I do realize and it's the one exception I considered where Stream might come in handy, but not handy enough to justify its existence, imo. Usually in a new point release there might be a small handful of packages that need re-basing, out of those the number of packages that would need to have the spec file tweaked to build them would be minimal (at a complete guess three or less) and out of those the number that would require a change to the tool set would likely average out to be less than one per point release. In a worst-case scenario it might save a day or two on a particularly nasty point release, and this would easily be recouped in the amount of time it would save if the CentOS team did not have to maintain Stream at all.
Now these are just semi-educated guesses and I don't have the experience to justify this so I'm happy to consider real numbers that prove me wrong.
Hi,
Now I have RHEL 8 installed for my test machine and some test Virtual Machines.
I then subscribed to the RHSA-announce mailing list.
Now I wonder why a particular package has not been released for CentOS 8 while it has been some time on RHEL OS and mailing list.
With CentOS 7, I had no RHEL developer access, so I never wondered why a particular update has not been released CentOS 7
I just used CentOS 7 and was happy.
Is this a valid reason for my impatience?
- Lee
On Sat, Jun 20, 2020 at 5:41 AM Peter peter@pajamian.dhs.org wrote:
This is all well and good, but I don't think that CentOS was ever meant to be a testing ground for RHEL. As the name actually stands for it is a "Community Enterprise OS" and it has always been a rebuild of the RHEL sources. Stream is basically RHEL Rolling Beta, and that can hardly be considered "Enterprise".
I and I think many others find this focus on Stream to be rather distressing, and it does have the appearance to be taking focus away from the core OS. This is further evidenced by the long wait times for release.
I have to shake my head at this. You're telling the guy who builds, and had built, CentOS for years what the project is and where it is going? He told you what Stream is and what it will be used for already. It is the new future of the OS. That's it. You're arguing semantics about having Enterprise in the name, while ignoring that it's still Enterprise Linux, even if it's not the final end product because it is the future source project of Enterprise Linux.
Am 17.06.20 um 09:16 schrieb Nicolas Kovacs:
Hi,
I just read this blog article from austrian Linux expert Michael Kofler. For those among you who don't know the guy, he's my home country's number one Linux expert (known as "der Kofler") and most notably the author of a series of excellent books about Linux over the last 25 years.
https://kofler.info/centos-8-wertlose-langzeitunterstuetzung/
Disclaimer : I've been a CentOS user (and fan) since 4.x, I'm using it on all my servers, and yes, I know the difference between upstream RHEL and CentOS.
The article is in german, but the statistics graph is eloquent enough for the non-german-speaking users. It focuses on updates for CentOS 8, and more exactly the extended periods of time where there have been no updates available.
The author's theory ("unspoken truth"): while it's a positive thing that Red Hat is sponsoring CentOS, the amount of sponsoring is just insufficient enough so that the product is "starved to death" by Red Hat (e. g. IBM) to encourage users to move to RHEL.
The author's conclusion is quite severe: in the current state of things, CentOS 8 is not recommendable for production as updates are lagging too much behind. While CentOS 7 may be usable, CentOS 8 has been "degraded to teaching and testing purposes".
Still according to Mister Kofler, this "sorry state of things" will probably encourage users to move to Oracle Linux, the other big RHEL clone.
After some hesitation, I decided to share this on the mailing list. Since this raises some concerns, I'd be curious to have your take on this.
Site note: Despite the implications of update delays. The numbers provided by Michael are not comparable. The needed effort between every major release to build the OS is different. So, the numbers should be normalized.
Quote from https://wiki.centos.org/About/Building_8:
"The differences ... has changed drastically, the repository format has added 'modules' and RPMS have grown many features that EL7 and before do not have."
I wonder about the authors conclusion; the fact that RHEL is the choice for critical applications (what ever critical is) is known since the early days. This applies randomly to C5.11, C4.9 or C8.2.2004.
So - cold soup get cooked again :-)
-- Leon
On 6/17/20 10:19 AM, Leon Fauster via CentOS wrote:
... I wonder about the authors conclusion; the fact that RHEL is the choice for critical applications (what ever critical is) is known since the early days. This applies randomly to C5.11, C4.9 or C8.2.2004.
So - cold soup get cooked again :-)
Indeed. The author's conclusion has been the case since White Box Enterprise Linux was a thing. Anyone and everyone can get the sources from git.centos.org as soon as they are released and build the stuff themselves if they think it can be done faster; that's how WBEL got started, as a one-user project that just happened to be publicly released. Building from source has never really been any easier; the lack of .src.rpms is not an impediment to just getting something built. But the CentOS value-add is that those rebuilt sources have been tested for binary compatibility and are from a trusted source. A one-person project like WBEL would have a much more difficult time today, with modularity especially. Build times for these packages is not zero.
On Wed, Jun 17, 2020 at 12:09:07PM -0400, Lamar Owen wrote:
On 6/17/20 10:19 AM, Leon Fauster via CentOS wrote:
... I wonder about the authors conclusion; the fact that RHEL is the choice for critical applications (what ever critical is) is known since the early days. This applies randomly to C5.11, C4.9 or C8.2.2004.
So - cold soup get cooked again :-)
Indeed. The author's conclusion has been the case since White Box Enterprise Linux was a thing. Anyone and everyone can get the sources from git.centos.org as soon as they are released and build the stuff themselves if they think it can be done faster; that's how WBEL got started, as a one-user project that just happened to be publicly released. Building from source has never really been any easier; the lack of .src.rpms is not an impediment to just getting something built. But the CentOS value-add is that those rebuilt sources have been tested for binary compatibility and are from a trusted source. A one-person project like WBEL would have a much more difficult time today, with modularity especially. Build times for these packages is not zero.
And there was also Tao Linux, which I was using back in the day. another one-person project, which that one person did a good job with, but when time came to change jobs he had to let the project go. He had the foresight and grace to work with Johnny Hughes to define a way to transition to CentOS, which worked fine, so here I am still in the CentOS camp.
And my thanks to that original developer (whose name I cannot dredge out of my back-brain right now), to Johnny, and all the other folks who make CentOS great!
Fred
Hi,
I am the author of said blog article.
FIRST: It was never my intention to criticize the CentOS team. I appreciate the hard work you are doing. If my blog text (which is in German langugage) gave a wrong impression, I apologize.
SECOND: I LOVE CentOS. Otherwise it would not matter to me. I use CentOS to teach Linux administration at university, I promote CentOS in my books and I use it personally on some servers.
THIRD: It is a fact that the update gaps for CentOS 8 are currently too long for productive use. Basically, that's why I now warn against using CentOS 8 on live systems.
---
One might argue, CentOS was never intended for productive use. Perhaps I misunderstood this. And with me all administrators of some million web servers running on CentOS. Hm. Time to rethink?
The way I see it, there is a need for free Linux systems. No support, sure, but updates. In the past (and for CentOS 7, still), I considered CentOS as 'good enough' for many purposes. Not for the Bank of England, they can affort whatever they like. But for a school. For a small company needing a plain web and mail server. Etc.
The CentOS webpage says: 'CentOS Linux ... suits a wide variety of deployments.' Currently, I really fail to see a wide range of possible deployments.
Sure, there are other options. Out of my point of view, Ubuntu LTS is one. Debian is. Oracle Linux (free without support) is, too. I am not entirely in love with this company -- but if I had the need to deploy a RHEL 8 compatible system right now, and no budget to pay for RHEL, I would prefer it to CentOS. Sorry about this.
---
I truly believe, Red Hat has the means to make live for the CentOS team easier. Either by simply increasing the team, the infrastructure to build packages faster, whatever. Or by making the clone process easier.
My guess is, they don't want. And this is OK -- who am I be to advice a multi billion dollar company? The question is, what does this mean for the future of CentOS? Is CentOS to become an open development platform for Red Hat, but no more?
These are my thoughts.
Best wishes,
Michael
On 2020-06-17 12:38, Michael Kofler wrote:
Hi,
I am the author of said blog article.
FIRST: It was never my intention to criticize the CentOS team. I appreciate the hard work you are doing. If my blog text (which is in German langugage) gave a wrong impression, I apologize.
SECOND: I LOVE CentOS. Otherwise it would not matter to me. I use CentOS to teach Linux administration at university, I promote CentOS in my books and I use it personally on some servers.
THIRD: It is a fact that the update gaps for CentOS 8 are currently too long for productive use. Basically, that's why I now warn against using CentOS 8 on live systems.
One might argue, CentOS was never intended for productive use. Perhaps I misunderstood this. And with me all administrators of some million web servers running on CentOS. Hm. Time to rethink?
Some of us fled servers from Linux (of whichever flavor) to one of BSD descendants. That has more to do with frequent reboots due to glibc or kernel security updates or other unpleasantnesses of Linux. Many of those who did this, still use/support Linux on workstations and laptops.
The way I see it, there is a need for free Linux systems. No support, sure, but updates. In the past (and for CentOS 7, still), I considered CentOS as 'good enough' for many purposes. Not for the Bank of England, they can affort whatever they like. But for a school. For a small company needing a plain web and mail server. Etc.
The CentOS webpage says: 'CentOS Linux ... suits a wide variety of deployments.' Currently, I really fail to see a wide range of possible deployments.
Sure, there are other options. Out of my point of view, Ubuntu LTS is one. Debian is. Oracle Linux (free without support) is, too. I am not entirely in love with this company -- but if I had the need to deploy a RHEL 8 compatible system right now, and no budget to pay for RHEL, I would prefer it to CentOS. Sorry about this.
I truly believe, Red Hat has the means to make live for the CentOS team easier. Either by simply increasing the team, the infrastructure to build packages faster, whatever. Or by making the clone process easier.
RedHat has not and never had any obligation to financially or otherwise support its clones (released free for use). To think they do is delusional. And CentOS project I'm sure never expected that (I am not part of CentOS project, though I greatly appreciate what they do).
Valeri
My guess is, they don't want. And this is OK -- who am I be to advice a multi billion dollar company? The question is, what does this mean for the future of CentOS? Is CentOS to become an open development platform for Red Hat, but no more?
These are my thoughts.
Best wishes,
Michael _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On 6/17/20 10:38 AM, Michael Kofler wrote:
Hi,
I am the author of said blog article.
FIRST: It was never my intention to criticize the CentOS team. I appreciate the hard work you are doing. If my blog text (which is in German langugage) gave a wrong impression, I apologize.
SECOND: I LOVE CentOS. Otherwise it would not matter to me. I use CentOS to teach Linux administration at university, I promote CentOS in my books and I use it personally on some servers.
[snip]
I truly believe, Red Hat has the means to make live for the CentOS team easier. Either by simply increasing the team, the infrastructure to build packages faster, whatever. Or by making the clone process easier.
IMHO this is the crux of the problem. I feel for the CentOS team every time they get beat up by users asking why things take so long, and they're forced to explain over and over again how they have to re-engineer processes that the RHEL team has already engineered.
In theory, both RHEL and CentOS start from the same sources -- git.centos.org -- which is a great thing. But, RHEL obviously has package build infrastructure, release composition, release management, and QA (among other) systems that are requisite steps to building and releasing, say, RHEL 8.2. It makes me sad that the CentOS devs (most of whom are Red Hat employees, as I understand it) are forced to re-implement what the RHEL team has already implemented, without any advice, guidance, or tooling from the RHEL engineering team. (i.e. the CentOS team has to discover that "these packages have to be built in this order, or with this modified build environment", etc. on their own)
It's not clear why 2 different groups at the same company doing the same thing can't combine resources. Why can't one group at Red Hat produce binary RPMs from git.centos.org that find their way into both a RHEL compose and a CentOS compose? And would the composes then be so different if the only thing that varied was the package set and branding?
Perhaps the duplication of engineering effort stems from the history of CentOS being a separate organization that's still undergoing integration with other Red Hat teams. And I'd love to be enlightened if any or all of my assumptions above are wrong; my perspective is just that of a long-time Red Hat Linux, RHEL, Fedora, and CentOS user (since 1998 or so).
-Greg
On 17/06/2020 18:38, Michael Kofler wrote:
Hi,
I am the author of said blog article.
FIRST: It was never my intention to criticize the CentOS team. I appreciate the hard work you are doing. If my blog text (which is in German langugage) gave a wrong impression, I apologize.
SECOND: I LOVE CentOS. Otherwise it would not matter to me. I use CentOS to teach Linux administration at university, I promote CentOS in my books and I use it personally on some servers.
THIRD: It is a fact that the update gaps for CentOS 8 are currently too long for productive use. Basically, that's why I now warn against using CentOS 8 on live systems.
One might argue, CentOS was never intended for productive use. Perhaps I misunderstood this. And with me all administrators of some million web servers running on CentOS. Hm. Time to rethink?
As far as I'm aware that has always been the case. Johnny has never been slow in coming forward and saying if you need updates, or a service level agreement, or support then you should buy RHEL. That is what it is for. If not, then use CentOS for free. But don't use CentOS for free on production servers and then shout or act surprised when you don't have updates on a timescale you consider appropriate.
Nothing has changed in this regard for as long as I've been a CentOS user or been involved in the CentOS community. If you are now having to rethink your approach then you probably either haven't given it sufficient thought in the first place or you originally came to the wrong conclusion.
This is a non-issue. Nothing has changed. Things were exactly the same with CentOS 4, CentOS 5, CentOS 6, CentOS 7, and by it's very nature it will be the same in CentOS 9... The simple matter is it takes time to rebuild a complete OS and there will always be a lag. Either that is acceptable to you and you use it, or you purchase a RHEL license for your publicly facing infrastructure. The only issue here is people's unrealistic expectations, and to be fair the CentOS Project can hardly be accused of falsely raising peoples expectations having consistently stated it will be ready when it's done for at least the last 15 years.
On Jun 17, 2020, at 3:02 PM, Phil Perry pperry@elrepo.org wrote:
Nothing has changed in this regard for as long as I've been a CentOS user or been involved in the CentOS community.
This is the essence of the question, to me. I agree that _in_principle_ nothing has changed, and I don't even see any disagreement with that in the list. However, there is a separate question, and that is whether _in_practice_ the lag between RHEL and CentOS updates has increased with CentOS 8. I don't know what the answer is, because I'm not paying attention since I'm far from adopting CentOS 8, but it's a legitimate (and in fact empirical) question.
Noam
On 17/06/2020 20:06, Noam Bernstein via CentOS wrote:
On Jun 17, 2020, at 3:02 PM, Phil Perry pperry@elrepo.org wrote:
Nothing has changed in this regard for as long as I've been a CentOS user or been involved in the CentOS community.
This is the essence of the question, to me. I agree that _in_principle_ nothing has changed, and I don't even see any disagreement with that in the list. However, there is a separate question, and that is whether _in_practice_ the lag between RHEL and CentOS updates has increased with CentOS 8. I don't know what the answer is, because I'm not paying attention since I'm far from adopting CentOS 8, but it's a legitimate (and in fact empirical) question.
I get what you are saying, but what difference does it make if it has? What does it matter if the lag is 1 week, or 1 month, or more? The only reason it will matter to you is if you are trying to do something with CentOS that is time critical - e.g, publicly facing server that needs security updates, using CentOS on test servers to validate production releases for RHEL, etc. At which point you probably should be using RHEL if it is important to you, not CentOS, and it was a mistake to deploy CentOS in those roles in the first place.
People need to hold their hands up and say, I took a gamble that CentOS would get updates out quick and I wouldn't get hacked in a week, and now updates are taking a lot longer my gamble is no longer paying off and I need to get myself a RHEL subscription or switch to Ubuntu or whatever other flavor you like the taste of. Coming here and complaining when (you) made a bet and lost doesn't achieve anything.
On my home file server for example, which is not connected to the internet, what does it matter if the release is 1 month or 3 months out of date? I can install the server in the knowledge it's going to work, and be supported with updates for 10 years and I can largely forget about it. My el5 box ran for more than 10 years until the hardware eventually died.
On Jun 17, 2020, at 3:32 PM, Phil Perry pperry@elrepo.org wrote:
I get what you are saying, but what difference does it make if it has? What does it matter if the lag is 1 week, or 1 month, or more? The only reason it will matter to you is if you are trying to do something with CentOS that is time critical - e.g, publicly facing server that needs security updates, using CentOS on test servers to validate production releases for RHEL, etc. At which point you probably should be using RHEL if it is important to you, not CentOS, and it was a mistake to deploy CentOS in those roles in the first place.
And yet in practice many of us have found CentOS to be perfectly adequate for such applications in the past, up to and including CentOS 7. If this is no longer true for CentOS 8, for whatever reason, it's useful to know. I'm not saying RHEL doesn't have its place - just that perhaps the boundary in the range of applicability between it and CentOS has therefore also changed.
Noam
Am 17.06.20 um 21:37 schrieb Noam Bernstein via CentOS:
On Jun 17, 2020, at 3:32 PM, Phil Perry pperry@elrepo.org wrote:
I get what you are saying, but what difference does it make if it has? What does it matter if the lag is 1 week, or 1 month, or more? The only reason it will matter to you is if you are trying to do something with CentOS that is time critical - e.g, publicly facing server that needs security updates, using CentOS on test servers to validate production releases for RHEL, etc. At which point you probably should be using RHEL if it is important to you, not CentOS, and it was a mistake to deploy CentOS in those roles in the first place.
And yet in practice many of us have found CentOS to be perfectly adequate for such applications in the past, up to and including CentOS 7. If this is no longer true for CentOS 8, for whatever reason, it's useful to know. I'm not saying RHEL doesn't have its place - just that perhaps the boundary in the range of applicability between it and CentOS has therefore also changed.
The answer is not inherently in the distribution itself. Make your analysis about your needs an requirements and the choice is then yours.
One could argue that the gap between disclosure of one security issues and the update via RHEL subscription is to big. Then a contract with the upstream developer of the corresponding software component is a better choice then relying in RHEL, right?
-- Leon
On Jun 17, 2020, at 3:46 PM, Leon Fauster via CentOS centos@centos.org wrote:
The answer is not inherently in the distribution itself. Make your analysis about your needs an requirements and the choice is then yours.
One could argue that the gap between disclosure of one security issues and the update via RHEL subscription is to big. Then a contract with the upstream developer of the corresponding software component is a better choice then relying in RHEL, right?
Of course. My only question is whether the observation that the gap for CentOS 8 is indeed larger than we have come to be used to for CentOS 7. I'm certainly not, and I don't think anyone is, claiming that the CentOS teams owes us any particular response time. I just want to know if the claim that it's systematically significantly longer for 8 than 7 is in fact (empirically) true.
Noam
Once upon a time, Noam Bernstein noam.bernstein@nrl.navy.mil said:
Of course. My only question is whether the observation that the gap for CentOS 8 is indeed larger than we have come to be used to for CentOS 7.
So, I took a look... and the answer is "it's not" (with a small sample set). I took dates from Wikipedia for RHEL and the archived release notes for CentOS. I didn't bother with the .0 releases (since that's a lot of new work anyway). Right now, CentOS 8 is far faster than CentOS 7 and 6 were at this stage.
release RHEL date CentOS date days 6.1 2011-05-19 2011-12-12 207 6.2 2011-12-06 2012-07-24 231 6.3 2012-05-20 2012-09-30 133 6.4 2013-02-21 2013-05-21 89 6.5 2013-11-21 2014-02-26 97 6.6 2014-10-13 2014-11-15 33 6.7 2015-07-22 2015-09-05 45 6.8 2016-05-10 2016-07-28 79 6.9 2017-03-21 2017-04-05 15 6.10 2018-06-19 2018-07-03 14
7.1 2015-03-05 2015-10-11 220 7.2 2015-11-19 2016-02-19 92 7.3 2016-11-03 2016-12-21 48 7.4 2017-08-01 2018-03-21 232 7.5 2018-04-10 2018-10-30 203 7.6 2018-10-30 2019-01-28 90 7.7 2019-08-06 (didn't find release notes) 7.8 2020-03-31 2020-04-27 27
8.1 2019-11-05 2020-01-15 71 8.2 2020-04-28 2020-06-15 48
On Wed, Jun 17, 2020 at 1:53 PM Chris Adams linux@cmadams.net wrote:
Once upon a time, Noam Bernstein noam.bernstein@nrl.navy.mil said:
Of course. My only question is whether the observation that the gap
for CentOS 8 is indeed larger than we have come to be used to for CentOS 7.
So, I took a look... and the answer is "it's not" (with a small sample set). I took dates from Wikipedia for RHEL and the archived release notes for CentOS. I didn't bother with the .0 releases (since that's a lot of new work anyway). Right now, CentOS 8 is far faster than CentOS 7 and 6 were at this stage.
release RHEL date CentOS date days 6.1 2011-05-19 2011-12-12 207 6.2 2011-12-06 2012-07-24 231 6.3 2012-05-20 2012-09-30 133 6.4 2013-02-21 2013-05-21 89 6.5 2013-11-21 2014-02-26 97 6.6 2014-10-13 2014-11-15 33 6.7 2015-07-22 2015-09-05 45 6.8 2016-05-10 2016-07-28 79 6.9 2017-03-21 2017-04-05 15 6.10 2018-06-19 2018-07-03 14
7.1 2015-03-05 2015-10-11 220 7.2 2015-11-19 2016-02-19 92 7.3 2016-11-03 2016-12-21 48 7.4 2017-08-01 2018-03-21 232 7.5 2018-04-10 2018-10-30 203 7.6 2018-10-30 2019-01-28 90 7.7 2019-08-06 (didn't find release notes) 7.8 2020-03-31 2020-04-27 27
8.1 2019-11-05 2020-01-15 71 8.2 2020-04-28 2020-06-15 48
7.7 2019-08-06 2019-09-17 42
On Jun 17, 2020, at 4:53 PM, Chris Adams linux@cmadams.net wrote:
Once upon a time, Noam Bernstein noam.bernstein@nrl.navy.mil said:
Of course. My only question is whether the observation that the gap for CentOS 8 is indeed larger than we have come to be used to for CentOS 7.
So, I took a look... and the answer is "it's not" (with a small sample set). I took dates from Wikipedia for RHEL and the archived release notes for CentOS. I didn't bother with the .0 releases (since that's a lot of new work anyway). Right now, CentOS 8 is far faster than CentOS 7 and 6 were at this stage.
Did you look at the German blog that started this discussion? I don't know what determines the archived release notes dates, but I just picked the longest delay, CentOS 7.4. You list:
release RHEL date CentOS date days 7.4 2017-08-01 2018-03-21 232
which is indeed the "last updated" dated on the archived notes. However, the German blog post that started this thread lists the much earlier 2017-09-13 for CentOS, 43 days. On the mailing list this message https://lists.centos.org/pipermail/centos-announce/2017-September/022532.htm... https://lists.centos.org/pipermail/centos-announce/2017-September/022532.htmlappears to confirm the earlier date.
I'm not sure the difference shown in that blog (assuming the other dates are also correct) is really quite so dramatic as to justify the conclusion that CentOS 8 is now too slow in getting updates for a substantial number of situations where the CentOS 7 lag was acceptable, but it's apparently not faster.
Noam
On 6/17/20 3:53 PM, Chris Adams wrote:
Once upon a time, Noam Bernstein noam.bernstein@nrl.navy.mil said:
Of course. My only question is whether the observation that the gap for CentOS 8 is indeed larger than we have come to be used to for CentOS 7.
So, I took a look... and the answer is "it's not" (with a small sample set). I took dates from Wikipedia for RHEL and the archived release notes for CentOS. I didn't bother with the .0 releases (since that's a lot of new work anyway). Right now, CentOS 8 is far faster than CentOS 7 and 6 were at this stage.
release RHEL date CentOS date days 6.1 2011-05-19 2011-12-12 207 6.2 2011-12-06 2012-07-24 231 6.3 2012-05-20 2012-09-30 133 6.4 2013-02-21 2013-05-21 89 6.5 2013-11-21 2014-02-26 97 6.6 2014-10-13 2014-11-15 33 6.7 2015-07-22 2015-09-05 45 6.8 2016-05-10 2016-07-28 79 6.9 2017-03-21 2017-04-05 15 6.10 2018-06-19 2018-07-03 14
7.1 2015-03-05 2015-10-11 220 7.2 2015-11-19 2016-02-19 92 7.3 2016-11-03 2016-12-21 48 7.4 2017-08-01 2018-03-21 232 7.5 2018-04-10 2018-10-30 203 7.6 2018-10-30 2019-01-28 90 7.7 2019-08-06 (didn't find release notes) 7.8 2020-03-31 2020-04-27 27
8.1 2019-11-05 2020-01-15 71 8.2 2020-04-28 2020-06-15 48
Your dates are significantly off Wikipedia has a delay listed in a table:
It is, for CentOS-7, For example:
7.0 27 7.1 26 7.2 25 7.3 39 7.4 43 7.5 31 7.6 34 7.7 42 7.8 28
For 6 .. since 6.2, it has bee3n between 10 and 18 days.
For 8:
8.0 140 8.1 71 8.2 48
And EL8 is exponentially harder with an entirely new build system and the requirement to build modules.
On 20/06/20 3:50 am, Johnny Hughes wrote:
Your dates are significantly off Wikipedia has a delay listed in a table:
It is, for CentOS-7, For example:
7.0 27 7.1 26 7.2 25 7.3 39 7.4 43 7.5 31 7.6 34 7.7 42 7.8 28
For 6 .. since 6.2, it has bee3n between 10 and 18 days.
For 8:
8.0 140 8.1 71 8.2 48
So the delays for 8 are significantly longer than they ever were for 7.
And EL8 is exponentially harder with an entirely new build system and the requirement to build modules.
But it seems like every major release has had reasons to be exponentially harder than the last. With 7 it was the shift to using the git sources instead of the SRPMS that were previously provided by Red Hat, thereby necessitating an entirely new build system, plus the change to systemd and the introduction of a new point release numbering scheme, not to mention the move to entirely new infrastructure because of the change to Red Hat sponsorship. So given those I find it hard to believe that the changes in 8 are so much different as to have had such longer delays than 7.
I'd also like to point out something else, from: https://wiki.centos.org/About/Building_8.x#Current_Timeline_8.2.2004
It would appear that the package build was completed on the 4th of May, why didn't we get a CR repo dump this time around so that CentOS users wouldn't have to wait another month before getting critical updates?
Peter
On 20/06/20 10:50 pm, Peter wrote:
On 20/06/20 3:50 am, Johnny Hughes wrote:
8.0 140 8.1 71 8.2 48
So the delays for 8 are significantly longer than they ever were for 7.
I should also say that the time lag for each point release of 8 seems to be dropping exponentially. Hopefully this means that 8.3 will have a significantly shorter delay and we will be in the range of less than 30 days for that, if so I can accept the delays up to this point.
Peter
On 6/20/20 6:50 AM, Peter wrote:
On 20/06/20 3:50 am, Johnny Hughes wrote:
And EL8 is exponentially harder with an entirely new build system and the requirement to build modules.
But it seems like every major release has had reasons to be exponentially harder than the last. With 7 it was the shift to using the git sources instead of the SRPMS that were previously provided by Red Hat, thereby necessitating an entirely new build system, plus the change to systemd and the introduction of a new point release numbering scheme, not to mention the move to entirely new infrastructure because of the change to Red Hat sponsorship.
Using git to build sources from, while a change, doesn't significantly increase the number of packages that need building. The actual package building - compilation - stage can take days on even the fastest hardware. Yes, the build infrastructure is different, and yes there is a learning curve involved, but I have found that the mechanics of building the source is not really that different from doing it with source RPMs. Instead of 'wget src.rpm; rpmbuild --rebuild src.rpm' you do a 'git clone $rpm-package; git checkout $tag; get_sources.sh; rpmbuild -ba $tree' for a full manual build; I've now done this a few times, and it's not really difficult, just a few more keys to press. What can't now be used is the %{BUILDTIME} embedded in the source RPM that can give you clues as to build order; a git commit of a thousand packages can be atomic and all at the same time.
The use of systemd has nothing at all to do with the build difficulty; it's just another package for that purpose; likewise, the new release numbering has nothing to do with build and QA difficulty.
... It would appear that the package build was completed on the 4th of May, why didn't we get a CR repo dump this time around so that CentOS users wouldn't have to wait another month before getting critical updates?
CentOS users who can't wait for CentOS to release 'critical' updates really should reconsider using something else; that might be RHEL, that might be something completely different. This is part of the calculated risk of using CentOS, and this hasn't changed since 2.1. That's not going to change, nor can it thanks to all the reasons Johnny mentioned. Building a full distribution from sources can be a hard problem.
As far as the first build iteration completion date is concerned, that should be considered a rough draft build; if the QA process finds binary incompatibility (library linkage versions for the most part) then several if not all packages need rebuilding as part of the QA stage. Do you REALLY want to use first-draft stage 1 packages in production? You can look in the %{BUILDTIME} query tag for build order; use the following command to get the order: rpm -qa --queryformat "%{BUILDTIME} %{NAME} %{EPOCH} %{VERSION} %{RELEASE}\n" | sort
Or you can go through the released packages on centos.org and look at the build dates to see how many packages were built pretty late; for instance, bpftool-4.18.0-193.el8.x86_64.rpm apparently wasn't built in the released version until 2020-05-29. I'm reasonably sure the first pass at this was built probably a month earlier based on the dates of other packages, but something was probably found in QA that required a rebuild, or the rebuild depended on something else that was late to build.
AND since this rebuilt package HAS TO KEEP THE SAME Name/Epoch/Version/Release (NEVR) tuple to be RHEL-compatible, the first N=bpftool V=4.18.0 R= 193.el8 could not be released; who knows, there may have been a dozen of the same NVR (bpftool's %{EPOCH} is (none) and so I don't count it)... If the first N=bpftool V=4.18.0 R= 193.el8 were to be released, then how, within the RPM update decision mechanism that only uses the EVR to update, is dnf/yum/rpm going to know the version built on 2020-05-29 is updating the one built on say 2020-04-26? No, ${BUILDTIME} is NOT used for update decisions, sorry, and bumping %{EPOCH} is the nuclear sledgehammer of RPM versioning and thus is strongly discouraged. For complete compatibility you want every package to have the same NEVR in CentOS as in RHEL except where CentOS-specific changes had to be made (branding, mostly).
And then there's modularity in CentOS 8, which means packages might have to build against a whole 'nother set of packages (I'm thinking specifically of the three different PostgreSQL versions that are modularized, or the mariadb versus MySQL modules with their libraries, and so on) that does indeed dramatically increase the number of package builds that not only need building but then need QA testing. Or do you want to run untested packages in production to get the packages sooner?
On 6/24/20 12:27 PM, Lamar Owen wrote:
... You can look in the %{BUILDTIME} query tag for build order; use the following command to get the order: rpm -qa --queryformat "%{BUILDTIME} %{NAME} %{EPOCH} %{VERSION} %{RELEASE}\n" | sort
So, replying to my own post here, as build order is only part of the story. Determining the contents of what was in the actual BUILDROOT at %{BUILDTIME} can be right difficult, since the BUILDROOT is not guaranteed to have every package that has a prior %{BUILDTIME} in it.
Am 24.06.20 um 18:37 schrieb Lamar Owen:
On 6/24/20 12:27 PM, Lamar Owen wrote:
... You can look in the %{BUILDTIME} query tag for build order; use the following command to get the order: rpm -qa --queryformat "%{BUILDTIME} %{NAME} %{EPOCH} %{VERSION} %{RELEASE}\n" | sort
So, replying to my own post here, as build order is only part of the story. Determining the contents of what was in the actual BUILDROOT at %{BUILDTIME} can be right difficult, since the BUILDROOT is not guaranteed to have every package that has a prior %{BUILDTIME} in it.
Additionally; I think the "build loop 0" is build on Fedora and loop 1 on the output of loop 0 and so on ... which makes things more difficult.
-- Leon
On 6/17/20 3:32 PM, Phil Perry wrote:
On my home file server for example, which is not connected to the internet, what does it matter if the release is 1 month or 3 months out of date? I can install the server in the knowledge it's going to work, and be supported with updates for 10 years and I can largely forget about it. My el5 box ran for more than 10 years until the hardware eventually died.
EL5... how modern... from a production application server VM, not internet-connected: [root@c6-2850 ~]# ssh root@10.1.x.y root@10.1.x.y's password: Last login: Tue Jan 28 19:53:32 2020 unknown terminal "xterm-256color" unknown terminal "xterm-256color" [root@localhost root]# cat /etc/centos-release CentOS Linux Advanced Server release 2.1AS (Slurm) [root@localhost root]#
This one has to be hard reset every day or two (virsh reset rhel2.1) since the bridge to the guest just dies randomly, and a reboot inside the guest hangs hard before finishing the reboot. The hard reset has to manually load the ethernet kernel module after it's booted up so far; if the ethernet module loads too soon it will never connect.... haven't found the reason for that, either, just run a pinging script every fifteen minutes on the host to check for connectivity and 'virsh reset rhel2.1' when it fails. The appserver is hard reboot resilient, and the software does a very specific task, and there's no budget for a rewrite. At least I did upgrade it from Red Hat Linux 5.2 a couple of years ago (the RHL5.2 box, an old AMD K6/2-450 with 128MB of RAM, ran almost continuously for 20 years).
Thanks, CentOS!
On 17/06/2020 21:05, Lamar Owen wrote:
On 6/17/20 3:32 PM, Phil Perry wrote:
On my home file server for example, which is not connected to the internet, what does it matter if the release is 1 month or 3 months out of date? I can install the server in the knowledge it's going to work, and be supported with updates for 10 years and I can largely forget about it. My el5 box ran for more than 10 years until the hardware eventually died.
EL5... how modern... from a production application server VM, not internet-connected: [root@c6-2850 ~]# ssh root@10.1.x.y root@10.1.x.y's password: Last login: Tue Jan 28 19:53:32 2020 unknown terminal "xterm-256color" unknown terminal "xterm-256color" [root@localhost root]# cat /etc/centos-release CentOS Linux Advanced Server release 2.1AS (Slurm) [root@localhost root]#
This one has to be hard reset every day or two (virsh reset rhel2.1) since the bridge to the guest just dies randomly, and a reboot inside the guest hangs hard before finishing the reboot. The hard reset has to manually load the ethernet kernel module after it's booted up so far; if the ethernet module loads too soon it will never connect.... haven't found the reason for that, either, just run a pinging script every fifteen minutes on the host to check for connectivity and 'virsh reset rhel2.1' when it fails. The appserver is hard reboot resilient, and the software does a very specific task, and there's no budget for a rewrite. At least I did upgrade it from Red Hat Linux 5.2 a couple of years ago (the RHL5.2 box, an old AMD K6/2-450 with 128MB of RAM, ran almost continuously for 20 years).
Thanks, CentOS!
Indeed!
You have clearly had more success with the longevity of your hardware than me. I was plagued with expanding capacitors about 15 years ago which killed off most of my hardware at the time, and the replacements were taken out of service as they subsequently died meaning I'm exclusively running el7|8 now :-)
-
Akemi Yagi -
Alessandro Baggi -
Chris Adams -
Fred Smith -
Greg Bailey -
John Pierce -
Johnny Hughes -
Jon Pruente -
Lamar Owen -
Leon Fauster -
Michael Kofler -
Nicolas Kovacs -
Noam Bernstein -
Pete Biggs -
Peter -
Peter Ajamian
-
Phil Perry -
Scott Robbins -
Simon Matter -
Stephen John Smoogen -
Thomas Bendler -
Thomas Stephen Lee -
Tom Bishop -
Valeri Galtsev