On Friday, February 19, 2021 5:47 PM, Josh Boyer jwboyer@redhat.com wrote:
On Fri, Feb 19, 2021 at 5:47 PM redbaronbrowser redbaronbrowser@protonmail.com wrote:
I'm not asking for anything ahead of RHEL customers getting it. All I am asking for is pre-approval from management to get access as an Upstream once a problem with RHEL is confirmed.
If you mean "get access to the fixed code" or "get access to a fixed build", in a situation like the post-Boot Hole scenario there is no pre-approval necessary. We will do our best to resolve the issue in Stream as quickly as possible.
You keep coming back to the Fix for the Broken-Fix. I believe you when you say Red Hat has always been working as quickly as possible to Fix any Broken-Fixes. However, once they have the fix the lifecycle of the bug is over. Stream isn't interesting from the perspective of completed bug lifecycles. The interesting aspect to closing the openness gap is the mid-lifecycle of the bug.
I am trying to focus on how quickly will Stream users be able to contribute to the fixing of a Broken-Fix.
I am sorry if I am parsing too much into Mike's response, but it read to me like it is possible for the code to a Broken-Fix to, by policy, be withheld to only the "entitled" RHEL subscribers to access.
I'm asking to what degree Red Hat expects to be able to take advantage us while an active Bugzilla entry exist because a CVE "fix" had unintended side-effects. Can we get quick transparency into accessing broken packages and code even if that broken build is a CVE "fix" related?
Or is the policy that we will sit on the sidelines for Red Hat by itself to fix broken security patches much like CentOS has always operated before Stream?
On Sat, Feb 20, 2021 at 5:20 PM redbaronbrowser via CentOS-devel < centos-devel@centos.org> wrote:
On Friday, February 19, 2021 5:47 PM, Josh Boyer jwboyer@redhat.com wrote:
On Fri, Feb 19, 2021 at 5:47 PM redbaronbrowser redbaronbrowser@protonmail.com wrote:
I'm not asking for anything ahead of RHEL customers getting it. All I
am asking for is pre-approval from management to get access as an Upstream once a problem with RHEL is confirmed.
If you mean "get access to the fixed code" or "get access to a fixed build", in a situation like the post-Boot Hole scenario there is no pre-approval necessary. We will do our best to resolve the issue in Stream as quickly as possible.
You keep coming back to the Fix for the Broken-Fix. I believe you when you say Red Hat has always been working as quickly as possible to Fix any Broken-Fixes. However, once they have the fix the lifecycle of the bug is over. Stream isn't interesting from the perspective of completed bug lifecycles. The interesting aspect to closing the openness gap is the mid-lifecycle of the bug.
I am trying to focus on how quickly will Stream users be able to contribute to the fixing of a Broken-Fix.
I am sorry if I am parsing too much into Mike's response, but it read to me like it is possible for the code to a Broken-Fix to, by policy, be withheld to only the "entitled" RHEL subscribers to access.
You were.
I'm asking to what degree Red Hat expects to be able to take advantage us while an active Bugzilla entry exist because a CVE "fix" had unintended side-effects. Can we get quick transparency into accessing broken packages and code even if that broken build is a CVE "fix" related?
Or is the policy that we will sit on the sidelines for Red Hat by itself to fix broken security patches much like CentOS has always operated before Stream?
The policy is CVEs go out via RHEL first, just like it was with CentOS. When you see behavior that is counter to that policy, the policy was broken (this is a Red Hat policy, not a CentOS Stream policy).
-Mike
On Saturday, February 20, 2021 5:30 PM, Mike McGrath mmcgrath@redhat.com wrote:
On Sat, Feb 20, 2021 at 5:20 PM redbaronbrowser via CentOS-devel centos-devel@centos.org wrote:
On Friday, February 19, 2021 5:47 PM, Josh Boyer jwboyer@redhat.com wrote:
On Fri, Feb 19, 2021 at 5:47 PM redbaronbrowser redbaronbrowser@protonmail.com wrote:
I'm not asking for anything ahead of RHEL customers getting it. All I am asking for is pre-approval from management to get access as an Upstream once a problem with RHEL is confirmed.
If you mean "get access to the fixed code" or "get access to a fixed build", in a situation like the post-Boot Hole scenario there is no pre-approval necessary. We will do our best to resolve the issue in Stream as quickly as possible.
You keep coming back to the Fix for the Broken-Fix. I believe you when you say Red Hat has always been working as quickly as possible to Fix any Broken-Fixes. However, once they have the fix the lifecycle of the bug is over. Stream isn't interesting from the perspective of completed bug lifecycles. The interesting aspect to closing the openness gap is the mid-lifecycle of the bug.
I am trying to focus on how quickly will Stream users be able to contribute to the fixing of a Broken-Fix.
I am sorry if I am parsing too much into Mike's response, but it read to me like it is possible for the code to a Broken-Fix to, by policy, be withheld to only the "entitled" RHEL subscribers to access.
You were.
I'm asking to what degree Red Hat expects to be able to take advantage us while an active Bugzilla entry exist because a CVE "fix" had unintended side-effects. Can we get quick transparency into accessing broken packages and code even if that broken build is a CVE "fix" related?
Or is the policy that we will sit on the sidelines for Red Hat by itself to fix broken security patches much like CentOS has always operated before Stream?
The policy is CVEs go out via RHEL first, just like it was with CentOS. When you see behavior that is counter to that policy, the policy was broken (this is a Red Hat policy, not a CentOS Stream policy).
I wasn't claiming it was a Stream policy. And I'm not asking for CVEs to be released for CentOS first. I was just trying to determine, what is the expected latency between the time it is determine a CVE "fix" for RHEL causes broken behavior and how soon Stream users can contribute to fixing it.
On Sat, Feb 20, 2021 at 5:35 PM redbaronbrowser < redbaronbrowser@protonmail.com> wrote:
On Saturday, February 20, 2021 5:30 PM, Mike McGrath mmcgrath@redhat.com wrote:
On Sat, Feb 20, 2021 at 5:20 PM redbaronbrowser via CentOS-devel < centos-devel@centos.org> wrote:
On Friday, February 19, 2021 5:47 PM, Josh Boyer jwboyer@redhat.com wrote:
On Fri, Feb 19, 2021 at 5:47 PM redbaronbrowser redbaronbrowser@protonmail.com wrote:
I'm not asking for anything ahead of RHEL customers getting it. All I
am asking for is pre-approval from management to get access as an Upstream once a problem with RHEL is confirmed.
If you mean "get access to the fixed code" or "get access to a fixed build", in a situation like the post-Boot Hole scenario there is no pre-approval necessary. We will do our best to resolve the issue in Stream as quickly as possible.
You keep coming back to the Fix for the Broken-Fix. I believe you when you say Red Hat has always been working as quickly as possible to Fix any Broken-Fixes. However, once they have the fix the lifecycle of the bug is over. Stream isn't interesting from the perspective of completed bug lifecycles. The interesting aspect to closing the openness gap is the mid-lifecycle of the bug.
I am trying to focus on how quickly will Stream users be able to contribute to the fixing of a Broken-Fix.
I am sorry if I am parsing too much into Mike's response, but it read to me like it is possible for the code to a Broken-Fix to, by policy, be withheld to only the "entitled" RHEL subscribers to access.
You were.
I'm asking to what degree Red Hat expects to be able to take advantage us while an active Bugzilla entry exist because a CVE "fix" had unintended side-effects. Can we get quick transparency into accessing broken packages and code even if that broken build is a CVE "fix" related?
Or is the policy that we will sit on the sidelines for Red Hat by itself to fix broken security patches much like CentOS has always operated before Stream?
The policy is CVEs go out via RHEL first, just like it was with CentOS. When you see behavior that is counter to that policy, the policy was broken (this is a Red Hat policy, not a CentOS Stream policy).
I wasn't claiming it was a Stream policy. And I'm not asking for CVEs to be released for CentOS first. I was just trying to determine, what is the expected latency between the time it is determine a CVE "fix" for RHEL causes broken behavior and how soon Stream users can contribute to fixing it.
I don't think we've done this long enough to know what the lag will be. I can tell you there isn't some sort of management sleep() injected into the process. It will be up to the individual engineering teams and their development workflow. I don't expect we'll be opening CVE development to CentOS Stream.
-Mike
On Saturday, February 20, 2021 5:57 PM, Mike McGrath mmcgrath@redhat.com wrote:
On Sat, Feb 20, 2021 at 5:35 PM redbaronbrowser redbaronbrowser@protonmail.com wrote:
On Saturday, February 20, 2021 5:30 PM, Mike McGrath mmcgrath@redhat.com wrote:
On Sat, Feb 20, 2021 at 5:20 PM redbaronbrowser via CentOS-devel centos-devel@centos.org wrote:
On Friday, February 19, 2021 5:47 PM, Josh Boyer jwboyer@redhat.com wrote:
On Fri, Feb 19, 2021 at 5:47 PM redbaronbrowser redbaronbrowser@protonmail.com wrote:
I'm not asking for anything ahead of RHEL customers getting it. All I am asking for is pre-approval from management to get access as an Upstream once a problem with RHEL is confirmed.
If you mean "get access to the fixed code" or "get access to a fixed build", in a situation like the post-Boot Hole scenario there is no pre-approval necessary. We will do our best to resolve the issue in Stream as quickly as possible.
You keep coming back to the Fix for the Broken-Fix. I believe you when you say Red Hat has always been working as quickly as possible to Fix any Broken-Fixes. However, once they have the fix the lifecycle of the bug is over. Stream isn't interesting from the perspective of completed bug lifecycles. The interesting aspect to closing the openness gap is the mid-lifecycle of the bug.
I am trying to focus on how quickly will Stream users be able to contribute to the fixing of a Broken-Fix.
I am sorry if I am parsing too much into Mike's response, but it read to me like it is possible for the code to a Broken-Fix to, by policy, be withheld to only the "entitled" RHEL subscribers to access.
You were.
I'm asking to what degree Red Hat expects to be able to take advantage us while an active Bugzilla entry exist because a CVE "fix" had unintended side-effects. Can we get quick transparency into accessing broken packages and code even if that broken build is a CVE "fix" related?
Or is the policy that we will sit on the sidelines for Red Hat by itself to fix broken security patches much like CentOS has always operated before Stream?
The policy is CVEs go out via RHEL first, just like it was with CentOS. When you see behavior that is counter to that policy, the policy was broken (this is a Red Hat policy, not a CentOS Stream policy).
I wasn't claiming it was a Stream policy. And I'm not asking for CVEs to be released for CentOS first. I was just trying to determine, what is the expected latency between the time it is determine a CVE "fix" for RHEL causes broken behavior and how soon Stream users can contribute to fixing it.
I don't think we've done this long enough to know what the lag will be. I can tell you there isn't some sort of management sleep() injected into the process. It will be up to the individual engineering teams and their development workflow. I don't expect we'll be opening CVE development to CentOS Stream.
Thank you for your quick response.
To be clear, I wasn't expecting a CVE response team to be part of Stream. I was expecting regression response to be part of Stream of which CVE fixes is one potential source of regressions. I want to be able to work on CI/CD testing and regression fixes regardless of the source instead of only 80% of them.
On 2/20/21 6:00 PM, redbaronbrowser via CentOS-devel wrote:
On Saturday, February 20, 2021 5:57 PM, Mike McGrath mmcgrath@redhat.com wrote:
On Sat, Feb 20, 2021 at 5:35 PM redbaronbrowser <redbaronbrowser@protonmail.com mailto:redbaronbrowser@protonmail.com> wrote:
On Saturday, February 20, 2021 5:30 PM, Mike McGrath <mmcgrath@redhat.com <mailto:mmcgrath@redhat.com>> wrote:On Sat, Feb 20, 2021 at 5:20 PM redbaronbrowser via CentOS-devel <centos-devel@centos.org <mailto:centos-devel@centos.org>> wrote: On Friday, February 19, 2021 5:47 PM, Josh Boyer <jwboyer@redhat.com <mailto:jwboyer@redhat.com>> wrote: > On Fri, Feb 19, 2021 at 5:47 PM redbaronbrowser > redbaronbrowser@protonmail.com <mailto:redbaronbrowser@protonmail.com> wrote: > > > I'm not asking for anything ahead of RHEL customers getting it. All I am asking for is pre-approval from management to get access as an Upstream once a problem with RHEL is confirmed. > > If you mean "get access to the fixed code" or "get access to a fixed > build", in a situation like the post-Boot Hole scenario there is no > pre-approval necessary. We will do our best to resolve the issue in > Stream as quickly as possible. You keep coming back to the Fix for the Broken-Fix. I believe you when you say Red Hat has always been working as quickly as possible to Fix any Broken-Fixes. However, once they have the fix the lifecycle of the bug is over. Stream isn't interesting from the perspective of completed bug lifecycles. The interesting aspect to closing the openness gap is the mid-lifecycle of the bug. I am trying to focus on how quickly will Stream users be able to contribute to the fixing of a Broken-Fix. I am sorry if I am parsing too much into Mike's response, but it read to me like it is possible for the code to a Broken-Fix to, by policy, be withheld to only the "entitled" RHEL subscribers to access. You were. I'm asking to what degree Red Hat expects to be able to take advantage us while an active Bugzilla entry exist because a CVE "fix" had unintended side-effects. Can we get quick transparency into accessing broken packages and code even if that broken build is a CVE "fix" related? Or is the policy that we will sit on the sidelines for Red Hat by itself to fix broken security patches much like CentOS has always operated before Stream? The policy is CVEs go out via RHEL first, just like it was with CentOS. When you see behavior that is counter to that policy, the policy was broken (this is a Red Hat policy, not a CentOS Stream policy).I wasn't claiming it was a Stream policy. And I'm not asking for CVEs to be released for CentOS first. I was just trying to determine, what is the expected latency between the time it is determine a CVE "fix" for RHEL causes broken behavior and how soon Stream users can contribute to fixing it.I don't think we've done this long enough to know what the lag will be. I can tell you there isn't some sort of management sleep() injected into the process. It will be up to the individual engineering teams and their development workflow. I don't expect we'll be opening CVE development to CentOS Stream.
Thank you for your quick response.
To be clear, I wasn't expecting a CVE response team to be part of Stream. I was expecting regression response to be part of Stream of which CVE fixes is one potential source of regressions. I want to be able to work on CI/CD testing and regression fixes regardless of the source instead of only 80% of them.
Basically .. when a CVE is released in RHEL, there are two possibilities.
1) The current package in RHEL and Stream are the same.
2) The current package in RHEL is slightly older than the one in Stream.
If it is #1 .. after the package is released in RHEL .. if the new package is now newer than stream .. then 1 of 2 things will happen
A) That package will be put into stream too since it will also be in the next point release of RHEL.
or
B) If a rebase of that package is going to happen in the next point release .. they may pull in that rebased package and fic the CVE (the way they will do it in the next point release).
If instead they already have rebased the package (#2 above .. Stream is slightly ahead of RHEL).
In this case, they will do B) above .. roll the cve fix into the rebased package.
====
So .. how fast will they do B) ?
They will do it as fast as they can .. because .. they will be using whatever is in the build root to build other things and if they don't roll in the fixes in B), then anything they build against the older B) could have issues. No one wants to build a package in Stream that has a security issue (or other bug) that they will then need to rebuild again, it means more work for them.
Therefore, it is in the best intrest of the engineers to get the stuff into Stream as soon as they can.
That is why you can believe it will likely happen .. it is the thing that is the less amount of work for the engineers doing the work. It is also the best overall situation. When those lineup .. things are good :)
On Thu, Feb 25, 2021 at 10:44 AM Johnny Hughes johnny@centos.org wrote:
Basically .. when a CVE is released in RHEL, there are two possibilities.
The current package in RHEL and Stream are the same.
The current package in RHEL is slightly older than the one in Stream.
Or: the version in Stream never got updated and is out-of-date due to a compatibility issue with another Stream tested component, and the security issue is great enough to release it in RHEL without ever making it to Stream. Unless there is a guarantee of some sort that Stream will *always* get new releases with or ahead of the production RHEL, there are very likely to be missed updates in Stream. I've run such split distributions for various development environments, such a consistency is a promise likely to be broken on occasion, especially since CentOS Stream does not have a support contract that would prevent it.
If it is #1 .. after the package is released in RHEL .. if the new package is now newer than stream .. then 1 of 2 things will happen
Or: packages in Stream have dependencies incompatible with the update, especially if thoe packages have never been released in RHEL. Anything module based such as python modules, or software with version specific library dependenices, are vulnerable to this problem. And modularity..... modularity makes the resolution of such upgrade dependencies more dangerous. I particularly expect trouble with third party repos, such as EPEL.
Please, don't leave out possibilities when assessing risks. I've painful experiences of subtly incompatible upgrades compatible only with older, locked down, stable systems.
A) That package will be put into stream too since it will also be in the next point release of RHEL.
or
B) If a rebase of that package is going to happen in the next point release .. they may pull in that rebased package and fic the CVE (the way they will do it in the next point release).
Or other packages will also need to be updated. It's part of the problem with "you can't change just one thing".
If instead they already have rebased the package (#2 above .. Stream is slightly ahead of RHEL).
I hope you understand my skepticism that stream will be stable enough for anything resembling production work, and the lingering suspicion that stream is *deliberately* destabilizing to discourage peopole from using CentOS for production work.
In this case, they will do B) above .. roll the cve fix into the rebased package.
====
So .. how fast will they do B) ?
They will do it as fast as they can .. because .. they will be using whatever is in the build root to build other things and if they don't roll in the fixes in B), then anything they build against the older B) could have issues. No one wants to build a package in Stream that has a security issue (or other bug) that they will then need to rebuild again, it means more work for them.
Therefore, it is in the best intrest of the engineers to get the stuff into Stream as soon as they can.
That is why you can believe it will likely happen .. it is the thing that is the less amount of work for the engineers doing the work. It is also the best overall situation. When those lineup .. things are good :)
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On Thu, Feb 25, 2021 at 10:50 AM Nico Kadel-Garcia nkadel@gmail.com wrote:
On Thu, Feb 25, 2021 at 10:44 AM Johnny Hughes johnny@centos.org wrote:
Basically .. when a CVE is released in RHEL, there are two possibilities.
The current package in RHEL and Stream are the same.
The current package in RHEL is slightly older than the one in Stream.
Or: the version in Stream never got updated and is out-of-date due to a compatibility issue with another Stream tested component, and the security issue is great enough to release it in RHEL without ever making it to Stream. Unless there is a guarantee of some sort that Stream will *always* get new releases with or ahead of the production RHEL, there are very likely to be missed updates in Stream. I've run such split distributions for various development environments, such a consistency is a promise likely to be broken on occasion, especially since CentOS Stream does not have a support contract that would prevent it.
Eh, we tried hard not to make CentOS Stream a separate thing. For most people most of the time, they'll be working directly in stream and scripts and automation will pull it into RHEL. This helps us avoid having the engineers work in two separate places and the out-of-date issue. I'm not saying it will never happen, but it should be rare outside of the aforementioned CVE or NDA concerns.
If it is #1 .. after the package is released in RHEL .. if the new package is now newer than stream .. then 1 of 2 things will happen
Or: packages in Stream have dependencies incompatible with the update, especially if thoe packages have never been released in RHEL. Anything module based such as python modules, or software with version specific library dependenices, are vulnerable to this problem. And modularity..... modularity makes the resolution of such upgrade dependencies more dangerous. I particularly expect trouble with third party repos, such as EPEL.
Please, don't leave out possibilities when assessing risks. I've painful experiences of subtly incompatible upgrades compatible only with older, locked down, stable systems.
A) That package will be put into stream too since it will also be in the next point release of RHEL.
or
B) If a rebase of that package is going to happen in the next point release .. they may pull in that rebased package and fic the CVE (the way they will do it in the next point release).
Or other packages will also need to be updated. It's part of the problem with "you can't change just one thing".
If instead they already have rebased the package (#2 above .. Stream is slightly ahead of RHEL).
I hope you understand my skepticism that stream will be stable enough for anything resembling production work, and the lingering suspicion that stream is *deliberately* destabilizing to discourage peopole from using CentOS for production work.
There is no goal implied or explicit to deliberately destabilize CentOS Stream.
-Mike
In this case, they will do B) above .. roll the cve fix into the rebased package.
====
So .. how fast will they do B) ?
They will do it as fast as they can .. because .. they will be using whatever is in the build root to build other things and if they don't roll in the fixes in B), then anything they build against the older B) could have issues. No one wants to build a package in Stream that has a security issue (or other bug) that they will then need to rebuild again, it means more work for them.
Therefore, it is in the best intrest of the engineers to get the stuff into Stream as soon as they can.
That is why you can believe it will likely happen .. it is the thing that is the less amount of work for the engineers doing the work. It is also the best overall situation. When those lineup .. things are good :)
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On 2/25/21 10:49 AM, Nico Kadel-Garcia wrote:
On Thu, Feb 25, 2021 at 10:44 AM Johnny Hughes johnny@centos.org wrote:
Basically .. when a CVE is released in RHEL, there are two possibilities.
The current package in RHEL and Stream are the same.
The current package in RHEL is slightly older than the one in Stream.
Or: the version in Stream never got updated and is out-of-date due to a compatibility issue with another Stream tested component, and the security issue is great enough to release it in RHEL without ever making it to Stream. Unless there is a guarantee of some sort that Stream will *always* get new releases with or ahead of the production RHEL, there are very likely to be missed updates in Stream. I've run such split distributions for various development environments, such a consistency is a promise likely to be broken on occasion, especially since CentOS Stream does not have a support contract that would prevent it.
If it is #1 .. after the package is released in RHEL .. if the new package is now newer than stream .. then 1 of 2 things will happen
Or: packages in Stream have dependencies incompatible with the update, especially if thoe packages have never been released in RHEL. Anything module based such as python modules, or software with version specific library dependenices, are vulnerable to this problem. And modularity..... modularity makes the resolution of such upgrade dependencies more dangerous. I particularly expect trouble with third party repos, such as EPEL.
Please, don't leave out possibilities when assessing risks. I've painful experiences of subtly incompatible upgrades compatible only with older, locked down, stable systems.
A) That package will be put into stream too since it will also be in the next point release of RHEL.
or
B) If a rebase of that package is going to happen in the next point release .. they may pull in that rebased package and fic the CVE (the way they will do it in the next point release).
Or other packages will also need to be updated. It's part of the problem with "you can't change just one thing".
If instead they already have rebased the package (#2 above .. Stream is slightly ahead of RHEL).
I hope you understand my skepticism that stream will be stable enough for anything resembling production work, and the lingering suspicion that stream is *deliberately* destabilizing to discourage peopole from using CentOS for production work.
I could care less what you use .. but your FUD is silly. Stream is critical to the development of RHEL. Breaking it on purpose would be ridiculous.
Use what you want , but take your conspiracy theories to a prepper list.
And yes .. 3rd party repos will need to build some things for Stream .. or their things will be broken. That will be required. If they want to do it or not do it has nothing to do with how we are doing Stream.
If you want to use Stream , use it .. if you don't then don't.
But I will not have the Project be accused of purposely destabilizing things.
On Thu, Feb 25, 2021 at 11:49:38AM -0500, Nico Kadel-Garcia wrote:
Or: the version in Stream never got updated and is out-of-date due to a compatibility issue with another Stream tested component, and the security issue is great enough to release it in RHEL without ever making it to Stream. Unless there is a guarantee of some sort that Stream will *always* get new releases with or ahead of the production RHEL, there are very likely to be missed updates in Stream. I've run
This scenario doesn't make any sense to me. The next RHEL minor release will come from Stream, and that will definitely need the fix too, so there's not room for a long-standing divergence like this.
On 2/25/21 11:49 AM, Nico Kadel-Garcia wrote:
I hope you understand my skepticism that stream will be stable enough for anything resembling production work, and the lingering suspicion that stream is*deliberately* destabilizing to discourage peopole from using CentOS for production work.
The notion that Red Hat would deliberately break something, and then tell the whole world "this is what you can expect in the next version of RHEL" (which is, in fact, the core of the CentOS Stream messaging) makes no sense whatsoever, and would be exactly the same as saying "We plan for the next version of RHEL to be garbage."
I encourage you to read Phil's article - https://medium.com/swlh/centos-stream-why-its-awesome-5c45d944fb22 - if you have not already done so.
On Thu, Feb 25, 2021 at 2:49 PM Rich Bowen rbowen@redhat.com wrote:
On 2/25/21 11:49 AM, Nico Kadel-Garcia wrote:
I hope you understand my skepticism that stream will be stable enough for anything resembling production work, and the lingering suspicion that stream is*deliberately* destabilizing to discourage peopole from using CentOS for production work.
The notion that Red Hat would deliberately break something, and then tell the whole world "this is what you can expect in the next version of RHEL" (which is, in fact, the core of the CentOS Stream messaging) makes no sense whatsoever, and would be exactly the same as saying "We plan for the next version of RHEL to be garbage."
I encourage you to read Phil's article - https://medium.com/swlh/centos-stream-why-its-awesome-5c45d944fb22 - if you have not already done so.
I just read it. It seems.... to echo the party line. His enthusiasm for simplifying bug reporting and reporting bugs to CentOS Stream, rather than having to push it through RHEL subscribed ticketing systems, is not one I'd focused on. .It applauds the great things about CentOS stream being upstream of RHEL.
All of the desirable CentOS Stream releases are orthogonal to dropping the point releases based on the RHEL point releases, especially the distribution media ISO images.
* you can always mirror locally and snapshot the repos and update them every month or two or whatever works for you.
There's a difficulty of picking when to do these snapshots. They won't match the RHEL point releases, especially the updates in /etc/redhat-release, /etc/os-release, and the distribution media with the full software suites for network free installation. Is there any evidence that Red Hat is discarding those?
On Thursday, February 25, 2021 10:49 AM, Nico Kadel-Garcia nkadel@gmail.com wrote:
If instead they already have rebased the package (#2 above .. Stream is slightly ahead of RHEL).
I hope you understand my skepticism that stream will be stable enough for anything resembling production work, and the lingering suspicion that stream is deliberately destabilizing to discourage peopole from using CentOS for production work.
Please read Stef Walter's blog post here: https://blog.centos.org/2020/12/centos-stream-is-continuous-delivery/
What you seem to be suggesting is RHEL / CentOS would deliberately push out packages regardless of CI/CD results. I don't believe that is true.
Despite changes to Stream coming slightly ahead of RHEL, we should by nature of our contributions to CI/CD tests to establish the requirements for the package releases into Stream. Those tests can include cases that characterize our production use.
The question should not be if we can stabilize Stream for production use but rather WHEN. My best guess is most evaluations for switching to Stream will be taking place 60 to 90 days before CentOS 8 ends. So, to best encourage adoption of Stream by CentOS 8 users requires having CI/CD testing improved before October.
I sympathize with your skepticism but do not agree with it. Biggest threat to Stream right now is time. Claims there is a deliberate attempt to destablize Stream does not fit with the whole point of improved testing. There is just a lot of work to do and a window of opportunity that is closing fast.
On 25/02/2021 17:49, Nico Kadel-Garcia wrote: <snip>
I hope you understand my skepticism that stream will be stable enough for anything resembling production work, and the lingering suspicion that stream is *deliberately* destabilizing to discourage peopole from using CentOS for production work.
Hi Nico,
I resisted for a long time participating in debates/threads on centos-devel about Stream but conspiracy theories are always getting on my nerves ...
Everybody is free to believe for himself what he wants (including that Earth is flat, etc ..) but spreading FUD isn't helping.
I don't want to dive into a "arguments fight" on a mailing-list but let me just give you some facts, and talking here with my SysAdmin hat on (for centos.org infra) :
Since the announce I decided to default to CentOS Stream for all new deployments, including critical one : the new complete infra/environment dedicated to build the new Stream version (aka '9', as people can guess it also from Brian's presentation at CentOS Dojo and DevConf.cz) *is* completely built on top of CentOS Stream.
Various roles were all automatically deployed by Ansible, using previous roles tested on CentOS 8, without any modification needed for Stream (to be expected, as it's 8).
All pkgs build through infra8* tags on koji (https://cbs.centos.org/koji/search?match=glob&type=tag&terms=infra8*) work without a mod for CentOS Stream
Have I tested all the Ansible roles from centos.org infra against Stream ? not yet, but so far our baseline / monitoring / kojihub / kojiweb / kojid / kvm-host / kvm-guest / postgresql / httpd / mysql / nfs / iscsi (target and initiators) /haproxy (and others, let me stop here ...) roles were all working directly, and that on x86_64, ppc64le and aarch64 architectures ...
Now let's just take two seconds to think about it : *if* Red Hat would really like on purposes to make Stream unstable for production use, *why* would we even just deploy our critical build infra , used on the critical path for RHEL9, on top of Stream ? Just think about it and read this sentence again ;-)
Am I trying to sell you something ? no, and you're free to pick any solution that fits *your* needs Am I claiming that Stream is perfect and will be "bug free" ? no (and nobody did) :-)
Let me finish with this : life is too short to spend time fighting with each others. 2020 was hard enough so let's *try* to make 2021 'better' (for things where we can have influence that is) ..
And let me also quote Smooge's signature here, himself using an interesting quote : <quote> Let us be kind to one another, for most of us are fighting a hard battle - -- Ian MacClaren </quote>
Kind Regards,
On Friday, February 26, 2021 1:32 AM, Fabian Arrotin arrfab@centos.org wrote:
Now let's just take two seconds to think about it : if Red Hat would really like on purposes to make Stream unstable for production use, why would we even just deploy our critical build infra , used on the critical path for RHEL9, on top of Stream ? Just think about it and read this sentence again ;-)
I agree with the point you are trying to make. It is my belief that the CentOS project intends for Stream to stable for production use.
However, Red Hat is sending a mixed message by offering free RHEL licenses to the Stream project. The next logical step after making the offer would be for the CentOS Goverance Board to mandate a policy of using RHEL for Stream project infrastructure. Given Red Hat policy of withholding security updates from Stream, such a mandate would make sense.
If CentOS really will be using Stream in the long term for critical build infrastructure, it would be nice to hear from the CentOS governance board that they do not intend to change that.
On 26/02/2021 10:02, redbaronbrowser via CentOS-devel wrote:
On Friday, February 26, 2021 1:32 AM, Fabian Arrotin arrfab@centos.org wrote:
<snip>
However, Red Hat is sending a mixed message by offering free RHEL licenses to the Stream project.
Just curious : where have you seen/read that ? I'm not aware of that fact ... (and if that's true, ideally I'd be in a position to be aware of that or someone forgot to send the message to infra team :-) )
I guess that yes, with the newly announced 'RHEL for OSI' (just announced yesterday) we can eventually ask to be part of that program and so benefit from it, but so far it hasn't been discussed :)
All that should probably its own thread though
On Fri, 2021-02-26 at 09:02 +0000, redbaronbrowser via CentOS-devel wrote:
Given Red Hat policy of withholding security updates from Stream
This is a fundamental misstatement of the workflow.
For EMBARGOED security errata, RHEL will be getting the fix ahead of Stream.
Stream is built in the open where anyone can see what is built, any patches, and changelogs. If an embargoed update is built in stream before the announcement date, the embargo is violated.
RHEL is built in private. They can build the embargoed update whenever they want, stage it for release, and maintain the privacy of the CVE.
This means there is a certainty that EMBARGOED updates will get into RHEL first.
This gets more complex if the stream package is ahead of the RHEL update. If the stream and RHEL packages are identical, the source code sync process will automatically get the update built and published.
If the stream package is ahead of the RHEL package, then the patch will need to be ported over. This will take some time and be done in public. It may take minutes or hours.
If you've a way to improve this workflow while honoring the commitments to embargoes and build transparency, I'm certain it would be well received.
Pat
On Fri, Feb 26, 2021 at 02:23:47PM +0000, Patrick Riehecky wrote:
RHEL is built in private. They can build the embargoed update whenever they want, stage it for release, and maintain the privacy of the CVE.
This means there is a certainty that EMBARGOED updates will get into RHEL first.
This is the same situation with Fedora. It is sometimes but not always the case that Fedora package maintainers are aware of the embargoed issue (either as part of RH work or through some other connection), but often isn't. Generally, once the issue is public, RH maintainers work very quickly to get the fix into Fedora Linux.
On 2/26/21 3:02 AM, redbaronbrowser via CentOS-devel wrote:
On Friday, February 26, 2021 1:32 AM, Fabian Arrotin arrfab@centos.org wrote:
Now let's just take two seconds to think about it : if Red Hat would really like on purposes to make Stream unstable for production use, why would we even just deploy our critical build infra , used on the critical path for RHEL9, on top of Stream ? Just think about it and read this sentence again ;-)
I agree with the point you are trying to make. It is my belief that the CentOS project intends for Stream to stable for production use.
However, Red Hat is sending a mixed message by offering free RHEL licenses to the Stream project. The next logical step after making the offer would be for the CentOS Goverance Board to mandate a policy of using RHEL for Stream project infrastructure. Given Red Hat policy of withholding security updates from Stream, such a mandate would make sense.
If CentOS really will be using Stream in the long term for critical build infrastructure, it would be nice to hear from the CentOS governance board that they do not intend to change that.
No one in the CentOS Project ever said that Red Hat (the corporation) ever wanted anyone to deploy CetnOS Linux (or CentOS Stream) anywhere in production. Red Hat says that RHEL is the only thing they offer that is production ready. What else would they say?
In reality, they are correct. RHEL is the only production ready enterprise offering, simply because they offer things you need in that environment that is not offered other places. (A Service Level Agreement, Software Assurance, etc.)
Red Hat is also offering many more ways to get RHEL for free, as a service to the community. Currently the 2 programs .. more to come.
None of that has any impact on the CentOS Stream process. For some users, CentOS Stream will be usable for them in production .. for others it will not be.
But from a user perspective, packages built from source code that will become the next RHEL minor release in less than 6 months is absolutely as "stable" and "usable" as any enterprise distribution out there besides RHEL. The fact that it has a 5 year lifetime and is free is as good as any released distribution out there.
I get it, people want what they had. Hell, I want it too. If / When the other downstream RHEL source code builds happen, use them if that is what you want. None of that requires bashing CentOS. CentOS is not bashing any of those distros.
On Friday, February 26, 2021 8:33 AM, Johnny Hughes johnny@centos.org wrote:
I get it, people want what they had. Hell, I want it too. If / When the other downstream RHEL source code builds happen, use them if that is what you want. None of that requires bashing CentOS. CentOS is not bashing any of those distros.
Ok. But why is that what you want too?
Why isn't KW's blog post enough to consider Stream a win-win?
There seems to be belief that CentOS being a downstream will provide better QA than CentOS as an upstream.
I think KW was right that with a community drive of CD/CI that being an Upstream will be a win-win.
I see two reasons why on the CentOS 8 termination date why adoption of Stream 8 might be poor.
First is it sounds like Stream 8 will already be a de-focused project by that date. It isn't clear my request for a breakdown on the Stream 8 kernel patches will ever be honored. I see going from CentOS 8 to Stream 9 as much more jarring then transitioning to Stream 8. The project should be putting it's best foot forward on Stream 8 first and then focusing on 9.
Second is the timing of getting CD/CI to a mature state.
Some of the CD/CI test I see needed are complex and I can't find any current public example code of how to perform. If there is a CVE which causes a denial of service, what is the establish procedure of testing if the problem still exists? Is there any existing CD/CI tests that are performed in a VM with external monitoring to see if the test passes or fails (if the VM becomes unresponsive)?
If I am able to get a complex test that a current C8S package fails in April, how soon should we expect a fix and rebuild of the package? What if the fix reveals more tests are needed that it then also fails?
Is there any point in which a potential Stream adopter starts a 90 day evaluation of Stream in October while there are packages that are failing CD/CI tests because the packages haven't been rebuilt yet?
What was the basis for the determination Jan 1, 2022 is the best date to promote Stream focus/adoption?
Rich Bowen has pointed out multiple times the community through it's contributions gets a say in the success or failure of Stream. I believe to some extent that is true. But at the same time, we have already been vetoed on deciding when we are in a ready state. There is no attempt to seek community consensus on the target date.
If we just drive people to other RHEL clones then we have lost building a focus with those people around Stream. Isn't that enough reason to lay out a set of prerequisites for a "when Stream is ready" criteria before shutting down CentOS 8?
On 2/26/21 12:57 PM, redbaronbrowser via CentOS-devel wrote:
On Friday, February 26, 2021 8:33 AM, Johnny Hughes johnny@centos.org wrote:
I get it, people want what they had. Hell, I want it too. If / When the other downstream RHEL source code builds happen, use them if that is what you want. None of that requires bashing CentOS. CentOS is not bashing any of those distros.
Ok. But why is that what you want too?
Why isn't KW's blog post enough to consider Stream a win-win?
If you have to ask this question .. then you know nothing about me or the CentOS Project.
I personally built and released 95% of the CentOS Linux packages from CentOS Linux 4.0 on. It is really my life's passion for the last 17 years.
I was very upset / depressed when CentOS Linux 8 was given an early EOL. I do not now, nor will I ever like (or agree with) the decision. I get to have my opinion. I also work for Red Hat. I am not the decider .. I am an employee. So I do what I can do.
That fact has nothing to do with CentOS Stream, which I do feel is the best open source enterprise distro available since there will be no CentOS Linux moving forward (except CentOS 7 until the nomral EOL). I also think Stream helps both Red Hat and RHEL customers.
I have no idea what your agenda is. I do know who is working on the CentOS Stream project and I know what is being done on a daily basis.
Stream will absolutely be a great distribution.
Whether it meets your needs or not, only you can decide.
On Saturday, February 27, 2021 7:52 AM, Johnny Hughes johnny@centos.org wrote:
On 2/26/21 12:57 PM, redbaronbrowser via CentOS-devel wrote:
On Friday, February 26, 2021 8:33 AM, Johnny Hughes johnny@centos.org wrote:
I get it, people want what they had. Hell, I want it too. If / When the other downstream RHEL source code builds happen, use them if that is what you want. None of that requires bashing CentOS. CentOS is not bashing any of those distros.
Ok. But why is that what you want too? Why isn't KW's blog post enough to consider Stream a win-win?
If you have to ask this question .. then you know nothing about me or the CentOS Project.
I personally built and released 95% of the CentOS Linux packages from CentOS Linux 4.0 on. It is really my life's passion for the last 17 years.
Good point--I am sorry, I should have worded the question better. Thank you for everything you have done for the last 17 years and what you are continuing to do for CentOS 7 and Stream.
What I was trying to ask is as a CentOS user what would it take for you to be just as comfortable with Stream. As you are with CentOS 8 such that you would consider the transition a win-win.
I was very upset / depressed when CentOS Linux 8 was given an early EOL. I do not now, nor will I ever like (or agree with) the decision. I get to have my opinion. I also work for Red Hat. I am not the decider .. I am an employee. So I do what I can do.
That fact has nothing to do with CentOS Stream, which I do feel is the best open source enterprise distro available since there will be no CentOS Linux moving forward (except CentOS 7 until the nomral EOL). I also think Stream helps both Red Hat and RHEL customers.
I have no idea what your agenda is. I do know who is working on the CentOS Stream project and I know what is being done on a daily basis.
Stream will absolutely be a great distribution.
Whether it meets your needs or not, only you can decide.
I agree with everything you have said above.
My furstration ("agenda") is mostly with the hard termination date of CentOS 8. And also with the lack of hard dates/status for achieving Upstream features.
While it will always be true everyone will have to decide for themselves if Stream works for them. But I think tweaks to the deadline for CentOS 8 could have a huge impact on the adoption of Stream.
For example, if RHEL 8.6 freeze and fork of Stream takes place in Feb/Mar of 2022, that would be a better date to force for a CentOS 8 to Stream transistion.
Instead, it is being indicated that Red Hat is trying give CentOS more independence. But the foundation of that "independence" is to dictate the transition date. It might be true that users can choose for themselves a date to transition before Jan 1st. However, less than 13 months from the announcement is not enough time for a project to have achieved project independence.
I was hoping for a little more leeway from Red Hat for making Stream ready before running out of the time on C8. Preferably around RHEL 8.7 but since that doesn't seem realistic to ask for then at least until around RHEL 8.6.
On Sat, Feb 27, 2021 at 1:30 PM redbaronbrowser via CentOS-devel < centos-devel@centos.org> wrote:
On Saturday, February 27, 2021 7:52 AM, Johnny Hughes johnny@centos.org wrote:
On 2/26/21 12:57 PM, redbaronbrowser via CentOS-devel wrote:
On Friday, February 26, 2021 8:33 AM, Johnny Hughes johnny@centos.org
wrote:
I get it, people want what they had. Hell, I want it too. If / When the other downstream RHEL source code builds happen, use them if
that is
what you want. None of that requires bashing CentOS. CentOS is not bashing any of those distros.
Ok. But why is that what you want too? Why isn't KW's blog post enough to consider Stream a win-win?
If you have to ask this question .. then you know nothing about me or the CentOS Project.
I personally built and released 95% of the CentOS Linux packages from CentOS Linux 4.0 on. It is really my life's passion for the last 17 years.
Good point--I am sorry, I should have worded the question better. Thank you for everything you have done for the last 17 years and what you are continuing to do for CentOS 7 and Stream.
What I was trying to ask is as a CentOS user what would it take for you to be just as comfortable with Stream. As you are with CentOS 8 such that you would consider the transition a win-win.
I was very upset / depressed when CentOS Linux 8 was given an early EOL. I do not now, nor will I ever like (or agree with) the decision. I get to have my opinion. I also work for Red Hat. I am not the decider .. I am an employee. So I do what I can do.
That fact has nothing to do with CentOS Stream, which I do feel is the best open source enterprise distro available since there will be no CentOS Linux moving forward (except CentOS 7 until the nomral EOL). I also think Stream helps both Red Hat and RHEL customers.
I have no idea what your agenda is. I do know who is working on the CentOS Stream project and I know what is being done on a daily basis.
Stream will absolutely be a great distribution.
Whether it meets your needs or not, only you can decide.
I agree with everything you have said above.
My furstration ("agenda") is mostly with the hard termination date of CentOS 8. And also with the lack of hard dates/status for achieving Upstream features.
While it will always be true everyone will have to decide for themselves if Stream works for them. But I think tweaks to the deadline for CentOS 8 could have a huge impact on the adoption of Stream.
For example, if RHEL 8.6 freeze and fork of Stream takes place in Feb/Mar of 2022, that would be a better date to force for a CentOS 8 to Stream transistion.
Instead, it is being indicated that Red Hat is trying give CentOS more independence. But the foundation of that "independence" is to dictate the transition date. It might be true that users can choose for themselves a date to transition before Jan 1st. However, less than 13 months from the announcement is not enough time for a project to have achieved project independence.
I was hoping for a little more leeway from Red Hat for making Stream ready before running out of the time on C8. Preferably around RHEL 8.7 but since that doesn't seem realistic to ask for then at least until around RHEL 8.6.
On the Red Hat side of this equation, I can say our focus has been trying to get people into RHEL proper and getting our new programs created, terms legally vetted, etc. On the community side of this equation, alternative builds already exist (and I'm aware of at least one other one announced for March I think?) it seems extremely unlikely we'll revisit the RHEL8 lifecycle.
-Mike
On Saturday, February 27, 2021 2:15 PM, Mike McGrath mmcgrath@redhat.com wrote:
On Sat, Feb 27, 2021 at 1:30 PM redbaronbrowser via CentOS-devel centos-devel@centos.org wrote:
On the Red Hat side of this equation, I can say our focus has been trying to get people into RHEL proper and getting our new programs created, terms legally vetted, etc. On the community side of this equation, alternative builds already exist (and I'm aware of at least one other one announced for March I think?) it seems extremely unlikely we'll revisit the RHEL8 lifecycle.
-Mike
Sure, Red Hat is working to try to make RHEL attractive to others.
As you point out there is a community clone coming in March. There is also a non-commmunity build already trying to poach users through twitter announcements. Neither of those will achieve what Stream is attempting to.
I expect the majority of CentOS 8 users will choose RHEL and to a lesser extent the alternative clones as the C8 end gets close. I have not problem with that and I expect that to remain to be true regardless of when C8 ends.
But I'm not asking for RHEL8 lifecycle to change, instead I am asking for the CentOS 8 lifecycle to be changed--just a little bit. Having 10 months remaining of runway for the community involvement to take off is highly aggressive.
The amount of time given for community involvement in CI/CD testing is going to be critical for motivating as large a group as possible to adopt Stream. For example, let's say staying with the Dec 31st date results in for every 1,000 systems coverted to RHEL there is 10 systems that covert to Stream. But if giving 3-6 more months runway to Stream to build up means it is 11 systems that go to Stream, that is a 10% boost to Stream. Even if that 11th system takes away from the RHEL pool, the numbers for RHEL still remain 99.9% the same. Wouldn't shifting that one system from downstream to Upstream be more vaulable in the long run that going from downstream to midstream?
Look, I understand you have decided I'm non-human and looking to "blow up" Red Hat like some crazy robotic terrorist. I'm sorry, I get it, don't trust me.
Please instead ask yourself these two questions:
If you could send an email to yourself that goes back in time a year ago, is there anything you would have told yourself to do differently to prepare for the December announcement?
And if yourself from a year in the future could email you today, what do you think he would be telling you?
I am not trying to attack you or anyone else at Red Hat with these questions. I am interested in what the answers would be but if you don't want to share them with me, I completely understand.
However, is there any chance there are additional things you (or others) could do if allowed more than 10 remaining months on the C8 stopwatch that would benefit Stream and Red Hat more in the long run?
On Fri, Feb 26, 2021 at 9:34 AM Johnny Hughes johnny@centos.org wrote:
But from a user perspective, packages built from source code that will become the next RHEL minor release in less than 6 months is absolutely as "stable" and "usable" as any enterprise distribution out there besides RHEL. The fact that it has a 5 year lifetime and is free is as good as any released distribution out there.
It doesn't necessarily work, without all the other bits being updated at the same time. I'm particularly thinking of the "yum update --security" update that included curl-libs and broke wget until you did "yum update wget" or "yum update" for all components. There was also the "python3" versus "python36" adventure in RHEL 7, which is ongoing, and I anticipate similar adventures if and when python 3.8 is released for CentOs 8 Stream^H^H^H^H RHEL 8. I try to play nice, and have worked with updates to various components (published to third party repos, in particular) that were broken when the base OS got updates. I'm particularly concerned about breaking EPEL components.
This is an unavoidable risk of beta environments, whose updates of critical components and libraries may occur at any time and without any warning, kind of like the announcement of CentOS 8 stopping the publicaiton of point releases. I'm very skeptical, due to a lot of scar tissue, that critical daily use components like "awscli" from EPEL will not be broken on an entirely unpredictable basis with an update in CentOS Stream to, say, the "PyYAML".Python module. Python and many other modular tools, can be very vulnerable to one component being upgraded without the rest being upgraded.
I get it, people want what they had. Hell, I want it too. If / When the other downstream RHEL source code builds happen, use them if that is what you want. None of that requires bashing CentOS. CentOS is not bashing any of those distros.
It's very difficult to trust CentOS right now. Production grade services are... very mistrustful of "beta platforms", with good historical reason. The answer seems to be "Get RHEL."
On Friday, February 26, 2021 6:14 PM, Nico Kadel-Garcia nkadel@gmail.com wrote:
On Fri, Feb 26, 2021 at 9:34 AM Johnny Hughes johnny@centos.org wrote:
But from a user perspective, packages built from source code that will become the next RHEL minor release in less than 6 months is absolutely as "stable" and "usable" as any enterprise distribution out there besides RHEL. The fact that it has a 5 year lifetime and is free is as good as any released distribution out there.
It doesn't necessarily work, without all the other bits being updated at the same time. I'm particularly thinking of the "yum update --security" update that included curl-libs and broke wget until you did "yum update wget" or "yum update" for all components. There was also the "python3" versus "python36" adventure in RHEL 7, which is ongoing, and I anticipate similar adventures if and when python 3.8 is released for CentOs 8 Stream^H^H^H^H RHEL 8. I try to play nice, and have worked with updates to various components (published to third party repos, in particular) that were broken when the base OS got updates. I'm particularly concerned about breaking EPEL components.
CentOS now seems more receptive to greatly expanding the number of SIGs. Hopefully this will mean critical EPEL packages will migrate under CentOS. Once that has been done, it should be possible to get a more consistent CD/CI across both Stream and the Stream Extended packages inherented from EPEL.
On Sat, Feb 27, 2021 at 01:18:46AM +0000, redbaronbrowser via CentOS-devel wrote:
CentOS now seems more receptive to greatly expanding the number of SIGs. Hopefully this will mean critical EPEL packages will migrate under CentOS. Once that has been done, it should be possible to get a more consistent CD/CI across both Stream and the Stream Extended packages inherented from EPEL.
Why duplicate? I'd rather see CentOS SIGs with interest in EPEL packages actually work in EPEL.
On 3/1/21 9:44 AM, Matthew Miller wrote:
On Sat, Feb 27, 2021 at 01:18:46AM +0000, redbaronbrowser via CentOS-devel wrote:
CentOS now seems more receptive to greatly expanding the number of SIGs. Hopefully this will mean critical EPEL packages will migrate under CentOS. Once that has been done, it should be possible to get a more consistent CD/CI across both Stream and the Stream Extended packages inherented from EPEL.
Why duplicate? I'd rather see CentOS SIGs with interest in EPEL packages actually work in EPEL.
I would agree with this for items where you are trying to maintain the same versions. no ned to duplicate work.
If they need different versions for the SIG than are in EPEL .. maybe maintain those in the SIG.
On Monday, March 1, 2021 10:44 AM, Johnny Hughes johnny@centos.org wrote:
On 3/1/21 9:44 AM, Matthew Miller wrote:
On Sat, Feb 27, 2021 at 01:18:46AM +0000, redbaronbrowser via CentOS-devel wrote:
CentOS now seems more receptive to greatly expanding the number of SIGs. Hopefully this will mean critical EPEL packages will migrate under CentOS. Once that has been done, it should be possible to get a more consistent CD/CI across both Stream and the Stream Extended packages inherented from EPEL.
Why duplicate? I'd rather see CentOS SIGs with interest in EPEL packages actually work in EPEL.
I would agree with this for items where you are trying to maintain the same versions. no ned to duplicate work.
If they need different versions for the SIG than are in EPEL .. maybe maintain those in the SIG.
How will CD/CI work across those lines?
For example, rsyslog depends on glibc, I assume a patch applied to glibc will also result in the CD/CI tests for rsyslog also being run. If the patch breaks rsyslog, the patch will be reviewed before release.
One of the concerns being brought up seems to be if the nature of Stream will break EPEL packages. As far as I see it, if CD/CI testing is integrated between the two then that concern should be addressed over time as testing continue to improve.
So, take for example, syslog-ng instead of rsyslog. Is there any method for that package to also trigger CD/CI tests for glibc changes while remaining in a repo that is technically external to Stream?
If Stream 8 could have a change which breaks EPEL 8 packages, couldn't that indicate an API/ABI break which could create larger issues if ever propigated into RHEL?
To be clear, I'm not asking if Stream 9 might break EPEL 8 packages. I'm also not asking how quickly EPEL 9 will be available after the release of Stream 9. I'm just asking how CD/CI testing will work across like versions (Stream/EPEL 8, Stream/EPEL 9, etc).
On 3/1/21 12:00 PM, redbaronbrowser via CentOS-devel wrote:
On Monday, March 1, 2021 10:44 AM, Johnny Hughes johnny@centos.org wrote:
On 3/1/21 9:44 AM, Matthew Miller wrote:
On Sat, Feb 27, 2021 at 01:18:46AM +0000, redbaronbrowser via CentOS-devel wrote:
CentOS now seems more receptive to greatly expanding the number of SIGs. Hopefully this will mean critical EPEL packages will migrate under CentOS. Once that has been done, it should be possible to get a more consistent CD/CI across both Stream and the Stream Extended packages inherented from EPEL.
Why duplicate? I'd rather see CentOS SIGs with interest in EPEL packages actually work in EPEL.
I would agree with this for items where you are trying to maintain the same versions. no ned to duplicate work.
If they need different versions for the SIG than are in EPEL .. maybe maintain those in the SIG.
How will CD/CI work across those lines?
For example, rsyslog depends on glibc, I assume a patch applied to glibc will also result in the CD/CI tests for rsyslog also being run. If the patch breaks rsyslog, the patch will be reviewed before release.
One of the concerns being brought up seems to be if the nature of Stream will break EPEL packages. As far as I see it, if CD/CI testing is integrated between the two then that concern should be addressed over time as testing continue to improve.
So, take for example, syslog-ng instead of rsyslog. Is there any method for that package to also trigger CD/CI tests for glibc changes while remaining in a repo that is technically external to Stream?
If Stream 8 could have a change which breaks EPEL 8 packages, couldn't that indicate an API/ABI break which could create larger issues if ever propigated into RHEL?
So, how EPEL will test for and decide if stream packages work with current EPEL8 packages is not something we really handle on this list .. it is something that EPEL (managed by the Fedora Project) needs to figure out.
Carl George (who works on CentOS Stream and is a member of the EPEL board) probably knows what they are doing .. if he wishes to elaborate here.
To be clear, I'm not asking if Stream 9 might break EPEL 8 packages. I'm also not asking how quickly EPEL 9 will be available after the release of Stream 9. I'm just asking how CD/CI testing will work across like versions (Stream/EPEL 8, Stream/EPEL 9, etc).
Another issue is if the actual EPEL repos are available in the CentOS CI/CD system .. and if EPEL is available to build against for the CentOS Community Build System (CBS)
Both of those issues will need to be address to prevent duplicate work of rolling EPEL Packages into CentOS SIGs.
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On Mon, Mar 01, 2021 at 10:44:36AM -0600, Johnny Hughes wrote:
If they need different versions for the SIG than are in EPEL .. maybe maintain those in the SIG.
Possibly, or consider modularity. I know it's not perfect, but this is the kind of thing it was made for.
On 2/26/21 6:14 PM, Nico Kadel-Garcia wrote:
On Fri, Feb 26, 2021 at 9:34 AM Johnny Hughes johnny@centos.org wrote:
But from a user perspective, packages built from source code that will become the next RHEL minor release in less than 6 months is absolutely as "stable" and "usable" as any enterprise distribution out there besides RHEL. The fact that it has a 5 year lifetime and is free is as good as any released distribution out there.
It doesn't necessarily work, without all the other bits being updated at the same time. I'm particularly thinking of the "yum update --security" update that included curl-libs and broke wget until you did "yum update wget" or "yum update" for all components. There was also the "python3" versus "python36" adventure in RHEL 7, which is ongoing, and I anticipate similar adventures if and when python 3.8 is released for CentOs 8 Stream^H^H^H^H RHEL 8. I try to play nice, and have worked with updates to various components (published to third party repos, in particular) that were broken when the base OS got updates. I'm particularly concerned about breaking EPEL components.
yum --update security has never, ever worked in CentOS Linux .. not in any version. It will also not work on CentOS Stream.
This is an unavoidable risk of beta environments, whose updates of critical components and libraries may occur at any time and without any warning, kind of like the announcement of CentOS 8 stopping the publicaiton of point releases. I'm very skeptical, due to a lot of scar tissue, that critical daily use components like "awscli" from EPEL will not be broken on an entirely unpredictable basis with an update in CentOS Stream to, say, the "PyYAML".Python module. Python and many other modular tools, can be very vulnerable to one component being upgraded without the rest being upgraded.
It is not a beta environment. It is continuously released. Huge difference. This process is faster and more agile than, for example, the RHEL process. Will there be more issues in the code? Probably. But, you are not starting out with untested and unreleased code .. you are starting out with things that have been tested in Fedora and the upstream projects (GNOME, OpenOffice, Apache, OpenSSH, OpenSSL, etc) for a long period of time already.
This software gets pulled out at tree freeze time and stabilized. Everything else that happens is happening to that Enterprise (and frozen / stabilized code). It is not like you are rolling in items from Rawhide, etc.
I get it, people want what they had. Hell, I want it too. If / When the other downstream RHEL source code builds happen, use them if that is what you want. None of that requires bashing CentOS. CentOS is not bashing any of those distros.
It's very difficult to trust CentOS right now. Production grade services are... very mistrustful of "beta platforms", with good historical reason. The answer seems to be "Get RHEL."
Then move on to another distro. I will do my best work regardless of if anyone actually uses this or not. It will be available, use or don't use it, totally up to you.
Am 26.02.21 um 08:32 schrieb Fabian Arrotin: <snip>
All pkgs build through infra8* tags on koji (https://cbs.centos.org/koji/search?match=glob&type=tag&terms=infra8*) work without a mod for CentOS Stream
CS8 has clang 11 now.
I can not be build some internal packages on C8S now.
So, heads up.
-- Leon
-
Fabian Arrotin -
Johnny Hughes -
Leon Fauster -
Matthew Miller -
Mike McGrath -
Nico Kadel-Garcia -
Patrick Riehecky -
redbaronbrowser -
Rich Bowen