hi,
Johnny and I spoke about this a few days back and we think the best way to solve the problem around delayed srpms + waiting for isos, build and QA at the time of a point release is to get something like a 'Continous Release' repository going.
The Problem: At a point release stage, users dont get updates till such time as the ISOS are ready ( eg. during the 5.6 release process ), the updates are built, QA has had time to verify content and the mirrors sync up. For new installs, this isnt an issue; however for people with existing installs this can potentially expose their installations.
One solution: Export packages as they are built from the c[456]bsys into a repository that people can opt into, that would allow them to get early access to packages.
Couple of notes: - it would have to be opt-in, rather than default for everyone
- packages into the CR repo's will not be announced via the regular channels; the announcements will still only happen when rpms are visible in the os/ and updates/ repos.
- it will not mean a lesser or a worse quality of builds, we hope to have enough automation in place that rpms will only be released into this repo once we are sure that it meets the release requirements.
- this repository will run off centos.org machines initially, and will not be released to third party mirrors. The exact details of how the release-to-external-mirrors takes place, and how we manage the issues around that would be a conversation we can relegate into the future for now.
- the CR repos will go-away once rpms are released in the regular process, it will not transfer into a rolling release type repo.
At this stage, we are looking for comments as we prep for future release process.
- KB
On 30 May 2011 22:52, Karanbir Singh mail-lists@karan.org wrote:
Johnny and I spoke about this a few days back and we think the best way to solve the problem around delayed srpms + waiting for isos, build and QA at the time of a point release is to get something like a 'Continous Release' repository going.
<snip>
One solution: Export packages as they are built from the c[456]bsys into a repository that people can opt into, that would allow them to get early access to packages.
That reads like a very good solution, to me.
I would certainly appreciate the updated packages that resolve particular CVEs, whereas for plain bug-fixes I could wait.
Alan.
On 5/30/2011 6:12 PM, Alan Bartlett wrote:
Johnny and I spoke about this a few days back and we think the best way to solve the problem around delayed srpms + waiting for isos, build and QA at the time of a point release is to get something like a 'Continous Release' repository going.
<snip>
One solution: Export packages as they are built from the c[456]bsys into a repository that people can opt into, that would allow them to get early access to packages.
That reads like a very good solution, to me.
I would certainly appreciate the updated packages that resolve particular CVEs, whereas for plain bug-fixes I could wait.
Agreed on the security-related fixes being the important ones, but I suspect that build-order dependencies will apply anyway and there's no reason to hold back working updates. In any case, prioritizing the update stream ahead of work on anaconda and iso-building makes sense for the same reasons 5.6 was pushed ahead of 6.x work. It's just bad for everyone to leave known security vulnerabilities on currently running machines. Personally, I'd consider that important enough to make it the default, although in that case maybe they should go though the 'testing' repo first and require some large-scale feedback first.
On 31 May 2011 16:12, Les Mikesell lesmikesell@gmail.com wrote:
On 5/30/2011 6:12 PM, Alan Bartlett wrote:
I would certainly appreciate the updated packages that resolve particular CVEs, whereas for plain bug-fixes I could wait.
Agreed on the security-related fixes being the important ones, but I suspect that build-order dependencies will apply anyway and there's no reason to hold back working updates. In any case, prioritizing the update stream ahead of work on anaconda and iso-building makes sense for the same reasons 5.6 was pushed ahead of 6.x work. It's just bad for everyone to leave known security vulnerabilities on currently running machines. Personally, I'd consider that important enough to make it the default, although in that case maybe they should go though the 'testing' repo first and require some large-scale feedback first.
I had given a brief thought to the build-order dependencies and decided that if a security bug-fix could be pushed out as soon as it could be built, I would then -- once the full point update had been released -- perform a "yum reinstall" for all those "fast" security fixes. A bit hazy around the edges, so I would leave the fuller details to those greater wizards to ponder.
Alan.
On 05/31/2011 04:12 PM, Les Mikesell wrote:
machines. Personally, I'd consider that important enough to make it the default, although in that case maybe they should go though the 'testing' repo first and require some large-scale feedback first.
It would be interesting to see what you think are the tests these packages should go through first. I dont disagree, and I have a list of my own - but it would be great to get some more opinions on this subject.
- KB
On 5/31/2011 5:50 PM, Karanbir Singh wrote:
On 05/31/2011 04:12 PM, Les Mikesell wrote:
machines. Personally, I'd consider that important enough to make it the default, although in that case maybe they should go though the 'testing' repo first and require some large-scale feedback first.
It would be interesting to see what you think are the tests these packages should go through first. I dont disagree, and I have a list of my own - but it would be great to get some more opinions on this subject.
It is just hard to duplicate the whole 'real-world' in tests so no matter what you do you may have surprises when things are deployed to a large number of users. To a certain extent this is a special case since it should be identical to the upstream binaries which already have hit a large user base, though - but there are still unpredictable things that can go wrong. My opinion is that it would be best to expose the initial release as 'test' quality and let a large number of people try it in a large number of environments - knowing that they should treat it as a test.
On 06/01/2011 12:33 AM, Les Mikesell wrote:
can go wrong. My opinion is that it would be best to expose the initial release as 'test' quality and let a large number of people try it in a large number of environments - knowing that they should treat it as a test.
In which case, how would one estimate 'enough' people have used it and 'enough' people have said ok ? Or, in other words 'enough' people have not reported anything breaking for them.
One of the reasons why we want to keep the Continuous Release repo on .centos.org machines is to be able to 'watch' log files...
- KB
On 5/31/11 6:38 PM, Karanbir Singh wrote:
On 06/01/2011 12:33 AM, Les Mikesell wrote:
can go wrong. My opinion is that it would be best to expose the initial release as 'test' quality and let a large number of people try it in a large number of environments - knowing that they should treat it as a test.
In which case, how would one estimate 'enough' people have used it and 'enough' people have said ok ? Or, in other words 'enough' people have not reported anything breaking for them.
Off the top of my head I'd say a few dozen people explicitly reporting a tested-good status or a few thousand downloads and a few days with no negative reports. Pretty hard to generalize since there are going to be code paths that are very rarely exercised. But, you have to trade the risk of pushing minimally-tested code against leaving known vulnerabilities exposed even if they are 'local' type exploits. I see an assortment of probes for application level vulnerabilities (struts, php, etc.) that simply post a success notice to a central site when they work, which is later followed with attempts to use that hole to send commands that try local privilege escalation - so I'm fairly nervous about vulnerabilities that have been published.
One of the reasons why we want to keep the Continuous Release repo on .centos.org machines is to be able to 'watch' log files...
How big of a problem will it be to update something that needs a rebuild without a version bump?
On 01/06/11 06:16, Les Mikesell wrote:
On 5/31/11 6:38 PM, Karanbir Singh wrote:
On 06/01/2011 12:33 AM, Les Mikesell wrote:
can go wrong. My opinion is that it would be best to expose the initial release as 'test' quality and let a large number of people try it in a large number of environments - knowing that they should treat it as a test.
In which case, how would one estimate 'enough' people have used it and 'enough' people have said ok ? Or, in other words 'enough' people have not reported anything breaking for them.
Off the top of my head I'd say a few dozen people explicitly reporting a tested-good status or a few thousand downloads and a few days with no negative reports.
Les,
From where I'm standing, what you describe is pretty much exactly what we currently have in place with the current QA system. The whole point of a CR repository is to get the updates out quickly rather than wait for ISO to be built and pushed to the mirror network. If you add an additional layer of testing that's not going to happen.
Unless I misunderstand, KB is talking about releasing packages to CR as soon as they are ready (meaning as soon as they have passed all internal tests + QA) rather than delaying their release whilst the ISOs are subsequently built and everything is synced up to the mirrors. The packages released to CR would be the exact same set used to build the ISOs and eventually released to the mirrors, just you'd get access to them maybe a week earlier.
If more testing is needed then we need to add that to the existing QA phase, either by adding the required tests and/or by recruiting more testers. Anyone can contribute to that process by writing automated test suites.
On 6/1/2011 2:49 PM, Ned Slider wrote:
can go wrong. My opinion is that it would be best to expose the initial release as 'test' quality and let a large number of people try it in a large number of environments - knowing that they should treat it as a test.
In which case, how would one estimate 'enough' people have used it and 'enough' people have said ok ? Or, in other words 'enough' people have not reported anything breaking for them.
Off the top of my head I'd say a few dozen people explicitly reporting a tested-good status or a few thousand downloads and a few days with no negative reports.
Les,
From where I'm standing, what you describe is pretty much exactly what we currently have in place with the current QA system. The whole point of a CR repository is to get the updates out quickly rather than wait for ISO to be built and pushed to the mirror network. If you add an additional layer of testing that's not going to happen.
Not quite. You need to balance the risk of not making the updates available as the default 'yum update' action (the bazillion internet connected Centos boxes aren't all going to enable some new funky repository just to get an update quicker) against the risk of pushing a broken build. My opinion leans toward pushing security fixes as fast as possible to everyone with default settings (which, I suppose, could involve pushing a new centos-release that activates a new repo but then it takes 2 updates to get them). But that doesn't mean it should bypass testing entirely or that it should wait for anaconda or iso completion.
Unless I misunderstand, KB is talking about releasing packages to CR as soon as they are ready (meaning as soon as they have passed all internal tests + QA) rather than delaying their release whilst the ISOs are subsequently built and everything is synced up to the mirrors. The packages released to CR would be the exact same set used to build the ISOs and eventually released to the mirrors, just you'd get access to them maybe a week earlier.
If more testing is needed then we need to add that to the existing QA phase, either by adding the required tests and/or by recruiting more testers. Anyone can contribute to that process by writing automated test suites.
There are probably some new possibilities for problems with an out-of-order delivery or some new build issues that don't show up until the whole system is completed. It might be a good test to populate the updates into a stock upstream system to match what you expect CentOS to duplicate when the point release is completed - but you'd need a licensed copy to do that.
Hi,
On 06/01/2011 09:18 PM, Les Mikesell wrote:
From where I'm standing, what you describe is pretty much exactly what we currently have in place with the current QA system. The whole point of a CR repository is to get the updates out quickly rather than wait for ISO to be built and pushed to the mirror network. If you add an additional layer of testing that's not going to happen.
Not quite. You need to balance the risk of not making the updates available as the default 'yum update' action (the bazillion internet connected Centos boxes aren't all going to enable some new funky
The CR rpms will only be available by making a manual choice on the machines, so people need to opt-in rather than opt-out. So mass rollout should not happen.
repository just to get an update quicker) against the risk of pushing a broken build. My opinion leans toward pushing security fixes as fast as
The important point to keep in mind is that packages from the buildsystem will only be released when they have been through the regular package level as well as update from centos-<release-being-worked-on>-1. The aim is to reduce the need to ever reissue a package that has already been through that process.
The time that gets cut out is from not needing to wait for the distro stuff ( isos, mirrors, qa of the distro etc ) - which is a bulk of the time between upstream release and our release being available.
- KB
On 06/03/2011 12:31 PM, Karanbir Singh wrote:
The CR rpms will only be available by making a manual choice on the machines, so people need to opt-in rather than opt-out. So mass rollout should not happen.
I want to push this conversation a bit further and see if we can get agreement and some ideas on potential courses of exactly how that opt-in would work as we get ready for 5.7 in the near future, and 6.0 -> 6.1 -> 6.1+updates in the more immediate future.
One option we have is to make the CR repo available on only a few machines, isolate those from regular mirror.c.o taffic and push a centos-release-CR rpm that people need to manually install on their machines to 'opt in'. We could then obsolte: that centos-release-CR rpm with the centos-release that comes down the road when the isos/ are in place and the new release announced. This seems to be the cleanest way to do things. It also means we can clean out the CR rpms once the point release is published and not need to maintain it forever as a giant well of rpms.
Thoughts ?
- KB
On 06/21/2011 07:39 AM, Karanbir Singh wrote:
One option we have is to make the CR repo available on only a few machines, isolate those from regular mirror.c.o taffic and push a centos-release-CR rpm that people need to manually install on their machines to 'opt in'. We could then obsolte: that centos-release-CR rpm with the centos-release that comes down the road when the isos/ are in place and the new release announced. This seems to be the cleanest way to do things. It also means we can clean out the CR rpms once the point release is published and not need to maintain it forever as a giant well of rpms.
I think that this makes sense. With the added benefit that since the -CR repo rpm would be removed so the user wouldn't get an ugly surprise on during the next point release cycle if they weren't wanting the updates at that time.
Would there be any mechanism for ensuring that if you participate in the CR repo that if you installed a 'preview' rpm but it had some issues and was rebuilt that you would get the rebuilt rpm? Or would that just be the chance you take and you would have to hope that the package gets an update and release # increment to get updated to 'stable'?
David Hollis wrote:
On 06/21/2011 07:39 AM, Karanbir Singh wrote:
One option we have is to make the CR repo available on only a few machines, isolate those from regular mirror.c.o taffic and push a centos-release-CR rpm that people need to manually install on their machines to 'opt in'. We could then obsolte: that centos-release-CR rpm with the centos-release that comes down the road when the isos/ are in place and the new release announced. This seems to be the cleanest way to do things. It also means we can clean out the CR rpms once the point release is published and not need to maintain it forever as a giant well of rpms.
I think that this makes sense. With the added benefit that since the -CR repo rpm would be removed so the user wouldn't get an ugly surprise on during the next point release cycle if they weren't wanting the updates at that time.
Would there be any mechanism for ensuring that if you participate in the CR repo that if you installed a 'preview' rpm but it had some issues and was rebuilt that you would get the rebuilt rpm? Or would that just be the chance you take and you would have to hope that the package gets an update and release # increment to get updated to 'stable'?
If nothing else, yum in 6.x has history of what was updated, and if I remember correctly you can reinstall those same packages. If this is true, you can manually or automatically order yum to reinstall those packages just in case. Maybe mandatory reinstall of all packages installed that indicate they are installed from "@cr-repo" should be in order.
I am not sure about 5.x. If something similar exists for 5.x (and 4.x?), in for of a yum plugin, centos-release-CR could depend on install of that plugin and use it to remember all packages installed from that repository, so reinstall can be forced on those packages, just in case.
Ljubomir
On 06/21/2011 03:53 PM, Ljubomir Ljubojevic wrote:
David Hollis wrote:
On 06/21/2011 07:39 AM, Karanbir Singh wrote:
One option we have is to make the CR repo available on only a few machines, isolate those from regular mirror.c.o taffic and push a centos-release-CR rpm that people need to manually install on their machines to 'opt in'. We could then obsolte: that centos-release-CR rpm with the centos-release that comes down the road when the isos/ are in place and the new release announced. This seems to be the cleanest way to do things. It also means we can clean out the CR rpms once the point release is published and not need to maintain it forever as a giant well of rpms.
I think that this makes sense. With the added benefit that since the -CR repo rpm would be removed so the user wouldn't get an ugly surprise on during the next point release cycle if they weren't wanting the updates at that time.
Would there be any mechanism for ensuring that if you participate in the CR repo that if you installed a 'preview' rpm but it had some issues and was rebuilt that you would get the rebuilt rpm? Or would that just be the chance you take and you would have to hope that the package gets an update and release # increment to get updated to 'stable'?
If nothing else, yum in 6.x has history of what was updated, and if I remember correctly you can reinstall those same packages. If this is true, you can manually or automatically order yum to reinstall those packages just in case. Maybe mandatory reinstall of all packages installed that indicate they are installed from "@cr-repo" should be in order.
this makes sense..
I am not sure about 5.x. If something similar exists for 5.x (and 4.x?),
the "history" option appeared in 6.0
in for of a yum plugin, centos-release-CR could depend on install of that plugin and use it to remember all packages installed from that repository, so reinstall can be forced on those packages, just in case.
there is yum.log. but it's tedious.
On 06/21/2011 01:11 PM, David Hollis wrote:
I think that this makes sense. With the added benefit that since the -CR repo rpm would be removed so the user wouldn't get an ugly surprise on during the next point release cycle if they weren't wanting the updates at that time.
Right, and we dont need to maintain that CR repo for the life of the point release it was setup for. Since everyone would have the default $releaever/updates/$basearch repo enabled anyway. Since CR would not replace updates/ it would be an additional repo.
Would there be any mechanism for ensuring that if you participate in the CR repo that if you installed a 'preview' rpm but it had some issues and was rebuilt that you would get the rebuilt rpm? Or would that just be the chance you take and you would have to hope that the package gets an update and release # increment to get updated to 'stable'?
If there is a need to rebuild a package that is already in the CR repo, in order to maintain sanity we would need to bump release. The distrosync plugin for yum would help - but its hard to make sure that everyone using CR would have it and have it enabled.
- KB
On 06/21/2011 02:39 PM, Karanbir Singh wrote:
On 06/03/2011 12:31 PM, Karanbir Singh wrote:
The CR rpms will only be available by making a manual choice on the machines, so people need to opt-in rather than opt-out. So mass rollout should not happen.
I want to push this conversation a bit further and see if we can get agreement and some ideas on potential courses of exactly how that opt-in would work as we get ready for 5.7 in the near future, and 6.0 -> 6.1 -> 6.1+updates in the more immediate future.
One option we have is to make the CR repo available on only a few machines, isolate those from regular mirror.c.o taffic and push a centos-release-CR rpm that people need to manually install on their machines to 'opt in'.
so far so good...
We could then obsolte: that centos-release-CR rpm with the centos-release that comes down the road when the isos/ are in place and the new release announced.
I am not sure I get this part. Since the packages are not changed ( I presume the NEVR remains the same when the packages are moved from CR to "stable"), how would this "obsolete" process happen ? I am used to the fedora / fedora epel "testing" phase which is basically - packages are first pushed to a testing repo which is not enabled by default on client computers - after a) enough people test and validate a package or b) enough time has passed since the package was pushed to the testing-repo the package can be moved to the stable repo
This seems to be the cleanest way to do things. It also means we can clean out the CR rpms once the point release is published and not need to maintain it forever as a giant well of rpms.
Thoughts ?
Manuel Wolfshant wrote:
We could then obsolte: that centos-release-CR rpm with the centos-release that comes down the road when the isos/ are in place and the new release announced.
I am not sure I get this part. Since the packages are not changed ( I presume the NEVR remains the same when the packages are moved from CR to "stable"), how would this "obsolete" process happen ? I am used to the fedora / fedora epel "testing" phase which is basically
- packages are first pushed to a testing repo which is not enabled by
default on client computers
- after a) enough people test and validate a package or b) enough time
has passed since the package was pushed to the testing-repo the package can be moved to the stable repo
Trick is that CentOS will not change NEVR or version number if they recompile package with diferent build environment. It could look exactly the same, size and all, but it would act differently.
In Fedora/EPEL way, when you change anything you also change version number, but it is not so in RHEL rebuilds. version and NEVR *must* stay the same, even if correction is made. Hence the original problem in question, how to pass the info about rebuilded packages to yum.
Ljubomir
On 06/21/2011 04:24 PM, Ljubomir Ljubojevic wrote:
Manuel Wolfshant wrote:
We could then obsolte: that centos-release-CR rpm with the centos-release that comes down the road when the isos/ are in place and the new release announced.
I am not sure I get this part. Since the packages are not changed ( I presume the NEVR remains the same when the packages are moved from CR to "stable"), how would this "obsolete" process happen ? I am used to the fedora / fedora epel "testing" phase which is basically
- packages are first pushed to a testing repo which is not enabled by
default on client computers
- after a) enough people test and validate a package or b) enough time
has passed since the package was pushed to the testing-repo the package can be moved to the stable repo
Trick is that CentOS will not change NEVR or version number if they recompile package with diferent build environment. It could look exactly the same, size and all, but it would act differently.
In Fedora/EPEL way, when you change anything you also change version number, but it is not so in RHEL rebuilds. version and NEVR *must* stay the same, even if correction is made. Hence the original problem in question, how to pass the info about rebuilded packages to yum.
you are 100% correct. hence my "how would this "obsolete" process happen?" question
On Tuesday, June 21, 2011 09:24:11 AM Ljubomir Ljubojevic wrote:
In Fedora/EPEL way, when you change anything you also change version number, but it is not so in RHEL rebuilds. version and NEVR *must* stay the same, even if correction is made. Hence the original problem in question, how to pass the info about rebuilded packages to yum.
Checksum. This is used by the yum 3.4.x option 'distro-sync full' to do just exactly that; it does the distro-sync, and also will reinstall packages that have the same NEVR but different checksums.
You can try it out on Fedora, but you'll need yum 3.4 or later installed to do it. Yum 3.2.28 (F14) and yum 3.2.29 (current EL6, as referenced from a RHEL 6.1 install) have distro-sync, but not the 'full' option. The 'full' patch might apply cleanly to 3.2.29, but I have not had a chance to try it.
This discussion was up-thread already.....back on June 3rd.
On 06/21/2011 02:33 PM, Lamar Owen wrote:
You can try it out on Fedora, but you'll need yum 3.4 or later installed to do it. Yum 3.2.28 (F14) and yum 3.2.29 (current EL6, as referenced from a RHEL 6.1 install) have distro-sync, but not the 'full' option. The 'full' patch might apply cleanly to 3.2.29, but I have not had a chance to try it.
there is a yum-3.4 in the pipeline for c5-testing, should be there tomorrow at some point.
- KB
On 06/21/2011 04:37 PM, Karanbir Singh wrote:
On 06/21/2011 02:33 PM, Lamar Owen wrote:
You can try it out on Fedora, but you'll need yum 3.4 or later installed to do it. Yum 3.2.28 (F14) and yum 3.2.29 (current EL6, as referenced from a RHEL 6.1 install) have distro-sync, but not the 'full' option. The 'full' patch might apply cleanly to 3.2.29, but I have not had a chance to try it.
there is a yum-3.4 in the pipeline for c5-testing, should be there tomorrow at some point.
and it will be happily ignored by anyone wishing to stay close to upstream. but I assume those will happily ignore -CR , too :)
On Tuesday, June 21, 2011 09:45:33 AM Manuel Wolfshant wrote:
On 06/21/2011 04:37 PM, Karanbir Singh wrote:
there is a yum-3.4 in the pipeline for c5-testing, should be there tomorrow at some point.
and it will be happily ignored by anyone wishing to stay close to upstream. but I assume those will happily ignore -CR , too :)
Make yum >= 3.4 a requires for the CR release package..... :-)
Karanbir, is yum 3.4 going into main, or is it going to be CentOS Plus?
On 06/21/2011 02:02 PM, Manuel Wolfshant wrote:
We could then obsolte: that centos-release-CR rpm with the centos-release that comes down the road when the isos/ are in place and the new release announced.
I am not sure I get this part. Since the packages are not changed ( I presume the NEVR remains the same when the packages are moved from CR to "stable"), how would this "obsolete" process happen ? I am used to the fedora / fedora epel "testing" phase which is basically
so lets take an example: 5.6/os/centos-release-5-6.0.i386.rpm 5.6/os/ < has no centos-release rpm > 5.6/CR/updates/centos-release-CR-5-7.0.i386.rpm ( which only contains the repo definition for the CR repo )
and when there is a 5.7 : 5.7/os/centos-release-5-7.0.i386.rpm ( with an Obsoletes: centos-release-CR <= 5-7.0; and drop a copy of this rpm into the 5.6/CR/ repo as well.
Would that work ?
- packages are first pushed to a testing repo which is not enabled by
default on client computers
- after a) enough people test and validate a package or b) enough time
has passed since the package was pushed to the testing-repo the package can be moved to the stable repo
we could/should do that anyway with the QA repo's - but as we saw in the past, that sort of testing can be done by a dedicated group of people fairly quickly when they are doing the testing for the sake of testing by design.
- KB
On 06/21/2011 04:29 PM, Karanbir Singh wrote:
On 06/21/2011 02:02 PM, Manuel Wolfshant wrote:
We could then obsolte: that centos-release-CR rpmwith the centos-release that comes down the road when the isos/ are in place and the new release announced.
I am not sure I get this part. Since the packages are not changed ( I presume the NEVR remains the same when the packages are moved from CR to "stable"), how would this "obsolete" process happen ? I am used to the fedora / fedora epel "testing" phase which is basically
so lets take an example: 5.6/os/centos-release-5-6.0.i386.rpm 5.6/os/< has no centos-release rpm> 5.6/CR/updates/centos-release-CR-5-7.0.i386.rpm ( which only contains the repo definition for the CR repo )
and when there is a 5.7 : 5.7/os/centos-release-5-7.0.i386.rpm ( with an Obsoletes: centos-release-CR<= 5-7.0; and drop a copy of this rpm into the 5.6/CR/ repo as well.
Would that work ?
I meant the packages themselves, not the centos-release.rpm. this one is trivial my concern is the mechanism which would trigger reinstallation of packages which are modified between their existence in CR and the stable/updates phase
On 06/21/2011 02:48 PM, Manuel Wolfshant wrote:
I meant the packages themselves, not the centos-release.rpm. this one is trivial my concern is the mechanism which would trigger reinstallation of packages which are modified between their existence in CR and the stable/updates phase
we need to make sure that we dont get into that situation. So only ship to CR once we are happy with the package itself.
it does mean that some packages like anaconda and friends will never make it to CR.
- KB
On 06/21/2011 04:50 PM, Karanbir Singh wrote:
On 06/21/2011 02:48 PM, Manuel Wolfshant wrote:
I meant the packages themselves, not the centos-release.rpm. this one is trivial my concern is the mechanism which would trigger reinstallation of packages which are modified between their existence in CR and the stable/updates phase
we need to make sure that we dont get into that situation. So only ship to CR once we are happy with the package itself.
Well, the mere presence in CR implies "we are not yet sure those are ready for prime time". Or are they ?
it does mean that some packages like anaconda and friends will never make it to CR.
This aspect requires a bit of [additional] attention from the person who does the pushes. And maybe a blacklist of some sort ?
On 06/21/2011 02:53 PM, Manuel Wolfshant wrote:
Well, the mere presence in CR implies "we are not yet sure those are ready for prime time". Or are they ?
we are saying that they are early-access to the updates packages and *should* be ready for prime time.
it does mean that some packages like anaconda and friends will never make it to CR.
This aspect requires a bit of [additional] attention from the person who does the pushes. And maybe a blacklist of some sort ?
I think a whitelist to allow stuff through on a per rpm basis would be a better fit.
- KB
On 06/21/2011 05:16 PM, Karanbir Singh wrote:
On 06/21/2011 02:53 PM, Manuel Wolfshant wrote:
Well, the mere presence in CR implies "we are not yet sure those are ready for prime time". Or are they ?
we are saying that they are early-access to the updates packages and *should* be ready for prime time.
In this case I see no problems with the approach that you have suggested. Who wants to use them installs and activates manually the repo. Once the packages are moved to the stable tree the repo will be empty and behave just like the current empty ones . win-win situation
it does mean that some packages like anaconda and friends will never make it to CR.
This aspect requires a bit of [additional] attention from the person who does the pushes. And maybe a blacklist of some sort ?
I think a whitelist to allow stuff through on a per rpm basis would be a better fit.
Or that. But given that the package list changes per release ( since RH introduces new packages during the first half of the distro cycle ) a blacklist might be easier to maintain. Especially as I do not [fore]see many packages to be " forbidden ". To be reanalyzed when time comes ?
On 06/21/2011 03:23 PM, Manuel Wolfshant wrote:
I think a whitelist to allow stuff through on a per rpm basis would be a better fit.
Or that. But given that the package list changes per release ( since RH introduces new packages during the first half of the distro cycle ) a blacklist might be easier to maintain. Especially as I do not [fore]see many packages to be " forbidden ". To be reanalyzed when time comes ?
I was thinking that once the packages start moving into the buildsystem, the whitelist will be empty : at that stage we just need some way for the QA team members to be able to click a link or enable a tick box somewhere and have it added to the whitelist.
Also, i think we should have the basic rpm level tests running inside the buildsystem for 5.7, so that will be an additional point of reference / data point to consider ( and for fail's to get marked as such automatically ).
- KB
On 6/21/2011 9:16 AM, Karanbir Singh wrote:
On 06/21/2011 02:53 PM, Manuel Wolfshant wrote:
Well, the mere presence in CR implies "we are not yet sure those are ready for prime time". Or are they ?
we are saying that they are early-access to the updates packages and *should* be ready for prime time.
it does mean that some packages like anaconda and friends will never make it to CR.
This aspect requires a bit of [additional] attention from the person who does the pushes. And maybe a blacklist of some sort ?
I think a whitelist to allow stuff through on a per rpm basis would be a better fit.
Why is it that you think having the new rpm will be worse than continuing to have the problem the update is intended to fix?
On 06/21/2011 03:51 PM, Les Mikesell wrote:
I think a whitelist to allow stuff through on a per rpm basis would be a better fit.
Why is it that you think having the new rpm will be worse than continuing to have the problem the update is intended to fix?
a kernel that does not boot can kind of do that...
On 6/21/2011 9:53 AM, Karanbir Singh wrote:
On 06/21/2011 03:51 PM, Les Mikesell wrote:
I think a whitelist to allow stuff through on a per rpm basis would be a better fit.
Why is it that you think having the new rpm will be worse than continuing to have the problem the update is intended to fix?
a kernel that does not boot can kind of do that...
But is that really worse than one that allows anyone to become root, which might be the other choice? And if you can't take the chance or think you are firewalled well enough that it doesn't matter, why update at all?
On 06/21/2011 04:06 PM, Les Mikesell wrote:
a kernel that does not boot can kind of do that...
But is that really worse than one that allows anyone to become root, which might be the other choice? And if you can't take the chance or think you are firewalled well enough that it doesn't matter, why update at all?
I'm guessing you are just ranting here for the sake of ranting. Or do you really expect rpms to be pushed from build to public repos online without any testing at all ?
- KB
On 6/21/2011 10:09 AM, Karanbir Singh wrote:
On 06/21/2011 04:06 PM, Les Mikesell wrote:
a kernel that does not boot can kind of do that...
But is that really worse than one that allows anyone to become root, which might be the other choice? And if you can't take the chance or think you are firewalled well enough that it doesn't matter, why update at all?
I'm guessing you are just ranting here for the sake of ranting. Or do you really expect rpms to be pushed from build to public repos online without any testing at all ?
I'm pointing out that running for any length of time without fixing known vulnerabilities is a very bad. Even if it is a local root escalation - if you also have an exploit in a network app (like the bazillion in php and its apps, struts, etc.) the two can be combined to take over the machine and it is mostly a matter of time until it happens (and yes, this is from experience...). And I thought last time around you said these packages would go through the normal qa process before even going into the option CR repo, so I'll repeat the question as to why you think something is going to be wrong with them. I can see wanting some reasonable number of machines to run them as a test, but still don't understand why anyone would want to continue to run with known problems instead of having them fixed.
On 06/21/2011 04:41 PM, Les Mikesell wrote:
I'm pointing out that running for any length of time without fixing known vulnerabilities is a very bad. Even if it is a local root escalation - if you also have an exploit in a network app (like the bazillion in php and its apps, struts, etc.) the two can be combined to take over the machine and it is mostly a matter of time until it happens (and yes, this is from experience...). And I thought last time around you said these packages would go through the normal qa process before even going into the option CR repo, so I'll repeat the question as to why you think something is going to be wrong with them. I can see wanting some reasonable number of machines to run them as a test, but still don't understand why anyone would want to continue to run with known problems instead of having them fixed.
I think you need to re-read the thread a bit, you are getting confused about what we are doing and what Wolfy said was happening in Fedora.
- KB
On 06/21/2011 05:35 PM, Karanbir Singh wrote:
(and yes, this is from experience...). And I thought last time around you said these packages would go through the normal qa process before even going into the option CR repo, so I'll repeat the question as to why you think something is going to be wrong with them.
To re-iterate:
- rpms will be built, and will go through the usual tests - once QA is happy each rpm will be individually added to a whitelist allowing it to be pushed automatically to the CR repo. - packages from CR will move into the /os/ or /updates/ repo for the point release coming along when its done - if we need to rebuild a package that has already been released publicly, there will be a EVR bump.
- KB
On Tuesday, June 21, 2011 12:51:30 PM Karanbir Singh wrote:
- if we need to rebuild a package that has already been released
publicly, there will be a EVR bump.
This moots the 'distro-sync full' bits in yum, so no need for yum-3.4. Although having it in CentOS-Plus would be nice for those who know that that will break strict compatibility, since I know you'd prefer to not Require: it for CR.
And that also makes it cleaner. Much cleaner.
On 6/21/2011 11:51 AM, Karanbir Singh wrote:
On 06/21/2011 05:35 PM, Karanbir Singh wrote:
(and yes, this is from experience...). And I thought last time around you said these packages would go through the normal qa process before even going into the option CR repo, so I'll repeat the question as to why you think something is going to be wrong with them.
To re-iterate:
- rpms will be built, and will go through the usual tests
- once QA is happy each rpm will be individually added to a whitelist
allowing it to be pushed automatically to the CR repo.
- packages from CR will move into the /os/ or /updates/ repo for the
point release coming along when its done
- if we need to rebuild a package that has already been released
publicly, there will be a EVR bump.
So, if you were managing an internet connected host running CentOS, would you configure it to track the CR repo or not? Or what criteria would you use to make this decision? I'm still having trouble seeing why, if upstream decided they should go out, that someone running what is essentially identical to that upstream code doesn't need them for the same reasons. Or why to think the risk of installing them outweighs the risk of continuing to run what upstream had its reasons to replace.
Les Mikesell wrote:
So, if you were managing an internet connected host running CentOS, would you configure it to track the CR repo or not? Or what criteria would you use to make this decision? I'm still having trouble seeing why, if upstream decided they should go out, that someone running what is essentially identical to that upstream code doesn't need them for the same reasons. Or why to think the risk of installing them outweighs the risk of continuing to run what upstream had its reasons to replace.
We are not talking about regular updates, but **only** the time between RHEL point release and CentOS point release. So completed packages do not wait for *all* packages and ISO's to be released, but are available as soon as QA team approves them. If there is fundamental error for a base package that requires for some of those packages to be recompiled, we need to have some kind of automatic protection for that case scenario.
Ljubomir
On 6/21/2011 3:00 PM, Ljubomir Ljubojevic wrote:
Les Mikesell wrote:
So, if you were managing an internet connected host running CentOS, would you configure it to track the CR repo or not? Or what criteria would you use to make this decision? I'm still having trouble seeing why, if upstream decided they should go out, that someone running what is essentially identical to that upstream code doesn't need them for the same reasons. Or why to think the risk of installing them outweighs the risk of continuing to run what upstream had its reasons to replace.
We are not talking about regular updates, but **only** the time between RHEL point release and CentOS point release. So completed packages do not wait for *all* packages and ISO's to be released, but are available as soon as QA team approves them. If there is fundamental error for a base package that requires for some of those packages to be recompiled, we need to have some kind of automatic protection for that case scenario.
That doesn't address the risk of *not* installing these updates. Generally speaking, I think most users of CentOS trust upstream's choices and for me that includes when it is time to fix the bugs they shipped last time around. And generally speaking, I trust the CentOS project to be able to recompile working source and catch obvious mistakes before pushing them out.
So, again, under what circumstances does anyone think it is a good idea to not opt into this repo and instead keep running code that will very likely have published exploits over a time span that we've seen can run for months? I agree that this is a fuzzy area where not all of the point release updates address vulnerabilities or even serious bugs, but some certainly will. It just seems to me that not doing them is betting against the upstream wisdom or the project's building/QA capability. But I also agree that yum should be smarter and know something about rebuilds with no source change.
On 06/21/2011 09:26 PM, Les Mikesell wrote:
So, again, under what circumstances does anyone think it is a good idea to not opt into this repo and instead keep running code that will very likely have published exploits over a time span that we've seen can run for months?
Sounds like a good question to bring up at your next user group meeting. From the CentOS perspective, its important we give people the opportunity to get these packages as soon as possible so they can make their choice.
I dont particularly care about their religious choice or their internal implementation policies, and this list isn't the place to bash them around.
- KB
On 6/21/2011 5:36 PM, Karanbir Singh wrote:
On 06/21/2011 09:26 PM, Les Mikesell wrote:
So, again, under what circumstances does anyone think it is a good idea to not opt into this repo and instead keep running code that will very likely have published exploits over a time span that we've seen can run for months?
Sounds like a good question to bring up at your next user group meeting. From the CentOS perspective, its important we give people the opportunity to get these packages as soon as possible so they can make their choice.
I dont particularly care about their religious choice or their internal implementation policies, and this list isn't the place to bash them around.
Let's say we disagree about choosing to continue to run software with known/published exploits. I think you need very, very, good reasons to make that choice, which is why I think the choice should be opt-out, not in. It may be a matter of faith one way or another, but I think there is a lot more reason to risk installing the fixes than to leave it as a matter of time until someone takes over your machines for DDOS attacks against others or worse.
On 06/21/2011 11:48 PM, Les Mikesell wrote:
make that choice, which is why I think the choice should be opt-out, not in. It may be a matter of faith one way or another, but I think there
A vast majority of these updates are not Security related, they are the BA / EA variety, and if there is a security issue - we can always push those packages into the regular /5/updates/ repo.
quite a few people run private repo store's and they might not even come across any of the CR stuff; so major security issues *should* go into the regular /5/updates in an either and/or with CR
- KB
On 6/21/11 6:27 PM, Karanbir Singh wrote:
On 06/21/2011 11:48 PM, Les Mikesell wrote:
make that choice, which is why I think the choice should be opt-out, not in. It may be a matter of faith one way or another, but I think there
A vast majority of these updates are not Security related, they are the BA / EA variety, and if there is a security issue - we can always push those packages into the regular /5/updates/ repo.
quite a few people run private repo store's and they might not even come across any of the CR stuff; so major security issues *should* go into the regular /5/updates in an either and/or with CR
I'd expect it to be common for the kernels and probably glibc's included with a point release or soon thereafter to include security fixes. If you push those, you have the biggest risk of affecting everything else - so what's the point of isolating the rest?
Les Mikesell wrote:
On 6/21/11 6:27 PM, Karanbir Singh wrote:
On 06/21/2011 11:48 PM, Les Mikesell wrote:
make that choice, which is why I think the choice should be opt-out, not in. It may be a matter of faith one way or another, but I think there
A vast majority of these updates are not Security related, they are the BA / EA variety, and if there is a security issue - we can always push those packages into the regular /5/updates/ repo.
quite a few people run private repo store's and they might not even come across any of the CR stuff; so major security issues *should* go into the regular /5/updates in an either and/or with CR
I'd expect it to be common for the kernels and probably glibc's included with a point release or soon thereafter to include security fixes. If you push those, you have the biggest risk of affecting everything else - so what's the point of isolating the rest?
All I can see is you pushing extreme case scenario on something that is good will of the devs to lower aggravation of people waiting for point release to be completed, with agenda to push for 2-days delay between upstream and CentOS point releases, knowing it can not physically happen. It's like watching my 2-years old nephew screaming for his bottle of milk even tho he can see his mother pouring it just in front of him.
The packages that **can** be released faster *will* be released faster, those that could brake things will be held back, it is simple as that, at least in my book.
I will even dare to speculate that main reason for people to opt-in for CR repo will be so they can see how many packages are finished and to see packages coming out so they do not freak out without a visible progress. Side affect will be that some of them will be able to busy them selfs with comparing against upstream packages.
Ljubomir
On 6/22/2011 4:17 AM, Ljubomir Ljubojevic wrote:
I'd expect it to be common for the kernels and probably glibc's included with a point release or soon thereafter to include security fixes. If you push those, you have the biggest risk of affecting everything else - so what's the point of isolating the rest?
All I can see is you pushing extreme case scenario on something that is good will of the devs to lower aggravation of people waiting for point release to be completed, with agenda to push for 2-days delay between upstream and CentOS point releases, knowing it can not physically happen. It's like watching my 2-years old nephew screaming for his bottle of milk even tho he can see his mother pouring it just in front of him.
The packages that **can** be released faster *will* be released faster, those that could brake things will be held back, it is simple as that, at least in my book.
It's speculation at this point, but I think security fixes in the kernel and major libs are to be expected instead of being some extreme case, and those are precisely the most likely things that would cause something to break if done incorrectly. The point of planning the early release concept in the first place should be to get these fixes out to the people who otherwise become targets of well-known exploits and rootkits. Assume, for example, that another flaw is found in php or a web app that allows remote command execution, and another glibc flaw like the one recently fixed that allowed root escalation if you could make a symlink to a suid file. Now assume that the fixes for these vulnerabilities comes in or immediately after the point release. That scenario seems normal, expected, and what the early release planning should be all about instead of holding these back until a working ananconda and iso layout is ready and tested.
I will even dare to speculate that main reason for people to opt-in for CR repo will be so they can see how many packages are finished and to see packages coming out so they do not freak out without a visible progress. Side affect will be that some of them will be able to busy them selfs with comparing against upstream packages.
I think this is unlikely - unless they are unaware of the pending security issues, don't watch the news, and never look at their logs - or don't have an internet connection.
Les Mikesell wrote:
On 6/22/2011 4:17 AM, Ljubomir Ljubojevic wrote:
I'd expect it to be common for the kernels and probably glibc's included with a point release or soon thereafter to include security fixes. If you push those, you have the biggest risk of affecting everything else - so what's the point of isolating the rest?
All I can see is you pushing extreme case scenario on something that is good will of the devs to lower aggravation of people waiting for point release to be completed, with agenda to push for 2-days delay between upstream and CentOS point releases, knowing it can not physically happen. It's like watching my 2-years old nephew screaming for his bottle of milk even tho he can see his mother pouring it just in front of him.
The packages that **can** be released faster *will* be released faster, those that could brake things will be held back, it is simple as that, at least in my book.
It's speculation at this point, but I think security fixes in the kernel and major libs are to be expected instead of being some extreme case, and those are precisely the most likely things that would cause something to break if done incorrectly. The point of planning the early release concept in the first place should be to get these fixes out to the people who otherwise become targets of well-known exploits and rootkits. Assume, for example, that another flaw is found in php or a web app that allows remote command execution, and another glibc flaw like the one recently fixed that allowed root escalation if you could make a symlink to a suid file. Now assume that the fixes for these vulnerabilities comes in or immediately after the point release. That scenario seems normal, expected, and what the early release planning should be all about instead of holding these back until a working ananconda and iso layout is ready and tested.
So in another words, you approve creation of CR repository in it's current designed form.
And you also support devs when they release important security related packages through "updates" repo as fast as humanly possible, just I failed to understand what you are saying.
I will even dare to speculate that main reason for people to opt-in for CR repo will be so they can see how many packages are finished and to see packages coming out so they do not freak out without a visible progress. Side affect will be that some of them will be able to busy them selfs with comparing against upstream packages.
I think this is unlikely - unless they are unaware of the pending security issues, don't watch the news, and never look at their logs - or don't have an internet connection.
Ljubomir
On 6/22/2011 12:09 PM, Ljubomir Ljubojevic wrote:
It's speculation at this point, but I think security fixes in the kernel and major libs are to be expected instead of being some extreme case, and those are precisely the most likely things that would cause something to break if done incorrectly. The point of planning the early release concept in the first place should be to get these fixes out to the people who otherwise become targets of well-known exploits and rootkits. Assume, for example, that another flaw is found in php or a web app that allows remote command execution, and another glibc flaw like the one recently fixed that allowed root escalation if you could make a symlink to a suid file. Now assume that the fixes for these vulnerabilities comes in or immediately after the point release. That scenario seems normal, expected, and what the early release planning should be all about instead of holding these back until a working ananconda and iso layout is ready and tested.
So in another words, you approve creation of CR repository in it's current designed form.
It is sort of irrelevant to my concerns if things that have CVE's associated don't go there. But I'm not sure it can be that simple.
And you also support devs when they release important security related packages through "updates" repo as fast as humanly possible, just I failed to understand what you are saying.
Yes, this is the important part, but I don't think the updates of a point release are neatly divided into security/bug/feature sets and worse, there are usually a flurry of new updates immediately after the point release. It would be reasonable to expect and plan for something like a php, mod_perl or mod_python security update to come soon after a point release that would have dependencies on a whole bunch of other stuff in their point-release versions even if those hadn't been pushed as updates on their own yet. If the CR plan covers this, then I'm happy. I just see the security issue as a fairly sure thing and the risk being one of running version combinations that aren't well tested together. If you push anything due to security needs, I'm not sure that risk is reduced by holding parts back that upstream would have tested together.
Karanbir Singh wrote:
On 06/21/2011 11:48 PM, Les Mikesell wrote:
make that choice, which is why I think the choice should be opt-out, not in. It may be a matter of faith one way or another, but I think there
A vast majority of these updates are not Security related, they are the BA / EA variety, and if there is a security issue - we can always push those packages into the regular /5/updates/ repo.
quite a few people run private repo store's and they might not even come across any of the CR stuff; so major security issues *should* go into the regular /5/updates in an either and/or with CR
- KB
If it goes into updates, there is no need to have it in the CR repo as well since CR will just be like optional repo "updates2".
Ljubomir
On 06/22/2011 09:52 AM, Ljubomir Ljubojevic wrote:
quite a few people run private repo store's and they might not even come across any of the CR stuff; so major security issues *should* go into the regular /5/updates in an either and/or with CR
If it goes into updates, there is no need to have it in the CR repo as well since CR will just be like optional repo "updates2".
yeah, hence my and/or
keep in mind that when a previous point release moves to vault, we want to try and keep it as close to the updates/ and os/ state as was in the lifetime of that point release.
- KB
Les Mikesell wrote:
On 6/21/2011 3:00 PM, Ljubomir Ljubojevic wrote:
Les Mikesell wrote:
So, if you were managing an internet connected host running CentOS, would you configure it to track the CR repo or not? Or what criteria would you use to make this decision? I'm still having trouble seeing why, if upstream decided they should go out, that someone running what is essentially identical to that upstream code doesn't need them for the same reasons. Or why to think the risk of installing them outweighs the risk of continuing to run what upstream had its reasons to replace.
We are not talking about regular updates, but **only** the time between RHEL point release and CentOS point release. So completed packages do not wait for *all* packages and ISO's to be released, but are available as soon as QA team approves them. If there is fundamental error for a base package that requires for some of those packages to be recompiled, we need to have some kind of automatic protection for that case scenario.
That doesn't address the risk of *not* installing these updates. Generally speaking, I think most users of CentOS trust upstream's choices and for me that includes when it is time to fix the bugs they shipped last time around. And generally speaking, I trust the CentOS project to be able to recompile working source and catch obvious mistakes before pushing them out.
So, again, under what circumstances does anyone think it is a good idea to not opt into this repo and instead keep running code that will very likely have published exploits over a time span that we've seen can run for months? I agree that this is a fuzzy area where not all of the point release updates address vulnerabilities or even serious bugs, but some certainly will. It just seems to me that not doing them is betting against the upstream wisdom or the project's building/QA capability. But I also agree that yum should be smarter and know something about rebuilds with no source change.
I finally understand what you are talking about. Who said anything about not releasing critical updates as soon as srpms are available (in "updates" repo)? I am sure that every security patch will still be released as soon as it is rebuild, unless it requires a package that has build problems that will actually hold entire build process, in which case even CR repo will not help.
Ljubomir
On 6/21/2011 5:41 PM, Ljubomir Ljubojevic wrote:
I finally understand what you are talking about. Who said anything about not releasing critical updates as soon as srpms are available (in "updates" repo)? I am sure that every security patch will still be released as soon as it is rebuild, unless it requires a package that has build problems that will actually hold entire build process, in which case even CR repo will not help.
Is there any distinction? I thought the CR repo and opt-in was going to apply to all packages at and past the point release, which is bound to include some security fixes.
On 06/21/2011 07:03 PM, Les Mikesell wrote:
So, if you were managing an internet connected host running CentOS, would you configure it to track the CR repo or not? Or what criteria
I would yes, but thats my personal choice on my personal machines.
would you use to make this decision? I'm still having trouble seeing why, if upstream decided they should go out, that someone running what is essentially identical to that upstream code doesn't need them for the same reasons. Or why to think the risk of installing them outweighs the risk of continuing to run what upstream had its reasons to replace.
You basically hit the nail on the head there - the testing that we hope to go through before putting the packages into CR is exactly the testing that that needs to be done to make sure that things are how they should be. Done on a per-rpm level, in smaller groups and closer to build times means that we can get them out faster to a public place.
The 'issue' is how we execute the process on the public (~ user) side of things, what their process of opting in would be, and how they would get back onto the regular os/ and updates/ repo once that point release goes through.
- KB
Karanbir Singh wrote:
On 06/21/2011 02:02 PM, Manuel Wolfshant wrote:
We could then obsolte: that centos-release-CR rpm with the centos-release that comes down the road when the isos/ are in place and the new release announced.
I am not sure I get this part. Since the packages are not changed ( I presume the NEVR remains the same when the packages are moved from CR to "stable"), how would this "obsolete" process happen ? I am used to the fedora / fedora epel "testing" phase which is basically
so lets take an example: 5.6/os/centos-release-5-6.0.i386.rpm 5.6/os/ < has no centos-release rpm > 5.6/CR/updates/centos-release-CR-5-7.0.i386.rpm ( which only contains the repo definition for the CR repo )
and when there is a 5.7 : 5.7/os/centos-release-5-7.0.i386.rpm ( with an Obsoletes: centos-release-CR <= 5-7.0; and drop a copy of this rpm into the 5.6/CR/ repo as well.
Would that work ?
- KB
Will centos-release-5-7.0.i386.rpm delete "centos-CR.repo" file in *any* case, or are there possibilities that "centos-CR.repo" somehow survives?
If it's name is changed by accident, CR repository will stay enabled. Can that affect 5.7 repository in any way by like dist-sync but in the opposite direction 5.7 -> 5.6/CR?
I am not well versed in this type of problems so I might ask stupid questions.
Ljubomir
On 06/21/2011 02:52 PM, Ljubomir Ljubojevic wrote:
Will centos-release-5-7.0.i386.rpm delete "centos-CR.repo" file in *any* case, or are there possibilities that "centos-CR.repo" somehow survives?
If it's name is changed by accident, CR repository will stay enabled. Can that affect 5.7 repository in any way by like dist-sync but in the opposite direction 5.7 -> 5.6/CR?
yes, local edits will mean that the best we can do is have the file be moved to .rpmsave - but that would in turn cause the .repo to get disabled.
in either way, once 5.7/os and 5.7/updates are in place, and the /5/ link moves from 5.6 -> 5.7, the only packages in CR would be the centos-release rpm from 5.7. The other packages will be moved. If we decide to go down the workflow mentioned above in this thread.
- KB
On Tue, 21 Jun 2011, Karanbir Singh wrote:
On 06/21/2011 02:52 PM, Ljubomir Ljubojevic wrote:
Will centos-release-5-7.0.i386.rpm delete "centos-CR.repo" file in *any* case, or are there possibilities that "centos-CR.repo" somehow survives?
If it's name is changed by accident, CR repository will stay enabled. Can that affect 5.7 repository in any way by like dist-sync but in the opposite direction 5.7 -> 5.6/CR?
yes, local edits will mean that the best we can do is have the file be moved to .rpmsave - but that would in turn cause the .repo to get disabled.
in either way, once 5.7/os and 5.7/updates are in place, and the /5/ link moves from 5.6 -> 5.7, the only packages in CR would be the centos-release rpm from 5.7. The other packages will be moved. If we decide to go down the workflow mentioned above in this thread.
If you are going to do this I would like to request that the srpms for the centos-release package are released at the same time as the distro. I use custom repos for my machines and rebuild the centos-release package with a higher NVR so that all of my machines pull from my repos. This gets broken every time a new update comes out and updates the centos-release package. In the recent past the centos-release srpm has been MIA for some period of time after the distro release.
Also, would the CR repo be accessible via rsync?
Regards,
On 06/21/2011 04:22 PM, me@tdiehl.org wrote:
If you are going to do this I would like to request that the srpms for the centos-release package are released at the same time as the distro. I use
absolutely, the packages which are distro specific, like anaconda, centos-release and the notes etc will never make it to CR, since they change almost upto the time that the isos are ready. But we can try and make sure that the srpms for those are pushed at release time ( where release time would be the date/time when the isos and distro for the updated point release is pushed out )
Also, would the CR repo be accessible via rsync?
I think thats a different conversation. It would depend on how far we want to push the contents. if you need a a local copy, reposync should work anyway.
- KB
On 05/31/2011 02:12 AM, Alan Bartlett wrote:
On 30 May 2011 22:52, Karanbir Singhmail-lists@karan.org wrote:
Johnny and I spoke about this a few days back and we think the best way to solve the problem around delayed srpms + waiting for isos, build and QA at the time of a point release is to get something like a 'Continous Release' repository going.
<snip>
One solution: Export packages as they are built from the c[456]bsys into a repository that people can opt into, that would allow them to get early access to packages.
That reads like a very good solution, to me.
I would certainly appreciate the updated packages that resolve particular CVEs, whereas for plain bug-fixes I could wait.
+1
On Mon, May 30, 2011 at 10:52:01PM +0100, Karanbir Singh wrote:
- this repository will run off centos.org machines initially, and will
not be released to third party mirrors. The exact details of how the release-to-external-mirrors takes place, and how we manage the issues around that would be a conversation we can relegate into the future for now.
Works for me as a start, but we'd be really happy to mirror them for you as soon as possible.
Hi Matt,
On 05/31/2011 12:44 PM, Matthew Miller wrote:
On Mon, May 30, 2011 at 10:52:01PM +0100, Karanbir Singh wrote:
- this repository will run off centos.org machines initially, and will
not be released to third party mirrors. The exact details of how the
Works for me as a start, but we'd be really happy to mirror them for you as soon as possible.
Thanks for the offer.
The reason why we were thinking of keeping it on centos.org machines is so we can manage it a bit. In case we need to remove content from the repo, or replace content we would like to be able to do that really-quickly. Also, we need to workout a mechanism to make this CR repo go away in an elegant manner once content is released into the main os/ and updates/ repo.
Would be interesting if the CR repo could have a hard dependancy on the OS/ and Updates/ repo ( ~ it would only work if the machine also had the os/ and updates/ repo enabled ).
- KB
on 5/30/2011 2:52 PM Karanbir Singh spake the following: <snip>
At this stage, we are looking for comments as we prep for future release process.
- KB
Some type of tagging would be nice, so CR packages could easily be grep'd and yum --reinstalled if you felt so inclined.
Am 01.06.11 18:05, schrieb Scott Silva:
on 5/30/2011 2:52 PM Karanbir Singh spake the following:
<snip> > > At this stage, we are looking for comments as we prep for future release > process.
Some type of tagging would be nice, so CR packages could easily be grep'd and yum --reinstalled if you felt so inclined.
That might be hard to do, as the only working thing coming to my mind is adding to the R of NEVR - and what happens if that package doesn't get updated ever after that? You'll then have that release/repo tag for the rest of the package's lifetime. Reissuing the package within the point release might not work then either, if rpm/yum doesn't deem the then released package to be newer than the "rolling" package.
Or did you have something else in mind?
Ralph
On Thursday, June 02, 2011 08:53:49 AM Ralph Angenendt wrote:
That might be hard to do, as the only working thing coming to my mind is adding to the R of NEVR - and what happens if that package doesn't get updated ever after that? You'll then have that release/repo tag for the rest of the package's lifetime. Reissuing the package within the point release might not work then either, if rpm/yum doesn't deem the then released package to be newer than the "rolling" package.
And this is a problem 'developed' (as opposed to 'rebuilt') distributions don't have..... The mantra there is simply 'NEVR must change if the package changes in any way, shape, form, or build' and that eliminates the problem for 'developed' distributions.
Or did you have something else in mind?
:-)
yum-plugin-rolling-rebuild. Wish I had such written....or even specified.... A yum plugin that compares something unique between packages with identical NEVR tuples, but also something that is present in the repo files and is carried into the rpm database. Would be a prerequisite for successfully using CR in a seamless fashion, methinks.
Since the build time (field 'time_build') is present in the repo metadata, perhaps this plugin could force a reinstall during 'yum update' of package(s) whose time_build has changed. But is time_build carried over from the yum sqlite repo databases to the rpm databases? That I don't know, and my Berkely db skills aren't even close to my SQL skills.... So finding something that will tag the package as having changed even though NEVR has not changed, and that the plugin can then verify properly and then issue the reinstall of that package as part of the update is what is needed for this specific use case.
Ok, looking at possibilities, the yum repo data field 'time_build' might just work, since RPM apparently does store that information: [root@migration base]# rpm -qa --queryformat="%{NAME} %{BUILDTIME} \n"|head gnome-audio 1168123179 libICE 1168062878 libjpeg 1168063870 info 1173922742 libattr 1168060218 libxslt 1217530262 libogg 1173898173 readline 1253572717 gmp 1173887159 libattr 1168060345 [root@migration base]#
Thoughts?
On 06/02/2011 02:24 PM, Lamar Owen wrote:
yum-plugin-rolling-rebuild. Wish I had such written....or even specified.... A yum plugin that compares something unique between packages with identical NEVR tuples, but also something that is present in the repo files and is carried into the rpm database. Would be a prerequisite for successfully using CR in a seamless fashion, methinks.
are you trying to rehash this : http://lists.baseurl.org/pipermail/yum-devel/2011-March/008071.html
- KB
On Thursday, June 02, 2011 10:19:44 PM Karanbir Singh wrote:
On 06/02/2011 02:24 PM, Lamar Owen wrote:
yum-plugin-rolling-rebuild. Wish I had such written....or even specified.... A yum plugin that compares something unique between packages with identical NEVR tuples, but also something that is present in the repo files and is carried into the rpm database. Would be a prerequisite for successfully using CR in a seamless fashion, methinks.
are you trying to rehash this : http://lists.baseurl.org/pipermail/yum-devel/2011-March/008071.html
Hmmm, didn't know that patch to distro-sync existed. But, yeah, that would be the ticket. Assuming it could go into the distributed yum. Can yum for C5 be retrofitted for distro-sync? Hmmm, it looks like a minor version bump from 3.2.22 in C5 to 3.2.29 in EL6. But, yes, this would, IMO, fit the bill.
On 06/03/2011 02:04 PM, Lamar Owen wrote:
are you trying to rehash this : http://lists.baseurl.org/pipermail/yum-devel/2011-March/008071.html
Hmmm, didn't know that patch to distro-sync existed. But, yeah, that would be the ticket. Assuming it could go into the distributed yum. Can yum for C5 be retrofitted for distro-sync? Hmmm, it looks like a minor version bump from 3.2.22 in C5 to 3.2.29 in EL6. But, yes, this would, IMO, fit the bill.
go ahead, git it a shot! ( get yum-from-c5 working with this patch )
- KB
On Friday, June 03, 2011 09:29:53 AM Karanbir Singh wrote:
On 06/03/2011 02:04 PM, Lamar Owen wrote:
Hmmm, didn't know that patch to distro-sync existed. But, yeah, that would be the ticket. Assuming it could go into the distributed yum. Can yum for C5 be retrofitted for distro-sync? Hmmm, it looks like a minor version bump from 3.2.22 in C5 to 3.2.29 in EL6. But, yes, this would, IMO, fit the bill.
go ahead, git it a shot! ( get yum-from-c5 working with this patch )
It's certainly a worthwhile attempt.... looks like I'll be pulling out some more python docs in the near future.....
Might mean an upgrade of yum to 3.2.29, or other version more recent than 3.2.22. So time to read some release notes for yum as well....
On 06/03/2011 03:42 PM, Lamar Owen wrote:
Hmmm, didn't know that patch to distro-sync existed. But, yeah, that would be the ticket. Assuming it could go into the distributed yum. Can yum for C5 be retrofitted for distro-sync? Hmmm, it looks like a minor version bump from 3.2.22 in C5 to 3.2.29 in EL6. But, yes, this would, IMO, fit the bill.
It's certainly a worthwhile attempt.... looks like I'll be pulling out some more python docs in the near future.....
cool :)
Might mean an upgrade of yum to 3.2.29, or other version more recent than 3.2.22. So time to read some release notes for yum as well....
In an ideal world, we should try and keep yum at 3.2.22, lets first confirm that the distro-sync patch does not apply to 3.2.22 as we ship it first.
- KB
On Jun 3, 2011, at 12:19 PM, Karanbir Singh wrote:
In an ideal world, we should try and keep yum at 3.2.22, lets first confirm that the distro-sync patch does not apply to 3.2.22 as we ship it first.
This is the flaw in "back ports" like distro-sync -> yum 3.2.22
When you retrofit functionality and don't change the version then noone knows what tools are "supposed" to do anymore.
This is the "real" not the "ideal" world.
You might as well be honest and change the version too imho. ymmv.
73 de Jeff
On Friday, June 03, 2011 12:19:52 PM Karanbir Singh wrote:
In an ideal world, we should try and keep yum at 3.2.22, lets first confirm that the distro-sync patch does not apply to 3.2.22 as we ship it first.
It's hard to patch functionality that isn't even present. That is, 3.2.22 doesn't do distro-sync at all, and this patch patches existing distro-sync code.
The 'distro-sync full' patch was applied to yum git on 3/25/2011, so current yum from git has this functionality; yum-3.4.0 was first shipping version with it, and is the stable version immediately after 3.2.29.
I'm not even going to speculate on how difficult or how easy actually going to yum 3.4 would be, but I am going to read up on it.
It would be easier for C6 than for C5, as the distro-sync full patch might apply cleanly to the 3.2.29 cli.py. As to backporting the patch, that would require a backport of the entirety of the distro-sync functionality (which, IMO, isn't in itself a bad thing).
-
Alan Bartlett -
David Hollis -
Jeff Johnson -
Karanbir Singh -
Lamar Owen -
Les Mikesell -
Ljubomir Ljubojevic -
Manuel Wolfshant -
Matthew Miller -
me＠tdiehl.org -
Ned Slider -
Ralph Angenendt -
Scott Silva