-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hello Folks,
As discussed in the past on this mailing list for the Centos build service we need to define different koji targets for the SIGs. I 'll try to present you a possible solution.
In the koji world, a "target" has a "build tag" and "destination tag". To simply the "build tag" is the buildroot and "destination tag" is the tag that will contain all the packages.
If you look at Fedora koji instance (http://koji.fedoraproject.org/koji/buildtargets) you see that it is quite simple : fXX target and the associated "build tag" is fXX-build. That implies they build only packages with a single "disttag" (from the macro file) for a "destination tag".
As we want to provide for each "sig" a set of targets we need to think about the naming.
The SIGs will be able to choose a "disttag" ; which can be default el7.centos, el7 or something else.
Let's say we have SIG="storage" DISTRIBUTION="el7".
storage7-el7-build storage7-el7.centos-build storage7-el7_0-build
On top of that some products we want to build/rebuild use software collection so we will need an additional buildroot (e.g: *python33-build):
storage7-el7-python33-build storage7-el7_0-python33-build
This part is needed for some rebuild, if the packaging is done correctly we should not need more than Requires:COLLECTION-build in the spec files.
So my proposal is:
For the build tags : <SIG><MAJOR>-<TAG>[-<COLLECTION>,]-build And the target : <SIG><MAJOR>-<TAG>[-<COLLECTION>,] The destination can be <SIG><MAJOR>-testing/<SIG><MAJOR>-release for all the target. (or keep the Fedora naming dist-*, but it looks more confusing to me)
Maybe we can find something better, let me know.
I think we need to take decision on the naming so we can integrate with centpkg (don't forget about tomorrow centpkg demo : http://lists.centos.org/pipermail/centos-devel/2014-July/011474.html)
The tools need to be able depending on the "disttag" to use the correct "target" on the koji CLI, so it will be transparent to build service users. e.g: koji build storage7-el7.centos https://git.centos.org/pkg?#tag
I presented this as our SIG use case but please keep in mind that we will have the same issues in the future ("Force sig to use 1 disttag only" argument) if we want to add more rebuild use cases (scl, devtoolset, etc...)
I hope I did not confused you more :-) I am waiting for your feedback.
- -- Thomas
On 07/15/2014 09:49 AM, Thomas Oulevey wrote:
So my proposal is:
For the build tags : <SIG><MAJOR>-<TAG>[-<COLLECTION>,]-build And the target : <SIG><MAJOR>-<TAG>[-<COLLECTION>,] The destination can be <SIG><MAJOR>-testing/<SIG><MAJOR>-release for all the target. (or keep the Fedora naming dist-*, but it looks more confusing to me)
I think this works, the only consideration is that we might need to ( we really should ) give the SIG's a -devel a -testing and -production target, so they can build and release into vairous repos.
At the very least we would need a -devel where they can do whatever, and a -testing where they can then push builds that need to be made public on the repos, and once some criteria is satisfied, we can push content from -testing to -production ( I dont think we need a rebuild for that, just moving the rpms over should be good enough )
- KB
On 07/15/2014 04:37 PM, Karanbir Singh wrote:
On 07/15/2014 09:49 AM, Thomas Oulevey wrote:
So my proposal is:
For the build tags : <SIG><MAJOR>-<TAG>[-<COLLECTION>,]-build And the target : <SIG><MAJOR>-<TAG>[-<COLLECTION>,] The destination can be <SIG><MAJOR>-testing/<SIG><MAJOR>-release for all the target. (or keep the Fedora naming dist-*, but it looks more confusing to me)
I think this works, the only consideration is that we might need to ( we really should ) give the SIG's a -devel a -testing and -production target, so they can build and release into vairous repos.
At the very least we would need a -devel where they can do whatever, and a -testing where they can then push builds that need to be made public on the repos, and once some criteria is satisfied, we can push content from -testing to -production ( I dont think we need a rebuild for that, just moving the rpms over should be good enough )
If we have facility to scratch build, do we need a "-devel" tag as we can do scratch build as many time we want? As I mentioned in a previous mail, we can promote a build from -test to -production after it meets certain criteria. Building it for multiple targets will be overhead IMO.
Thanks, Lala #lalatenduM
On 07/15/2014 04:15 PM, Lalatendu Mohanty wrote:
On 07/15/2014 04:37 PM, Karanbir Singh wrote:
On 07/15/2014 09:49 AM, Thomas Oulevey wrote:
So my proposal is:
For the build tags : <SIG><MAJOR>-<TAG>[-<COLLECTION>,]-build And the target : <SIG><MAJOR>-<TAG>[-<COLLECTION>,] The destination can be <SIG><MAJOR>-testing/<SIG><MAJOR>-release for all the target. (or keep the Fedora naming dist-*, but it looks more confusing to me)
I think this works, the only consideration is that we might need to ( we really should ) give the SIG's a -devel a -testing and -production target, so they can build and release into vairous repos.
At the very least we would need a -devel where they can do whatever, and a -testing where they can then push builds that need to be made public on the repos, and once some criteria is satisfied, we can push content from -testing to -production ( I dont think we need a rebuild for that, just moving the rpms over should be good enough )
If we have facility to scratch build, do we need a "-devel" tag as we can do scratch build as many time we want? As I mentioned in a previous mail, we can promote a build from -test to -production after it meets certain criteria. Building it for multiple targets will be overhead IMO.
that works too - the important thing is to have some process that people can use to do these builds
On 07/15/2014 09:05 PM, Karanbir Singh wrote:
On 07/15/2014 04:15 PM, Lalatendu Mohanty wrote:
On 07/15/2014 04:37 PM, Karanbir Singh wrote:
On 07/15/2014 09:49 AM, Thomas Oulevey wrote:
So my proposal is:
For the build tags : <SIG><MAJOR>-<TAG>[-<COLLECTION>,]-build And the target : <SIG><MAJOR>-<TAG>[-<COLLECTION>,] The destination can be <SIG><MAJOR>-testing/<SIG><MAJOR>-release for all the target. (or keep the Fedora naming dist-*, but it looks more confusing to me)
I think this works, the only consideration is that we might need to ( we really should ) give the SIG's a -devel a -testing and -production target, so they can build and release into vairous repos.
At the very least we would need a -devel where they can do whatever, and a -testing where they can then push builds that need to be made public on the repos, and once some criteria is satisfied, we can push content from -testing to -production ( I dont think we need a rebuild for that, just moving the rpms over should be good enough )
If we have facility to scratch build, do we need a "-devel" tag as we can do scratch build as many time we want? As I mentioned in a previous mail, we can promote a build from -test to -production after it meets certain criteria. Building it for multiple targets will be overhead IMO.
that works too - the important thing is to have some process that people can use to do these builds
Yup, agree. Thats where bodhi like automated system comes in to picture. Where it forces certain regulations (what ever community decides) on the package without which we can't move packages between repos. We can discuss about it more and decide what system we want to use for build promotion.
On Jul 15 16:35, Karanbir Singh wrote:
On 07/15/2014 04:15 PM, Lalatendu Mohanty wrote:
On 07/15/2014 04:37 PM, Karanbir Singh wrote:
On 07/15/2014 09:49 AM, Thomas Oulevey wrote:
So my proposal is:
For the build tags : <SIG><MAJOR>-<TAG>[-<COLLECTION>,]-build And the target : <SIG><MAJOR>-<TAG>[-<COLLECTION>,] The destination can be <SIG><MAJOR>-testing/<SIG><MAJOR>-release for all the target. (or keep the Fedora naming dist-*, but it looks more confusing to me)
I think this works, the only consideration is that we might need to ( we really should ) give the SIG's a -devel a -testing and -production target, so they can build and release into vairous repos.
At the very least we would need a -devel where they can do whatever, and a -testing where they can then push builds that need to be made public on the repos, and once some criteria is satisfied, we can push content from -testing to -production ( I dont think we need a rebuild for that, just moving the rpms over should be good enough )
If we have facility to scratch build, do we need a "-devel" tag as we can do scratch build as many time we want? As I mentioned in a previous mail, we can promote a build from -test to -production after it meets certain criteria. Building it for multiple targets will be overhead IMO.
that works too - the important thing is to have some process that people can use to do these builds
+1 for koji+bodhi. Rpkg already speaks both, but of course we can always do our own thing if we want. Possible, -1 for bodhi because I'm not sure the karma-gathering process maps well to the more centralized promotion workflow (presumably promotions fall to the core team?). Anyone with more bodhi experience care to chime in about karma?
Brian
-- Brian Stinson bstinson@ksu.edu | IRC: bstinson | Bitbucket/Twitter: bstinsonmhk
On 07/15/2014 04:53 PM, Brian Stinson wrote:
+1 for koji+bodhi. Rpkg already speaks both, but of course we can always do our own thing if we want. Possible, -1 for bodhi because I'm not sure the karma-gathering process maps well to the more centralized promotion workflow (presumably promotions fall to the core team?). Anyone with more bodhi experience care to chime in about karma?
how well has the karma thing worked for epel / fedora ?
On Tue, Jul 15, 2014 at 11:10 AM, Karanbir Singh mail-lists@karan.org wrote:
On 07/15/2014 04:53 PM, Brian Stinson wrote:
+1 for koji+bodhi. Rpkg already speaks both, but of course we can always do our own thing if we want. Possible, -1 for bodhi because I'm not sure the karma-gathering process maps well to the more centralized promotion workflow (presumably promotions fall to the core team?). Anyone with more bodhi experience care to chime in about karma?
how well has the karma thing worked for epel / fedora ?
The 'salt' package might be a good example to check in epel. It has a fairly frequent update schedule and a (sometimes) requirement to update the master ahead of the minions that makes delays awkward in a mixed environment. And I think it often lingers in epel-testing for a couple of weeks.
On 07/15/2014 02:19 PM, Thomas Oulevey wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hello Folks,
As discussed in the past on this mailing list for the Centos build service we need to define different koji targets for the SIGs. I 'll try to present you a possible solution.
In the koji world, a "target" has a "build tag" and "destination tag". To simply the "build tag" is the buildroot and "destination tag" is the tag that will contain all the packages.
If you look at Fedora koji instance (http://koji.fedoraproject.org/koji/buildtargets) you see that it is quite simple : fXX target and the associated "build tag" is fXX-build. That implies they build only packages with a single "disttag" (from the macro file) for a "destination tag".
As we want to provide for each "sig" a set of targets we need to think about the naming.
The SIGs will be able to choose a "disttag" ; which can be default el7.centos, el7 or something else.
Let's say we have SIG="storage" DISTRIBUTION="el7".
storage7-el7-build storage7-el7.centos-build storage7-el7_0-build
On top of that some products we want to build/rebuild use software collection so we will need an additional buildroot (e.g: *python33-build):
storage7-el7-python33-build storage7-el7_0-python33-build
I am not sure if I understand you correctly. Is it about using packages which are not part of build-tag but have been built using koji for a SIG. This is solved in Fedora by using "BuildRootOverride" [1]. I think koji+bodhi work flow works pretty well here.
[1] https://fedoraproject.org/wiki/Bodhi/BuildRootOverrides
This part is needed for some rebuild, if the packaging is done correctly we should not need more than Requires:COLLECTION-build in the spec files.
So my proposal is:
For the build tags : <SIG><MAJOR>-<TAG>[-<COLLECTION>,]-build And the target : <SIG><MAJOR>-<TAG>[-<COLLECTION>,] The destination can be <SIG><MAJOR>-testing/<SIG><MAJOR>-release for all the target. (or keep the Fedora naming dist-*, but it looks more confusing to me)
What about packages which are needed to multiple SIGs? For example a newer libvirt package which required to "virtualization", "cloud" and "storage" SIG? For example if storage SIG have built a libvirt and other SIGs want to use the same package in their buildroot.
Maybe we can find something better, let me know.
We should keep the destination tags to small number as much as possible and should use a build promoting mechanism i.e. build for testing repo, then promote it to stable repo when it is ready.
I think we need to take decision on the naming so we can integrate with centpkg (don't forget about tomorrow centpkg demo : http://lists.centos.org/pipermail/centos-devel/2014-July/011474.html)
The tools need to be able depending on the "disttag" to use the correct "target" on the koji CLI, so it will be transparent to build service users. e.g: koji build storage7-el7.centos https://git.centos.org/pkg?#tag I presented this as our SIG use case but please keep in mind that we will have the same issues in the future ("Force sig to use 1 disttag only" argument) if we want to add more rebuild use cases (scl, devtoolset, etc...)
I hope I did not confused you more :-) I am waiting for your feedback.
On 15/07/14 19:23, Trevor Hemsley wrote:
On 15/07/14 09:49, Thomas Oulevey wrote:
Let's say we have SIG="storage" DISTRIBUTION="el7".
storage7-el7-build storage7-el7.centos-build storage7-el7_0-build
A minor cosmetic point but isn't one of those 7's redundant? Will there be a storage8-el7? Why not just storage-el7?
Agreed if we enforce a sig to have "major" in the "disttag", but it may not be always the case.
T
CentOS-devel mailing list CentOS-devel@centos.org http://lists.centos.org/mailman/listinfo/centos-devel
-
Brian Stinson -
Karanbir Singh -
Lalatendu Mohanty -
Les Mikesell -
Thomas Oulevey -
Trevor Hemsley