Hi All,
In the CBS/Infra meeting on Monday we agreed to start a discussion here on the mailing list about how to handle "ad-hoc" upstreams. An ad-hoc upstream could best be described as a project we would like to ship that is developed within the CentOS community (centpkg is one example).
Developers of an ad-hoc upstream need some extra infra (e.g. a git repo for doing active development) in addition to the dist-git repo on git.centos.org where the package specs live.
I would like to start the policy and procedure discussion with the following proposals:
- Host the ad-hoc development repositories on git.centos.org in separate Gitblit projects
- Host the ad-hoc development repositories on Github, linked to the CentOS project group
- Host the ad-hoc development repositories someplace else?
Thanks! Brian
-- Brian Stinson bstinson@ksu.edu | IRC: bstinson | Bitbucket/Twitter: bstinsonmhk
On Wed, Oct 22, 2014 at 10:42 AM, Brian Stinson bstinson@ksu.edu wrote:
Hi All,
In the CBS/Infra meeting on Monday we agreed to start a discussion here on the mailing list about how to handle "ad-hoc" upstreams. An ad-hoc upstream could best be described as a project we would like to ship that is developed within the CentOS community (centpkg is one example).
Is there some reason the work cannot or should not go into getting the packages into Fedora and EPEL? I realize there's a much more professional relationship with RHEL now that git.centos.org is the RHEL 7 publication repo, and that EPEL will not publish tools that overlap with RHEL upstream and thus overlap with CentOS, for a lot of very good reasons.
Developers of an ad-hoc upstream need some extra infra (e.g. a git repo for doing active development) in addition to the dist-git repo on git.centos.org where the package specs live.
I would like to start the policy and procedure discussion with the following proposals:
Host the ad-hoc development repositories on git.centos.org in separate Gitblit projects
Host the ad-hoc development repositories on Github, linked to the CentOS project group
Host the ad-hoc development repositories someplace else?
Thanks! Brian
It takes some work if you're overlapping the core OS packages, work that I'm sure CentOS developers are familiar with. I personally publish toolsl for that for RT version 4 and Samba version 4 at https://github.com/nkadel/.
On Oct 22 23:19, Nico Kadel-Garcia wrote:
On Wed, Oct 22, 2014 at 10:42 AM, Brian Stinson bstinson@ksu.edu wrote:
Hi All,
In the CBS/Infra meeting on Monday we agreed to start a discussion here on the mailing list about how to handle "ad-hoc" upstreams. An ad-hoc upstream could best be described as a project we would like to ship that is developed within the CentOS community (centpkg is one example).
Is there some reason the work cannot or should not go into getting the packages into Fedora and EPEL? I realize there's a much more professional relationship with RHEL now that git.centos.org is the RHEL 7 publication repo, and that EPEL will not publish tools that overlap with RHEL upstream and thus overlap with CentOS, for a lot of very good reasons.
Developers of an ad-hoc upstream need some extra infra (e.g. a git repo for doing active development) in addition to the dist-git repo on git.centos.org where the package specs live.
I would like to start the policy and procedure discussion with the following proposals:
Host the ad-hoc development repositories on git.centos.org in separate Gitblit projects
Host the ad-hoc development repositories on Github, linked to the CentOS project group
Host the ad-hoc development repositories someplace else?
Thanks! Brian
It takes some work if you're overlapping the core OS packages, work that I'm sure CentOS developers are familiar with. I personally publish toolsl for that for RT version 4 and Samba version 4 at https://github.com/nkadel/.
I can't speak for the others who were looking for this sort of workflow, but it makes sense to host upstream development (that is, the day-to-day commits) for Centpkg and centos-packager someplace the CentOS project controls.
Originally I was in favor of having the stuff we develop go in separate projects on git.c.o but that may end up being extra administrative work for the infra team.
Brian
-- Brian Stinson bstinson@ksu.edu | IRC: bstinson | Bitbucket/Twitter: bstinsonmhk
On 10/22/2014 10:19 PM, Nico Kadel-Garcia wrote:
On Wed, Oct 22, 2014 at 10:42 AM, Brian Stinson bstinson@ksu.edu wrote:
Hi All,
In the CBS/Infra meeting on Monday we agreed to start a discussion here on the mailing list about how to handle "ad-hoc" upstreams. An ad-hoc upstream could best be described as a project we would like to ship that is developed within the CentOS community (centpkg is one example).
Is there some reason the work cannot or should not go into getting the packages into Fedora and EPEL? I realize there's a much more professional relationship with RHEL now that git.centos.org is the RHEL 7 publication repo, and that EPEL will not publish tools that overlap with RHEL upstream and thus overlap with CentOS, for a lot of very good reasons.
For some projects, sadly yes there is. Fedora's not approved or come up with any policy around software collections yet. For some projects that rely heavily on software collections, this is problematic. In other cases, the code may already be in fedora but simply needs to be 'brought back' into the EL ecosystem. Further, things like the Xen4CentOS project may need an updated or patched libvirt, where there may not be any interest upstream because Xen isn't supported there.
Docker is also example of needing something newer. We're working to try to keep an updated 'upstream' docker instance available for users who want it. Right now in -Extras we have 0.11 (soon to be 1.1.2) in keeping with RH, however on the community side there's a tremendous push for a newer docker. Lokesh (the fedora dev responsible for docker) has been working with us to ensure that we have a current build as well. In this case the code lands upstream in fedora first, and is then built for CentOS as appropriate.
Fedora should absolutely stay the upstream, but it doesn't always make sense in every single case.
Developers of an ad-hoc upstream need some extra infra (e.g. a git repo for doing active development) in addition to the dist-git repo on git.centos.org where the package specs live.
I would like to start the policy and procedure discussion with the following proposals:
Host the ad-hoc development repositories on git.centos.org in separate Gitblit projects
Host the ad-hoc development repositories on Github, linked to the CentOS project group
Host the ad-hoc development repositories someplace else?
Thanks! Brian
It takes some work if you're overlapping the core OS packages, work that I'm sure CentOS developers are familiar with. I personally publish toolsl for that for RT version 4 and Samba version 4 at https://github.com/nkadel/.
It does indeed. This circles back to a discussion several months ago on this list about tiered repositories, best practices, etc. We've known for a while that this would come up, but before it was mostly just discussion and theory. Now we're looking at putting it into practice with deliverable content.
On Thu, Oct 23, 2014 at 7:02 AM, Jim Perrin jperrin@centos.org wrote:
It does indeed. This circles back to a discussion several months ago on this list about tiered repositories, best practices, etc. We've known for a while that this would come up, but before it was mostly just discussion and theory. Now we're looking at putting it into practice with deliverable content.
So was/is there a theoretical resolution of how to handle tiered/alternative packages either where concurrent installations conflict or where concurrent installations are desirable along with a way to distinguish which instance you want to run. In this context, I'm not sure it is even relevant whether or not these packages come from different repositories.
On Thu, Oct 23, 2014 at 2:44 PM, Les Mikesell lesmikesell@gmail.com wrote:
On Thu, Oct 23, 2014 at 7:02 AM, Jim Perrin jperrin@centos.org wrote:
It does indeed. This circles back to a discussion several months ago on this list about tiered repositories, best practices, etc. We've known for a while that this would come up, but before it was mostly just discussion and theory. Now we're looking at putting it into practice with deliverable content.
So was/is there a theoretical resolution of how to handle tiered/alternative packages either where concurrent installations conflict or where concurrent installations are desirable along with a way to distinguish which instance you want to run. In this context, I'm not sure it is even relevant whether or not these packages come from different repositories.
The only completely reliable way to do this is user-tuning of exclusions in the local configurations. As long as you have only a few well built repositories that are careful not to overlap the core, such as EPEL and RPMfusion enabled, or a well defined target repository such as I publish hooks for Samba 4, you're OK. But as soon as you start updating a lot of Perl modules to resolve Perl dependency hell, for example to update something like RT 4 or MusicBrainz, or to introduce overlapping 3rd party repositories like JPackage, well, you're in for adventures. It's no longer plug and play, and you have to be ready to set exclusions in other repository configurations or make plugins to provide compatibility.
Particular examples for particular packages include jpackage-utils from JPackage, and the rather incompatible layouts of nagios-plugins from Repoforge and from EPEL. Managing that sort of thing can get very complex very quickly if you want unlimited access to all such repos, and don't want to allow your setups for one to touch your setups for the other.
Nico Kadel-Garcia
On Fri, Oct 24, 2014 at 7:20 AM, Nico Kadel-Garcia nkadel@gmail.com wrote:
So was/is there a theoretical resolution of how to handle tiered/alternative packages either where concurrent installations conflict or where concurrent installations are desirable along with a way to distinguish which instance you want to run. In this context, I'm not sure it is even relevant whether or not these packages come from different repositories.
The only completely reliable way to do this is user-tuning of exclusions in the local configurations. As long as you have only a few well built repositories that are careful not to overlap the core, such as EPEL and RPMfusion enabled, or a well defined target repository such as I publish hooks for Samba 4, you're OK. But as soon as you start updating a lot of Perl modules to resolve Perl dependency hell, for example to update something like RT 4 or MusicBrainz, or to introduce overlapping 3rd party repositories like JPackage, well, you're in for adventures. It's no longer plug and play, and you have to be ready to set exclusions in other repository configurations or make plugins to provide compatibility.
Particular examples for particular packages include jpackage-utils from JPackage, and the rather incompatible layouts of nagios-plugins from Repoforge and from EPEL. Managing that sort of thing can get very complex very quickly if you want unlimited access to all such repos, and don't want to allow your setups for one to touch your setups for the other.
Yes, that may be the best you can do now, but it is mostly a matter of luck since it relies on unknown future changes in likely unrelated repositories. A real solution would work even with all the packages in the same repository where the packaged could be maintained to always work.
hi,
On 10/22/2014 03:42 PM, Brian Stinson wrote:
Hi All,
In the CBS/Infra meeting on Monday we agreed to start a discussion here on the mailing list about how to handle "ad-hoc" upstreams. An ad-hoc upstream could best be described as a project we would like to ship that is developed within the CentOS community (centpkg is one example).
to be clear, these upstreams are code repos, not dist-git like repos to host rpm packages; some of these might not even be targetting rpms ( eg. the test suites that a SIg develops or any scripting that they need, docs etc would also be included in this )
Developers of an ad-hoc upstream need some extra infra (e.g. a git repo for doing active development) in addition to the dist-git repo on git.centos.org where the package specs live.
I would like to start the policy and procedure discussion with the following proposals:
- Host the ad-hoc development repositories on git.centos.org in separate Gitblit projects
+1
- Host the ad-hoc development repositories on Github, linked to the CentOS project group
For projects that want to, we can share namespace under either github.com/CentOS or /CentOSProject
- Host the ad-hoc development repositories someplace else?
if the target is CentOS and the larger EL ecosystem around it, hosting the end result git repos on git.centos.org ( or atleast hosting it via a mirror from somewhere else ) would be the way to go, I think. It also reduces the barrier to sync releases and dist-git content when needed.
- KB
On Oct 24 14:36, Karanbir Singh wrote:
hi,
On 10/22/2014 03:42 PM, Brian Stinson wrote:
Hi All,
In the CBS/Infra meeting on Monday we agreed to start a discussion here on the mailing list about how to handle "ad-hoc" upstreams. An ad-hoc upstream could best be described as a project we would like to ship that is developed within the CentOS community (centpkg is one example).
to be clear, these upstreams are code repos, not dist-git like repos to host rpm packages; some of these might not even be targetting rpms ( eg. the test suites that a SIg develops or any scripting that they need, docs etc would also be included in this )
Developers of an ad-hoc upstream need some extra infra (e.g. a git repo for doing active development) in addition to the dist-git repo on git.centos.org where the package specs live.
I would like to start the policy and procedure discussion with the following proposals:
- Host the ad-hoc development repositories on git.centos.org in separate Gitblit projects
+1
- Host the ad-hoc development repositories on Github, linked to the CentOS project group
For projects that want to, we can share namespace under either github.com/CentOS or /CentOSProject
- Host the ad-hoc development repositories someplace else?
if the target is CentOS and the larger EL ecosystem around it, hosting the end result git repos on git.centos.org ( or atleast hosting it via a mirror from somewhere else ) would be the way to go, I think. It also reduces the barrier to sync releases and dist-git content when needed.
- KB
-- Karanbir Singh +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh GnuPG Key : http://www.karan.org/publickey.asc _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org http://lists.centos.org/mailman/listinfo/centos-devel
Another plus for hosting on git.c.o is that it encourages discussion here on the mailing list, instead of fragmenting into multiple streams of communication (as would happen with github pull requests).
What are some other examples of projects that would benefit from this workflow?
Brian
-- Brian Stinson bstinson@ksu.edu | IRC: bstinson | Bitbucket/Twitter: bstinsonmhk
On 24/10/14 16:07, Brian Stinson wrote:
What are some other examples of projects that would benefit from this workflow?
test suites for specific stuff ( eg. t_xen that checks if a new xen stack + kernel can still provision and run xen vm's ), documentation and code that maps to centos related efforts ( like centpkg and its dep chain ); these are all examples of the sort of projects I would expect to see using git.c.o as their upstream dev git's.
Also, SIG's have shown interest in moving some upstream content into git.centos.org - eg. the most recent conversations in the Atomic SIG about having some of their devel repos be moved into git.centos.org - and potentially doing some automated CI / testing around it as well.
- KB
On 10/27/2014 02:23 PM, Karanbir Singh wrote:
On 24/10/14 16:07, Brian Stinson wrote:
What are some other examples of projects that would benefit from this workflow?
test suites for specific stuff ( eg. t_xen that checks if a new xen stack + kernel can still provision and run xen vm's ), documentation and code that maps to centos related efforts ( like centpkg and its dep chain ); these are all examples of the sort of projects I would expect to see using git.c.o as their upstream dev git's.
Also, SIG's have shown interest in moving some upstream content into git.centos.org - eg. the most recent conversations in the Atomic SIG about having some of their devel repos be moved into git.centos.org - and potentially doing some automated CI / testing around it as well.
Hi all - I don't really have input into the "how" of this, but I do have questions about the "when" - as in, do we have a target to coming to consensus, someone driving this, and ensuring it wraps up in a timely fashion.
(KB's comment is the last one I can find on the thread, and that was a week ago - so I'm concerned this has stalled without coming to resolution.)
Thanks,
jzb
On 11/04/2014 03:04 AM, Joe Brockmeier wrote:
Hi all - I don't really have input into the "how" of this, but I do have questions about the "when" - as in, do we have a target to coming to consensus, someone driving this, and ensuring it wraps up in a timely fashion.
(KB's comment is the last one I can find on the thread, and that was a week ago - so I'm concerned this has stalled without coming to resolution.)
I think we should be able to do both, arbitary git repos and dist-git format repos for upstream / sig specific repos in git.centos.org by the end of next week. But we need to workout how we will signal cbs.centos.org to consume dist-git style repos hosted in this manner.
Lets bring this up in the next buildsys meeting ( should be Monday 14:00 UTC ). As long as were not doing something that will need a migration later on, we should go able to start rolling in the upstreams fairly soon.
- KB
-
Brian Stinson -
Jim Perrin -
Joe Brockmeier -
Karanbir Singh -
Les Mikesell -
Nico Kadel-Garcia