At the moment, we're essentially using yum repos as the "patch" mechanism to CentOS Base: SIGs populate a repo with packages they want to add / override, and all other packages default to Base.
The current setup is that every sig has exactly one production repo per CentOS version.
Also, we seem to be wanting to see SIGs not as individual projects, but to make the 'G' an actual "group": so, for instance, the Virt SIG has both Xen and Docker, and we've recently approved oVirt.
But there's a bit of a conflict here: what if one project in the SIG wants to use the package from Base, and another project in the SIG wants to use a custom-built package? Or what if both projects want to customize their packages in different ways?
In particular: the Docker project in the Virt SIG wants to supply updated versions of Docker; but at the moment uses the kernel package from Base. The Xen project, on the other hand, needs to supply a different kernel package that has dom0 support enabled.
Up until this point, this hasn't been an issue because I've been doing Xen development in virt6-testing, and Lokesh has been doing Docker development in virt7-testing. But I'm hoping to get Xen for CentOS 7 out soon, and I was just about to upload the Xen kernel to the CBS, when it occurred to me that if I did so, then everyone using the virt7-testing repo for Docker would suddenly get my updated kernel as well.
So it seems we have a couple of ways to approach it.
1. Keep it one repo per SIG, and make all projects in a SIG sort out shared dependencies.
2. Allow one repo per *project* within a SIG (e.g., virt6-xen, virt7-docker)
3. Have some alternate way of doing a "patch", instead of repos.
In the case of xen / docker, I haven't talked with Lokesh about this at all; it may be that he's fine with sharing a kernel with Xen. But it does increase the workload: now Lokesh needs to maintain the Docker side of the kernel, whereas before they could just rely on our upstream to do a lot of that testing. Furthermore, it increases the complexity of development and testing: before when doing kernel work, I could just test Xen; now the kernel needs to be tested by both Xen and Docker. And finally, now users of both projects get a kernel update whenever either one does.
Which, of course, is pretty much the same as any other distro; but as a whole, at the moment at least, we have a lot less resources than any other distro our size and popularity.
On the whole I'm inclined to prefer #2.
Thoughts?
-George
On 01/19/2015 05:38 PM, George Dunlap wrote:
At the moment, we're essentially using yum repos as the "patch" mechanism to CentOS Base: SIGs populate a repo with packages they want to add / override, and all other packages default to Base.
The current setup is that every sig has exactly one production repo per CentOS version.
Also, we seem to be wanting to see SIGs not as individual projects, but to make the 'G' an actual "group": so, for instance, the Virt SIG has both Xen and Docker, and we've recently approved oVirt.
But there's a bit of a conflict here: what if one project in the SIG wants to use the package from Base, and another project in the SIG wants to use a custom-built package? Or what if both projects want to customize their packages in different ways?
In particular: the Docker project in the Virt SIG wants to supply updated versions of Docker; but at the moment uses the kernel package from Base. The Xen project, on the other hand, needs to supply a different kernel package that has dom0 support enabled.
Up until this point, this hasn't been an issue because I've been doing Xen development in virt6-testing, and Lokesh has been doing Docker development in virt7-testing. But I'm hoping to get Xen for CentOS 7 out soon, and I was just about to upload the Xen kernel to the CBS, when it occurred to me that if I did so, then everyone using the virt7-testing repo for Docker would suddenly get my updated kernel as well.
Yup, for e.g. in Storage SIG we are going have extra kernel modules , may be a rebuilt kernel in future. So we are having couple of SIGs which are having or going to have modified kernels.
So it seems we have a couple of ways to approach it.
- Keep it one repo per SIG, and make all projects in a SIG sort out
shared dependencies.
Allow one repo per *project* within a SIG (e.g., virt6-xen, virt7-docker)
Have some alternate way of doing a "patch", instead of repos.
In the case of xen / docker, I haven't talked with Lokesh about this at all; it may be that he's fine with sharing a kernel with Xen. But it does increase the workload: now Lokesh needs to maintain the Docker side of the kernel, whereas before they could just rely on our upstream to do a lot of that testing. Furthermore, it increases the complexity of development and testing: before when doing kernel work, I could just test Xen; now the kernel needs to be tested by both Xen and Docker. And finally, now users of both projects get a kernel update whenever either one does.
Which, of course, is pretty much the same as any other distro; but as a whole, at the moment at least, we have a lot less resources than any other distro our size and popularity.
I like #1, even if it is more work. I think it is logical thing to do as SIGs should not conflict with each other when a user consumes them. I mean one of the purpose of SIGs to improve the user experience for consuming stuffs that are not present in CentOS core.
Having different build targets for each SIG I think complicate overall integration of different SIGs. I am sure there would be a lot of use cases when multiple SIGs would be used together.
-Lala
On the whole I'm inclined to prefer #2.
Thoughts?
-George _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org http://lists.centos.org/mailman/listinfo/centos-devel
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 19.01.2015 13:40, Lalatendu Mohanty wrote:
I like #1, even if it is more work. I think it is logical thing to do as SIGs should not conflict with each other when a user consumes them. I mean one of the purpose of SIGs to improve the user experience for consuming stuffs that are not present in CentOS core.
Being not a sig member but a user:
I dislike option 1 for several reasons:
1. it complicates development a _lot_ 2. I can't imagine any usecase where you e.g. want to mix xen-kernel with ovirt, as ovirt has no xen support (yet), so those don't need to be compatible (at least for now). 3. possibly adding more bloat: in order to satisfy this one-repo-for-all you might need to define some silly dependencies (libvirt requiring xen-kernel, when you don't need xen-kernel, but just ovirt/kvm).
kind regards
Sven
On 01/20/2015 12:03 AM, Sven Kieske wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 19.01.2015 13:40, Lalatendu Mohanty wrote:
I like #1, even if it is more work. I think it is logical thing to do as SIGs should not conflict with each other when a user consumes them. I mean one of the purpose of SIGs to improve the user experience for consuming stuffs that are not present in CentOS core.
Being not a sig member but a user:
I dislike option 1 for several reasons:
- it complicates development a _lot_
- I can't imagine any usecase where you e.g.
want to mix xen-kernel with ovirt, as ovirt has no xen support (yet), so those don't need to be compatible (at least for now).
What about an use-case where Xen would be used with Storage SIG i.e. GlusterFS or Ceph or OpenAFS? Also Ovirt can be used for management of GlusterFS.
- possibly adding more bloat:
in order to satisfy this one-repo-for-all you might need to define some silly dependencies (libvirt requiring xen-kernel, when you don't need xen-kernel, but just ovirt/kvm).
Honestly I don't want to complicate stuff but just concerned that user does not face these kind of issues when he tries to use multiple SIGs.
-Lala
kind regards
Sven -----BEGIN PGP SIGNATURE----- Version: GnuPG v2
iQGcBAEBAgAGBQJUvU3cAAoJEAq0kGAWDrqlIh8L/RM9NGbLIdYh1Mnfq/h0vCS1 bjWojMpiQUKaCxKsFjqAPtXSF6eXDXEiI8/Ymw9rMQMHY86WPzFltArf7hi/6phG NBn77hPt0QBu+KukIEPa5X6tf/xhTwf0HQw6tGBni4ynSCQVFu+soE5EFshPBZiI MpxEKlCJKUFMQVHXOkRQlr0bpvWN+eLWSBJ4k6bxW9SJsRsmXqWzhVhnurz/sCgx N+cj1h2FIHC7F/gkwXwO24Q4CFe4ahbUBvtZbFZl4rWFg8ZbywN96qRRuLnRiUPS hQH2YwvT4ojQL45uSp+4Z1vcyeGfyQzKY9k8yRVxg46vhfjKtNkPp6S7K2Vlg4o7 HGcXs1IhXLvP7RJvQaY1VHJ5Yg0br+wb6kjdbTggJrIZT7hyx7KFX68W/yA5ijmm oGB9Jggo/oTMYOVDpMiczvs8guJMXPC2lSS72kZ6zQNJz2nbDDFZFdPHxupdqcfK BWie6fyGv1Iu490VDwZy9pkrbPjW2NeLGbm/3dQk2Q== =jjzb -----END PGP SIGNATURE----- _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org http://lists.centos.org/mailman/listinfo/centos-devel
On Tue, Jan 20, 2015 at 8:47 AM, Lalatendu Mohanty lmohanty@redhat.com wrote:
On 01/20/2015 12:03 AM, Sven Kieske wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 19.01.2015 13:40, Lalatendu Mohanty wrote:
I like #1, even if it is more work. I think it is logical thing to do as SIGs should not conflict with each other when a user consumes them. I mean one of the purpose of SIGs to improve the user experience for consuming stuffs that are not present in CentOS core.
Being not a sig member but a user:
I dislike option 1 for several reasons:
- it complicates development a _lot_
- I can't imagine any usecase where you e.g.
want to mix xen-kernel with ovirt, as ovirt has no xen support (yet), so those don't need to be compatible (at least for now).
What about an use-case where Xen would be used with Storage SIG i.e. GlusterFS or Ceph or OpenAFS? Also Ovirt can be used for management of GlusterFS.
Cross-repo compatbility is along-standing problem. There isrt of a solution by using the "include" option in yum.conf to include something like '/etc/yum.excludes/[repo.exclude] to apply a set of excludes for the entire yum configuration. I've used this to protect the EPEL from RPMforge overlapping components, especially mismatched "nagios-plugins" and perl modules. The reason it does not work now with setting up /etc/yum.repos.d/[name].repo files is because those can't provide exclusions for the base configuration, or other repositories.
A more flexible includes structure for yum, much like the individual repo files but not restricted to actual repositories, could help this a lot.
- possibly adding more bloat:
in order to satisfy this one-repo-for-all you might need to define some silly dependencies (libvirt requiring xen-kernel, when you don't need xen-kernel, but just ovirt/kvm).
Honestly I don't want to complicate stuff but just concerned that user does not face these kind of issues when he tries to use multiple SIGs.
-Lala
See above. It would take one 'includes' line added to /etc/yum.conf, and a practice of allowing one repo's "repo-reelease" package to add exclusions. This would be enormously helpful for things like alternative MySQL repositories: activating the 'mysql-libe' exclusion to prevent that dependency from being "included" and updated by the Percona or MariaDB packages has been a configuration nightmare. In this case, it should be possible to exclude the default Xen packages from RHEL or CentOS and to enable the separate Xen repo kernels in an appropriate, possibly designated sub-repository with only the kernels and an "include" option.
That's a package management issue. Now, whether this much work on Xen makes any sense when Red Hat is explicitly pursuing KVM support is not so clear to me. Personally, I'd much rather see the effort spent on making virt-manager more sane, and network configuration tools for virtual hosts. Xen hypervisors, and other virtualization tools, all need pair-bonding, VLAN tagging, and some of them need network bridge configuration. *NONE* of the current generation of network configuration tools support these for either the hyperviros or for the virtual hosts. If someone out there with GUI design skills wants to work on that, I'd be thrilled to help support that for all virtualization environments.
But that's me. (And yes, I'm pretty familiar with Xen: I published the first publicly available RPM's for it back in 2007.)
On 01/19/2015 06:08 AM, George Dunlap wrote:
At the moment, we're essentially using yum repos as the "patch" mechanism to CentOS Base: SIGs populate a repo with packages they want to add / override, and all other packages default to Base.
The current setup is that every sig has exactly one production repo per CentOS version.
Also, we seem to be wanting to see SIGs not as individual projects, but to make the 'G' an actual "group": so, for instance, the Virt SIG has both Xen and Docker, and we've recently approved oVirt.
But there's a bit of a conflict here: what if one project in the SIG wants to use the package from Base, and another project in the SIG wants to use a custom-built package? Or what if both projects want to customize their packages in different ways?
In particular: the Docker project in the Virt SIG wants to supply updated versions of Docker; but at the moment uses the kernel package from Base. The Xen project, on the other hand, needs to supply a different kernel package that has dom0 support enabled.
Up until this point, this hasn't been an issue because I've been doing Xen development in virt6-testing, and Lokesh has been doing Docker development in virt7-testing. But I'm hoping to get Xen for CentOS 7 out soon, and I was just about to upload the Xen kernel to the CBS, when it occurred to me that if I did so, then everyone using the virt7-testing repo for Docker would suddenly get my updated kernel as well.
So it seems we have a couple of ways to approach it.
- Keep it one repo per SIG, and make all projects in a SIG sort out
shared dependencies.
Allow one repo per *project* within a SIG (e.g., virt6-xen, virt7-docker)
Have some alternate way of doing a "patch", instead of repos.
In the case of xen / docker, I haven't talked with Lokesh about this at all; it may be that he's fine with sharing a kernel with Xen. But it does increase the workload: now Lokesh needs to maintain the Docker side of the kernel, whereas before they could just rely on our upstream to do a lot of that testing. Furthermore, it increases the complexity of development and testing: before when doing kernel work, I could just test Xen; now the kernel needs to be tested by both Xen and Docker. And finally, now users of both projects get a kernel update whenever either one does.
Which, of course, is pretty much the same as any other distro; but as a whole, at the moment at least, we have a lot less resources than any other distro our size and popularity.
On the whole I'm inclined to prefer #2.
I believe option 2 was always the plan, with a few caveats.
1. We start with one repo to test procedures and make sure things are working that way first.
2. We work to keep repo sprawl to a minimum, lest sigs start creating a few dozen project repos when just a couple will do.
I mashed send without finishing the thought. Not enough coffee yet today.
On 01/19/2015 08:37 AM, Jim Perrin wrote:
On 01/19/2015 06:08 AM, George Dunlap wrote:
At the moment, we're essentially using yum repos as the "patch" mechanism to CentOS Base: SIGs populate a repo with packages they want to add / override, and all other packages default to Base.
The current setup is that every sig has exactly one production repo per CentOS version.
Also, we seem to be wanting to see SIGs not as individual projects, but to make the 'G' an actual "group": so, for instance, the Virt SIG has both Xen and Docker, and we've recently approved oVirt.
But there's a bit of a conflict here: what if one project in the SIG wants to use the package from Base, and another project in the SIG wants to use a custom-built package? Or what if both projects want to customize their packages in different ways?
In particular: the Docker project in the Virt SIG wants to supply updated versions of Docker; but at the moment uses the kernel package from Base. The Xen project, on the other hand, needs to supply a different kernel package that has dom0 support enabled.
Up until this point, this hasn't been an issue because I've been doing Xen development in virt6-testing, and Lokesh has been doing Docker development in virt7-testing. But I'm hoping to get Xen for CentOS 7 out soon, and I was just about to upload the Xen kernel to the CBS, when it occurred to me that if I did so, then everyone using the virt7-testing repo for Docker would suddenly get my updated kernel as well.
So it seems we have a couple of ways to approach it.
- Keep it one repo per SIG, and make all projects in a SIG sort out
shared dependencies.
Allow one repo per *project* within a SIG (e.g., virt6-xen, virt7-docker)
Have some alternate way of doing a "patch", instead of repos.
In the case of xen / docker, I haven't talked with Lokesh about this at all; it may be that he's fine with sharing a kernel with Xen. But it does increase the workload: now Lokesh needs to maintain the Docker side of the kernel, whereas before they could just rely on our upstream to do a lot of that testing. Furthermore, it increases the complexity of development and testing: before when doing kernel work, I could just test Xen; now the kernel needs to be tested by both Xen and Docker. And finally, now users of both projects get a kernel update whenever either one does.
Which, of course, is pretty much the same as any other distro; but as a whole, at the moment at least, we have a lot less resources than any other distro our size and popularity.
On the whole I'm inclined to prefer #2.
I believe option 2 was always the plan, with a few caveats.
- We start with one repo to test procedures and make sure things are
working that way first.
- We work to keep repo sprawl to a minimum, lest sigs start creating a
few dozen project repos when just a couple will do.
3. We work to consolidate common packages as appropriate in a shared repository. There is still some ongoing debate as to if this would be epel, Extras, or would require the creation of a sig-common repo. If a sig requires a version that cannot be shared, this should be documented and the sig would need to carry that lib in their repo(s).
On Mon, Jan 19, 2015 at 2:37 PM, Jim Perrin jperrin@centos.org wrote:
- Allow one repo per *project* within a SIG (e.g., virt6-xen, virt7-docker)
[snip]
On the whole I'm inclined to prefer #2.
I believe option 2 was always the plan, with a few caveats.
- We start with one repo to test procedures and make sure things are
working that way first.
- We work to keep repo sprawl to a minimum, lest sigs start creating a
few dozen project repos when just a couple will do.
Great! So I think we've probably hit the point in the Virt SIG (unless Lokesh wants to argue otherwise); how do we proceed to getting new repos, so I can actually start the Xen4CentOS7 work?
-George
On Tue, Jan 20, 2015 at 12:20 PM, George Dunlap dunlapg@umich.edu wrote:
On Mon, Jan 19, 2015 at 2:37 PM, Jim Perrin jperrin@centos.org wrote:
- Allow one repo per *project* within a SIG (e.g., virt6-xen, virt7-docker)
[snip]
On the whole I'm inclined to prefer #2.
I believe option 2 was always the plan, with a few caveats.
- We start with one repo to test procedures and make sure things are
working that way first.
- We work to keep repo sprawl to a minimum, lest sigs start creating a
few dozen project repos when just a couple will do.
Great! So I think we've probably hit the point in the Virt SIG (unless Lokesh wants to argue otherwise); how do we proceed to getting new repos, so I can actually start the Xen4CentOS7 work?
Ping?
-George
-
George Dunlap -
Jim Perrin -
Lalatendu Mohanty -
Nico Kadel-Garcia -
Sven Kieske