Hi Niels and Kaleb,
I'm wondering how we could best build nfs-ganesha's ceph FSALs in CBS.
Right now, since we'll likely support multiple versions of nfs-ganesha simultaneously for different versions of Ceph, I'm leaning towards using a custom "dist tag" like thing in the Release field so we don't collide with what Kaleb's doing at http://cbs.centos.org/koji/packageinfo?packageID=98
Eventually it would be nice to have a unified nfs-ganesha build shared between Ceph and Gluster. If we built it in storage7-common-candidate, we would have to tag gluster and ceph into that, which sounds kinda messy, because you only get to choose one version of each. It'd be more manageable to have a separate set of tags and targets just for nfs-ganesha versions. Like:
storage7-nfs-ganesha-26-el7 storage7-nfs-ganesha-27-el7 ... etc
But that still leaves the problem of shipping a nfs-ganesha-gluster sub-package that will not have its runtime deps fulfilled in http://mirror.centos.org/centos/7/storage/x86_64/ceph-nautilus/ . Vice versa, we'd have a nfs-ganesha-ceph sub-package that lacks runtime deps in http://mirror.centos.org/centos/7/storage/x86_64/gluster-5/
What do you think? Just have separate nfs-ganesha builds for now?
- Ken
On 12/4/18 5:38 PM, Ken Dreyer wrote:
Hi Niels and Kaleb,
I'm wondering how we could best build nfs-ganesha's ceph FSALs in CBS.
Right now, since we'll likely support multiple versions of nfs-ganesha simultaneously for different versions of Ceph, I'm leaning towards using a custom "dist tag" like thing in the Release field so we don't collide with what Kaleb's doing at http://cbs.centos.org/koji/packageinfo?packageID=98
Eventually it would be nice to have a unified nfs-ganesha build shared between Ceph and Gluster. If we built it in storage7-common-candidate, we would have to tag gluster and ceph into that, which sounds kinda messy, because you only get to choose one version of each. It'd be more manageable to have a separate set of tags and targets just for nfs-ganesha versions. Like:
storage7-nfs-ganesha-26-el7 storage7-nfs-ganesha-27-el7 ... etc
But that still leaves the problem of shipping a nfs-ganesha-gluster sub-package that will not have its runtime deps fulfilled in http://mirror.centos.org/centos/7/storage/x86_64/ceph-nautilus/ . Vice versa, we'd have a nfs-ganesha-ceph sub-package that lacks runtime deps in http://mirror.centos.org/centos/7/storage/x86_64/gluster-5/
I too would like to get to a place where we have a single ganesha package.
Gluster's GFAPI guarantees its ABI by using versioned symbols. A Gluster FSAL built against glusterfs-4.1 (or earlier) will work with glusterfs-5's libgfapi.so. And glusterfs-6's. And so on.
So we don't build separate ganesha-2.7 packages for both glusterfs-4.1 and glusterfs-5. We build a single ganesha-2.7 package with glusterfs-4.1 and it is tagged into both the glusterfs-4.1 and glusterfs-5 releases.
I think it should be achievable. I try and build ganesha with as many FSALs enabled as possible. It's been a while though since I looked at which version of Ceph was in CBS for the Storage SIG.
If Ceph doesn't guarantee its ABIs though that's going to make things harder.
--
Kaleb
On Tue, Dec 4, 2018 at 4:09 PM Kaleb S. KEITHLEY kkeithle@redhat.com wrote:
I think it should be achievable. I try and build ganesha with as many FSALs enabled as possible. It's been a while though since I looked at which version of Ceph was in CBS for the Storage SIG.
If Ceph doesn't guarantee its ABIs though that's going to make things harder.
I confirmed with Matt Benjamin on ceph-devel today that this ok. We'll be able to build nfs-ganesha-2.7 against Luminous and then cross-tag that into our newer releases (eg mimic and nautilus). So we can follow the same model that you're following with building nfs-ganesha in the "oldest supported" build target.
In terms of sharing one single nfs-ganesha build across ceph and gluster, I think that will require more work. We'd probably want:
1) A custom nfs-ganesha "-build" tag/target that contains the "oldest supported" versions of both ceph and gluster,
2) cross-tag the nfs-ganesha builds from there into both the ceph and gluster -testing tags,
3) Update the mash configuration to exclude the irrelevant sub-packages when building ceph and gluster's "testing" and "release" repositories.
Unfortunately that last bit looks really tricky. I'm not sure there is a way to tell mash itself to exclude a particular sub-package, and https://git.centos.org/blob/sig-core!cbs-tools.git/master/scripts!mash_run.s... uses a basic mash config template for all CBS tags.
- Ken
On Wed, Dec 05, 2018 at 09:11:58AM -0700, Ken Dreyer wrote:
On Tue, Dec 4, 2018 at 4:09 PM Kaleb S. KEITHLEY kkeithle@redhat.com wrote:
I think it should be achievable. I try and build ganesha with as many FSALs enabled as possible. It's been a while though since I looked at which version of Ceph was in CBS for the Storage SIG.
If Ceph doesn't guarantee its ABIs though that's going to make things harder.
I confirmed with Matt Benjamin on ceph-devel today that this ok. We'll be able to build nfs-ganesha-2.7 against Luminous and then cross-tag that into our newer releases (eg mimic and nautilus). So we can follow the same model that you're following with building nfs-ganesha in the "oldest supported" build target.
In terms of sharing one single nfs-ganesha build across ceph and gluster, I think that will require more work. We'd probably want:
A custom nfs-ganesha "-build" tag/target that contains the "oldest supported" versions of both ceph and gluster,
cross-tag the nfs-ganesha builds from there into both the ceph and gluster -testing tags,
Update the mash configuration to exclude the irrelevant sub-packages when building ceph and gluster's "testing" and "release" repositories.
Unfortunately that last bit looks really tricky. I'm not sure there is a way to tell mash itself to exclude a particular sub-package, and https://git.centos.org/blob/sig-core!cbs-tools.git/master/scripts!mash_run.s... uses a basic mash config template for all CBS tags.
This is something that I'm very much looking forward to see happening. The 3rd point is indeed something that (from my understanding) is not possible. I have thought of an alternative, but that is still a little ugly. Not sure if we want to go that route.
- add a new repository with all nfs-ganesha-x.y packages (direct dependencies and sub-packages), without ceph/gluster
- add a new repository definition in centos-release-gluster and centos-release-ceph packages. This entry in the .repo file should use the 'exclude=' directive to skip the Ceph sub-package for Gluster, and the Gluster sub-package for Ceph.
For additional projects (like Samba) this would work in a similar way.
Opinions? Niels
On Wed, Dec 5, 2018 at 9:53 AM Niels de Vos ndevos@redhat.com wrote:
Opinions?
The biggest reason I want to exclude the package server side is that will allow repoclosure to pass for the repo. repoclosure provides me a sanity-check that the entire "product" is installable and I'm not missing any packages. With the way we're growing more and more packages, I want to make this as automated as possible. Ideally repoclosure would automatically gate a package going from -candidate to -testing.
One problem I see with configuring this exclusion client side is that I'm not sure I can rely on everyone using the exact same .repo file. I eventually want to ship packaged mock configs for Ceph, so we'd have to duplicate that there as well :(
I'd like to know if the CentOS admins plan to stay on mash long-term, and if so, let's look into filtering at that level, like Pungi can do. What do you think?
- Ken
On Wed, Dec 05, 2018 at 10:47:53AM -0700, Ken Dreyer wrote:
On Wed, Dec 5, 2018 at 9:53 AM Niels de Vos ndevos@redhat.com wrote:
Opinions?
The biggest reason I want to exclude the package server side is that will allow repoclosure to pass for the repo. repoclosure provides me a sanity-check that the entire "product" is installable and I'm not missing any packages. With the way we're growing more and more packages, I want to make this as automated as possible. Ideally repoclosure would automatically gate a package going from -candidate to -testing.
Yes, I understand that. This is an item on my wishlist for the Gluster repositories for a looong time already.
One problem I see with configuring this exclusion client side is that I'm not sure I can rely on everyone using the exact same .repo file. I eventually want to ship packaged mock configs for Ceph, so we'd have to duplicate that there as well :(
Ah, yes. I have my mock-config files from the 'cbs' tool, but it is not friendly for others to use. Packaging them would be really nice too.
I'd like to know if the CentOS admins plan to stay on mash long-term, and if so, let's look into filtering at that level, like Pungi can do. What do you think?
This definitely would have my preference as well!
Thanks, Niels
-
Kaleb S. KEITHLEY -
Ken Dreyer -
Niels de Vos