Hello Everyone,
I have been working on reworking the upstream spec file to work with centos. I've been keeping track of my progress on my github page [1] and I welcome pull requests if you find a problem.
I have pulled out all of the code related to the building the kmod, as I think that should at least be rewritten from scratch and quite possibly should live in it's own spec file. I did turn off the transarch paths configure option, which will break external scripts that expect the legacy paths. Turning off the transarch paths also meant I had to put moved files back into subpackages and I might have misplaced some files and not noticed.
Finally, I've gotten the userspace packages building in koji [2] with minimal testing. Use at your own risk; they might kick kittens. Note that these are only userspace packages. You'll need the kmod from some other source to make the client work.
Ross Smith
[1] https://github.com/gaurdro/openafs
[2] http://koji.fedoraproject.org/koji/taskinfo?taskID=7703952
On Fri, 26 Sep 2014 14:34:37 -0400 ross smith gaurdro@gaurdro.net wrote:
I have been working on reworking the upstream spec file to work with centos. I've been keeping track of my progress on my github page [1] and I welcome pull requests if you find a problem.
Thanks for doing this. I had some comments on this I wanted to write down somewhere, so I'll just bring that up here; let me know if any such discussion should move elsewhere. I should be at the meeting tomorrow, too, where any of this can be discussed in realtime if anyone would like. (For anyone who cares and doesn't know, I am an upstream developer, and was advocating for the RPM packaging to move downstream.)
- I guess it's too late for this for the commit history, but it would be nice if it was noted anywhere where the original spec file came from before any modifications. e.g. what openafs.git git commit hash, or if it just came from the upstream openafs 1.6.9 source tarball release.
- The 'compat' subpackage has been removed, but I'm not sure why. This seems like a very low level of effort to maintain, but this is actually useful for compatibility with old tooling.
- The 'transarc' paths have been removed, which is good, but a compatibility subpackage with symlinks for the old 'transarc' paths would be very useful. This is perhaps more important than the 'compat' package referenced above, since transarc paths are used in all existing upstream RHEL/CentOS packaging. I was thining of a 'transarc' subpackage, but I guess there's no reason this couldn't just be rolled into the 'compat' subpackage.
- Just to note: I originally thought that CentOS would just provide packages for RHEL/CentOS 7, and that users of 6 could use the existing upstream packaging. That way, the transition of moving from openafs.org to centos.org RPMs would occur when migrating from 6 to 7, and there would be no migration within the same base system. I just thought it would be easier for existing users this way (and it can make the spec file a bit simpler), but I don't know if you agree. At the very least, I thought 7 would be a priority, since upstream will not be providing 7 packages.
Also, who should the "packager" be set to? Does a single person volunteer to be the official maintainer of the packaging, or should it be assigned to the Storage SIG or something like that?
I have pulled out all of the code related to the building the kmod, as I think that should at least be rewritten from scratch and quite possibly should live in it's own spec file.
That sounds good, but do we know how the kmod builds are going to work at all? My understanding is that we cannot rely on detecting what kernel headers are installed, and we also cannot accept kernel versions to build against via rpmbuild parameters. So, how do we determine what kernels to build kmods for?
Or I think I heard some mention of dynamically building kmods via a kernel trigger or something, which sounded a lot like DKMS. I'm not clear on how that would be beneficial over DKMS...? Some users do want more traditional kmod packages, but most should be satisfied by DKMS if that's the only practical option in the existing build framework.
On Tue, Sep 30, 2014 at 4:41 PM, Andrew Deason adeason@sinenomine.net wrote:
On Fri, 26 Sep 2014 14:34:37 -0400 ross smith gaurdro@gaurdro.net wrote:
(For anyone who cares and doesn't know, I am an upstream developer, and was advocating for the RPM packaging to move downstream.)
Glad to have someone from the upstream looking at this too.
- I guess it's too late for this for the commit history, but it would be nice if it was noted anywhere where the original spec file came from before any modifications. e.g. what openafs.git git commit hash, or if it just came from the upstream openafs 1.6.9 source tarball release.
I was going to make this the first entry in the changelog that it was forked from the packaged 1.6.9 openafs.org spec file. I imagine the spec file is going to land in a centos.org git repo at some point and its origin can be noted at that push as well.
- The 'compat' subpackage has been removed, but I'm not sure why.
I was pruning from the original package and it ended up empty, so I dropped it instead of figuring out what should be included. I assumed it would come up before it was stabilized if someone cared about it. I wouldn't be opposed to recreating it for <7 with or without the transarch paths. I think RHEL/CentOS 7 is a big enough change that starting as clean as possible would be worthwhile.
- Just to note: I originally thought that CentOS would just provide packages for RHEL/CentOS 7, and that users of 6 could use the existing upstream packaging. That way, the transition of moving from openafs.org to centos.org RPMs would occur when migrating from 6 to 7, and there would be no migration within the same base system. I just thought it would be easier for existing users this way (and it can make the spec file a bit simpler), but I don't know if you agree. At the very least, I thought 7 would be a priority, since upstream will not be providing 7 packages.
At least for userspace I think the complexity is small enough to support
both 6 and 7. I don't think a final destination for the packages has been decided yet. I'd like to have RHEL/CentOS 6 packages available and leave it up to the sysadmins to decide when makes sense for them to migrate. I'm certainly not discounting el7 as an important build target though.
Also, who should the "packager" be set to? Does a single person
volunteer to be the official maintainer of the packaging, or should it be assigned to the Storage SIG or something like that?
This might be a good argument for setting up a Storage SIG mailing list even if we do most of our discussion here.
I have pulled out all of the code related to the building the kmod, as I think that should at least be rewritten from scratch and quite possibly should live in it's own spec file.
That sounds good, but do we know how the kmod builds are going to work at all?
I understand there's some infrastructure already in place with CentOS to rebuild packages when the kernel package updates. The updated packages would happen at the repo level, not at the system level.
So, how do we determine what
kernels to build kmods for?
I've not heard a definitive answer on this myself. I suspect that there will be at most one kernel-devel package installed at build time, which should simply things a bit. I haven't dug around in koji to see if my suspicions are correct, however. This would be a good topic for the meeting tomorrow.
Ross Smith
gaurdro@gaurdro.net
On 30/09/14 22:57, ross smith wrote:
On Tue, Sep 30, 2014 at 4:41 PM, Andrew Deason adeason@sinenomine.net wrote:
On Fri, 26 Sep 2014 14:34:37 -0400 ross smith gaurdro@gaurdro.net wrote:
I have pulled out all of the code related to the building the kmod, as I think that should at least be rewritten from scratch and quite possibly should live in it's own spec file.
That sounds good, but do we know how the kmod builds are going to work at all?
RHEL (and hence CentOS) has contained the framework to solve that particular problem since el5. It's part of the Driver Update Program (DUP) which allows 3rd party kABI-compatible kmod packages to be built against one kernel and work seamlessly with all kABI-compatible kernels. This is the RHEL way of doing things so I would strongly suggest you do likewise.
elrepo.org have already done that work for el5 / el6:
https://github.com/elrepo/packages/tree/master/openafs-kmod
It should be trivial to make a kmod package for el7 using elrepo's spec file template:
https://github.com/elrepo/templates/tree/master/el7
Happy to help if you have any questions.
I understand there's some infrastructure already in place with CentOS to rebuild packages when the kernel package updates. The updated packages would happen at the repo level, not at the system level.
Not needed. A kABI-tracking kmod package built against kernel-3.10.0-123.el7.x86_64 should work seamlessly with all RHEL7 kernels, at least in the 7.0 release. There is a possibility that your kmod package might need to be rebuilt against the 7.1 kernel when that is released if the openafs driver uses kernel symbols that are not on the RHEL kABI whitelist and the ABI of those symbols change, but it isn't going to need to be rebuilt against each and every kernel release.
So, how do we determine what
kernels to build kmods for?
That's up to you. Typically one would build against the base kernel for a point release, so for RHEL7 I would build against kernel-3.10.0-123.el7.x86_64.
I've not heard a definitive answer on this myself. I suspect that there will be at most one kernel-devel package installed at build time, which should simply things a bit. I haven't dug around in koji to see if my suspicions are correct, however. This would be a good topic for the meeting tomorrow.
On Tue, 30 Sep 2014 23:52:19 +0100 Ned Slider ned@unixmail.co.uk wrote:
On 30/09/14 22:57, ross smith wrote:
On Tue, Sep 30, 2014 at 4:41 PM, Andrew Deason adeason@sinenomine.net wrote:
That sounds good, but do we know how the kmod builds are going to work at all?
RHEL (and hence CentOS) has contained the framework to solve that particular problem since el5. It's part of the Driver Update Program (DUP) which allows 3rd party kABI-compatible kmod packages to be built against one kernel and work seamlessly with all kABI-compatible kernels.
The openafs kernel module uses symbols outside of the kABI whitelist out of necessity [1], and has been proven to break previously on kernel updates that trusted the DUP.
So, we try to disable and avoid any mechanisms involving the kABI/DUP stuff as much as possible. If it is not possible to build against every kernel in CentOS's buildsystem, that's not a showstopper, since 'most' users would be happy with DKMS, and the rest can find their own way to develop their own kmods or get them from somewhere else. But I'm just always inquiring if it's just possible to build for every kernel, since that will work for everyone. (And it's what upstream does now, and what someone will have to do anyway.)
[1] While I have not actually spent any significant time trying to avoid the 'bad' symbols, the last list I saw for EL6 included some symbols that seemed pretty impractical: http://article.gmane.org/gmane.comp.file-systems.openafs.general/35234 (not all of them, but in particular some of the keyring and fs related ones). And even if it were possible, the effort required for doing so I think would make this impractical for the short term.
On 10/01/2014 12:52 AM, Andrew Deason wrote:
So, we try to disable and avoid any mechanisms involving the kABI/DUP stuff as much as possible. If it is not possible to build against every kernel in CentOS's buildsystem, that's not a showstopper, since 'most'
The distro buildsystem can do hooks, one of which is a dep chain. It can trigger builds for packages based on build status of others. So its easy to add a srpm for the kernel mod, and have that build just after any kernel does.
The thing we need to workout is going to be how that might cascade down to the koji setup at cbs.centos.org which we intend to use for all SIG work, and eventually for all builds in CentOS.
Maybe for now, the hook against distro kernel might be to just call the koji command that builds the openafs kernel module
On Tue, 30 Sep 2014 17:57:59 -0400 ross smith gaurdro@gaurdro.net wrote:
- The 'compat' subpackage has been removed, but I'm not sure why.
I was pruning from the original package and it ended up empty, so I dropped it instead of figuring out what should be included. I assumed it would come up before it was stabilized if someone cared about it. I wouldn't be opposed to recreating it for <7 with or without the transarch paths. I think RHEL/CentOS 7 is a big enough change that starting as clean as possible would be worthwhile.
I could see the 'transarc' compatibility symlinks going away after some releases, but certainly not yet with just EL7 (since the openafs.org EL6 RPMs still use them). These are paths that are mostly of concern only to administrators (but I would double-check that at some point).
For the compat symlinks that are in the existing 'compat' subpackage, though, these are more visible to users and decades-old scripts and such. Some things are run from /afs and don't care if they are on EL6 or EL7 or whatever, but they assume /usr/afsws/bin/fs or somesuch always exists, as a kind of public interface established a long time ago.
I won't argue terribly hard for this, since I (or someone) can always maintain a small package separately that does this if it's needed. But the maintenance just seems so small that getting rid of it doesn't seem like much gain.
- Just to note: I originally thought that CentOS would just provide packages for RHEL/CentOS 7, and that users of 6 could use the
At least for userspace I think the complexity is small enough to support both 6 and 7.
For the spec file maintenance side of things, I was thinking of the systemd/non-systemd, which I suppose isn't so bad.
-
Andrew Deason -
Karanbir Singh -
Ned Slider -
ross smith