<paste> sigs would like to use centpkg / lookaside, build direct through git to koji authentication requirements to accounts.centos.org Fabian to evaluate git solutions and report back to sig chairs. mrunge has volunteered to be the "guinea pig" of the new system </paste>
So let's start the thread, to be sure that all involved people would be able to comment. SIGs would like to start building from git, and not from SRPMs they have to create/upload themselves.
For GIT itself, several options exist : - using git.centos.org : that would mean SIG would need access to specific repositories and also a "lookaside cache" feature to not store binary blogs/tarballs in git itself. And that would also need an authentication backend that the current solution would need to be tied to
- using github : most people have probably a GH account, so everything can be hosted there, and for the "lookaside cache", Github has LFS support built-in so that would need re-tooling correctly centos-packager to use that (but no rpm yet for git-lfs client side). Problem would be suddenly that we only rely on GH to build packages/artifacts on cbs.centos.org
- using other "on premise" git solution , close to the CBS builders : from my quick research, I'll probably have a look at gitlab, but gitea was on my radar and it implements natively : - LFS support (so "lookaside" cache without having to write an app for that, and so using directly also the repo ACL to let people store tarballs/blobs - openid/oauth2 support (so we can then also reuse https://accounts.centos.org, through https://id.centos.org IdP
Waiting for comments/input/feedback on those points
On Tue, Oct 24, 2017 at 9:59 AM, Fabian Arrotin arrfab@centos.org wrote:
<paste> sigs would like to use centpkg / lookaside, build direct through git to koji authentication requirements to accounts.centos.org Fabian to evaluate git solutions and report back to sig chairs. mrunge has volunteered to be the "guinea pig" of the new system </paste>
So let's start the thread, to be sure that all involved people would be able to comment. SIGs would like to start building from git, and not from SRPMs they have to create/upload themselves.
For GIT itself, several options exist :
- using git.centos.org : that would mean SIG would need access to
specific repositories and also a "lookaside cache" feature to not store binary blogs/tarballs in git itself. And that would also need an authentication backend that the current solution would need to be tied to
- using github : most people have probably a GH account, so everything
can be hosted there, and for the "lookaside cache", Github has LFS support built-in so that would need re-tooling correctly centos-packager to use that (but no rpm yet for git-lfs client side). Problem would be suddenly that we only rely on GH to build packages/artifacts on cbs.centos.org
- using other "on premise" git solution , close to the CBS builders :
from my quick research, I'll probably have a look at gitlab, but gitea was on my radar and it implements natively :
- LFS support (so "lookaside" cache without having to write an app for
that, and so using directly also the repo ACL to let people store tarballs/blobs
- openid/oauth2 support (so we can then also reuse
https://accounts.centos.org, through https://id.centos.org IdP
Waiting for comments/input/feedback on those points
From our discussion, I remember that with the "lookaside cache", it
should be possible for a "drive-by" contributor to submit a change which included a new tarball, by submitting a pull request that had the proper hash; I could then download the tarball from the upstream website myself, verify the hash, and upload it to the lookaside cache when merging the PR.
For an on-prem gitlab with lfs support, would it be possible for "drive-by" contributors to send changes which changed tarballs in a similar way?
FWIW my preference would be: * On-prem git solution with git-LFS, probably gitlab * github + lookaside cache * git.centos.org + traditional look-aside cache * github + LFS support
-George
On Tue, Oct 24, 2017 at 03:46:28PM +0100, George Dunlap wrote:
On Tue, Oct 24, 2017 at 9:59 AM, Fabian Arrotin arrfab@centos.org wrote:
<paste> sigs would like to use centpkg / lookaside, build direct through git to koji authentication requirements to accounts.centos.org Fabian to evaluate git solutions and report back to sig chairs. mrunge has volunteered to be the "guinea pig" of the new system </paste>
Waiting for comments/input/feedback on those points
Thank you for kicking this off!
Storing specs + upstream sources somewhere would solve my primary concern with creating some more reproducible builds. Even in a small team, it seems scary to upload "somehow" created srpms to get them built in cbs.
From our discussion, I remember that with the "lookaside cache", it should be possible for a "drive-by" contributor to submit a change which included a new tarball, by submitting a pull request that had the proper hash; I could then download the tarball from the upstream website myself, verify the hash, and upload it to the lookaside cache when merging the PR.
Yes, I remember we discussed it briefly, on how to enable drive-by contributions or how to lower the barrier for contributors.
I'd be fine with patches/pull-requests/whatever for spec files. I'd try to pull down sources myself anyways.
Ideally, any solution would be supported by a central tool, comparable to fedpkg for fedora. I know there is centpkg, but I'm currently unsure how git and source upload is handled there.
Best, Matthias
On Oct 24 18:08, Matthias Runge wrote:
On Tue, Oct 24, 2017 at 03:46:28PM +0100, George Dunlap wrote:
On Tue, Oct 24, 2017 at 9:59 AM, Fabian Arrotin arrfab@centos.org wrote:
<paste> sigs would like to use centpkg / lookaside, build direct through git to koji authentication requirements to accounts.centos.org Fabian to evaluate git solutions and report back to sig chairs. mrunge has volunteered to be the "guinea pig" of the new system </paste>
Waiting for comments/input/feedback on those points
Thank you for kicking this off!
Storing specs + upstream sources somewhere would solve my primary concern with creating some more reproducible builds. Even in a small team, it seems scary to upload "somehow" created srpms to get them built in cbs.
From our discussion, I remember that with the "lookaside cache", it should be possible for a "drive-by" contributor to submit a change which included a new tarball, by submitting a pull request that had the proper hash; I could then download the tarball from the upstream website myself, verify the hash, and upload it to the lookaside cache when merging the PR.
Yes, I remember we discussed it briefly, on how to enable drive-by contributions or how to lower the barrier for contributors.
I'd be fine with patches/pull-requests/whatever for spec files. I'd try to pull down sources myself anyways.
Ideally, any solution would be supported by a central tool, comparable to fedpkg for fedora. I know there is centpkg, but I'm currently unsure how git and source upload is handled there.
Centpkg currently only deals with source RPMs. This is blocked on some sort of git solution with proper credentialing such that the SIG members can do basic operations. If such a thing came up, centpkg could easily become a thing again, and could be the right "central tool" for the job.
Best, Matthias -- Matthias Runge mrunge@matthias-runge.de _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
--Brian
On 26/10/17 22:11, Brian Stinson wrote:
On Oct 24 18:08, Matthias Runge wrote:
On Tue, Oct 24, 2017 at 03:46:28PM +0100, George Dunlap wrote:
On Tue, Oct 24, 2017 at 9:59 AM, Fabian Arrotin arrfab@centos.org wrote:
<paste> sigs would like to use centpkg / lookaside, build direct through git to koji authentication requirements to accounts.centos.org Fabian to evaluate git solutions and report back to sig chairs. mrunge has volunteered to be the "guinea pig" of the new system </paste>
Waiting for comments/input/feedback on those points
Thank you for kicking this off!
Storing specs + upstream sources somewhere would solve my primary concern with creating some more reproducible builds. Even in a small team, it seems scary to upload "somehow" created srpms to get them built in cbs.
From our discussion, I remember that with the "lookaside cache", it should be possible for a "drive-by" contributor to submit a change which included a new tarball, by submitting a pull request that had the proper hash; I could then download the tarball from the upstream website myself, verify the hash, and upload it to the lookaside cache when merging the PR.
Yes, I remember we discussed it briefly, on how to enable drive-by contributions or how to lower the barrier for contributors.
I'd be fine with patches/pull-requests/whatever for spec files. I'd try to pull down sources myself anyways.
Ideally, any solution would be supported by a central tool, comparable to fedpkg for fedora. I know there is centpkg, but I'm currently unsure how git and source upload is handled there.
Centpkg currently only deals with source RPMs. This is blocked on some sort of git solution with proper credentialing such that the SIG members can do basic operations. If such a thing came up, centpkg could easily become a thing again, and could be the right "central tool" for the job.
I haven't tested LFS myself, but as Gitea (that I deployed as a PoC, so that Matthias could play with it) supports that, I was wondering if that couldn't be a simple solution to store blobs/tarballs, without a need to write a kind of "lookaside cache" solution that would have to do ACL verification. AFAICS, LFS through git does that automatically through git permissions
The client side would need to be worked on though : git-lfs seems to exist in recent Fedora, but nothing in Epel7. https://bugzilla.redhat.com/show_bug.cgi?id=1504322 https://src.fedoraproject.org/rpms/git-lfs/blob/master/f/git-lfs.spec
I haven't tried a rebuild, as from a quick look in the .spec, it would need quite some packages to be available , including higher git (or can we force SCLo for this ?)
-
Brian Stinson -
Fabian Arrotin -
George Dunlap -
Matthias Runge