In August, the CentOS Project and Fedora announced the intention to combine their dist-git sources, and since then Fabian Arrotin of CentOS and Patrick Uiterwijk of Fedora have worked to make that happen. Last week at the CERN dojo, they demonstrated this work, and today we’d like to welcome community testing of it. We’ll be keeping this content updated, so you'll be able to continue to track sources appropriately.
You can help us by testing the new system, providing feedback, and contributing additional tooling to help improve the overall state of the project.
Structure:
Currently git.stg.centos.org and src.stg.fedoraproject.org are linked via RepoSpanner[1], and sharing common repositories. Packagers for the roughly 6700 common packages in both communities will see the additional branches..
For the Fedora community, there should be no changes required for the day to day operation of dist-git or packaging.
CentOS community members will note that the platform change to pagure brings a variety of new features, including a new API[2] , a new UI, pull requests, and issue tracking. The current tooling for CentOS sources will continue to work (after updating to the staging server) and we plan to add support for tooling for centpkg, which would mirror the functionality found in fedpkg.
In the coming weeks we'll be putting out documentation around the expected uses for pull requests, and the issue tracking features within pagure, as we don't intend it to replace bugs.centos.org.
Permissions and ownership:
There are protections enabled to prevent either community from accidentally overwriting code from the other, so only CentOS community members will be able to push to c#-* branches, while only Fedora community members can push to f#*/epel#* branches. With the new dynamic ACL validation in place, CentOS SIG members will automatically get the rights to push on their specific branches.
While the repositories and content are shared between the communities, authentication is not. To push to c# branches, you will have to authenticate against accounts.centos.org. Similarly Fedora community members will continue to authenticate against accounts.fedoraproject.org.
1. https://github.com/repoSpanner/repoSpanner 2. https://git.stg.centos.org/api/0/
Looking at that API, I've got a few questions:
Under "List projects", is there a way to search for projects with a specific branch (ie, find projects with c7 branch)? Under "List projects", is there a way to search for projects with any commits/activity after a certain date range (so I can only check for updates within the last few days)?
Generically, will the scripts in https://git.centos.org/summary/centos-git-common.git be ported over? Our work flow is entirely dependent on : centos.git.repolist.py into_srpm.sh show_possible_srpms.sh
Pat
On 10/25/2018 11:25 AM, Jim Perrin wrote:
In August, the CentOS Project and Fedora announced the intention to combine their dist-git sources, and since then Fabian Arrotin of CentOS and Patrick Uiterwijk of Fedora have worked to make that happen. Last week at the CERN dojo, they demonstrated this work, and today we’d like to welcome community testing of it. We’ll be keeping this content updated, so you'll be able to continue to track sources appropriately.
You can help us by testing the new system, providing feedback, and contributing additional tooling to help improve the overall state of the project.
Structure:
Currently git.stg.centos.org and src.stg.fedoraproject.org are linked via RepoSpanner[1], and sharing common repositories. Packagers for the roughly 6700 common packages in both communities will see the additional branches..
For the Fedora community, there should be no changes required for the day to day operation of dist-git or packaging.
CentOS community members will note that the platform change to pagure brings a variety of new features, including a new API[2] , a new UI, pull requests, and issue tracking. The current tooling for CentOS sources will continue to work (after updating to the staging server) and we plan to add support for tooling for centpkg, which would mirror the functionality found in fedpkg.
In the coming weeks we'll be putting out documentation around the expected uses for pull requests, and the issue tracking features within pagure, as we don't intend it to replace bugs.centos.org.
Permissions and ownership:
There are protections enabled to prevent either community from accidentally overwriting code from the other, so only CentOS community members will be able to push to c#-* branches, while only Fedora community members can push to f#*/epel#* branches. With the new dynamic ACL validation in place, CentOS SIG members will automatically get the rights to push on their specific branches.
While the repositories and content are shared between the communities, authentication is not. To push to c# branches, you will have to authenticate against accounts.centos.org. Similarly Fedora community members will continue to authenticate against accounts.fedoraproject.org.
On 10/25/18 10:08 AM, Pat Riehecky wrote:
Looking at that API, I've got a few questions:
Under "List projects", is there a way to search for projects with a specific branch (ie, find projects with c7 branch)? Under "List projects", is there a way to search for projects with any commits/activity after a certain date range (so I can only check for updates within the last few days)?
I'd defer to Fabian on these, as I don't have the answers.
Generically, will the scripts in https://git.centos.org/summary/centos-git-common.git be ported over? Our work flow is entirely dependent on : centos.git.repolist.py into_srpm.sh show_possible_srpms.sh
So most of them should keep working. The repo paths aren't changing, so into_srpm.sh and show_possible_srpms.sh should still work. The centos.git.repolist.py script would need to be updated.
Pat
On 10/25/2018 11:25 AM, Jim Perrin wrote:
In August, the CentOS Project and Fedora announced the intention to combine their dist-git sources, and since then Fabian Arrotin of CentOS and Patrick Uiterwijk of Fedora have worked to make that happen. Last week at the CERN dojo, they demonstrated this work, and today we’d like to welcome community testing of it. We’ll be keeping this content updated, so you'll be able to continue to track sources appropriately.
You can help us by testing the new system, providing feedback, and contributing additional tooling to help improve the overall state of the project.
Structure:
Currently git.stg.centos.org and src.stg.fedoraproject.org are linked via RepoSpanner[1], and sharing common repositories. Packagers for the roughly 6700 common packages in both communities will see the additional branches..
For the Fedora community, there should be no changes required for the day to day operation of dist-git or packaging.
CentOS community members will note that the platform change to pagure brings a variety of new features, including a new API[2] , a new UI, pull requests, and issue tracking. The current tooling for CentOS sources will continue to work (after updating to the staging server) and we plan to add support for tooling for centpkg, which would mirror the functionality found in fedpkg.
In the coming weeks we'll be putting out documentation around the expected uses for pull requests, and the issue tracking features within pagure, as we don't intend it to replace bugs.centos.org.
Permissions and ownership:
There are protections enabled to prevent either community from accidentally overwriting code from the other, so only CentOS community members will be able to push to c#-* branches, while only Fedora community members can push to f#*/epel#* branches. With the new dynamic ACL validation in place, CentOS SIG members will automatically get the rights to push on their specific branches.
While the repositories and content are shared between the communities, authentication is not. To push to c# branches, you will have to authenticate against accounts.centos.org. Similarly Fedora community members will continue to authenticate against accounts.fedoraproject.org.
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_repoSpanner_...
https://urldefense.proofpoint.com/v2/url?u=https-3A__git.stg.centos.org_api_...
On 25/10/18 19:08, Pat Riehecky wrote:
Looking at that API, I've got a few questions:
Under "List projects", is there a way to search for projects with a specific branch (ie, find projects with c7 branch)? Under "List projects", is there a way to search for projects with any commits/activity after a certain date range (so I can only check for updates within the last few days)?
Generically, will the scripts in https://git.centos.org/summary/centos-git-common.git be ported over? Our work flow is entirely dependent on : centos.git.repolist.py into_srpm.sh show_possible_srpms.sh
Pat
So, myself I just got a quick look at the self-documented api (https://git.stg.centos.org/api/0/) but that doesn't seem possible, at least not through simple query
You can still query for all the projects (/api/0/projects) and then parse json output for each project including date_created and date_modified (epoch time).
Same for each project : /api/0/<namespace>/<repo>/git/branches
But as pagure is also open , pretty sure that if you come with a PR against it to implement "missing" api features, they'll consider those ;-)
WRT to centos-git-common, it will be migrated, probably on a new namespace, but still there. But , as Jim said, the goal is to have all the logic (for git *and* lookaside push) in a new centpkg/fedpkg tool.
The previous get_sources.sh tool continues also to work, but for the .stg. Instance, you need to specify the url to use to pull the sources from. Example with the zsh rpm :
<example> pushd $(mktemp -d) git clone https://git.stg.centos.org/rpms/zsh.git cd zsh/ git checkout c7 ~/bin/get_sources.sh --surl https://git.stg.centos.org/sources rpmbuild --define "%_topdir `pwd`" -bs --nodeps SPECS/zsh.spec Wrote: /tmp/tmp.1B78dTLtkx/zsh/SRPMS/zsh-5.0.2-28.el7.src.rpm popd </example>
And we already have in place some RewriteRules so that people would had local git cloned repositories from the /rpms/ namespaces on current gitblit/git.centos.org will be able to continue to "git pull" when the A record will have been switched and so migration done (no date yet, as we're now in staging mode, also reason why community can now start investigating what they need in their workflow)
So if you see some things that you'd like to change in the current centos-git-common repo, patches are welcome, and that will even be easier on the new system, through PR :-)
On Thu, Oct 25, 2018 at 10:20:45PM +0200, Fabian Arrotin wrote:
On 25/10/18 19:08, Pat Riehecky wrote:
Looking at that API, I've got a few questions:
Under "List projects", is there a way to search for projects with a specific branch (ie, find projects with c7 branch)? Under "List projects", is there a way to search for projects with any commits/activity after a certain date range (so I can only check for updates within the last few days)?
Generically, will the scripts in https://git.centos.org/summary/centos-git-common.git be ported over? Our work flow is entirely dependent on : centos.git.repolist.py into_srpm.sh show_possible_srpms.sh
Pat
So, myself I just got a quick look at the self-documented api (https://git.stg.centos.org/api/0/) but that doesn't seem possible, at least not through simple query
That's correct.
You can still query for all the projects (/api/0/projects) and then parse json output for each project including date_created and date_modified (epoch time).
The date_modified field refers to the last time information/settings about the project were changed, not the last time a commit was made
Same for each project : /api/0/<namespace>/<repo>/git/branches
But as pagure is also open , pretty sure that if you come with a PR against it to implement "missing" api features, they'll consider those ;-)
Feel also free to open a ticket there if you cannot work on the PR yourself, so the idea doesn't get lost :)
Thanks, Pierre
On 10/26/2018 04:34 AM, Pierre-Yves Chibon wrote:
On Thu, Oct 25, 2018 at 10:20:45PM +0200, Fabian Arrotin wrote:
On 25/10/18 19:08, Pat Riehecky wrote:
Looking at that API, I've got a few questions:
Under "List projects", is there a way to search for projects with a specific branch (ie, find projects with c7 branch)? Under "List projects", is there a way to search for projects with any commits/activity after a certain date range (so I can only check for updates within the last few days)?
Generically, will the scripts in https://git.centos.org/summary/centos-git-common.git be ported over? Our work flow is entirely dependent on : centos.git.repolist.py into_srpm.sh show_possible_srpms.sh
Pat
So, myself I just got a quick look at the self-documented api (https://git.stg.centos.org/api/0/) but that doesn't seem possible, at least not through simple query
That's correct.
You can still query for all the projects (/api/0/projects) and then parse json output for each project including date_created and date_modified (epoch time).
The date_modified field refers to the last time information/settings about the project were changed, not the last time a commit was made
Same for each project : /api/0/<namespace>/<repo>/git/branches
But as pagure is also open , pretty sure that if you come with a PR against it to implement "missing" api features, they'll consider those ;-)
Feel also free to open a ticket there if you cannot work on the PR yourself, so the idea doesn't get lost :)
Thanks, Pierre
I've got them open:
https://pagure.io/pagure/issue/3386 https://pagure.io/pagure/issue/3956
Larger scale question: With this merge would it make sense to evaluate copr for CentOS SIGs? It has native support for "This repo depends on this other repo". Fedora DNF has a copr plugin which could be useful in prep for RHEL8.
Pat
On 10/25/2018 11:25 AM, Jim Perrin wrote:
In August, the CentOS Project and Fedora announced the intention to combine their dist-git sources, and since then Fabian Arrotin of CentOS and Patrick Uiterwijk of Fedora have worked to make that happen. Last week at the CERN dojo, they demonstrated this work, and today we’d like to welcome community testing of it. We’ll be keeping this content updated, so you'll be able to continue to track sources appropriately.
You can help us by testing the new system, providing feedback, and contributing additional tooling to help improve the overall state of the project.
Structure:
Currently git.stg.centos.org and src.stg.fedoraproject.org are linked via RepoSpanner[1], and sharing common repositories. Packagers for the roughly 6700 common packages in both communities will see the additional branches..
For the Fedora community, there should be no changes required for the day to day operation of dist-git or packaging.
CentOS community members will note that the platform change to pagure brings a variety of new features, including a new API[2] , a new UI, pull requests, and issue tracking. The current tooling for CentOS sources will continue to work (after updating to the staging server) and we plan to add support for tooling for centpkg, which would mirror the functionality found in fedpkg.
In the coming weeks we'll be putting out documentation around the expected uses for pull requests, and the issue tracking features within pagure, as we don't intend it to replace bugs.centos.org.
Permissions and ownership:
There are protections enabled to prevent either community from accidentally overwriting code from the other, so only CentOS community members will be able to push to c#-* branches, while only Fedora community members can push to f#*/epel#* branches. With the new dynamic ACL validation in place, CentOS SIG members will automatically get the rights to push on their specific branches.
While the repositories and content are shared between the communities, authentication is not. To push to c# branches, you will have to authenticate against accounts.centos.org. Similarly Fedora community members will continue to authenticate against accounts.fedoraproject.org.
For right now, I prefer not to. One of the goals for SIGs (that we admittedly haven't done a good in achieving) was to promote the collaboration between SIGs and the sharing of packages. While copr can handle repo dependencies as you point out, it also tends to promote repository isolation. You can see this in the fact that there are a number of the same package repos built by different people in the Fedora copr setup.
I'd prefer putting a bit more time (and guidance from Rich) into better leadership and SIG work, before trying to achieve the same goal via a different tool. I'd like to give time for pull requests and automatic builds from pagure->cbs to get into SIG workflows and see what impact that has first.
On 10/26/18 6:11 AM, Pat Riehecky wrote:
Larger scale question: With this merge would it make sense to evaluate copr for CentOS SIGs? It has native support for "This repo depends on this other repo". Fedora DNF has a copr plugin which could be useful in prep for RHEL8.
Pat
On 10/25/2018 11:25 AM, Jim Perrin wrote:
In August, the CentOS Project and Fedora announced the intention to combine their dist-git sources, and since then Fabian Arrotin of CentOS and Patrick Uiterwijk of Fedora have worked to make that happen. Last week at the CERN dojo, they demonstrated this work, and today we’d like to welcome community testing of it. We’ll be keeping this content updated, so you'll be able to continue to track sources appropriately.
You can help us by testing the new system, providing feedback, and contributing additional tooling to help improve the overall state of the project.
Structure:
Currently git.stg.centos.org and src.stg.fedoraproject.org are linked via RepoSpanner[1], and sharing common repositories. Packagers for the roughly 6700 common packages in both communities will see the additional branches..
For the Fedora community, there should be no changes required for the day to day operation of dist-git or packaging.
CentOS community members will note that the platform change to pagure brings a variety of new features, including a new API[2] , a new UI, pull requests, and issue tracking. The current tooling for CentOS sources will continue to work (after updating to the staging server) and we plan to add support for tooling for centpkg, which would mirror the functionality found in fedpkg.
In the coming weeks we'll be putting out documentation around the expected uses for pull requests, and the issue tracking features within pagure, as we don't intend it to replace bugs.centos.org.
Permissions and ownership:
There are protections enabled to prevent either community from accidentally overwriting code from the other, so only CentOS community members will be able to push to c#-* branches, while only Fedora community members can push to f#*/epel#* branches. With the new dynamic ACL validation in place, CentOS SIG members will automatically get the rights to push on their specific branches.
While the repositories and content are shared between the communities, authentication is not. To push to c# branches, you will have to authenticate against accounts.centos.org. Similarly Fedora community members will continue to authenticate against accounts.fedoraproject.org.
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_repoSpanner_...
https://urldefense.proofpoint.com/v2/url?u=https-3A__git.stg.centos.org_api_...
On Fri, Oct 26, 2018 at 9:53 AM Jim Perrin jperrin@centos.org wrote:
For right now, I prefer not to. One of the goals for SIGs (that we admittedly haven't done a good in achieving) was to promote the collaboration between SIGs and the sharing of packages. While copr can handle repo dependencies as you point out, it also tends to promote repository isolation. You can see this in the fact that there are a number of the same package repos built by different people in the Fedora copr setup.
I'd prefer putting a bit more time (and guidance from Rich) into better leadership and SIG work, before trying to achieve the same goal via a different tool. I'd like to give time for pull requests and automatic builds from pagure->cbs to get into SIG workflows and see what impact that has first.
Note that the main missing functionality is a mechanism for binding submissions and reviews with its dist-git backend and the build system. If that was supported, then the possibility of short-lived forks that would then merge back into upstream projects in COPR would make it a natural fit to work with Pagure.
On 10/26/2018 08:53 AM, Jim Perrin wrote:
For right now, I prefer not to. One of the goals for SIGs (that we admittedly haven't done a good in achieving) was to promote the collaboration between SIGs and the sharing of packages. While copr can handle repo dependencies as you point out, it also tends to promote repository isolation. You can see this in the fact that there are a number of the same package repos built by different people in the Fedora copr setup.
I've got a kind of half formed view in my head, so this may be jumbled up....
On www.softwarecollections.org there are different kinds of policy/project things. If the COPR moved a bit in that direction with some sort of "rational types" and review required for "non-personal types", this could help reduce the fragmentation a bit. (For what it's worth I'm super happy with the SCL.org website as a whole UI). I think this could really help promote the SIG as "The Place" to get certain things.
The "project types" might also provide a clean way for proto SIGs to privately play around while they sort things out.
In theory the "project types" could provide a hint to folks "Hey I see you are building AWESOMEPACKAGE for RELEASE, did you know we've got an Approved group that is building AWESOMEPACKAGE for RELEASE over here".
Also having the SIGs located up there could help drive the special interest development and then re-merge (similar to Neal's comments). I feel like this could be handy for Fedora but can't think of a good example.....
The builtin support for project docs via the "Overview" tab makes a clean place to put some items. Might be simpler than giving people rights to https://wiki.centos.org/SpecialInterestGroup/%7BSIGNAME%7D
From a larger unification standpoint this would help simplify searching a bit. Right now there are a few different places to look for packages. Merging the SIGs into COPR would reduce the locations and help folks who search before making a COPR repo see what SIGs are out there.
It would also eliminate the need to maintain CBS and sort out the occasional package version collisions. CBS currently shows 1523 Koji tags.
Free support for Modularity is a big bonus.
I'd prefer putting a bit more time (and guidance from Rich) into better leadership and SIG work, before trying to achieve the same goal via a different tool. I'd like to give time for pull requests and automatic builds from pagure->cbs to get into SIG workflows and see what impact that has first.
On 10/26/18 6:11 AM, Pat Riehecky wrote:
Larger scale question: With this merge would it make sense to evaluate copr for CentOS SIGs? It has native support for "This repo depends on this other repo". Fedora DNF has a copr plugin which could be useful in prep for RHEL8.
Pat
On 10/25/2018 11:25 AM, Jim Perrin wrote:
In August, the CentOS Project and Fedora announced the intention to combine their dist-git sources, and since then Fabian Arrotin of CentOS and Patrick Uiterwijk of Fedora have worked to make that happen. Last week at the CERN dojo, they demonstrated this work, and today we’d like to welcome community testing of it. We’ll be keeping this content updated, so you'll be able to continue to track sources appropriately.
You can help us by testing the new system, providing feedback, and contributing additional tooling to help improve the overall state of the project.
Structure:
Currently git.stg.centos.org and src.stg.fedoraproject.org are linked via RepoSpanner[1], and sharing common repositories. Packagers for the roughly 6700 common packages in both communities will see the additional branches..
For the Fedora community, there should be no changes required for the day to day operation of dist-git or packaging.
CentOS community members will note that the platform change to pagure brings a variety of new features, including a new API[2] , a new UI, pull requests, and issue tracking. The current tooling for CentOS sources will continue to work (after updating to the staging server) and we plan to add support for tooling for centpkg, which would mirror the functionality found in fedpkg.
In the coming weeks we'll be putting out documentation around the expected uses for pull requests, and the issue tracking features within pagure, as we don't intend it to replace bugs.centos.org.
Permissions and ownership:
There are protections enabled to prevent either community from accidentally overwriting code from the other, so only CentOS community members will be able to push to c#-* branches, while only Fedora community members can push to f#*/epel#* branches. With the new dynamic ACL validation in place, CentOS SIG members will automatically get the rights to push on their specific branches.
While the repositories and content are shared between the communities, authentication is not. To push to c# branches, you will have to authenticate against accounts.centos.org. Similarly Fedora community members will continue to authenticate against accounts.fedoraproject.org.
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_repoSpanner_...
https://urldefense.proofpoint.com/v2/url?u=https-3A__git.stg.centos.org_api_...
-
Fabian Arrotin -
Jim Perrin -
Neal Gompa -
Pat Riehecky -
Pierre-Yves Chibon