Hi all (especially SIG members/contributors) ,
As announced by Jim some time ago, we wanted to redesign the current signing process that is in place for https://cbs.centos.org for CentOS 6 and 7 (so quite some years now)
The goal is to automate as much as possible the workflow, and working for all releases (yeah for CentOS 8 and Stream)
Thomas and myself have worked on the following idea and we're now happy with the results (in our Dev environment) : - when someone builds a pkg, and that it's tagged to -testing, koji sends a notification (through koji callback - https://fedoraproject.org/wiki/Koji/WritingKojiCode#Event_Plugin) to a bus - signing machine listens to the bus, process the tag and push directly to buildlogs - when someone tag-build said pkg to -release, the node signs it with correct gpg key id (from the SIG), push generated repository (including for debuginfo/src.rpm packages) to mirror CDN
As we were working on such automation, we also confirm that if one uses "untag-build", the same rule applies : it triggers automatically koji to reprocess the tag/repo and so removes the pkg from mirror (we got some requests for that in the past)
The process so would work 24/7, and also have monitoring built-in (so that we can detect - the infra team - when something doesn't work, and so we can still process the tag again)
We recently upgraded cbs.centos.org to newer koji version that permits some of the needed operations for this to happen.
At this stage, we are just missing the SIG GPG private keys, but we were promised to have those this week.
We don't have any ETA yet for the migration to new signing system, but we were thinking about such plan:
- deploy and configure (automatically) new signing node (this week) - import GPG keys in the new system and validate that we can sign (when we get those) - changes some flags/parameters on tags on cbs.centos.org (this week)
Once we have this in place, we'd like to start validating the signing process but *not* pushing to mirror CDN (yet). Same for the bus : we'd like to process all tags in advance as we'll have to import RPM signatures back in koji/cbs.
After full validation, we announce that new system would be fully in place, and kojihub restarted with the notification plugin. From that point, the system would be fully operational and so when SIG members would build/tag pkgs, that would trigger the push (and signing if needed for -release tag)
Worth knowing that with the update to some components in the chain, the directory layout will change a little bit (like people are already used to if using Fedora and Epel) : the Packages will be splitted by first letter. Here is an example from our testing :
└── storage └── x86_64 └── gluster-7 ├── Packages │ ├── g │ │ ├── glusterfs-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-api-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-api-devel-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-cli-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-client-xlators-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-cloudsync-plugins-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-devel-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-events-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-extra-xlators-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-fuse-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-geo-replication-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-libs-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-rdma-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-resource-agents-7.2-1.el7.noarch.rpm │ │ ├── glusterfs-server-7.2-1.el7.x86_64.rpm │ │ └── glusterfs-thin-arbiter-7.2-1.el7.x86_64.rpm │ ├── p │ │ └── python2-gluster-7.2-1.el7.x86_64.rpm │ └── u │ ├── userspace-rcu-0.7.16-3.el7.x86_64.rpm │ └── userspace-rcu-devel-0.7.16-3.el7.x86_64.rpm └── repodata ├── 047503df3313d596e14a1a0d097b6c2261b26690000ecd4260d212a6f5e5632d-other.sqlite.bz2 ├── 34c1f5dd954099816d6cce289e52f96e61cc2e6430e13072194e35c063b0e3da-primary.xml.gz ├── 6293e07e4c5a7acc2c610beb5acb84b996329ef3c6d61c03e375f2afc6a71536-filelists.sqlite.bz2 ├── 6f89e8b43fb7e384f86f041c4de84d9afeb15dca981559b15230cb48887095db-other.xml.gz ├── bef236563d185417fbda48e3b4dd1d62714d1af2b7a7d6dc1a7213af5de1aa9b-primary.sqlite.bz2 ├── fb66c148e8e2738bd5d19b88d1b3507b0383aaab0938de3809b399babb1a0b44-filelists.xml.gz ├── repomd.xml └── repomd.xml.asc
From a usage perspective that will not change the way pkgs can be consumed, as repodata will always be at the right place, but in the case of SCLs that will mean no subdirectory like before, but instead just same "first letter" name (while repodata will be global)
So in the current stage, we'd like to deploy and fully validate the signing setup and we'll announce on this list when we'll fully switch to new system.
Also worth noting that with the new system in place, in theory we'll *not* use the old cbs-content-control git repo (as we'll query automatically koji for tags, enabled architectures, etc) so tagging to -testing and -release should trigger automatically the "push to mirrors" step
If you have questions, comments, feel free to answer in this thread and/or #centos-devel on irc.freenode.net
Kind Regards,
Thomas and Fabian
On 10/03/2020 16:17, Fabian Arrotin wrote:
Hi all (especially SIG members/contributors) ,
<snip>
If you have questions, comments, feel free to answer in this thread and/or #centos-devel on irc.freenode.net
Replying to myself based on a discussion we had with Alfredo in #centos-devel : We'll also stop using mash to generate the initial repositories, so https://cbs.centos.org/repos will completely disappear after the full switch to new system (FWIW)
Normally it shouldn't have been exposed outside, and so we'd like to ensure that SIG aren't pointing to those unsigned repositories (as it will be fully gone)
The way to consume SIGs rpms will be : - https://buildlogs.centos.org (for -testing) - mirrorlist/mirrors (for -release, with signed pkgs and repodata)
Hope that it clarifies things :)
On Tue, Mar 10, 2020 at 10:27 AM Fabian Arrotin arrfab@centos.org wrote:
Replying to myself based on a discussion we had with Alfredo in #centos-devel : We'll also stop using mash to generate the initial repositories, so https://cbs.centos.org/repos will completely disappear after the full switch to new system (FWIW)
What is the replacement?
Will the signing pieces and repository pieces be open-source?
- Ken
On 10/03/2020 17:30, Ken Dreyer wrote:
On Tue, Mar 10, 2020 at 10:27 AM Fabian Arrotin arrfab@centos.org wrote:
Replying to myself based on a discussion we had with Alfredo in #centos-devel : We'll also stop using mash to generate the initial repositories, so https://cbs.centos.org/repos will completely disappear after the full switch to new system (FWIW)
What is the replacement?
Will the signing pieces and repository pieces be open-source?
- Ken
Hi Ken,
Yes, as all ansible roles are on github
To orchestrate the signing, it's this role : https://github.com/CentOS/ansible-role-stylo/tree/dev (actually only in -dev- branch during our testings)
For the koji counter part, it was already pushed to staging branch : https://github.com/CentOS/ansible-role-kojihub/commit/b99cd542bd4fe0f4887093...
People following activity on github.com/CentOS can see all ansible roles :)
Now, can you elaborate on "repository pieces be open-source" ?
On Tue, Mar 10, 2020 at 10:40 AM Fabian Arrotin arrfab@centos.org wrote:
Now, can you elaborate on "repository pieces be open-source" ?
Sure, I mean if you are not using mash, what are you using to generate the repositories with signed and unsigned packages?
- Ken
On 10/03/2020 20:31, Ken Dreyer wrote:
On Tue, Mar 10, 2020 at 10:40 AM Fabian Arrotin arrfab@centos.org wrote:
Now, can you elaborate on "repository pieces be open-source" ?
Sure, I mean if you are not using mash, what are you using to generate the repositories with signed and unsigned packages?
"koji dist-repo $options $tag_name $gpg_key_id" and koji , as it will have the rpm signatures imported, will generate the repo with signed pkgs in place, also splitting debuginfo and src.rpm packages
That's great, thank you.
Can I see application or configuration for how often you will run it? Is it event-driven, or on a timer?
- Ken
On Tue, Mar 10, 2020 at 2:10 PM Fabian Arrotin arrfab@centos.org wrote:
On 10/03/2020 20:31, Ken Dreyer wrote:
On Tue, Mar 10, 2020 at 10:40 AM Fabian Arrotin arrfab@centos.org wrote:
Now, can you elaborate on "repository pieces be open-source" ?
Sure, I mean if you are not using mash, what are you using to generate the repositories with signed and unsigned packages?
"koji dist-repo $options $tag_name $gpg_key_id" and koji , as it will have the rpm signatures imported, will generate the repo with signed pkgs in place, also splitting debuginfo and src.rpm packages
-- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On 10/03/2020 22:18, Ken Dreyer wrote:
That's great, thank you.
Can I see application or configuration for how often you will run it? Is it event-driven, or on a timer?
Well, in the initial mail I mentioned a message bus with koji callback plugin (even the link to it ;-) ) and the process about trigger through that callback .. so as soon as someone will have tagged/untagged a build , a json payload will be posted on the bus (to the MQTT broker but eventually we'll switch to something else later when we'll fully converge with Fedora infra) and a listener, subscribed to the bus reacts directly (or put it in a queue that is processed in serial)
So ideally that will happen in the seconds after the operation finished in Koji/CBS, but if multiple SIGs are tagging at the same time, they'll be queued and processed in serial (also directly after) , so no "timer" involved (no cron if that's your question)
On Wed, Mar 11, 2020 at 12:18 AM Fabian Arrotin arrfab@centos.org wrote:
On 10/03/2020 22:18, Ken Dreyer wrote:
That's great, thank you.
Can I see application or configuration for how often you will run it? Is it event-driven, or on a timer?
Well, in the initial mail I mentioned a message bus with koji callback plugin (even the link to it ;-) ) and the process about trigger through that callback .. so as soon as someone will have tagged/untagged a build , a json payload will be posted on the bus (to the MQTT broker but eventually we'll switch to something else later when we'll fully converge with Fedora infra) and a listener, subscribed to the bus reacts directly (or put it in a queue that is processed in serial)
So ideally that will happen in the seconds after the operation finished in Koji/CBS, but if multiple SIGs are tagging at the same time, they'll be queued and processed in serial (also directly after) , so no "timer" involved (no cron if that's your question)
Ok thanks. I see the dist-repo commands now in https://github.com/CentOS/ansible-role-stylo/commit/36cb2c05fdd66014d10b826b...
- Ken
On Tue, Mar 10, 2020, at 12:27 PM, Fabian Arrotin wrote:
On 10/03/2020 16:17, Fabian Arrotin wrote:
Hi all (especially SIG members/contributors) ,
<snip> > > If you have questions, comments, feel free to answer in this thread > and/or #centos-devel on irc.freenode.net >
Replying to myself based on a discussion we had with Alfredo in #centos-devel : We'll also stop using mash to generate the initial repositories, so https://cbs.centos.org/repos will completely disappear after the full switch to new system (FWIW)
What's the replacement for these repos? Specifically, the -candidate versions.
V/r, James Cassell
On 10/03/2020 17:32, James Cassell wrote:
On Tue, Mar 10, 2020, at 12:27 PM, Fabian Arrotin wrote:
On 10/03/2020 16:17, Fabian Arrotin wrote:
Hi all (especially SIG members/contributors) ,
<snip> > > If you have questions, comments, feel free to answer in this thread > and/or #centos-devel on irc.freenode.net >
Replying to myself based on a discussion we had with Alfredo in #centos-devel : We'll also stop using mash to generate the initial repositories, so https://cbs.centos.org/repos will completely disappear after the full switch to new system (FWIW)
What's the replacement for these repos? Specifically, the -candidate versions.
V/r, James Cassell
-candidate has never been really something that was supposed to be exposed as repo to be consumed from. That's the reason only -testing (as name implies it) and -release were pushed out. We'll not generate "ready-to-be-consumed" repos for -candidate tags, but individually people can still "cbs download-build $build_id" straight from koji/cbs if they want to .
So said differently, we expect SIG to test (through CI and also externally) what's in the -testing tag. Does it make sense ? :)
On Tue, Mar 10, 2020, at 12:42 PM, Fabian Arrotin wrote:
On 10/03/2020 17:32, James Cassell wrote:
On Tue, Mar 10, 2020, at 12:27 PM, Fabian Arrotin wrote:
On 10/03/2020 16:17, Fabian Arrotin wrote:
Hi all (especially SIG members/contributors) ,
<snip> > > If you have questions, comments, feel free to answer in this thread > and/or #centos-devel on irc.freenode.net >
Replying to myself based on a discussion we had with Alfredo in #centos-devel : We'll also stop using mash to generate the initial repositories, so https://cbs.centos.org/repos will completely disappear after the full switch to new system (FWIW)
What's the replacement for these repos? Specifically, the -candidate versions.
V/r, James Cassell
-candidate has never been really something that was supposed to be exposed as repo to be consumed from. That's the reason only -testing (as name implies it) and -release were pushed out. We'll not generate "ready-to-be-consumed" repos for -candidate tags, but individually people can still "cbs download-build $build_id" straight from koji/cbs if they want to .
So said differently, we expect SIG to test (through CI and also externally) what's in the -testing tag. Does it make sense ? :)
Is/will everything from the -candidate repos be/available in the testing repos? What's the intended purpose/use of -candidate today?
V/r, James Cassell
On Tue, Mar 10, 2020 at 04:17:42PM +0100, Fabian Arrotin wrote:
Hi all (especially SIG members/contributors) ,
Content trimmed.
Fabian,
This is astounding. Gets rid of yet another SPOF within the project that has needed to be dealt with for, quite literally, years now.
Spoken as merely a consumer... thank you for all this effort.
John
Thanks for the update, looking forward to this! Also thanks for your hard work on this!
Il giorno mar 10 mar 2020 alle ore 16:18 Fabian Arrotin arrfab@centos.org ha scritto:
Hi all (especially SIG members/contributors) ,
As announced by Jim some time ago, we wanted to redesign the current signing process that is in place for https://cbs.centos.org for CentOS 6 and 7 (so quite some years now)
The goal is to automate as much as possible the workflow, and working for all releases (yeah for CentOS 8 and Stream)
Thomas and myself have worked on the following idea and we're now happy with the results (in our Dev environment) :
- when someone builds a pkg, and that it's tagged to -testing, koji
sends a notification (through koji callback - https://fedoraproject.org/wiki/Koji/WritingKojiCode#Event_Plugin) to a bus
- signing machine listens to the bus, process the tag and push directly
to buildlogs
- when someone tag-build said pkg to -release, the node signs it with
correct gpg key id (from the SIG), push generated repository (including for debuginfo/src.rpm packages) to mirror CDN
As we were working on such automation, we also confirm that if one uses "untag-build", the same rule applies : it triggers automatically koji to reprocess the tag/repo and so removes the pkg from mirror (we got some requests for that in the past)
The process so would work 24/7, and also have monitoring built-in (so that we can detect - the infra team - when something doesn't work, and so we can still process the tag again)
We recently upgraded cbs.centos.org to newer koji version that permits some of the needed operations for this to happen.
At this stage, we are just missing the SIG GPG private keys, but we were promised to have those this week.
We don't have any ETA yet for the migration to new signing system, but we were thinking about such plan:
- deploy and configure (automatically) new signing node (this week)
- import GPG keys in the new system and validate that we can sign (when
we get those)
- changes some flags/parameters on tags on cbs.centos.org (this week)
Once we have this in place, we'd like to start validating the signing process but *not* pushing to mirror CDN (yet). Same for the bus : we'd like to process all tags in advance as we'll have to import RPM signatures back in koji/cbs.
After full validation, we announce that new system would be fully in place, and kojihub restarted with the notification plugin. From that point, the system would be fully operational and so when SIG members would build/tag pkgs, that would trigger the push (and signing if needed for -release tag)
Worth knowing that with the update to some components in the chain, the directory layout will change a little bit (like people are already used to if using Fedora and Epel) : the Packages will be splitted by first letter. Here is an example from our testing :
└── storage └── x86_64 └── gluster-7 ├── Packages │ ├── g │ │ ├── glusterfs-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-api-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-api-devel-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-cli-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-client-xlators-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-cloudsync-plugins-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-devel-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-events-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-extra-xlators-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-fuse-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-geo-replication-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-libs-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-rdma-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-resource-agents-7.2-1.el7.noarch.rpm │ │ ├── glusterfs-server-7.2-1.el7.x86_64.rpm │ │ └── glusterfs-thin-arbiter-7.2-1.el7.x86_64.rpm │ ├── p │ │ └── python2-gluster-7.2-1.el7.x86_64.rpm │ └── u │ ├── userspace-rcu-0.7.16-3.el7.x86_64.rpm │ └── userspace-rcu-devel-0.7.16-3.el7.x86_64.rpm └── repodata ├──
047503df3313d596e14a1a0d097b6c2261b26690000ecd4260d212a6f5e5632d-other.sqlite.bz2 ├──
34c1f5dd954099816d6cce289e52f96e61cc2e6430e13072194e35c063b0e3da-primary.xml.gz ├──
6293e07e4c5a7acc2c610beb5acb84b996329ef3c6d61c03e375f2afc6a71536-filelists.sqlite.bz2 ├──
6f89e8b43fb7e384f86f041c4de84d9afeb15dca981559b15230cb48887095db-other.xml.gz ├──
bef236563d185417fbda48e3b4dd1d62714d1af2b7a7d6dc1a7213af5de1aa9b-primary.sqlite.bz2 ├──
fb66c148e8e2738bd5d19b88d1b3507b0383aaab0938de3809b399babb1a0b44-filelists.xml.gz ├── repomd.xml └── repomd.xml.asc
From a usage perspective that will not change the way pkgs can be consumed, as repodata will always be at the right place, but in the case of SCLs that will mean no subdirectory like before, but instead just same "first letter" name (while repodata will be global)
So in the current stage, we'd like to deploy and fully validate the signing setup and we'll announce on this list when we'll fully switch to new system.
Also worth noting that with the new system in place, in theory we'll *not* use the old cbs-content-control git repo (as we'll query automatically koji for tags, enabled architectures, etc) so tagging to -testing and -release should trigger automatically the "push to mirrors" step
If you have questions, comments, feel free to answer in this thread and/or #centos-devel on irc.freenode.net
Kind Regards,
Thomas and Fabian
-- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On Tue, Mar 10, 2020 at 04:17:42PM +0100, Fabian Arrotin wrote:
Hi all (especially SIG members/contributors) ,
As announced by Jim some time ago, we wanted to redesign the current signing process that is in place for https://cbs.centos.org for CentOS 6 and 7 (so quite some years now)
The goal is to automate as much as possible the workflow, and working for all releases (yeah for CentOS 8 and Stream)
This is really great! Thanks for all the work and the sharing the current status.
Niels
On Tue, Mar 10, 2020 at 4:18 PM Fabian Arrotin arrfab@centos.org wrote:
Hi all (especially SIG members/contributors) ,
As announced by Jim some time ago, we wanted to redesign the current signing process that is in place for https://cbs.centos.org for CentOS 6 and 7 (so quite some years now)
The goal is to automate as much as possible the workflow, and working for all releases (yeah for CentOS 8 and Stream)
Thomas and myself have worked on the following idea and we're now happy with the results (in our Dev environment) :
- when someone builds a pkg, and that it's tagged to -testing, koji
sends a notification (through koji callback - https://fedoraproject.org/wiki/Koji/WritingKojiCode#Event_Plugin) to a bus
- signing machine listens to the bus, process the tag and push directly
to buildlogs
- when someone tag-build said pkg to -release, the node signs it with
correct gpg key id (from the SIG), push generated repository (including for debuginfo/src.rpm packages) to mirror CDN
Any recommendation for eficient bulk tagging. In some cases we (CloudSIG) tag more that 300 builds together, will kojiclient multicall api feature help the new system to do the sign/push more efficiently (btw, we are already using that multicall feature when tagging).
As we were working on such automation, we also confirm that if one uses "untag-build", the same rule applies : it triggers automatically koji to reprocess the tag/repo and so removes the pkg from mirror (we got some requests for that in the past)
The process so would work 24/7, and also have monitoring built-in (so that we can detect - the infra team - when something doesn't work, and so we can still process the tag again)
We recently upgraded cbs.centos.org to newer koji version that permits some of the needed operations for this to happen.
At this stage, we are just missing the SIG GPG private keys, but we were promised to have those this week.
We don't have any ETA yet for the migration to new signing system, but we were thinking about such plan:
- deploy and configure (automatically) new signing node (this week)
- import GPG keys in the new system and validate that we can sign (when
we get those)
- changes some flags/parameters on tags on cbs.centos.org (this week)
Currently, there are builds in -testing and -release tags which are not signed nor shipped (both, centos8 and centos7 multiarch, i.e.), will this pending builds be processed somehow when moving to the new system, or an untag/tag will be needed?
Once we have this in place, we'd like to start validating the signing process but *not* pushing to mirror CDN (yet). Same for the bus : we'd like to process all tags in advance as we'll have to import RPM signatures back in koji/cbs.
After full validation, we announce that new system would be fully in place, and kojihub restarted with the notification plugin. From that point, the system would be fully operational and so when SIG members would build/tag pkgs, that would trigger the push (and signing if needed for -release tag)
Worth knowing that with the update to some components in the chain, the directory layout will change a little bit (like people are already used to if using Fedora and Epel) : the Packages will be splitted by first letter. Here is an example from our testing :
└── storage └── x86_64 └── gluster-7 ├── Packages │ ├── g │ │ ├── glusterfs-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-api-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-api-devel-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-cli-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-client-xlators-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-cloudsync-plugins-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-devel-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-events-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-extra-xlators-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-fuse-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-geo-replication-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-libs-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-rdma-7.2-1.el7.x86_64.rpm │ │ ├── glusterfs-resource-agents-7.2-1.el7.noarch.rpm │ │ ├── glusterfs-server-7.2-1.el7.x86_64.rpm │ │ └── glusterfs-thin-arbiter-7.2-1.el7.x86_64.rpm │ ├── p │ │ └── python2-gluster-7.2-1.el7.x86_64.rpm │ └── u │ ├── userspace-rcu-0.7.16-3.el7.x86_64.rpm │ └── userspace-rcu-devel-0.7.16-3.el7.x86_64.rpm └── repodata ├──
047503df3313d596e14a1a0d097b6c2261b26690000ecd4260d212a6f5e5632d-other.sqlite.bz2 ├──
34c1f5dd954099816d6cce289e52f96e61cc2e6430e13072194e35c063b0e3da-primary.xml.gz ├──
6293e07e4c5a7acc2c610beb5acb84b996329ef3c6d61c03e375f2afc6a71536-filelists.sqlite.bz2 ├──
6f89e8b43fb7e384f86f041c4de84d9afeb15dca981559b15230cb48887095db-other.xml.gz ├──
bef236563d185417fbda48e3b4dd1d62714d1af2b7a7d6dc1a7213af5de1aa9b-primary.sqlite.bz2 ├──
fb66c148e8e2738bd5d19b88d1b3507b0383aaab0938de3809b399babb1a0b44-filelists.xml.gz ├── repomd.xml └── repomd.xml.asc
From a usage perspective that will not change the way pkgs can be consumed, as repodata will always be at the right place, but in the case of SCLs that will mean no subdirectory like before, but instead just same "first letter" name (while repodata will be global)
So in the current stage, we'd like to deploy and fully validate the signing setup and we'll announce on this list when we'll fully switch to new system.
Also worth noting that with the new system in place, in theory we'll *not* use the old cbs-content-control git repo (as we'll query automatically koji for tags, enabled architectures, etc) so tagging to -testing and -release should trigger automatically the "push to mirrors" step
If you have questions, comments, feel free to answer in this thread and/or #centos-devel on irc.freenode.net
Kind Regards,
Thomas and Fabian
-- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On 12/03/2020 10:35, Alfredo Moralejo Alonso wrote:
<snip>
Any recommendation for eficient bulk tagging. In some cases we (CloudSIG) tag more that 300 builds together, will kojiclient multicall api feature help the new system to do the sign/push more efficiently (btw, we are already using that multicall feature when tagging).
That's a good call : I'll verify myself in Dev, as I have no clue about how Koji does that : hopefully indeed it does send through the callback only one tagBuild notification, and if it doesn't, that would mean 300 triggers in the same queue, each signing one pkg and calling dist-repo ... that would still work and end result would be the same, but just longer ... and at the same time probably faster than $current_process :)
On Fri, Mar 13, 2020 at 7:53 AM Fabian Arrotin arrfab@centos.org wrote:
On 12/03/2020 10:35, Alfredo Moralejo Alonso wrote:
<snip> > > > Any recommendation for eficient bulk tagging. In some cases we > (CloudSIG) tag more that 300 builds together, will kojiclient multicall > api feature help the new system to do the sign/push more efficiently > (btw, we are already using that multicall feature when tagging). >
That's a good call : I'll verify myself in Dev, as I have no clue about how Koji does that : hopefully indeed it does send through the callback only one tagBuild notification, and if it doesn't, that would mean 300 triggers in the same queue, each signing one pkg and calling dist-repo ... that would still work and end result would be the same, but just longer ... and at the same time probably faster than $current_process :)
I'm eager to see the new process working and I'm sure it will improve the current situation :). I was just thinking about possible optimizations for our specific workflows.
-- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On 10/03/2020 16:17, Fabian Arrotin wrote:
Hi all (especially SIG members/contributors) ,
As announced by Jim some time ago, we wanted to redesign the current signing process that is in place for https://cbs.centos.org for CentOS 6 and 7 (so quite some years now)
The goal is to automate as much as possible the workflow, and working for all releases (yeah for CentOS 8 and Stream)
Thomas and myself have worked on the following idea and we're now happy with the results (in our Dev environment) :
- when someone builds a pkg, and that it's tagged to -testing, koji
sends a notification (through koji callback - https://fedoraproject.org/wiki/Koji/WritingKojiCode#Event_Plugin) to a bus
- signing machine listens to the bus, process the tag and push directly
to buildlogs
- when someone tag-build said pkg to -release, the node signs it with
correct gpg key id (from the SIG), push generated repository (including for debuginfo/src.rpm packages) to mirror CDN
Hi,
thank you for putting this stuff together, this is really great.
Can we have tags for opstools and messaging SIG?
opstools[78]-collectd-5-{candidate|testing|release} messaging[78]-qpid-0.30-{candidate|... messaging[78]-rabbitmq-3.8-{candidate|...
Thank you. Matthias
-
Alfredo Moralejo Alonso -
Fabian Arrotin -
James Cassell -
John R. Dennison -
Ken Dreyer -
Matthias Runge -
Niels de Vos -
Sandro Bonazzola