On 05/12/2022 12:45, Peter Georg wrote: > On 02/12/2022 09.13, Fabian Arrotin wrote: >> Just for awareness : I noticed this morning through monitoring that >> the cache/proxy used by kojid builders (needed to reach out to >> gitlab/etc) was down during the night, so while kojihub itself was >> available, the kojid nodes weren't even checking in. >> >> I quickly reconfigured (with ansible) another caching proxy and so >> restored connectivity. If you noticed some build issues during some >> hours, please resubmit these, but it seems most were just queued and >> so picked up "normally" as soon as connection was restored. > > Since approx. 1st of December I've been noticing issues running cbs > commands within GitLab CI. Most of the cbs commands are running into a > time out, no connection can be established. No tasks are ever created. > This issue still exists. > > Might this issue be related to the cache/proxy used for the kojid nodes? No, not related, but last week we also suffered from massive remote koji api calls , that was taking cbs/koji down on its knees. After having discussed with Fedora infra team colleagues, they said they had the same issue the week[s] before and had to just drop IP ranges to ensure koji.fedoraproject.org was back online. I quickly took same emergency measure (had really other things to do) to restore cbs.centos.org availability and so just also dropped some ranges. Probably gitlab is hosted in one of these ranges ? In all cases, it's better to open an infra ticket (in parallel, as it's also good to discuss on the centos-devel list too) -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 840 bytes Desc: OpenPGP digital signature URL: <http://lists.centos.org/pipermail/centos-devel/attachments/20221205/a9e317db/attachment-0003.sig>