[CentOS-devel] unplanned outage : community cage network issue

Sun Oct 1 20:42:33 UTC 2023
Fabian Arrotin <arrfab at centos.org>

Hi all,

this Sunday, our zabbix monitoring platform informed us that there were 
plenty of nodes unreachable in community cage, and after some 
investigation it's a whole rack that was cut off access from network 
switches.
That causes issues for multiple workloads, including duffy CI infra env, 
CentOS Stream env (aka https://testing.stream.centos.org) or 
storage/mirror servers used for https://cbs.centos.org.

As we don't know when the root cause will be investigated by the network 
team managing the network infra, we have disabled the koji builders, so 
don't even try to submit builds to cbs.centos.org as they'd just be 
stuck waiting for builders to be available (themselves disabled until we 
have access to internal mirror)

-- 
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <http://lists.centos.org/pipermail/centos-devel/attachments/20231001/00b514d4/attachment.sig>