CI-users

ci-users@lists.centos.org

416 discussions

[infra] Planned outage : RDU2 network maintenance (please read) : 2023-11-17
by Fabian Arrotin 15 Nov '23

15 Nov '23

Due to a mandatory network maintenance at the RDU2 (Community Cage) facility/DC, we'll have to power off some CentOS infrastructure components hosted in that DC. Impacted services: - https://cbs.centos.org (impossible to submit any build, nor push to delivery network) - ns1.centos.org - all centos linux / stream 8 / stream 9 release engineering workflow (stream 8/9 can be built but not pushed out, while for the rest, all will be down) - https://git.centos.org - CI infra: - while openshift in aws is self-contained, there will be no way to request duffy nodes for tests - various other services Migration is scheduled for """"Friday November 17th, 1:00 pm UTC time"""". You can convert to local time with $(date -d '2023-11-17 13:00 UTC') The scheduled maintenance window (communicated to us) is estimated to 3h, but we'll restart infra services as soon as possible, following ourselves the current status internally. Thanks for your understanding and patience. on behalf of the Infra team, -- -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]

1 0

Scheduled Upgrade Notification: CentOS CI Production Openshift Cluster
by Fabian Arrotin 09 Nov '23

09 Nov '23

Dear CentOS CI User, Please be advised that the CentOS CI Production Openshift cluster is due for scheduled maintenance from 08:00 to 10:00 UTC on 2023-11-13 . During this period necessary updates will be applied to the cluster. The upgrade is not expected to be service affecting, however there might be periods of slight performance degradation during this maintenance period. We apologize for any inconvenience that this may cause. Kind regards, CentOS CI Infrastructure. -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]

1 0

unplanne outage : community cage network issue
by Fabian Arrotin 02 Oct '23

02 Oct '23

Hi all, this Sunday, our zabbix monitoring platform informed us that there were plenty of nodes unreachable in community cage, and after some investigation it's a whole rack that was cut off access from network switches. That causes issues for multiple workloads, including duffy CI infra env. We'll verify with RH IT (managing the network switches) how network connectivity can be restored but if that's due to faulty network switch, we'll have no ETA, as (reminder) everything in that cage runs on out-of-warranty hardware, and so without any SLA/SLE. We'll keep you informed about status when possible -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]

1 1

Scheduled Upgrade Notification: CentOS CI Production Openshift Cluster
by David Kirwan 29 Sep '23

29 Sep '23

Dear CentOS CI User, Please be advised that the CentOS CI Production Openshift cluster is due for scheduled maintenance from 08:00 to 10:00 UTC on 2023-09-29. During this period necessary updates will be applied to the cluster. The upgrade is not expected to be service affecting, however there might be periods of slight performance degradation during this maintenance period. We apologise for any inconvenience that this may cause. Kind regards, CentOS CI Infrastructure. -- David Kirwan Senior Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108

1 2

Scheduled Upgrade Notification: CentOS CI Staging Openshift Cluster
by David Kirwan 28 Sep '23

28 Sep '23

Dear CentOS CI User, Please be advised that the CentOS CI Staging Openshift cluster is due for scheduled maintenance from 08:00 to 10:00 UTC on 2023-09-28. During this period necessary updates will be applied to the cluster. The upgrade is not expected to be service affecting, however there might be periods of slight performance degradation during this maintenance period. We apologise for any inconvenience that this may cause. Kind regards, CentOS CI Infrastructure. -- David Kirwan Senior Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108

1 1

Scheduled Upgrade Notification: CentOS CI Production Openshift Cluster
by David Kirwan 11 Sep '23

11 Sep '23

Dear CentOS CI User, Please be advised that the CentOS CI Production Openshift cluster is due for scheduled maintenance from 08:00 to 10:00 UTC on 2023-09-11. During this period necessary updates will be applied to the cluster. The upgrade is not expected to be service affecting, however there might be periods of slight performance degradation during this maintenance period. We apologise for any inconvenience that this may cause. Kind regards, CentOS CI Infrastructure. -- David Kirwan Senior Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108

1 1

Scheduled Upgrade Notification: CentOS CI Staging Openshift Cluster
by David Kirwan 08 Sep '23

08 Sep '23

Dear CentOS CI User, Please be advised that the CentOS CI Staging Openshift cluster is due for scheduled maintenance from 08:00 to 10:00 UTC on 2023-09-08. During this period necessary updates will be applied to the cluster. The upgrade is not expected to be service affecting, however there might be periods of slight performance degradation during this maintenance period. We apologise for any inconvenience that this may cause. Kind regards, CentOS CI Infrastructure. -- David Kirwan Senior Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108

1 1

Infra - planned outage (short notice) : duffy.ci.centos.org
by Fabian Arrotin 07 Aug '23

07 Aug '23

Hi, Context : https://pagure.io/centos-infra/issue/1211 We'll apply some duffy codebase update and also have to pause Duffy API to ensure Duffy DB and provisioned aws infra can be reconciled. Migration is scheduled for """"Monday August 7th 12:00 am UTC time"""". You can convert to local time with $(date -d '2023-08-07 12:00 UTC') The expected "downtime" is estimated to ~10 minutes , time needed to : - stop duffy API service - upgrade to version 3.3.9 (and upgrade database schema) - terminate all Duffy EC2 instances (including the ones in "unknown" state, from a Duffy DB PoV) - clean up Duffy DB to reflect "starting from scratch" state - restart Duffy API service (from that point, Duffy will reprovision all the pools automatically) Thanks for your understanding and patience. on behalf of the Infra team, -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]

1 1

Scheduled Upgrade Notification: CentOS CI Production Openshift Cluster
by David Kirwan 18 Jul '23

18 Jul '23

Dear CentOS CI User, Please be advised that the CentOS CI Production Openshift cluster is due for scheduled maintenance from 08:00 to 10:00 UTC on 2023-07-17. During this period necessary updates will be applied to the cluster. The upgrade is not expected to be service affecting, however there might be periods of slight performance degradation during this maintenance period. We apologise for any inconvenience that this may cause. Kind regards, CentOS CI Infrastructure. -- David Kirwan Senior Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108

1 2

Scheduled Upgrade Notification: CentOS CI Staging Openshift Cluster
by David Kirwan 13 Jul '23

13 Jul '23

Dear CentOS CI User, Please be advised that the CentOS CI Staging Openshift cluster is due for scheduled maintenance from 08:00 to 10:00 UTC on 2023-07-13. During this period necessary updates will be applied to the cluster. The upgrade is not expected to be service affecting, however there might be periods of slight performance degradation during this maintenance period. We apologise for any inconvenience that this may cause. Kind regards, CentOS CI Infrastructure. -- David Kirwan Senior Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108

1 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

CI-users