CI-users

ci-users@lists.centos.org

416 discussions

Infra - planned outage : duffy.ci.centos.org
by Fabian Arrotin 27 Jun '23

27 Jun '23

Hi, Due to small (unscheduled) outage we had with Duffy earlier today (more info on https://pagure.io/centos-infra/issue/1204), we'll have to pause Duffy API to ensure Duffy DB and provisioned aws infra can be reconciled. Migration is scheduled for """"Tuesday June 27th, 7:00 am UTC time"""". You can convert to local time with $(date -d '2023-06-27 07:00 UTC') The expected "downtime" is estimated to ~10 minutes , time needed to : - stop duffy API service - terminate all Duffy EC2 instances (including the ones in "unknown" state, from a Duffy DB PoV) - clean up Duffy DB to reflect "starting from scratch" state - restart Duffy API service (from that point, Duffy will reprovision all the pools automatically) Thanks for your understanding and patience. on behalf of the Infra team, -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]

1 1

cannot reserve machines
by Vladimir Benes 24 May '23

24 May '23

Hi all, any idea why I see this in all jobs? I have just one job running, so my 10 machines quota is not reached. 24/May/2023 08:52:54:DEBUG:executed 'duffy client --url https://duffy.ci.centos.org/api/v1 --auth-name networkmanager --auth- key $CICO_API_KEY request-session pool=virt-ec2-t2-centos-9s- x86_64,quantity=2': returned 0 24/May/2023 08:52:54:DEBUG:machine content { "error": { "detail": "can't reserve nodes: quantity=2 pool='virt-ec2-t2- centos-9s-x86_64'" } } Thanks, Vlad

2 2

Scheduled Upgrade Notification: CentOS CI Production Openshift Cluster
by David Kirwan 02 May '23

02 May '23

Dear CentOS CI User, Please be advised that the CentOS CI Production Openshift cluster is due for scheduled maintenance from 06:30 to 09:00 UTC on 2023-05-02. During this period necessary updates will be applied to the cluster, and will be upgraded from version 4.11.35 to 4.11.37, and from there to 4.12.13. The upgrade is not expected to be service affecting, however there might be periods of slight performance degradation during this maintenance period. We apologise for any inconvenience that this may cause. Kind regards, CentOS CI Infrastructure. -- David Kirwan Senior Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108

4 5

Scheduled Upgrade Notification: CentOS CI Staging Openshift Cluster
by David Kirwan 28 Apr '23

28 Apr '23

Dear CentOS CI User, Please be advised that the CentOS CI Staging Openshift cluster is due for scheduled maintenance from 06:30 to 09:00 UTC on 2023-04-28. During this period necessary updates will be applied to the cluster, and will be upgraded from version 4.11.35 to 4.11.37, and from there to 4.12.13. The upgrade is not expected to be service affecting, however there might be periods of slight performance degradation during this maintenance period. We apologise for any inconvenience that this may cause. Kind regards, CentOS CI Infrastructure. -- David Kirwan Senior Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108

1 1

Scheduled Upgrade Notification: CentOS CI Production Openshift Cluster
by David Kirwan 24 Apr '23

24 Apr '23

Dear CentOS CI User, Please be advised that the CentOS CI Production Openshift cluster is due for scheduled maintenance from 08:00 to 09:00 UTC on 2023-04-24. During this period necessary updates will be applied to the cluster, and will be upgraded from version 4.11.30 to 4.11.35, from there we will move the production cluster onto the 4.12 stable branch. The upgrade is not expected to be service affecting, however there might be periods of slight performance degradation during this maintenance period. We apologise for any inconvenience that this may cause. Kind regards, CentOS CI Infrastructure. -- David Kirwan Senior Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108

1 1

Scheduled Upgrade Notification: CentOS CI Staging Openshift Cluster
by David Kirwan 21 Apr '23

21 Apr '23

Dear CentOS CI User, Please be advised that the CentOS CI Staging Openshift cluster is due for scheduled maintenance from 08:00 to 09:00 UTC on 2023-04-21. During this period necessary updates will be applied to the cluster, and will be upgraded from version 4.11.30 to 4.11.35, from there we will move the staging cluster onto the 4.12 stable branch. The upgrade is not expected to be service affecting, however there might be periods of slight performance degradation during this maintenance period. We apologise for any inconvenience that this may cause. Kind regards, CentOS CI Infrastructure. -- David Kirwan Senior Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108

1 1

Latest kernel does not boot
by Filip Pokryvka 12 Apr '23

12 Apr '23

Hi all, we tried to upgrade the system and reboot before running the tests. CentOS 8 stream works fine, however newest kernel in 9 stream does not boot in AWS cloud, machine remains unresponsive, unable to ping, SSH... Build 283 works, 295 and also newer from kojihub are broken, I did not bisect to find the exact build, but it is somewhere in the range. Is this a known issue? Thanks, Filip

3 3

wrong checksums
by Vladimir Benes 05 Apr '23

05 Apr '23

heya, there seems to be some inconsistency when firing up a machine. any idea what's wrong? Yesterday it was all good, we saw it first here: Wednesday, April 5, 2023 at 12:00:01 AM Thanks, Vladimir STDERR: Errors during downloading metadata for repository 'baseos': - Downloading successful, but checksum doesn't match. Calculated: 67f0fad4fe457471c91c70a50245e5700a98c4b49d509d35a8b6643eed55e789b3afb5c ed888da13b63a3bdfd122e7aecc8b6c485a9dedcd1afca23cf81fa288(sha512) Expected: 60642857dc1e825a77340c6ffe13eb4c84f0c3c1972dd4dfc7ab8e00c6b1980f63a5d33 1bd97f5a9808466c657325a10631840267e80cfb5a87b1adb28d52947(sha512) Error: Failed to download metadata for repo 'baseos': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

3 3

[Action Required] : Decommissioning (on premises) OpenShift CI cluster
by Fabian Arrotin 03 Apr '23

03 Apr '23

Hi all, Camilla announced this in december already (https://lists.centos.org/pipermail/ci-users/2022-December/thread.html) and we sent multiple reminders about that. The plan was already knew since it was decided to move the whole CI infra to AWS (announced in June 2022) but the last bit that had to move was the openshift CI cluster (aka *.apps.ocp.ci.centos.org) to AWS (https://console-openshift-console.apps.ocp.cloud.ci.centos.org/) As the new cluster in AWS is deployed since end of November you still have to migrate your workload from old to new cluster, and that *before* end of March 2023 (see https://sigs.centos.org/guide/ci/#migration-to-new-ci-instance) We'll continue to send such reminders as long as we see tenants who haven't migrated (yet) We prefer announce it multiple times so that nobody will ask "where is my app gone ?" when the whole cluster will be powered down end of March. PS: if you have already migrated, please confirm to us so that we can already delete your project/namespace/storage PV in the previous cluster. Kind Regards, -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab

6 11

Scheduled UPGRADE Notification: CentOS CI PROD Openshift Cluster
by Pedro Moura 17 Mar '23

17 Mar '23

Dear CentOS CI User, Please be advised that the CentOS CI PROD Openshift cluster is due for scheduled maintenance from 12pm to 2pm UTC on Mar 17th. During this period necessary updates will be applied to the cluster, and will be upgraded from version 4.11.28 to 4.11.30 Every service under **apps.ocp.cloud.ci.centos.org <http://apps.ocp.cloud.ci.centos.org/>* might be affected. Pods will be restarted, so it's suggested to pause your Jenkins jobs, and there might be periods of slight performance degradation during this maintenance period. We apologise for any inconvenience that this may cause. Kind regards, Pedro Moura he/him/his Software Engineer Red Hat <https://www.redhat.com> pmoura(a)redhat.com <https://www.redhat.com>

1 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

CI-users