CI-users

ci-users@lists.centos.org

416 discussions

Updated cico-workspace container in OCP cluster
by Fabian Arrotin 21 Oct '21

21 Oct '21

Follow-up on the previous mail about supporting 9-stream in CI infra. We have refreshed the cico-workspace container that is deployed as jenkins agent container and it's available on https://quay.io/repository/centosci/cico-workspace That's the version that is automatically deployed in the OCP cluster. Changes: - updated to CentOS 7.9 + updates - updated python-cicoclient to 0.4.6 (version that supports requesting for '9-stream' nodes to duffy) - added ansible 2.9.27 (per request from some tenants, avoiding to have to install it themselves) Next steps: We'd like to rebase that container to 8-stream in the near future but we'd just need first to have python-cicoclient available as pkg and tested with python3. Once we'll have such test image, we'll rebase and keep you informed about it -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab

1 0

[infra] : Planned outage : Jenkins/CI environment
by Fabian Arrotin 28 Sep '21

28 Sep '21

Due to some needed software updates, we'll have to shutdown and restart jenkins on the 'legacy' ci.centos.org setup (reminder : moving to openshift would be great and we'll have to put a hard deadline for this if some tenants are still on the old setup) Migration is scheduled for """"Tuesday September 28th, 9:00 am UTC time"""". You can convert to local time with $(date -d '2021-09-28 09:00 UTC') The expected "downtime" is estimated to ~30 minutes , time needed to put jenkins in "shutdown/quiet mode" , wait for some some jobs to finish, update and restart jenkins. Thanks for your comprehending and patience. on behalf of the Infra team, -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab

1 1

Fwd: IMPORTANT - ACTION MAY BE NEEDED - TravisCI security issues
by Mark O'Brien 16 Sep '21

16 Sep '21

Raising awareness here. I am not sure if anyone uses this on our openshift cluster but better safe than sorry. ---------- Forwarded message --------- From: Jay Madison <madisonj(a)redhat.com> Date: Wed, Sep 15, 2021 at 8:06 PM Subject: IMPORTANT - ACTION MAY BE NEEDED - TravisCI security issues To: <announce-list(a)redhat.com> Hi all, TL;DR: If your software development projects use TravisCI, please rotate your secrets as soon as possible, but by no later than close of business, September 17th. If you use TravisCI and have seen any first time contributors between Sep 03 - Sep 10, 2021, follow the steps below in the “What you need to do” section and contact infosec(a)redhat.com if you have any questions. If you are not involved with software development activities, using tools such as GitHub, GitLab or CI/CD tooling, this message very likely does not apply to you, and you may ignore it. What happened Travis CI is a hosted continuous integration service used to build and test software projects hosted on source code repositories such as GitHub. On September 13, TravisCI released a security bulletin <https://travis-ci.community/t/security-bulletin/12081>[1] advising that secret environment variables of any public repositories may have been leaked. This issue has been designated as CVE-2021-41077[2] This issue was reported to TravisCI by the community on September 7 and a patch was deployed by TravisCI on September 10. It is believed that all use between September 3rd and 10th may have been subject to this vulnerability, at a minimum. Given the limited information published by the Travis CI Team, it is impossible to rule out a broader range of potential impact. Information Security is in the process of scanning known Red Hat repositories, but we need your help. What you need to do If you have a repository that uses TravisCI: - Rotate your secrets as soon as possible but by no later than close of business September 17th. - Secrets refers to secure environment variables of all public repos using TravisCI. Items such as Signing Keys, Access Credentials, and API Keys. - Check for any external pull requests between September 3rd - 10th. - This includes first time pull request submitters, and people who don’t submit often. - In particular, look for tags FIRST_TIME_CONTRIBUTOR, FIRST_TIMER, or NONE. - To learn more about specific environment variables that may have been exposed, please visit: https://source.redhat.com/departments/it/it-information-security/wiki/septe… - For any pull requests of this nature, check the diff to see if it does something unusual, for example, dumping env variables. - If you are unsure of any of these steps, notice anything unusual, and/or unexpected activity please contact infosec(a)redhat.com. Thank you for your diligence in helping us keep Red Hat secure. As always, if there are any concerns, questions, or you wish to report an anomaly or potential incident, please contact infosec(a)redhat.com directly. Regards, J. Links: [1] TravisCI security bulletin: https://travis-ci.community/t/security-bulletin/12081 [2] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-41077 -- Jay Madison Vice President - Trust, Risk, Assurance & Compliance Red Hat, Inc. Forward any comments to mailto:memo-list@redhat.com for open discussion.

1 0

[infra] : Planned outage : CI environment
by Fabian Arrotin 17 Aug '21

17 Aug '21

Due to a previous hardware issue (https://lists.centos.org/pipermail/ci-users/2020-October/002124.html), we had a workaround in place that is still in place but is actually blocking a reorg of the infra used behind ci.centos.org. We are announcing a maintenance window that will permit us to move network switches, and reconfigure the network properly , allowing also us to redeploy/load balance our workload across remaining available infrastructure. Migration is scheduled for """"Tuesday August 17th, 1:00 pm UTC time"""". You can convert to local time with $(date -d '2021-08-17 13:00 UTC') The expected "downtime" is estimated to ~60 minutes , time needed to migrate services, reconfigure network switch and restart the nodes connected to new network switch. Impacted services in CI env: - duffy API (unavailable during the move to other hypervisor, so normally unavailable for ~10min) : no way to request a duffy node for bare-metal tests - jenkins (aka ci.centos.org) and all agents connected to it - cloud.cico (opennebula infra used to provide Virtual Machines for CI tenants) Thanks for your comprehending and patience. on behalf of the Infra team, -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab

1 1

Re: [Ci-users] URGENT - https://ci.centos.org/ is down
by Vipul Siddharth 17 Aug '21

17 Aug '21

On Tue, Aug 17, 2021 at 7:27 PM Jonathan Beakley <jbeakley(a)redhat.com> wrote: > +Ant and Stefan, because Leigh is on PTO. > Please see Fabian's previous announcement of scheduled outage [0] https://lists.centos.org/pipermail/ci-users/2021-August/002177.html > > -JB > > On Tue, Aug 17, 2021 at 9:54 AM Jonathan Beakley <jbeakley(a)redhat.com> > wrote: > >> +Leigh and Aoife >> >> Leigh and Aoife, >> >> Can you help get some attention on this issue? Thank you, >> >> -JB >> >> >> On Tue, Aug 17, 2021 at 9:35 AM Serhii Kryzhnii <skryzhni(a)redhat.com> >> wrote: >> >>> Hello, >>> >>> https://ci.centos.org/ returning 503 Service Unaviable >>> >>> Is there anyone that can look into this? >>> >>> Thanks, >>> >>> -- >>> Serhii Kryzhnii >>> Application SRE team, Service Delivery >>> Red Hat >>> skryzhni(a)redhat.com >>> >> >> >> -- >> >> Jonathan Beakley >> >> Senior Engineering Manager, Site Reliability Engineering Services >> >> Red Hat <https://www.redhat.com/>, 314 Littleton Road, Westford, MA >> 01886 >> >> jbeakley(a)redhat.com | Mobile: 617-529-2828 >> >> I respect your work-life balance. There is no need to answer this email >> outside of your office hours. >> <https://www.redhat.com/> >> > > > -- > > Jonathan Beakley > > Senior Engineering Manager, Site Reliability Engineering Services > > Red Hat <https://www.redhat.com/>, 314 Littleton Road, Westford, MA 01886 > > jbeakley(a)redhat.com | Mobile: 617-529-2828 > > I respect your work-life balance. There is no need to answer this email > outside of your office hours. > <https://www.redhat.com/> > -- Vipul Siddharth He/His/Him Infrastructure @ Fedora and CentOS <http://vipul.dev>

1 0

Planned Outage: CentOS CI OCP4 09:00 - 10:00 UTC Monday 28th June
by David Kirwan 28 Jun '21

28 Jun '21

Hi folks, Something to be aware of: https://pagure.io/centos-infra/issue/371 "' There will be a planned outage for the CentOS CI OCP4 cluster on Monday 28th June for a period of 1 hour from 09:00 UTC until 10:00 UTC but should not take the full hour to complete this work. This work is to allow us to restore our primary storage `storage02` back into production. This should drastically improve performance as we have moved to a RAID10 configuration. "' Thank you, if you have any questions, please comment them on the ticket -- David Kirwan Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108 IM: @dkirwan

1 1

CentOS CI OCP 4 cluster down for emergency maintenance
by David Kirwan 10 Jun '21

10 Jun '21

Hi ci-users, We're currently suffering an issue with our storage on the CentOS CI OCP 4 cluster, we'll be taking the cluster down for emergency maintenance immediately. Apologies for the inconvenience, we'll keep you updated once we know more. -- David Kirwan Software Engineer Community Platform Engineering @ Red Hat T: +(353) 86-8624108 IM: @dkirwan

2 2

Jobs failing with JNLP error
by Anjan Nath 04 Jun '21

04 Jun '21

Hi, We are seeing jobs[0] failing with some JNLP error from yesterday (console snip added below), not sure if this is known already or maybe related to the upgrade done? ``` 06:42:21 Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from slave02.ci.centos.org/172.19.0.15:57126 06:42:21 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800) 06:42:21 at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) 06:42:21 at hudson.remoting.Channel.call(Channel.java:1001) 06:42:21 at hudson.FilePath.act(FilePath.java:1158) 06:42:21 at hudson.FilePath.act(FilePath.java:1147) 06:42:21 at org.jenkinsci.plugins.gitclient.Git.getClient(Git.java:121) ``` [0] https://ci.centos.org/job/codeready-crc-pr/3612/console -- ANJAN NATH Red Hat - India Devtools Team, CDK

4 4

Why the file /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt disappeared
by Petr "Stone" Hracek 02 Jun '21

02 Jun '21

Hi folks, I am testing container in ci.centos.org and last week to looks like file /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt disappeared. ./tests/squash/squash.sh *10:19:49* open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory *10:19:50* *10:19:50* Traceback (most recent call last): *10:19:50* File "/root/sources/tests/squash/../../squash.py", line 22, in <module> *10:19:50* from_layer=sys.argv[2]).run() *10:19:50* File "/usr/local/python-tools/lib/python3.6/site-packages/docker_squash/squash.py", line 61, in run *10:19:50* return self.squash(image) *10:19:50* File "/usr/local/python-tools/lib/python3.6/site-packages/docker_squash/squash.py", line 91, in squash *10:19:50* new_image_id=image.squash() *10:19:50* File "/usr/local/python-tools/lib/python3.6/site-packages/docker_squash/image.py", line 73, in squash *10:19:50* self._before_squashing() *10:19:50* File "/usr/local/python-tools/lib/python3.6/site-packages/docker_squash/v2_image.py", line 15, in _before_squashing *10:19:50* super(V2Image, self)._before_squashing() *10:19:50* File "/usr/local/python-tools/lib/python3.6/site-packages/docker_squash/image.py", line 153, in _before_squashing *10:19:50* self.old_image_id = self.docker.inspect_image(self.image)['Id'] *10:19:50* File "/usr/local/python-tools/lib/python3.6/site-packages/docker/utils/decorators.py", line 17, in wrapped *10:19:50* 'Resource ID was not provided' *10:19:50* docker.errors.NullResource: Resource ID was not provided *10:19:50* make: *** [check-squash] Error 1 I am not able to pull an image registry.access.redhat.com/ubi8-micro Can you please help me what package is missing? Greetings Petr

1 0

ci.centos.org (Jenkins instance) update
by Vipul Siddharth 01 Jun '21

01 Jun '21

Good Morning everyone, On Wednesday (June 2, 2021) at 12 UTC, I will update ci.centos.org certificates and Jenkins version (with all compatible Plugins). This is just a heads up for that. Downtime shouldn't be more than a few minutes but if we notice anything wrong, will update here in the thread. Thank you -- Vipul Siddharth He/His/Him Fedora and CentOS Infrastructure

1 0

← Newer
1
...
4
5
6
7
8
9
10
...
42
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

CI-users