CI-users

ci-users@lists.centos.org

416 discussions

Nodes provided by ci.centos.org are not clean
by Katerina Foniok 02 Dec '19

02 Dec '19

Hi guys, from the last week, errors regarding node provided increased. For me, it seems like the nodes are not cleaned correctly, because a lot of stuff we use is already installed there and sometimes the folder with logs remains there too. I've summed up some errors in this issue: https://github.com/redhat-developer/rh-che/issues/1715 Thank you, have a nice day! Katka Foniok

2 1

Re: [Ci-users] Potentially mis-provisioned CentOS 8-stream nodes
by Vipul Siddharth 01 Dec '19

01 Dec '19

On Sun, Dec 1, 2019 at 8:09 PM František Šumšal <frantisek(a)sumsal.cz> wrote: > > On 12/1/19 3:04 PM, Vipul Siddharth wrote: > > Thank you for keeping me in cc, somehow I missed the email from Friday. > > Since your last email, I checked all 8-stream nodes to see if it's a > > thing with all but so far out of 15, I found just one node that has > > DNF missing (which is worse). I am trying a manual 8-Stream install to > > see if it's in a newly provisioned nodes or older/est ones (possibly > > catch the issue). > > Thank you for looking into this! While writing this email, I re-scheduled > two failed jobs which use CentOS 8 (not 8-stream), and both got a new > machine which had the same issue (missing `dnf`), so it's definitely not > isolated to CentOS 8-stream only. hmm, that's not good. CentOS 8-stream that I just tried manually had the dnf installed. WIll now try CentOS 8 and see if I can catch something. I will update here once I have figured out why this is happening. Sorry for this inconvenience. > > > All the ver=7 I checked have bash installed).. if possible, can you > > send me the ip/hostname for C7 node so that I can check it (if it > > hasn't deprovisioned) > > Unfortunately, the machine is long gone: > > ``` > Pseudo-terminal will not be allocated because stdin is not a terminal. > Warning: Permanently added 'n62.pufty,172.19.3.126' (RSA) to the list of known hosts. > bash: systemd-centos-ci/agent/bootstrap.sh: /usr/bin/bash: bad interpreter: No such file or directory > ``` > > However, so far it's been only a one time thing (let's hope it's going to > stay this way). > > > > > and the storage issue is now fixed. It was the whole central (Jenkins) > > master which was full and not the slave01. > > > > Thank you very much! > > > > On Sun, Dec 1, 2019 at 6:48 PM František Šumšal <frantisek(a)sumsal.cz> wrote: > >> > >> Hello again, > >> > >> Apart from the issue mentioned in the previous email (and which affects CentOS 8 nodes as well), > >> it looks like CentOS 7 nodes have a similar issue, but in this case it's missing `bash`. > >> > >> And to finish this beautiful weekend off, `systemd-ci-slave01` is apparently out of space: > >> > >> ``` > >> Archiving artifacts > >> ERROR: Step ‘Archive the artifacts’ aborted due to exception: > >> java.io.IOException: No space left on device > >> at sun.nio.ch.FileDispatcherImpl.write0(Native Method) > >> at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60) > >> at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) > >> at sun.nio.ch.IOUtil.write(IOUtil.java:65) > >> at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:211) > >> at java.nio.channels.Channels.writeFullyImpl(Channels.java:78) > >> at java.nio.channels.Channels.writeFully(Channels.java:101) > >> at java.nio.channels.Channels.access$000(Channels.java:61) > >> at java.nio.channels.Channels$1.write(Channels.java:174) > >> at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2315) > >> ``` > >> > >> On 11/29/19 10:56 AM, František Šumšal wrote: > >>> Hello! > >>> > >>> Today I noticed that approx. half of the CentOS 8-stream nodes our CI got its hands on today > >>> were missing `dnf`: > >>> > >>> ``` > >>> 2019-11-29 09:34:59,493 [agent-control/allocate_node] INFO: Attempting to allocate a node (8-stream x86_64) > >>> 2019-11-29 09:34:59,493 [agent-control/_execute_api_command] INFO: Duffy request URL: http://admin.ci.centos.org:8080/Node/get > >>> 2019-11-29 09:34:59,507 [connectionpool/_new_conn] INFO: Starting new HTTP connection (1): admin.ci.centos.org > >>> 2019-11-29 09:35:00,669 [agent-control/allocate_node] INFO: Successfully allocated node 'n29.pufty' (e819a934) > >>> 2019-11-29 09:35:00,669 [agent-control/<module>] INFO: PHASE 1: Setting up basic dependencies to configure CI repository > >>> ... > >>> Pseudo-terminal will not be allocated because stdin is not a terminal. > >>> Warning: Permanently added 'n29.pufty,172.19.3.93' (ECDSA) to the list of known hosts. > >>> bash: dnf: command not found > >>> ``` > >>> > >>> I'm not sure if it's already known, but it's indeed quite strange :-) > >>> > >>> Thanks! > >>> > >>> > >>> _______________________________________________ > >>> Ci-users mailing list > >>> Ci-users(a)centos.org > >>> https://lists.centos.org/mailman/listinfo/ci-users > >>> > >> > >> -- > >> PGP Key ID: 0xFB738CE27B634E4B > >> > > > > > > -- > PGP Key ID: 0xFB738CE27B634E4B > -- Vipul Siddharth He/His/Him Fedora | CentOS CI Infrastructure Team Red Hat w: vipul.dev

1 0

Re: [Ci-users] Potentially mis-provisioned CentOS 8-stream nodes
by Vipul Siddharth 01 Dec '19

01 Dec '19

Thank you for keeping me in cc, somehow I missed the email from Friday. Since your last email, I checked all 8-stream nodes to see if it's a thing with all but so far out of 15, I found just one node that has DNF missing (which is worse). I am trying a manual 8-Stream install to see if it's in a newly provisioned nodes or older/est ones (possibly catch the issue). All the ver=7 I checked have bash installed).. if possible, can you send me the ip/hostname for C7 node so that I can check it (if it hasn't deprovisioned) and the storage issue is now fixed. It was the whole central (Jenkins) master which was full and not the slave01. On Sun, Dec 1, 2019 at 6:48 PM František Šumšal <frantisek(a)sumsal.cz> wrote: > > Hello again, > > Apart from the issue mentioned in the previous email (and which affects CentOS 8 nodes as well), > it looks like CentOS 7 nodes have a similar issue, but in this case it's missing `bash`. > > And to finish this beautiful weekend off, `systemd-ci-slave01` is apparently out of space: > > ``` > Archiving artifacts > ERROR: Step ‘Archive the artifacts’ aborted due to exception: > java.io.IOException: No space left on device > at sun.nio.ch.FileDispatcherImpl.write0(Native Method) > at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60) > at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) > at sun.nio.ch.IOUtil.write(IOUtil.java:65) > at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:211) > at java.nio.channels.Channels.writeFullyImpl(Channels.java:78) > at java.nio.channels.Channels.writeFully(Channels.java:101) > at java.nio.channels.Channels.access$000(Channels.java:61) > at java.nio.channels.Channels$1.write(Channels.java:174) > at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2315) > ``` > > On 11/29/19 10:56 AM, František Šumšal wrote: > > Hello! > > > > Today I noticed that approx. half of the CentOS 8-stream nodes our CI got its hands on today > > were missing `dnf`: > > > > ``` > > 2019-11-29 09:34:59,493 [agent-control/allocate_node] INFO: Attempting to allocate a node (8-stream x86_64) > > 2019-11-29 09:34:59,493 [agent-control/_execute_api_command] INFO: Duffy request URL: http://admin.ci.centos.org:8080/Node/get > > 2019-11-29 09:34:59,507 [connectionpool/_new_conn] INFO: Starting new HTTP connection (1): admin.ci.centos.org > > 2019-11-29 09:35:00,669 [agent-control/allocate_node] INFO: Successfully allocated node 'n29.pufty' (e819a934) > > 2019-11-29 09:35:00,669 [agent-control/<module>] INFO: PHASE 1: Setting up basic dependencies to configure CI repository > > ... > > Pseudo-terminal will not be allocated because stdin is not a terminal. > > Warning: Permanently added 'n29.pufty,172.19.3.93' (ECDSA) to the list of known hosts. > > bash: dnf: command not found > > ``` > > > > I'm not sure if it's already known, but it's indeed quite strange :-) > > > > Thanks! > > > > > > _______________________________________________ > > Ci-users mailing list > > Ci-users(a)centos.org > > https://lists.centos.org/mailman/listinfo/ci-users > > > > -- > PGP Key ID: 0xFB738CE27B634E4B > -- Vipul Siddharth He/His/Him Fedora | CentOS CI Infrastructure Team Red Hat w: vipul.dev

1 0

Potentially mis-provisioned CentOS 8-stream nodes
by František Šumšal 29 Nov '19

29 Nov '19

Hello! Today I noticed that approx. half of the CentOS 8-stream nodes our CI got its hands on today were missing `dnf`: ``` 2019-11-29 09:34:59,493 [agent-control/allocate_node] INFO: Attempting to allocate a node (8-stream x86_64) 2019-11-29 09:34:59,493 [agent-control/_execute_api_command] INFO: Duffy request URL: http://admin.ci.centos.org:8080/Node/get 2019-11-29 09:34:59,507 [connectionpool/_new_conn] INFO: Starting new HTTP connection (1): admin.ci.centos.org 2019-11-29 09:35:00,669 [agent-control/allocate_node] INFO: Successfully allocated node 'n29.pufty' (e819a934) 2019-11-29 09:35:00,669 [agent-control/<module>] INFO: PHASE 1: Setting up basic dependencies to configure CI repository ... Pseudo-terminal will not be allocated because stdin is not a terminal. Warning: Permanently added 'n29.pufty,172.19.3.93' (ECDSA) to the list of known hosts. bash: dnf: command not found ``` I'm not sure if it's already known, but it's indeed quite strange :-) Thanks! -- Frantisek Sumsal GPG key ID: 0xFB738CE27B634E4B

1 0

Nodes are not provisioned
by Katerina Foniok 27 Nov '19

27 Nov '19

Hi guys, we are experiencing the issue with provisioning nodes on ci.centos.org. The issue started for us ~9 PM UTC yesterday. Our jobs are running on devtools-ci-slave04. Do you have any information about this? Thank you, have a great day! Katka Foniok Che QE

2 2

CentOS 8 and CentOS 8-Stream now available in CI
by Vipul Siddharth 13 Nov '19

13 Nov '19

Hello everyone, This is an email to inform you that now you can checkout CentOS 8 and CentOS 8-Stream nodes in CI. We have upgraded all the agents with the latest cico client which supports the parameter --release=8 and --release=8-stream. Here is a minimal example to get an 8/8-stream node `cico node get --release=8` `cico node get --release=8-stream` You should be able to use all other parameters like earlier. In case you see any problem with them, please let us know here or on #centos-ci@Freenode. Thank you -- Vipul Siddharth He/His/Him Fedora | CentOS CI Infrastructure Team Red Hat w: vipul.dev

8 15

Temporarily disabling CentOS 6 VMs in Duffy
by Brian Stinson 29 Oct '19

29 Oct '19

Hi Folks, We're noticing some trouble provisioning nodes in duffy. This trouble is isolated to CentOS 6. When we hit this behavior, quite a few instances end up in the 'Failed' state, which sets them aside so they can't be reprovisioned. In order to minimize the affect on CentOS 7 tests, we are temporarily disabling CentOS 6 in duffy until we can find a solution. Thank you all for your patience while we look into this. -- Brian Stinson brian(a)bstinson.com

2 1

Leveraging Zuul for job orchestration on Duffy nodes
by David Moreau Simard 10 Oct '19

10 Oct '19

Hi, While RDO does have some amount of jobs in the ci.centos.org Jenkins, the vast majority of our CI is driven by different instances of Zuul hosted by OpenStack [1] and Software Factory [2]. Jobs in Zuul are written as Ansible roles and playbooks and have the notion of inheritance so you can share jobs between projects and even between different instances of Zuul. This is convenient because we can easily re-use jobs that have been defined upstream and run them in different environments or with different parameters. For the time being, we'd like to know if it would be possible to experiment running Zuul jobs against powerpc test nodes provided by Duffy ? This would not be a full standalone Zuul deployment, it would be a single executor node that would connect to softwarefactory-project.io to fetch jobs and report results. In practice, we could probably recycle one of our existing Jenkins slave for this purpose. Could you let us know if this is possible ? Let me know if you have any questions, Thanks ! [1]: https://zuul.openstack.org/ [2]: https://softwarefactory-project.io/zuul/t/rdoproject.org/status David Moreau Simard dmsimard = [irc, github, twitter]

3 4

Fwd: Github is blocking account creation
by Tibor Dancs 03 Oct '19

03 Oct '19

---------- Forwarded message --------- From: Tibor Dancs <tdancs(a)redhat.com> Date: Tue, Oct 1, 2019 at 2:54 PM Subject: Github is blocking account creation To: osio-ci <osio-ci(a)redhat.com> Hello guys. I need to create a new SA for use with ci.centos.org jenkins jobs. Github, however, is flagging and blocking the account just as soon as I create them. They don't respond to support tickets. Do you have any ideas what could be done? Thanks, Tibor Dancs

2 2

Re: [Ci-users] duffy refuses to hand out nodes for devtools slave04 jobs - no inventory is available.
by Katerina Foniok 18 Sep '19

18 Sep '19

Hi, we have the same problem today. Seems to be happening from ~22 PM yesterday. Our jobs are running on slave04. On Wed, Sep 18, 2019 at 9:55 AM Katerina Foniok <kkanova(a)redhat.com> wrote: > Hi, we have the same problem today. Seems to be happening from ~22 PM > yesterday. Our jobs are running on slave04. > > On Mon, Sep 16, 2019 at 3:09 PM Vipul Siddharth <vipul(a)redhat.com> wrote: > >> Hi, >> thank you for informing us. >> This is taken care of and things should be back in a couple of minutes. >> >> On Mon, Sep 16, 2019 at 6:35 PM František Šumšal <frantisek(a)sumsal.cz> >> wrote: >> > >> > Hello, >> > >> > Can confirm, the same thing is happening to our jobs >> (systemd-ci-slave01). >> > Duffy has been returning 'Insufficient Nodes in READY State' for the >> last >> > ~two hours. >> > >> > On 9/16/19 2:57 PM, Jean-Francois Chevrette wrote: >> > > Hi, >> > > >> > > we've noticed that duffy refuses to hand out nodes for devtools >> slave04 jobs. >> > > >> > > This has been happening since at least ~12:00 UTC >> > > >> > > Thanks in advance >> > > >> > > -- >> > > Regards, >> > > Jean-Francois Chevrette >> > > Principal Software Engineer | Service Delivery - Application SRE >> > > jfchevrette = [irc, github, twitter] >> > > >> > > _______________________________________________ >> > > Ci-users mailing list >> > > Ci-users(a)centos.org >> > > https://lists.centos.org/mailman/listinfo/ci-users >> > > >> > >> > >> > -- >> > Frantisek Sumsal >> > GPG key ID: 0xFB738CE27B634E4B >> > >> > _______________________________________________ >> > Ci-users mailing list >> > Ci-users(a)centos.org >> > https://lists.centos.org/mailman/listinfo/ci-users >> >> >> >> -- >> Vipul Siddharth >> He/His/Him >> Fedora | CentOS CI Infrastructure Team >> Red Hat >> w: vipul.dev >> >> _______________________________________________ >> Ci-users mailing list >> Ci-users(a)centos.org >> https://lists.centos.org/mailman/listinfo/ci-users >> >

1 0

← Newer
1
...
9
10
11
12
13
14
15
...
42
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

CI-users