CI-users

ci-users@lists.centos.org

1 participants
421 discussions

How to abort running CI jobs
by Vinzenz Feenstra 19 Jun '17

19 Jun '17

Hi, Our team uses the CentOS CI infrastructure together within GitHub PR to check those to be in shape. At times it happens that due to programming mistakes we would like to cancel/abort jobs. As those runs take usually up to 45 minutes or even longer or sometimes are just stuck because they are waiting for user input that ofc never is going to happen. Is there some way to stop them? In this particular case I am talking about the leapp-to project @ github ( https://github.com/leapp-to/prototype ) Thanks -- Vinzenz Feenstra Senior Software Developer Red Hat Czech

5 8

Helping with CentOS CI
by Karanbir Singh 16 Jun '17

16 Jun '17

Hi Guys, Ari and some of the other people in his team have offered to help with running and maintenance around CentOS Ci, which would be very welcome. In order to get the ball rolling, Ari can you and the guys maybe go ahead and post some details on yourself, the work you are looking to do and what sort of things you had in mind w.r.t helping on the CentOS CI bits ? regards, -- Karanbir Singh, Project Lead, The CentOS Project +44-207-0999389 | http://www.centos.org/ | twitter.com/CentOS GnuPG Key : http://www.karan.org/publickey.asc

3 3

safe restart coming
by Karanbir Singh 14 Jun '17

14 Jun '17

hi Ari has disabled the jobs that were causing the problems for now. I've setup the jenkins instance to saferestart and judging by the openstack jobs running, this is going to happen in about 2 hrs time from now. that translates to : 02:50 14th Jun 2017 UTC 23:50 14th Jun 2017 UTC If the system load goes up again, I will disable the JMS plugins and do another restart, but hopefully there wont be a need for it. Regards -- Karanbir Singh, Project Lead, The CentOS Project +44-207-0999389 | http://www.centos.org/ | twitter.com/CentOS GnuPG Key : http://www.karan.org/publickey.asc

1 0

Unexpected outage 17:00 UTC Today - Service Restored
by Brian Stinson 14 Jun '17

14 Jun '17

Hi Folks, Jenkins was leaking file descriptors and hit a limit today at 17:00 UTC, service was degraded for about 10 minutes, and service was fully restored at around 17:24. I've increased the open-files limit for jenkins and am working on tuning the garbage collector to mitigate this in the future. Thanks for your patience, and apologies for any inconvenience. -- Brian Stinson

5 13

Devicemapper error with Docker
by Ratnadeep Debnath 12 Jun '17

12 Jun '17

Hi all, We have been experiencing some devicemapper issues with Docker since the last few weeks on CentOS CI and CentOS in general. This issue is reported here: https://github.com/moby/moby/issues/23089, however the workaround does not work. Did anyone face similar issue? Does anyone know how to workaround it. Debug info ======== [root@nxx ~]# docker --version Docker version 1.12.6, build 1398f24/1.12.6 [root@nxx ~]# systemctl status docker -l ● docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Mon 2017-06-12 08:14:41 BST; 1min 22s ago Docs: http://docs.docker.com Process: 300 ExecStart=/usr/bin/dockerd-current --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current --default-runtime=docker-runc --exec-opt native.cgroupdriver=systemd --userland-proxy-path=/usr/libexec/docker/docker-proxy-current $OPTIONS $DOCKER_STORAGE_OPTIONS $DOCKER_NETWORK_OPTIONS $ADD_REGISTRY $BLOCK_REGISTRY $INSECURE_REGISTRY (code=exited, status=1/FAILURE) Main PID: 300 (code=exited, status=1/FAILURE) Jun 12 08:14:40 n41.pufty.ci.centos.org systemd[1]: Starting Docker Application Container Engine... Jun 12 08:14:40 n41.pufty.ci.centos.org dockerd-current[300]: time="2017-06-12T08:14:40.732651146+01:00" level=warning msg="[!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING [!]" Jun 12 08:14:40 n41.pufty.ci.centos.org dockerd-current[300]: time="2017-06-12T08:14:40.742206601+01:00" level=info msg="libcontainerd: new containerd process, pid: 308" Jun 12 08:14:41 n41.pufty.ci.centos.org dockerd-current[300]: time="2017-06-12T08:14:41.758615859+01:00" level=warning msg="devmapper: Usage of loopback devices is strongly discouraged for production use. Please use `--storage-opt dm.thinpooldev` or use `man docker` to refer to dm.thinpooldev section." Jun 12 08:14:41 n41.pufty.ci.centos.org dockerd-current[300]: time="2017-06-12T08:14:41.759174439+01:00" level=error msg="[graphdriver] prior storage driver \"devicemapper\" failed: devmapper: Base Device UUID and Filesystem verification failed: devicemapper: Can't set cookie dm_task_set_cookie failed" Jun 12 08:14:41 n41.pufty.ci.centos.org dockerd-current[300]: time="2017-06-12T08:14:41.759576207+01:00" level=fatal msg="Error starting daemon: error initializing graphdriver: devmapper: Base Device UUID and Filesystem verification failed: devicemapper: Can't set cookie dm_task_set_cookie failed" Jun 12 08:14:41 n41.pufty.ci.centos.org systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE Jun 12 08:14:41 n41.pufty.ci.centos.org systemd[1]: Failed to start Docker Application Container Engine. Jun 12 08:14:41 n41.pufty.ci.centos.org systemd[1]: Unit docker.service entered failed state. Jun 12 08:14:41 n41.pufty.ci.centos.org systemd[1]: docker.service failed. Thanks, rtnpro -- Ratnadeep Debnath, https://www.waartaa.com GPG Fingerprint: 033C 8041 A0E9 CDBA 2E02 B785 2119 5486 F245 DFD6

1 0

Jenkins Gerrit Trigger problem, please investigate
by Attila Darazs 09 Jun '17

09 Jun '17

Dear CentOS CI admins, We realized that some of our jobs are not triggering properly when we submit changes to review.openstack.org. None of our jobs that is using this gerrit server triggers any more. We did not change anything in the jobs configs recently, the jobs are set up to use the "rdo-ci-openstack.org" gerrit server and trigger on any change. What's weird is that if we go to https://ci.centos.org/gerrit_manual_trigger/ then we can manually trigger the jobs that were supposed to be running automatically. Example test change: https://review.openstack.org/472267 Submitted: 2017-06-08 13:44 UTC One of the gates jobs that were supposed to trigger, but nothing happened: https://ci.centos.org/view/rdo/view/tripleo-gate/job/tripleo-quickstart-gat… Manually triggered at (with rdo-ci-openstack.org) 2017-06-08 13:58 UTC Triggered Builds 472267,1 tripleo-quickstart-gate-ocata-delorean-quick-basic #530 tripleo-quickstart-gate-master-delorean-quick-basic #2290 tripleo-quickstart-gate-newton-delorean-quick-basic #1453 As far as I can tell this was the last of our jobs that was still properly triggered by gerrit, 11 days ago: https://ci.centos.org/job/tripleo-quickstart-extras-gate-newton-delorean-fu… Please look into this, it seems to me that somehow the connection that listens to Gerrit events is down and it's important for us to have these gates running. Thanks, Attila

1 1

Unplanned Saferestart 14:35 UTC
by Brian Stinson 08 Jun '17

08 Jun '17

Hi Folks, To handle the Gerrit Triggering issue, I have issued a saferestart starting at 14:35 today. I expect this to finish up in the next hour or so. In the meantime jobs will build up in the queue and be executed shortly after jenkins returns. Thanks for your patience. -- Brian Stinson

2 2

Ongoing delayed triggers, and Github PR updates in CICO
by Brian Stinson 08 Jun '17

08 Jun '17

Hi Folks, Over the past few days, we've heard reports of delayed Github job triggers and PR status updates. Quite a few jobs are polling PRs instead of relying on the webhooks such that we are exhausting our Github API rate limit. We'll be reaching out to some of you individually if your jobs are affecting this, but I thought I'd give a single update on the situation here. If you have questions, please let us know. Cheers! -- Brian Stinson

1 0

Weird issue in CentOS CI GUI and jobs are taking longer time
by Budh Gurung 07 Jun '17

07 Jun '17

Hi Folks, Thanks for the wonderful CI platform. Recently, we are noticing that few of our jobs [1] are taking way longer than the usual time. Also, it is not notifying back to the PR which triggered those jobs. I noticed that on the list of executors view [2], one/many of the jobs are in the state of progress (see screenshot 1) and on clicking on the progress bar to go to the respective job [3](screenshot 2), that job seems to have completed quite a long back. We are also experiencing jobs taking a longer time in recent time. Anyone experiencing same? Any idea what is happening here? Regards, Budh Ram Gurung Software Engineer - Devtools

1 1

irregular execution of NM jobs
by Vladimir Benes 02 Jun '17

02 Jun '17

Hi all, not sure if it's expected but I see some irregularity in my jobs execution https://ci.centos.org/view/NetworkManager/job/NetworkManager-nm-1-4/ and others since job 55 it was executed every day at the same time but since then (May 25th) it was executed just two times. Any hints? CI itself doesn't seem to be super busy. thanks, Vladimir

2 1

← Newer
1
...
23
24
25
26
27
28
29
...
43
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

CI-users