CI-users May 2016

ci-users@lists.centos.org

14 participants
11 discussions

CI Maintenance Window 27-May-2016 01h00 - 02h00 UTC (26-May-2016 21h00 - 22h00 EDT)
by Brian Stinson 24 Oct '16

24 Oct '16

Hi Folks, We will be having a maintenance window starting at 1AM UTC on Friday, May 27th to do the following work: - Upgrade to the latest Jenkins LTS - Upgrade all plugins (including the Github Plugin) to their latest versions - Install the Pipeline Plugin (bug: 10825) - Update the ansible version on slave01.ci.centos.org 1.9.2 -> 1.9.6 - Reboot the storage node The following services will be affected: - ci.centos.org: Jenkins Frontend - artifacts.ci.centos.org: File availability As usual we will have a quiet period starting 1 hour before in order to let pending jobs clear out. If there are any questions please let us know. Cheers! -- Brian Stinson CentOS CI Infrastructure Team

5 10

Networking query (Vagrant + Duffy + Jenkins)
by Dharmit Shah 27 May '16

27 May '16

Hi all, I'm trying to get access to OpenShift service running inside the Vagrant box on one of the systems provided by Duffy. I'm attempting to access it from Jenkins slave. I tried to forward the port 8443 inside the Vagrant box to port 8443 on Duffy provided system. And then when I try to access it from Jenkins slave, it fails with error: $ ./oc login <duffy-provided-slave> Unable to connect to the server: dial tcp <ip-address>:8443: getsockopt: no route to host A rough mapping of the systems involved should look like this: Jenkins Slave --> Duffy provisioned host (CentOS 7) --> Vagrant box running OpenShift (CentOS 7) I want to check with the group if someone's ever tried something like this and succeeded. Would love to hear some ideas that we should be trying. Regards, Dharmit.

1 0

Test results not sent to github PR
by Honza Horak 23 May '16

23 May '16

I'm fighting with github+jenkins integration; specifically I'd like to make jenkins to run a job [1] for every commit in PR [2] or after writing message '[test]' in this PR. Then I expect the test result to be set in github PR. One issue I see is that when trying to follow [3] and adding 'https://ci.centos.org/ghbrphook', I see 301 response in 'Recent Deliveries' section. Well, when adding [test] comment, the job is run, but the output is not sent into github. I'd appreciate any help here, since I'm quite new to jenkins.. [1] https://ci.centos.org/view/SCLo/job/SCLo-mariadb-container-pr/ [2] https://github.com/sclorg/mariadb-container/pull/1 [3] https://wiki.centos.org/QaWiki/CI/GithubIntegration Honza

2 3

Jenkins SafeRestart to add extra CSRF Protection 19-Apr-2016 14h30 UTC (09h30 EDT)
by Brian Stinson 21 May '16

21 May '16

Hi Folks, In response to news of directed attacks against public Jenkins instances[0], we are enabling some of the CSRF protections in ci.centos.org To do this we will issue a SafeRestart at 14:30 UTC Today! Running jobs will be given a chance to clear and new jobs should be queued up and will execute as soon as the restart finishes. Potential Impact: - If you are using the Jenkins REST interface you may need to modify your scripts to send the appropriate headers[1] - Jenkins Job Builder is tracking an issue to enable CSRF support[2]. Some basic tests were performed on our side, and simple jobs were configured correctly, but you may notice strange behavior if you are using JJB. [0]: https://groups.google.com/d/topic/jenkinsci-advisories/lJfvDs5s6bk [1]: https://wiki.jenkins-ci.org/display/JENKINS/Remote+access+API#RemoteaccessA… [2]: https://storyboard.openstack.org/#!/story/2000556 If you have any questions or comments, let us know here or find one of us in #centos-devel on Freenode. Cheers! -- Brian Stinson CentOS CI Infrastructure Team

3 7

Important infra outage notification - dates to be discussed
by Fabian Arrotin 20 May '16

20 May '16

Due to some reorganization at the DC/Cage level, we'll have to shutdown/move/reconfigure a big part of our hosted infra for the following services : - cbs.centos.org (Koji) - accounts.centos.org (auth backend) - ci.centos.org (jenkins-driven CI environment) We're working on a plan to minimize the downtime/reconfiguration part, but at first sight, due to the hardware move of the racks/recabling parts/etc, the announced downtime will be probably ~48h. What does that mean ? That during this window, nobody will be able to build/tests packages, nor be able to triggers automatically CI jobs (important). As said, we're working on an agenda with the team operating the DC, but we'd like you (cbs and ci users) to give us feedback on the best (or worst ?) time line for such migration. For example if you know that your $project will have a release soon, and already have an agenda for such release (and so build/ci) and that you rely on that infra, we'd like you to communicate those informations to us, so that we can try to find the best possible time slot for the migration, minimizing the impact on the whole CentOS ecosystem (and so for all our users) Feel free to answer in this thread, or find us in #centos-devel on freenode. -- Fabian Arrotin The CentOS Project | http://www.centos.org gpg key: 56BEC54E | twitter: @arrfab

1 0

centos-ci-skeleton v2016.1
by Colin Walters 19 May '16

19 May '16

Hi, https://github.com/cgwalters/centos-ci-skeleton/releases/tag/v2016.1 is a repo I put together today that's factoring out some recent work I did on a new duffy wrapper script: https://github.com/cgwalters/centos-ci-skeleton/releases/tag/v2016.1 Combined with JJB templates: https://github.com/cgwalters/centos-ci-skeleton/blob/master/jjb-tmpl/cciske… My high level goal is to try to establish a bit more of a shared baseline codebase. It seems for example that most CentOS CI users are using Jenkins Job Builder. (If you're not, you should really consider it). If you look at my demo job: https://github.com/cgwalters/centos-ci-skeleton/tree/master/jjb-demo It shows how to pair together things so that you get a reusable node that's provisioned via an Ansible playbook.

2 2

Request for virtual IP-addresses in the CI for fail-over testing
by Niels de Vos 19 May '16

19 May '16

Hi, there is an integration in place with Gluster, NFS-Ganesha and Pacemaker. This combination makes it possible to have an active-active high-available NFS-server backed by Gluster volumes. We'd like to add automated testing for functional fail-over in the CI. This requires the use of virtual-IPs that get assigned to the different NFS-Ganesha servers, which will migrate to other servers upon failure. On https://wiki.centos.org/QaWiki/PubHardware is a mentioning of "reserved IP addresses" where the Gluster project in the CI would like to get listed too. What is the process to request a few IPs, and what are the restrictions we need to be aware of (and how to put them in the Jenkins job)? Thanks, Niels

2 1

duffy requests
by Colin Walters 18 May '16

18 May '16

1) Public source code 2) Arbitrary metadata (could be JSON or just a blob) like /Node/get/?key=blah&metadata=<base64 encoded json> Then a new /InventoryExt verb that returns JSON like: [ { "name": "n49.pufty", "ssid": "abcde", "metadata": "<base64 encoded json>" }, { "name": "a29.pufty", "ssid": "abcde", "metadata": "<base64 encoded json>" } ] 3) Also add a new option to /get/ named "longpoll" which means the machine is automatically deallocated when the requester's TCP connection closes. This would help ensure that e.g. cancelling the job or a Jenkins restart etc. automatically deallocated the machine

4 5

Request for https://wiki.jenkins-ci.org/display/JENKINS/PostBuildScript+Plugin
by Colin Walters 16 May '16

16 May '16

It's a handy way to implement cleanup actions, such as de-provisioning a duffy machine regardless of whether or not a job succeeds.

2 2

Strategy for scaling jenkins slaves ?
by David Moreau Simard 16 May '16

16 May '16

Hi, The RDO slave currently has 15 execution threads running on two cores and 4GB of RAM. As we use ci.centos more and more for gating and periodic jobs, I can see that the slave starts to struggle a little bit CPU-wise when the execution threads are busy. The bulk of the jobs are offloaded to an ephemeral bare metal node but ansible still runs from the slave. Some jobs do run locally, such as smaller tox jobs. What's the strategy if we want to scale this ? For example, do we want to scale in increments of "10 threads/2 cores/4 GB RAM" by adding more slaves as need be or do we want to grow the specifications of the existing slave and grow the amount of threads ? I have a personal preference to have more smaller slaves as this also has the added benefit of redundancy so we're not left entirely out in the cold if this one big slave happens to crash. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter]

5 7

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

CI-users May 2016