CI-users October 2016

ci-users@lists.centos.org

17 participants
16 discussions

Infinite loop with multiple repos?
by Laurentiu Pancescu 13 Oct '16

13 Oct '16

Hello, while adapting the cloudinstance-vagrant-manual-build job on ci.c.o, I thought to skip the manual cloning of a second repo via shell scripts and add a second repo via the Jenkins UI (I expected it would be cloned into a sibling directory to the directory hosting the clone for the first repo, but it seems to have just overwritten it). When I ran the job, it not only failed, but retried the job immediately, causing some sort of infinite loop. Cancelling the current job was useless, since a new one started immediately, and I ended with more than 250 failed jobs in just 1-2 minutes; I had to disable the project to stop it. I only had a trigger to start cloudinstance-vagrant-test on success - no idea why it retried again and again... Is that expected behavior? I switched to manual git cloning, this works without issues. Laurentiu

1 0

ci.centos.org is now running the latest Jenkins LTS (Jenkins 2)
by Brian Stinson 12 Oct '16

12 Oct '16

Hi Folks, During the migration this week we went ahead and migrated our Jenkins instance to the latest LTS (which means Jenkins 2!). The core functionality is completely backwards compatible with Jenkins 1, and we kept all of our plugins pinned to the versions we had running before. Just a reminder for the only known effect: - If you're using JJB to configure some of your jobs you _may_ need to set 'query_plugins_info = False' in the [jenkins] section of your config file. If you notice any trouble, please let us know here. Cheers! -- Brian Stinson CentOS CI Infrastructure Team

2 1

[infra] : Hardware migration status (full/correct version)
by Fabian Arrotin 11 Oct '16

11 Oct '16

As announced some time ago ( https://lists.centos.org/pipermail/centos-announce/2016-September/022065.ht… ) , there was a big hardware migration/DC relocation happening yesterday that impacted several of our public services. Here is a small report of that migration, that finally took less time that previously announced. * 2016-10-10 - 1pm UTC : we started to preconfigure all changes, and powered down all the services in correct order * 2016-10-10 - 2pm => 6pm UTC : hardware was moved/migrated * 2016-10-10 - 6pm => 8pm UTC : hardware was (in serial order) plugged back in and ipmi/oob interfaces reconfigured, and network started to be validated * 2016-10-10 - 8:30pm UTC : https://accounts.centos.org was back online and operational * 2016-10-10 - 9pm UTC : https://cbs.centos.org was back online but builders were still disabled for validation * 2016-10-10 - 9:45pm UTC : https://cbs.centos.org : builders were enabled and working again * 2016-10-11 - 3:30am UTC : https://ci.centos.org now fully operational (after some network reconfiguration needed post-migration, and validations Thanks a lot to Brian Stinson (on-site) and DC people who helped for that migration (Matt, Jon, Nathan, Nina, Phil) Should you encounter an issue , feel free to discuss with us in #centos-devel on irc.freenode.net PS : sorry for the previous and incomplete mail : sent by accident before finished and probably due to lack of coffee and short night ... -- Fabian Arrotin The CentOS Project | http://www.centos.org gpg key: 56BEC54E | twitter: @arrfab

3 2

[infra] : Hardware migration status
by Fabian Arrotin 11 Oct '16

11 Oct '16

1 0

OpenStack cluster in ci.centos.org
by Karanbir Singh 06 Oct '16

06 Oct '16

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Folks, If anyone is using the openstack infra in ci.centos.org - please note that when the ci hardware is relocated this monday morning on the 10th Oct, the openstack cluster will NOT be coming back online. Our plan is to reinstall the entire openstack hypervisors and redeploy with neuton the following day and bringup an entirely clear cluster. If anyone has content there that they need to preserve, please back that up and shutdown the VMs before Monday morning UTC 10th October. Regards, - -- Karanbir Singh, Project Lead, The CentOS Project +44-207-0999389 | http://www.centos.org/ | twitter.com/CentOS GnuPG Key : http://www.karan.org/publickey.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJX9lvpAAoJEI3Oi2Mx7xbtXosIAKXfGzj81JhKe6xbqOxp1vjE bTb3/2BK0EJxLzD2QGPuXmFMzgZf5him+cxJUu/jjh5pGoPSgto4rEu1UzBLhapy RzQvfCn9e877hhwUaPmBrJyaNlsfF8mIKKtoXCqwYKXRGvka3cJbXTp0WbsTjEmi +JEOWKt3cYcDio7Wj+cJMG8mqEXmgDb1pUxiiQPnzrx0BqbG/DUg0O42Oj+AMqPX pI4HI+jUZRtMooYLd9PoGdZsAZf8Cxy/dXFPJz1zy9C+iZcDFCt0vY8OvxDUuEfA N41dgEpxapYN81l9BSz9fXOW8kc+AzUIxkFI5BearcrrFVkSkt77QGRWzr3NnwA= =Z5Qz -----END PGP SIGNATURE-----

1 0

[Infra] - Major outage for several services (please read)
by Fabian Arrotin 05 Oct '16

05 Oct '16

As pre-announced earlier this year (https://lists.centos.org/pipermail/centos-devel/2016-May/014792.html) we'll have to move a part of our existing hardware to a new DC. That means that the following public services will be powered off and unreachable : - https://cbs.centos.org (Koji build farm front end and also builders/storage nodes behind) - https://accounts.centos.org (auth backend) - https://ci.centos.org (jenkins-driven CI environment and all nodes in that dedicated/isolated environment) What does that mean ? - Nobody from the SIGs (https://wiki.centos.org/SpecialInterestGroup) will be able to submit build jobs/tasks (so no update at all during that migration) - No CI job will be able to track CBS nor external repositories (and so that means also that http://artifacts.ci.centos.org will be unreachable) - Not possible for existing users to update their details on accounts.centos.org, and not possible to create new accounts either We're working on a plan to minimize the downtime/reconfiguration part, but at first sight, due to the hardware move of the racks/cabling parts/etc, the announced downtime will be probably ~48h. Migration is scheduled for """"Monday October 10th, 1:00 pm UTC time"""". You can convert to local time with $(date -d '2016-10-10 13:00 UTC') We'll start to restore services as soon as possible, while working in coordination with the DC people closely (and after validation about network ports, connection) During this period you can come talk to us in #centos-devel and we will be posting updates there. on behalf of the Infra team, -- Fabian Arrotin The CentOS Project | http://www.centos.org gpg key: 56BEC54E | twitter: @arrfab

2 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

CI-users October 2016