CI-users May 2020

ci-users@lists.centos.org

3 participants
2 discussions

Update on OpenShift cluster upgrade
by Vipul Siddharth 19 May '20

19 May '20

Hi folks, Here is a quick and short update on what CI Infrastructure team has been up to for the last few days. # Done * identified infrastructure suitable for hosting the new ocp4 cluster and curated inventory * configured DNS for the new ocp4 ocp.ci.centos.org cluster * provisioned 2 haproxy load balancers * ansible template for PXE ocp4 nodes # Hiccups we had * Faced issue pulling machine config operator image from quay.io on the ocp4 bootstrap node. We reached out to Jonathan Lebon of Fedora CoreOS related help (Thank you Jonathan for the help). The issue was resolved by recreating the ignition files from scratch. * Faced issue with multiple ethernet interfaces on the RHCOS nodes and DHCP. Resolved by disabling DHCP, and configuring static IPs on a single interface. # In progress * Provisioning OCP4 nodes (Master, Worker, and Bootstrap) # Blockers * RHEL CoreOS does not support RAID on the root device Thank you and Stay Safe / Brian Stinson, David Kirwan, Fabian Arrotin & -- Vipul Siddharth Fedora | CentOS CI Infrastructure Team

2 3

CI state and efforts
by Vipul Siddharth 01 May '20

01 May '20

Greetings everyone This email is to update you all on some of the efforts we are about to put/putting in place within the CentOS CI Infrastructure. Some of the things have been long due and we realize that. For a couple of years, our efforts have mainly been under keeping the fires away and general maintenance (adding new releases, arches, making things in the background a bit sophisticated, etc). Right now we have started working on some things that you would notice and will impact you (in a better way hopefully!) and we will be keeping you updated. * Monitoring - Done * A couple of months ago, All I had was basic ansible scripts to show me data, and that was my way of monitoring things. As one would expect, I often missed things, and email on the list would pop up with `insufficient nodes` or `no space on Jenkins master`. While we tried to be relatively quick on the resolution, we needed a better way. In early February, Fabian (Thank you) set up a monitoring system (added ci.centos.org to CentOS Infra Zabbix instance) which in turn reduced such incidence by a lot. I hope you have not noticed those emails/complaints in the last 2-3 months. We will be adding monitoring to more areas as we go. * OpenShift cluster upgrade - In Progress * https://console.apps.ci.centos.org:8443/ is using openshift version 3.6 which has been outdated for a while. We are upgrading the cluster to OpenShift 4.3 which handles some of the group permissions in a better way. Apart from the shiny new deployment which has a bunch of improvements/features, it will allow us to give admin access to more members easily which in turn will decrease the wait time for new accounts/users (it won't wait on a single person to do all the work). * Deployment of OpenNebula for VM checkouts through cico - Upcoming * We will be deploying OpenNebula and will extend duffy api to support VMs in the check out process. This will be useful for the jobs who don't need bare metals to do their tests and need slightly less resources. This will also enable us to support Fedora as an on-demand offering distro (in vm) that can be used to run your test. Ideally, it would look like `cico node get --type vm` or `cico node get --type vm --distro fedora -- version rawhide`. This needs more grooming and planning before we start planning exact images/distros and ways but I am hopeful that it will make your experience better. We are hoping to update you all with the things going on in the background pretty frequently and the updates will show up as weekly emails `progress of $initiatives_mentioned_above` and "probably" monthly email of some matrices of things that happened in that month (not sure yet). Once we have upgraded the openshift cluster, we will need your help in migrating the jobs from legacy `ci.centos.org` to openshift. You will get a namespace where you can control all things related. This will introduce easier workflow and management for admins and more freedom to you, tenants! If you want to discuss things more, feel free to contact us or catch us at #centos-ci@Freenode Thank you! Be safe -- @siddharthvipul He/His/Him Fedora | CentOS CI Infrastructure Team Red Hat

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

CI-users May 2020