CI-users September 2020

ci-users@lists.centos.org

4 participants
2 discussions

Infra : scheduled hardware maintenance (Openshift/NFS)
by Fabian Arrotin 01 Oct '20

01 Oct '20

Due to a hardware maintenance that needs to take place on the NFS storage node used by openshift ("legacy" and current one - ocp ), we'll have to shutdown the openshift cluster, and then proceed with hardware maintenance on the NFS server (that itself needs to be powered down, no way to actually do that "online") Migration is scheduled for """"Wednesday September 30th, 12:00 pm UTC time"""". You can convert to local time with $(date -d '2020-09-30 12:00 UTC') The expected "downtime" is estimated to ~60 minutes , time needed to shutdown the machine, install new disks, restart the machine and also do some updates and tuning on the setup. For more informations about this, here are some relevant tickets that were created for the perf issue in openshift and nfs : https://pagure.io/centos-infra/issue/53 https://pagure.io/centos-infra/issue/105 https://pagure.io/centos-infra/issue/85 https://pagure.io/centos-infra/issue/26 <subliminal message> PS : worth noting that while we'll investigate reports on new ocp cluster, we'll probably not spend time investigating in the old/legacy one, that projects are supposed to migrate away from soon, as the legacy openshift setup will disappear soon (see https://pagure.io/centos-infra/issue/16) </subliminal message> Thanks for your comprehending and patience. on behalf of the CI Infra team, -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab

3 4

Infra Pre-Announce : moving CI ssh jump host soon, please read !
by Fabian Arrotin 07 Sep '20

07 Sep '20

Hi, As you noticed recently, we started to refresh the infra used for CentOS CI (not the hardware, still the same, but the software stack and the way to control/manage it). One of the identified nodes still being used and that needs to be converted to the new infra layout is the ssh jumphost (see https://wiki.centos.org/QaWiki/CI/GettingStarted#How_to_use_it) Normally, some of you have switched to OpenShift workload, (including to the new Openshift 4/OCP setup that went live recently) but some Projects are still on the old setup with sometimes a need to reach dedicated/shared VMs acting as Jenkins agent[s], connected to Jenkins behind https://ci.centos.org. We have already provisioned a new VM in the new setup (that can reach the whole CI subnet and VLAN) but here are some points to consider, reason why we wanted to pre-announce long time in advance before we do the real switch) : * New ssh jump host is CentOS 8 based, versus CentOS 6, meaning that if you used ssh-dss key (instead of ssh-rsa), you'll *not* be able to connect through that new host. We already identified such keys and Vipul will try (when it's tied to a real email address for the project) to reach out. But better to announce it here too, so that you have time to ask us to reflect a change (through ticket on https://pagure.io/centos-infra/issues) * Old VM allowed shell access, but it will be disallowed on the new one (there is no need for shell on that intermediate node anyway). Reminder that you can configure your ssh config to directly use ProxyCommand or even now ProxyJump (on recent openssh-client). See https://wiki.centos.org/TipsAndTricks/SshTips/JumpHost) * Because the host has a new sshd_host_key, it will come with a new fingerprint too, so if you have automation and that you don't trust our CA already, the fingerprint for new host will be : [fingerprint] rsa=3072 SHA256:n7y0qZS/FvhjaskOBds3TTKQh5EtgNQ25E7cmTNBATg (RSA) rsa_md5=3072 MD5:9e:83:46:d0:c5:8a:a0:94:50:10:58:9d:af:ca:50:19 (RSA) ecdsa=256 SHA256:ZQacwDsWkKBYL9HJJYwHr94Ny1sMhHMDnz9GiLFb8Uc (ECDSA) ecdsa_md5=256 MD5:dd:24:ea:6a:fd:8b:29:3d:1d:d0:a9:32:8c:b2:ea:62 (ECDSA) As we know that it's August and that some of you are probably on PTO (coming back or leaving soon), after discussion with Vipul , David and myself, we considered that we'll probably go live around beginning of September. Should you have any question around that migration, feel free to reply to this thread (ideally on dedicated ci-users mailing list), or on irc.freenode.net (#centos-ci) On behalf of the CentOS CI infra team, -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab

2 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

CI-users September 2020