Due to a mandatory core switches upgrade in the DC where some services
are hosted, we'll lose connectivity between some services/storage
servers and some publicly available services will be impacted.
Impacted services:
- https://git.centos.org
- https://cbs.centos.org (builders will be put on hold during
maintenance window)
- https://lists.centos.org - CentOS mailing lists
- CentOS CI env (https://duffy.ci.centos.org api endpoint)
Maintenance is scheduled for """"Tuesday September 22nd, 6:00 am UTC
time"""".
You can convert to local time with $(date -d '2020-09-22 06:00 UTC')
The expected/communicated "downtime" was estimated to be 4h, but we'll
just announce when all services will be available again.
Thanks for your understanding and patience.
on behalf of the Infra team,
--
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]
Hi CI tenants !
Just a reminder that (as you know it) CentOS Stream 8 will go EOL in May
and then CentOS Linux 7 itself going EOL in June (so very soon)
That would mean that we'll just remove the various pools that provide
CentOS 8s and 7 for your tests.
Reason is that mirror network itself will be unavailable and so all
tests would fail anyway.
Happy to discuss here but we'll send reminders about when we'll just
remove the various pools in Duffy.
I also need to find an option for something else that you *still* all
use when using Jenkins on openshift cluster : our cico-workspace image
(https://quay.io/repository/centosci/cico-workspace)
That one is "problematic" as it's based on upstream openshift jenkins
agent
(https://github.com/openshift/jenkins/tree/master?tab=readme-ov-file#hosted-…)
These ones are based on UBI8 and it seems that there is (still) no
momentum to move to UBI9 (don't have direct contact with them though)
While our current images were based from the upstream ones, we could
just "convert" these to 8s
(https://github.com/centosci/images/blob/master/cico-workspace/Dockerfile#L1
and
https://github.com/centosci/images/blob/master/cico-workspace/Dockerfile#L8)
We need to find a solution but was wondering if just using vault
(archived mirror content) would be a good solution or not : tempted
myself to answer "no" as that would mean running container that will
*never* be updated for security reasons (CentOS Stream 8 going EOL in May)
Opinions, thoughts ?
--
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]
Recently , after the openshift cluster upgrade to ocp 4.15, some tenants
asked about lack of update for Jenkins (stuck on version 2.401.1 , EOL
LTS version)
Long story short (people can read
https://pagure.io/centos-infra/issue/1409) :
- Openshift (during releases) modified the default jenkins
imagestreamtag
(https://access.redhat.com/documentation/en-us/openshift_container_platform/…)
- deployed jenkins pods were using DeploymentConfig using jenkins:2
imagestreamtag (not up2date)
What is required from you , CI tenants :
you can just edit (for now) your deploymentconfig like this, and that
will automatically trigger a pod redeploy, using latest jenkins lts
version from registry ( v2.440.3 when writing this) :
```
oc edit dc/jenkins
replace "name: jenkins:2" with "name: jenkins:ocp-upgrade-redeploy" and exit
```
Watch for your logs and then update your plugins
Worth knowing that we'll enforce this rule on June 17th, by making (ocp
cluster wise in openshift namespace) jenkins:2 an alias to
jenkins:ocp-upgrade-redeploy, so redeploying *all* jenkins pods that
aren't migrated yet
Summary :
So if you prefer to do that yourself and watch process, and pause your
ci jobs, etc , do it now
If you don't do it, it will be done automatically (but in unattended
way) on June 17th
--
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]
As announced some time ago, CentOS Stream 8 reached EOL (End Of Life)
and so will be archived today.
For that reason, CentOS Stream 8 duffy nodes (see
https://sigs.centos.org/guide/ci/#duffy-ephemeral-bare-metalvirtual-machine…)
were removed from configuration and pool cleared out
Kind Regards,
--
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]
Please be advised that the CentOS CI Production Openshift cluster is due
for scheduled maintenance from 07:00 to 08:00 UTC on 2024-05-07 . During
this period necessary updates will be applied to the cluster.
The upgrade is not expected to be service affecting, however there might
be periods of slight performance degradation (pods restarting on other
workers) during this maintenance period. We apologize for any
inconvenience that this may cause.
PS : this is a follow-up to last week update, and this time the upgrade
will bring the cluster from 4.14.x to 4.15.x (latest stable released
version)
Kind regards,
CentOS CI Infrastructure.
--
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]
Please be advised that the CentOS CI Production Openshift cluster is due
for scheduled maintenance from 07:00 to 08:00 UTC on 2024-05-02 . During
this period necessary updates will be applied to the cluster.
The upgrade is not expected to be service affecting, however there might
be periods of slight performance degradation (pods restarting on other
workers) during this maintenance period. We apologize for any
inconvenience that this may cause.
Kind regards,
CentOS CI Infrastructure.
--
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]
Due to a needed upgrade , we'll have to move the existing CentOS
mailman instance (aka https://lists.centos.org) to a new server/host.
Migration is scheduled for """"Tuesday April 8th, 7:00 am UTC time"""".
You can convert to local time with $(date -d '2024-04-08 07:00 UTC')
The expected "downtime" is estimated to ~60 minutes , time needed to :
- take last mailman2 backup
- reimport / convert mailman2 archives to mailman3 DB
- DNS propagation for A/AAAA/MX records
Here are also some important information about the mailman2 => mailman3
migration :
# Renamed lists
Worth knowing that, based on open discussion on the centos-devel list
(see whole thread at
https://lists.centos.org/pipermail/centos-devel/2024-March/165576.html),
existing lists will be *renamed* , so while we'll put aliases for
incoming mails, each list member will start receiving list mails from
new list name. So start updating your filters if you filter on email
address instead of "subject:"
Here is the overview of the new lists names :
arm-dev at centos.org => arm-dev at lists.centos.org
centos at centos.org => discuss at lists.centos.org
centos-devel at centos.org => devel at lists.centos.org
centos-announce at centos.org => announce at lists.centos.org
centos-automotive-sig at centos.org => automotive-sig at lists.centos.org
centos-{cz,de,es,fr,nl,pt-br,zh}(a)centos.org =>
discuss-{cz,de,es,fr,nl,pt-br,zh}(a)lists.centos.org
ci-users at centos.org => ci-users at lists.centos.org
centos-gsoc: => gsoc at lists.centos.org
centos-mirror at centos.org => mirror at lists.centos.org
centos-mirror-announce at centos.org => mirror-announce at lists.centos.org
centos-newsletter at centos.org => newsletter at lists.centos.org
centos-promo at centos.org => promo at lists.centos.org
centos-virt at centos.org => virt at lists.centos.org
# Authentication
Mailman2 had no real concept of authentication so you could just
subscribe to one or more lists, and have a password associated with your
email address for that/these subscription(s).
Mailman3 itself is split into "core" and "webui" components, so when
we'll import mailman2 lists/config into mailman3, your existing
subscriptions will continue to work *but* not your password.
Mailman3 will be configured to support SSO, and so if you already have a
FAS/ACO account (https://accounts.centos.org) you'll be able to login
directly into new webui and manage your settings/subscriptions *if* your
ACO email address of course matches the one you initially subscribed
with for lists.centos.org.
If that's not the case, either create an ACO/FAS account that will match
and you'll be then able to "link" your mailman3 account with FAS and so
manage your settings/subscriptions.
If you don't want to, there is always the documented process :
https://docs.mailman3.org/en/latest/userguide.html#making-a-mailman-account
Thanks for your understanding and patience.
on behalf of the Infra team,
--
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]
Dear CentOS CI User,
Please be advised that the CentOS CI Production Openshift cluster is due
for scheduled maintenance from 08:00 to 10:00 UTC on 2023-12-11 . During
this period necessary updates will be applied to the cluster.
The upgrade is not expected to be service affecting, however there might
be periods of slight performance degradation during this maintenance
period. We apologize for any inconvenience that this may cause.
Kind regards,
CentOS CI Infrastructure.
--
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]
Due to a mandatory network maintenance at the RDU2 (Community Cage)
facility/DC, we'll have to power off some CentOS infrastructure
components hosted in that DC.
Impacted services:
- https://cbs.centos.org (impossible to submit any build, nor push to
delivery network)
- ns1.centos.org
- all centos linux / stream 8 / stream 9 release engineering workflow
(stream 8/9 can be built but not pushed out, while for the rest, all
will be down)
- https://git.centos.org
- CI infra:
- while openshift in aws is self-contained, there will be no way to
request duffy nodes for tests
- various other services
Migration is scheduled for """"Friday November 17th, 1:00 pm UTC time"""".
You can convert to local time with $(date -d '2023-11-17 13:00 UTC')
The scheduled maintenance window (communicated to us) is estimated to
3h, but we'll restart infra services as soon as possible, following
ourselves the current status internally.
Thanks for your understanding and patience.
on behalf of the Infra team,
--
--
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]
Dear CentOS CI User,
Please be advised that the CentOS CI Production Openshift cluster is due
for scheduled maintenance from 08:00 to 10:00 UTC on 2023-11-13 . During
this period necessary updates will be applied to the cluster.
The upgrade is not expected to be service affecting, however there might
be periods of slight performance degradation during this maintenance
period. We apologize for any inconvenience that this may cause.
Kind regards,
CentOS CI Infrastructure.
--
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 17F3B7A1 | @arrfab[@fosstodon.org]