Hi everyone,

Welcome to the CPE team weekly project update mail!

Background:

The Community Platform Engineering group is the Red Hat team combining IT and release engineering from Fedora and CentOS. Our goal is to keep core servers and services running and maintained, build releases, and other strategic tasks that need more dedicated time than volunteers can give.

For better communication, we will be giving weekly reports to the CentOS and Fedora communities about the general tasks and work being done. Also for better communication between our groups we have created #redhat-cpe on Freenode IRC! Please feel free to catch us there, a mail has landed on both the CentOS and Fedora devel lists with context here.

Note:

This document is currently built from individual reports rolled into a document which we edit and copy into a final document. We are aware that this causes problems with some email readers, and are working on a method to make this less problematic.

High Level Project Updates:

CentOS

Wiki.centos.org migration (last monday)

Finishing templates for mailman ansible role (new look with community members)

Preparing the next migration (sponsor relocates DC and new machines):

www.centos.org (on track, to be announced)

Forums (scheduled for next monday)

Starting new ansible roles to automate deploy + upgrade of those migrated services

Working on koji (cbs.dev) to see when we can import 8.1 content to let SIGs build against/for .el8 and .el8s (Stream)

Collaborating with Artwork SIG for new design for http exposed services (new centos design)

QA for 8-Stream and 8.1.1911 continues

Fedora

Production issues with Bodhi:

Bodhi is under investigation for misbehaving. Creating update can take a long time or just fail. The status can be tracked here https://status.fedoraproject.org/

Rawhide Gating

Fixed: https://pagure.io/fedora-ci/general/issue/70

repoSpanner

This has been turned off on Fedora side except for src.stg.fedoraproject.org

The team are now working on whether to convert src.stg or redeploy

Pagure Updates

Pagure 5.8.1 has been released

Pagure 5.8.1 has been pushed to staging & production

Pagure-dist-git 1.5.1 and 1.5.2 has been released

Pagure-dist-git 1.5.1 has been pushed to staging

Pagure and 1.5.2 has been pushed to staging & production

Fedora Docs Updates

Merges:

https://pagure.io/fedora-docs/quick-docs/pull-request/166

https://pagure.io/fedora-docs/quick-docs/pull-request/161

https://pagure.io/fedora-docs/quick-docs/pull-request/159

Fedora docs was published to localization stats to stage

Community Fires Team

Made the side-tag workflow work in stable releases

Fixed the distgit-bugzilla-sync script

New pagure and pagure-dist-git have been released and deployed

Fix showing CI results on PR on dist-git

Added a new API endpoint to Pagure to install and remove git hook

Tiger Team Updates:

OSBS

Sprint focused on getting the aarch64 VMs in staging

Deployed Fedora 30 on the VMs and started deploying OpenShift

Currently blocked by the authentication to quay.io to get the aarch64 images.

Focus for the next sprint is to have a working OpenShift on the aarch64 hardware.

Bodhi CD

Ability to build a docker image in quay.io on a push to github branch

Ability to trigger a redeploy in openshift automatically after image has been rebuilt in quay

Upgrade the database schema automatically with openshift deployment, instead of manually running ansible playbooks

Application Retirements

Elections

Everything should be ready now to move Elections to Communishift

Planning will start soon for a move date!

Fedocal

No progress on kanban board last seven weeks - looks abandoned https://teams.fedoraproject.org/project/fedora-calendar/kanban

Jlanda’s permission error in communishift should be fixed now https://pagure.io/fedora-infrastructure/issue/8274

Another permission issue occurred during test

Nuancier

Benson Muite is now working on OIDC authentication

The team have recommended to create a PR on Github to be able to see the progress

PR from sebwoj - Porting to Fedora messaging

Sebwoj is now working on adding scheme as part of the PR

EPEL 8 Modularity

EPEL 8 Modularity work tested in stage and was deployed to production on Tuesday 10th December

EPEL 8 Playground Modularity work is on hold.

Blocker: https://pagure.io/fm-orchestrator/issue/1541

Misc Updates

Tests to jms-messaging plugin are now successfully running & waiting for review

Buildvmhost-01/02/03/04 has been reinstalled with rhel8

Buildvm-01 to 32 has been reinstalled

Autocloud has been decommissioned

ppc9-02 has been reinstalled with f31, and buildvm-ppc64le-01 to 09

ppc8-01 has been reinstalled with rhel8

F29 signed builds, updates, updates-testing, container, cloud composes in /mnt/koji/composes were cleaned up this week

Holiday Season Shutdown Notices:

Red Hat does a 2 week end of year shutdown where nearly everyone is encouraged to spend time away from computers and family. During that time, some operators will be monitoring tickets and services for large scale outages, but will not be active on IRC or email. The ability of the operators to fix problems is also limited because some operations require staff who will be away. People on duty will do what they can to repair and maintain services but do not give any ETA on how long it will take for it to be done.

If you are making changes to infrastructure during this time, please treat it like a ‘frozen’ period. Unless it is an emergency repair, do not push changes without a review from a main system administrator. We would encourage you to bump your ticket or request early in the New Year (full service resumes January 6th) for our attention, otherwise we will work through the backlog of requests.

We wish you and yours a happy holiday season and look forward to working with you in collaboration throughout 2020 and beyond.

Comments? Suggestions? Feedback? Let Us Know!

Kindest regards, Aoife