devel

devel@lists.centos.org

18 participants
4604 discussions

CentOS SIGs and lookaside cache
by Pierre-Yves Chibon 28 Feb '22

28 Feb '22

Good Morning Everyone, There are currently two lookaside caches in use around the CentOS project: * One used by CentOS-Stream: https://sources.stream.centos.org/sources it's not browsable, but it uses the structure: `baseurl/pkgname/tarball/hashtype/hash/tarball`. Example: https://sources.stream.centos.org/sources/rpms/kernel/linux-5.14.0-62.el9.t… * One used by CentOS-Linux, CentOS-Stream 8 and the SIGs: https://git.centos.org/sources/ this one is browsable and as you can see uses the structure: `baseurl/pkgname/branch/hash`. Example: https://git.centos.org/sources/kernel/c8s/0c4e10577cfd4b4f8e3d83c0406da8ab0… The rest of this email focuses on this second one. SIGs upload to it using the route: https://git.centos.org/sources/upload.cgi In an email last week [1] was proposed an idea for how SIGs could leverage the centos namespace in gitlab for those who wishes. One of the benefits of using gitlab would be increased flexibility for SIGs and a clear example for this would be the ability to drop the branch structures currently imposed on the git repositories. That structure is imposed because the git repositories are shared between CentOS-Linux, CentOS-Stream and (potentially multiple) SIGs, so that structure ensures groups are not stepping on each other's toes. By moving the SIGs out of these shared repositories, imposing that structure is no longer needed. However, since the lookaside cache relies on branch name, lifting that structure would break the lookaside cache. I have already brought this idea to a few folks to see if the idea was sane. The consensus that emerged is: * Introduce a new upload endpoint next to the existing one, something like: https://git.centos.org/sources/sig_upload.cgi * That new endpoint would upload the sources given using the same structure as the one used for CentOS-Stream, but ensuring that the person uploading is member of at least one SIG. The idea of using `sig_upload.cgi` instead of just replacing `upload.cgi` is the assumption that we want to preserve the current structure used for CentOS-Linux and CentOS-Stream, allowing to find more easily which sources are used where and not impacting the process Red Hat uses to push its releases. Since the structures used by the two upload scripts are different, they will not conflict. What we will end up seeing is something like: sources │ ├── pkg1 │ ├── c7 │ │ ├── hash1 │ │ └── hash2 │ ├── c8 │ │ ├── hash3 │ │ └── hash4 │ ├── tarball1 │ │ └── sha name │ │ └── hash5 │ │ └── tarball1 │ └── tarball2 │ └── sha name │ └── hash6 │ └── tarball2 │ └── pkg2 ├── c8 │ ├── hash7 │ └── hash8 ├── c8s │ ├── hash9 │ └── hash10 ├── tarball3 │ └── sha name │ └── hash11 │ └── tarball3 └── tarball4 └── sha name └── hash12 └── tarball4 and so on On CBS, the script that downloads the sources, will then need to be adjusted to try first the old structure before trying the new one. This may slow down a little bit the builds, but that should be most of the time, at most by a single http request. In this email I'm calling for feedback, do you like the idea? I'm happy to work on making it happen if there is consensus on this :) Looking forward for your thoughts, Pierre [1] https://lists.centos.org/pipermail/centos-devel/2022-February/120216.html

5 11

Using lookaside_upload in combination with the "flat" dist-git layout
by Daan De Meyer 25 Feb '22

25 Feb '22

Hi, Recently there was an announcement that the "flat" layout can now be used by SIGs that want to pull in changes from the Fedora RPM repos directly. I'm trying to switch the hyperscale branches for the systemd RPM to use the "flat" layout but I'm running into issues trying to upload the systemd tarball to the CentOS lookaside cache. I've successfully used the lookaside_upload script from centos-git-common to upload tarballs for use with the rpmbuild metadata layout before, but I'm struggling to get it to work with the "flat" layout. The main issue comes from the fact that in the "flat" layout, the sources file is used which contains a SHA512 hash that's used to find the corresponding tarball in the centos lookaside cache. However, the lookaside_upload script tries to upload tarballs to the lookaside cache using the sha1sum of the tarball as the identifier. This leads to CBS not finding the tarball when trying to do a build. I tried modifying the lookaside_upload script to use sha512sum instead of sha1sum but when doing this, I get a 500 HTTP internal error from the lookaside cache (https://paste.centos.org/view/5e7ef15c) Any other way to upload a tarball under the SHA512 identifier to the lookaside cache? Cheers, Daan De Meyer

1 1

[Storage-SIG] cephadm is not last version
by Luis Domingues 21 Feb '22

21 Feb '22

Hello, We are currently testing deploying clusters using CentOS Storage SIG. And while almost all packages are available on last version (16.2.7), we notice that cephadm was only on 16.2.5. Is any reason for that? Thanks, Luis Domingues Proton AG

4 4

Announcing possibility for SIGs to use "flat layout" on git.centos.org
by Fabian Arrotin 21 Feb '22

21 Feb '22

Recently, a thread (see https://lists.centos.org/pipermail/centos-devel/2022-January/120171.html) was started onn this list to discuss the possibility to let SIGs pushing to git in a "flat layout" style instead "rpmbuild layout" style on git.centos.org. As you know, since the beginning of git.centos.org, Red Hat pushed (as is still pushing, no change in sight for that part) in each /rpms/<package_name> git project in the "rpmbuild layout" style (so with SPECS/<pkg_name>.spec , eventually SOURCES/*) That's what SIGs have been using for years now, and it's how it's also documented in the SIG guide (see https://sigs.centos.org/guide/git/) Some SIGs wanted to bring "flat layout" style support so that they can just import existing branch from Fedora/Epel directly on git.centos.org without a need to change the dist-git layout, so get_sources.sh (from https://git.centos.org/centos-git-common) was recently modified to detect in which format the git repo is laid out and so how to retrieve sources from https://git.centos.org/sources. This request was all tracked in ticket https://pagure.io/centos-infra/issue/659 The needed new centpkg-minimal (https://cbs.centos.org/koji/packageinfo?packageID=8196) was built/pushed "live" today on cbs.centos.org , as some --scratch builds were tested with automotive SIG to confirm that it was working fine. We'll just add a simple note on the SIG Guide to mention that it's now possible to use the "flat layout" style (like Fedora/Epel) on git.centos.org for SIGs willing to use that layout. -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab

1 0

Linux Engineering update
by Josh Boyer 16 Feb '22

16 Feb '22

Hi All, I wanted to take a moment to thank the CentOS Stream contributor community and give an update on where we're at from the Red Hat Linux Engineering side. The contributions coming in for CentOS Stream, particularly Stream 9, have been fantastic. This is the kind of feedback loop we are trying to build, and the willingness to jump in headfirst from the community even though we still have some bumpy spots in workflow and guidance is much appreciated. We've seen: - package rebases - compose adjustments - bug fixes - subpackage additions and more from the community. While not all of them were accepted into CentOS Stream, overall it's been great. As we march towards spring, the Linux Engineering team is going to be focusing on finishing up the releases of Red Hat Enterprise Linux 8.6 and 9.0. Thankfully, the continuous development model of CentOS Stream means that it won't stop and community contributions should keep on going. With that being said, what each Linux Eng team needs to do varies widely and it may result in some slower or intermittent responses to contributions for the next few months. If someone doesn't respond quickly or asks to defer a contribution, please have a little patience. We'll get to it eventually. Lastly, I wanted to give some forward-looking contribution advice. CentOS Stream 9 has seen quite a bit of flux since we made it available. This has been quite helpful to shape the Red Hat Enterprise Linux 9 Beta, and what will be the RHEL 9 GA. After RHEL 9 GA, things will solidify more into what one would normally expect from an Enterprise Linux operating system. Package versions will settle, rebases will be less common, and our Linux Eng maintainers will need to evaluate CentOS Stream 9 contributions fully against the standards we have for RHEL. For contributions that are just outside of those standards in some way, we have a variety of other avenues within the CentOS project. Several CentOS SIGs are active today, building additional content on top of CentOS Stream. We also have associated projects like EPEL. If you have a contribution and you're not quite sure if it's something CentOS Stream can accommodate, please reach out to the Feature Request SIG [1]. They're there to help you navigate the best place for a contribution, and to help shephard along where they can. Thank you so much for the success we've had to date. It's far from perfect, but building out a new project is fun in and of itself. josh [1] https://wiki.centos.org/SpecialInterestGroup/StreamFeatureRequest

1 0

CentOS SIGs and gitlab
by Pierre-Yves Chibon 15 Feb '22

15 Feb '22

Good Morning Everyone, Some people have already noticed some asks being made to the CentOS infrastructure team and may be wondering what is the end goal of all of these. So this email is to try to share the end goal/vision of what I would to like to reach and open it up for comments/thoughts. I know of folks wanting to use gitlab for their workflow, this is for example the case of the automotive SIG folks. One reason for this is the automation that has been developed around the gitlab API in other contexes but that can be re-used for the SIG's benefits. So I have been looking at ways to make it possible for the Automotive SIG to use gitlab, but also keeping in mind that other SIGs may also be interested in re-using that model. First of all, a disclaimer: *all of the following only applies to SIGs who are interested. It is entirely opt-in. There is absolutely no desire to change what is working today for SIGs who do not wish to change*. So the way I envision things is the following: * SIGs who want they will be able to request a namespace at gitlab.com/centos/<sig_name> * The SIG chairs will be made owner of that namespace, giving them full power to organize it the way they wish * Guidance will be provided in the SIG's guide as to the different approaches possible (from the most simple to more complexes [1]). Those will be just that: guidance, SIGs can choose to follow them or ignore them * Gitlab groups will be mapped to groups in the CentOS accout system * Upon requests, groups can be created in the CentOS account system - This will allow SIG to fine-tune their access control if they so choose * SIGs will be able to use that gitlab namespace to host the git component of their dist-git * SIGs will be able to use either the flat dist-git layout or the exploded SRPM layout in their git repo, at their choosing, regardless of where they host their repositories * The lookaside cache used by the SIGs will be reworked to drop the requirement on branch name in the path. The end result will likely look like the Fedora or CentOS-Stream's lookaside cache. There are potentially a few ways to arrive to this, I have not started any conversation around this I have no ETA for most of these items, some are already work in progress (such as the possibility to use flat-layout in dist-git [2]), others are quite far down the line (cf the point about the lookaside cache). What do you all think of these ideas? Is that something that would be of interest? Did I miss something? If you would like more realtime discussion on this topic, it will be on the agenda of the infra SIG at 14 UTC on Monday the 21st in #centos-meeting. Looking forward to hear your thoughts! Pierre [1] for example: https://pagure.io/centos-infra/issue/645#comment-779917

2 1

Feedback on office hours
by Shaun McCance 15 Feb '22

15 Feb '22

Hi all, We have three sets of office hours that I'd like to gather feedback on. 1) CentOS Stream office hours are monthly on the second Wednesday, the same day as board meetings, at 17:00 UTC. This is a video meeting on meet.opensuse.org. It always has a few attendees. Sometimes productive stuff happens. Sometimes it's just a social hour, and that's ok. We should keep this one. I wouldn't mind a time shift or a shift in which week, because I happen to have another monthly meeting that conflicts. 2) CentOS office hours are monthly on the very next day on Thursday, at 10:00 UTC. This happens on IRC. Nobody attends, but we don't promote it at all. The time slot is awful for the US. It's a great time slot to bridge Europe and Asia. If we want to keep a time slot for Europe and Asia, we need to find attendees from those time zones, because I'm not awfully fond of waking up at 5am to look at a blank screen. 3) CentOS board office hours happen the week after the board meeting, so third week, Wednesday at 15:00 UTC. This is a video meeting on a Google Meet room I set up each month. The time slot is kind of early for US Pacific (7am). The purpose of this one is to ask the board questions about board stuff, so it's kind of important we have some board folks there. I'm open to rescheduling, canceling, consolidating, changing venues, or any level of bike shedding. Office hours can have value. They can also be a waste of time. Let's make sure we have something that adds value for our community. Thoughts? -- Shaun

2 1

Announcing Fedora-messaging notifications for cbs.centos.org
by Fabian Arrotin 14 Feb '22

14 Feb '22

We recently received this request from the Automotive SIG to enable fedora-messaging notification on the cbs.centos.org koji instance. (see https://pagure.io/centos-infra/issue/650) After some tests in dev/stg environment (thanks Pingou !), we pushed live the change earlier today and that means that if you're already using fedora-messaging, you'll be able to track activities from cbs.centos.org on fedora-messaging, or through datagrepper. For example, you'll be able to use https://apps.fedoraproject.org/datagrepper/v2/search?category=cbs to see recent activity on cbs.centos.org More information about fedora-messaging on https://fedora-messaging.readthedocs.io/ and https://apps.fedoraproject.org/datagrepper/ -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab

1 0

Board office hours, Feb 16
by Shaun McCance 11 Feb '22

11 Feb '22

This week was a board meeting, so next week we'll have board office hours. Join the board Wednesday, February 16, at 15:00 UTC. Video call link: https://meet.google.com/qud-kgjn-tfg If you need dial-in details instead, just email me off-list. Thanks, Shaun

1 0

CPE Weekly Update – Week of February 7th – 11th
by Michal Konecny 10 Feb '22

10 Feb '22

Hi everyone, This is a weekly report from the CPE (Community Platform Engineering) Team. If you have any questions or feedback, please respond to this report or contact us on #redhat-cpe channel on libera.chat (https://libera.chat/) If you wish to read this in form of a blog post, check the post on Fedora community blog: https://communityblog.fedoraproject.org/cpe-weekly-update-week-of-february-… # Highlights of the week ## Infrastructure & Release Engineering Goal of this Initiative ----------------------- Purpose of this team is to take care of day to day business regarding CentOS and Fedora Infrastructure and Fedora release engineering work. It’s responsible for services running in Fedora and CentOS infrastructure and preparing things for the new Fedora release (mirrors, mass branching, new namespaces etc.). The ARC (which is a subset of the team) investigates possible initiatives that CPE might take on. Update ------ ### Fedora Infra * Got koji issues sorted and things back on track * Worked with Fedora CoreOS folks on bringing up a ppc64le builder, which exposed a virt issue we have. Will be installing a power9 box with Fedora 35 to test if it fixes it. * Got more ocp4 workers added, upgraded clusters a few times without incident * First live prod app on ocp4 ([blockerbugs](https://qa.fedoraproject.org/blockerbugs/)) * Outage Friday caused by disk space issue on proxies. ;( * 2 10TB iscsi volumes setup on fedora iad2 netapp for CentOS folks. * Fedimg is broken apparently, caused ci issues. Still need to find a fix to it. ( https://pagure.io/fedora-infrastructure/issue/10532 ) ### CentOS Infra including CentOS CI * Still storage/hardware issues to work on * [Dell server](https://pagure.io/centos-infra/issue/649) for iscsi/netapp usage * CentOS [backup server](https://pagure.io/centos-infra/issue/618) * https://debuginfod.centos.org should be [live/announced](https://lists.centos.org/pipermail/centos-devel/2022-Februa… (content for CentOS Stream 8 and 9 and SIGs packages) * Bussiness as usual ### Release Engineering * Branching of F36 * Rawhide nightlies are enabled again * F37 builds are being signed with F37 key * We have a Fedora 36 branched compose done already * Started work on [automation of scm requests](https://pagure.io/releng/issue/9274) ## CentOS Stream Goal of this Initiative ----------------------- This initiative is working on CentOS Stream/Emerging RHEL to make this new distribution a reality. The goal of this initiative is to prepare the ecosystem for the new CentOS Stream. Updates ------- * February planning done * Adding centpkg to EPEL * Continuing aligning c8s + c9s workflows with an optimum delivery date now agreed * Content Resolver upgrades continuing with maintainer page being added ## CentOS Duffy CI Goal of this Initiative ----------------------- Duffy is a system within CentOS CI Infra which allows tenants to provision and access bare metal resources of multiple architectures for the purposes of CI testing. We need to add the ability to checkout VMs in CentOS CI in Duffy. We have OpenNebula hypervisor available, and have started developing playbooks which can be used to create VMs using the OpenNebula API, but due to the current state of how Duffy is deployed, we are blocked with new dev work to add the VM checkout functionality. Updates ------- * Legacy API meta client merged * Provisioning/Deprovisioning implemented (in review) * Outstanding: integration tests & loose ends ## Image builder for Fedora IoT Goal of this Initiative ----------------------- Integration of Image builder as a service with Fedora infra to allow Fedora IoT migrate their pipeline to Fedora infra. Updates ------- * local dev environment successfully deployed * koji in containers, mocked oidc auth, builder in containers, and osbuild-compose running as a service * We can run a full koji build using the image-builder plugins to make a successful compose ## Bodhi Goal of this Initiative ----------------------- This initiative is to separate Bodhi into multiple sub packages, fix integration and unit tests in CI, fix dependency management and automate part of the release process. Read ARC team findings in detail at: https://fedora-arc.readthedocs.io/en/latest/bodhi/index.html Updates ------- * Move Bodhi from python-openid to OIDC (ongoing) * Automate staging process for RPM building in Koji (ongoing) * Dependency management via Poetry and dependabot (ongoing) ## EPEL Goal of this initiative ----------------------- Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL). EPEL packages are usually based on their Fedora counterparts and will never conflict with or replace packages in the base Enterprise Linux distributions. EPEL uses much of the same infrastructure as Fedora, including buildsystem, bugzilla instance, updates manager, mirror manager and more. Updates ------- * EPEL9 up to 1775 source packages (increase of 127 from last week) * [RDO considering shipping openstack client packages in EPEL](https://meetings.opendev.org/meetings/rdo_meeting___2022_02_09/2022/r… * EPEL talks at CentOS Dojo * [State of EPEL on CentOS](https://youtu.be/JqnENGdvG88) * [Bootstrapping new EPEL releases with ebranch](https://youtu.be/VjPZmq_h2Rk) * [EPEL Packaging Hackfest](https://youtu.be/vZ-CX5XJV2Q) Kindest regards, CPE Team

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

devel