Upfront apologies for being forced to use Outlook to respond. I tried to make it right. Comments inline (If I did it right).
On 3/7/23, 9:01 AM, "CentOS-devel on behalf of Colin Walters" <centos-devel-bounces@centos.org mailto:centos-devel-bounces@centos.org on behalf of walters@verbum.org mailto:walters@verbum.org> wrote:
[. . .]
However, as we've been doing about this...I've been saying to people that I wish I had a time machine to go back and do bootable containers from the start. There's a lot of things we're doing today that I think we should stop doing, e.g.:
- Switching to kernel-rt by fiddling with each node; we should be simply pulling a pre-built bootable container image with that kernel (more on this below)
- Getting away from injecting so much persistent state by default (both via Ignition and outside of it)
And crucially, I think we should be developing tools and techniques that apply *outside* of Kubernetes/OpenShift and also work well with it. To be direct, I'd like to eventually productize some of what's happening here in RHEL, not in OpenShift.
As part of this (potential) re-architecture of how we think of systems management, I created the https://github.com/containers/bootc https://github.com/containers/bootc project. To be direct: If successful, I think bootc will be the successor to (rpm-)ostree. It's also intended to much more closely align with the github.com/containers organization.
A simple way to think of this is: One can (build and) run *application* containers with podman; and these containers can also be run in e.g. Kubernetes/OpenShift. One can build *bootable* containers using any tooling (including podman build), but *running* them is via bootc on the end machine. bootc understands kernels etc.
But there's a lot to figure out here - and I want to have a space to figure out this stuff and experiment with it outside of a direct-to-product path. I think a CentOS SIG makes sense for this.
I have a lot to learn here so, my curiosity is piqued.
So what I'd like to do is either:
- Add a new effort to the Cloud SIG, which currently (IMO a bit confusingly) hosts OpenStack/RDO and OpenShift/OKD things which would be a 3rd thing. The bootc work would then be the "base OS" split for OKD/SCOS. But of course, nothing stops one from building bootable host images that are instead designed to be RDO/OpenStack hosts.
- Or, create a new SIG
I don't see a reason to support another SIG for this image model. You are talking about supporting a different kernel though and that might (though I suspect it is not) be a greater burden on image builds. Personally, I have high confidence that if we run into pipeline issues, you are going to be able to provide a lot os support there.
Personally, I lean towards the latter because honestly I find the naming "Cloud" to be misleading - bootc is also intended to be useful for standalone, non-cloud-infrastructure settings (such as desktops and IoT).
Cloud, to me, is focused around that managed experience that I think you are describing, be it isolated to a desktop or a raspberry-pi. Sure there are much deeper layers of abstraction to use, but I think that those are great environments for experimentation that later ends up feeding a larger cloud experience. I think that this is a good fit.
Specifically, I'd like to transfer the existing code that lives in https://github.com/cgwalters/bootc-demo-base-images https://github.com/cgwalters/bootc-demo-base-images (specifically https://github.com/cgwalters/bootc-demo-base-images/blob/main/c9s.yaml https://github.com/cgwalters/bootc-demo-base-images/blob/main/c9s.yaml ) into something CentOS-affiliated and explicitly maintained by a team. (Though I'm not super excited to move it to pagure like at least some other SIG content, but let's not get distracted by git hosting too much here).
It doesn't seem to me that you think this is going to be in "demo" for that much longer and you have high confidence that this will be a better fit ultimately for the community. Recent discussions have also pointed me in that direction.
I have heard other advanced developers discuss getting to this lower level in my own world and I would love to see if you can make it work faster with the support of the team. Selfishly, I'd also like to have your help on what you might consider misleading about the word "cloud".
Another way to say it is that I'd love to ship quay.io/centos/centos-boot:stream9 (notice the -boot). Or failing that, it'd be quay.io/centos-boot/centos-boot:stream9 or so. There's a *lot* to discuss in terms of what actually goes in these base images, and also ensuring it's equally ergonomic for users to build their own base images. So really it's very likely there wouldn't be just *one* base image. In fact, I recently introduced a -rt variant with the RT kernel: https://github.com/cgwalters/bootc-demo-base-images/commit/68afb072a5a1396c7... https://github.com/cgwalters/bootc-demo-base-images/commit/68afb072a5a1396c7424ed536a896293fff8287d - and this was specifically motivated by issues we hit in OCP. But again, I want to have a space where we try to do more of a "clean(er) slate" approach for a while, with notes "not for production use" - for a while. Everything done here though *is* made with that as an explicit goal though (e.g. it's a toplevel design goal too that existing ostree-based systems can be seamlessly switched to be container-based without
reprovisioning).
At the same time, bootc already introduces some quite new things that need design iteration; for example: https://github.com/containers/bootc#using-bootc-install https://github.com/containers/bootc#using-bootc-install - we ship tooling such that a container can install itself (without going through a raw disk image as is used by both OCP and Edge deployments today). And at the same time, I'd like to aim to get the Anaconda changes to install these bootable containers in https://github.com/rhinstaller/anaconda/pull/4561 https://github.com/rhinstaller/anaconda/pull/4561
[. . . ] _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org mailto:CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel https://lists.centos.org/mailman/listinfo/centos-devel