I've volunteered to help maintain official CentOS Docker ( https://www.docker.io/) images, and Karanbir suggested I email the list to "thrash out what we need at the infra end to achieve these images."
Currently, there are 'centos' images available in the top-level Docker registry namespace, but these are maintained by dotcloud. I think we can do better. :)
As I see it, there are two ways we can do this:
1. Use ami-creator (https://github.com/katzj/ami-creator), or something derived from it, to build Docker images from a kickstart file. This is how CentOS AMIs are currently built.
Pros: Consistent with how AMIs are usually built; Kickstart is a known entity.
Cons: ami-creator won't actually work for us as-is; requires a full kickstart config.
2. Use the mkimage-yum.sh script from Docker ( https://github.com/dotcloud/docker/blob/master/contrib/mkimage-yum.sh), customized as necessary to produce a minimal base image.
Pros: Consistent with how Docker images are usually built; we can contribute changes to mkimage-yum.sh upstream to help other Yum-based distros; only requires a yum config.
Cons: Requires changing a shell script to make changes to the base image, not just a Kickstart.
Unfortunately, Docker Trusted Build ( http://blog.docker.io/2013/11/introducing-trusted-builds/) can't be used for base images, AFAICT.
In either case, we'll need a git repo to hold the code and configs used to build the images. If we follow the example of the official CentOS AMIs ( http://wiki.centos.org/Cloud/AWS), it should be sufficient to provide a reasonably up-to-date base image ("latest" or "nightly" or similar), and a few recent dot releases. Of course, once CentOS 7 is released we'll need to do that for both 6 and 7, but that shouldn't be difficult if we do this right.
Thoughts on process and/or infrastructure?
Hi,
On 03/16/2014 07:30 PM, Chris St. Pierre wrote:
I've volunteered to help maintain official CentOS Docker (https://www.docker.io/) images, and Karanbir suggested I email the list to "thrash out what we need at the infra end to achieve these images."
Currently, there are 'centos' images available in the top-level Docker registry namespace, but these are maintained by dotcloud. I think we can do better. :)
As I see it, there are two ways we can do this:
- Use ami-creator (https://github.com/katzj/ami-creator), or something
derived from it, to build Docker images from a kickstart file. This is how CentOS AMIs are currently built.
Pros: Consistent with how AMIs are usually built; Kickstart is a known entity.
Cons: ami-creator won't actually work for us as-is; requires a full kickstart config.
ami_creator is still used for the pv images, but for the hvm images its now done as a native kvm image, then migrated. I think having a kickstart that people can contribute into isnt a bad idea, and as you mentioned - lots of people already familiar with it
- Use the mkimage-yum.sh script from Docker
(https://github.com/dotcloud/docker/blob/master/contrib/mkimage-yum.sh), customized as necessary to produce a minimal base image.
Pros: Consistent with how Docker images are usually built; we can contribute changes to mkimage-yum.sh upstream to help other Yum-based distros; only requires a yum config.
Cons: Requires changing a shell script to make changes to the base image, not just a Kickstart.
how about just using anaconda to drive the install ( so running the build like a native kvm image, then trimming). The yum installroot for @core is going to bring in a kernel etc as well, and we could --excludedocs and set installlang etc at the anaconda level. It also means, and I quite like, the idea that a single build process could be used for other things as well.
Unfortunately, Docker Trusted Build (http://blog.docker.io/2013/11/introducing-trusted-builds/) can't be used for base images, AFAICT.
afaik, trusted-builds are a CI component of app containers and might not be very relevant to the platform generic containers we might end up doing initially ? or am I missing something else here. Are they trying to setup trusted builds as a 'validated/verified good build with a trackable upstream' ? commit access to a github repo might not be much of in terms of traceability.
In either case, we'll need a git repo to hold the code and configs used to build the images. If we follow the example of the official CentOS AMIs (http://wiki.centos.org/Cloud/AWS), it should be sufficient to provide a reasonably up-to-date base image ("latest" or "nightly" or similar), and a few recent dot releases. Of course, once CentOS 7 is released we'll need to do that for both 6 and 7, but that shouldn't be difficult if we do this right.
We can work with or setup a new git repo for the docker configs, metadata etc and I would say we use the same workflow that has been proposed for the other cloud-instance sig stuff : ie. mirror on github for easier import, and manually bridge to git.centos.org, with triggers to build instance at git.centos.org.
If we can get to the AWS pages in the wiki, that would be good - and easy to get to, just needs someone to work on it a bit. So thanks for raising the hand :)
In terms of versions, we should do CentOS-5 as well. And we should perhaps, if we can get away with it, drop the updated-in-time images and stick to just the release content in CentOS-5/6 and then 7. The updated-in-time VM's have been a train wreck at AWS, and for 6.5 I didnt do it at all.
- KB
On 03/18/2014 09:03 AM, Karanbir Singh wrote:
In terms of versions, we should do CentOS-5 as well. And we should perhaps, if we can get away with it, drop the updated-in-time images and stick to just the release content in CentOS-5/6 and then 7.
Also, at this point we should target the easiest wins and ship a base OS image, but keep in mind that shipping app containers might be an interesting option as well in the future ( gluster cluster as an example ? )
- KB
On Tue, Mar 18, 2014 at 5:03 AM, Karanbir Singh mail-lists@karan.orgwrote:
how about just using anaconda to drive the install ( so running the build like a native kvm image, then trimming). The yum installroot for @core is going to bring in a kernel etc as well, and we could --excludedocs and set installlang etc at the anaconda level. It also means, and I quite like, the idea that a single build process could be used for other things as well.
I'm sold. It sounds like we'd need the build process to have three components:
* A kickstart; * A post-processing script that packages the image up (into either an AMI or a Docker image or whatever else comes along); and, of course * The harness that ties it all together.
If you can point me to the code that's used to create HVM AMIs, I can start poking at this.
afaik, trusted-builds are a CI component of app containers and might not be very relevant to the platform generic containers we might end up doing initially ? or am I missing something else here. Are they trying to setup trusted builds as a 'validated/verified good build with a trackable upstream' ? commit access to a github repo might not be much of in terms of traceability.
You're not missing anything -- they can't be used for base images, and they're not very trusted, either. I just mentioned them to definitively rule them out.
We can work with or setup a new git repo for the docker configs, metadata etc and I would say we use the same workflow that has been proposed for the other cloud-instance sig stuff : ie. mirror on github for easier import, and manually bridge to git.centos.org, with triggers to build instance at git.centos.org.
Any reason we can't just host it entirely on github and use webhooks to trigger the builds at git.centos.org?
In terms of versions, we should do CentOS-5 as well. And we should perhaps, if we can get away with it, drop the updated-in-time images and stick to just the release content in CentOS-5/6 and then 7. The updated-in-time VM's have been a train wreck at AWS, and for 6.5 I didnt do it at all.
What has been painful about them? If they're not always stable for the end user, we could consider tagging them with "nightly" and leaving a stable build (e.g., stock 6.5) as "latest", so that people have to specifically ask for the fully-updated image.
E.g., 'docker run ... centos/centos6' runs CentOS 6.5, or whatever the most recent "stable build" is. 'docker run ... centos/centos6:nightly' runs a fully-updated image.
BTW, I did snag the 'centos' username on the Docker index; I'll contact you or Jim on IRC today to transfer it to you guys.
Also, at this point we should target the easiest wins and ship a base OS image, but keep in mind that shipping app containers might be an interesting option as well in the future ( gluster cluster as an example
? )
Yes! If we're willing to make this a separate process, this would be super easy with Dockerfiles. We could even use Trusted Builds for this, based on our own images, and not have to use CentOS' own resources. :)
On Tue, Mar 18, 2014 at 07:27:35AM -0400, Chris St. Pierre wrote:
- A kickstart;
- A post-processing script that packages the image up (into either an AMI
or a Docker image or whatever else comes along); and, of course
- The harness that ties it all together.
If you can point me to the code that's used to create HVM AMIs, I can start poking at this.
You may want to look at the Docker plugin in ImageFactory. Not yet live, but the Fedora plan is to use that plugin in Koji to generate our official images.
On 03/18/2014 09:35 PM, Matthew Miller wrote:
On Tue, Mar 18, 2014 at 07:27:35AM -0400, Chris St. Pierre wrote:
- A kickstart;
- A post-processing script that packages the image up (into either an AMI
or a Docker image or whatever else comes along); and, of course
- The harness that ties it all together.
If you can point me to the code that's used to create HVM AMIs, I can start poking at this.
You may want to look at the Docker plugin in ImageFactory. Not yet live, but the Fedora plan is to use that plugin in Koji to generate our official images.
There are 2 issues I have with ImageFactory : 1) it does not solve any problem that isnt already addressed by the libvirt tools, and is actually limiting in its scope. 2) it needs yet another piece of metadata that is in yet another differerent file format.
Whatever route we take - there will be some need to integrate with the tools, and i just feel that the code might be better utilised, repeatedly shared as well, if we target the libvirt tools instead.
On Tue, Mar 18, 2014 at 09:03:55AM +0000, Karanbir Singh wrote:
build like a native kvm image, then trimming). The yum installroot for @core is going to bring in a kernel etc as well, and we could --excludedocs and set installlang etc at the anaconda level. It also means, and I quite like, the idea that a single build process could be used for other things as well.
This is the plan in Fedora, by the way, and I'm working with the Anaconda team to make enhancements so there won't need to be as much trimming. The latter part probably help you guys for a few years :) but it'd be nice to be using the same basic approach.
In terms of versions, we should do CentOS-5 as well. And we should perhaps, if we can get away with it, drop the updated-in-time images and stick to just the release content in CentOS-5/6 and then 7. The updated-in-time VM's have been a train wreck at AWS, and for 6.5 I didnt do it at all.
Could you tell me more about the train wreck here? I want to make sure we don't recreate it. :)
On 03/18/2014 09:28 PM, Matthew Miller wrote:
On Tue, Mar 18, 2014 at 09:03:55AM +0000, Karanbir Singh wrote:
build like a native kvm image, then trimming). The yum installroot for @core is going to bring in a kernel etc as well, and we could --excludedocs and set installlang etc at the anaconda level. It also means, and I quite like, the idea that a single build process could be used for other things as well.
This is the plan in Fedora, by the way, and I'm working with the Anaconda team to make enhancements so there won't need to be as much trimming. The latter part probably help you guys for a few years :) but it'd be nice to be using the same basic approach.
Sounds good, where can we track this effort ?
Also, it would be good to sync and share with the Fedora cloud folks, my attempts at doing this in the past have mostly been met with cold sholder like responses.
In terms of versions, we should do CentOS-5 as well. And we should perhaps, if we can get away with it, drop the updated-in-time images and stick to just the release content in CentOS-5/6 and then 7. The updated-in-time VM's have been a train wreck at AWS, and for 6.5 I didnt do it at all.
Could you tell me more about the train wreck here? I want to make sure we don't recreate it. :)
Nightly builds are good, as long as they are communicated as such and people understand that if they want to own the feature spec, they need to own their own infra to build, test and deploy based on those. For everyone else, setting a baseline in time does not seem to map very well to any app requirements. Reducing those to map to iso release points has been far more productive, and people have a better ( and easier, requiring far less communication to get across ) map of content and where things are.
The question it boils down to is : why are we building these images, where do we anticipate them being used and for what purpose.
On Wed, Mar 19, 2014 at 11:39:05AM +0000, Karanbir Singh wrote:
This is the plan in Fedora, by the way, and I'm working with the Anaconda team to make enhancements so there won't need to be as much trimming. The latter part probably help you guys for a few years :) but it'd be nice to be using the same basic approach.
Sounds good, where can we track this effort ?
I'm writing a feature proposal for F21, which should be _part_ of tracking it. I'm also working on getting our own tracking in a bit better shape... someone in our SIG is setting up a Cantas (open source trello-alike) board which I hope will help keep this organized better.
Also, it would be good to sync and share with the Fedora cloud folks, my attempts at doing this in the past have mostly been met with cold sholder like responses.
I hope that wasn't me. I think it's mostly a lack of time, not actual maliciousness. Speaking personally, I'm terrible at followups and apologize in advance. :)
Could you tell me more about the train wreck here? I want to make sure we don't recreate it. :)
Nightly builds are good, as long as they are communicated as such and people understand that if they want to own the feature spec, they need to own their own infra to build, test and deploy based on those. For everyone else, setting a baseline in time does not seem to map very well to any app requirements. Reducing those to map to iso release points has been far more productive, and people have a better ( and easier, requiring far less communication to get across ) map of content and where things are. The question it boils down to is : why are we building these images, where do we anticipate them being used and for what purpose.
Makes sense. Thanks.
-
Chris St. Pierre -
Karanbir Singh -
Matthew Miller