To consolidate the docker effort moving around the -devel list [1][2], I've been talking with the docker folks about taking over the 'official' CentOS images they have in place, and transferring the responsibility to the cloud sig. Currently Chris StPierre and Adam Miller have been helping to do the work and get us in contact with the right people.
As it stands right now, the docker folks use stackbrew[3] to generate their images, jam the resulting image tarball into git, and submit a pull request when updates are required. As long as the image can be imported into docker, they don't seem to have a problem with the way we're currently[4] doing things
The recent openssl bug has gotten them a bit more hurried to transition the responsibility back to us.
Immediate requirements: - An updated docker image, built from the most recent updates so that the openssl fix is in place.
Short term requirements: - A brief description of the centos image, with url for more information
- Wiki page documenting the basics of the docker image, how it's created, and how to use it.
Long term requirements: - Update plan for for keeping the docker image updated, along with 'emergency roll-out plan' for critical updates. - established policy for docker and other cloud images.
Thoughts?
[1] http://lists.centos.org/pipermail/centos-devel/2014-April/010070.html [2] http://lists.centos.org/pipermail/centos-devel/2014-April/010090.html [3] https://github.com/dotcloud/stackbrew [4] https://github.com/CentOS/sig-cloud-instance-build/tree/master/docker
On Tue, Apr 08, 2014 at 09:34:28AM -0500, Jim Perrin wrote:
Long term requirements:
- Update plan for for keeping the docker image updated, along with
'emergency roll-out plan' for critical updates.
- established policy for docker and other cloud images.
I saw some comments in the CentOS github about coordinating with Fedora. Even if we end up with different tooling for initial creation, I think a lot of the surrounding effort can still be shared.
On 04/08/2014 12:51 PM, Matthew Miller wrote:
On Tue, Apr 08, 2014 at 09:34:28AM -0500, Jim Perrin wrote:
Long term requirements:
- Update plan for for keeping the docker image updated, along with
'emergency roll-out plan' for critical updates.
- established policy for docker and other cloud images.
I saw some comments in the CentOS github about coordinating with Fedora. Even if we end up with different tooling for initial creation, I think a lot of the surrounding effort can still be shared.
Yes. The TL;DR for the github exchange is this (for those not following):
We currently use ami_creator several tasks, including the docker images. Fedora uses another tool suite, appliance-tools to build docker.
We're questioning the merits of adding appliance-tools, where ami_creator currently fills our need.
So lets see if we can get a broader discussion going here:
KB, can you provide some details on what ami_creator is currently used for beyond docker?
Adam/Matt, could you provide some detail as to the benefits of appliance-tools, and the capabilities?
If there's enough overlap, it may make sense to move. If not, maybe we keep using ami_creator and look at other areas of cooperation.
Folks interested in the Cloud SIG, thoughts, comments, fact-based-opinions?
On Tue, Apr 08, 2014 at 12:59:18PM -0500, Jim Perrin wrote:
Adam/Matt, could you provide some detail as to the benefits of appliance-tools, and the capabilities?
Wellllllll. We are actively migrating away from appliance tools, actually. the two benefits were:
- uses a standard input format we are all familiar with (kickstart) - integrated into koji for builds (so we get the repeatability and traceability benefits that brings)
Now with Koji 1.9, we have an anaconda/imagefactory toolchain, which has these advantages:
- also uses kickstart - appliance-creator in maintenance mode only, anaconda actively developed - even better koji integration (tarball outputs for docker to take in)
But, I think even that is intermediary. There is preliminary work on an idea which uses rpm in mock as the build format:
- uses a standard input format we are all familiar with (spec files) - much lighter weight than anaconda - in active development - can handle _layered_ images - will be integrated with koji (i think also through imagefactory)
The nice thing here is that the output format (and fedbus messages) should be the same if we make this second transition in the future, so the koji -> docker registry bits can be the same.
On 04/08/2014 02:41 PM, Matthew Miller wrote:
On Tue, Apr 08, 2014 at 12:59:18PM -0500, Jim Perrin wrote:
Adam/Matt, could you provide some detail as to the benefits of appliance-tools, and the capabilities?
Wellllllll. We are actively migrating away from appliance tools, actually. the two benefits were:
- uses a standard input format we are all familiar with (kickstart)
so that's pretty much the same as what we're doing now then.
- integrated into koji for builds (so we get the repeatability and traceability benefits that brings)
and we're not using koji for the core builds.
Now with Koji 1.9, we have an anaconda/imagefactory toolchain, which has these advantages:
- also uses kickstart
- appliance-creator in maintenance mode only, anaconda actively developed
- even better koji integration (tarball outputs for docker to take in)
But, I think even that is intermediary. There is preliminary work on an idea which uses rpm in mock as the build format:
Since el6 will still be current/relevant to us for quite some time, would this new method be able to create el6 images as well?
- uses a standard input format we are all familiar with (spec files)
- much lighter weight than anaconda
- in active development
- can handle _layered_ images
- will be integrated with koji (i think also through imagefactory)
The nice thing here is that the output format (and fedbus messages) should be the same if we make this second transition in the future, so the koji -> docker registry bits can be the same.
On Tue, Apr 8, 2014 at 3:40 PM, Jim Perrin jperrin@centos.org wrote:
On 04/08/2014 02:41 PM, Matthew Miller wrote:
On Tue, Apr 08, 2014 at 12:59:18PM -0500, Jim Perrin wrote:
Adam/Matt, could you provide some detail as to the benefits of appliance-tools, and the capabilities?
Wellllllll. We are actively migrating away from appliance tools,
actually.
the two benefits were:
- uses a standard input format we are all familiar with (kickstart)
so that's pretty much the same as what we're doing now then.
- integrated into koji for builds (so we get the repeatability and traceability benefits that brings)
and we're not using koji for the core builds.
Now with Koji 1.9, we have an anaconda/imagefactory toolchain, which has these advantages:
- also uses kickstart
- appliance-creator in maintenance mode only, anaconda actively
developed
- even better koji integration (tarball outputs for docker to take in)
But, I think even that is intermediary. There is preliminary work on an
idea
which uses rpm in mock as the build format:
Since el6 will still be current/relevant to us for quite some time, would this new method be able to create el6 images as well?
For what it's worth I figured I'd mention the github comment thread that kind of kicked off this topic.
https://github.com/CentOS/sig-cloud-instance-build/pull/4
Also, in the event this ends up going anywhere I've moved forward with getting appliance-tools into EPEL6
https://admin.fedoraproject.org/updates/appliance-tools-007.7-2.1.el6
-AdamM
- uses a standard input format we are all familiar with (spec files)
- much lighter weight than anaconda
- in active development
- can handle _layered_ images
- will be integrated with koji (i think also through imagefactory)
The nice thing here is that the output format (and fedbus messages)
should
be the same if we make this second transition in the future, so the koji -> docker registry bits can be the same.
-- Jim Perrin The CentOS Project | http://www.centos.org twitter: @BitIntegrity | GPG Key: FA09AD77 _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org http://lists.centos.org/mailman/listinfo/centos-devel
On 04/08/2014 06:59 PM, Jim Perrin wrote:
KB, can you provide some details on what ami_creator is currently used for beyond docker?
not sure how i missed this thread...
Ami_creator is used for the ami's! and for the docker image, and its used for some testing stuff internally. Anything that works pv or does not need a kernel to boot, builds via ami_creator.
Everything else, ie. HVM needed, builds via virt-install and kickstarts. In many cases the kickstart is shared between ami_creator and virt-install.
Personally, i think virt-install is a good place to end up with everything - it really does represent a virtualised instance and runs the real distro installer. And the xml fluffery is well abstracted away.
ImgFac and Appliance Tools are both worth considering.
The big question is : do we try and use one and only one tool ? or do we use whatever tool gets us the biggest win for the specific delivery requirements, with the least amount of 'ownership' required. Both of those come with their own challenges.
and lets not forget ostree in CentOS Seven
On Mon, May 12, 2014 at 06:44:59PM +0100, Karanbir Singh wrote:
The big question is : do we try and use one and only one tool ? or do we use whatever tool gets us the biggest win for the specific delivery requirements, with the least amount of 'ownership' required. Both of those come with their own challenges. and lets not forget ostree in CentOS Seven
I don't know how it will work for CentOS, but there's work to get Anaconda to support ostree directly, which means that any process (imagefactory, virt-install, etc) that runs anaconda will get ostree support automatically.
On 05/12/2014 06:47 PM, Matthew Miller wrote:
On Mon, May 12, 2014 at 06:44:59PM +0100, Karanbir Singh wrote:
The big question is : do we try and use one and only one tool ? or do we use whatever tool gets us the biggest win for the specific delivery requirements, with the least amount of 'ownership' required. Both of those come with their own challenges. and lets not forget ostree in CentOS Seven
I don't know how it will work for CentOS, but there's work to get Anaconda to support ostree directly, which means that any process (imagefactory, virt-install, etc) that runs anaconda will get ostree support automatically.
I wonder if EL7 anaconda will get this functionality.. if not, we might need to wait on 8;
having an ostree install post-anaconda run install sounds great though. I presume this is still going to do the actual ostree builds ( in which case, how does one setup the remote repo ? )
On Mon, May 12, 2014 at 1:50 PM, Karanbir Singh kbsingh@centos.org wrote:
I wonder if EL7 anaconda will get this functionality.. if not, we might need to wait on 8;
I can't guarantee anything, but I hope this will become available.
having an ostree install post-anaconda run install sounds great though. I presume this is still going to do the actual ostree builds ( in which case, how does one setup the remote repo ? )
The current rpm-ostree patchset for Anaconda pulls a tree instead of laying down packages. That's about the only difference.
The process of assembling packages into trees on a compose server I call "treecompose". I used to call it "building" but there's no actual source code being compiled, merely a transformation of RPM content, so I think "compose" is clearer.
In this model Anaconda is replicating, not composing. Now you do raise an interesting point in that some people might want tree composition at install time, which would be very possible. I can definitely imagine some use cases for this.
-
Adam Miller -
Colin Walters -
Jim Perrin -
Karanbir Singh -
Karanbir Singh -
Matthew Miller