Hi all,
Here is the update for work recently done for the CentOS Community Container Pipeline.
- Docker builds in the CCCP-Service were taking a long time for building each images. We tracked down the issue and got in conclusion of building the images sequentially. This increase the performance as the other option of creating docker lvm pool is already in place. - - For getting the builds sequential we created beanstalkd tubes managing the builds from jenkins. Jenkins pushes all the builds to beankstalkd tube. beanstalk client starts the build one after another is complete. - We were checking with the facts to include OSBS as build client for CCCP. After analyzing different facts of benefits of osbs and our current issues for the system, we are postponing OSBS adoption till our first release. We will be looking for adopting this after that. - - We got atomic registry built from CCCP-Service with proper namespace. This is also demoed as end to end user story to the team. Currently we are taking the origin base images from outside of registry.centos.org
In the coming days, we will be focusing on :
- Getting Openshift origin completely built from registry.centos.org with all its base images (also indirectly solving problem with dependent container images) - Write a wiki page on wiki.centos.org for atomic registry. - Setting up end to end delivery with test suites (container scanning etc) and build sequence.
*Mohammed Zeeshan Ahmed* Associate Software Engineer, Redhat Developers Team (Devtools) IRC : mzee1000
mohammedzee1000.wordpress.com
Old PGP Public Key : http://pgp.mit.edu/pks/lookup?op=get&search=0x726A119E62847E37 (REVOKED) New PGP Public Key : https://pgp.mit.edu/pks/lookup?op=get&search=0x147A8569AFC5E9ED
We slightly touched the topic of Pipeline with KB today and I promised to kick-off some further discussion. What is our intention:
* We have sources for docker containers based on SCLs at e.g.: https://github.com/sclorg/mariadb-container/
* For every pull request and also commit, we're able to check whether the tests pass, so after every commit we should be pretty convinced the image is ready to be built (that is our intention now to make this working withing ci.centos.org, which seems to be good enough for it already now)
* After building the docker image we'd like to run the test again and if tests pass, then push the image to the docker hub under centos/ namespace. (it could be probably done in CentOS CI as well, but KB seems to be against doing it given we have CCCP being developed)
I have couple of questions regarding CCCP and it's current status though: * Which steps from the above we can already do in the CCCP today? * Which steps from the above will we be able to do in the CCCP soon? (ETA?) * Which steps from the above won't we be able to do in CCCP any time soon? * [1] talks about Nulecule a lot -- what if our images are not Nulecule-ized, will we be able to use CCCP still?
[1] https://github.com/kbsingh/cccp-index
Regards, Honza
On 06/20/2016 01:31 PM, Mohammed Ahmed wrote:
Hi all,
Here is the update for work recently done for the CentOS Community Container Pipeline.
- Docker builds in the CCCP-Service were taking a long time for building each images. We tracked down the issue and got in conclusion of building the images sequentially. This increase the performance as the other option of creating docker lvm pool is already in place.
- For getting the builds sequential we created beanstalkd tubes managing the builds from jenkins. Jenkins pushes all the builds to beankstalkd tube. beanstalk client starts the build one after another is complete.
- We were checking with the facts to include OSBS as build client for CCCP. After analyzing different facts of benefits of osbs and our current issues for the system, we are postponing OSBS adoption till our first release. We will be looking for adopting this after that.
- We got atomic registry built from CCCP-Service with proper namespace. This is also demoed as end to end user story to the team. Currently we are taking the origin base images from outside of registry.centos.org http://registry.centos.org
In the coming days, we will be focusing on :
- Getting Openshift origin completely built from registry.centos.org http://registry.centos.org with all its base images (also indirectly solving problem with dependent container images)
- Write a wiki page on wiki.centos.org http://wiki.centos.org for atomic registry.
- Setting up end to end delivery with test suites (container scanning etc) and build sequence.
*Mohammed Zeeshan Ahmed* Associate Software Engineer, Redhat Developers Team (Devtools) IRC : mzee1000
mohammedzee1000.wordpress.com http://mohammedzee1000.wordpress.com
Old PGP Public Key : http://pgp.mit.edu/pks/lookup?op=get&search=0x726A119E62847E37 (REVOKED) New PGP Public Key : https://pgp.mit.edu/pks/lookup?op=get&search=0x147A8569AFC5E9ED
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
Hi Honza,
On Mon, Jun 20, 2016 at 9:06 PM, Honza Horak hhorak@redhat.com wrote:
We slightly touched the topic of Pipeline with KB today and I promised to kick-off some further discussion. What is our intention:
- We have sources for docker containers based on SCLs at e.g.:
https://github.com/sclorg/mariadb-container/
- For every pull request and also commit, we're able to check whether the
tests pass, so after every commit we should be pretty convinced the image is ready to be built (that is our intention now to make this working withing ci.centos.org, which seems to be good enough for it already now)
- After building the docker image we'd like to run the test again and if
tests pass, then push the image to the docker hub under centos/ namespace. (it could be probably done in CentOS CI as well, but KB seems to be against doing it given we have CCCP being developed)
I have couple of questions regarding CCCP and it's current status though:
- Which steps from the above we can already do in the CCCP today?
All the above points are included in CCCP now. We are running the tests on the recently built image from using test script provided by the project team.
* Which steps from the above will we be able to do in the CCCP soon? (ETA?)
We are building image for any change to the source repo (may be each push). may be we will be moving for each commit also soon.
- Which steps from the above won't we be able to do in CCCP any time soon?
- [1] talks about Nulecule a lot -- what if our images are not
Nulecule-ized, will we be able to use CCCP still?
For now we are not building Nulecule-ized images. Current CCCP builds images directly from Dockerfile.
Please let us know if you have something more to clarify.
Thanks, Regards Bamacharan
[1] https://github.com/kbsingh/cccp-index
Regards, Honza
On 06/20/2016 01:31 PM, Mohammed Ahmed wrote:
Hi all,
Here is the update for work recently done for the CentOS Community Container Pipeline.
- Docker builds in the CCCP-Service were taking a long time for building each images. We tracked down the issue and got in conclusion of building the images sequentially. This increase the performance as the other option of creating docker lvm pool is already in place.
- For getting the builds sequential we created beanstalkd tubes managing the builds from jenkins. Jenkins pushes all the builds to beankstalkd tube. beanstalk client starts the build one after another is complete.
- We were checking with the facts to include OSBS as build client for CCCP. After analyzing different facts of benefits of osbs and our current issues for the system, we are postponing OSBS adoption till our first release. We will be looking for adopting this after that.
- We got atomic registry built from CCCP-Service with proper namespace. This is also demoed as end to end user story to the team. Currently we are taking the origin base images from outside of registry.centos.org http://registry.centos.org
In the coming days, we will be focusing on :
- Getting Openshift origin completely built from registry.centos.org http://registry.centos.org with all its base images (also indirectly solving problem with dependent container images)
- Write a wiki page on wiki.centos.org http://wiki.centos.org for atomic registry.
- Setting up end to end delivery with test suites (container scanning etc) and build sequence.
*Mohammed Zeeshan Ahmed* Associate Software Engineer, Redhat Developers Team (Devtools) IRC : mzee1000
mohammedzee1000.wordpress.com http://mohammedzee1000.wordpress.com
Old PGP Public Key : http://pgp.mit.edu/pks/lookup?op=get&search=0x726A119E62847E37 (REVOKED) New PGP Public Key : https://pgp.mit.edu/pks/lookup?op=get&search=0x147A8569AFC5E9ED
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
-
Bamacharan Kundu -
Honza Horak -
Mohammed Ahmed