devel

devel@lists.centos.org

4 participants
4592 discussions

Introducing Flamingo: A Lightweight Cloud Instance Contextualization Tool for CentOS Atomic Host
by Tamer TAS 23 Jul '15

23 Jul '15

When using on-demand instantiation of virtual machines in cloud computing, users pass configuration data to the cloud and that data is used for configuring the instances. This process is called contextualization. Contextualization includes identity(users, groups), network, system services and disk configuration.Flamingo is a contextualization tool that is being developed as a GSoC projectthat aims to handle early initialization of cloud instances. The current de facto standard for instance virtualization is cloud-init. However, there are some problems with it. The most prominent ones are:The usage of a scripting language (Python in this case), since scripting languages have the overhead of the interpreter, its dependencies and being slower than compiled ones due to their dynamic nature.The documentation is lacking at best. There are examples of common use-cases. However, most of the code-base and plug-ins are undocumented. Inspecting the code-base is a prerequisite to extend the functionality or even understand it.Test coverage is low. Making it hard to extend, maintain, and improve. Flamingo is a replacement for cloud-init that solves these issues. Flamingo aims to solve the following problems encountered in cloud contextualization;SpeedDependencyMaintainabilityExtensibility The current focus of Flamingo is Atomic Host and CentOS Current work-in progress is to build test images that are using Flamingo instead of cloud-init as thecontextualization tool for EC2, GCE, Openstack, and OpenNebula. You can find the source code for the tool here. For more details please check this blog post Feel free to make suggestions or ask questions.

1 0

No IRC log for last Cloud SIG meeting?
by Neil Jerram 23 Jul '15

23 Jul '15

Hi all, I participated briefly towards the end of last Thursday's Cloud SIG meeting, and just went looking for the log of that. Unfortunately it appears that it doesn't exist: http://www.centos.org/minutes/2015/july/ has a complete set of entries for the Atomic SIG at 2015-07-16-16.07, but only a single truncated text file for the Cloud SIG at 2015-07-16-15.02. Does anyone know why that is? Is there any other source for a trace of the Cloud SIG meeting? Many thanks, Neil

2 2

c7-ppc64 (big endian) progress
by James O'Connor 21 Jul '15

21 Jul '15

I stepped back from my unified power64 koji master plan and went back to using mock configs and shell for loops. I am still using a Fedora 22 ppc64 host but now have a mock config that bootstraps against a local Fedora 19 ppc64/ppc repository. I have built half of the 2500 CentOS 7 x86_64 source rpms. Toolchain (binutils/gcc/glibc) and build group packages are solid. kernel-3.10.0-229.el7.ppc64.rpm finished this morning. I will continue marching through the other 1200 source rpms and also build a bootable c7-ppc64 test image. After the majority of the 2500 ppc64 rpms are built, I will rebuild them in a second phase without the Fedora 19 backing repository. config_opts['root'] = 'rhel-7b1-ppc64' config_opts['basedir'] = '/var/lib/mock' config_opts['target_arch'] = 'ppc64' config_opts['legal_host_arches'] = ('ppc64',) config_opts['chroot_setup_cmd'] = 'install bash bzip2 coreutils cpio diffutils system-release findutils gawk gcc gcc-c++ grep gzip info make patch redhat-rpm-config rpm-build sed shadow-utils tar unzip util-linux which xz' config_opts['dist'] = 'el7' # only useful for --resultdir variable subst config_opts['%centos_ver'] = "7" config_opts['macros']['%centos_ver'] = "7" config_opts['macros']['%rhel'] = "7" config_opts['macros']['%el7'] = "1" config_opts['macros']['%redhat'] = "7" config_opts['macros']['%vendor'] = "CentOS" config_opts['macros']['%packager'] = "CentOS BuildSystem <http://bugs.centos.org>" config_opts['macros']['%_vendor'] = "redhat" config_opts['macros']['%_vendor_host'] = "redhat" config_opts['macros']['%_host'] = "ppc64-redhat-linux-gnu"

2 1

CBS/Infra Meeting 20-Jul-2015 14:00 UTC
by Brian Stinson 20 Jul '15

20 Jul '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi All, It's time for another meeting to talk about the Community Build System[0]. We will meet in #centos-devel on irc.freenode.net Proposed Agenda: - - Status Updates: - General - Central Auth - - Upcoming Projects Cheers! Brian [0]: http://wiki.centos.org/HowTos/CommunityBuildSystem -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJVrHNgAAoJEIMGvNKzCweM9BcP/i1Whefb7KftCk/s5K41LckG +dZ4FyoLDwk48rPBLyQ6vhL+gGVGRXaLV6xOIPuMH+QPL7aosbqHGhNY+vSvg5ji PCwwMX5pz2dHJHmclsxU9+5LrvwjlPm9akYvkNq3pv0M+EYoLvib021MLQXhFQAv OVHlksjxs7nrT3BLPSCUWuEGAJUMsYHrfmbl0aVkz4KAOMCPtqQz1FXT3zWnCKvL JYmqAfcC0SRng48GwPKEKeXXP6iAzdDGy8pYwLcAc23tzaIxaaT9uk0l5kmVmxbS iWeCDOlDDZw5ftzY674HxuoXLCXCOgdazyvSKLVO4Vqc2xni6K/sn/OBuCax+74F UH+lfOaytlTC7cMrsfZuVxYhG2GxvB/606XKt6AfLrluCMsCQTnj1CdymfzGzY8r zZDyejIpkP3C6wHPEBes+ISK0GnNm9C35baY8/Jibd3hWeYmznvLshImMimXo8Ti mZZukp+8Ix/AGPJjiT7apwqzqR6yMWctozQiLVBHGo8tbNr1S+CnvEQGdo+5xmpx yH5r7Q9xT/n4CWB1qzwMeSIatrPnX9lVyeNTCICMbrw9J96/7wMUdcvSCaFiIjoc g9Q5CXXWSQ85TDKpeX3EMJgvK3bik/PkX7r6blt81wvlxK1IynOEuASUxaXfQZBi 7z+9JjNQNFwbnRU7PtA2 =FN6H -----END PGP SIGNATURE-----

1 0

numbering and building the CentOS Atomic story
by Karanbir Singh 18 Jul '15

18 Jul '15

Hi, This email is the amalgamation of many conversations that have been running in various venues for the last couple of months. I am posting this here for comments, as we move towards implementing the process ( which has been validated with builds and releases already, the glue components are what we are working on now ). If you are seeing this somewhere other than CentOS-Devel mailing list, I highly encourage you to join that list for comments and to track progress of this effort. ------ At this point were looking at building two sets of atomic machine instances: a) the 4 weekly cycle that fits into the 2/4 model from upstream where Fedora runs a 2 weekly cycle, CentOS Atomic SIG runs the 4weekly downstream from there. The actual process by which CentOS Atomic SIG maps to every alternate Fedora build is a TBH point, but there needs to be some specific map there in the short term. b) the downstream build, where we consume the content delivered via the RHEL Atomic Host content, and deliver a similar functional story. Due to the nature of how the atomic builds work, we wont have an identical build, but we should get to a close enough feature compatibility. any atomic platform is backed by an ostree repo, which is composed of the rpms and any other content needed for the atomic platform to run on. it would be to a atomic host what a yum repo is to a rpm based CentOS Linux install. For (a) above, I propose we run an ostree repo, managed by a centos-release-projectatomic rpm file; the ostree repo for this should be delivered from buildlogs.centos.org, and updated in sync with the rest of the content in the CentOS Linux repos, as well as the atomic sig run repos in the community build service. In order to keep things simple, it might be good enough to run this ostree update on a nightly basis. I propose this nightly build be run from an automated harness at ci.centos.org, which would pull down the last ostree repo, do its updates, commit the changeset, and have a way to push back into the buildlogs.centos.org host the new repo. As an additional downstream trigger, we should then build nightly the images derived from this ostree repo ( this would be the GenericCloud, Vagrant, ISO files etc for the atomic build that maps to the latest commit in that ostree repo). For numbering, I propose we use datestamp with a build id eg: 20150715_01, this maps well to user expectations and also sets suitable confidence in the build. Key requirement to applying updates is that the atomic host needs a reboot to consume the latest ostree propagated changes. So while the updated tree would be available nightly, uses might chose to still apply it nightly, they can control to a fairly good granularity the point in time they wish to run. The atomic sig can then decide which point in time build they wish to promote as their 4wk 'release' image. The ostree repo + images + documentation for that build can then be promoted to a release location. This would then be announced via projectatomic.io I'm also proposing that this build be called the project atomic build. for (b) above, I'm proposing we use the same process as for (a), except for the following major changes : - content in (a) is signed, so while we can build things nightly, we cant promote that content as is. When the Atomic SIG decides on a good tree to promote, the entire buildrun will then be done, against the SIG's accepted metadata, inside the CentOS Build Services ( reimzul ). Resulting in a testable, signed atomic content sit ( ostree repo + updates + images, all signed ). The SIG can then chose to redo, rework, release as needed. - content for (b) will be released to mirror.centos.org (ostree repos), and cloud.centos.org ( backing instance images ). the centos-release-atomic will point at mirror.centos.org's ostree repo. - the update cycle for the ostree repo should be undecided, so we can promote a new update if there are security issues that merit this work, and also when there is a new version of a core component that we might want to push out out of band ( i.e not wait on the next upstream build ) - numbering for (b) should be date driven as well, to look similar as YYYYMMDD_BuildId - The release cycle for this should be documented in an 'Atomic Host' section on wiki.centos.org/Download, and announced via the generic centos announcement channels. - This release would be called the CentOS Atomic Host. regards, ref: * https://lists.fedoraproject.org/pipermail/cloud/2015-March/004976.html * http://www.projectatomic.io/blog/2015/06/centos-atomic-host-rebuild-release… -- Karanbir Singh +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh GnuPG Key : http://www.karan.org/publickey.asc

4 6

Re: [CentOS-devel] numbering and building the CentOS Atomic story
by Colin Walters 16 Jul '15

16 Jul '15

Thanks for posting this! On Wed, Jul 15, 2015, at 06:18 AM, Karanbir Singh wrote: > > For (a) above, I propose we run an ostree repo, managed by a > centos-release-projectatomic rpm file; the ostree repo for this should > be delivered from buildlogs.centos.org, and updated in sync with the > rest of the content in the CentOS Linux repos, as well as the atomic sig > run repos in the community build service. In order to keep things > simple, it might be good enough to run this ostree update on a nightly > basis. With current rpm-ostree it is very inexpensive to just run it constantly (e.g every 5 minutes), or better, edge triggered from push notification from one of the rpm-md input repos being regenerated, and it will quickly do nothing if there's nothing to do. See: https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2015-Jun… > I propose this nightly build be run from an automated harness at > ci.centos.org, which would pull down the last ostree repo, do its > updates, commit the changeset, and have a way to push back into the > buildlogs.centos.org host the new repo. Transitioning it from CI to CD then. There's lots of interesting aspects to this... > As an additional downstream > trigger, we should then build nightly the images derived from this Not sure I understand the rationale of this nightly thing here either. Why not just trigger all images when the ostree commit changes? That can be implemented by a few lines of shell to compare the previous/new commit IDs, and/or rpm-ostree compose tree --touch-if-changed. > ostree repo ( this would be the GenericCloud, Vagrant, ISO files etc for > the atomic build that maps to the latest commit in that ostree repo). > For numbering, I propose we use datestamp with a build id eg: > 20150715_01, this maps well to user expectations and also sets suitable > confidence in the build. That versioning seems fine to me. > Key requirement to applying updates is that the atomic host needs a > reboot to consume the latest ostree propagated changes. So while the > updated tree would be available nightly, uses might chose to still apply > it nightly, they can control to a fairly good granularity the point in > time they wish to run. That sounds like a different branch name? > The atomic sig can then decide which point in time build they wish to > promote as their 4wk 'release' image. The ostree repo + images + > documentation for that build can then be promoted to a release location. > This would then be announced via projectatomic.io > > I'm also proposing that this build be called the project atomic build. > > for (b) above, I'm proposing we use the same process as for (a), except > for the following major changes : > - content in (a) is signed, Is *not* signed? Or is? Note that OSTree actually supports multiple GPG signatures per commit, intended for "binary promotion" workflows. But: > so while we can build things nightly, we > cant promote that content as is. Right, and the SIG build may have things that aren't in the rebuild, so it wouldn't make sense to binary-promote. > - the update cycle for the ostree repo should be undecided, so we can > promote a new update if there are security issues that merit this work, > and also when there is a new version of a core component that we might > want to push out out of band ( i.e not wait on the next upstream build ) I doubt anyone would object if we unconditionally always included the Z stream errata and released the tree commit immediately after the RPM rebuild.

2 1

CI for kubernetes, docker, etcd and other atomic packages
by Jan Chaloupka 15 Jul '15

15 Jul '15

Hi, at the moment kubernetes, docker, etcd and other atomic packages has no CI testing in CentOS. The packages built for CentOS are taken from Fedora's repositories. There is already an effort to create CI for Fedora builds. Currently kubernetes has its own CI job which can be taken as an template for other jobs for docker, etcd, etc. in Fedora. The job among other things consists of a builder, i.e. script that can be broken down into two parts: installation and testing. The installation part takes care of installing, updating and downloading all necessary builds. Testing part runs tests. As the script is universal and differs only in the installation part, other distributions can reuse it. RHEL does that after a simple modification of the installation part. The workflow is following: update kubernetes ci job for Fedora, if it works, update it for RHEL. I would like to continue in the same sense for CentOS. Create a job for centos's kubernetes. The question is where should it run? All jobs (for Fedora and RHEL) are run in redhat's internal Jenkins. For CentOS it would be better to run the job in its own CI environment as this is not the only job. Docker, possible etcd and cadvisor can have its own set of sets that are suitable. So distributing CI effort among all distributions that are involved is natural. Another option is to run all centos jobs related to atomic packages in the internal Jenkins. Why should we bother with running centos's jobs in redhat's environment? I have all jobs at one place so I can update them all at once and they don't get out of sync. It does not take me much time to create another ci job for CentOS once I have created one for Fedora. However, I need a trigger that will trigger the job (I have started a discussion on setting up fedmsg for CentOS that could provide such a trigger [1]). Second, the CI environment is internal so everyone who wants to take a look at run jobs must have RH credentials. If CentOS's CI is used the responsibility for maintaining all jobs and keeping them in sync is completely on CentOS side. Or something between? What about I will first test the new or updated job for CentOS in internal env and then forward all changes to someone responsible for CentOS's CI which would be who? I am not against taking care of this part if I get an access/credentials to CentOS to access your CI. Some jobs in internal RH Ci environment contain private data so I can not go public. What can go public is the builder (script) for Fedora. However I don't see any benefit sharing the script without the entire *.yaml files that create the job. On the other hand if you provide a public repository I can contribute to I will update CentOS's jobs as changes or new jobs appears. Provided your template for a job. [1] http://lists.centos.org/pipermail/centos-devel/2015-June/013415.html Kind Regards Jan

4 4

Feedback on C7 Vagrant Box
by Jeff Sheltren 13 Jul '15

13 Jul '15

Hi, I've started doing some testing on the Virtualbox Vagrant box ( http://cloud.centos.org/centos/7/vagrant/x86_64/images/CentOS-7-Vagrant-150… ) I've hit a couple of issues: 1) The disk size, at about 8 GB, is too small. Could it be bumped to 50-100 GB? Since the on-disk image will grow dynamically this won't take up that much space by default but will make it usable for those that need more storage on the root partition (and I don't want to have to custom mount additional partitions). 2) The VirtualBox Guest Additions aren't included. I'm assuming this is due to a licensing issue and not that they were simply not considered but wanted to double check. Otherwise it's been great, and saves me having to build my own custom box, which is definitely a win. -Jeff

6 15

participate/contribute to SCL
by Rob Mokkink 08 Jul '15

08 Jul '15

Hi, I would like to participate/contribute to SCL SIG. I have RPM building experience, i also contribute to elrepo (bumblebee, bbswitch). How can i get involved? Thanks. Regards, Rob

2 2

Problems with custom centos-atomic-host build
by Andres Toomsalu 08 Jul '15

08 Jul '15

Hi, Im trying to do custom centos-atomic-host build - from unmodified https://github.com/CentOS/sig-atomic-buildscripts configuration source and following instruction in https://developerblog.redhat.com/2015/01/08/creating-custom-atomic-trees-im… blog post series. Im able to produce ostree and installer.iso image - BUT rpm-ostree-toolbox creates ostree tree with missing grub file, resulting in a failing installer. More details about this problem here: https://github.com/projectatomic/rpm-ostree-toolbox/issues/77 Commands used for ostree and installer creation were: --- rpm-ostree-toolbox treecompose -c ~/repos/sig-atomic-buildscripts/config.ini --ostreerepo /srv/rpm-ostree/centos-atomic-host/7/ rpm-ostree-toolbox installer -c /root/repos/sig-atomic-buildscripts/config.ini \ --ostreerepo /srv/rpm-ostree/centos-atomic-host/7/ \ --outputdir /var/www/html/latest/ --overwrite cp ~/repos/sig-atomic-buildscripts/centos-atomic-host-7.ks /var/www/html/ virt-install --name=atomic-iso --memory=1024 --vcpus=1 \ --disk=/var/lib/libvirt/images/test.qcow2,size=5 \ --location /var/www/html/latest/images/installer.iso \ --noautoconsole --accelerate --os-type=linux --os-variant=rhel7 \ --extra-args "ks=http://192.168.122.1/centos-atomic-host-7.ks" --- Another issue is that also imagefactory KVM centos-atomic-host image build fails in anaconda install stage - complaining about unknown missing file - might be the same problem - not sure... Command used for kvm image creation: --- rpm-ostree-toolbox imagefactory \ -c /root/repos/sig-atomic-buildscripts/config.ini -i kvm \ --ostreerepo /srv/rpm-ostree/centos-atomic-host/7/ \ --outputdir /srv/rpm-ostree/centos-atomic-host/7/images --- Error screenshot from imagefactory launched build VM console: http://prntscr.com/7lq38e From the build VM /tmp/anaconda.log: --- 21:40:35,454 INFO anaconda: Creating xfs on /dev/vda1 21:40:36,242 INFO anaconda: executing ostreesetup=<pykickstart.commands.ostreesetup.RHEL7_OSTreeSetup object at 0x7fdd038d3110> 21:40:36,570 ERR anaconda: Failed to pull from repository: Server returned status 404: Not Found --- Last lines from the build VM /tmp/program.log: --- 21:40:36,339 INFO program: Running... ostree --repo=/mnt/sysimage/ostree/repo remote add --set=gpg-verify=false installmedia http://192.168.122.1:47990/ 21:40:36,377 DEBUG program: Return code: 0 --- Any suggestions how to debug it further? Where should I file a bug report? Kind regards, -- <http://www.getpostbox.com>---------------------------------------------- Andres Toomsalu,andres(a)opennodecloud.com <mailto:andres@opennodecloud.com>

3 4

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

devel