[Ci-users] Question about plugin installation: build-pipeline

Mon Oct 5 13:56:44 UTC 2015
David Moreau Simard <dms at redhat.com>

Hi Arie,

I gave a bird's eye view of what I planned to do, but I see you're
hungry for details so this is what I have in mind right now for the
short term.

The actual packstack CI jobs will accept a repository hash as a
parameter, this will allow different jobs to test against the same
Right now they are picking up whatever is the latest each time, making
it hard to do multiple tests against the same repo.

As for the jobs:
- "Prepare job": A timed job that will fetch the "current" (master)
repository and do some basic tests with it
-- i.e Configure the repo, makecache/repolist - ensure we don't waste
time creating VMs to install packstack if the repo won't even work.
It's end goal is to retrieve the actual repository hash and pass it on
to downstream jobs.
- The prepare job, if successful, launches the Packstack AIO jobs (one
selinux permissive, one selinux enforcing)
- If the Packstack AIO jobs are successful, they launch their
respective multi-node jobs (one selinux permissive, one selinux
- There is one last job: the promotion job, this one is triggered
based on the status of the multi-node jobs. If the multi node jobs are
triggered and are successful, the repository hash that was tested is
promoted as "passed-ci".

In a pipeline, it looks like this [1]. The general idea is to avoid
running jobs that we will know will fail ahead of time.
Right now we are running a very minimal test with tempest. Once I get
smoke tests to run reliably, I want to bump the tests to smoke and
this generally adds ~30 minutes to the job as per my current
In this regard, I'm juggling with the idea of running:
"prepare" ->
"packstack aio selinux enforcing minimal tests" ->
"packstack multi-node selinux enforcing smoke tests" and ''packstack
multi-node selinux permissive smoke tests" <- promote job

This removes the need for the packstack aio selinux permissive job as
we assume there won't be any problems in this one that we won't see in
the enforcing job.

[1] http://i.imgur.com/BQG7vGK.png

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]

On Sun, Oct 4, 2015 at 4:01 PM, Arie Bregman <abregman at redhat.com> wrote:
> Hi,
> Glad to hear someone working on improving RDO CI =)
> About pipeline plugin - it is known to be very useful when you have
> distinguished steps that depends on each other (e.g Test -> Deploy) but
> installing RDO on single or multiple nodes are quite similar. I also believe
> you will find there is a minor difference in time run between those two
> types of jobs so it can save a lot of time running those in parallel
> (especially when resources are limited).
> perhaps ""Install RDO on a single node & Install RDO on multiple nodes
> (+testing)" -> "Promote the tested set of packages in the 'passed-ci'
> repository" ?
> Cheers,
> Arie
> On Fri, Oct 2, 2015 at 7:23 PM, David Moreau Simard <dms at redhat.com> wrote:
>> Hi there,
>> We're currently working on improving the RDO and RDO Manager CI.
>> As part of those efforts, I'd like to use the build-pipeline plugin
>> [1] to build a clear dependency flow of jobs to stay in the logic of
>> "fail fast" and avoid needless jobs where possible.
>> At a bird's eye view, it would look like this:
>> "Install RDO on a single node and do very minimal testing" ->
>> "Install RDO on multiple nodes with decent/exhaustive testing" ->
>> "Promote the tested set of packages in the 'passed-ci' repository"
>> This means we wouldn't bother doing exhaustive testing if the basic
>> installation or the minimal testing fails and we would only promote
>> the tested set of packages if the exhaustive testing passes.
>> What's great with the pipelines plugin is that the dependencies are
>> very clear and the dashboard it provides is very insightful to see
>> what is passing or what is failing [2].
>> We will eventually be moving the RDO and RDO Manager CI jobs to the
>> ci.centos infrastructure so knowing whether or not we can use this
>> plugin is important. There are other ways around doing this (with
>> another plugin, multi jobs) but it's not as straightforward, IMO.
>> Please let me know what you think,
>> Thanks !
>> [1]: https://wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin
>> [2]:
>> https://wiki.jenkins-ci.org/download/attachments/54723106/bpp1.png?version=2&modificationDate=1340695983000
>> David Moreau Simard
>> Senior Software Engineer | Openstack RDO
>> dmsimard = [irc, github, twitter]
>> _______________________________________________
>> Ci-users mailing list
>> Ci-users at centos.org
>> https://lists.centos.org/mailman/listinfo/ci-users