[CentOS-devel] deliverable infra for the ci.centos.org effort

On Thu, Dec 18, 2014 at 7:57 AM, Karanbir Singh <mail-lists at karan.org> wrote:
>
> Over the last few weeks I've been talking with a few projects who are
> interested in coming onboard to consume the ci.centos.org infra. and
> based on those conversations, keeping in mind what sort of hardware we
> have in place and coming up : it looks like were going to need an
> abstracted api layer between the ci interface and the test hardware itself.
>
> As a bootstrap this means jenkins jobs will need to 'request' a machine
> instance from this middleware api ( lets call it duffy ), once the
> machine instance comes up, it will need to login or use a preseeded
> jenkins agent inside the instance to run the job. once the tests are
> done, we can optionally also collect the artifacts and then let duffy
> know ( which can then either recycle, or shutdown or rebuild or whatever ).

I think the commercial/supported jenkins version includes support for
spinning up slaves as needed from a cloud  - haven't used it myself.

> With that in mind, duffy needs to implement atleast the following calls:
> + requesting new machine for <user>
> + list of allocated machines for <user>
> + 'done' with machine for <user>
>
> within the duffy mechanism we need to implement a way so the user gets
> the same'ish platform everytime ( ie. if they are using the intel
> machines, they should always get the intel machines ), and also make
> sure we stress every machine in the cluster.
>
> The second aspect we need to implement here is machine types. We clearly
> want most of the people to use baremetal and stress the machines as hard
> as they can ( with the aim that they deliver tests on a platform that
> looks most like the user machines will, and also so that they can be
> 'done' really quick ). However, we are also going to need, for some
> small numbers, Virtual Machines for some of the tests to run on. There
> are some use cases where the test suites actually expect VMs instead of
> physical machines, and we would need to cater for those.
>
> Metadata around what type of machines is being requested ( VM or
> baremetal ) could be something sent to duffy in the request for new machine.
>
> Finally, how i envison this as running is that duffy can just be a thin
> python script running behind bottle.py or flask, backed by some db to
> store audit trail etc, and executing the tasks using ansible cookbooks.
> that way we dont need to reinvent the whole interface access to the
> SeaMicro machines nor whatever Virtualisation environ we end up using
> for the VM's.

Have you waded through all of the existing plugins for this sort of
thing?  I don't think there is an exact match as things are typically
tuned to work with either VMs or physical hosts instead of a mix.
The 'gearman' plugin might have a reasonable abstraction if you wanted
to split out multiple jenkins masters for different but overlapping
purposes.   And you might be able to glue some steps into the new
workflow plugin (set) to spin up and allocate nodes before using them.
   If you do build something new to manage sets of users and
associated nodes I'd love to see it made available.

-- 
   Les Mikesell
     lesmikesell at gmail.com