On Thu, Dec 18, 2014 at 7:57 AM, Karanbir Singh mail-lists@karan.org wrote:
Over the last few weeks I've been talking with a few projects who are interested in coming onboard to consume the ci.centos.org infra. and based on those conversations, keeping in mind what sort of hardware we have in place and coming up : it looks like were going to need an abstracted api layer between the ci interface and the test hardware itself.
As a bootstrap this means jenkins jobs will need to 'request' a machine instance from this middleware api ( lets call it duffy ), once the machine instance comes up, it will need to login or use a preseeded jenkins agent inside the instance to run the job. once the tests are done, we can optionally also collect the artifacts and then let duffy know ( which can then either recycle, or shutdown or rebuild or whatever ).
I think the commercial/supported jenkins version includes support for spinning up slaves as needed from a cloud - haven't used it myself.
With that in mind, duffy needs to implement atleast the following calls:
- requesting new machine for <user>
- list of allocated machines for <user>
- 'done' with machine for <user>
within the duffy mechanism we need to implement a way so the user gets the same'ish platform everytime ( ie. if they are using the intel machines, they should always get the intel machines ), and also make sure we stress every machine in the cluster.
The second aspect we need to implement here is machine types. We clearly want most of the people to use baremetal and stress the machines as hard as they can ( with the aim that they deliver tests on a platform that looks most like the user machines will, and also so that they can be 'done' really quick ). However, we are also going to need, for some small numbers, Virtual Machines for some of the tests to run on. There are some use cases where the test suites actually expect VMs instead of physical machines, and we would need to cater for those.
Metadata around what type of machines is being requested ( VM or baremetal ) could be something sent to duffy in the request for new machine.
Finally, how i envison this as running is that duffy can just be a thin python script running behind bottle.py or flask, backed by some db to store audit trail etc, and executing the tasks using ansible cookbooks. that way we dont need to reinvent the whole interface access to the SeaMicro machines nor whatever Virtualisation environ we end up using for the VM's.
Have you waded through all of the existing plugins for this sort of thing? I don't think there is an exact match as things are typically tuned to work with either VMs or physical hosts instead of a mix. The 'gearman' plugin might have a reasonable abstraction if you wanted to split out multiple jenkins masters for different but overlapping purposes. And you might be able to glue some steps into the new workflow plugin (set) to spin up and allocate nodes before using them. If you do build something new to manage sets of users and associated nodes I'd love to see it made available.