[Arm-dev] CentOS 7 ARMv7 build status

[Gordan Bobic]

On 24/05/15 14:26, Mandar Joshi wrote:
>> Actually, I'm also using the list of packages in failed status, to
>> requeue those at the end, automatically. A best option would be to do
>> that in a "clever" way, so trying to build a map of BuildRequires: and
>> so which package would need to be built before another one, but my
>> time is limited at the moment ( <hint> welcome ! </hint> )
>>
>
> I do not know if there is an easy way to do this but I was just
> playing with the code for yum-builddep and it seems that with better
> understanding of yum internals we can use the code in it to generate
> an ordered list of packages to compile.
>
> yum-builddep installs packages required to build a particular src.rpm
> on the "current" system. We can't use yum-builddep directly because we
> do not have an installed CentOS ARM to run yum-builddep on but we can
> use its code on an x86_64 to find dependencies for packages we want to
> compile.
>
> I put a simple print statement in the get_build_deps function and was
> able to get a dependency list for a package. For input 'bash', I got
> the list as
> ------------------
> texinfo
> ncurses-devel
> gettext
> bison
> autoconf
> ------------------
> We need to do this recursively until we reach a point where the only
> dependencies are ones satisfied by our buildroot. eg. libtool
> Then we feed this list to plague and have it compile packages in the
> order we desire. My understanding of plague is limited at this point
> but I do foresee a problem. If we do generate a list of packages in
> the right order, how do we make sure plague doesn't mess up the order
> in the process of distributing the compile jobs to plague-builders?
> Could we use just one plague-builder instance and use the processing
> power of remaining nodes with DistCC?
>
> I am new to this. So if there is any flaw in my approach please correct me.

As the first step, you need to get the packages from the @buildsys-build 
group built, along with all of their dependencies. This is the most 
difficult part.

As far as the dependencies go, this problem is neither yum dependant nor 
architecture specific. You could simply rpm -ivh all the .src.rpm files 
and for each .spec file look for BuildRequires lines. You could then 
create a directed graph of dependencies and write an algorithm to solve 
the traveling salesman problem on it, to work out the optimal build order.

There are additional gotchas - some packages have cut-down build options 
with fewer dependencies. This can be invoked using the --with-bootstrap 
build option, but I would not want to bet money that this is consistent 
across all of the packages, so some manual intervention may be required. 
You would need to account for this in the solver, and your .spec file 
parser would need to differentiate between normal and bootstrap options 
in terms of dependencies and treat the bootstrap and final packages 
separately during the solving process.

If you were to write a tool to do this and spit out something like a 
Makefile containing the full dependency list, that would be quite 
amazingly awesome, since that would enable simply running "make" and 
having the entire build proceed in the optimal order with no wasted time 
and resources on builds that fail due to missing dependencies.

Extra points for also writing a mock wrapper taking account of a 
supplied list of builders (even if it is just a list of mock config 
files, each pointing at a different build chroot), so that the resulting 
program could be used with a mock wrapper using, "make -j" and having it 
build everything everything possible in parallel as far as the 
dependencies allow.

Gordan