On 24/05/15 14:26, Mandar Joshi wrote: >> Actually, I'm also using the list of packages in failed status, to >> requeue those at the end, automatically. A best option would be to do >> that in a "clever" way, so trying to build a map of BuildRequires: and >> so which package would need to be built before another one, but my >> time is limited at the moment ( <hint> welcome ! </hint> ) >> > > I do not know if there is an easy way to do this but I was just > playing with the code for yum-builddep and it seems that with better > understanding of yum internals we can use the code in it to generate > an ordered list of packages to compile. > > yum-builddep installs packages required to build a particular src.rpm > on the "current" system. We can't use yum-builddep directly because we > do not have an installed CentOS ARM to run yum-builddep on but we can > use its code on an x86_64 to find dependencies for packages we want to > compile. > > I put a simple print statement in the get_build_deps function and was > able to get a dependency list for a package. For input 'bash', I got > the list as > ------------------ > texinfo > ncurses-devel > gettext > bison > autoconf > ------------------ > We need to do this recursively until we reach a point where the only > dependencies are ones satisfied by our buildroot. eg. libtool > Then we feed this list to plague and have it compile packages in the > order we desire. My understanding of plague is limited at this point > but I do foresee a problem. If we do generate a list of packages in > the right order, how do we make sure plague doesn't mess up the order > in the process of distributing the compile jobs to plague-builders? > Could we use just one plague-builder instance and use the processing > power of remaining nodes with DistCC? > > I am new to this. So if there is any flaw in my approach please correct me. As the first step, you need to get the packages from the @buildsys-build group built, along with all of their dependencies. This is the most difficult part. As far as the dependencies go, this problem is neither yum dependant nor architecture specific. You could simply rpm -ivh all the .src.rpm files and for each .spec file look for BuildRequires lines. You could then create a directed graph of dependencies and write an algorithm to solve the traveling salesman problem on it, to work out the optimal build order. There are additional gotchas - some packages have cut-down build options with fewer dependencies. This can be invoked using the --with-bootstrap build option, but I would not want to bet money that this is consistent across all of the packages, so some manual intervention may be required. You would need to account for this in the solver, and your .spec file parser would need to differentiate between normal and bootstrap options in terms of dependencies and treat the bootstrap and final packages separately during the solving process. If you were to write a tool to do this and spit out something like a Makefile containing the full dependency list, that would be quite amazingly awesome, since that would enable simply running "make" and having the entire build proceed in the optimal order with no wasted time and resources on builds that fail due to missing dependencies. Extra points for also writing a mock wrapper taking account of a supplied list of builders (even if it is just a list of mock config files, each pointing at a different build chroot), so that the resulting program could be used with a mock wrapper using, "make -j" and having it build everything everything possible in parallel as far as the dependencies allow. Gordan