-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I've been busy on various things (including personal and family related issues) but I had some time to work on the plague build farm. After a discussion with Johnny, KB and Jim, we thought that it would be a good idea to try to focus first on a "self-contained" buildroot, and only then trying to rebuild the 7.1503 SRPMS against that buildroot, and not against F19/F20 (there will be some issues that will need to be solved though)
I got a "self-contained" buildroot, rebuilt from 7.1503 packages, and I started with gcc, then glibc, then kernel (4.0.0-1, for compat reason with aarch64), and each time repodata was regenerated , so glibc has been rebuilt with gcc from 7.1.1503, etc ..
The current c7-buildroot is here : http://armv7.dev.centos.org/repodir/c7-buildroot/
Then I put all the 7.1503 in the queue, and target name being c7-pass-1 , and the built packages are located here : http://armv7.dev.centos.org/repodir/c7-pass-1/
As plague-server required mod_python (deprecated now) we don't have any Web UI people can have a look at. But I created a "quick-and-dirty" script that gathers information from the mysql DB, and outputs that here : - - Packages in the current queue : http://armv7.dev.centos.org/queue.html - - Packages in a "failed" status : http://armv7.dev.centos.org/report.html (with link to the log files)
Those pages are generated every 5 minutes, through a cron job.
Now that we have something that can be used, I'll work on the access for contributors (two names are on my list).
Actually, I'm also using the list of packages in failed status, to requeue those at the end, automatically. A best option would be to do that in a "clever" way, so trying to build a map of BuildRequires: and so which package would need to be built before another one, but my time is limited at the moment ( <hint> welcome ! </hint> )
I'm not expecting that to work for a complete and final tree though. So we'll have to have a look at the build logs, and sometimes we'll probably have to use a specific mock config file for one or two packages (or even a "patch" will be needed too) so the best/easiest/fastest will probably be to do that "locally" and not use plague (even if using the same build nodes)
I had to exclude some packages from the queue, as armv7hl was excluded, but we can probably investigate if that can be built or not ...
I guess people don't want me to spam the list every day with a long list of packages still in the queue :-) , so everybody can have a look at the website.
If you have brilliant ideas, opinions, etc .. feel free to comment and participate.
Sorry if that took more time that what I was expecting ...
Kind Regards - --
Fabian Arrotin The CentOS Project | http://www.centos.org gpg key: 56BEC54E | twitter: @arrfab
Actually, I'm also using the list of packages in failed status, to requeue those at the end, automatically. A best option would be to do that in a "clever" way, so trying to build a map of BuildRequires: and so which package would need to be built before another one, but my time is limited at the moment ( <hint> welcome ! </hint> )
I do not know if there is an easy way to do this but I was just playing with the code for yum-builddep and it seems that with better understanding of yum internals we can use the code in it to generate an ordered list of packages to compile.
yum-builddep installs packages required to build a particular src.rpm on the "current" system. We can't use yum-builddep directly because we do not have an installed CentOS ARM to run yum-builddep on but we can use its code on an x86_64 to find dependencies for packages we want to compile.
I put a simple print statement in the get_build_deps function and was able to get a dependency list for a package. For input 'bash', I got the list as ------------------ texinfo ncurses-devel gettext bison autoconf ------------------ We need to do this recursively until we reach a point where the only dependencies are ones satisfied by our buildroot. eg. libtool Then we feed this list to plague and have it compile packages in the order we desire. My understanding of plague is limited at this point but I do foresee a problem. If we do generate a list of packages in the right order, how do we make sure plague doesn't mess up the order in the process of distributing the compile jobs to plague-builders? Could we use just one plague-builder instance and use the processing power of remaining nodes with DistCC?
I am new to this. So if there is any flaw in my approach please correct me.
Regards Mandar Joshi
On 05/24/2015 03:26 PM, Mandar Joshi wrote:
Actually, I'm also using the list of packages in failed status, to requeue those at the end, automatically. A best option would be to do that in a "clever" way, so trying to build a map of BuildRequires: and so which package would need to be built before another one, but my time is limited at the moment ( <hint> welcome ! </hint> )
I do not know if there is an easy way to do this but I was just playing with the code for yum-builddep and it seems that with better understanding of yum internals we can use the code in it to generate an ordered list of packages to compile.
yum-builddep installs packages required to build a particular src.rpm on the "current" system. We can't use yum-builddep directly because we do not have an installed CentOS ARM to run yum-builddep on but we can use its code on an x86_64 to find dependencies for packages we want to compile.
I put a simple print statement in the get_build_deps function and was able to get a dependency list for a package. For input 'bash', I got the list as
texinfo ncurses-devel gettext bison autoconf
We need to do this recursively until we reach a point where the only dependencies are ones satisfied by our buildroot. eg. libtool Then we feed this list to plague and have it compile packages in the order we desire. My understanding of plague is limited at this point but I do foresee a problem. If we do generate a list of packages in the right order, how do we make sure plague doesn't mess up the order in the process of distributing the compile jobs to plague-builders? Could we use just one plague-builder instance and use the processing power of remaining nodes with DistCC?
I am new to this. So if there is any flaw in my approach please correct me.
Keep in mind that, besides the builddeps, you also need the regular install deps.
To follow your example, you need to build autoconf before building bash. At this point you'll need all the build requirements for autoconf AND all the install requirements for autoconf.
Next to that, there will be *many* circular dependencies, which will need to be resolved by hand. Sometimes one of the items in such a circle has a 'bootstrap' option in the spec file, but more often you'll need to find a good replacement rpm from another repo (or a noarch from the regular C7)
I've been down this road once and it's a lot of work to resolve all the issues.
My advise would be to build all against F19, so you'll have a good starting point and then do it again. Especially, because you seem to have enough CPUpower to not mind building things more than once. (I have not tried this myself)
autoconf is, by the way, a nice example. This was one of the packages I wanted to build first, because it was needed so often. It ended up being build as one of the last, because of it's many dependencies. IIRC if you follow the tree, you'll need kde, openoffice, tex and a whole lot of other packages.
Good luck,
Jacco
On 24/05/15 14:26, Mandar Joshi wrote:
Actually, I'm also using the list of packages in failed status, to requeue those at the end, automatically. A best option would be to do that in a "clever" way, so trying to build a map of BuildRequires: and so which package would need to be built before another one, but my time is limited at the moment ( <hint> welcome ! </hint> )
I do not know if there is an easy way to do this but I was just playing with the code for yum-builddep and it seems that with better understanding of yum internals we can use the code in it to generate an ordered list of packages to compile.
yum-builddep installs packages required to build a particular src.rpm on the "current" system. We can't use yum-builddep directly because we do not have an installed CentOS ARM to run yum-builddep on but we can use its code on an x86_64 to find dependencies for packages we want to compile.
I put a simple print statement in the get_build_deps function and was able to get a dependency list for a package. For input 'bash', I got the list as
texinfo ncurses-devel gettext bison autoconf
We need to do this recursively until we reach a point where the only dependencies are ones satisfied by our buildroot. eg. libtool Then we feed this list to plague and have it compile packages in the order we desire. My understanding of plague is limited at this point but I do foresee a problem. If we do generate a list of packages in the right order, how do we make sure plague doesn't mess up the order in the process of distributing the compile jobs to plague-builders? Could we use just one plague-builder instance and use the processing power of remaining nodes with DistCC?
I am new to this. So if there is any flaw in my approach please correct me.
As the first step, you need to get the packages from the @buildsys-build group built, along with all of their dependencies. This is the most difficult part.
As far as the dependencies go, this problem is neither yum dependant nor architecture specific. You could simply rpm -ivh all the .src.rpm files and for each .spec file look for BuildRequires lines. You could then create a directed graph of dependencies and write an algorithm to solve the traveling salesman problem on it, to work out the optimal build order.
There are additional gotchas - some packages have cut-down build options with fewer dependencies. This can be invoked using the --with-bootstrap build option, but I would not want to bet money that this is consistent across all of the packages, so some manual intervention may be required. You would need to account for this in the solver, and your .spec file parser would need to differentiate between normal and bootstrap options in terms of dependencies and treat the bootstrap and final packages separately during the solving process.
If you were to write a tool to do this and spit out something like a Makefile containing the full dependency list, that would be quite amazingly awesome, since that would enable simply running "make" and having the entire build proceed in the optimal order with no wasted time and resources on builds that fail due to missing dependencies.
Extra points for also writing a mock wrapper taking account of a supplied list of builders (even if it is just a list of mock config files, each pointing at a different build chroot), so that the resulting program could be used with a mock wrapper using, "make -j" and having it build everything everything possible in parallel as far as the dependencies allow.
Gordan
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 24/05/15 15:26, Mandar Joshi wrote:
Actually, I'm also using the list of packages in failed status, to requeue those at the end, automatically. A best option would be to do that in a "clever" way, so trying to build a map of BuildRequires: and so which package would need to be built before another one, but my time is limited at the moment ( <hint> welcome ! </hint> )
I do not know if there is an easy way to do this but I was just playing with the code for yum-builddep and it seems that with better understanding of yum internals we can use the code in it to generate an ordered list of packages to compile.
yum-builddep installs packages required to build a particular src.rpm on the "current" system. We can't use yum-builddep directly because we do not have an installed CentOS ARM to run yum-builddep on but we can use its code on an x86_64 to find dependencies for packages we want to compile.
I put a simple print statement in the get_build_deps function and was able to get a dependency list for a package. For input 'bash', I got the list as ------------------ texinfo ncurses-devel gettext bison autoconf ------------------ We need to do this recursively until we reach a point where the only dependencies are ones satisfied by our buildroot. eg. libtool Then we feed this list to plague and have it compile packages in the order we desire. My understanding of plague is limited at this point but I do foresee a problem. If we do generate a list of packages in the right order, how do we make sure plague doesn't mess up the order in the process of distributing the compile jobs to plague-builders? Could we use just one plague-builder instance and use the processing power of remaining nodes with DistCC?
Well, plague is nothing more than a distributed system on top of mock. So one thing to keep in mind is that plague will "just" submit the SRPMs in the order you submit/upload those. But then, it doesn't "chain" those, so if you submit srpm A,B,C, and that you have three builders, each one will get a job .. but if package B needs A as BuildRequires, it will fail, and also when a package will need a package still compiling/building. That's the reason why I just used mock directly on one of the plague builders to init the c7-buildroot. I used yum-builddep to do what you described too, but , as Jacco pointed out too, yum-builddep just shows the BuildRequires, but then you also have to build the packages that are themselves listed a Requires: for those packages in the buildroot. I got a "self-contained" buildroot last wednesday (as explained in the first mail in that thread) and started to submit all SRPMS then in plague. Problem is that EL7 (same for previous versions) doesn't build on itself cleanly, and has some circular dependencies.
Just as an example, for bash, as you said, you need bison, which then needs perl-data-dumper .. but the issue with perl-data-dumper is that itself it needs perl-test-simple, which itself needs perl-data-dumper (inception ! :-) ) So in that case, I had to bootstrap from somewhere and I built perl-data-dumper (7.1406 srpms) in a "staging" repo (against f19) to have a starting point, then rebuild it with plague against that buildroot including that specific package (see http://armv7.dev.centos.org/rpmbuild/c7-pass-1/5787-perl-Data-Dumper-2.145-3...)
It would be good to see if we can have all the mandatory packages for a minimal install, and then work on a RootFS (isn't that what you were interested in ? ) and see if we can produce various images for some armv7hl boards (and then the fun begins, with probably all the variants, and different way of booting those)
- --
Fabian Arrotin The CentOS Project | http://www.centos.org gpg key: 56BEC54E | twitter: @arrfab
-
Fabian Arrotin -
Gordan Bobic -
Jacco Ligthart -
Mandar Joshi