hi, i write this mail after more people ask me in private. and since the centos core devel team not share any info about it's build system, status, there modifications, doesn't have any public revision system and not really accept any help or advice for outside. but i still believe these are useful info for many others so i try to summarize our rhel-6 rebuild experience.
first question is the build host and base repo selection. it's advice to choose both the build system and the base repo as close as to rhel-6 as possible. unfortunately even with official rhel-6 isos you can't get the optional packages in binary. so imho the best base system and base repo is a union of rhel-6 rc2 and rhel-6 beta. both are available freely in binary but only the beta include the optional packages in binary from. after you finish you can rebuild packages on itself again.
another important note that most update packages (there're a few dozens) have higher buildreq packages then in the base repo, but these buildreqs are not versioned in the specs. so you've to add the updates pacakges to the build system's base packages otherwise you'll get build errors.
i recommend mock-1.1.7 for building. unfortunately at rh it's not required to be able to build src.rpm is mock, they use only rpmbuild (!) which of course cause a few dozen of packages won't build:-( imho it'd be useful something like this for rhel too: http://fedoraproject.org/wiki/FTBFS
so let we see what's are the problematic packages (fortunately most of the packages build clean). all of these packages bellow has a bugzilla entry and most of them already contains the solutions. so just go and search for it (i'd not like to link all bz entry:-()
- virt-top (missing ocaml-camomile-data) - spice-client (missing pixman-devel) these two packages can't be rebuild on rhel-6 since they has such buildreq which is not shipped in rhel-6 (simple rh forget about it:-) so you can either add to the base repo these packages from epel or from a rebuild from fedora or simple drop them.
- bash - zsh can't be rebuild in mock-1.x since a mock tty bug. in this case imho the easiest solution: mock <params> --rebuild bash... ctrl-c during build, then mock <params> --shell rpmbuild --rebuild /builddir/build/originals/bash...
- nss can't be rebuild in mock since during %check try to connect to the localhost to test ssl, but it's not working in mock. i suggest the above solution plus ctrl-c during the rpmbuild when the build hang at the ssl connection test and after the you'll got a successful build:-)
- kernel unfortunately kernel spec is not updated to the new noarch subpackage style. which means you've to build the arch and noarch packages separately (as in rhel-5). but the new spec file even worst then the rhel-5 was so eg: a simple: mock <params> --target "noarch,i686" --rebuild kernel... no longer works, but mock <params> --target "noarch" --rebuild kernel... mock <params> --target "i686" --rebuild kernel... works:-) imho the best would be to update the kernel spec to the new noarch style subpackage, but afais rh so conservative about kernel spec it won't be happened in the rhel-6.x series:-(
- opal has a wrong rh patch which cause ekiga not compile, so you've to fix opal first.
- java-1.6.0-openjdk-1.6.0.0-1.21.b17.el6 can't be build, unfortunately bz with fixes for this package are not public, but there's an update which can be build (may be we'd have to leave this package out).
and the remaining packages has different problems in the spec file, but all has (mostly a simple) solution in bz, but doesn't have update package: - guile - openmpi - foomatic - hivex - mod_auth_kerb - gnome-doc-utils - pilot-link - linuxdoc-tools - perl-Sys-Virt - perl-Test-MockObject - perl-Test-Memory-Cycle - perl-GSSAPI - perl-Makefile-Parser - perl-CGI-Session - perl-IPC-Run3 - python-lxml - python-sqlalchemy - rome - epydoc - kdepim-runtime
i hope it can help for a few poeple.
anyway happy xmas!
On Fri, 24 Dec 2010, Farkas Levente wrote:
i recommend mock-1.1.7 for building. unfortunately at rh it's not required to be able to build src.rpm is mock, they use only rpmbuild (!) which of course cause a few dozen of packages won't build:-( imho it'd be useful something like this for rhel too: http://fedoraproject.org/wiki/FTBFS
I don't know where you get your info but everything is built in mock.
-sv
On 12/24/2010 04:30 PM, Seth Vidal wrote:
On Fri, 24 Dec 2010, Farkas Levente wrote:
i recommend mock-1.1.7 for building. unfortunately at rh it's not required to be able to build src.rpm is mock, they use only rpmbuild (!) which of course cause a few dozen of packages won't build:-( imho it'd be useful something like this for rhel too: http://fedoraproject.org/wiki/FTBFS
I don't know where you get your info but everything is built in mock.
see comment 4: https://bugzilla.redhat.com/show_bug.cgi?id=613392#c4
anyway if thwy would use mock then can't be so much problems and if rh try to rebuild the it's release on itself FTBFS (i assume they've got enough cpu cycle:-) then no package can be missing from the release.
On Fri, Dec 24, 2010 at 10:38 AM, Farkas Levente lfarkas@lfarkas.org wrote:
On 12/24/2010 04:30 PM, Seth Vidal wrote:
On Fri, 24 Dec 2010, Farkas Levente wrote:
i recommend mock-1.1.7 for building. unfortunately at rh it's not required to be able to build src.rpm is mock, they use only rpmbuild (!) which of course cause a few dozen of packages won't build:-( imho it'd be useful something like this for rhel too: http://fedoraproject.org/wiki/FTBFS
I don't know where you get your info but everything is built in mock.
see comment 4: https://bugzilla.redhat.com/show_bug.cgi?id=613392#c4
anyway if thwy would use mock then can't be so much problems and if rh try to rebuild the it's release on itself FTBFS (i assume they've got enough cpu cycle:-) then no package can be missing from the release.
There is nothing that says all BuildRequires packages have to be shipped in the release. It's only important to them that they can build RHEL, not anyone else.
josh
On Fri, 24 Dec 2010, Josh Boyer wrote:
enough cpu cycle:-) then no package can be missing from the release.
There is nothing that says all BuildRequires packages have to be shipped in the release. It's only important to them that they can build RHEL, not anyone else.
True enough -- closed and complete 'self-hosting' has never been a goal upstream in their enterprise product; I understand the wish of those here (and even more strongly in EPEL which this piece was crossposted to as well ...) to have it, however
include my stock IAAL disclaimer, and also not sent from a centos email address
For those components of the upstream RHEL sources that are GPLv2 licensed (not all of them, by a growing margin) and to the people to whom binary content is distributed, this is possibly not the case
GPLv2, section 3 a) Accompany it with the complete corresponding machine-readable source code
[later un-numbered para portion] ... For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable
Assuming, arguendo that there are 'tweaks' from Release Engineering, or such to make mock stand up and dance, on the offending 'make check' absence of PTY [Clark William's comment on one], or such [or a need to manually 'walk a build through' outside of mock -- I make no such representation, and my approach on distribution buildsystems is not CentOS' present one], ...
... the build system deviations from what is stock 'mock' would appear to constitute either part of the 'complete source code' to attain a binary, or alternatively part of 'the scripts used to control compilation' ...
and be part of the releasables in proper and a somewhat narrow set of cases
* shrug *
and GPLv3 does not have such an argument chain available in it that I can see presently
The bugs referenced in this thread here and on the EPEL mailing list, seem to refer to Artistic licensed code, however
As to builders, I looked, and upstream released its v 6 product beta in late July, and the gold release drop in early November. This new major release, with the changes I've noted before on one of the mailing lists on the difficulty of doing a yum migration from 5 to 6 should be taken to confirm that it is indeed challenging to solve it all
I've privately solved subsets of the upstream's 6 sources, not for CentOS but for other purposes, and it can be tricky ;) I guess it is time to write my annual buildsystems piece. If bugs in this regard are filed in the CentOS tracker, or in the upstream tracker on such problems, and you do no see me 'cc into them, please feel free to send a private email to me at herrold owlriver com and I'll look in on them
-- Russ herrold
Farkas Levente wrote:
hi, i write this mail after more people ask me in private. and since the centos core devel team not share any info about it's build system, status, there modifications, doesn't have any public revision system and not really accept any help or advice for outside. but i still believe these are useful info for many others so i try to summarize our rhel-6 rebuild experience.
Thanks for the summary of your experience. In my case I created a build environment much as you laid out but I am still doing everything as rpmbuild --rebuild rather than a Mock environment. I ended up with 36 packages not building and did have to import a couple of fedora packages to get it to that number. In comparing my list to yours, you have solutions to 11 of those on my list. I will be attempting those when I have the time. My question is this. Are these the only packages you were unable to rebuild, or do you still have some left to resolve? I noticed a few of mine failed with unpackaged elements so I assume those may be resolved with a change in build options. I used to believe that if I just found the right environment all of RedHat's sources would rebuild. I now have found out that, this belief was very naive.
Thanks again Hubert
On Mon, Dec 27, 2010 at 08:03, Hubert Bahr hab@hbahr.org wrote:
Farkas Levente wrote:
hi, i write this mail after more people ask me in private. and since the centos core devel team not share any info about it's build system, status, there modifications, doesn't have any public revision system and not really accept any help or advice for outside. but i still believe these are useful info for many others so i try to summarize our rhel-6 rebuild experience.
Thanks for the summary of your experience. In my case I created a build environment much as you laid out but I am still doing everything as rpmbuild --rebuild rather than a Mock environment. I ended up with 36 packages not building and did have to import a couple of fedora packages to get it to that number. In comparing my list to yours, you have solutions to 11 of those on my list. I will be attempting those when I have the time. My question is this. Are these the only packages you were unable to rebuild, or do you still have some left to resolve? I noticed a few of mine failed with unpackaged elements so I assume those may be resolved with a change in build options. I used to believe that if I just found the right environment all of RedHat's sources would rebuild. I now have found out that, this belief was very naive.
everything else build, except that i've to find the reason why yum wouldn't like to install 32 bit packages into a 64bit system in mock (but this is bot an rh problem).
Hello Farkas,
On Mon, 2010-12-27 at 22:01 +0100, Farkas Levente wrote:
everything else build, except that i've to find the reason why yum wouldn't like to install 32 bit packages into a 64bit system in mock (but this is bot an rh problem).
Perhaps you are referring to some packages that have conflicting documentation files for the i686 and x86_64 versions? That is a problem with doxygen creating slightly different doc files for both versions.
I couldn't tell you how to fix that for mock, but on a plain build system you can safely force install those rpms (after you verified it's only the doc files that are clashing of course).
Regards, Leonard.
Farkas Levente wrote:
hi, i write this mail after more people ask me in private. and since the centos core devel team not share any info about it's build system, status, there modifications, doesn't have any public revision system and not really accept any help or advice for outside. but i still believe these are useful info for many others so i try to summarize our rhel-6 rebuild experience.
- nss
can't be rebuild in mock since during %check try to connect to the localhost to test ssl, but it's not working in mock. i suggest the above solution plus ctrl-c during the rpmbuild when the build hang at the ssl connection test and after the you'll got a successful build:-)
I tried this with both versions of nss currently out and do not get the hang on the ssl connection. I get a report of 5 errors and then rebuild ends since errors are greater than 0. I do not understand the significance of 5 errors out of 5000+tests.
- opal
has a wrong rh patch which cause ekiga not compile, so you've to fix opal first.
- epydoc
My interpretation of the above two is not that the don't build, but they build with an error that cause other packages not to build. Is it your intent that we use these patched packages to replace the defective ones in our repository and then build ekiga or python-nss.
i hope it can help for a few poeple.
anyway happy xmas!
Thanks for your info I have now reduced my unbuilt packages down to 10. I haven't used all your patches yet so I expect to reduce them still further. Hubert
On Fri, Dec 31, 2010 at 10:58 PM, Hubert Bahr hab@hbahr.org wrote:
Farkas Levente wrote:
- opal
has a wrong rh patch which cause ekiga not compile, so you've to fix opal first.
My interpretation of the above two is not that the don't build, but they build with an error that cause other packages not to build. Is it your intent that we use these patched packages to replace the defective ones in our repository and then build ekiga or python-nss.
This is the bugzilla Levente filed:
https://bugzilla.redhat.com/show_bug.cgi?id=661769
Akemi
Farkas Levente wrote:
hi, i write this mail after more people ask me in private. and since the centos core devel team not share any info about it's build system, status, there modifications, doesn't have any public revision system and not really accept any help or advice for outside. but i still believe these are useful info for many others so i try to summarize our rhel-6 rebuild experience.
- nss
can't be rebuild in mock since during %check try to connect to the localhost to test ssl, but it's not working in mock. i suggest the above solution plus ctrl-c during the rpmbuild when the build hang at the ssl connection test and after the you'll got a successful build:-)
In an earlier reply I indicated that instead of a hang the build indicated 5 errors. I changed the spec file to allow up to 5 errors and it completed the build. so now I am down to one unbuilt src.rpm
and the remaining packages has different problems in the spec file, but all has (mostly a simple) solution in bz, but doesn't have update package:
- linuxdoc-tools
This is it. I am afraid I am extremely rusty on spec files and while I tried to follow the bz instructions, I don't know whether I improved things or made them worse. While I managed to get rid of the not found errors, I ended up with a long list of unpackaged files. Could I get a copy of your spec file for linuxdoc-tools or a diff/patch file from the distribution spec.
Thanks Hubert
Hello Hubert,
On Mon, 2011-01-03 at 01:33 -0600, Hubert Bahr wrote:
I changed the spec file to allow up to 5 errors and it completed the build.
That sounds ugly. If you do this kind of patching you should patch out the specific test failures, not set a number of random tests that are allowed to fail. Just a thought.
Regards, Leonard.
Leonard den Ottolander wrote:
Hello Hubert,
On Mon, 2011-01-03 at 01:33 -0600, Hubert Bahr wrote:
I changed the spec file to allow up to 5 errors and it completed the build.
That sounds ugly. If you do this kind of patching you should patch out the specific test failures, not set a number of random tests that are allowed to fail. Just a thought.
All a matter of perspective. This is one of over 2000 source packages to build binary's and it passes 99.9% of the test cases, and this 5 character change to a specfile allowed it to build. The previous alternative was to kill it during a test and then completing the build. The other one was to skip the testing in total. There are 433 bugs filed against nss. Some were just expired certificates causing problems. I chose not worry about it unless I run into further specific problems that point me back to nss. In this case I just wanted to have a baseline should I have need to modify nss source (unlikely unless it is for branding purposes). If I make an error that increases the count above current value, I would still be alerted. My intuition says it is probably a problem with the test case, not a problem with the source or build technique. I would like to hear from anybody who has built rhel-6 nss from RedHat's source rpms, on their results and what technique they applied.
Hubert
Regards, Leonard.
On 01/03/2011 08:33 AM, Hubert Bahr wrote:
- linuxdoc-tools
This is it. I am afraid I am extremely rusty on spec files and while I tried to follow the bz instructions, I don't know whether I improved things or made them worse. While I managed to get rid of the not found errors, I ended up with a long list of unpackaged files. Could I get a copy of your spec file for linuxdoc-tools or a diff/patch file from the distribution spec.
Farkas Levente wrote:
On 01/03/2011 08:33 AM, Hubert Bahr wrote:
Thank you for the Spec file. I was a lot closer than I thought I was. It solved my problem. I believe I have now built all of Red Hat's source rpms. I will now attempt to scrip everything, put to-geather a repository based on these builds and rebuild everything again just to verify I have remembered everything. In my case I use rpmbuild rebuild for about 99% of the build and mock for the remainder. This speeds up the process signifantly on my cluster.
Thanks again Hubert
-
Akemi Yagi -
Farkas Levente -
Hubert Bahr -
Josh Boyer -
Leonard den Ottolander -
R P Herrold -
Seth Vidal