On 2014-07-03 11:42, Johnny Hughes wrote: > On 07/03/2014 05:19 AM, D.S. Ljungmark wrote: >> Excellent information, I'd love the scripts, and post-weekend sounds >> as >> if it'd fit well with my schedule. >> >> I'll see about taking the time to document steps as well so we might >> get >> a wiki started on how to do this, seems as if there are a few people >> who >> have interest, and at least documenting the basics might be good. >> >> Regards, >> D.S. >> >> >> On 03/07/14 12:14, Gordan Bobic wrote: >>> On 2014-07-03 11:00, D.S. Ljungmark wrote: >>>> Thanks for the head's up on that. >>>> >>>> So, plan of action would be: >>>> * Find / prepare a F19 bootable image. >>> Technically, as Karanbir said, you don't have to run F19 >>> on the build host, just use the F19 respository for mock >>> builds. OTOH, for first pass you may find it a lot faster >>> to install F19 (install _all_ packages), and instead of >>> mock, use just straight rpm to build the first pass. >>> >>> This will save you a tonne of time because the chroot won't >>> have to be built every time (it takes time even if it's >>> tarred and cached rather than yum installed each time). >>> >>> Expect spurious failures if you do that - in EL6 I noticed >>> there are packages that fail to build if other packages >>> that aren't in the dependency list are installed. This >>> is because the package's configure finds the extra >>> packages and tries to build against them, which fails >>> (or worse, produces a broken binary). If you remove the >>> extra package, the build will succeed. >>> >>> But for the first pass it should be OK because you >>> are only going to use what comes out of it to build >>> the second pass. >>> >>> Then you rebuild it all again, just to make sure, >>> and you should be good for an alpha test, and start >>> working on genuine build failures, erroneous arch >>> restrictions, etc. It is this stage that takes >>> hundreds of man-hours. Everything else is mostly CPU >>> time. >>> >>> For building with multiple machines, I use a simple >>> script on all the builders that places a lock file >>> on uncached NFS when a package is picked for build, >>> and if a builder sees there's a lock file there, >>> goes on to the next package in the list. It's >>> trivially simple and works very well. It would be >>> nice to have something that resolves all dependencies >>> for building and tries to build the packages in the >>> dependency tree order, but that's mostly useful for >>> bootstrapping from scratch, and we are cheating by >>> bootstrapping on F19, so it isn't as big a problem. >>> >>>> * Install mock (git or are the packages ok?) >>> See above - you can save a lot of time for the first >>> build pass by not using mock. Install all Fedora >>> packages, and then simply use: >>> >>> rpmbuild --rebuild $package.rpm >>> >>>> * build a mock F19 starter, test compile something traditional >>>> (bash?) >>>> * Duplicate this environment to the various machines >>>> * set up nfs for compile target >>>> * wrap some scripts around pssh to do parallel builds >>>> >>>> -- Am I missing something major here? >>> That's pretty much it. I am happy to share the scripts >>> I use. If I don't post them by the weekend ping me >>> to remind me. I can't get to them right now because my >>> build farm is behind I firewall I don't have a hole on. > > I would be happy to setup mock for you if you want ... or even just put > what I have been using for mock configs here on this list for a test > build. > > I personally would rather produce the RPMs via mock as that prevents > pulling in spurious links for packages because the buildroot is too fat > (ie, in RHEL, package-y.x.z does not link against package-a.b.c because > it is not a BuildRequire and not installed in the mock build root .. > but > if run with the package-a.b.c in the buildroot, the configure process > checks for and links against it. Indeed, I touched upon that in my previous post, but that doesn't really matter too much for the first build pass, as you are going to rebuild everything again anyway. And you can pick off the few build failures arising from too much junk in the build environment before the second pass via mock rebuilds. > All mock does is build a minimum clean build root for each package > where > only the specific requires for building are in the build root so that > each package gets only what it needs to build and builds are more > consistently. Sure, but the time it takes to rm -rf the build root and then untar a cached build root copy is a non-trivial fraction of the build time for a lot of the packages. It is certainly not trivial when you multiply it by around 2,000 for the number of packages you are going to need to build. From experience, anything you can do to get past the first stage build faster is usually a good idea if hardware is limited - and on ARM it usually is. Even on something like the Arndale Octa or the new Chromebook which have 3-4GB of RAM and 8 cores, building takes a while. It's not like throwing a full distro rebuild at a 24-thread 96GB of RAM Xeon monster and being able to expect it to have completed by tomorrow morning.