On 03/07/14 12:52, Gordan Bobic wrote: > On 2014-07-03 11:42, Johnny Hughes wrote: >> On 07/03/2014 05:19 AM, D.S. Ljungmark wrote: >>> Excellent information, I'd love the scripts, and post-weekend sounds >>> as >>> if it'd fit well with my schedule. >>> >>> I'll see about taking the time to document steps as well so we might >>> get >>> a wiki started on how to do this, seems as if there are a few people >>> who >>> have interest, and at least documenting the basics might be good. >>> >>> Regards, >>> D.S. >>> >>> >>> On 03/07/14 12:14, Gordan Bobic wrote: >>>> On 2014-07-03 11:00, D.S. Ljungmark wrote: >>>>> Thanks for the head's up on that. >>>>> >>>>> So, plan of action would be: >>>>> * Find / prepare a F19 bootable image. >>>> Technically, as Karanbir said, you don't have to run F19 >>>> on the build host, just use the F19 respository for mock >>>> builds. OTOH, for first pass you may find it a lot faster >>>> to install F19 (install _all_ packages), and instead of >>>> mock, use just straight rpm to build the first pass. >>>> >>>> This will save you a tonne of time because the chroot won't >>>> have to be built every time (it takes time even if it's >>>> tarred and cached rather than yum installed each time). >>>> >>>> Expect spurious failures if you do that - in EL6 I noticed >>>> there are packages that fail to build if other packages >>>> that aren't in the dependency list are installed. This >>>> is because the package's configure finds the extra >>>> packages and tries to build against them, which fails >>>> (or worse, produces a broken binary). If you remove the >>>> extra package, the build will succeed. >>>> >>>> But for the first pass it should be OK because you >>>> are only going to use what comes out of it to build >>>> the second pass. >>>> >>>> Then you rebuild it all again, just to make sure, >>>> and you should be good for an alpha test, and start >>>> working on genuine build failures, erroneous arch >>>> restrictions, etc. It is this stage that takes >>>> hundreds of man-hours. Everything else is mostly CPU >>>> time. >>>> >>>> For building with multiple machines, I use a simple >>>> script on all the builders that places a lock file >>>> on uncached NFS when a package is picked for build, >>>> and if a builder sees there's a lock file there, >>>> goes on to the next package in the list. It's >>>> trivially simple and works very well. It would be >>>> nice to have something that resolves all dependencies >>>> for building and tries to build the packages in the >>>> dependency tree order, but that's mostly useful for >>>> bootstrapping from scratch, and we are cheating by >>>> bootstrapping on F19, so it isn't as big a problem. >>>> >>>>> * Install mock (git or are the packages ok?) >>>> See above - you can save a lot of time for the first >>>> build pass by not using mock. Install all Fedora >>>> packages, and then simply use: >>>> >>>> rpmbuild --rebuild $package.rpm >>>> >>>>> * build a mock F19 starter, test compile something traditional >>>>> (bash?) >>>>> * Duplicate this environment to the various machines >>>>> * set up nfs for compile target >>>>> * wrap some scripts around pssh to do parallel builds >>>>> >>>>> -- Am I missing something major here? >>>> That's pretty much it. I am happy to share the scripts >>>> I use. If I don't post them by the weekend ping me >>>> to remind me. I can't get to them right now because my >>>> build farm is behind I firewall I don't have a hole on. >> >> I would be happy to setup mock for you if you want ... or even just put >> what I have been using for mock configs here on this list for a test >> build. >> >> I personally would rather produce the RPMs via mock as that prevents >> pulling in spurious links for packages because the buildroot is too fat >> (ie, in RHEL, package-y.x.z does not link against package-a.b.c because >> it is not a BuildRequire and not installed in the mock build root .. >> but >> if run with the package-a.b.c in the buildroot, the configure process >> checks for and links against it. > > Indeed, I touched upon that in my previous post, but that doesn't > really matter too much for the first build pass, as you are going > to rebuild everything again anyway. And you can pick off the few > build failures arising from too much junk in the build environment > before the second pass via mock rebuilds. > >> All mock does is build a minimum clean build root for each package >> where >> only the specific requires for building are in the build root so that >> each package gets only what it needs to build and builds are more >> consistently. > > Sure, but the time it takes to rm -rf the build root and then untar > a cached build root copy is a non-trivial fraction of the build > time for a lot of the packages. It is certainly not trivial when > you multiply it by around 2,000 for the number of packages you are > going to need to build. > > From experience, anything you can do to get past the first stage > build faster is usually a good idea if hardware is limited - and > on ARM it usually is. Even on something like the Arndale Octa or > the new Chromebook which have 3-4GB of RAM and 8 cores, building > takes a while. It's not like throwing a full distro rebuild at a > 24-thread 96GB of RAM Xeon monster and being able to expect it to > have completed by tomorrow morning. > > Oh trust me on that one, I know how this is, and the boards we have aren't monsters. (but we have a fair amount of them;) Ex Gentoo dev here, I know -everything- about obnoxious build times and how amazingly painful it can be. I suspect this will be a lot like going back to an old thunderbird CPU with PATA drives. Crossing it off onto many different machines might help. Not sure if there's a way to hook btrfs or lvm snapshots into the build/restore process ( I don't understand it enough yet to test ) but those certainly won't work over network filesystems. //D.S -- 8362 CB14 98AD 11EF CEB6 FA81 FCC3 7674 449E 3CFC -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 884 bytes Desc: OpenPGP digital signature URL: <http://lists.centos.org/pipermail/arm-dev/attachments/20140703/9f416127/attachment-0004.sig>