Hi, CentOSers,
I found out that icecc (icecream package), the distributed compiling system delivers a great advantage for me when building software. It's quite easy to set up mock 0.9.19 to use icecream by default. There must be the lines
config_opts['chroot_setup_cmd'] = 'install buildsys-build icecream' ... config_opts['macros']['%_smp_mflags'] = "-jX"
in the specific OS definition, where X in the _smp_mplags line is the summary of all CPU cores in your compiler network. You have to add a repository containing the icecream RPM. I'm using icecream-0.9.4. The icecream cache must be delegated to the mock chroot by adding
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append\ (('/var/cache/icecream', '/var/cache/icecream' ))
to the site-defaults.cfg file. This is why mock 0.9.19 is required for this. Be sure that your icecream package contains /etc/profile.d/icecream.sh which is needed to wrap cc,gcc,g++,... for icecc in the mock chroot. Or you simply take my RPM.
Remember, that the compile nodes must be in the same subnet, so it's a good idea that moving together your build machines using VPN bridges or whatever. Pure compile nodes may even be something else than CentOS (i.e. Debian) because the needed (target) compiler binaries are delivered over the icecc network, too - and held in a cache.
What do you think about that?
Cheers, Brian
On Wed, Mar 3, 2010 at 8:30 PM, Brian Schueler brian.schueler@gmx.de wrote:
Hi, CentOSers,
I found out that icecc (icecream package), the distributed compiling system delivers a great advantage for me when building software.
It would be nice to see quantitative numbers showing said "great advantage". Or, all the numbers which might prove more than one benefit. Even the package site[1] lacks real analysis.
It's quite easy to set up mock 0.9.19 to use icecream by default. There must be the lines
config_opts['chroot_setup_cmd'] = 'install buildsys-build icecream' ... config_opts['macros']['%_smp_mflags'] = "-jX"
in the specific OS definition, where X in the _smp_mplags line is the summary of all CPU cores in your compiler network. You have to add a repository containing the icecream RPM. I'm using icecream-0.9.4. The icecream cache must be delegated to the mock chroot by adding
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append\ (('/var/cache/icecream', '/var/cache/icecream' ))
to the site-defaults.cfg file. This is why mock 0.9.19 is required for this. Be sure that your icecream package contains /etc/profile.d/icecream.sh which is needed to wrap cc,gcc,g++,... for icecc in the mock chroot. Or you simply take my RPM.
Take an RPM from where? and who should install the RPM, and on what? Your audience here is fairly limited - those in the CentOS core, and the rest of us. :) Either way, the niche for Icecream seems to be systems already using distcc[1] and even distcc, though in dag[2], isn't in a repo shipped with CentOS.
Remember, that the compile nodes must be in the same subnet, so it's a good idea that moving together your build machines using VPN bridges or whatever. Pure compile nodes may even be something else than CentOS (i.e. Debian) because the needed (target) compiler binaries are delivered over the icecc network, too - and held in a cache.
What do you think about that?
I think you have not proven your case very well. :-)
jerry
[1] http://en.opensuse.org/Icecream [2] http://dag.wieers.com/rpm/packages/distcc/
Hi Jerry
My case is a very good one. I'm able to use 22 Dual Core PCs in a cluster which has nearly up to 10 times of performance of a single system.
The C5 package for icecream can be found on: http://repo1.pst.beuth-hochschule.de/repo/bs/el5/ (this OpenDNS Domain is hard to resolve from outside EU, so 141.64.26.2 should always work).
Brian.
On Thu, Mar 4, 2010 at 4:58 PM, Brian Schueler brian.schueler@gmx.de wrote:
Hi Jerry
My case is a very good one. I'm able to use 22 Dual Core PCs in a cluster which has nearly up to 10 times of performance of a single system.
That is great emotional support of the product, but, again, does not provide numerical proof of your claim. Case in point, you use 22 PCs, yet only get 10 times the performance of a single system, so I propose that the "icecream" package causes a performance *degradation* of 55%. ;-)
The C5 package for icecream can be found on: http://repo1.pst.beuth-hochschule.de/repo/bs/el5/ (this OpenDNS Domain is hard to resolve from outside EU, so 141.64.26.2 should always work).
Good, that's something at least.
jerry
Hi Jerry,
yes you're right, that the performance is not the optimum. This is also caused by the network bottleneck. Files have to be moved around. But first I also used distcc, but it is dangerous when non-CentOS systems or mixed 64 and 32-bit systems are in action. In these cases code linkage failes. So I switched to icecc, which first distributes the right compilers (which the system doing the make job has currently installed). There is also someone who compiles with Gentoo on the CentOS cluster with a different gcc version and it works perfectly.
Greetings, Brian.
-
Brian Schueler -
Jerry Amundson