[CentOS-devel] Interested in IA64 build.

Tue Jan 22 16:39:00 UTC 2013
Lamar Owen <lowen at pari.edu>

On 01/21/2013 12:28 PM, Karanbir Singh wrote:
> On 01/21/2013 04:57 PM, Lamar Owen wrote:
>> up the space heater ^H^H^H buildhost (again, photos of the buildhost can
>> be seen at http://forums.nekochan.net/viewtopic.php?f=14&t=16725868 ).....
> Wonder what the electric bill for that is :)

:-)  If run 24 hours a day for 31 days, the cost would be somewhere 
between $250 US and $435 US.  I say 'somewhere between' due to the way 
our power is figured, being a combination of usage plus demand, where 
demand is the highest 15 minute average load during a particular month; 
the cost of usage varies on a sliding scale based on demand (the higher 
the demand, the more each kilowatthour costs).

And it depends upon how much more load we draw; if taken by itself it 
would be $435; with our existing demand load in this building of 125KW, 
at a usage of about 64MWh (megawatthours) in a 31 day month, the 
additional 5.1KW of load (3KW direct, 2.1KW for the additional cooling 
to cover the 3KW additional heat load) would only add about $250 to the 
bill.  I know, somewhat strange.  But that is one of the things I have 
to figure in my role here.....

Those numbers work out to a cost of somewhere between $0.34 US and $0.58 
US per hour.  It took 34 hours to build 5.8 (I forgot to run 'time' on 
the 5.9 build, so I don't have figures for that), so that works out to 
somewhere between ~ $11.56US to ~$19.72US to build that update.  Not too 
bad.  The cost to get from SLC54 to C5.7 was much higher, probably about 
$200US.  Still not too excessive, and I had that much budget to work 
with for that project (the cost of power was a small fraction of the 
budget, which included my time, and the cost of my time was a lot more 
than the cost of power; so much so that the cost of the power wasn't 
accounted for in great detail).

>>   I'm using smock),
> Have you looked at mockchain ?

I did; I don't recall right off hand why I didn't use it.... that may be 
something I've documented; there were some version issues with 
something; I may take another look at it now that I know more about what 
I'm doing.....

>> Third-party repos are also on the plate, since there are several
>> packages we need that are in either EPEL or RepoForge.  Those are
>> probably higher on the list than making the build fully self-hosted, really.
> One thing that you might run into is that the packagers on those repo's
> would almost certainly not have been testing builds on IA64, so consider
> some sort of a feedback loop : maybe even just a http interface to the
> logs and status ( let apache do the listing with options indexes' turned
> on ).

I'm weighing my options; it may be that I only build what I need.  
Repoforge is huge, and I know that I don't need everything there.  EPEL 
likewise is huge, and likewise I know I don't need everything there.  
The biggest package I actually know that I need is going to be blender; 
this box is ideal for renderfarm duty, and rendering of images/videos 
for planetarium use is the targeted primary use for the 30 CPU box.  
Secondarily, this box makes an ideal on-demand helper for our website, 
e-mail server, and other intensive tasks that are bursty.  If we're 
having an event, and we expect our website to be a big draw, I can see 
running Pound on a box for the reverse proxy load balancer and having 
25-26 instances of the Plone client (we run ZEO on the backend) and 
making a large cache to handle the peak load that would otherwise sink 
our normal webserver.  Scripting the SGI L2 to power up the box and 
bring things online is possible, and a good use for the machine, IMO 
(and if the L2 can be scripted to bring up the box, it can be scripted 
to power it back down once the load goes below a threshold).  If it were 
just rendering, GPU driven solutions would be more cost-effective; but 
these big Altix systems are wonderful at handling crushing loads of 
general-purpose things, not just rendering.

The other possible use is 'big data' type database correlations for 
astronomical images from our plate archive; but the software needs to be 
written for that use.

> Happy to promote this build effort a bit and try to drum up support. 
> It would essentially boil down to : if we start it, if we promote it - 
> are we then in a place where we can keep the builds going over a 
> reasonable period of time. If the answer there is, YES, then lets 
> start making the noise. w.r.t the system updates and tracking that - 
> the buildsystem at this end which does the i686 / x86_64 / powerpc 
> builds is capable of triggers via http call back. If that helps, lets 
> set that up.

Let's see how much response we get; while I may not be able to commit 
the 30 CPU machine to the task I can probably commit the 4 CPU box.  
With sponsorship of the build that covers power and admin I can commit 
the 30 CPU box.  But, as it is right now, this is just for my internal 
use here, but I am willing to make the results (in unsigned packages) 
available to interested parties directly.  With enough interest I'll 
sign what I build, with my own signing key (which I've not generated yet).

And, yes, I think I'd like to see about integrating the builds of 
updates, once I replicate the working build setup from the 30CPU box to 
the 4 CPU box, since for the normal update cycle 4 CPUs is plenty.  Very 
few of the package builds take advantage of more than a couple of CPU's; 
the major exceptions are glibc, gcc, and the kernel builds, which can 
effectively use the additional CPUs (the kernel build in particular 
really stresses the system, and watching 30CPUs spin up and completing 
the build in short order is fun; it takes a lot longer to write out the 
packages than it takes to actually build the files).  And I'm leaving 
the 4 CPU box online 24/7; it takes far less power for it.

I'll keep you updated as to the progress of the smaller box becoming the 
primary build host, with the bigger box brought online for builds only 
at point release time.  Seems a better use of the resources; just 
haven't gotten the 4 CPU box fully set up for the builds yet.  We have a 
20CPU box, but it has some hardware issues that may reduce the number of 
usable CPU's to 12 or 16 instead of 20.  I had actually started with 
that box, and had some strange results; I was able to build PostgreSQL 
successfully, for instance, but not the kernel.  And there seems to be 
some I/O corruption going on somewhere.  So I'm unsure of the 20CPU 
box's future, since documentation from SGI at the level of detail I need 
to track this thing down isn't available to normal users.

Anyway, that's the status......