[CentOS-devel] Interested in IA64 build.
lowen at pari.edu
Tue Jan 22 16:39:00 UTC 2013
On 01/21/2013 12:28 PM, Karanbir Singh wrote:
> On 01/21/2013 04:57 PM, Lamar Owen wrote:
>> up the space heater ^H^H^H buildhost (again, photos of the buildhost can
>> be seen at http://forums.nekochan.net/viewtopic.php?f=14&t=16725868 ).....
> Wonder what the electric bill for that is :)
:-) If run 24 hours a day for 31 days, the cost would be somewhere
between $250 US and $435 US. I say 'somewhere between' due to the way
our power is figured, being a combination of usage plus demand, where
demand is the highest 15 minute average load during a particular month;
the cost of usage varies on a sliding scale based on demand (the higher
the demand, the more each kilowatthour costs).
And it depends upon how much more load we draw; if taken by itself it
would be $435; with our existing demand load in this building of 125KW,
at a usage of about 64MWh (megawatthours) in a 31 day month, the
additional 5.1KW of load (3KW direct, 2.1KW for the additional cooling
to cover the 3KW additional heat load) would only add about $250 to the
bill. I know, somewhat strange. But that is one of the things I have
to figure in my role here.....
Those numbers work out to a cost of somewhere between $0.34 US and $0.58
US per hour. It took 34 hours to build 5.8 (I forgot to run 'time' on
the 5.9 build, so I don't have figures for that), so that works out to
somewhere between ~ $11.56US to ~$19.72US to build that update. Not too
bad. The cost to get from SLC54 to C5.7 was much higher, probably about
$200US. Still not too excessive, and I had that much budget to work
with for that project (the cost of power was a small fraction of the
budget, which included my time, and the cost of my time was a lot more
than the cost of power; so much so that the cost of the power wasn't
accounted for in great detail).
>> I'm using smock),
> Have you looked at mockchain ?
I did; I don't recall right off hand why I didn't use it.... that may be
something I've documented; there were some version issues with
something; I may take another look at it now that I know more about what
>> Third-party repos are also on the plate, since there are several
>> packages we need that are in either EPEL or RepoForge. Those are
>> probably higher on the list than making the build fully self-hosted, really.
> One thing that you might run into is that the packagers on those repo's
> would almost certainly not have been testing builds on IA64, so consider
> some sort of a feedback loop : maybe even just a http interface to the
> logs and status ( let apache do the listing with options indexes' turned
> on ).
I'm weighing my options; it may be that I only build what I need.
Repoforge is huge, and I know that I don't need everything there. EPEL
likewise is huge, and likewise I know I don't need everything there.
The biggest package I actually know that I need is going to be blender;
this box is ideal for renderfarm duty, and rendering of images/videos
for planetarium use is the targeted primary use for the 30 CPU box.
Secondarily, this box makes an ideal on-demand helper for our website,
e-mail server, and other intensive tasks that are bursty. If we're
having an event, and we expect our website to be a big draw, I can see
running Pound on a box for the reverse proxy load balancer and having
25-26 instances of the Plone client (we run ZEO on the backend) and
making a large cache to handle the peak load that would otherwise sink
our normal webserver. Scripting the SGI L2 to power up the box and
bring things online is possible, and a good use for the machine, IMO
(and if the L2 can be scripted to bring up the box, it can be scripted
to power it back down once the load goes below a threshold). If it were
just rendering, GPU driven solutions would be more cost-effective; but
these big Altix systems are wonderful at handling crushing loads of
general-purpose things, not just rendering.
The other possible use is 'big data' type database correlations for
astronomical images from our plate archive; but the software needs to be
written for that use.
> Happy to promote this build effort a bit and try to drum up support.
> It would essentially boil down to : if we start it, if we promote it -
> are we then in a place where we can keep the builds going over a
> reasonable period of time. If the answer there is, YES, then lets
> start making the noise. w.r.t the system updates and tracking that -
> the buildsystem at this end which does the i686 / x86_64 / powerpc
> builds is capable of triggers via http call back. If that helps, lets
> set that up.
Let's see how much response we get; while I may not be able to commit
the 30 CPU machine to the task I can probably commit the 4 CPU box.
With sponsorship of the build that covers power and admin I can commit
the 30 CPU box. But, as it is right now, this is just for my internal
use here, but I am willing to make the results (in unsigned packages)
available to interested parties directly. With enough interest I'll
sign what I build, with my own signing key (which I've not generated yet).
And, yes, I think I'd like to see about integrating the builds of
updates, once I replicate the working build setup from the 30CPU box to
the 4 CPU box, since for the normal update cycle 4 CPUs is plenty. Very
few of the package builds take advantage of more than a couple of CPU's;
the major exceptions are glibc, gcc, and the kernel builds, which can
effectively use the additional CPUs (the kernel build in particular
really stresses the system, and watching 30CPUs spin up and completing
the build in short order is fun; it takes a lot longer to write out the
packages than it takes to actually build the files). And I'm leaving
the 4 CPU box online 24/7; it takes far less power for it.
I'll keep you updated as to the progress of the smaller box becoming the
primary build host, with the bigger box brought online for builds only
at point release time. Seems a better use of the resources; just
haven't gotten the 4 CPU box fully set up for the builds yet. We have a
20CPU box, but it has some hardware issues that may reduce the number of
usable CPU's to 12 or 16 instead of 20. I had actually started with
that box, and had some strange results; I was able to build PostgreSQL
successfully, for instance, but not the kernel. And there seems to be
some I/O corruption going on somewhere. So I'm unsure of the 20CPU
box's future, since documentation from SGI at the level of detail I need
to track this thing down isn't available to normal users.
Anyway, that's the status......
More information about the CentOS-devel