I have a 10 blade cluster of just hardware - I can install what I want, how I want. What options are there if I wanted to build the 10 blades as one large beast, but _NOT_ necessarily for someone doing grid-type work? Some users don't now how to program that way, but they'd like to have their program run on something that acts like a single processor, but a massive single processor with lots of memory. Some code is designed for a single, powerful workstation. I'd like this cluster to act as one powerful workstation.
Thanks for insights.
Scott
Scott Ehrlich wrote:
Some code is designed for a single, powerful workstation. I'd like this cluster to act as one powerful workstation.
Good luck with that..
http://www.theregister.co.uk/2009/11/04/scalemp_vsmp_smb_cloud/
Is the only product I've heard of that claims to do what your aiming for.
Suggest your programmers get with the game and learn how to effectively scale their apps horizontally or be left behind.
nate
nate wrote:
http://www.theregister.co.uk/2009/11/04/scalemp_vsmp_smb_cloud/ Is the only product I've heard of that claims to do what your aiming for.
um, that just does away with the expensive infiniband switch in favor of an infiniband 'loop' architecture (eg, each host still uses an infiniband adapter), and virtualizes the switch.
this is simply a cost reducing measure in building conventional HP clustering where infiniband is often used for interprocess message passing because of its high performance and low latency.
you still need to program for a message passing environment and break your workload down into small chunks that can run independently
Scott:
Some code is designed for a single, powerful workstation. I'd like this cluster to act as one powerful workstation.
I think your application has to be designed to run in a cluster environment. Splitting up a job into multiple streams of execution requires some the application to coordinate the different pieces that are running.
Neil
-- Neil Aggarwal, (281)846-8957, http://www.JAMMConsulting.com CentOS 5.4 KVM VPS $55/mo, no setup fee, no contract, dedicated 64bit CPU 1GB dedicated RAM, 40GB RAID storage, 500GB/mo premium BW, Zero downtime
On Sat, 7 Nov 2009, Scott Ehrlich wrote:
I have a 10 blade cluster of just hardware - I can install what I want, how I want. What options are there if I wanted to build the 10 blades as one large beast, but _NOT_ necessarily for someone doing grid-type work? Some users don't now how to program that way, but they'd like to have their program run on something that acts like a single processor, but a massive single processor with lots of memory. Some code is designed for a single, powerful workstation. I'd like this cluster to act as one powerful workstation.
I have had okay experiences with ScaleMPs vSMP software. Depending on workload it may or may not be good for your needs but it does work, however, you'll also need to ensure that the nodes are connected via a high speed interconnect like Infiniband. The vSMP system is based around a COMA architecture. As such some memory is reserved for caching to hide the latencies. We had issues with job affinities. Jobs would migrate from CPU to CPU frequently leading to rather poor performance. All of these findings were reported to them as we were an early adoptor.
While this can for for some of your jobs, I would advise that if you are looking to the long term, consider taking advantage of MPI, OpenMP or concurrent languages like Erlang. You'll be happier for it.
On Sat, Nov 7, 2009 at 7:57 PM, Scott Ehrlich srehrlich@gmail.com wrote:
Some code is designed for a single, powerful workstation. I'd like this cluster to act as one powerful workstation.
I was going to suggest OpenMOSIX but that turned out to be quite dead :-(
2 minutes of frantic googling led to this project: http://www.kerrighed.org which at least seems to be alive, but I haven't tried it myself.
with kind regards, Bent Terp
On Mon, Nov 9, 2009 at 7:12 AM, Bent Terp bent@nagstrup.dk wrote:
On Sat, Nov 7, 2009 at 7:57 PM, Scott Ehrlich srehrlich@gmail.com wrote:
Some code is designed for a single, powerful workstation. I'd like this cluster to act as one powerful workstation.
I was going to suggest OpenMOSIX but that turned out to be quite dead :-(
2 minutes of frantic googling led to this project: http://www.kerrighed.org which at least seems to be alive, but I haven't tried it myself.
You might also want to look at this one: http://www.xtreemos.org/
BR Bent
Bent Terp wrote:
2 minutes of frantic googling led to this project: http://www.kerrighed.org which at least seems to be alive, but I haven't tried it myself.
You might also want to look at this one: http://www.xtreemos.org/
those are both grid solutions. as any high performance clustering solution inherently must be. You still have to program with MPI or anther grid-oriented programming methodology and break your workload up into parallel chunks
The OP specifically stated he wants his cluster to function as a single processor, so a single conventionally written (and presumably single-threaded) program would run faster, and thats just not possible.
Scott Ehrlich wrote:
I have a 10 blade cluster of just hardware - I can install what I want, how I want. What options are there if I wanted to build the 10 blades as one large beast, but _NOT_ necessarily for someone doing grid-type work? Some users don't now how to program that way, but they'd like to have their program run on something that acts like a single processor, but a massive single processor with lots of memory. Some code is designed for a single, powerful workstation. I'd like this cluster to act as one powerful workstation.
9 women can't make a baby in one month.
From: Scott Ehrlich srehrlich@gmail.com
I have a 10 blade cluster of just hardware - I can install what I want, how I want. What options are there if I wanted to build the 10 blades as one large beast, but _NOT_ necessarily for someone doing grid-type work? Some users don't now how to program that way, but they'd like to have their program run on something that acts like a single processor, but a massive single processor with lots of memory. Some code is designed for a single, powerful workstation. I'd like this cluster to act as one powerful workstation.
Maybe check http://open.eucalyptus.com/
JD
-
Bent Terp -
James A. Peltier -
John Doe -
John R Pierce -
nate -
Neil Aggarwal -
Scott Ehrlich