[CentOS] does anyone have experience with clusters?

[Laurence Liew]

Hi,

I have a tutorial on HPC 101.. check it out at

http://www.hpccommunity.org/f55/kusu-101-what-beowulf-cluster-computing-table-contents-234/

It should give you an understanding of HPC.

As for 1 huge disk and plenty of RAM

- huge disk - you will need a cluster filesystem.. you may wish to  
check out PVFS2

- for RAM - shared memory across multiple nodes using commodity  
hardware and software (distributed shared memory DSM type software)  
will work.. but latency will be the biggest problem you will face...  
good for academic exercise, but in the real world, such shared memory  
systems are built with proprietary high speed interconnects such as  
Quadrics or Infiniband...


Hope this helps.




-- Laurence
Kusu - HPC Management Simplified
http://www.hpccommunity.org/kusu

-------------- next part --------------
A non-text attachment was scrubbed...
Name: kusu_icon.jpg
Type: image/jpeg
Size: 925 bytes
Desc: not available
URL: <http://lists.centos.org/pipermail/centos/attachments/20081204/abab4313/attachment.jpg>
-------------- next part --------------







On 3 Dec 2008, at 11:08 PM, Rudi Ahlers wrote:

> On Wed, Dec 3, 2008 at 4:29 PM, Tom Brown <tom at ng23.net> wrote:
>>
>>>
>>> Thank you for the input. Let's forget about XEN for a moment, I was
>>> actually looking at setting up a cluster which has fail-over & load
>>> balancing capabilities, regardless of what runs on it. If XEN
>>> enterprise is the only option,then I'm not going to bother. I don't
>>> see why I need to pay for a tool which has a helpdesk and
>>> "professional technicians standing by" to help me when I get  
>>> stuck, if
>>> XEN can do the same.
>>>
>>
>>
>> i cant speak for others but when i talk of clusters and load  
>> balancing i
>> talk of different things. For load balancing i'd lean towards LVS and
>> for clusters then it very much depends on what you are clustering.
>> Application servers, databases, mail servers etc etc. For a MySQL
>> 'cluster' i'd probably go for master<>master depending on how many  
>> nodes
>> i need and the application type etc. If its application clusters then
>> things like tomcat can know about each other and take over if one of
>> them dies. I think that the point i'm trying to make is that the
>> solution very much depends on what you are trying to achieve, so to  
>> me
>> 'regardless what runs on it' is not really something to aim a good
>> answer at.
>>
>> As mentioned i am pretty sure that if you want to make your own  
>> 'cloud'
>> in todays speak then you may well be looking commercial.
>>
>> Thats just my thoughts and its most probable i am wrong.
>> _______________________________________________
>
>
>
> Hi Tom,
>
> I do use MySQL clusters, but this is an application level cluster, and
> is limited. I would like to go further and do an OS level cluster.
> With DRBD, one could mirror 2 servers identical, i.e. everything on 1
> server to the other, which is even better than MySQL clustering. But,
> DRBD only offers high-availability, i.e. if one server goes down, the
> other can take over.
>
> What I'm looking for, is how to build what is called a super computer.
> Google used to, or still even does this, where they put hundreds of
> computers into the same "cluster" / super computer, and end up with a
> 1 huge hard drive, and plenty of RAM to use :) So, my question is, how
> does one do this? I know that I can pay someone a LOT of money for it,
> but I don't have a lot of money for this. If it's not possible, I'll
> probably just go and purchase VMWare's grid application and use that,
> but I would prefer to try this myself if possible.
>
> -- 
>
> Kind Regards
> Rudi Ahlers
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos