Hey folks,
I just went through the archives trying to find some info on this but did not come up with much other than it seems there are a few experts here on the list.
I have no experience with clustering and have just taken over a Stem Cell Research Lab that has a Grid Engine cluster. I have not yet dug into the details of Grid Engine (only been here a week now) but am just trying to get up to speed on clustering in general.
I was just looking at Red Hat's site and they have this HPC thing http://www.redhat.com/promo/mrg/ but damned if I can find any actual details on it there - that data sheet they link to is just a bunch of marketing gobble-de-gook as far as I can make sense of it anyway.
Quick question : what are Red Hat using to do that, and can CentOS do the same thing? How hard is it to configure? How does it compare to Grid Engine?
I have to say I'm a bit hesitant about Grid Engine because of the whole Oracle takeover. I just don't trust Oracle.
Basically I'd like to get up to speed really quickly on different clustering technologies, and maybe even set up a CentOS (or Scientific) based cluster in a sandbox to play with.
I guess - looking for reading to get up to speed on clustering, and wondering what my options are with CentOS, RHEL and Scientific.
thanks, -Alan
I recommend you check out ROCKS
CentOS based clustering with lots of built in goodness.
Tony Schreiner
On 11/15/2011 9:50 PM, Alan McKay wrote:
Hey folks,
I just went through the archives trying to find some info on this but did not come up with much other than it seems there are a few experts here on the list.
I have no experience with clustering and have just taken over a Stem Cell Research Lab that has a Grid Engine cluster. I have not yet dug into the details of Grid Engine (only been here a week now) but am just trying to get up to speed on clustering in general.
I was just looking at Red Hat's site and they have this HPC thing http://www.redhat.com/promo/mrg/ but damned if I can find any actual details on it there - that data sheet they link to is just a bunch of marketing gobble-de-gook as far as I can make sense of it anyway.
Quick question : what are Red Hat using to do that, and can CentOS do the same thing? How hard is it to configure? How does it compare to Grid Engine?
I have to say I'm a bit hesitant about Grid Engine because of the whole Oracle takeover. I just don't trust Oracle.
Basically I'd like to get up to speed really quickly on different clustering technologies, and maybe even set up a CentOS (or Scientific) based cluster in a sandbox to play with.
I guess - looking for reading to get up to speed on clustering, and wondering what my options are with CentOS, RHEL and Scientific.
thanks, -Alan
On 11/15/11 7:09 PM, Tony Schreiner wrote:
I recommend you check out ROCKS
awhile back I setup a little test OSCAR cluster, which used CentOS, and found it quite nicely packaged and easy to deploy with a rich set of tools... but I have no idea where its gone since (and this was years ago)
IIRC (again, this was years ago), you setup Oscar on a master server running CentOS with a pile of slaves 'behind' it, then net boot up the slaves and they install themselves off the master, and run whatever MPI applications you have for the cluster, deployed and managed by the master. Ganglia was used to monitor the whole mess. It had a bunch of other stuff that I forget what all, as basically we decided it wasn't suitable for where we were going so put it aside.
ah, here's the project webpile... http://svn.oscar.openclustergroup.org/trac/oscar
Le 16/11/2011 04:09, Tony Schreiner a écrit :
I recommend you check out ROCKS
CentOS based clustering with lots of built in goodness.
Hi,
I also recommend Rocks Cluster, that I used on my site. Recently, they switch to OGS, Open Grid Schduler, the open source version of SGE (there is another one too, SoGE, Son of Grid Engine), that does not depend on Oracle. In fact, SGE was relaesed by SUN under an open source license, SISSL, so open sources derivatives are allowed.
For information, most SGE developpers from Oracle were hired by Univa, a company which claimed at first they would develop SGE as open source, but are now closing it...
Alain
On 11/15/2011 09:50 PM, Alan McKay wrote:
Basically I'd like to get up to speed really quickly on different clustering technologies, and maybe even set up a CentOS (or Scientific) based cluster in a sandbox to play with.
I guess - looking for reading to get up to speed on clustering, and wondering what my options are with CentOS, RHEL and Scientific.
thanks, -Alan
I suspect this is not of interest to you, but it does fall under "different clustering technologies", if under the High-Availability side of things. You want performance clustering, which generally requires purpose-built solutions (save for simple web/dns/mail load balancers).
Older but complete HA VM clustering on EL5 (inc. CentOS);
https://alteeve.com/w/Red_Hat_Cluster_Service_2_Tutorial
More up to date but not yet complete update of the above for EL6 with redundant networking;
https://alteeve.com/w/2-Node_Red_Hat_KVM_Cluster_Tutorial
hth
----- Original Message ----- | Hey folks, | | I just went through the archives trying to find some info on this but | did not come up with much other than it seems there are a few experts | here on the list. | | I have no experience with clustering and have just taken over a Stem | Cell Research Lab that has a Grid Engine cluster. I have not yet dug | into the details of Grid Engine (only been here a week now) but am | just trying to get up to speed on clustering in general. | | I was just looking at Red Hat's site and they have this HPC thing | http://www.redhat.com/promo/mrg/ but damned if I can find any actual | details on it there - that data sheet they link to is just a bunch of | marketing gobble-de-gook as far as I can make sense of it anyway. | | Quick question : what are Red Hat using to do that, and can CentOS do | the same thing? How hard is it to configure? How does it compare to | Grid Engine? | | I have to say I'm a bit hesitant about Grid Engine because of the | whole Oracle takeover. I just don't trust Oracle. | | Basically I'd like to get up to speed really quickly on different | clustering technologies, and maybe even set up a CentOS (or | Scientific) based cluster in a sandbox to play with. | | I guess - looking for reading to get up to speed on clustering, and | wondering what my options are with CentOS, RHEL and Scientific. | | thanks, | -Alan
I'm not sure what is going to be happening with SGE, but we use Torque and Maui for our deparmental HPC clusters and Torque and MOAB for our Western Canada HPC environment (Westgrid). There are a *lot* of aspects to HPC clusters that you need to be familiar with. The resource managers and schedulers are the least of your problems. The software toolchain and optimization are *the most important*. Understanding how to optimize for the processors, troubleshooting inefficient code, etc. That's where you should focus.
FWIW: MRG is based around Condor. Aeolus the new cloud product (OpenForms) is also based around Condor.
-
Alain Péan -
Alan McKay -
Digimer -
James A. Peltier -
John R Pierce -
Tony Schreiner