Hi
I am after a solution where i can easily kickstart many, read hundreds, of boxes in a short time frame. Perhaps the way i install software is to actually re-kix the box with a new software baseline - that type of idea.
I have looked at rocks and it looks good but it seems a little rigid in that i need to be able to determine certain things like hostname etc as in our env hostname represents box location etc.
Are there any tools out there that do things as 'easily' as rocks but with more customization in them? Rocks seems to be based on beowulf type clusters where our setup is more an application type affair where communication it between the apps and not through the master box.
thanks for any pointers
On 2/9/07, Tom Brown tom@ng23.net wrote:
Hi
I am after a solution where i can easily kickstart many, read hundreds, of boxes in a short time frame. Perhaps the way i install software is to actually re-kix the box with a new software baseline - that type of idea.
I have looked at rocks and it looks good but it seems a little rigid in that i need to be able to determine certain things like hostname etc as in our env hostname represents box location etc.
Are there any tools out there that do things as 'easily' as rocks but with more customization in them? Rocks seems to be based on beowulf type clusters where our setup is more an application type affair where communication it between the apps and not through the master box.
thanks for any pointers
Check http://www.cs.inf.ethz.ch/CoPs/patagonia/ . Nettee or Dolly+ cloning tools seem to be time optimal. Some customization can probably be done through DHCP.
Tom,
FYI, If you are handy with SQL, you can customize a lot of things in ROCKS cluster, including node names. Instead of using their packaged tools to deploy the nodes, you'd just have to pre-populate the SQL database with the correct information. Unfortunately, the documentation is very poor so you need to be willing and able to look into the code and database schema to see how things work. I've deployed ROCKS cluster with additional file servers (in addition to the front-end), customized node names, and two cluster private networks (instead of just one) connecting all nodes for more data- intensive computing applications. One thing I have NOT figured out is the limitation of having one front-end, which is a single source of failure for accessing the cluster.
Steven
On Feb 9, 2007, at 11:50 AM, Tom Brown wrote:
Hi
I am after a solution where i can easily kickstart many, read hundreds, of boxes in a short time frame. Perhaps the way i install software is to actually re-kix the box with a new software baseline
- that type of idea.
I have looked at rocks and it looks good but it seems a little rigid in that i need to be able to determine certain things like hostname etc as in our env hostname represents box location etc.
Are there any tools out there that do things as 'easily' as rocks but with more customization in them? Rocks seems to be based on beowulf type clusters where our setup is more an application type affair where communication it between the apps and not through the master box.
thanks for any pointers _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
One thing I have NOT figured out is the limitation of having one front-end, which is a single source of failure for accessing the cluster.
That is my main issue with Rocks as well. That and the fact that you have to have a private and a public network (which although a good idea, not always desirable).
Back to the original poster, are you looking for a way to rebuild the nodes automatically, remotely, and completely hands off, or just merely to rebuild them quickly? I only deal with a couple of dozen nodes (not hundreds), and using kickstart and storing the config files on the server allows me to rebuild a node in about 15 minutes, of which only the first minute or two requires my attention and physical presence. I wrote a simple Perl script that generates the kickstart file for each node.
Alfred
Back to the original poster, are you looking for a way to rebuild the nodes automatically, remotely, and completely hands off, or just merely to rebuild them quickly? I only deal with a couple of dozen nodes (not hundreds), and using kickstart and storing the config files on the server allows me to rebuild a node in about 15 minutes, of which only the first minute or two requires my attention and physical presence. I wrote a simple Perl script that generates the kickstart file for each node.
thanks for the reply's - the main issue i have is that when i build a new box currently my kix has to go something like
./addnewbox 'mac-yada-yada-yada' <hostname> <ip> <image name>
then walk away - yes i know thats not that hard and requires minimum effort - I do however have to find mac's out first etc which on a lot of boxes at once can be a pain.
i like the idea in rocks to be able to go next mac you see install with x image so that i can just power up the boxes in order.
I guess id'd also like to be able to select a bunch of already running boxes and have them rebuild to where they were last install just like rocks can also do so i guess i'm after a mix of what i have now and some features that rocks has. I would also like to be able to force a pix boot next reboot just like the 'shoot-node' command in rocks but i dont seem to be able to figure out how that works!!
thanks
in rocks when you run the shoot-node command it reboots the box to PXE, even if PXE is not first in the boot order it seems, does anyone know how that is achieved? Is it in the grub config and if so how?
thanks
Back to the original poster, are you looking for a way to rebuild the nodes automatically, remotely, and completely hands off, or just merely to rebuild them quickly? I only deal with a couple of dozen nodes (not hundreds), and using kickstart and storing the config files on the server allows me to rebuild a node in about 15 minutes, of which only the first minute or two requires my attention and physical presence. I wrote a simple Perl script that generates the kickstart file for each node.
thanks for the reply's - the main issue i have is that when i build a new box currently my kix has to go something like
./addnewbox 'mac-yada-yada-yada' <hostname> <ip> <image name>
then walk away - yes i know thats not that hard and requires minimum effort - I do however have to find mac's out first etc which on a lot of boxes at once can be a pain.
i like the idea in rocks to be able to go next mac you see install with x image so that i can just power up the boxes in order.
I guess id'd also like to be able to select a bunch of already running boxes and have them rebuild to where they were last install just like rocks can also do so i guess i'm after a mix of what i have now and some features that rocks has. I would also like to be able to force a pix boot next reboot just like the 'shoot-node' command in rocks but i dont seem to be able to figure out how that works!!
thanks _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Tue, 20 Feb 2007 12:42:33 +0000 "TB" == Tom Brown tom@ng23.net wrote:
TB> in rocks when you run the shoot-node command it reboots the TB> box to PXE, even if PXE is not first in the boot order it TB> seems, does anyone know how that is achieved? Is it in the TB> grub config and if so how?
There is a pseudo-service rocks-grub installed on every rocks-node. When the node boots it sets the grub-configuration in such a way that grub will call pxe-boot. At the end of a regular shutdown it resets the grub-configuration to the regular harddisk-boot. That way every time something bad happens to the node (in other words: was not shutdown regularily) the node 'boots' by PXE (and is reinstalled).
The bottom line is that the BIOS-Bootorder does not get changed (the node always boots from harddisk, just into different configurations)
>> >>> Back to the original poster, are you looking for a way to >>> rebuild the nodes automatically, remotely, and completely >>> hands off, or just merely to rebuild them quickly? I only >>> deal with a couple of dozen nodes (not hundreds), and using >>> kickstart and storing the config files on the server allows me >>> to rebuild a node in about 15 minutes, of which only the first >>> minute or two requires my attention and physical presence. I >>> wrote a simple Perl script that generates the kickstart file >>> for each node. >> thanks for the reply's - the main issue i have is that when i >> build a new box currently my kix has to go something like >> ./addnewbox 'mac-yada-yada-yada' <hostname> <ip> <image name> >> then walk away - yes i know thats not that hard and requires >> minimum effort - I do however have to find mac's out first etc >> which on a lot of boxes at once can be a pain. i like the idea >> in rocks to be able to go next mac you see install with x image >> so that i can just power up the boxes in order. I guess id'd >> also like to be able to select a bunch of already running boxes >> and have them rebuild to where they were last install just like >> rocks can also do so i guess i'm after a mix of what i have now >> and some features that rocks has. I would also like to be able >> to force a pix boot next reboot just like the 'shoot-node' >> command in rocks but i dont seem to be able to figure out how >> that works!! thanks >> _______________________________________________ CentOS mailing >> list CentOS@centos.org >> http://lists.centos.org/mailman/listinfo/centos
TB> _______________________________________________ CentOS mailing TB> list CentOS@centos.org TB> http://lists.centos.org/mailman/listinfo/centos
There is a pseudo-service rocks-grub installed on every rocks-node. When the node boots it sets the grub-configuration in such a way that grub will call pxe-boot. At the end of a regular shutdown it resets the grub-configuration to the regular harddisk-boot. That way every time something bad happens to the node (in other words: was not shutdown regularily) the node 'boots' by PXE (and is reinstalled).
can you give me any examples of this grub configuration at all?
thanks
On Wed, 21 Feb 2007 11:09:19 +0000 "TB" == Tom Brown tom@ng23.net wrote:
>> There is a pseudo-service rocks-grub installed on every >> rocks-node. When the node boots it sets the grub-configuration >> in such a way that grub will call pxe-boot. At the end of a >> regular shutdown it resets the grub-configuration to the >> regular harddisk-boot. That way every time something bad >> happens to the node (in other words: was not shutdown >> regularily) the node 'boots' by PXE (and is reinstalled).
TB> can you give me any examples of this grub configuration at TB> all?
I can, but I doubt whether it will help you too much:
/etc/grub.conf is:
# grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You do not have a /boot partition. This means that # all kernel and initrd paths are relative to /, eg. # root (hd0,0) # kernel /boot/vmlinuz-version ro root=/dev/sda1 # initrd /boot/initrd-version.img #boot=/dev/sda default=0 timeout=5 hiddenmenu title Rocks Reinstall root (hd0,0) kernel /boot/kickstart/default/vmlinuz ro root=LABEL=/ ramdisk_size=150000 kssendmac ks selinux=0 initrd /boot/kickstart/default/initrd.img title Rocks (2.6.9-42.0.2.ELsmp) root (hd0,0) kernel /boot/vmlinuz-2.6.9-42.0.2.ELsmp ro root=LABEL=/ initrd /boot/initrd-2.6.9-42.0.2.ELsmp.img title Rocks-up (2.6.9-42.0.2.EL) root (hd0,0) kernel /boot/vmlinuz-2.6.9-42.0.2.EL ro root=LABEL=/ initrd /boot/initrd-2.6.9-42.0.2.EL.img
For a graceful reboot the first option is removed. The kernel that is used for Reinstalling ist part of a rocks-boot.rpm (the script that does all that is part of rocks-boot.auto.rpm). The sources for those (which you'll propably want) are not on my system and I couldn't locate them at ftp://ftp.rocksclusters.org, but maybe you have more luck.
The init.d-script that exchanges the configuration won't be too much help too, becuase cluster-kickstart is a binary (I'll send it anyway):
. /etc/rc.d/init.d/functions
# Set the file perms for cluster-kickstart to SUID /bin/chmod 4750 /boot/kickstart/cluster-kickstart
case "$1" in start) action "Rocks GRUB: Setting boot action to 'reinstall':" \ /boot/kickstart/cluster-kickstart --start touch /var/lock/subsys/rocks-grub ;;
stop) action "Rocks GRUB: Setting boot action to 'boot current kernel':" \ /boot/kickstart/cluster-kickstart --stop rm -f /var/lock/subsys/rocks-grub ;;
*) echo "Usage: $0 {start|stop}" RETVAL=1 esac
exit $RETVAL
If you need any info apart from that (as I said, I don't have the sources for the stuff) send me a mail (drop the _list from the address). Otherwise the Rocks-Mailinglist might help you.
On 09/02/07, Tom Brown tom@ng23.net wrote:
Are there any tools out there that do things as 'easily' as rocks but with more customization in them? Rocks seems to be based on beowulf type clusters where our setup is more an application type affair where communication it between the apps and not through the master box.
Have a look at oscar? http://oscar.openclustergroup.org/
-
Alfred von Campe -
Alvin Chang -
Bernhard Gschaider -
bgschaid_lists＠ice-sf.at
-
Eduardo Grosclaude -
Steven Lee -
Tom Brown