Good morning/day and Happy New Year.
We have a geographically distributed environment (marketing speak: cloud) where we regularly need to migrate individual systems to new hardware (for bigger disks or for better geographical placement, for example). We currently use Cobbler to do our base installs automatically and I am now looking at integrating Clonezilla with Cobbler. The goal is take an *individual* system that has been customized and migrate it in an automated fashion.
We currently do this using Cobbler and then running rsync and mysqldump in a script along with other system -> userland module configurations (such as PHP modules) but the process is tedious and generally a PITA. We also must have the ability to run this across different datacenters on different continents.
Any pointers to good automated solutions?
References:
On Sat, Jan 1, 2011 at 6:50 AM, Geoff Galitz geoff@galitz.org wrote:
Good morning/day and Happy New Year.
We have a geographically distributed environment (marketing speak: cloud) where we regularly need to migrate individual systems to new hardware (for bigger disks or for better geographical placement, for example). We currently use Cobbler to do our base installs automatically and I am now looking at integrating Clonezilla with Cobbler. The goal is take an *individual* system that has been customized and migrate it in an automated fashion.
The "RHEL" way is to use RHN. I don't recommend it, it's expensive and requires an Oracle server even if you want to install your own based on the "Spacewalk" toolkit.
More efficiently, you can use 'mock' or other tools to build a stripped chroot cage, tarball it up, then un-tarball it and apply patches as needed. Then push tools to partition the disk, mount the partitions, apply the tarball on it, and configure to your particular needs. I've probably done..... 20,000 systems this way, with the approach dating all the way back to RedHat 5.2.
Properly done, you can even write a remote installer that grabs your existing swap space, stuffs the OS image *there*, then rebuilds the OS around the existing allocated swap space, then releases and re-apportions. That's less trivial.
We currently do this using Cobbler and then running rsync and mysqldump in a script along with other system -> userland module configurations (such as PHP modules) but the process is tedious and generally a PITA. We also must have the ability to run this across different datacenters on different continents.
Push or pull the OS image tarball first. You can strip out the sensitive data. such as unlocked passwords, and push commands from your central repository to reset those passwords with their encrypted versions if necessary.
Any pointers to good automated solutions?
References:
https://fedorahosted.org/cobbler/wiki/ClonezillaIntegration
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 1/1/11 5:50 AM, Geoff Galitz wrote:
Good morning/day and Happy New Year. We have a geographically distributed environment (marketing speak: cloud) where we regularly need to migrate individual systems to new hardware (for bigger disks or for better geographical placement, for example). We currently use Cobbler to do our base installs automatically and I am now looking at integrating Clonezilla with Cobbler. The goal is take an *individual* system that has been customized and migrate it in an automated fashion. We currently do this using Cobbler and then running rsync and mysqldump in a script along with other system -> userland module configurations (such as PHP modules) but the process is tedious and generally a PITA. We also must have the ability to run this across different datacenters on different continents. Any pointers to good automated solutions? References: https://fedorahosted.org/cobbler/wiki/ClonezillaIntegration
Clonezilla is probably the only thing that doesn't lock you into a single OS/distribution. It works as long as the hardware is similar and the target disk is as large or larger than the source, but unless you use DHCP for IP addressing on all NICs you can expect problems with losing your static configs when cloning. I don't think it would be efficient to clone a large number of individualized copies remotely from a central location though. You'd probably want a generic image you could cache locally, followed by a scripted update of the differences. Clonezilla also does not have a handy way to clone an existing machine to one with larger disks letting you specify new arbitrarily-sized partitions (although it can resize them all proportionally).
On Sun, Jan 2, 2011 at 11:55 AM, Les Mikesell lesmikesell@gmail.com wrote:
On 1/1/11 5:50 AM, Geoff Galitz wrote:
Good morning/day and Happy New Year. We have a geographically distributed environment (marketing speak: cloud) where we regularly need to migrate individual systems to new hardware (for bigger disks or for better geographical placement, for example). We currently use Cobbler to do our base installs automatically and I am now looking at integrating Clonezilla with Cobbler. The goal is take an *individual* system that has been customized and migrate it in an automated fashion. We currently do this using Cobbler and then running rsync and mysqldump in a script along with other system -> userland module configurations (such as PHP modules) but the process is tedious and generally a PITA. We also must have the ability to run this across different datacenters on different continents. Any pointers to good automated solutions? References: https://fedorahosted.org/cobbler/wiki/ClonezillaIntegration
Clonezilla is probably the only thing that doesn't lock you into a single OS/distribution. It works as long as the hardware is similar and the target disk is as large or larger than the source, but unless you use DHCP for IP addressing on all NICs you can expect problems with losing your static configs when cloning. I don't think it would be efficient to clone a large number of individualized copies remotely from a central location though. You'd probably want a generic image you could cache locally, followed by a scripted update of the differences. Clonezilla also does not have a handy way to clone an existing machine to one with larger disks letting you specify new arbitrarily-sized partitions (although it can resize them all proportionally).
-- Les Mikesell lesmikesell@gmail.com
That's what the post-install wrapper scripts are for, and definitely why enterprise class setups need DHCP reservation based setups, so that re-installs automatically get the right network setups.
CentOS and RHEL 5 DHCP do have a flaw. Their DHCP is so old that it does not follow the "domain search order" directives of DHCP RFC's that already existed when they were published, but weren't in the main DHCP codeline. So if you have "example.com", "mail.example.com", and "test.example.com" in your local searchable domains, you won't get it in your resolv.conf unless you manually insert the entirely undocumented "SEARCH=" settings in /etc/sysconfig/network. Neither system-config-network nor NetworkManager get this right.
And yes, I reported this as a feature request for documentation of this DHCP configuration requirement on my RHEL systems.
On Sat, Jan 1, 2011 at 6:50 AM, Geoff Galitz geoff@galitz.org wrote:
Good morning/day and Happy New Year.
We have a geographically distributed environment (marketing speak: cloud) where we regularly need to migrate individual systems to new hardware (for bigger disks or for better geographical placement, for example). We currently use Cobbler to do our base installs automatically and I am now looking at integrating Clonezilla with Cobbler. The goal is take an *individual* system that has been customized and migrate it in an automated fashion.
We currently do this using Cobbler and then running rsync and mysqldump in a script along with other system -> userland module configurations (such as PHP modules) but the process is tedious and generally a PITA. We also must have the ability to run this across different datacenters on different continents.
Any pointers to good automated solutions?
Do you use cfengine or puppet in your environment? If you install the base packages via post installation scripts, you can use cfengine to layer on specific host identities (hostname, IP, configuration, packages, etc.). With either of these tools you can import configurations from a running host and then layer them back to a new host. There is some considerable setup to get it to that point, but lots of scripts available online so much of the work is done.
On Mon, Jan 3, 2011 at 1:55 PM, Kwan Lowe kwan.lowe@gmail.com wrote:
On Sat, Jan 1, 2011 at 6:50 AM, Geoff Galitz geoff@galitz.org wrote:
[knip]
Any pointers to good automated solutions?
Do you use cfengine or puppet in your environment? If you install the base packages via post installation scripts, you can use cfengine to layer on specific host identities (hostname, IP, configuration, packages, etc.). With either of these tools you can import configurations from a running host and then layer them back to a new host. There is some considerable setup to get it to that point, but lots of scripts available online so much of the work is done.
+1 on cfengine. (Kikckstart|Jumpstart|Fai) + cfengine = admin nirvana.
This also applies to puppet/chef/whatever. Just use management software.
On Sat, Jan 1, 2011 at 6:50 AM, Geoff Galitz geoff@galitz.org wrote:
We have a geographically distributed environment (marketing speak: cloud) where we regularly need to migrate individual systems to new hardware (for bigger disks or for better geographical placement, for example). We currently use Cobbler to do our base installs automatically and I am now looking at integrating Clonezilla with Cobbler. The goal is take an *individual* system that has been customized and migrate it in an automated fashion.
[snip]
Any pointers to good automated solutions?
Check out this document: http://www.ibm.com/developerworks/cloud/library/cl-automatevm/index.html?ca=...
Not exactly your case, but it seems it could address some of the concerns you had.
-
Geoff Galitz -
Kwan Lowe -
Les Mikesell -
Natxo Asenjo -
Nico Kadel-Garcia