I have a tested copy of EL6 that I would like to duplicate to a number of similar servers, but can't seem to find a sane howto on the subject. TLDP's "Hard disk upgrade howto" is embarrassingly antiquated: when's the last time you saw LILO?
What's the recommended procedure for doing an HDD copy/upgrade from one disk to another? Is there a nice, detailed howto that explains this in detail?
My set up is pretty straight forward. My partitions are all set up as simple partitions, with UUID's in /etc/fstab. I'm not using LVM or RAID.
Any ideas?
-Ben
On 7/29/11 9:45 PM, Benjamin Smith wrote:
I have a tested copy of EL6 that I would like to duplicate to a number of similar servers, but can't seem to find a sane howto on the subject. TLDP's "Hard disk upgrade howto" is embarrassingly antiquated: when's the last time you saw LILO?
What's the recommended procedure for doing an HDD copy/upgrade from one disk to another? Is there a nice, detailed howto that explains this in detail?
My set up is pretty straight forward. My partitions are all set up as simple partitions, with UUID's in /etc/fstab. I'm not using LVM or RAID.
If the machines are pretty much identical, clonezilla should work. Boot the machine with a 'clonezilla-live' CD or USB drive. If you can attach the target drive to the same machine you can go disk->disk. Otherwise, connect to something on the network with enough space to hold the image (ssh, nfs, or a windows share) and clone disk->image, then boot the target machines and copy image->disk. There is also a server version that will pxe-boot the targets if you need to do a large number of them.
On Friday, July 29, 2011 08:02:20 PM Les Mikesell wrote:
If the machines are pretty much identical, clonezilla should work. Boot the machine with a 'clonezilla-live' CD or USB drive. If you can attach the target drive to the same machine you can go disk->disk. Otherwise, connect to something on the network with enough space to hold the image (ssh, nfs, or a windows share) and clone disk->image, then boot the target machines and copy image->disk. There is also a server version that will pxe-boot the targets if you need to do a large number of them.
Thanks for the tip! I'll be trying this out on Monday, but what is the "native" way of doing this?
On 7/30/11 2:03 AM, Benjamin Smith wrote:
On Friday, July 29, 2011 08:02:20 PM Les Mikesell wrote:
If the machines are pretty much identical, clonezilla should work. Boot
the machine with a 'clonezilla-live' CD or USB drive. If you can attach
the target drive to the same machine you can go disk->disk. Otherwise,
connect to something on the network with enough space to hold the image
(ssh, nfs, or a windows share) and clone disk->image, then boot the target
machines and copy image->disk. There is also a server version that will
pxe-boot the targets if you need to do a large number of them.
Thanks for the tip! I'll be trying this out on Monday, but what is the "native" way of doing this?
For disk image copies, you can use 'dd' with the raw disks. But it won't know anything about the filesystem and will copy all sectors. Clonezilla will use partclone (or ntfsclone for windows) and only copy the used blocks so it is much faster with large disks. In any case the partitions must be unmounted or at least not changed while being copied, so you need to boot from something else. I recommend the ubuntu-based version of clonezilla for better hardware support. Also, you'll need to fix up the network settings and hostname on each clone after you bring it up.
At Sat, 30 Jul 2011 09:12:28 -0500 CentOS mailing list centos@centos.org wrote:
On 7/30/11 2:03 AM, Benjamin Smith wrote:
On Friday, July 29, 2011 08:02:20 PM Les Mikesell wrote:
If the machines are pretty much identical, clonezilla should work. Boot
the machine with a 'clonezilla-live' CD or USB drive. If you can attach
the target drive to the same machine you can go disk->disk. Otherwise,
connect to something on the network with enough space to hold the image
(ssh, nfs, or a windows share) and clone disk->image, then boot the target
machines and copy image->disk. There is also a server version that will
pxe-boot the targets if you need to do a large number of them.
Thanks for the tip! I'll be trying this out on Monday, but what is the "native" way of doing this?
For disk image copies, you can use 'dd' with the raw disks. But it won't know anything about the filesystem and will copy all sectors. Clonezilla will use partclone (or ntfsclone for windows) and only copy the used blocks so it is much faster with large disks. In any case the partitions must be unmounted or at least not changed while being copied, so you need to boot from something else.
dump/restore is a 'native' file system copy method. Doing this in single-user mode works extremely well, although the really paranoid can always boot up either a live cd or the boot/install cd in rescue mode. I've done this countless times in either single-user mode and sometimes in full multi-user mode (on a 'quiet' system). (dump can dump *unmounted* file systems.)
dd can be problematical if the target and source disks are different (sizes, geometry, etc.), since dd will do a literal sector-by-sector copy, which is not generally advisable (and why o why to people *keep* suggesting it? -- it is really a misuse of dd, unless you *really* know what you are doing).
I recommend the ubuntu-based version of clonezilla for better hardware support. Also, you'll need to fix up the network settings and hostname on each clone after you bring it up.
On Saturday, July 30, 2011 11:18:34 AM Robert Heller wrote:
dd can be problematic if the target and source disks are different (sizes, geometry, etc.), since dd will do a literal sector-by-sector copy, which is not generally advisable (and why o why to people *keep* suggesting it? -- it is really a misuse of dd, unless you *really* know what you are doing).
Geometry on an LBA device should not be an issue. If it is an issue, that is a bug, since it is impossible to specify the actual geometry of the disk, even if the manufacturer makes that information available due to varying numbers of sectors per track across the platters. CHS geometry is an archaic thing.
For direct clones dd and its variants work well, and get data that isn't in any filesystem or partition (boot loaders, in particular, often use the space before the first partition on the disk). The default upstream EL6.1 install leaves a full megabyte in front of the first partition; some bootloaders and other utilities use this space.
Things like Dell's MediaDirect, for instance..... and that's but one example. Restore partitions (not just for Windows; Dell systems with Ubuntu preinstalled have them, and extended MBR boot sectors to handle them (LBA 3 is a common place to put the 'extended' boot loader for such things)).
And that's part of the reason I'll use dd or a variant (ddrescue or dd_rescue, etc) if doing a clone to a disk of the same size or larger. Then I can resize partitions as needed for the larger disk using any one of a number of tools for the job.
At Sat, 30 Jul 2011 11:51:04 -0400 CentOS mailing list centos@centos.org wrote:
On Saturday, July 30, 2011 11:18:34 AM Robert Heller wrote:
dd can be problematic if the target and source disks are different (sizes, geometry, etc.), since dd will do a literal sector-by-sector copy, which is not generally advisable (and why o why to people *keep* suggesting it? -- it is really a misuse of dd, unless you *really* know what you are doing).
Geometry on an LBA device should not be an issue. If it is an issue, that is a bug, since it is impossible to specify the actual geometry of the disk, even if the manufacturer makes that information available due to varying numbers of sectors per track across the platters. CHS geometry is an archaic thing.
For direct clones dd and its variants work well, and get data that isn't in any filesystem or partition (boot loaders, in particular, often use the space before the first partition on the disk). The default upstream EL6.1 install leaves a full megabyte in front of the first partition; some bootloaders and other utilities use this space.
Things like Dell's MediaDirect, for instance..... and that's but one example. Restore partitions (not just for Windows; Dell systems with Ubuntu preinstalled have them, and extended MBR boot sectors to handle them (LBA 3 is a common place to put the 'extended' boot loader for such things)).
And that's part of the reason I'll use dd or a variant (ddrescue or dd_rescue, etc) if doing a clone to a disk of the same size or larger. Then I can resize partitions as needed for the larger disk using any one of a number of tools for the job.
Presumably you '*really* know what you are doing'. I would recogmend *against* dd or a variant for a newbie, who is likely to get him or herself into various sorts of trouble (yes, with modern LBA devices, geometry is not going to be a problem) -- there was a thread not too long ago about someone who was cloning SSD devices with dd and having various problems. If the destination disk is larger (or worse smaller), dd can cause 'problems' -- if larger, there will be unused parts of the disk requiring some sort of post hacking (futzing with the partition table, fun with resize2fs, etc.), if smaller (even by the odd cylinder's worth of sectors or so -- after all a 500G Seagate disk might have *exactly* the same number of sectors/blocks as a 500G WD or Fujitzu disk), well, opps!
Also: dd can be *slow* with a large disk that is nowhere near full. It can be a waste of time, etc. to copy 400G of empty sectors, when all you really needed to copy was 100G worth of used file system space. dd will also preserve all of the 'warts' of the file system (fragmentation, deleted files, directories that have grown huge from having lots of files that have since been deleted, etc.). dump/restore, tar, etc. don't do that.
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 7/30/11 12:30 PM, Robert Heller wrote:
At Sat, 30 Jul 2011 11:51:04 -0400 CentOS mailing listcentos@centos.org wrote:
On Saturday, July 30, 2011 11:18:34 AM Robert Heller wrote:
dd can be problematic if the target and source disks are different (sizes, geometry, etc.), since dd will do a literal sector-by-sector copy, which is not generally advisable (and why o why to people *keep* suggesting it? -- it is really a misuse of dd, unless you *really* know what you are doing).
Geometry on an LBA device should not be an issue. If it is an issue, that is a bug, since it is impossible to specify the actual geometry of the disk, even if the manufacturer makes that information available due to varying numbers of sectors per track across the platters. CHS geometry is an archaic thing.
For direct clones dd and its variants work well, and get data that isn't in any filesystem or partition (boot loaders, in particular, often use the space before the first partition on the disk). The default upstream EL6.1 install leaves a full megabyte in front of the first partition; some bootloaders and other utilities use this space.
Things like Dell's MediaDirect, for instance..... and that's but one example. Restore partitions (not just for Windows; Dell systems with Ubuntu preinstalled have them, and extended MBR boot sectors to handle them (LBA 3 is a common place to put the 'extended' boot loader for such things)).
And that's part of the reason I'll use dd or a variant (ddrescue or dd_rescue, etc) if doing a clone to a disk of the same size or larger. Then I can resize partitions as needed for the larger disk using any one of a number of tools for the job.
Presumably you '*really* know what you are doing'. I would recogmend *against* dd or a variant for a newbie, who is likely to get him or herself into various sorts of trouble (yes, with modern LBA devices, geometry is not going to be a problem) -- there was a thread not too long ago about someone who was cloning SSD devices with dd and having various problems. If the destination disk is larger (or worse smaller), dd can cause 'problems' -- if larger, there will be unused parts of the disk requiring some sort of post hacking (futzing with the partition table, fun with resize2fs, etc.), if smaller (even by the odd cylinder's worth of sectors or so -- after all a 500G Seagate disk might have *exactly* the same number of sectors/blocks as a 500G WD or Fujitzu disk), well, opps!
Also: dd can be *slow* with a large disk that is nowhere near full. It can be a waste of time, etc. to copy 400G of empty sectors, when all you really needed to copy was 100G worth of used file system space. dd will also preserve all of the 'warts' of the file system (fragmentation, deleted files, directories that have grown huge from having lots of files that have since been deleted, etc.). dump/restore, tar, etc. don't do that.
Dump/restore, tar, also won't make the target disk bootable and you'll need to know an assortment of cruft to do that manually. Clonezilla uses sort of an intermediate approach, making partitions and copying with partclone (with dd as a fallback for unknown partition types), then doing the grub install cruft for you.
One other thing that might do all the grunge work right is 'ReaR'. http://rear.sourceforge.net/. It builds a bootable iso with the tools from your running system that knows how to reconstruct the filesystem layout and restore a backup (of various sorts) on it. It knows more about raid than clonezilla and might even get combinations of lvm and raid right. Not sure if it works on 6.x yet, but I think it is in epel for 5.x at least. It is intended to get a backup of a working system up quickly on a replacement but it should work for cloning too.
--- Les Mikesell lesmikesell@gmail.com
At Sat, 30 Jul 2011 14:19:40 -0500 CentOS mailing list centos@centos.org wrote:
On 7/30/11 12:30 PM, Robert Heller wrote:
At Sat, 30 Jul 2011 11:51:04 -0400 CentOS mailing listcentos@centos.org wrote:
On Saturday, July 30, 2011 11:18:34 AM Robert Heller wrote:
dd can be problematic if the target and source disks are different (sizes, geometry, etc.), since dd will do a literal sector-by-sector copy, which is not generally advisable (and why o why to people *keep* suggesting it? -- it is really a misuse of dd, unless you *really* know what you are doing).
Geometry on an LBA device should not be an issue. If it is an issue, that is a bug, since it is impossible to specify the actual geometry of the disk, even if the manufacturer makes that information available due to varying numbers of sectors per track across the platters. CHS geometry is an archaic thing.
For direct clones dd and its variants work well, and get data that isn't in any filesystem or partition (boot loaders, in particular, often use the space before the first partition on the disk). The default upstream EL6.1 install leaves a full megabyte in front of the first partition; some bootloaders and other utilities use this space.
Things like Dell's MediaDirect, for instance..... and that's but one example. Restore partitions (not just for Windows; Dell systems with Ubuntu preinstalled have them, and extended MBR boot sectors to handle them (LBA 3 is a common place to put the 'extended' boot loader for such things)).
And that's part of the reason I'll use dd or a variant (ddrescue or dd_rescue, etc) if doing a clone to a disk of the same size or larger. Then I can resize partitions as needed for the larger disk using any one of a number of tools for the job.
Presumably you '*really* know what you are doing'. I would recogmend *against* dd or a variant for a newbie, who is likely to get him or herself into various sorts of trouble (yes, with modern LBA devices, geometry is not going to be a problem) -- there was a thread not too long ago about someone who was cloning SSD devices with dd and having various problems. If the destination disk is larger (or worse smaller), dd can cause 'problems' -- if larger, there will be unused parts of the disk requiring some sort of post hacking (futzing with the partition table, fun with resize2fs, etc.), if smaller (even by the odd cylinder's worth of sectors or so -- after all a 500G Seagate disk might have *exactly* the same number of sectors/blocks as a 500G WD or Fujitzu disk), well, opps!
Also: dd can be *slow* with a large disk that is nowhere near full. It can be a waste of time, etc. to copy 400G of empty sectors, when all you really needed to copy was 100G worth of used file system space. dd will also preserve all of the 'warts' of the file system (fragmentation, deleted files, directories that have grown huge from having lots of files that have since been deleted, etc.). dump/restore, tar, etc. don't do that.
Dump/restore, tar, also won't make the target disk bootable and you'll need to know an assortment of cruft to do that manually. Clonezilla uses sort of an
Yes. But it is *less* dangerious and less weird to copy the file systems using proper file system disk-to-disk copy tool (eg dump/restore or tar or cpio or the like), and then do the necessary grub-install or lilo or whatever bootloader install step. Over or under copying with dd is either a disaster or confusing, to a newbie: "why did dd crash copying from my 500G WD disk to my new 500G Seagate disk? How do I get access to the rest of my 1TB disk after I dd my 500G disk to it?". Also, dd can be a total disaster copying a *live, mounted* file system. While there are warning about dump and live file systems, tar or cpio are safe for use with a live, mounted file system. Sometimes it is easier or necessary to do this cloning with a live system (idealy in single user mode).
intermediate approach, making partitions and copying with partclone (with dd as a fallback for unknown partition types), then doing the grub install cruft for you.
One other thing that might do all the grunge work right is 'ReaR'. http://rear.sourceforge.net/. It builds a bootable iso with the tools from your running system that knows how to reconstruct the filesystem layout and restore a backup (of various sorts) on it. It knows more about raid than clonezilla and might even get combinations of lvm and raid right. Not sure if it works on 6.x yet, but I think it is in epel for 5.x at least. It is intended to get a backup of a working system up quickly on a replacement but it should work for cloning too.
Les Mikesell lesmikesell@gmail.com
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
At Fri, 29 Jul 2011 19:45:42 -0700 CentOS mailing list centos@centos.org wrote:
I have a tested copy of EL6 that I would like to duplicate to a number of similar servers, but can't seem to find a sane howto on the subject. TLDP's "Hard disk upgrade howto" is embarrassingly antiquated: when's the last time you saw LILO?
*I* still use LILO, even with my CentOS 5.6 systems.
What's the recommended procedure for doing an HDD copy/upgrade from one disk to another? Is there a nice, detailed howto that explains this in detail?
http://www.deepsoft.com/2009/01/how-to-transfer-a-linux-system-from-one-disk...
My set up is pretty straight forward. My partitions are all set up as simple partitions, with UUID's in /etc/fstab. I'm not using LVM or RAID.
Any ideas?
-Ben
On Friday 29 July 2011 22:45, Benjamin Smith wrote:
I have a tested copy of EL6 that I would like to duplicate to a number of similar servers, but can't seem to find a sane howto on the subject. TLDP's "Hard disk upgrade howto" is embarrassingly antiquated: when's the last time you saw LILO?
When I wrote that How-To, I wanted to copy Linux from a 120 Mb hard disk to a 250 Mb hard disk.
I've thought about updating the How-To, but I assumed that modern hard disks are so large that people no longer needed to copy Linux just because they ran out of disk space.
Even back then, the How-To wasn't intended for someone who wanted to make several copies. As others have suggested, Clonezilla seems appropriate for your situation: http://clonezilla.org/
I still use LILO to boot from thumb drives, by the way.
Yves
On Saturday, July 30, 2011 06:00:26 PM Yves Bellefeuille wrote:
On Friday 29 July 2011 22:45, Benjamin Smith wrote:
I have a tested copy of EL6 that I would like to duplicate to a number of similar servers, but can't seem to find a sane howto on the subject. TLDP's "Hard disk upgrade howto" is embarrassingly antiquated: when's the last time you saw LILO?
When I wrote that How-To, I wanted to copy Linux from a 120 Mb hard disk to a 250 Mb hard disk.
I've thought about updating the How-To, but I assumed that modern hard disks are so large that people no longer needed to copy Linux just because they ran out of disk space.
Even back then, the How-To wasn't intended for someone who wanted to make several copies. As others have suggested, Clonezilla seems appropriate for your situation: http://clonezilla.org/
I still use LILO to boot from thumb drives, by the way.
I feel like I'm speaking to an old friend. =) I did several upgrades with that guide until RH switched to grub. It's been painful ever since. :/ Mostly, I throw in new drives and mount them all over the place. I have systems with a 20 GB / partition with up to 10 TB of drives and partitions crufted all over the place. (yuk!)
My issue is that the instructions for LILO don't do much for me with respect to grub. There's numerous howtos mention hdparms and partition numbers and chroot and stuff, but nothing that spells it out for a dolt like me.
... and nobody here's suggested a recommended way to do this. I would *love* it if somebody who knew could ammend your excellent howto with information on grub. (which has been the RHEL/CentOS default for several years)
I tried using Clonezilla to clone a 300 GB SAS drive to a 1 TB SATA drive. I don't care *at* *all* if there's some wasted space - I just want it to *work*. But it thew a screen full of random textual garbage on the screen and not much helpful information when I hit the "go" button.
(sigh)
On 08/02/2011 06:16 AM, Benjamin Smith wrote:
On Saturday, July 30, 2011 06:00:26 PM Yves Bellefeuille wrote:
On Friday 29 July 2011 22:45, Benjamin Smith wrote:
I have a tested copy of EL6 that I would like to duplicate to a
number of similar servers, but can't seem to find a sane howto on the
subject. TLDP's "Hard disk upgrade howto" is embarrassingly
antiquated: when's the last time you saw LILO?
When I wrote that How-To, I wanted to copy Linux from a 120 Mb hard disk
to a 250 Mb hard disk.
I've thought about updating the How-To, but I assumed that modern hard
disks are so large that people no longer needed to copy Linux just
because they ran out of disk space.
Even back then, the How-To wasn't intended for someone who wanted to
make several copies. As others have suggested, Clonezilla seems
appropriate for your situation: http://clonezilla.org/
I still use LILO to boot from thumb drives, by the way.
I feel like I'm speaking to an old friend. =) I did several upgrades with that guide until RH switched to grub. It's been painful ever since. :/ Mostly, I throw in new drives and mount them all over the place. I have systems with a 20 GB / partition with up to 10 TB of drives and partitions crufted all over the place. (yuk!)
My issue is that the instructions for LILO don't do much for me with respect to grub. There's numerous howtos mention hdparms and partition numbers and chroot and stuff, but nothing that spells it out for a dolt like me.
... and nobody here's suggested a recommended way to do this. I would *love* it if somebody who knew could ammend your excellent howto with information on grub. (which has been the RHEL/CentOS default for several years)
I tried using Clonezilla to clone a 300 GB SAS drive to a 1 TB SATA drive. I don't care *at* *all* if there's some wasted space - I just want it to *work*. But it thew a screen full of random textual garbage on the screen and not much helpful information when I hit the "go" button.
Have you tried "dd if=<old-drive> of=<new-drive> bs=4096"? That may be slow but it usually is the best bet to get an exact copy of the old disk (the new disk must at least have the same size as the old one of course).
Regards, Dennis
On Tuesday 02 August 2011 00:16, Benjamin Smith lists@benjamindsmith.com wrote:
... and nobody here's suggested a recommended way to do this. I would *love* it if somebody who knew could ammend your excellent howto with information on grub. (which has been the RHEL/CentOS default for several years)
I did try at one point to write a "Grub Mini How-To" to complement the Hard Disk Upgrade Mini How-To. I gave up because I was having too much trouble trying to work out all possible situations, but I'll send you what I wrote at that time by E-mail.
Yves
On Tuesday, August 02, 2011 02:22:49 PM Yves Bellefeuille wrote:
On Tuesday 02 August 2011 00:16, Benjamin Smith
lists@benjamindsmith.com wrote:
... and nobody here's suggested a recommended way to do this. I would *love* it if somebody who knew could ammend your excellent howto with information on grub. (which has been the RHEL/CentOS default for several years)
I did try at one point to write a "Grub Mini How-To" to complement the Hard Disk Upgrade Mini How-To. I gave up because I was having too much trouble trying to work out all possible situations, but I'll send you what I wrote at that time by E-mail.
Yves
That would be just awesome. Thanks!
-Ben
On Tue, 2011-08-02 at 17:22 -0400, Yves Bellefeuille wrote:
On Tuesday 02 August 2011 00:16, Benjamin Smith lists@benjamindsmith.com wrote:
... and nobody here's suggested a recommended way to do this. I would *love* it if somebody who knew could ammend your excellent howto with information on grub. (which has been the RHEL/CentOS default for several years)
I did try at one point to write a "Grub Mini How-To" to complement the Hard Disk Upgrade Mini How-To. I gave up because I was having too much trouble trying to work out all possible situations, but I'll send you what I wrote at that time by E-mail.
How about including that on the Centos Wiki ?
I just wasted many hours with a Grub problem.
On 8/1/2011 11:16 PM, Benjamin Smith wrote:
I tried using Clonezilla to clone a 300 GB SAS drive to a 1 TB SATA drive. I don't care *at* *all* if there's some wasted space - I just want it to *work*. But it thew a screen full of random textual garbage on the screen and not much helpful information when I hit the "go" button.
(sigh)
I've used clonezilla frequently and have never seen that kind of failure (although I haven't tried it on an ext4 filesystem yet). That sounds like a bad download or iso burn to me. Did you verify the md5sum? Also, it is usually safest to use the latest available 'alternative-stable' (ubuntu based) release which would be the iso from: http://clonezilla.org/downloads/alternative/iso-zip-files.php
If you have time to experiment, it would also be interesting to see how you fare with 'rear', which you can install via yum from epel. You run it on your working system and it creates a bootable iso that knows how to reconstruct the existing filesystem (lvm/md/etc.) layout and restore a backup (of various types). The documentation leaves a lot to be desired: http://rear.sourceforge.net/ but it does seem to understand more about linux quirks than anything similar that I've seen.
-
Always Learning -
Benjamin Smith -
Dennis Jacobfeuerborn -
Lamar Owen -
Les Mikesell -
Robert Heller -
Yves Bellefeuille