I have a 500GB Sata drive about 15% used I would like to make an exact copy of too another Sata 500GB drive as a spare. That way if something happens to the one in service I can plug in the spare quickly and restore one of the weekly backups without reinstalling the entire OS and all the little tweaks of setup on this mail/web server.
How do I do this? That is make an exact bootable copy of a linux drive. Its running Centos 4.6 if that matters.
Matt
Matt wrote:
I have a 500GB Sata drive about 15% used I would like to make an exact copy of too another Sata 500GB drive as a spare. That way if something happens to the one in service I can plug in the spare quickly and restore one of the weekly backups without reinstalling the entire OS and all the little tweaks of setup on this mail/web server.
How do I do this? That is make an exact bootable copy of a linux drive. Its running Centos 4.6 if that matters.
brute force approach...
dd if=/dev/sda of=/dev/sdb bs=16384
that will copy the whole physical drive. I would do this in single user mode (init 1) for best results, as any changes to the file system while a copy is in progress will be inconsistent
brute force approach...
dd if=/dev/sda of=/dev/sdb bs=16384
That's probably the most safest, but you'll have to remove it before you boot if you use lvm right?
Doesn't clonzilla support lvm? Could you boot off a live cd to only copy actual data which should be quick in your case?
jlc
Joseph L. Casale wrote:
brute force approach...
dd if=/dev/sda of=/dev/sdb bs=16384
That's probably the most safest, but you'll have to remove it before you boot if you use lvm right?
Doesn't clonzilla support lvm? Could you boot off a live cd to only copy actual data which should be quick in your case?
Clonezilla will handle LVMs, but I'm not sure if it is happy with cloned identifiers done disk<->disk in the same machine. It may drop back to dd for that anyway. If you have space on the network to store the image, let clonezilla copy to the network from the source box and clone it to disk on a different one. Unless the contents are extremely critical you probably don't need to clone back to a real disk anyway. That part would only take a few extra minutes when you need it.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org]On Behalf Of Les Mikesell Sent: Friday, August 01, 2008 3:16 PM To: CentOS mailing list Subject: Re: [CentOS] Mirroring Hard Drive
Joseph L. Casale wrote:
brute force approach...
dd if=/dev/sda of=/dev/sdb bs=16384
That's probably the most safest, but you'll have to remove it before you boot if you use lvm right?
Doesn't clonzilla support lvm? Could you boot off a live cd to only copy actual data which should be quick in your case?
Clonezilla will handle LVMs, but I'm not sure if it is happy with cloned identifiers done disk<->disk in the same machine. It may drop back to dd for that anyway. If you have space on the network to store the image, let clonezilla copy to the network from the source box and clone it to disk on a different one. Unless the contents are extremely critical you probably don't need to clone back to a real disk anyway. That part would only take a few extra minutes when you need it.
I did this the other day with clonezilla and it worked great! It only took 15-20 minutes. I put the new clone in a different box and now use rsynce to keep things current. If you wanted to keep the drives in the same box I'm sure theres a way, with linux there always is :) Anyone?
Dan
<snip>
If you wanted to keep the drives in the same box I'm sure theres a way, with linux there always is :) Anyone?
Well, if the disc used LVM, you would obviously have to rename the VG/LV's or there would be duplicates. How about the GUID?
At any rate, I think the OP didnt have this in mind so now we are hijacking his thread.
jlc
I have a 500GB Sata drive about 15% used I would like to make an exact copy of too another Sata 500GB drive as a spare. That way if something happens to the one in service I can plug in the spare quickly and restore one of the weekly backups without reinstalling the entire OS and all the little tweaks of setup on this mail/web server.
How do I do this? That is make an exact bootable copy of a linux drive. Its running Centos 4.6 if that matters.
brute force approach...
dd if=/dev/sda of=/dev/sdb bs=16384
that will copy the whole physical drive. I would do this in single user mode (init 1) for best results, as any changes to the file system while a copy is in progress will be inconsistent
Can I use this to copy from a single 500GB SATA drive with only 54GB in use to a hardware RAID 1 drive of 300GB? The RAID will be two 300GB SATA drives.
Matt
Matt wrote:
I have a 500GB Sata drive about 15% used I would like to make an exact copy of too another Sata 500GB drive as a spare. That way if something happens to the one in service I can plug in the spare quickly and restore one of the weekly backups without reinstalling the entire OS and all the little tweaks of setup on this mail/web server.
How do I do this? That is make an exact bootable copy of a linux drive. Its running Centos 4.6 if that matters.
brute force approach...
dd if=/dev/sda of=/dev/sdb bs=16384
that will copy the whole physical drive. I would do this in single user mode (init 1) for best results, as any changes to the file system while a copy is in progress will be inconsistent
Can I use this to copy from a single 500GB SATA drive with only 54GB in use to a hardware RAID 1 drive of 300GB? The RAID will be two 300GB SATA drives.
no, as dd is a raw block copy of the storage device. i dont actually recommend usinig DD on file systems at all
instead, assuming the raid is freshly formatted, and temporarily mounted as /mnt I would use something like...
dump 0vf - /dev/sda1 | (cd /mnt; restore -rf - )
if there's more than one file system on the source, repeat this for each one. note that the source file systems must be unmounted when you do this, hence you would need to do this from a CD boot if its the system drive.
On Sun, 2008-12-07 at 12:36 -0800, John R Pierce wrote:
Matt wrote:
I have a 500GB Sata drive about 15% used I would like to make an exact copy of too another Sata 500GB drive as a spare. That way if
<snip>
no, as dd is a raw block copy of the storage device. i dont actually recommend usinig DD on file systems at all
instead, assuming the raid is freshly formatted, and temporarily mounted as /mnt I would use something like...
dump 0vf - /dev/sda1 | (cd /mnt; restore -rf - )if there's more than one file system on the source, repeat this for each one. note that the source file systems must be unmounted when you do this, hence you would need to do this from a CD boot if its the system drive.
I've always wondered why so many folks use the above construct when a _fast_ less "expensive" solution STM to be something like
cd <your source mount point> ; find . <other params you desire> \ | cpio -p<other params you want> /mnt
Am I missing something? Just old fashioned? Cpio has all the params you want and can be _very_ fast with the righ parameters. Similar to the above dump/restore set I've seen many use tar/untar equivalents.
Just curious is all.
<snip sig stuff>
William L. Maltby wrote:
On Sun, 2008-12-07 at 12:36 -0800, John R Pierce wrote:
Matt wrote:
I have a 500GB Sata drive about 15% used I would like to make an exact copy of too another Sata 500GB drive as a spare. That way if
<snip>
no, as dd is a raw block copy of the storage device. i dont actually recommend usinig DD on file systems at all
instead, assuming the raid is freshly formatted, and temporarily mounted as /mnt I would use something like...
dump 0vf - /dev/sda1 | (cd /mnt; restore -rf - )if there's more than one file system on the source, repeat this for each one. note that the source file systems must be unmounted when you do this, hence you would need to do this from a CD boot if its the system drive.
I've always wondered why so many folks use the above construct when a _fast_ less "expensive" solution STM to be something like
cd <your source mount point> ; find . <other params you desire> \ | cpio -p<other params you want> /mnt
Am I missing something? Just old fashioned? Cpio has all the params you want and can be _very_ fast with the righ parameters. Similar to the above dump/restore set I've seen many use tar/untar equivalents.
Or, on everything that has gnu cp (which would be at least every linux distro), 'cp -a . /mnt' should work. However, I usually use rsync since you can stop and restart keeping the completed work or repeat to get updates, and it works the same over ssh if the drive in question is on a different machine.
On Sun, 2008-12-07 at 16:04 -0600, Les Mikesell wrote:
William L. Maltby wrote:
<snip>
Am I missing something? Just old fashioned? Cpio has all the params you want and can be _very_ fast with the righ parameters. Similar to the above dump/restore set I've seen many use tar/untar equivalents.
Or, on everything that has gnu cp (which would be at least every linux distro), 'cp -a . /mnt' should work. However, I usually use rsync since you can stop and restart keeping the completed work or repeat to get updates, and it works the same over ssh if the drive in question is on a different machine.
Yep. I've recently began using rsync for several types of "local" copy, usually back-up related. I can't recall if the "cp -a" detects and handles hard-links to minimize space requirements though. I know cpio can/does. I guess I'll have to read up on cp some more and see if it leaves the access times alone (cpio parameter allows retaining that) and handles hard-links efficiently.
William L. Maltby wrote:
On Sun, 2008-12-07 at 16:04 -0600, Les Mikesell wrote:
William L. Maltby wrote:
<snip>
Am I missing something? Just old fashioned? Cpio has all the params you want and can be _very_ fast with the righ parameters. Similar to the above dump/restore set I've seen many use tar/untar equivalents.
Or, on everything that has gnu cp (which would be at least every linux distro), 'cp -a . /mnt' should work. However, I usually use rsync since you can stop and restart keeping the completed work or repeat to get updates, and it works the same over ssh if the drive in question is on a different machine.
Yep. I've recently began using rsync for several types of "local" copy, usually back-up related. I can't recall if the "cp -a" detects and handles hard-links to minimize space requirements though. I know cpio can/does. I guess I'll have to read up on cp some more and see if it leaves the access times alone (cpio parameter allows retaining that) and handles hard-links efficiently.
Rsync -a covers most options except hardlinks - you need to add -H for that because it adds significant overhead to track them.
2008/12/8 William L. Maltby CentOS4Bill@triad.rr.com:
Yep. I've recently began using rsync for several types of "local" copy, usually back-up related. I can't recall if the "cp -a" detects and handles hard-links to minimize space requirements though. I know cpio
Yes, it seems that "cp -a" is designed just for that kind of job. Might have to add "-x" to limit it to one file system if you are interested.
I noticed that, to my surprise, rsync is sometimes faster than a plain scp even when the destination is empty, and as someone else said it's nice to be able to stop/start and redo.
can/does. I guess I'll have to read up on cp some more and see if it leaves the access times alone (cpio parameter allows retaining that) and handles hard-links efficiently.
I'm not sure why you should care about atime so much - more and more people around (including Linus Torvalds) recommend to get rid of it altogether. Ubunut comes with "relatime" as a default config already.
According to Linus, disabling atime updates will give the single largest performance gain (in dozens of percentages, as far as I remember).
But back to the question - am I missing something too by not using dump/restore or cpio? dump/restore is so BSD 4/'80's :)
Cheers,
--Amos
On Mon, 2008-12-08 at 18:55 +1100, Amos Shapira wrote:
2008/12/8 William L. Maltby CentOS4Bill@triad.rr.com:
<snip>
can/does. I guess I'll have to read up on cp some more and see if it leaves the access times alone (cpio parameter allows retaining that) and handles hard-links efficiently.
I'm not sure why you should care about atime so much - more and more people around (including Linus Torvalds) recommend to get rid of it altogether. Ubunut comes with "relatime" as a default config already.
Atime seems to have utility for determining when items may be candidates for removal due to lack of interest (if it is not updated by utilities such as backup processes). I suppose some security activities might make use of it in forensic processes? How about profiling FS activities? Other than those (rather inconsequential?) items, not much use.
In reality, being raised on real UNIX(TM) systems from long ago and far away, it was just one of the things we wanted left unchanged when we did backups or shipped tapes to the outside world (one of my many jobs back then). There is the possibility that atime was tracked "because we can".
But with performance being what it was back then, profiling for performance tuning seems the most likely supportable reason for having it. Keep in mind that tools such as SAR didn't exist initially. On my systems back then, I would order directories and files with a mix of access times, update profiles, etc. to achieve unexpected performance. Combined with FS tunables, one could support a lot more users than expected on an old 386/486/586 systems with _teeny_ amounts of memory.
Never attempted multi-user on 286 systems. I think it would have been a waste of effort.
<snip>
--Amos
<snip sig stuff>
2008/12/8 William L. Maltby CentOS4Bill@triad.rr.com:
In reality, being raised on real UNIX(TM) systems from long ago and far away, it was just one of the things we wanted left unchanged when we did backups or shipped tapes to the outside world (one of my many jobs back then). There is the possibility that atime was tracked "because we can".
I've been there too. My first UNIX account was on a Vax 750 running BSD 4.2 in 1986, which after a year I started to manage (that's why I said that dump/restore "are so '80's" :).
Times changed. I believe the reasoning which says that atime is a waste of time (pun intended :). Move on.
Cheers,
--Amos
Or you could boot up with knoppix or some other livecd so the filesystem is not in use and mount both drives and do a:
mkdir /mnt/org mount /dev/hdx /mnt/org mkdir /mnt/bckup mount /dev/hdx /mnt/bckup
cp -af /mnt/org/* /mnt/bckup/.
umount both drives
then copy mbr
dd if=/dev/hdx of=/dev/hdx bs=512 count=1
I recalled this from memory but I don''t think I left anything out, but your mileage may very, at this point you could replace your orginal with your backup and it should boot just fine. I use this method to copy/resize new virtual images and works quite good and fast.
On Fri, Aug 1, 2008 at 2:27 PM, Matt lm7812@gmail.com wrote:
I have a 500GB Sata drive about 15% used I would like to make an exact copy of too another Sata 500GB drive as a spare. That way if something happens to the one in service I can plug in the spare quickly and restore one of the weekly backups without reinstalling the entire OS and all the little tweaks of setup on this mail/web server.
How do I do this? That is make an exact bootable copy of a linux drive. Its running Centos 4.6 if that matters.
Matt _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Or you could boot up with knoppix or some other livecd so the filesystem is not in use and mount both drives and do a:
mkdir /mnt/org mount /dev/hdx /mnt/org mkdir /mnt/bckup mount /dev/hdx /mnt/bckup
cp -af /mnt/org/* /mnt/bckup/.
Won't this command choke if there are too many files? I think I have run into that before.
Matt
umount both drives
then copy mbr
dd if=/dev/hdx of=/dev/hdx bs=512 count=1
Matt (lm7812@gmail.com) kirjoitteli (17.11.2008 18:42):
mkdir /mnt/org mount /dev/hdx /mnt/org mkdir /mnt/bckup mount /dev/hdx /mnt/bckup
cp -af /mnt/org/* /mnt/bckup/.
Won't this command choke if there are too many files? I think I have run into that before.
If it does, here's a way round (source: see the thread "[CentOS] ls and rm: "argument list too long"").
for i in /mnt/org/* do cp $i /mnt/bckup/ done
- Jussi -- Jussi Hirvi * Green Spot Topeliuksenkatu 15 C * 00250 Helsinki * Finland Tel. & fax +358 9 493 981 * Mobile +358 40 771 2098 (only sms) jussi.hirvi@greenspot.fi * http://www.greenspot.fi
Matt wrote:
I have a 500GB Sata drive about 15% used I would like to make an exact copy of too another Sata 500GB drive as a spare. That way if something happens to the one in service I can plug in the spare quickly and restore one of the weekly backups without reinstalling the entire OS and all the little tweaks of setup on this mail/web server.
How do I do this? That is make an exact bootable copy of a linux drive. Its running Centos 4.6 if that matters.
stupid question ... why not just put it in the machine and make it a raid1 mirror
then, if the first one dies, you just use the second one :D
I have a 500GB Sata drive about 15% used I would like to make an exact copy of too another Sata 500GB drive as a spare. That way if something happens to the one in service I can plug in the spare quickly and restore one of the weekly backups without reinstalling the entire OS and all the little tweaks of setup on this mail/web server.
How do I do this? That is make an exact bootable copy of a linux drive. Its running Centos 4.6 if that matters.
stupid question ... why not just put it in the machine and make it a raid1 mirror
then, if the first one dies, you just use the second one :D
How do you do that?
Matt
Matt wrote:
why not just put it in the machine and make it a raid1 mirror
then, if the first one dies, you just use the second one :D
How do you do that?
Detailed step by step instructions easily modified for CentOS:
http://www.howtoforge.com/software-raid1-grub-boot-debian-etch
I haven't tried this myself ... yet but plan on it in the next few weeks.
Paul R. Ganci wrote:
Matt wrote:
why not just put it in the machine and make it a raid1 mirror
then, if the first one dies, you just use the second one :D
How do you do that?
Detailed step by step instructions easily modified for CentOS:
http://www.howtoforge.com/software-raid1-grub-boot-debian-etch
I haven't tried this myself ... yet but plan on it in the next few weeks.
I haven't tried it either...yet... but there is also a version of the HOWTO for Fedora 8, which might require less interpolation. http://www.howtoforge.com/software-raid1-grub-boot-fedora-8
Thanks for the URL
Robert wrote:
Paul R. Ganci wrote:
Matt wrote:
why not just put it in the machine and make it a raid1 mirror
then, if the first one dies, you just use the second one :D
How do you do that?
Detailed step by step instructions easily modified for CentOS:
http://www.howtoforge.com/software-raid1-grub-boot-debian-etch
I haven't tried this myself ... yet but plan on it in the next few weeks.
I haven't tried it either...yet... but there is also a version of the HOWTO for Fedora 8, which might require less interpolation. http://www.howtoforge.com/software-raid1-grub-boot-fedora-8
Thanks for the URL
I should have had the common decency to report that I *did* try this howto and *was* successful. There were a couple things that caused some head-scratching.
1. I read somewhere that it's safest to resize the filesystems on the existing drive before doing anything else, to allow for a 4K superblock beginning on a 64k boundary at the end of the partition. I did that. (Straightforward instructions at http://lists.centos.org/pipermail/centos/2006-April/063687.html)
2. In part 2, page 7, there is a step "Next replace LABEL=/boot with /dev/md0 and LABEL=/ with /dev/md2 in /etc/mtab" that I kinda questioned. It was my understanding that /etc/mtab is maintained by the mount command. (From man mount: "The programs mount and umount maintain a list of currently mounted file systems in the file /etc/mtab".) This makes the command on page 9, "cp -dpRx / /mnt/md2" *appear* to be copying md2 to itself. Confusion aside, the command has the desired result.
Aside from those 2 points, it went very smoothly.
Robert wrote:
I should have had the common decency to report that I *did* try this howto and *was* successful. There were a couple things that caused some head-scratching.
Yes I also made it work this weekend. I ignored all the stuff regarding the creation of /etc/mtab and mdadm.conf ... they are not needed.
I also took the precaution of entering single user mode (init 1) before copying the file system, etc. The rationale for doing this first is so that the system is not writing anything while or after you do the copy. I read this from this URL
http://www.linuxconfig.org/Linux_Software_Raid_1_Setup
which describes the same basic procedure.
Also I am not sure that all the grub installs are necessary. I thought that once grub was installed it was unnecessary to re-install. It is only necessary to change the /boot/grub/grub.conf and the changes take effect. Similarly with the ramdisk. I am not sure it is necessary to keep running mkinitrd to create a new ramdisk just because the grub.conf changed. Can somebody who might know for sure comment? I admit I followed the instructions regarding the running of grub/mkinitrd to the letter only because I was chicken to do elsewise.
Ironically I discovered that the new raid drive (i.e. /dev/sdb from the documentation) was defective. When trying to sync with /dev/sda after I added it to the array the sync hit some bad blocks on the /dev/sdb and so the sync failed. The system went into an infinite loop of sorts (fail sync, try again, fail sync, try again). Fortunately it didn't just drop /dev/sdb and I was able to reverse the process to get everything back onto /dev/sda, replaced /dev/sdb with a different drive and tried again. The second time worked without any hitches. How I didn't loose any data is beyond me ... sometimes it is better to be lucky than good.
Other than to make sure your drives are good the procedure described in the links of this thread work very nicely.
On Sat, Aug 2, 2008 at 4:00 PM, Johnny Hughes johnny@centos.org wrote:
stupid question ... why not just put it in the machine and make it a raid1 mirror
cuz it's not an md volume to begin with?
Bent Terp wrote:
On Sat, Aug 2, 2008 at 4:00 PM, Johnny Hughes johnny@centos.org wrote:
stupid question ... why not just put it in the machine and make it a raid1 mirror
cuz it's not an md volume to begin with?
one method is to make the 2nd disk a 'half' mirror, with all the same partitions, then use dump|restore to copy your file systems over to it, fix up the grub boot on it, edit teh fstab on this new volume to refer to the md sets, then swap it and make it the primary drive, and add the original drive as the mirror.
if uptime is your goal, don't forget to mirror the swap too. too many people seem to think thats uneccessary, but if a drive containing unmirrored swap dies, the system can only crash and burn (eg, panic).
now, do note, a mirror is ONLY good for protecting against physical drive failure, and maintaining uptime, its /NOT/ a substitute for backups, as a software glitch, or user error can wipe the mirror out just as quick as can be.
-
Amos Shapira -
Bent Terp -
Dan Carl -
John R Pierce -
Johnny Hughes -
Joseph L. Casale -
Jussi Hirvi -
Les Mikesell -
Matt -
Paul R. Ganci -
Robert -
Tom Bishop -
William L. Maltby