Hello,
we have several physical servers (CentOS 5.*) with rather critical applications where (because of stability) we don't do regularly 'yum update'. In virtualized environemnts (under Vmware) we do a snapshot, then 'yum update', reboot and if something is wrong we rollback the snapshot. On physical servers we cannot do that. I have read about rollback option of rpm but not sure if this is reliable solution. What is your best practise regarding "rollbacking" 'yum update' on physical servers ?
Regards Przemyslaw Bak (przemol)
On 06/22/2012 09:42 AM, przemolicc@poczta.fm wrote:
Hello,
we have several physical servers (CentOS 5.*) with rather critical applications where (because of stability) we don't do regularly 'yum update'. In virtualized environemnts (under Vmware) we do a snapshot, then 'yum update', reboot and if something is wrong we rollback the snapshot. On physical servers we cannot do that. I have read about rollback option of rpm but not sure if this is reliable solution. What is your best practise regarding "rollbacking" 'yum update' on physical servers ?
One solution would be to use lvm snapshots. Create a snapshot of the root volume (lvcreate -s VolGroup00/LogVol00 -n rootsnapshot -L 10G), do an update and see if it works. If not, boot into rescue mode and copy the content from the snapshot back to the original. Make sure the snapshot gets the same size as the original volume. This is important if you want to copy back all the data.
Theo
On 22/6/2012 2:06 μμ, Theo Band wrote:
What is your best practise regarding "rollbacking" 'yum update' on physical servers ?
Assuming that you have problems due to a particular newly-installed package, you can downgrade:
rpm -Uvh --oldpackage package-2.4.0-1.el5.x86_64.rpm
or:
yum downgrade package (if it is in the repo)
Since problems are usually introduced by a particular package (and not by all), this might be enough in most cases.
Note, however, that dependencies are not resolved automatically with the above commands, so they must be handled manually.
Another, more complete solution, of course, would be to have a full system backup (regardless whether the system is physical or virtual) and in case things go wrong, restore from backup (always a bit risky, I know - it makes you feel uneasy). We use mondorescue without problems (see: http://www.mondorescue.org/). I have been able to even use the backup to restore a KVM guest (using LVM) under VMware (you may see http://mondorescue-mailing-list.679749.n3.nabble.com/Mondo-devel-Restore-fro...).
Or - if feasible - you can attempt to virtualize your physical server, either using mondorescue or VMware converter (or other commercial tools) and be ready to use the virtual machine instead.
I am interested on other solutions too, so your thread is interesting!
Nick
On 06/22/2012 01:58 PM, Nikolaos Milas wrote:
I am interested on other solutions too, so your thread is interesting!
dump
Assuming some form of ext[n] filesystem is being used. It has the advantage that is also works with incremental backups. You can dump the root file system and perhaps also the /boot filesystem. Instead of the root filesystem, I dump a snaphost that is created each evening. The snapshot has a frozen filesystem. So databases (mysql) should just be consistent. If you want to be 100% sure, stop the database, make a snapshot and start the database again. This is done within one second, so hardly any impact on the live server. Disadvantage of this method is that you still need the have a partition table if you need to fully restore. And LVM and boot sector need to be recreated. In case of disaster recovery you need this documented properly (try it out at least once). Advantage is that you can have daily system backups automatically created.
Theo
-
Nikolaos Milas -
przemolicc＠poczta.fm -
Theo Band