ACID-compliant filesystem (was:Re: [CentOS] Re: centos] 4.4 upgrade problems)

[Les Mikesell]

On Sat, 2006-09-09 at 16:57, Lamar Owen wrote:

> In the general case, I'd like to issue something like:
> 
> # acidfs-begin-transaction
> # yum -y update
> [bunch of output]
> # if yum-no-error-condition;
> #    acidfs-commit
> # else 
> #    acidfs-rollback
> # fi

Isn't that what LVM snapshots are supposed to provide?

> > Yum is managing the 
> > whole system.  
> 
> It is currently managing the whole filesystem.  But what about in-memory 
> program text?

That's up to the rpm layer and the scripts contained in the
packages.  Yum doesn't know much about that other than
dependencies.

> The currently loaded programs need to continue to have the 
> older libs available if needed;

This would not be possible without massive changes in the way
RPM works.  The new installs won't happen if they can't see
their dependent libs.

>  a single 'commit' operation at the end needs 
> to atomically reload program text and provide consistent library dependencies 
> post-run (the only way you can do this now is shut down the system, boot 
> rescue media, and yum update to a chroot (which is the system), then reboot 
> the system (aka 'booting an update CD'; anaconda does this quite well)).  The 
> in core text needs to be isolated from what's going on on the filesystem, and 
> if an error occurs (out of space, for instance, or a locked file) an atomic 
> rollback would be very nice (anaconda does not do this, though).

The only hope would be to make the equivalent of a virtual machine
where the old system keeps running until the new one is
completely constructed.

> > Given an ACID database, would you expect to be 
> > able to upgrade it to a new, potentially incompatible version of
> > an ACID database with transactions in progress? 
> 
> Of course I would; a filesystem/database (hmm, let's see, similar to, but 
> farther beyond what NILFS claims) would have to be guaranteed backwards 
> compatible. 

Backwards compatibility?  You seem to have confused Linux
distributions with something else.  Try, for example, to
copy a Centos 3.x distro onto filesystems created by
4.x and make it boot.

>  That's a given for a filesystem; it is unreasonable for the 
> authors of a filesystem to introduce such changes and expect seamless updates 
> of the filesystem code itself.

Are filesystem authors required to be reasonable?

> > What yum needs 
> > is just some special consideration when modifying its own
> > components.
> 
> Ok, let me repeat: I updated the yum components in the right order.  This is 
> NOT the sqlite/python-sqlite issue.  In my case, after following the advice 
> of updating python-sqlite, then sqlite, then yum and its dependencies, I got 
> a system with a lot of dupes and out of sync libraries.  Lots of out of sync 
> libraries; I've not finished the recovery yet, although bind at least is 
> working.

I think I'm missing something here.  If yum itself or the rpm
database wasn't broken, what went wrong? 

>   If there were a 'yum rollback'  (or a 'acidfsrollback 
> snapshot-prior-to-yum') then I could at least try it again.  But there isn't, 
> and I know there isn't, and I know all the deal about no support, etc etc.  

But you've made a big assumption here that yum itself would
work properly while doing this.  If yum was working right
you wouldn't have dups now.

> Just throwing an idea out, that's all, for discussion.  This systemic 
> non-atomicity and inconsistency is endemic to all linuxen at the moment.

And necessarily so, since each rpm package installs independently
and may have to complete with both process and filesystem changes
before some of the others will work.

-- 
  Les Mikesell
   lesmikesell at gmail.com