[CentOS] schily tools

[Les Mikesell]

On Tue, Feb 14, 2012 at 12:41 PM, Joerg Schilling
<Joerg.Schilling at fokus.fraunhofer.de> wrote:
>>
>> Gtar doesn't fail, it just requires some extra steps to accommodate some
>> very unlikely circumstances.  Your star runs will not work at all in very
>> likely circumstances (which you conveniently avoid testing).
>
> Trying to understand you leads me to the assumption that you of course know
> that gtar fails. As you seem to claim that there is a workaround but don't
> mention it, it seems that you like other people to fail when trying to restore
> incremental backups that have been made with gtar.

The only thing I've ever seen fail was where the full or earlier
incremental contained a directory which was subsequently replaced with
a plain file with the same name that was included in a higher level
incremental.  The gnutar code circa 2004 didn't correctly remove the
directory that should be overwritten during the incremental restore.
But the error was obvious and you already know how to remove a
directory.  Do it, repeat the restore of the missing item and everyone
is happy.   But that was a long time ago, maybe things have changed.

> Well, I did one test only and I know that gtar does not do what it should when
> restoring incrementals.

So you don't check for changes either?

> If you e.g. have a 1 TB filesystem with one directory at top level and rename
> that directory, you will end up with a 1 TB incremental if you use gtar.
> Looking at the way gtar works, I would assume that gtar needs an intermediate
> disk space of 2 TB to restore the data (if the restore works at all).
>
> Star will create an incremental archive of 10kB for the same change and star
> does not need additional space on the restore media.

Except that star won't do it at all if you want it to follow an
arbitrary directory tree.

>> Our definitions of 'OS and FS independent'  are very different.  I expect
>> that to mean that I can actually change the FS and have things keep
>> working.  Star doesn't.  And it doesn't work over nfs, on fat, ntfs, and on
>> and on.
>
> Do you like to tell everbody that it makes no sense to believe you, or why do
> you write claims that are obviously wrong?

Can it follow directory trees across mount points getting incrementals
right?  If I'm wrong, show me the commands needed to make that work.

> Star of course allows you to backup parts of a filesystem (e.g. a directory
> sub-tree).

And follow across mount points?

>> In a word, amanda.   If my backups don't fit on the tape there are no
>> backups at all, and the only reason for doing incrementals at all is that
>> you have to use them to make the data fit. I'm not interested in
>> calculating the right mix of incremental levels on a whole bunch of
>> directory trees to make each night's run fit.  Amanda does that
>
> You again harm your credibility by writing false suggestions.
>
> Star is able to automatically deal with the unpredictable size of modern tapes
> with HW compression, while amanda fails completely and needs to assume the
> smallest possible size. This is because amanda does not make use of the related
> features of the backup utility. Star can detect the actual end of a tape in
> write mode on the fly and then request a media change.

I don't want a media change.  No one is going to be there to change
it. I want the software to compute the mix of fulls and incrementals
that will fit my single daily tape.   Amanda did that faithfully for
many years..

>> And then there was the fact that at the time I implemented amanda, star did
>> not do incremental restores anyway...
>
> Star implements reliable incremental restores since 7 years, so you seem to be
> living in a long gone past.

Yes, I believed what you said here:
http://blog.gmane.org/gmane.comp.archivers.star.user/month=20040701
and haven't had any more reason to check again than you have to check gnutar.
Amanda kept doing what I needed until the tape drives wore out many
years later, with no more attention than swapping the tape sometime
during the day.

> What you like to do with gtar may require hours of manual intervention and is
> still based on an unreliable basic method (as you cannot use the needed
> snapshots).

There is nothing that would prevent you from using snapshots with tar,
but a filesystem snapshot isn't really enough to ensure that your
application's view of the data is consistent anyway.  Most things
don't care and the ones that do have application-level ways to dump
their data consistently.

> You are trying to do something in an unrelibale way that can be done reliably
> and fully automatically using star instead of gtar. So why do you still use
> gtar?

Because when I needed it, star wasn't even usable.   Even if it had
been able to do an incremental restore, it didn't work with amanda.

>> So first we need room for filesystem snapshots, but now storing a file big
>> enough to hold the equivalent of a directory listing is a problem?
>
> There of course is a difference between the need to store some data for a few
> minutes (this is what an incremental star backup takes) and the need to store
> a lot of data for the whole life-cycle of the FS (which is what you need to do
> with gtar).

What good does being able to release that space do if you have to
reserve it anyway?   I don't see how it has any other use.

>> Can you, using star, make incremental backups of a directory tree that work
>> without knowing./caring what file systems hold that tree or where the roots
>> or mount points are?
>
> Of course, if there are no mount points in that tree.

And if there are?   I don't see why you consider gnutar's issue with a
directory being replaced  with a file of the same name to be a big
problem and completely ignore the issue star has with a directory
being replaced by a mount point.    I've done the latter much more
often than the former.  It's not a reason to make backups of the file
contents fail.

>> I didn't see an option for reporting size in the man page - it might
>> actually be adaptable if anyone still cared about tapes.  But,  I find the
>> man page very confusing.  Most of the incremental options say it has to be
>> a single Posix-conforming FS but the -dump section talks about spanning
>> more than one.
>
> If you did never work with star, why do you claim so many "failures" of star?

One failing is enough.  Isn't that why you dismissed gnutar?  And for
me, being unable to do incrementals that cross mount points is a such
a failing.  And if I understand it correctly, even if I kept changing
the backup runs to match the source mount layouts,  I would not be
able to restore correctly if I wanted the replacement system to have a
different topology that would place a mount point in a location that
was previously a directory .   Did I misunderstand that?

> I recommend you to start using star and talk about it after you learn how it
> works. Star implements so many features, that you need to use it for some days
> to understand the concept, but I know nobody who did not switch to star after
> understanding the concept.

I hope to never need tapes again, except possibly for archival storage
where incrementals would not be a factor.   And going to disk targets,
rsync is much nicer than either gnutar or star.

-- 
   Les Mikesell
     lesmikesell at gmail.com