[CentOS] schily tools

Tue Feb 7 18:03:42 UTC 2012
Joerg Schilling <Joerg.Schilling at fokus.fraunhofer.de>

Les Mikesell <lesmikesell at gmail.com> wrote:

> On Tue, Feb 7, 2012 at 9:26 AM, Joerg Schilling
> <Joerg.Schilling at fokus.fraunhofer.de> wrote:
> >
> > When I implemented incremental restores for star in September 2004, I wrote a
> > simple script for a incremental testcase and tested the deltas with
> > ufsdump/ufsrestore, gtar and the star version at that time. Gtar was unable to
> > deal with my testcase, so I stopped testing it any further.
> My testcase for star was moving a subdirectory of what my backup runs
> covered onto a mounted volume.  Star failed and I stopped testing it
> any further.

So you tried to do something that cannot work for a filesystem oriented program.
This is not a problem from star but you just did not use star the right way.

Note that gtar uses a big database that is needed at the dump side and that 
still does not hold enough data to let gtar work correctly.

Star on the other side just needs to remember dump dates and creates a full 
data base et extract side when doing incremental restores. This is also more 
reliable than what gtar does as all needed information for the restore is in 
the archives.

> > If you like to discuss incrementals, you definitely need to discuss behavior at
> > restore time and restoring incrementals definitely does not work correclty with
> > gtar if you renamed directories.
> If that is the issue I recall, you can recover without losing data.

You are correct, you could in theory recover the data but this must be done 

> > Star is used to do incremental backups/restores on a dayly base in Berlios
> > since September 2004. Since Spring 2005, not a single problem was seen, so
> > there are more that 2500 successful incremental restores that verify no problem
> > even under stange conditions.
> So it all that time you have not mounted a new volume somewhere?  No
> remote backups of hosts where someone else might add space?  No one
> ever wanted to restore onto a different mount topology than the one
> where the backups were taken?   Being able to do those things is the
> reason I use tar instead of filesystem-dependent dump.

Star of course can do what you like. You just need to create more than one 
backup once you split filesystems. It would be easy to handle if you like to 
use it....

> > This problem is not caused by compression or not, it is a general gtar bug that
> > I reported in 1993 already. Nobody knows why it hits and the structure of the
> > gtar sources makes it really hard to debug this problem. The FSF was interested
> > to throw aywa gtar and replace it by star 10 years ago for this kind of
> > problems in gtar.
> So, you have a repeatable test case for this and no one has looked
> into it?  That's surprising considering the number of people who have
> contributed to gnutar.   And what drove the decision not to adopt
> star?

I had a repeatable case in 1993, but at that time, I send them an archive 
created by star. In any case, there is more than one gtar user who would be 
able and willing to provide gtar archives that trigger that case. 

The reason for not adopting star was that RMS did send me a contract that was 
illegal in Europe and RMS was unwilling to convert his contract into something 
that I could legally sign.

BTW: there have been two attempts to replace gtar by star and both ended the 
same way.

> > The following other problems are known with gtar:
> >
> > -       gtar is with aprox. 5% probability unable to read it's own
> >        continuation archives from multi volume archives. This cannot be fixed
> >        as it is caused by the concept used for multi volume archives in gtar.
> I assume you mean the version where you let the tape drive hit the
> end, not where you tell it the length.  Does star always work in that
> scenario?

As far as I can tell, yes. Star uses a completely different method to verify 
followup volumes and holds diffent sets of data that cannot cause this kind of 
failure. Gtar fails when it splits an archive at a location that is inside the 
(probably extended) tar header.

BTW: As star intentionally does not implement the "verification method" from 
gtar, star is able to restore such multi volume archives.

> > -       gtar has much less features than star
> Unless you would like it to do incrementals properly across mount
> points... And I thought there was another reason regarding  features
> that amanda used gtar as well - maybe it was the ability to quickly
> estimate sizes of incrementals.  If it had worked for amanda, I would
> probably have been using it for ages.  When I looked at it, it
> couldn't because of missing features.  These days I mostly use
> backuppc with rsync as the transport since online access is so much
> nicer than tapes and rsync obviously excels at detecting differences
> in incrementals, but I suppose there is still a place for archives.

AFAIK, amanda has too few features or there are no people who are willing to 
put efforts in adopting to star.

I currently cannot believe that there is really any important feature that is 
missing in star.


 EMail:joerg at schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js at cs.tu-berlin.de                (uni)  
       joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily