[CentOS] schily tools

Tue Feb 7 15:26:38 UTC 2012
Joerg Schilling <Joerg.Schilling at fokus.fraunhofer.de>

Let me comment some questions in one single mail:

> My basic requirement with what I'm doing is to use standard tools and
> formats so that archives I write today can be readable in 10 years.

Star becomes 30 in 4 months, any archive created since it's early beginning in 
summer 1982 can still be read back.

> I don't think there is any such general consensus.  Are you reading
> something that favors Solaris/*bsd over GNU based systems?

Schily tools (and in special star) implement support for Linux specific 
extensions. This is what you do not get from gtar at all. So why do Linux
distros prefer gtar even though there is no Linux support?

> I doubt if they are as well maintained in linux distros as the GNU
> tool set, particularly in terms of having recent fixes backported into
> the versions carried in enterprise distros.

gtar still did not fix bugs I reported in 1993 (e.g. the bug that causes gtar 
to complain with "skipping to next header" even on it's own archives). I am 
thus sure that star not not worse than gtar....

> It shouldn't matter if you don't use either of the version's
> extensions, and for archiving you probably don't need them.  For
> example, star and GNUtar use very different concepts for incremental
> backups - star is sort of like dump and must work on filesystem
> boundaries where GNUtar's --listed incremental needs a file to hold
> state but will work on arbitrary directories and can span mount
> points.  But for archiving, you probably only care about the maximum
> size of a file it can handle.

When I implemented incremental restores for star in September 2004, I wrote a 
simple script for a incremental testcase and tested the deltas with 
ufsdump/ufsrestore, gtar and the star version at that time. Gtar was unable to 
deal with my testcase, so I stopped testing it any further. 

If you like to discuss incrementals, you definitely need to discuss behavior at 
restore time and restoring incrementals definitely does not work correclty with 
gtar if you renamed directories. 

Star is used to do incremental backups/restores on a dayly base in Berlios 
since September 2004. Since Spring 2005, not a single problem was seen, so 
there are more that 2500 successful incremental restores that verify no problem 
even under stange conditions.

> I don't think so - I'm fairly sure I've seen GNUtar complain about bad
> headers, say 'skipping to next header'  and then find something. It
> won't do that if you used the -z option because you generally can't
> recover from errors in compression.  But, I've never seen a tape drive
> recover from an error and continue past it anyway so in practice
> that's not going to matter.  If you are concerned about errors, keep
> more copies.

This problem is not caused by compression or not, it is a general gtar bug that 
I reported in 1993 already. Nobody knows why it hits and the structure of the 
gtar sources makes it really hard to debug this problem. The FSF was interested 
to throw aywa gtar and replace it by star 10 years ago for this kind of 
problems in gtar.

> afio is an archiver (available from third-party repos, not base) which
> can compress yet still recover--it basically compresses each file
> individually instead of compressing the entire archive, so the file
> might be unrecoverable but the rest of the archive is still intact.
> I use it for my tape backups (though your point of not knowing if
> it'll fit on the tape is valid).

Be careful with what you believe. The CPIO archive format in general is worse 
with resyncing to a defective archive that TAR is. Also note that afio greates 
arhives that may start to be non CPIO compliant somewhere in the middle. So you 
can never know whether you are able to restore with anything other than afio. 
What if afio does not compile on your new platform because it is no longer 

Also note that the POSIX standard dropped CPIO as an actively supported 
archiveformat because (different from TAR) any extention to CPIO results in 
creating a new incompatible archive format.

The following other problems are known with gtar:

-       gtar is with aprox. 5% probability unable to read it's own
	continuation archives from multi volume archives. This cannot be fixed 
	as it is caused by the concept used for multi volume archives in gtar.
-       gtar created archives with defective sparse file until a few years ago 
	(up to ~ 2005) in case a file was bigger than a few GB. 
-       gtar has much less features than star 

-	gtar does not inlcude libfind, so there is no support for the find(1)
	command line syntax in gtar.

-       gtar needs aprox. 3x more CPU time as star 


 EMail:joerg at schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js at cs.tu-berlin.de                (uni)  
       joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily