Les Mikesell <lesmikesell at gmail.com> wrote: > > Well, I verified that gtar fails with even a simple case and I did not see > > any > > problem with star since 6 years even with thousands of incremental > > restores. > > So I am not sure what you are talkin about. > > > > Gtar doesn't fail, it just requires some extra steps to accommodate some > very unlikely circumstances. Your star runs will not work at all in very > likely circumstances (which you conveniently avoid testing). Trying to understand you leads me to the assumption that you of course know that gtar fails. As you seem to claim that there is a workaround but don't mention it, it seems that you like other people to fail when trying to restore incremental backups that have been made with gtar. > > How many successful automatical incremental restores did you try with gtar > > so > > far? 1/12 dozen? > > > > Enough to know it works. Well, I did one test only and I know that gtar does not do what it should when restoring incrementals. In addition, gtar incrementals are not space efficient: If you e.g. have a 1 TB filesystem with one directory at top level and rename that directory, you will end up with a 1 TB incremental if you use gtar. Looking at the way gtar works, I would assume that gtar needs an intermediate disk space of 2 TB to restore the data (if the restore works at all). Star will create an incremental archive of 10kB for the same change and star does not need additional space on the restore media. Also note that star is able to backup ACLs and extended attributes but gtar does not have the needed features to do so. > Our definitions of 'OS and FS independent' are very different. I expect > that to mean that I can actually change the FS and have things keep > working. Star doesn't. And it doesn't work over nfs, on fat, ntfs, and on > and on. Do you like to tell everbody that it makes no sense to believe you, or why do you write claims that are obviously wrong? Star gives you FS and OS independence and star is even at least 30% faster than ufsdump (under all conditions) or any fork that was created from ufsdump (such es ext*..). > What rules? If you are FS independent, you follow a directory tree, not a > rulebook. I want the content of my files backed up whether they are stored > in a way that matches your restricted rules or not. I don't want to have > to take a whole filesystem. I don't want a run to be restricted to a > single filesystem. You again harm your credibility by writing false suggestions. Star of course allows you to backup parts of a filesystem (e.g. a directory sub-tree). It seems that your problem is that you are not willing to inform you about any other program that gtar and it seems that you are not willing to think about the results from gtar problems. > In a word, amanda. If my backups don't fit on the tape there are no > backups at all, and the only reason for doing incrementals at all is that > you have to use them to make the data fit. I'm not interested in > calculating the right mix of incremental levels on a whole bunch of > directory trees to make each night's run fit. Amanda does that You again harm your credibility by writing false suggestions. Star is able to automatically deal with the unpredictable size of modern tapes with HW compression, while amanda fails completely and needs to assume the smallest possible size. This is because amanda does not make use of the related features of the backup utility. Star can detect the actual end of a tape in write mode on the fly and then request a media change. BTW: what amanda does may make sense with gtar as gtar frequently fails to accept own followup volumes but this is a problem that is absent with star. > automatically. Amanda can use dump or gnutar. Linus Torvald has claimed > that dump is not safe on live filesystems. I assume he's an expert on the > topic (and without looking at the code, I'd guess that star uses the same > approach to determine what to take in incrementals). That leaves gnutar. Well, besides the fact that Torvalds is not known of being a filesystem expert it is a bad idea to invert a true claim as you do here and believe that this is still true. ufsdump is not safe on life filesystems but gtar and star are not safe too. The most insecure backup program for a life file system if ufsdump or it's forks. This is because ufsdump first reads all fs meta data and later reads a raw filesystem at block level without taking care of changes. gtar is still insecure as it traverses the directory tree and is affected by changes on that tree while the backup is running. star is definitely less insecure than gtar as star does not need to keep a database of the filesystem structure in order to be able to work. In any case, I know of no serious sysadmin that makes backups from a life filesystem without using filesystem snapshots. If you however use snapshots that are needed for a reliable incremental backup, you cannot backup directory trees the way you seem to like..... Star matches the reliable case and if you use gtar without snapshots, you are creating unreliable backups. > I can live with a very unlikely chance that I'll have to give a few extra > commands to restore compared to not getting backups or having to compute > the size of incremental levels manually every day. > > And then there was the fact that at the time I implemented amanda, star did > not do incremental restores anyway... Star implements reliable incremental restores since 7 years, so you seem to be living in a long gone past. > > Well gtar failed with my first simple test already, why do you still > > believe it > > works? > > > > > Because the data is on the tape and can be extracted if you move the > same-named but different object type thing out of the way first (and admit > that it wouldn't be there except in a contrived test). The commands to do > it might not be convenient, but it is much, much better than using a tool > that in many circumstances won't have the data at all, or can't restore it > onto a different mount topology. Well star always works fully automatically with incrementals. What you like to do with gtar may require hours of manual intervention and is still based on an unreliable basic method (as you cannot use the needed snapshots). > OK, here's a test that is a very common scenario. Boot a dual-boot > windows/linux laptop into linux, back it up in a way that includes the > mounted VFAT and NTFS partitions. Make changes on the VFAT or NTFS > locations under linux or windows. Make an incremental backup that includes > all the changes. You are trying to do something in an unrelibale way that can be done reliably and fully automatically using star instead of gtar. So why do you still use gtar? > > As you stay with gtar, it seems that rather you are biased. > > > > > Strictly a pragmatic choice. GNUtar works, where star did not. Well gtar does not work and all your replies admit (even if you try to hide this fact) that gtar does not work automatically and that you need to add some secret manual work that may take a long time in order to go on in case of failures. Star on the other side works reliably and automatically. > But FS snapshots take extra disk space and not all file systems provided > them back then. That does not make the file contents less important. Just do not use filesystems that do not support snapshots for professional work and if you are forced to do backups on a fs without snapshots at least use the backup method with the least sensitivity against inconsistencies. This preferred backup method is star if we are comparing ufsdump, gtar and star. > > The file we are talking about may easily reach a size of several GB. > > > > > So first we need room for filesystem snapshots, but now storing a file big > enough to hold the equivalent of a directory listing is a problem? There of course is a difference between the need to store some data for a few minutes (this is what an incremental star backup takes) and the need to store a lot of data for the whole life-cycle of the FS (which is what you need to do with gtar). Also note that many filesystems store the actual data for the snapshot in the free blocks of the filesystem. In these cases, you do not even notice the need for space to support the incremental. > Can you, using star, make incremental backups of a directory tree that work > without knowing./caring what file systems hold that tree or where the roots > or mount points are? Of course, if there are no mount points in that tree. As mentioned before, the star method is enforced anyway if you like to have reliable incrementals that need snapshots. So gtar will not do what you like in case you implement reliable backups. > > Did you ever think about the fact that the various BSD dump descendants do > > not > > share a common archive format? > > > > > Does that matter if I have the code for the one that reads my format? If you like to restore more meta data than UNIX had in 1981, this definitely matters. Also note that star is at least 30% faster than ufsdump, so why do you like to use ufsdump when you can have star? > That would be nice if it were true. The most likely non-native target would > be windows. Should I plan on that working? What about the simpler case of > the same OS but an added mount point in the layout where I want a portion > of the restore to go? If you try to create backups on Win-DOS without using Cygwin or some equivalent system, you are lost, as MS ignores standards and does not include st_ino in struct stat. If you however use Cygwin, it works... > I expect actual backups to be i/o bound to a point that whatever else they > do performance-wise is irrelevant. If you were true, why is gtar slower than star? If you are backing up to a real tape, this may be extremely important because being a bit too slow may cause the tape to fail streaming. > Star introduced the -onull option more than 15 years ago and I asume that > > potential users start with reading the man page. > > > > > I didn't see an option for reporting size in the man page - it might > actually be adaptable if anyone still cared about tapes. But, I find the > man page very confusing. Most of the incremental options say it has to be > a single Posix-conforming FS but the -dump section talks about spanning > more than one. If you did never work with star, why do you claim so many "failures" of star? I recommend you to start using star and talk about it after you learn how it works. Star implements so many features, that you need to use it for some days to understand the concept, but I know nobody who did not switch to star after understanding the concept. Most of the features from star I am using on a daily base are missing in gtar. Jörg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js at cs.tu-berlin.de (uni) joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily