Les Mikesell <lesmikesell at gmail.com> wrote: > > In general, this does not work with gtar. You are exactly in the area that > > caused my conclusion that gtar is not useful at all for incremental > > backups. > > > > > No, a general file-oriented case would handle FAT and NTFS filesystems, and > continue to work even if you mount one of those in the path of your planned > backup or restore. Star fails that. Well, I verified that gtar fails with even a simple case and I did not see any problem with star since 6 years even with thousands of incremental restores. So I am not sure what you are talkin about. How many successful automatical incremental restores did you try with gtar so far? 1/12 dozen? Star's incrementals are file oriented and this is why it is able to offer a OS and FS independent incremental backup. Given the fact that gtar completely fails with it's incrementals even in simple cases, I cannot understand why people still use gtar's incrementals. Note that doing a backup is only half of the job and gtar fails with the restore... > > Star does what you need, you are just using it the wrong way. > > > > > I don't plan to arrange my life or my data around star's restricted > capabilities. This is a really strange claim, as you obviously follow the rules in case you create a new filesystem. Now please explain me why you are doing backups with a program that is known to fail with incremental restores? > As mentioned, gtar has major problems whe directory is envolved. How many > > successful restores did you pass? > > > > All of them. Even the contrived case you sent me when we had this same > conversation years ago. The saves always work and the only issue there > has ever been is when a restored directory is supposed to be deleted and > replaced by a file of the same name as an incremental update (or maybe the > other way around...). It's something that has never happened in years of > operations and recoverable if it does - just delete the offending item and > repeat the restore of the thing that ends up there. And it might be fixed > now - I haven't been concerned enough to check. Well gtar failed with my first simple test already, why do you still believe it works? > Here's the simple test case. Plug in a FAT-formatted USB key in the path > of your backup. Change something on it and repeat with your incremental > mode. Restore it with or without the key mounted in that position. If > that doesn't preserve the data in a way you can restore, I'm not willing to > trust it for my backups. Or if you are biased against FAT, use the > filesystem of your choice, just do it dynamically. Why do you like me to do some really strange test while your backup method is know not to work at all? Would'nt it be better to first switch to a working backup program? As you stay with gtar, it seems that rather you are biased. Also note that reliable backups need to use snapshots, so please rethink your backup method under the right constraints. > It may be that you miss the fact that modern filesystems like e.g. ZFS may > > be > > able to move a directory tree into a "new filesystem" without affecting > > ctime. > > > > > That breaks the definition of ctime, doesn't it? But it doesn't matter to > GNUtar because if it hits a directory in a location that doesn't match what > is listed in its -listed-incremental file it will back it up with > everything below it. No, ZFS is fully compatible with POSIX. ZFS permits file renames aross different filesystems in case these filesystems are on the same storage pool. See POSIX for rename(): [EXDEV] [CX] [Option Start] The links named by new and old are on different file systems and the implementation does not support links between file systems. [Option End] So if really like you move a sub-tree into a separate filesystem, only the top level directories are granted to get ctime updated. BTW: Why should gtar work in your case when it already fails with incremental restores for simple directory rename operations? > If gtar would be implemented correctly, it would need to maintain a huge > > database for the lifetime of the filesystems tobackup. > > > > No, it needs a small file for each state of a tree where you plan to do a > subsequent higher level incremental. And amanda manages this for you. You seem to have a strange concept of what's a "small" file. The file we are talking about may easily reach a size of several GB. Star needs less that 100 bytes per level of incrementals. So if you do just one level 0 dump and any amount of level 1 dumps, this star state file will be less than 200 bytes per filesystem. > Sometimes it is, sometimes it is plugging in a usb device, and sometimes it > is someone else's administrative work, unrelated to the backup > configuration. If you like to do reliable backups, you need to snapshots anyway. Believe me that the snapshot does not mirror the mount you are talking about, so it is obvious that the gtar concept is even wrong if you asume a reliable overall environment. > > What star allows you to do is to have incrementals that are as reliable as > > ufsdump but that are independent from the OS and the FS. > > > > > That would be better stated as 'as filesytem dependent as ufsdump'. It is > not independent from the FS at all or I'd be able to point it at an > arbitrary directory without knowing or caring if it encompasses a whole > filesystem or has additional mounts. Or if the layout is the same during > the restore. ??? > > Well as mentioned before, if you reconfigure your server, you also need to > > reconfigure the backup to match the new situation at the server. > > > > They are different machines. Why should I need to know if someone else > moved /home to a separate volume just to correctly copy the data that has > changed since the last backup run? Because such a change would affect snapshots that are definitely needed for reliable backups. > > This is an unproven claim from you. I grant you that star supports all you > > need > > for a reliable incremental backup and restore. > > > > It does, under the same restricted conditions dump would - and amanda > already works with dump. The reason I use tar instead of dump is to not be > restricted to those conditions. You told me that you use gtar and gtar has the limitation that incremental restrores do not work in many cases. I prefer to live with a method that may have any restriction, in case it is able to support reliable incremental restores. This is what you get from star. > > If amanda does not support star, it seems that the amanda people are not > > interested in reliable backups. > > > > No, they already have dump for the things that star could do. Did you ever think about the fact that the various BSD dump descendants do not share a common archive format? Star being OS and FS independent allows you to e.g. restore any incremental on any OS and does not force you to first manually port the actual "restore" program to your current restore OS. > > Star supports everything you need, if Amanda does not support star, this is > > obviously a filure at amanda's side. > > > > It just doesn't add capabilities the way GNUtar does so there is no need > for it. And I don't see any support for size estimates like dump provides, > which are very necessary for amanda to arrange each set of incremental and > full runs to fill a tape. Can you do anything that matches the speed of a > gnutar run of : 'tar --totals -cf /dev/null /path' ? I'll agree that what > GNUtar does in that special case is weird (or at least that the reason it > does it is weird) if you'll agree that it is necessary for practical > reasons. I cannot tell when gtar introduced this strange method that makes performance tests hard.... Star introduced the -onull option more than 15 years ago and I asume that potential users start with reading the man page. One problem for you with star may be that star does not match the performance of gtar.... star is aprox 2-3x faster than gtar ;-) I hope you can live with this limitation..... Jörg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js at cs.tu-berlin.de (uni) joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily