On 07/03/2014 09:48 PM, Lists wrote: > On 07/03/2014 12:19 PM, John R Pierce wrote: >> you do realize, adding/removing or even changing the length of a single >> line in a block of that pg_dump file will change every block after it as >> the data will be offset ? > > Yes. And I guess this is probably where the conversation should end. I'm > used to the capabilities of Mercurial DVCS as well as ZFS snapshots, and > was thinking/hoping that this type of capability might exist in a file > system. Perhaps it just doesn't belong there. > > On 07/03/2014 12:23 PM, Les Mikesell wrote: >> But, since this is about postgresql, the right way is probably just to >> set up replication and let it send the changes itself instead of doing >> frequent dumps. > > Whatever we do, we need the ability to create a point-in-time history. > We commonly use our archival dumps for audit, testing, and debugging > purposes. I don't think PG + WAL provides this type of capability. So at > the moment we're down to: > > A) run PG on a ZFS partition and snapshot ZFS. > B) Keep making dumps (as now) and use lots of disk space. > C) Cook something new and magical using diff, rdiff-backup, or related > tools. > Check out 7z from p7zip package. I use command: 7za a -t7z $YearNum-$MonthNum.7z -i at include.lst -mx$CompressionMetod -mmt$ThreadNumber -mtc=on for compressing a lot of similar files from sysinfo and kernel.log, files backuped every hour that do not change much. And I find out that it reuses already existing blocks/hashes/whatever and I guess just reference them with a pointer instead of storing them again. So, 742 files that uncompressed have 179 MB, compressed ocupy only 452 KB, which is only 0.2% of original size, 442 TIMES smaller : Listing archive: 2014-03.7z -- Path = 2014-03.7z Type = 7z Method = LZMA Solid = + Blocks = 1 Physical Size = 426210 Headers Size = 9231 Date Time Attr Size Compressed Name ------------------- ----- ------------ ------------ ------------------------ 2014-03-01 00:02:07 ....A 259517 416979 Silos-Srbobran-l5-kernlog-2014-03-01-00-02-07.txt 2014-03-01 01:01:52 ....A 259529 Silos-Srbobran-l5-kernlog-2014-03-01-01-01-52.txt ............................................... ............................................... ............................................... 2014-03-31 22:01:33 ....A 259502 Silos-Srbobran-l5-kernlog-2014-03-31-22-01-33.txt 2014-03-31 23:01:33 ....A 259485 Silos-Srbobran-l5-kernlog-2014-03-31-23-01-32.txt ------------------- ----- ------------ ------------ ------------------------ 184553028 416979 742 files, 0 folders Maybe, if you compress them into the same file (afterwords?) you can get the similar result. -- Ljubomir Ljubojevic (Love is in the Air) PL Computers Serbia, Europe StarOS, Mikrotik and CentOS/RHEL/Linux consultant