[CentOS] block level changes at the file system level?

[Ljubomir Ljubojevic]

On 07/03/2014 09:48 PM, Lists wrote:
> On 07/03/2014 12:19 PM, John R Pierce wrote:
>> you do realize, adding/removing or even changing the length of a single
>> line in a block of that pg_dump file will change every block after it as
>> the data will be offset ?
> 
> Yes. And I guess this is probably where the conversation should end. I'm 
> used to the capabilities of Mercurial DVCS as well as ZFS snapshots, and 
> was thinking/hoping that this type of capability might exist in a file 
> system. Perhaps it just doesn't belong there.
> 
> On 07/03/2014 12:23 PM, Les Mikesell wrote:
>> But, since this is about postgresql, the right way is probably just to
>> set up replication and let it send the changes itself instead of doing
>> frequent dumps.
> 
> Whatever we do, we need the ability to create a point-in-time history. 
> We commonly use our archival dumps for audit, testing, and debugging 
> purposes. I don't think PG + WAL provides this type of capability. So at 
> the moment we're down to:
> 
> A) run PG on a ZFS partition and snapshot ZFS.
> B) Keep making dumps (as now) and use lots of disk space.
> C) Cook something new and magical using diff, rdiff-backup, or related 
> tools.
> 

Check out 7z from p7zip package. I use command:

7za a -t7z $YearNum-$MonthNum.7z -i at include.lst -mx$CompressionMetod
-mmt$ThreadNumber -mtc=on

for compressing a lot of similar files from sysinfo and kernel.log,
files backuped every hour that do not change much. And I find out that
it reuses already existing blocks/hashes/whatever and I guess just
reference them with a pointer instead of storing them again.


So, 742 files that uncompressed have 179 MB, compressed ocupy only 452
KB, which is only 0.2% of original size, 442 TIMES smaller :

Listing archive: 2014-03.7z

--
Path = 2014-03.7z
Type = 7z
Method = LZMA
Solid = +
Blocks = 1
Physical Size = 426210
Headers Size = 9231

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------
------------------------
2014-03-01 00:02:07 ....A       259517       416979
Silos-Srbobran-l5-kernlog-2014-03-01-00-02-07.txt
2014-03-01 01:01:52 ....A       259529
Silos-Srbobran-l5-kernlog-2014-03-01-01-01-52.txt
...............................................
...............................................
...............................................
2014-03-31 22:01:33 ....A       259502
Silos-Srbobran-l5-kernlog-2014-03-31-22-01-33.txt
2014-03-31 23:01:33 ....A       259485
Silos-Srbobran-l5-kernlog-2014-03-31-23-01-32.txt
------------------- ----- ------------ ------------
------------------------
                            184553028       416979  742 files, 0 folders

Maybe, if you compress them into the same file (afterwords?) you can get
the similar result.


-- 
Ljubomir Ljubojevic
(Love is in the Air)
PL Computers
Serbia, Europe

StarOS, Mikrotik and CentOS/RHEL/Linux consultant