[CentOS] block level changes at the file system level?

[Ljubomir Ljubojevic]

On 07/07/2014 02:35 PM, SilverTip257 wrote:
> On Thu, Jul 3, 2014 at 4:50 PM, Ljubomir Ljubojevic <centos at plnet.rs> wrote:
> 
>>
>>> Whatever we do, we need the ability to create a point-in-time history.
>>> We commonly use our archival dumps for audit, testing, and debugging
>>> purposes. I don't think PG + WAL provides this type of capability. So at
>>> the moment we're down to:
>>>
>>> A) run PG on a ZFS partition and snapshot ZFS.
>>> B) Keep making dumps (as now) and use lots of disk space.
>>> C) Cook something new and magical using diff, rdiff-backup, or related
>>> tools.
>>>
>>
>> Check out 7z from p7zip package. I use command:
>>
>> 7za a -t7z $YearNum-$MonthNum.7z -i at include.lst -mx$CompressionMetod
>> -mmt$ThreadNumber -mtc=on
>>
> 
> Seems to be that 7zip uses LZMA for the 7z archives (though it supports
> other compression types).  Confirmed below.
> 
> 
>>
>> for compressing a lot of similar files from sysinfo and kernel.log,
>> files backuped every hour that do not change much. And I find out that
>> it reuses already existing blocks/hashes/whatever and I guess just
>> reference them with a pointer instead of storing them again.
>>
>>
>> So, 742 files that uncompressed have 179 MB, compressed ocupy only 452
>> KB, which is only 0.2% of original size, 442 TIMES smaller :
>>
>> Listing archive: 2014-03.7z
>>
>> --
>> Path = 2014-03.7z
>> Type = 7z
>> Method = LZMA
>>
> 
> Exactly ... LZMA ...
> Grab the "xz" package for CentOS 6
> 
> There's also an --lzma option for tar.
> 
> I am inclined to use xz utils as opposed to 7zip since 7zip comes from a
> 3rd party repo.
> 

xz needs to be checked for support of block optimization/reuse, I do not
think that is part of LZMA protokol (I might be wrong of course). Also,
check needs to be made if xz supports multitrheading like pk7zip.

Btw., p7zip package is in EPEL, should be safe enough,but I understand
your concerns.

-- 
Ljubomir Ljubojevic
(Love is in the Air)
PL Computers
Serbia, Europe

StarOS, Mikrotik and CentOS/RHEL/Linux consultant