On Wed, Jul 2, 2014 at 2:53 PM, Lists lists@benjamindsmith.com wrote:
I'm trying to streamline a backup system using ZFS. In our situation, we're writing pg_dump files repeatedly, each file being highly similar to the previous file. Is there a file system (EG: ext4? xfs?) that, when re-writing a similar file, will write only the changed blocks and not rewrite the entire file to a new set of blocks?
Assume that we're writing a 500 MB file with only 100 KB of changes. Other than a utility like diff, is there a file system that would only write 100KB and not 500 MB of data? In concept, this would work similarly to using the 'diff' utility...
There is something called rdiff-backup (http://www.nongnu.org/rdiff-backup/ and packaged in EPEL) that does reverse diffs at the application level. If it performs well enough it might be easier to manage than a de-duping filesystem. Or backuppc - which would store a complete copy if there are any changes at all between dumps but would compress them and automatically manage the number you need to keep.