On Wed, Nov 6, 2013 at 8:34 AM, <m.roth at 5-cent.us> wrote: > >> >> But even little automated things like logfile rotation can add up when >> you catch it across a bunch of noisy hosts. You don't really need to >> store the whole contents of yesterday's messages.1 and today's >> messages.2 separately when they are the same thing, just renamed. > > We don't back them up, except for /var/log on the central logging host. Are they rotated by renaming there? > But to return to the first para, there's no identical identical content. > There's similar content on development and prod servers for each team, but > that's not identical, so it's really not an issue. If the data is compressible, you'd still likely get 2x+ space saving from compression on the backup server side. If the data sets are something like time series data that just change as additional samples are added it might be worth working out a scheme to chunk it up so only the 'current' time range changes and all of the historic instances would stay identical. -- Les Mikesell lesmikesell at gmail.com