[CentOS] Tar so slow! Is there anything faster?

Fri Jan 7 05:32:17 UTC 2011
cpolish at surewest.net <cpolish at surewest.net>

On Thu, Jan 06, 2011 at 08:15:16PM +0100, Leonard den Ottolander wrote:
> Hello Dotan,
> 
> On Thu, 2011-01-06 at 16:19 +0200, Dotan Cohen wrote:
> > On Thu, Jan 6, 2011 at 16:08, Arun Khan <knura9 at gmail.com> wrote:
> > > bzip2 will slow down the operation.  If you don't really need
> > > compressed than simply do "tar cf  <tar file>  <dir/file list>"
> 
> > Yup, that's what I'm doing now! Thanks.
> 
> Gzip is pretty fast and still should give you decent compression. In
> most cases the highest compression will hardly give you better
> compression than the default level of 6, so just go with the default
> (tar cz). Use bzip2 only if space is a big concern.

pigz is a valuable tool for anyone needing gzip compression
and having > 1 cpu. It runs multiple (parallel) gzip's for
a huge speedup. I've been using it for months - very stable.

To use with Gnu tar: "--use-compress-program /usr/local/bin/pigz".

Download from:
http://pkgs.org/centos-5-rhel-5/rpmforge-i386/pigz-2.1.6-1.el5.rf.i386.rpm.html

Homepage: http://www.zlib.net/pigz/

Review: http://andrew.tumblr.com/post/344920968

-- 
Charles Polisher