On Fri, Sep 28, 2007 at 10:08:14AM -0400, Stefhen Hovland wrote: > > On 9/27/07, Andrew Sackville-West <andrew@farwestbilliards.com> wrote: > > On Thu, Sep 27, 2007 at 02:57:40PM -0400, Stefhen Hovland wrote: > > > Is this possible for pbzip2 to run on a tar which outputs to standard out? > > > > > > I am trying to speed up a backup process which takes hours, i have > > > about 100g of uncompressed data which will be tar'd up and is > > > currently running thru gzip. This will be running on a 16 cpu box > > > which would greatly speedup this issue. > > > > > > from: > > > > > > tar cf - . | gzip > /tmp/file.tar.gz > > > > > > to: > > > > > > tar cf - . | pbzip2 > /tmp/file.tar.bz2 > > > > > > > > > This doesnt seem to work, is it because there is no way to split the > > > stdio to multiple processors on the fly? > > > > per > > > > http://compression.ca/pbzip2/ > > > > ToDo > > > > - Add support for input from stdin & pipes > > > > google man! > > > Ahh crap, I looked thru that page, but didnt notice that. My fault :D :) my thought on this... I don't know how tar links in to bzip2 (which is a compression option for tar (-j?). But maybe its possible to substitute pbzip2 through something as simple as a symlink. Its a stretch I know, but worth a shot. Also, out of curiousity, why not tar -czf /tmp/file.tgz . instead of the pipe? (unless you're actually piping through ssh or something.) anyway, since the idea above (symlink) will almopst certainly not work, you'll probably have to sacrifice the storage space to tar it all up into a file and then pbzip2 it separately. or hack the code, but I'm willing to bet that parallelizing something coming from stdin is non-trivial (I'm no coder...) and that's why its ToDo... A
Attachment:
signature.asc
Description: Digital signature