[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Parallel GZIP



Ron Johnson wrote:
> On 12/22/08 09:13, Eugene V. Lyubimkin wrote:
>> Thomas Preud'homme wrote:
>>> The Monday 22 December 2008 13:53:43 Eugene V. Lyubimkin, you wrote :
>>>> Volkan YAZICI wrote:
>>>>> Hi,
>>>>>
>>>>> Are there any compression tools in the official Debian GNU/Linux
>>>>> repositories that can take the benefit of multiple cores/cpus?
>>>> pbzip2?
>>>
>>> I profit of this thread to ask you if you see amelioration with
>>> pbzip2. I made 2 tests on file of 50 MB and the result if
>>> approximately the same time as bzip2 (a little much longer with
>>> pbzip2 !!!) but a CPU twice time more used. I have a core 2 duo and
>>> time give me a usage of almost 200% with pbzip2
>>>
>>> Anyone has better results ?
>> Yes, see following:
>>
>> jackyf@work:~/temp$ time tar -cjf temp1.tar.bz2 debian
>>
>> real    0m18.830s
>> user    0m18.317s
>> sys    0m0.268s
>> jackyf@work:~/temp$ time bash -c "tar -cf - debian | pbzip2 -c >
>> temp2.tar.bz2"
>>
>> real    0m10.494s
>> user    0m19.557s
>> sys    0m0.488s
>>
>> 10 is much less than 18.
> 
> That might be because much of the data is already cached.  Each of those
> commands should be run 3 times to get good numbers.
We are not testing filesystem or disks, we testing CPU, and no even
modern desktop CPUs can cache 36 MiB of data.
But I retried these commands several times anyway to verify, and results are
almost the same, maybe +/- 5%.


-- 
Eugene V. Lyubimkin aka JackYF, JID: jackyf.devel(maildog)gmail.com
Ukrainian C++ Developer, Debian Maintainer, APT contributor

Attachment: signature.asc
Description: OpenPGP digital signature


Reply to: