Bug#328303: iconv breaks on piped input, because it tries to read whole input into a buffer / iconv: Feature request: add stream capabilities
Subject: /usr/bin/iconv: Feature request: add stream capabilities
Package: libc-bin
Version: 2.28-2
Severity: normal
File: /usr/bin/iconv
hi,
i also planned to add this as a feature request, please add some
parameter to the iconv command, like with script -f or grep
--line-buffered, to stream the output as fast as possible through iconv
no matter if EOF is reached or not. So iconv would be much more usable
in situations where the output input isnt completely generated yet.
Please add this if you are a programmer, understand the code of the
iconv command and have some time, it would make the iconv command a lot
more powerful.
Right now i have to start another iconv for every line in my stream
using xargs and some bash code to work around this which is a lot more
system load i guess do to repeated initialization and de-initialization
of the iconv application, of cause a lot more work to do when piping
stuff through iconv, and so on...
thanks a lot in advance!
treaki
On Wed, 14 Sep 2005 18:54:36 +0100 Chris Lightfoot <chris@ex-parrot.com>
wrote:
> Package: libc6
> Version: 2.3.2.ds1-22
>
> iconv(1) tries to read in its whole input before doing
> conversion. This is no good if you want to use it in a
> pipe.
>
> Example:
>
> yes | iconv -f utf-8 -t iso-8859-1
>
> expected result: ten lines of `y'
> actual result: iconv runs out of memory
>
> As a more serious example, consider
>
> zcat dump-of-wikipedia.xml.gz | iconv -f utf-8 -t $charset | ...
>
> --
> ``The Strategic Railway Authority is not in the business of railway
lines.''
> (Cambridgeshire County Councillor Shona Johnstone)
>
>
--
|_|0|_|
|_|_|0|
|0|0|0|
http://treaki.tk/
Reply to: