Re: remove/replace non-ascii characters from file

[Jim Holland <debian@mango.zw>]

On Thu, 3 Aug 2006, Johannes Wiedersich wrote:

> I have a silly Window$ application that is supposed to export ascii 
> data. In fact the file is 99% percent ascii (after dos2unix), but 
> contains a line starting with "Comment: " that contains non-ascii 
> garbage (represented as ^@^@^@^@^@^@ etc.)
> 
> I tried
> $ grep -v Comment
> but that just returns
> Binary file darkaa2.dat matches
> 
> Is there a simple way to remove this line?

Why not filter the file through "strings" first? eg:

	strings infile > outfile

That will remove any non-printable characters.

Or use "tr", eg:

	tr -cd '[:print:]\n' < infile > outfile

That will delete any characters that are not printable or newlines or 
spaces.

You will probably need to experiment a little to ensure that these don't 
remove any other characters that you still want.

Regards

Jim Holland
System Administrator
MANGO - Zimbabwe's non-profit e-mail service