Re: Uniq is not unique ?
Hi,
On Sun, Apr 13, 2008 at 12:57 AM, Bhasker C V <bhasker@unixindia.com> wrote:
> For fairly large file 100K+ lines
> uniq command does not filter the repetitive lines.
>
> Am I doing anything wrong on the usage ?
>
> For eg:-
>
> I had run this script in my home dir
>
> find . -name \* -type f -exec basename {} \; | uniq
> or send the output to a file and then run uniq on the file
>
> Both cases, the o/p shows repeated lines
I happened to know the source code for uniq and it should filter
repeated lines. By repeated lines, do you mean consecutive repeated
lines or separated by other lines? Uniq only filters consecutive
repeated lines, e.g.
A
A
B
A
will become
A
B
A
If you need it to filter such that only 1 unique line remains, you
will need to sort first then pipe to uniq (not a good solution for
really large files).
Regards,
Chris
>
>
> --
> Bhasker C V
> Registered Linux user: #306349 (counter.li.org)
> The box said "Requires Windows 95, NT, or better", so I installed Linux.
>
>
> --
> To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
>
>
--
contact: +65 97553292
e-mail: chrishenry.ni@gmail.com / ch_music@yahoo.com / chrishenry@nus.edu.sg
facebook: http://nus.facebook.com/profile.php?id=502687583
Reply to: