[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Removing duplicate text



Nyizsnyik Ferenc wrote:
On Sun, 24 May 2009 12:54:42 -0400
Tony Baldwin <photodharma@gmail.com> wrote:

Wilfred Zegwaard wrote:
Dear users,

I've got a textfile with a lot of duplicate text. How do I remove
it? I'm using Emacs 21.

Wilfred


That seems like a very general question, and one which has a broad plethora of answers.

Exactly; here is a simple one:
uniq -u infile outfile

Wow, that's cool.  I never saw uniq before.
I just read the man and tried it out.
Wouldn't he just want
uniq infile outfile
since -u option will remove all instances
of a repeated element, where as uniq f1 f2
will leave one instance of said repeating element?

I did
for 1 in 1 2 3 4 5 6 7 8 9; do echo banana >> banana; done
then echo "orange you glad I didn't say banana? " >> banana
then
uniq -u banana orange
cat orange
orange you glad I didn't say banana?

then did
uniq banana orange  # no -u
cat orange
banana
orange you glad I didn't say banana?

so it kept one banana.

/tony


--
http://www.photodharma.com
art & photos | tony baldwin


Reply to: