Re: remove duplicate lines



Tielman Koekemoer (TNE) wrote:

Wow, I'm really confused. I'm trying to remove duplicate lines from a marc21 text file. I have spent countless hours searching for scripts etc.

I'm also very new to Perl and wrote a long and newbyish script that does exactly what the Unix command "sort FILENAME | uniq" does just to see how it can be done.

How long? Because you can do that on one line in perl. :-)

perl -e'print sort grep !$seen{$_}++, <>' FILENAME



John
--
use Perl;
program
fulfillment
.



Relevant Pages

  • Re: Any TextMate Editor equivelent for Windows ?
    ... >> custom Ruby shell scripts and Unix command ports for finding and ... When I'm on windows the first thing I do is install MSys/MingGW. ... then hack together scripts using backticks or even platform ...
    (comp.lang.ruby)
  • Re: Any TextMate Editor equivelent for Windows ?
    ... >>>custom Ruby shell scripts and Unix command ports for finding and ... > When I'm on windows the first thing I do is install MSys/MingGW. ... > then hack together scripts using backticks or even platform ...
    (comp.lang.ruby)
  • Re: remove duplicate lines
    ... I have spent countless hours ... >>>searching for scripts etc. ... Because you can do that on one line in perl. ...
    (perl.beginners)
  • Re: Expect script needs to send a password that contains the # character
    ... > I've been searching the Net and this group, ... > formulating my query correctly or no-one else has as yet run into this ... > comment character in expect scripts. ...
    (comp.unix.questions)
  • Re: Dream Controls: TDcInfoTree
    ... to code, but this is not good, because I can't no longer change the Scripts with the Delphi-IDE. ... I am also searching for an replacement for DC but I see me to convert the sources to D2007. ...
    (borland.public.delphi.thirdpartytools.general)