Re: remove duplicate lines
- From: daggerquill@xxxxxxxxx (Jay Savage)
- Date: Mon, 30 May 2005 17:23:15 -0400
On 5/30/05, John W. Krahn <krahnj@xxxxxxxxx> wrote:
> Tielman Koekemoer (TNE) wrote:
> >
> >>Wow, I'm really confused. I'm trying to remove duplicate
> >>lines from a marc21 text file. I have spent countless hours
> >>searching for scripts etc.
> >
> > I'm also very new to Perl and wrote a long and newbyish script that
> > does exactly what the Unix command "sort FILENAME | uniq" does just to
> > see how it can be done.
>
> How long? Because you can do that on one line in perl. :-)
>
> perl -e'print sort grep !$seen{$_}++, <>' FILENAME
>
>
>
> John
You can also use just do:
$seen{$_}++ while <>;
print sort keys %seen;
Which will also let you know which items were repeated, and how many
times. It al depends on what you ultimately want to do with the
information. This is why we ask to see code you've tried, and where
you're headed. There are probably close to 1,000 ways to handle this
in Perl, each of them appropriate for a specific circumstance.
In this case, doing a search for MARC and/or Z3950 on search.cpan.org
will turn up some interesting results, too
HTH,
-- jay
--------------------
daggerquill [at] gmail [dot] com
http://www.engatiki.org
.
- References:
- RE: remove duplicate lines
- From: Tielman Koekemoer \
- Re: remove duplicate lines
- From: John W. Krahn
- RE: remove duplicate lines
- Prev by Date: Re: Perl 5.6.0, AIX, Missing Module
- Next by Date: mail header split
- Previous by thread: Re: Simple question
- Next by thread: Setting up the DBD::ODBC module
- Index(es):
Relevant Pages
|
