Re: run script on multiple files

Kirk Wythers wrote:

I think I see what is happening. John's script was crashing at the end
of the first file with an error that I saw earlier when writing my
script. There is a footer at the end of each file and as soon as the
script hits the footer junk, it gives:

Use of uninitialized value in concatenation (.) or string at ./
micis_final.pl line 38, <> line 37993.
Use of uninitialized value in concatenation (.) or string at ./
micis_final.pl line 38, <> line 37993.

In my script I solved this problem by telling the program to exit when
there is nothing in $year (essentially the first empty line which
always comes after the last line of data and before the footer). Since
Johns script was giving the same error, I added my two bit solution. My
guess is that I am telling the program to exit before the the file
handle closes and $ is reset.

Yes I saw where you tested $year but since I don't have the actual data to
test with I had to guess, and I guess I guessed wrong. :-)


I below is john's code with my addition. Any ideas how to get out of
each file when a blank line is hit, and still close the file handle
re-set $?

#!/usr/bin/perl -w
use strict;
use Date::Calc qw(Day_of_Year);
use DBI;


my $dbh = DBI->connect( 'DBI:Pg:dbname=met_data;host=localhost',
'pguser', 'pguser' )
or die "Couldn't connect to PostgreSQL: $DBI::errstr ($DBI::err) \n";

my $sth = $dbh->prepare( 'INSERT INTO weather (station_id, year,
month, day, doy, date, precip, tmin, tmax, snowfall, snowdepth, tmean)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?)' );


my $station_id = '';

while ( <> ) {

#Part 1. Loop through the header lines to identify the station id.
#The station ID has the format of:
#STATION: SOME_STATION, STATE (Station ID: ######)
if ( 1 .. 11 ) {
$station_id = $1 if /\(Station ID:\s*(\S+)\)/;
next;
}

#At eof close the input filehandle to reset $.
if ( eof ) {
close ARGV;
next;
}

#Part 2. Loop through the records and prepare SQL statement.
my ( $year, $month, $day, $precip, $tmin, $tmax, $snowfall,
$snowdepth, $tmean, $obstime, $datasource ) = split;

#Stop reading data at the end of the file, when $year is empty. This
#gets you out of the datafile before the program chokes on the footer.
exit unless $year;

Instead of exiting from the program you should change the eof test above to:

if ( eof || !/\S/ ) {
close ARGV;
next;
}


#Initialize and concatenate date as YYYMMDD.
my $date = $year . $month . $day;

#Initialize and calculate day of the year (doy)
my $doy = Day_of_Year( $year, $month, $day );

#Switch T (trace) to 0.01 and M (missing) to -999
$precip = 0.01 if $precip eq 'T';
for ( $precip, $tmin, $tmax, $snowfall, $snowdepth, $tmean ) {
$_ = -999 if $_ eq 'M';
}

$sth->execute( $station_id, $year, $month, $day, $doy, $date,
$precip, $tmin, $tmax, $snowfall, $snowdepth, $tmean );
#print join( "\t", $station_id, $year, $month, $day, $doy, $date,
$precip, $tmin, $tmax, $snowfall, $snowdepth, $tmean ), "\n";
}

#$sth->finish();

# Disconntect from database
$dbh->disconnect();

__END__



John
--
Perl isn't a toolbox, but a small machine shop where you can special-order
certain sorts of tools at low cost and in short order. -- Larry Wall
.

Relevant Pages

  • Re: embedding files in a script
    ... >>but I need to add a footer and a header before the pipe. ... in my script I have lines ...
    (comp.unix.shell)
  • Re: logon script for an examination account
    ... Oh, I should point out, Jason, I see that your script copies the ... pre-formatted word doc to the folder that's created. ... They also provide students with 50% of their final mark. ... The idea is that the students put the id# in the Word doc's footer rather ...
    (microsoft.public.scripting.vbscript)
  • Re: embedding files in a script
    ... > but I need to add a footer and a header before the pipe. ... in my script I have lines ...
    (comp.unix.shell)
  • Confused Login pg w/header & footer ascx
    ... script at the server vs html is rendered, ... footer, but it comes as header footer then body. ...
    (microsoft.public.dotnet.framework.aspnet.webcontrols)
  • Re: run script on multiple files
    ... use DBI; ... station id. ... # At eof close the input filehandle to reset $. ... I think this is suppose to allow the script to jump to the next file. ...
    (perl.beginners)