Re: data manipulation

From: Bob (bNOoONb_at_not.pilbara.net.au)
Date: 10/10/03 

Date: Fri, 10 Oct 2003 16:08:56 +0930

"Gunnar Hjalmarsson" <noreply@gunnar.cc> wrote in message
news:bm5fel$j9uep$1@ID-184292.news.uni-berlin.de...
> Bob wrote:
> >
> > -> push @chunks, $chunk if $chunk;
> > -> $chunk = '';
> >
> > After reading the push function description from "learning Perl" I
> > am failing to understand exactly what is happening here.
>
> It adds an element to the array @chunks with what's been stored in
> $chunk from previous iterations, and then it empties $chunk.
>
> The line
>
> push @chunks, $chunk if $chunk;
>
> can also be written
>
> if ($chunk) {
> push @chunks, $chunk;
> }
>
> My suggestion means that the whole log file ends up in memory in the
> array @chunks. I thought that made the code easier to understand, but
> it should be noted that if the log file is really big, it's not a good
> approach. In that case, you'd better do in the loop with respective
> 'generation' of $chunk whatever you want to do with it, and refrain
> from storing the whole file in an array.
>
> HTH
>
> --
> Gunnar Hjalmarsson
> Email: http://www.gunnar.cc/cgi-bin/contact.pl
>

Thanks for the explanation, And I already understood the implication of the
whole file being in memory. In this case, it is not a problem, the log is
generally ony 3-5Meg but never over 50Meg.

For those interested, I have included the full script below

Regards,

B

#!/usr/bin/perl
#
# NAME qmail-qreadto
# purpose - look thru the qmail-qread logs for particular messages
#
use strict;
use warnings;

if (@ARGV == "0" ) {
        print "\n\tqmail-qreadto \{ email to search for\} \n";
        print "\t\texample\: qmail-qreadto me\@example.net \n\n";
exit 0
}

my ($Log, @Logs);
my $file1 = "/var/log/qmail-qread";
my $file2 = "/var/log/qmail-qread1";

if (-e $file2 ) {
        open (MyFILE, $file2);
} else {
        open (MyFILE, $file1);
}

while (<MyFILE>) {
         if (/^\d{1,2}/) {
             push @Logs, $Log if $Log;
             $Log = '';
         }
         $Log .= $_;
     }

close MyFILE;

push @Logs, $Log;

print grep {/@ARGV/} @Logs;

exit 0;

Relevant Pages

  • Re: modifying putty to include the symmetric key in the logs
    ... 'keyspace' is an array of 'unsigned char' values, ... Of course such an attacker would become able to ... logged in cleartext _in_ the log file, so an attacker able to read ...
    (comp.security.ssh)
  • Re: write function
    ... >> Best Regards, ... I had meant to indicate the array element count with the ... second sizeof parameter and I indicated it incorrectly. ... number of chunks to be written. ...
    (comp.lang.cpp)
  • Re: regExp
    ... > It uses ReadAllto store the entire log file into an array then it uses ... ReadAll() is not suitable for "find" style log parsing because it's ... Of course if it's just a script for Juan to run himself, ...
    (microsoft.public.scripting.wsh)
  • Re: [PATCH] mm: use a radix-tree to make do_move_pages() complexity linear
    ... radix tree which would also avoid the vmalloc but still keep the need to ... * do_pages_statmay easily be rewritten without the huge pm array. ... * If we split the migration in small chunks, ...
    (Linux-Kernel)
  • Re: Releasing memory
    ... There something I don't understand in the object array solution. ... You say to create an array of object instances to obtain a single big memory ... instances placed in different chunks of memory? ... > Seperate key data from the real data. ...
    (microsoft.public.vb.general.discussion)