Re: a little help...

I am only concerned about the IP. The rest is just to verify the data for now. What code would I use to key the $IP in to hash for counting?. Most of the IP's are not static but are from broadband and don't change too often. An example log is:

-------------
[2005-09-28 10:05:03 -7:00] 127.0.0.1 71.32.59.249 216.163.137.3 - http://www.playboy.com/ blocked 0 PO
-------------
the IP I want to count is 71.32.59.249 (for this log) and the category is PO

Ryan Lamberton


----- Original Message ----- From: "Wagner, David --- Senior Programmer Analyst --- WGO" <David.Wagner@xxxxxxxxxxxxxxxxx>
To: "FamiLink Admin" <webmaster@xxxxxxxxxxxx>
Cc: <beginners@xxxxxxxx>
Sent: Wednesday, September 28, 2005 5:18 PM
Subject: RE: a little help...


FamiLink Admin wrote: 
Jeff ,
Thanks for all your help!  This is what I have now (below and this
time the whole thing):   I think I have included all that you talked
about plus others:

The sub scanlog does write the information to the files but it does
not return anything back to the main program and I also get the error:

Use of uninitialized value in split at ./test.pl line 9.

Also, is there a better way of counting the number of times each IP
address gets blocked with category PO?   Each time I get to the
blocklimit it writes to the file but I really just want the max
number of blocks over the limit. It will write the same IP each time
it gets over the blocklimit though.


If you are only concerned about $ip and if they went over that limit and not desiring the detail of said offense, then you could use the $ip as a key into a hash. Then you could count all the occurances. At the conclusion of that processing then you could loop through the hash and any count greater than your max, then you could write to the suspend file. For email, then could again use the hash to put together a list of $ip's that are over your limit.

I have not followed the topic, but unless you do something with the $ip, I would assume that the log is just that a log. You would have interspersed $ip and so I am unsure how you would be able to say $ip is at fault. I see nothing in your code which isolates to the $ip. Again, are these static ip addr or when someone logs out, they are ready for use by someone else. If it is released then you have to figure out when this occurs to get an accurate rcd. If static, then not a problem.

Wags ;)



------------------------------------------------------------------------------
#!/usr/bin/perl -w
require Mail::Send;
$|=1;           # no buffering
use constant IP_LIST_FILE => "/etc/squid/iplist.txt";
use constant SUSPEND_FILE => "/etc/squid/SuspendIpList.txt";
use constant LOG_FILE => "/opt/n2h2/logs/filter_log";
my $sysop = "webmaster\@familink.com";
my $flag = "PO";
my $hour = (split, localtime)[2];
my $blocklimit = 5;
my $matches = 0;
my $matched = 0;
{
        ($matched,$ip,$hour,$time,$category,$url) =
&Scanlog($flag,$hour,$blocklimit,$matches,);
        if($matched > $blocklimit){
          $msg = new Mail::Send Subject=>'SuspendIpList',
          To=>"$sysop"; $fh = $msg->open;
          print $fh "Someone has tried to access $matches banned sites
today\n";
          print $fh "Their IP address ($ip) has been added to
/etc/squid/SuspendIpList.txt\n";
          print $fh "To unblock them, remove their entry from the
file and run squid -k reconfigure\n";
          print $fh "$matches, $ip, $hour, $time, $category, $url\n";
          $fh->close;         # complete the message and send it
          $matched = 0;
       }
        else{
        open my $output2, ">", SUSPEND_FILE or die "Can't write
@{[SUSPEND_FILE]}: $!";
         print $output2 "10.0.0.252/32\n";
        close $output2;
       }
}
sub Scanlog {
        my ($flag,$hour,$blocklimit,$matches,)=@_;
        open my $slog, "-|", "tail -n 25000  @{[LOG_FILE]}" or die
"Unable to open $log:$!\n";
        open my $output, ">", IP_LIST_FILE or die "Can't write
@{[IP_LIST_FILE]}: $!";
        open my $output2, ">", SUSPEND_FILE or die "Can't write
@{[SUSPEND_FILE]}: $!";
        while (my $line = <$slog>){     # assigns each line in turn
           to $line #use an array slice to select the fields we want
           my ($time, $ip, $url, $category) = (split " ",
           $line)[1,4,7,10]; my ($hr) = split /:/, $time;
             if($flag eq $category and $hr eq $hour){
                $matches += 1 ;
             }
             if($matches > $blocklimit){
                print $output "$matches, $ip, $hour, $time, $category,
$url\n";
                print $output2 "$ip/32\n";
                $matched = $matches;
                $matches = 0;
             }
        }
        close $output;
        close $output2;
        return($matched,$ip,$hour,$time,$category,$url);
}



------------------------------------------------------------------
Ryan Lamberton


----- Original Message -----
From: "Jeff 'japhy' Pinyan" <japhy@xxxxxxxxxxxx>
To: "FamiLink Admin" <webmaster@xxxxxxxxxxxx>
Cc: <beginners@xxxxxxxx>
Sent: Wednesday, September 28, 2005 12:24 PM
Subject: Re: a little help...


On Sep 28, FamiLink Admin said:

I am trying to read a log file and get a list of how many times an
IP address get blocked each hour by category PO.  An example line
in the log with a block is: -------------
[2005-09-28 10:05:03 -7:00] 127.0.0.1 71.32.59.249 216.163.137.3 -
http://www.playboy.com/ blocked 0 PO
-------------
What I have kinda works but I am not sure if it is the best
practice. This is the first time programming in perl and this is
what I have so far:


Your indentation leaves much to be desired, so I've "fixed" it.

sub Scanlog {
  local($ipb) = @_;


No reason to use 'local'; stick with 'my' here.  But... what is
$ipb?  You don't use it anywhere!

  open my $slog, "-|", "tail -n 50000 $log" or die "Unable to open
  $log:$!\n"; open (OUTPUT,">/etc/squid/iplist.txt");
  open (OUTPUT2,">/etc/squid/SuspendIpList.txt");


You should also die if neither of those could be opened:

    open(OUTPUT, ">...") or die "can't create /etc/squid/iplist.txt:
$!";

  while (<$slog>){     # assigns each line in turn to $_
    # use an array slice to select the fields we want
    @data = (split ,$_)[1,4,10,5,7];
    $hr = (split /:/ ,$data[0])[0];
    $ip = "$data[1]";


Those three variables should all be declared with 'my'.  Your line
assigning to @data has a typo that hasn't effected you, but it might
eventually.

      my @data = (split)[1,4,10,5,7];  # why out of order?
      my $hr = (split /:/, $data[0])[0];
      my $ip = $data[1];  # no need to quote $data[1] here

if ($flag eq $data[2]) { 

Where is $flag coming from?

if ($hr eq $hour) { 

Where is $hour coming from?

Those two if statements can be combined into one, since you don't do
anything if they aren't both true.

      if ($flag eq $data[2] and $hr eq $hour) {

        foreach (/$data[2]/) {
          $matches += 1 ;
        }


I have a feeling this could lead to false positives.  How do you
know that 'PO' (or whatever else $data[2] might hold) won't appear
in the URL, for instance?  Perhaps this should just be

          $matches++;

But where is $matches coming from?!

if ($matches > $blocklimit) { 

Where does $blocklimit come from?!

$ip1 = "$data[1]/32"; 

Declare that with 'my'.

          print OUTPUT "$matches,", "$hour, ","$ip1, ",
"@data","\n";


You could just write that as

  print OUTPUT "$matches, $hour, $data[1]/32 @data\n";

          print OUTPUT2 "$ip1\n";
          $matched = $matches;
          $matches = 0;


Where did $matched come from?

        }
      }
    }
  }
  close (OUTPUT);
  close (OUTPUT2);
}


You should not use any variables in a function that you did not pass
to it or create IN it.

--
Jeff "japhy" Pinyan        %  How can we ever be the sold short or
RPI Acacia Brother #734    %  the cheated, we who for every service
http://www.perlmonks.org/  %  have long ago been overpaid?
http://princeton.pm.org/   %    -- Meister Eckhart

--
To unsubscribe, e-mail: beginners-unsubscribe@xxxxxxxx
For additional commands, e-mail: beginners-help@xxxxxxxx
<http://learn.perl.org/> <http://learn.perl.org/first-response>




*******************************************************
This message contains information that is confidential
and proprietary to FedEx Freight or its affiliates.
It is intended only for the recipient named and for
the express purpose(s) described therein.
Any other use is prohibited.
*******************************************************


--
To unsubscribe, e-mail: beginners-unsubscribe@xxxxxxxx
For additional commands, e-mail: beginners-help@xxxxxxxx
<http://learn.perl.org/> <http://learn.perl.org/first-response>


.

Relevant Pages

  • Re: a little help...
    ... Each time I get to the blocklimit it writes to the file but I really just want the max number of blocks over the limit. ... open my $output2, ">", SUSPEND_FILE or die "Can't write ... # use an array slice to select the fields we want ...
    (perl.beginners)
  • RE: a little help...
    ... > FamiLink Admin wrote: ... >> The sub scanlog does write the information to the files but it does ... >> it gets over the blocklimit though. ... > $ip as a key into a hash. ...
    (perl.beginners)