Re: Tracking Dependencies

Solved with simple script! (See below for info and script.)

Hal Vaughan wrote:

> I have some modules that started small (don't they all??), and has my
> project has grown, have gotten large enough that I could split them in 2
> or
> more smaller logical blocks, creating 2,3 or 4 modules instead of 1.
> While doing I've been doing this, I also have programs that have been
> revised that they may no longer need all the modules I currently have them
> using.
>
> I know when I run a Perl script and it calls a non-existing routine, I'll
> get an error message, but it's not always possible to quickly or easily
> make sure every possible path of program flow is checked to be sure of
> this.
>
> I was thinking of writing a script that would go through each program or
> module I have, collect the names of all routines called and compare them
> against Perl's own functions and all the functions defined in my modules,
> but I figure there's already a way to do that either built into Perl, or
> on CPAN (which seems to be down at the moment -- or at least I can't reach
> it
> from here). The only keyword I can think of to describe what I'm trying
> to do is dependency, and I'm not finding much useful using that.
>
> So is there a way to track the functions called in a program or module and
> get a listing of whether those functions are built-ins, defined in the
> program, or what module each function is defined in?
>
> Thanks for any help on this!
>
> Hal

I didn't get any info on packages or any other way to do this, and, needing
it resolved, I did a 3am hacked script, then commented it this morning. It
has a lot of shortcomings, but it works in general -- at least for me, so I
thought I'd post it here. (At end.) I am not interested in turning this
into a project. It works for me, and that's enough. However, I'd be
interested in hearing from people who find it useful.

Hal

=================Program Listing:========================

#!/usr/bin/perl
#==============================================================================
#Function Call Scanner (callscan)
#
#By Hal Vaughan
#Contact at hal@xxxxxxxxxxxxxxxxxxxx
#Licensed under GPL
#(c) 2005 Hal Vaughan
#
#(If you try to contact me, include something like "Perl Programming" in the
#subject to make it easy to see it isn't spam. If you don't get a reply, in
#a few days, somehow SpamAssassin classified your e-mail as spam, so try
#again.)
#
#I can't believe I'm writing this!! I would think there's been a need for a
#program like this before, so I can't see why there isn't something like
#this already out there. This is a 3 am hack so I could go through my
#modules and programs and elminate extra dependencies that cropped up during
#late nate debugging sessions.
#
#WHAT IT DOES: Run this program with one or two arguments (described below),
#one provides a list of directories for programs, the othe describes a list
#of directories for modules. First, we get the list of program and module
#directories from the arguments, then we scan the list of module directories
#and read in each module. We look for @EXPORT = qw() in the modules (you
#may want to change this if you use a different format for exporting
#function names). We pull out the list of functions and split it into an
#array. Once that is done, we go through all the programs and look for each
#use of each function in each module and print out a report, listing the
#line where a function name is seen, including the line number. We then do
#the same with the modules -- making sure when we're reading a module to
#skip scanning it for calls to its own functions.
#
#Arguments:
# --programs={comma separated list of programs}
# --modules={comma separated list of modules}
#
#NOTE: There are a number of weaknesses to this program, and I'm sure there
#is something much better on CPAN or elsewhere that I could not find and
#nobody on the newsgroups could point me to a package to do this. So, it
#was a quick hack to help me check which programs need actually needed which
#modules.
#
#Known shortcomings:
#- It will give false hits. For example, if you have a function named
#"scan", and a program calls "datascan", it will be reported as using the
#function "scan". (Comment lines and empty lines aren't checked.) This
#could be fixed by looking for "scan(", but it'll still pick up "datascan(".
#To overcome this it would be necessary to look for "scan" preceeded by
#whitespace and other characters that could preceed a function (like ( or
# { ). I'm sure there's more characters that fit that situation, so I
#haven't gone into it.
#- If you don't use "@EXPORT = qw();" (the list can be spread through
#different lines), it likely won't pick up the list of functions. This
#works for me, if you need a different format, then you can change the
#regexes.
#==============================================================================

use strict;

our ($dir, $file, $mod, $line, $list, $x);
our (@prog, @mod, @modfile, @moddir, @progdir, @progfile, @list, %call);

#
#Get the lists of program and module directories from the argument list
foreach (@ARGV) {
if ($_ =~ /-{1,2}prog/) {
$line = substr($_, index($_, "=") + 1);
@list = split(/,/, $line);
push(@progdir, @list);
}
if ($_ =~ /-{1,2}mod/) {
$line = substr($_, index($_, "=") + 1);
@list = split(/,/, $line);
push(@moddir, @list);
}
}

#
#Get all the modules in the directories, then get all the programs.
foreach (@moddir) {
$dir = $_."/*pm";
@list = glob($dir);
foreach (@list) {
if (-d $_) {next;}
push(@modfile, $_);
}
}
foreach (@progdir) {
$dir = $_."/*";
@list = glob($dir);
foreach (@list) {
if (-d $_) {next;}
push(@progfile, $_);
}
}
undef(@list);

#
#Now go through all the modules and get a list of all the functions in each
#module. Put it in a hash, with the module file name as the key
for $file (@modfile) {
$line = filetostring($file);
($list) = $line =~ /\@EXPORT *?= *?qw\((.*?)\)/is;
$list =~ s/\n/ /g;
$list =~ s/\t+/ /g;
$list =~ s/ +/ /g;
@list = split(/ +/, $list);
@{$call{$file}} = @list;
}

#
#Scan programs and modules for matches to function names
print "Scanning programs:\n\n";
scan(@progfile);
print "\n\nScanning Modules:\n\n";
scan(@modfile);

exit;

#
#scan a list of files (either modules or programs) for matches to functions
#in the different modules. Use 4 loops:
# Loop through all the files given, read each file into an array
# Loop through each target module in the module listing
# Loop through each line of the program/module we're scanning
# Loop through all the functions in the target module to see if they are
# used in that line. If they are, add the line number, the function,
# and the line itself to the output string.
#At the end of going through all the functions in the target module, see
#if there is an output string. If there is, print it out. If not, don't
#print it. Either way, go on to the next target module, then the next file.
sub scan {
my ($name, $file, $line, $mod, $func, $out, $x, $y, @data, @file, @func);
@file = @_;
for $x (0..$#file) {
$file = $file[$x];
$name = $file;
$name =~ s/.*?\///;
print "File: $name\n";
@data = filetoarray($file);
for $mod (sort keys %call) {
if ($mod eq $file) {next;}
@func = @{$call{$mod}};
$out = "";
for $y (0..$#data) {
$line = $data[$y];
if ($line =~ /^\s*#/ || !$line) {next;}
for $func (@func) {
if ($line =~ /$func *\(/) {
$line =~ s/\t//g;
$line =~ s/\s+/ /g;
$out = "$out\t\t$y\t$func\t$line\n";
}
}
}
if ($out) {
$line = $mod;
$line =~ s/\//::/g;
print "\tModule: $line\n$out";
}
}
}
print "\n";
return;
}

#
#Quick way to load a file into a string.
sub filetostring {
my $file = shift(@_);
my $eol = $/;
undef($/);
open (STREAM,"<$file");
my $data = <STREAM>;
close STREAM;
$/ = $eol;
return $data;
}

#
#Quick way to load a file into an array.
sub filetoarray {
my $file = shift(@_);
my $data = filetostring($file);
my @data = split(/\n/, $data);
return @data;
}

.

Relevant Pages

  • Re: Question for Paul and other AppleScripters
    ... I'm writing a script to do batch edits on my contacts. ... > objects or use a reference to to loop through the list of selected contacts to ... If you want to use AppleScript, ... enable whose clauses on lists. ...
    (microsoft.public.mac.office.entourage)
  • Re: Convert CSV file to multiple worksheets
    ... as used by the Microsoft script to iterate the MOF products list of software installed on each computer. ... The For loop works whether there is 0, ... The DisplayName retrieved by the MS script _is_ the software title. ... Find some way of adding software1 - softwareWhatever to the output line at the top of the script that lists the column headers. ...
    (microsoft.public.scripting.vbscript)
  • Re: Bash problem with find and mv
    ... Thanks for the insight on reading one line at a time. ... now obvious to me why my script didn't work in the first place. ... >> If a user has more than one file, it lists the files one after the other ... > rarely the right way to read a file; use a while loop (or set IFS ...
    (comp.unix.shell)
  • If Then Loop - help with my poor scripting
    ... all GroupID's held in a variable named GroupsAll and another list of ... Under the 2 lists above, I then tried (using a loop within a loop) to ... The script runs OK but no additional records are printed. ...
    (microsoft.public.inetserver.asp.db)
  • If Then Loop - help with my poor scripting
    ... all GroupID's held in a variable named GroupsAll and another list of ... Under the 2 lists above, I then tried (using a loop within a loop) to ... The script runs OK but no additional records are printed. ...
    (microsoft.public.scripting.vbscript)
Loading