Re: Very slow

Thanks Ben,

What the code actually is doing is to count the fields while respecting the number of \ escaped commas.
A 2k+1 number of slashes \ \\\ \\\\\ ... indicating a non separator
A 2k number of slashed \\ \\\\ ... is a valid comma separator
So your code does not give always correct results.
Have a look at the update I post. At my suprise the codes

while ( ... ) { ... } or
until ( ... ) { ... } or
for ( ... ) { ... }

are killing Perl speed. I can not explaing why but the aproach of

do
{
}
while ( ... )

is much faster. Time from 7 minutes decreased to 20 seconds (the same as your speed). I am not very happy with that, because my C collegue is loughing because perl cannot "for"














Στις 12/1/2012 4:21 μμ, ο/η Ben Morrow έγραψε:

Quoth "George Mpouras"<nospam.gravitalsun@xxxxxxxxxxxxxxxxxx>:
Create a test file with 20000000 same lines of 50 commas (it will be
1020000000 bytes)
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

Now I have a perl and C program running almost the same code. Perl needs
about 6 minutes to finish while the C version finishes at 20 seconds.
The difference is huge. What can I do for a faster perl version
Following the two programs

--== C ==--
<snip>
while (getline(&line,&len, stdin) != -1) {
n = 1;
^^^
0, otherwise you get an off-by-one error

<snip>
#!/usr/bin/perl

my $NC = 1;
open DATA, '<', '/work/test.txt' or die "$^E\n";

while (<DATA>)
{
my ($i, $n, $Length) = (0, 1, length $_);

while ( $i++< $Length )
{
my $c = substr $_, $i, 1;
if ( $c eq ',' ) { $n++ } elsif ($c eq '\\') {$i++}
}

$NC = $n if $n> $NC
}

close DATA;
print "Fields $NC\n"

Well, try not writing C in Perl :).

~% time ./ccommas<commas
50
0m3.29s real 0m3.08s user 0m0.20s system
~% time perl plcommas
Fields 50
6m37.54s real 6m35.89s user 0m1.61s system
~% time perl -nE'
BEGIN { $NC = 1 }
s/\\,//g; $c = tr/,//;
$c> $NC and $NC = $c;
END { say "Fields $NC" }'
Fields 50
0m16.74s real 0m15.48s user 0m1.25s system
~%

So, not quite as fast as the C version, but not too bad.

Ben


.

Relevant Pages

  • Re: dwarf stars histogram again
    ... >> CPU usage, so perl is out. ... Unless I hit a brick wall, ... sys 0m0.080s ...
    (comp.unix.shell)
  • why a.pl is faster than b.pl
    ... I have two perl scripts as following: ... use strict; ... open or die "$!"; ... time perl a.pl> /dev/null ...
    (perl.beginners)
  • Re: Random not completely random
    ... houghi writes: ... You know - in your script you called perl twice. ... Each time perl has to ... Could you pre-generate thousands of values ahead of time? ...
    (comp.unix.shell)
  • Re: Very slow
    ... Now I have a perl and C program running almost the same code. ... Perl needs ... ~% time perl plcommas ...
    (comp.lang.perl.misc)
  • Re: know-how(-not) about regular expressions
    ... "appending" to a variable. ... Perl would have to spend all its time on reallocbecause ... To isolate what I am seeing, I am posting a benchmark that simulates ...
    (comp.lang.perl.misc)