Re: Tcl faster than Perl/Python...but only with tricks...

Alexandre Ferrieux wrote:

A simple way of removing all the "first time" overheads (interpreter
init, bytecode compilation) would be to do the comparison on a file ten
times larger (just cat it over and over again).

Good point. I made the file 100 times larger now, so they do not fit into
memory and no in-memory-processing has an advantage (which destroys my
first and fastest solution with reading the whole file and the using
regexpt to that large string). I used the fastest Tcl version mentioned
here until now (Uwe Klein: while and gets in a proc).

Perl: 35.960s
Python: 1m, 31.896s
Tcl: 1m, 19.639s

But *if* [gets] is truly slower than it should (I'm not expecting
[while] to be the culprit !), then I'd look around the -translation
(the crlf) and -encoding (unicode conversion) aspects of the channel
(remember there's a mandatory unicode conversion for regexp; I don't
know if this could be a realistic cause for the slowdown though...).

I'm not sure how to eliminate that with fconfigure, but I doubt, that this
would be much improvement. When I commented out the regexp, I got only a
speedup of factor 2, which means half of the runtime is used by reading the
file with [gets] in a [while].

But your comment made me dig a little bit deeper, so I made this test: The
new very big file has 11.715.400 lines (819MB). The file can be read with
the following script:

---
set f [open bigfile r]
puts [time {
gets $f line
gets $f line
gets $f line
gets $f line
gets $f line
gets $f line
gets $f line
gets $f line
gets $f line
gets $f line
} 1171540 ]
---

So only the [gets] counts. The result is:
59.259s runtime total and 50.3546485822 microseconds per iteration.

Doing the same with ten times "while {1} break" runs only 2.338s. So
obviously [gets] is the slow part. So when processing files line by line,
one should try to get around [gets], if it should be really fast... The
regexp is not the bottleneck, which at first surprised me a little, but
delights me also and fits my experience when I once wrote the same parser
heavily using regular expressions in Python and Tcl, where the Tcl version
was, depending on the input, up to ten times faster.

Regards
Stephan

.

Relevant Pages

  • Re: TCL with Perl or Python?
    ... Now that you mentioned web guis, tcl is far behind on this :-( ... I want to add a small web gui to an existing app, and still I haven't found a solution... ... using Python or Perl. ...
    (comp.lang.tcl)
  • Re: Tcl faster than Perl/Python...but only with tricks...
    ... I have tried some small variations by mimicking your first python code. ... the tcl code that does similar to the python code is: ... python python.py: 582543 microseconds per iteration ...
    (comp.lang.tcl)
  • Re: Choosing Perl/Python for my particular niche
    ... >I was under the impression that Tcl ... >it's good to know that Tcl is an option to Perl and Python, ... >algorithms for programming coarse grain reconfigurable ...
    (comp.lang.perl.misc)
  • Re: Choosing Perl/Python for my particular niche
    ... >I was under the impression that Tcl ... >it's good to know that Tcl is an option to Perl and Python, ... >algorithms for programming coarse grain reconfigurable ...
    (comp.lang.python)
  • Re: Python for TCLer
    ... >I am almost brainwashed to forsake TCL for Python. ... >to ask a few preliminary questions before I jump in. ... One of my vendors supplies TCL package only. ... We have one control structure in our API that has done ...
    (comp.lang.python)