Re: Choosing Perl/Python for my particular niche
From: Fred Ma (fma_at_doe.carleton.ca)
Date: 28 Mar 2004 00:28:07 GMT
Re: Choosing Perl/Python for my particular niche
"Eric @ Zomething" wrote:
> Roy Smith wrote:
> > For most general purpose tasks, I reach for Python first. But if I needed
> > to provide a quick scripting interface to an existing C library,
> I would suggest that one might take an option other than the either/or
> (Perl/Python/____). It sounds like having at least some comfort with Perl is
> important to OP based on relevant code base. However, Python is so simple to
> learn, at least at the beginner/intermediate level, that there is not much
> cost to learning some Python as well.
> Personal experience: I struggled to becoming proficient with Perl due to its
> syntax and perhaps my lack of bottomless commitment (knowing Perl would not be
> my bread and butter). Somehow I stumbled upon a little bit of Python and got
> sucked in. Now I am probably more proficient at Perl than I was before I
> started using Python (except for needing to look up Perl syntax from time to
> time) because working with Python took my head out of the syntax and let my
> mind focus on the problem solving... and more importantly, the objective.
> Python has me feeling like I can make a program to do _anything_ I want, even
> though I doubt it is any more capable than Perl, etc. It's matter of clarity
> and better being able to see the forest, rather than a whole lot of trees.
> This might sound like something less than a firm choice, but perhaps OP might
> decide to start digging into Perl in April, but allow one weekend in that
> month for Python and let things evolve as best suits his situation.
Eric, you voiced exactly the opinion that was forming in my mind upon reading
the responses. There is a complicating factor (actually the most pressing
factor) behind my either/or approach. That is, I must wrap up my thesis very
soon, and what I've proposed is alot more than what there is time for. However,
given the responses, I agree that that most reasonable approach is to adopt one
and keep tabs on the other. Perl seems to be the natural choice for the first,
since I see it in alot of places, meaning I will probably have to poke through
it often. And the ease of Python will make it feasible to keep in touch with
it, if not immediately, then with time.
Aside from hacking other people's Perl code, I've already bloodied my nose on
Perl composition (quite a bit harder than small modifications to other people's
code, I found), so I'm already down that path a bit. I expect to be using this
primarily for "glue" work i.e. to mulch information meant for one utility (e.g.
verilog) so that it can be used for another e.g. verilator, or my own code.
For the computation engine, I will stick with C++ and matlab (for its graphing),
again due to the lack of time to ramp up thoroughly on something else. Also,
despite the availability of Perl code for evolutionary algorithms, I found that
in exploring algorithms, one can very quickly depart from the funcionality
offered by a standard package. Even C/C++ libraries for these algorithms
presume a certain general framework which can impact speed; I didn't use them,
but if I did, I expect I would have diverged from the code base very early.
On the topic of speed, It's surprising to hear that this can be possibly rivaled
by Perl/Python, considering that even my matlab code is about 10x slower than
C++. That's with extensive profiling and round-about coding styles to exploit
vectorization tricks, and no such effort in the C++ code. Some of the
limitations are pretty fundamential e.g. large speed penalties for invoking
nonbuiltin functions. Such limitations discourage clear coding practice e.g.
huge swaths of code to avoid encapsulating some functionality in its own
function. Together with the sometimes roundabout coding style, it can detract
from the matlab's high level ease of programming and clarity, especially
considering that C++ can be brought to a high level with its STL library. And
this speed disparity shows up despite matlab's mission to develop in a way that
rivals 3G languages. Speed considerations aside, however, matlab is infinitely
easier to use and to read. With time, perhaps the limitations will shrink.
This is not to say that Perl/Python will have the same limitations. They are,
however, more general purpose languages, so I expect that speed is less of a
priority than in scientific computing (compared to other important priorities).
Hence, learning curve aside, some time needs to be invested in exploring its
speed in a realistic scientific applications. For me, this is an option for the
more distant future rather than the immediate term. I spent quite alot of time
in this phase with matlab, trying to squeeze out all possible speed before
conceding that some of the limits were fundamental, at least in the current
One thing I learned was that speed characterization was not a simple. Many of
the tasks found in scientific computing are done blistering fast by matlab. It
is not until you put together a larger application when you start running into
the limitations. Aside from the speed penalties of calling nonbuiltins,
roundabout coding style is not such a big deal for small chunk of code. It can
be debilitating when you're trying to wrap your head around something large.
The large chunks of inline code exacerbate this.
A final reason to stick with C++ is the hardware simuation library ("SystemC").
The Perl translator I alluded to in an earlier post translates code in hardware
description language ("verilog") to C++ specifically for use with SystemC (well,
it does more than that, but that's my main use of it). This allows my C++ code
to more directly control the hardware model and its simulation, in contrast to
using a standalone simulator to simulate the original verilog. Depending on the
level of poking and prodding, it can be extremely arduous to go through the API
of the verilog simulator. This is especially true if you intend to change the
hardware models alot between simulations (or do things to achieve the same
effect), and all the more true if the functionality outside of the verilog
simuation is meant to supervise the various simulations instead of being
subordinate to them. An alternative is to have the C++ (or whatever) parent
program generate different verilog code between simulations and invoke the
simulator each time. This would be far too slow.
I realize that my inclination to write my own code sort of contradicts my want
to springboard off of existing Perl code base. I only had a vague idea of the
benefits of this, since I don't actually program in Perl right now. There are
several points I've tried to determine the relevance of. On one hand, I've
looked at and hacked Perl code to get things done, so knowing Perl helps. On
the other hand I need a good way to repackage data for different applications,
in which case I'll compose code from scratch. For this, I would appricate the
lesser pain and greater ease of Python (I imagine). On yet another hand, I use
Perl-based packages like the verilog-to-C++ translator, but I don't expect to
delve into such a complicated package, whose complexity probably lies in more
than just use of the Perl language. Finally, and most vaguest of all, I have
this idea that I might eventually be sharing Perl code, collaborating in its
composition, or revising/maintaining it. If "synergy" in that form is actually
typical in my area of specialization, then of course it would make sense to
start with Perl first, since that is the prevalent language in this area.
-- Fred Ma Dept. of Electronics, Carleton University 1125 Colonel By Drive, Ottawa, Ontario Canada, K1S 5B6