Re: python vs awk for simple sysamin tasks

From: Steve Lamb (grey_at_despair.dmiyu.org)
Date: 06/05/04 

Date: Fri, 04 Jun 2004 22:59:50 GMT

On 2004-06-04, William Park <opengeometry@yahoo.ca> wrote:
> Too many maybe's, and you're timing hypotheoretical tasks which you hope
> it would take that long.

    No, not too many maybes. That was based on a real-life experience which
is but a single example of many cases over the past several years of working
on my field (ISP/Web Hosting).

> Because that's what Shell/Awk is for... write, throw away, write, throw
> away, ...

    And you don't consider this wasteful. Ooook then.

>> It is amazing how much shell bogs down when you're running it over
>> several hundred thousand directories. :P

> Same would be true for Python here.

    Not as much as shell. With shell you have thousands upon thousands of
fork/execs getting in the way. In the case I am thinking of rewriting the
shell script into Perl and doing all the logic processing internally cut the
run time down from 7-8 hours down to *2*. I have seen similar perfomance
numbers from Python when compared to shell though not on that scale. I mean
the same could be said for C, or Assembly. Yes, iterations over large sets of
data is going to increase runtimes. However there are inefficiencies not
present in a scripting language which are present in shell which make it
exceptionally longer to run.

> "If hammer is all you have, then everything becomes a nail". :-)

    Fortunately I don't have just a hammer then, isn't it? I restate, I use
shell for one liners. Beyond that it has been my experience, practice and
recommendation that far more time is saved in the long run by using the proper
tool. A proper scripting language and not a hobbled-together pseudo-language
with severe performance issues. 

-- 
         Steve C. Lamb         | I'm your priest, I'm your shrink, I'm your
       PGP Key: 8B6E99C5       | main connection to the switchboard of souls.
-------------------------------+---------------------------------------------

Relevant Pages

  • Re: [PSH] Methods/functions and blank spaces
    ... to make because PowerShell is both a shell and a scripting language. ... Thus we need to support whitespace as a parameter delimiter. ...
    (microsoft.public.windows.server.scripting)
  • Re: [PSH] Methods/functions and blank spaces
    ... to make because PowerShell is both a shell and a scripting language. ... Thus we need to support whitespace as a parameter delimiter. ...
    (microsoft.public.windows.server.scripting)
  • Re: A Different Shell Philosophy
    ... in tcsh I might type something like this ... I'd prefer to make easy tasks easy and difficult tasks ... > in virtually every scripting language. ... Shell is a lot better to do ...
    (comp.unix.shell)
  • Re: *attaching* a file to /usr/bin/mail message
    ... > I'm not stuck with mail, I use it with some shell scripts....but yeah, I ... If you're doing scripting, why not use an actual scripting language, and ...
    (freebsd-questions)