Re: A question on python performance.

On Sep 26, 2:26 pm, "Joe Goldthwaite" <j...@xxxxxxxxxxxxxxxx> wrote:
Hi everyone,

I'm a developer who's been using python for a couple of years. I wrote a
fairly large application using it but I was learning the language at the
same time so it most of the code kind of sucks.

I've learned a lot since then and I've been going through my code trying to
organize it better and make better use of Python's features. I'm still not
an expert by any definition but I'm slowly getting better.

I've been working on a trend class that takes twelve monthly numbers and
returns a period to date, quarter to date, year to date and quarterly year
to date numbers for a specific period. This worked but I ended up with a lot
of code like this;

def getValue(trend, param, per):
if param == 'Ptd':
return trend.Ptd(per)
elif param == 'Qtd':
return trend.Qtd(per)
elif param == 'Ytd':
return trend.Ytd(per)
elif param == 'YtdQ':
return trend.YtdQ(per)

The code gets kind of wordy so I started trying to figure out how to call
them dynamically since the param type is the same as the method the
retrieves it. I came up with this;

def getValue(trend, param, per):
return trend.__class__.__dict__[param](trend, per)

That worked but it seems like the above line would have to do lots more
object look ups at runtime so I didn't think it would be very efficient. I
thought maybe I could add a caller method to the trend class and I came up
with this;

class trend:
...
...
...
def caller(self, param, *args):
return self.__class__.__dict__[param](self, *args)

This simplified the getValue function to this;

def getValue(trend, param, per):
return trend.caller(param, per)

Out of curiosity, I thought I'd do some benchmarking and see which one
performs the best. I executed three multiple times;

loop one. Time=11.71 seconds;
trend.Ptd(per)
trend.Qtd(per)
trend.Ytd(per)
trend.YtdQ(per)

loop two. 12.107 seconds;
trend.__class__.__dict__['Ptd'](trend, per)
trend.__class__.__dict__['Qtd'](trend, per)
trend.__class__.__dict__['Ytd'](trend, per)
trend.__class__.__dict__['YtdQ'](trend, per)

loop three. 17.085 seconds;
trend.caller('Ptd', per)
trend.caller('Qtd', per)
trend.caller('Ytd', per)
trend.caller('YtdQ', per)

The first surprise was how close the first and second loops were. I would
have thought the first loop would be much faster. The second surprise was
how much slower the third loop was. I know it has an extra call in there
but other than that, it's doing basically the same thing as loop two. Is
there that much overhead in making a class method call?

Can anyone explain the differences?

Makes perfect sense to me! Think about it:

method 1: looks up the method directly from the object (fastest)
method 2: looks up __class__, then looks up __dict__, then gets the
element from __dict__
method 3: looks up caller, looks up __class__, looks up __dict__, gets
element from __dict__

To get the element directly from the object (method 1), Python has to
internally check __class__.__dict__[element], which shows why method 1
and method 2 are nearly the same speed. The last version has to look
up caller in addition to the process described by method 2.

The best way to do what you are doing:

getattr(self, param)(self, *args)

.

Relevant Pages

  • A question on python performance.
    ... def getValue(trend, param, per): ... thought maybe I could add a caller method to the trend class and I came up ... loop one. ...
    (comp.lang.python)
  • Re: Generalized Method of Moments
    ... fhandle = myfunction; %here I changed as response to ... don't know the stopping loop value of param since I want to estimate ...
    (comp.soft-sys.matlab)
  • Delay in calls to stored procedure
    ... I'm running through a loop calling a stored procedure each time. ... the SqlCommand before entering the loop and just change the param values in ... the loop before each call to ExecuteNonQuery(). ... // assign param values ...
    (microsoft.public.dotnet.framework.adonet)
  • Re: c:if test with dynamic variable
    ... xyz_3 and I want to test if these variable are passed as param ... name in loop. ... forexample.. ... where var generates xyz_1, xyz_2,xyz,_3 .. ...
    (comp.lang.java.programmer)
  • Re: How to create "def method(item)= (value)" ?
    ... do as I wrote further on and have the param function return ... def initialize hash ... validation code, or having the object.param=value update something ...
    (comp.lang.ruby)