Re: timeit module: am I missing something obvious?

Peter Otten <__peter__@xxxxxx> writes:

Steven D'Aprano wrote:

When using the timeit module, you pass the code you
want to time as strings:
....
This is all very well, but it feels quite unnatural to
me. Why am I passing strings around when functions are
first class objects? Have I missed something obvious?

You are supposed to time small pieces of code where a function call would be
a significant overhead.

I think it can be useful to time longer code as well.

Moreover, when you pass in strings, you have to ensure that
the namespace is set-up correctly, which can be awkward at
times. If you could pass in a function, this would often
be much easier.

Just do it :-)

Yes, please do!

I would suggest a slight modification:

t = timeit.Timer.for_function(foo, x, y, a=..., b=...)

And while you are at it you could also move the code that dynamically
adjusts the number of repetitions from the main() function into a Timer
method and make its invocation the default for the timeit() method.

I agree.

Moreover, I think the calibration could be improved a lot. What I use
is below. It would be great if something like this could be merged
into timeit.py...

Notice how quickly it locks onto an appropriate number of iterations.
(Only 3% of the time is spent on calibration.)

import timer
timer.timing("3*2/6", goal=0.5, repeat=3, verbose=True)
1 loops -> 1e-05 secs
49933 loops -> 0.0396 secs
630325 loops -> 0.499 secs
630325 loops -> 0.499 secs 0.5 secs 0.501 secs
630325 loops, best of 3 trials: 0.792 usec per loop

Dan


"""Uses timeit to benchmark code, quickly choosing number of iterations.

Differences:

- This can be used interactively from python, not just from the
command line.
- Doesn't use separate creation and timing steps.
- The algorithm for choosing the number of iterations is faster
and more accurate. It also allows for just 1 iteration.
- The result from the final pass through the convergence loop
is used as one of the samples, which saves time
(especially if repeat=1).

Sample usage:

import timer
timer.timing("sin(10)",setup="from math import sin",repeat=3)
import time
timer.timing("3*2/6",repeat=3,goal=0.5,timer=time.time)

To do:

- add support for passing in a function instead of a string.
"""

import timeit
import time

def timing(stmt="pass",
setup="pass",
repeat=1,
goal=None,
number=None,
timer=timeit.default_timer,
verbose=False,
precision=3):
'''Eval setup, and then time stmt.
goal is desired time in seconds, a float.
number is desired number of iterations.
At most one of goal and number can be specified.
If neither is specified, default to a goal of 1 second.
When number is not specified, it is determined in an
efficient way.
repeat is the total number of times to run the test.
timer is the timer to use.
if verbose, show information about calibration.
precision is the number of significant digits to display.
'''

t = timeit.Timer(stmt,setup,timer=timer)
r = []

if number != None and goal != None:
raise ValueError,'Only one of number and goal may be specified.'

if number == None:
if goal == None:
goal = 1.0
# determine number so that total time >= 90% of goal.
number = 1
x = 0.0
while x < 0.9*goal:
x = t.timeit(number)

if verbose:
print "%9d loops -> %.*g secs" % (number, precision, x)
if x == 0:
# 1e2 is chosen because this is a common resolution
# for time.clock. Make sure that number gets increased
# by at least one, to avoid an infinite loop.
number = max(number+1,int(number*1e2*goal))
elif x < 0.9*goal:
# If x is very small, goal/x is large (possibly inf);
# to be cautious we limit the ratio to 1e6.
# The +1 is to ensure that number increases by at least 1,
# since goal/x > 1.
number = int(number*min(1e6,goal/x)+1)
r=[x]
repeat -= 1

r.extend(t.repeat(repeat, number))

best = min(r)
if verbose:
print "%9d loops ->" % number,
print " ".join(["%.*g secs" % (precision, x) for x in r])
print "%9d loops, best of %d trials:" % (number, len(r)),
usec = best * 1e6 / number
if usec < 1000:
print "%.*g usec per loop" % (precision, usec)
elif usec < 1e6:
print "%.*g milliseconds per loop" % (precision, usec/1000)
else:
print "%.*g seconds per loop" % (precision, usec/1e6)
.

Relevant Pages

  • Re: C# coding guidelines: use "this." or not when referring to member fields/properties within the
    ... Alphabetic for strings isn't quite so ... One example of where people go wrong is when they want to optimise loop ... implementation so that each iteration takes 10% less time will only ...
    (microsoft.public.dotnet.languages.csharp)
  • RE: Error handling in a Do Loop
    ... I made the changes you suggested to my strings - blanking them correctly now. ... have any further suggestion or pointers, ... > When loop is able to ping a computer and able to pull the information, ... when the loop does not ping a computer/unable to ...
    (microsoft.public.windows.server.scripting)
  • Re: Can this loop be made faster ?
    ... |> Optimisation: maximal 16 iterations for loops which contain one branch, ... |> otherwise the branch-prediction-table will reload (8 clocks penalty ... |> every/after 16 iterations on AMD). ... but if a second Jcc is within the loop then there are more ...
    (alt.lang.asm)
  • Re: Stupid Newbie Needs Help
    ... Without the loop the program works fine with the ... with 0-terminated strings, that way you can take advantage of C's ... hold up to 10 tokens, where each token may be up to 80 characters ... should give a clue of how the variable/constant/function/macro is ...
    (comp.lang.c)
  • Re: Can this loop be made faster ?
    ... increments, of course, the MEMORY at eax ... at least within the loop. ... maximal 16 iterations for loops which contain one branch, ... | routine, and the time is the divided by 100000 after all the calls. ...
    (alt.lang.asm)