Re: Nested scopes and class variables

From: Nick Coghlan (ncoghlan_at_iinet.net.au)
Date: 01/31/05


Date: Mon, 31 Jan 2005 20:15:42 +1000
To: Python List <python-list@python.org>

Dave Benjamin wrote:
> I ran into an odd little edge case while experimenting with functions that
> create classes on the fly (don't ask me why):

It gets even kookier:

Py> x = 5
Py> def f(y):
... class C(object):
... x = x
... print C.x
...
Py> f(5) # OK with x bound at global
5

Py> def f(x):
... class C(object):
... x = x
... print C.x
...
Py> f(6) # Oops, ignores the argument!
5

Py> def f(y):
... class C(object):
... x = y
... print C.x
...
Py> f(6) # OK with a different name
6
Py> y = 5
Py> def f(y):
... class C(object):
... x = y
... print C.x
...
Py> f(6) # Correctly use the nearest scope
6

That second case is the disturbing one - the class definition has silently
picked up the *global* binding of x, whereas the programmer presumably meant the
function argument.

With a nested function definition (instead of a class definition), notice that
*both* of the first two cases will generate an UnboundLocalError.

Anyway, the Python builtin disassembler is very handy when looking at behaviour
like this (I've truncated the dis output below after the interesting section):

Py> import dis
Py> def f1(x):
... class C(object):
... x = x
... print C.x
...
Py> def f2(y):
... class C(object):
... x = y
... print C.x
...

Py> dis.dis(f1)
   2 0 LOAD_CONST 1 ('C')
               3 LOAD_GLOBAL 0 (object)
               6 BUILD_TUPLE 1
               9 LOAD_CONST 2 (<code object C at 00B3E3E0, file "<s
tdin>", line 2>)
[...]

Py> dis.dis(f2)
   2 0 LOAD_CONST 1 ('C')
               3 LOAD_GLOBAL 0 (object)
               6 BUILD_TUPLE 1
               9 LOAD_CLOSURE 0 (y)
              12 LOAD_CONST 2 (<code object C at 00B3E020, file "<s
tdin>", line 2>)
[...]

Notice the extra LOAD_CLOSURE call in the second version of the code. What if we
define a function instead of a class?:

Py> def f3(x):
... def f():
... x = x
... print x
... f()
...

Py> def f4(y):
... def f():
... x = y
... print x
... f()
...

Py> dis.dis(f3)
   2 0 LOAD_CONST 1 (<code object f at 00B3EA60, file "<s
tdin>", line 2>)
[...]

Py> dis.dis(f4)
   2 0 LOAD_CLOSURE 0 (y)
               3 LOAD_CONST 1 (<code object f at 00B3EC60, file "<s
tdin>", line 2>)
[...]

Again, we have the extra load closure call. So why does the function version
give us an unbound local error, while the class version doesn't?. Again, we look
at the bytecode - this time of the corresponding internal code objects:

Py> dis.dis(f1.func_code.co_consts[2])
   2 0 LOAD_GLOBAL 0 (__name__)
               3 STORE_NAME 1 (__module__)

   3 6 LOAD_NAME 2 (x)
               9 STORE_NAME 2 (x)
              12 LOAD_LOCALS
              13 RETURN_VALUE

Py> dis.dis(f3.func_code.co_consts[1])
   3 0 LOAD_FAST 0 (x)
               3 STORE_FAST 0 (x)

   4 6 LOAD_FAST 0 (x)
               9 PRINT_ITEM
              10 PRINT_NEWLINE
              11 LOAD_CONST 0 (None)
              14 RETURN_VALUE

In this case, it's the LOAD_FAST opcode that blows up, while the LOAD_NAME falls
back on the globals and then the builtins. Looking at the class based version
that works also lets us see why:

Py> dis.dis(f2.func_code.co_consts[2])
   2 0 LOAD_GLOBAL 0 (__name__)
               3 STORE_NAME 1 (__module__)

   3 6 LOAD_DEREF 0 (y)
               9 STORE_NAME 3 (x)
              12 LOAD_LOCALS
              13 RETURN_VALUE

Here we can see the "LOAD_DEREF" instead of the "LOAD_NAME" that was present in
the version where the same name is reused. The dereference presumably picks up
the closure noted earlier.

I vote bug. If the assignment is going to be allowed, it should respect the
nested scopes.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net


Relevant Pages

  • Re: Anonymus functions revisited : tuple actions
    ... Checks locals, globals, and builtins for the varable name. ... def isa: ... otherwise return the varable obj. ...
    (comp.lang.python)
  • Re: modifying source at runtime - jython case
    ... modification in class definition. ... >>> def test: ... You can modify the behaviour of all existing instances of a class, e.g. their methods, by modifying the class itself. ... Any reference to the method on the instance will resolve to the method definition from its class, provided you haven't overwritten the method on the individual instance. ...
    (comp.lang.python)
  • Re: PEP-343 - Context Managment variant
    ... > def Synhronised: ... 'globals' means the module global namespace as returned by globals. ... We must declare local vars, to which we wish assign in "global" ... You cannot rebind enclosed local variables, ...
    (comp.lang.python)
  • Re: singletons
    ... this is a common approach in python. ... The one drawback to this is that it could require lots of globals ... def something: ...
    (comp.lang.python)
  • Re: undo a dictionary
    ...     for key, value in dd.items: ... globals() as the default names space, ... def undict(self, dict): ...
    (comp.lang.python)