Re: Article of interest: Python pros/cons for the enterprise

Paul Rubin wrote:
Jeff Schwab <jeff@xxxxxxxxxxxxxxxx> writes:
The most traditional, easiest way to open a file in C++ is to use an
fstream object, so the file is guaranteed to be closed when the
fstream goes out of scope.

Python has this too, except it's using a special type of scope
created by the "with" statement.

Yes, this seems to be the Python way: For each popular feature of some
other language, create a less flexible Python feature that achieves the
same effect in the most common cases (e.g. lambda to imitate function
literals, or recursive assignment to allow x = y = z).


CPython offers a similar feature, since
you can create a temporary object whose reference count will become
zero at the end of the statement where it is defined:

$ echo world >hello
$ python
>>> file('hello').read()
'world\n'

CPython does not guarantee that the reference count will become zero
at the end of the statement. It only happens to work that way in your
example, because the file.read operation doesn't make any new
references to the file object anywhere.

It doesn't "happen" to work that way in the example; it works that way
by design. I see what you're saying, though, and it is a good point.
Given a statements of the form:

some_class().method()

The method body could create an external reference to the instance of
some_class, such that the instance would not be reclaimed at the end of
the statement.


Other code might well do
something different, especially in a complex multi-statement scope.
Your scheme's

It's not "my" scheme. I got it from Martelli.


determinism relies on the programmer accurately keeping
track of reference counts in their head, which is precisely what
automatic resource management is supposed to avoid.

This is a special case of the reference count being 1, then immediately dropping to zero. It is simple and convenient. The approach is, as you rightly point out, not extensible to more complicated situations in Python, because the reference counting ceases to be trivial.

The point is that once you tie object lifetimes to scope, rather than unpredictable garbage collection, you can predict with perfect ease and comfort exactly where the objects are created and destroyed. If you can then request that arbitrary actions be taken automatically when those events happen, you can pair up resource acquisitions and releases very easily. Each resource has an "owner" object whose constructor acquires, and whose destructor releases. The resources are released in the reverse order, which is almost always exactly what you want.

Suppose you are using objects that have to be closed when you have finished with them. You would associate this concept with a type:

class Closer:
def __init__(self, closable):
self.closable = closable)
def __del__(self):
self.closable.close()

The C++-style paradigm would then let you do this:

def my_func(a, b, c):
a_closer = Closer(a)
b_closer = Closer(b)
c_closer = Closer(c)

# ... arbitrary code ...

If an exception gets thrown, the objects get closed. If you return normally, the objects get closed. This is what "with" is supposed to replace, except that it only seems to cover the trivial case of a single, all-in-one cleanup func. That's only a direct replacement for a single constructor/destructor pair, unless you're willing to have an additional, nested with-statement for each resource.

Now suppose there is an object type whose instances need to be "released" rather than "closed;" i.e., they have a release() method, but no close() method. No problem: You have the Closer class get its action indirectly from a mapping of types to close-actions. Whenever you have a type whose instances require some kind of cleanup action, you add an entry to the mapping.

class Closer:
actions = TypeActionMap()

# ...

def __del__(self):
actions[type(self.closable)](self.closable)

The mapping is not quite as simple as a dict, because of inheritance. This is what function overloads and C++ template specializations are meant to achieve. Similar functionality could be implemented in Python via pure Python mapping types, represented above by TypeActionMap.


If you want
reliable destruction it's better to set it up explicitly, using
"with".

That's true of the current language. I don't have enough experience with "with" yet to know whether it's a realistic solution to the issue. IMO, they are at least preferable to Java-style finally-clauses, but probably not a replacement for C++-style RAII.
.

Relevant Pages

  • Re: Article of interest: Python pros/cons for the enterprise
    ... Python has this too, except it's using a special type of scope ... CPython does not guarantee that the reference count will become zero ...
    (comp.lang.python)
  • Re: append
    ... the keyword search page for the Python Library Reference, ... Reference, and Python/C API manuals that you can find from a link on ... Full text searches (not limited to keywords like the resource above) ... There are two excellent free screencasts on ...
    (comp.lang.python)
  • Block-structured resource handling via decorators
    ... When handling resources in Python, where the scope of the resource is ... while one does not have to write an explicit cleanup. ...
    (comp.lang.python)
  • Re: "Help needed - I dont understand how Python manages memory"
    ... Python has a garbage collector. ... any scope is reclaimed, sooner or later. ... reference count of zero, or objects that participate in unreachable ... You cannot control when Python's garbage collector frees an object ...
    (comp.lang.python)
  • Re: function returns
    ... > scopes within Python? ... Also, does it matter ehat the type (str, int, ... When opne returns x from a function then x is a reference to a value, ... unlike the scope of names. ...
    (comp.lang.python)