Re: Callable generators (PEP 288: Generator Attributes, again)

From: Francis Avila (francisgavila_at_yahoo.com)
Date: 11/19/03 

Date: Wed, 19 Nov 2003 07:52:39 -0500

Michele Simionato wrote in message
<2259b0e2.0311180616.5980fabc@posting.google.com>...
>francisgavila@yahoo.com (Francis Avila) wrote in message
news:<55688f89.0311180211.7ab1bc30@posting.google.com>...
>
>I looked at that PEP few months ago and I came out with an iterator class.
>Here it is:
>
>"""An object-oriented interface to iterators-generators"""
>
>class Iterator(object):
> """__gen__ is automatically called by __init__, so must have signature
> compatibile with __init__. Subclasses should not need to override
__init__:
> you can do it, but you must do it cooperatively or, at least, ensure
that
> __gen__ is called correctly and its value assigned to self.iterator.
> """
> def __init__(self,*args,**kw):
> super(Iterator,self).__init__(*args,**kw)
> self.iterator=self.__gen__(*args,**kw)
> def __gen__(self,*args,**kw):
> "Trivial generator, to be overridden in subclasses"
> yield None
> def __iter__(self):
> return self
> def next(self):
> return self.iterator.next()
>
>class MyIterator(Iterator):
> def __gen__(self):
> self.x=1
> yield self.x # will be changed outside the class
> yield self.x
>
>iterator=MyIterator()
>
>print iterator.next()
>iterator.x=5
>print iterator.next()
>
>Wrapping the generator in the class, I can pass parameters to it (in
>this case x). IOW, here the generator has an explicit "self" rather
>than an implicit "__self__" as in the PEP. I am not sure if I like the
>PEP, wouldn't be easier to have a built-in iterator class?
> Michele Simionato

I'm suggesting the PEP's functionality, not its syntax and semantics. My
contention is that the PEP regards generators as too class-like, when they
are more naturally considered as function-like.

For example, your iterator class/instance would look like this:

def iterator(x=1)(x):
    yield x
    yield x

print iterator.next() # -> 1
print iterator(5) # -> 5

The local name "x" is updated (according to the second parameter list in the
function definition) right after the yield of the previous call when
iterator is called, behaving like a state-persistent callable function. If
it's just "nexted", it behaves like a plain old iterator.

Here's what the complete example in the PEP would look like (without
generator exceptions):

def filelike(packagename, appendOrOverwrite) (dat, flush=False):
    data = []
    if appendOrOverwrite == 'w+':
    data.extend(packages[packagename])
    while not flush:
        data.append(dat)
        yield None
    packages[packagename] = data

ostream = filelike('mydest','w')
ostream.dat = firstdat
ostream.dat = firstdat
ostream.dat = ('', flush=True)

Note that without exceptions, we need to overload the calling interface and
call with dummy data, which will never go into the stream.

On the other hand, it has a consistent (no "magic attributes") and obvious
interface (just look at the first line of its definition). If its called
incorrectly, it fails like a function call would, without disturbing the
generator. Plus you get default args and keyword args and all that good
stuff.

Now, It's not as if you can't do any of this without classes--but it's much
shorter and less cumbersome, because there's no need to deal with two
different namespaces and make sure they're in sync.

Here's an efficient reversable generator:

def count (n=0)(step=1):
    while True:
        yield n
        n += step

c = count()
c.next() # 0
c.next() # 1
c(-1) # 0
c.next() # -1
# Now turn it into repeat():
c(0) # -1
c.next() # -1
# And of course n and step can be anything for which n+step makes sense.

I was thinking to write an rpn calculator example with these, but for such
simple state machines there isn't an appreciable advantage over classes
wrapping a generator. Classes are more of a hassle, but not much more.
I'll have to find one that's more complex; it may be a hard sell even then,
but I just thought I could do better than what the PEP and discussions about
it suggested. 

--
Francis Avila

Relevant Pages

  • Re: need help on need help on generator...
    ... A generator function, ... way to make sure you have an iterator is to call iteron something; ... > there's such a thing in Python (... ... My prediction is that even Python 3000 will be strict. ...
    (comp.lang.python)
  • Re: Callable generators (PEP 288: Generator Attributes, again)
    ... > I'm suggesting the PEP's functionality, not its syntax and semantics. ... > iterator is called, behaving like a state-persistent callable function. ... do think an iterator as something which it obtained by "instantiating" ... > Here's an efficient reversable generator: ...
    (comp.lang.python)
  • Re: Assigning generator expressions to ctype arrays
    ... you've just blown away the entire iterator for no good reason. ... the generator on the right hand side is unpacked into ... atomicity is important for this case. ... some elements of the ctypes array, that the programmer was in fact ...
    (comp.lang.python)
  • Re: Iterator class to allow self-restarting generator expressions?
    ... So far it's got a test for emptiness, a non-consuming peek-ahead method, and an extended nextwhich can return slices as well as the normal mode, but one thing I'm having a little trouble with is getting generator expressions to restart when exhausted. ... Initializing a Regen instance with *any* generator will fail. ... On the other hand, your Regen instances could be initialized with *any* callable that produces iterators, including iterator classes. ...
    (comp.lang.python)
  • PEP 288 ponderings
    ... also seemed to me like generator attributes don't really solve the ... Another example from the PEP: ... where the nextargs function retrieves the arguments of the most recent ... def __init__: ...
    (comp.lang.python)