Re: 2.2.2 Annoyance

From: Michael Hudson (mwh_at_python.net)
Date: 11/25/03


Date: Tue, 25 Nov 2003 18:14:34 GMT


"Miklós" <nospam@nowhere.hu> writes:

> Michael Hudson <mwh@python.net> wrote in message
> news:m37k1pzkjw.fsf@pc150.maths.bris.ac.uk...
> > "Miklós" <nospam@nowhere.hu> writes:
> >
> > > """
> > > __getitem__(self, key)
> > > Called to implement evaluation of self[key]. For sequence types, the
> > > accepted keys should be integers and slice objects. Note that the
> special
> > > interpretation of
> > > """
> >
> > I think this is talking about implementing __getitem__, not calling it
> > directly. Historically these things have been quite different, though
> > they are becoming less so.
> >
> Well, I've never been really into the guts of Python ... but could you shed
> some light for me on that why these two things are different?

Hmm, that's potentially a large topic. I'll have a go.

At the C implementation level, Python distinguishes sequences (lists,
strings, tuples, arrays, etc) and mappings (dicts, basically).
However, Python-the-language uses the same notation -- brackets, [] --
to access the elements of both sequences and mappings, and there's
only one corresponding special method -- __getitem__.

If you're implementing a sequence in C, you're expected to fill out
the tp_as_sequence structure of the type object, which contains these
fields:

typedef struct {
        inquiry sq_length;
        binaryfunc sq_concat;
        intargfunc sq_repeat;
        intargfunc sq_item;
        intintargfunc sq_slice;
        intobjargproc sq_ass_item;
        intintobjargproc sq_ass_slice;
        objobjproc sq_contains;
        /* Added in release 2.0 */
        binaryfunc sq_inplace_concat;
        intargfunc sq_inplace_repeat;
} PySequenceMethods;

Notice that "sq_item" -- the function that retrieves an element of the
sequence -- is an "intargfunc", which is:

typedef PyObject *(*intargfunc)(PyObject *, int);

and "sq_slice" is an "intintargfunc", i.e:

typedef PyObject *(*intintargfunc)(PyObject *, int, int);

So, there's no way to actually implement a sequence that directly
handles a slice object! You have to make a mapping instead (the
corresponding function in PyMappingMethods is mp_subscript, a
binaryfunc:

typedef PyObject * (*binaryfunc)(PyObject *, PyObject *);

). This is the first reason you can't pass slice objects to
list.__getitem__ in 2.2 -- list just didn't implement mp_subscript.

The other reason is: just what is "list.__getitem__"? If list was
implemented in Python, it would be obvious, it would be an unbound
method. But list *isn't* implementing in Python, list is implemented
in C, so list.__getitem__ is a thing called a method-wrapper. When
you access it, magic happens to find the appropriate C level method
and wrap it up so you can call it from Python. But for __getitem__
specifically, there are two choices, sq_item and mp_subscript! 2.2
prefers sq_item, so even after the list type grew a mp_subscript
function [1].__getitem__(slice(0,1)) failed, because the
method-wrapper for sq_item didn't know what to do with the slice
object. 2.3 prefers mp_subscript over sq_item.

Hope that helped!

Cheers,
mwh

-- 
  The "of course, while I have no problem with this at all, it's
  surely too much for a lesser being" flavor of argument always
  rings hollow to me.                       -- Tim Peters, 29 Apr 1998


Relevant Pages

  • TOC of Python Cookbook now online (was Re: author index for Python Cookbook 2?)
    ... Processing a String One Character at a Time ... Finding a File on the Python Search Path ... Constructing Lists with List Comprehensions ... Looping over Items and Their Indices in a Sequence ...
    (comp.lang.python)
  • Re: Interface of the set classes
    ... >>It's about the interface of the set classes as defined in the PEP 218. ... the sets implements the sequence concept but without the impossible ... I don't know why this is different in Python! ... this interface incompatible with lists. ...
    (comp.lang.python)
  • Classical FP problem in python : Hamming problem
    ... I couldn't resist to test the new Python features about ... laziness on a classical FP problem, i.e. the "Hamming" problem. ... The name of the game is to produce the sequence of integers satisfying the ... -- Merges two infinite lists ...
    (comp.lang.python)
  • Re: Lists and Tuples
    ... > sequences of diverse items. ... > should use lists for longer sequences of similar items. ... You don't have to take my word for it, though, since Python itself ... arbitrary-length sequence of homogenious elements. ...
    (comp.lang.python)
  • Re: why cannot assign to function call
    ... I'm going to follow up here at the risk of annoying Mark, ... helpful in explaining things to Python beginners. ... it becomes a namespace mapping names to objects. ... to the list itself, while Steven held that Python lists ...
    (comp.lang.python)