what is lambda used for in real code?

From: Steven Bethard (steven.bethard_at_gmail.com)
Date: 12/31/04 

Date: Fri, 31 Dec 2004 06:53:55 GMT

I thought it might be useful to put the recent lambda threads into
perspective a bit. I was wondering what lambda gets used for in "real"
code, so I grepped my Python Lib directory. Here are some of the ones I
looked, classified by how I would rewrite them (if I could):

* Rewritable as def statements (<name> = lambda <args>: <expr> usage)
These are lambdas used when a lambda wasn't needed -- an anonymous
function was created with lambda and then immediately bound to a name.
Since this is essentially what def does, using lambdas here is (IMHO) silly.

pickletools.py: getpos = lambda: None
     def getpos(): return None
tarfile.py: normpath = lambda path:
                     os.path.normpath(path).replace(os.sep, "/")
     def normpath(path): os.path.normpath(path).replace(os.sep, "/")
urllib2.py: H = lambda x: md5.new(x).hexdigest()
     def H(x): md5.new(x).hexdigest()
urllib2.py: H = lambda x: sha.new(x).hexdigest()
     def H(x): sha.new(x).hexdigest()

* Rewritable with existing functions
Mainly these are examples of code that can benefit from using the
functions available in the operator module, especially
operator.itemgetter and operator.attrgetter (available in 2.4)

cgi.py: return map(lambda v: v.value, value)
     return map(operator.attrgetter('value'), value)
CGIHTTPServer.py: nobody = 1 + max(map(lambda x: x[2], pwd.getpwall()))
     nobody = 1 + max(map(operator.itemgetter(2), pwd.getpwall()))
SimpleXMLRPCServer.py: server.register_function(lambda x,y: x+y, 'add')
     server.register_function(operator.add, 'add')
SimpleXMLRPCServer.py: server.register_function(lambda x,y: x+y, 'add')
     server.register_function(operator.add, 'add')
sre_constants.py: items.sort(key=lambda a: a[1])
     items.sort(key=operator.itemgetter(1))
tarfile.py: return map(lambda m: m.name, self.infolist())
     return map(operator.attrgetter('name'), self.infolist())

* Rewritable with list comprehensions/generator expressions
Lambdas in map or filter expressions can often be replaced by an
appropriate list comprehension or generator expression (in Python 2.3/2.4)

cgi.py: plist = map(lambda x: x.strip(), line.split(';'))
     plist = [x.strip() for x in line.split(';')
cgi.py: return map(lambda v: v.value, value)
     return [v.value for v in value]
CGIHTTPServer.py: nobody = 1 + max(map(lambda x: x[2], pwd.getpwall()))
     nobody = 1 + max(x[2] for x in pwd.getpwall())
glob.py: names=filter(lambda x: x[0]!='.',names)
     names=[x for x in names if x[0] != '.']
hmac.py: return "".join(map(lambda x, y: chr(ord(x) ^ ord(y)),
                                       s1, s2))
     return "".join(chr(ord(x) ^ ord(y)) for x, y in zip(s1, s2))
imaplib.py: l = map(lambda x:'%s: "%s"' % (x[0], x[1][0] and
                            '" "'.join(x[1]) or ''), l)
     l = ['%s: "%s"' % (x[0], x[1][0] and '" "'.join(x[1]) or '')
          for x in l]
inspect.py: suffixes = map(lambda (suffix, mode, mtype):
                                     (-len(suffix), suffix, mode, mtype),
                                   imp.get_suffixes())
     suffixes = [(-len(suffix), suffix, mode, mtype)
                 for suffix, mode, mtype in imp.get_suffixes()
inspect.py: return join(map(lambda o, c=convert, j=join:
                                    strseq(o, c, j), object))
     return join([strseq(o, convert, join) for o in object])
mailcap.py: entries = filter(lambda e,key=key: key in e, entries)
     entries = [e for e in entries if key in e]
poplib.py: digest = ''.join(map(lambda x:'%02x'%ord(x), digest))
     digest = ''.join('%02x' % ord(x) for x in digest)
pstats.py: if line and not filter(lambda x,a=abbrevs:
                                               x not in a,line.split()):
     if line and not [x for x in line.split() if x not in abbrevs]:
tabnanny.py: firsts = map(lambda tup: str(tup[0]), w)
     firsts = [str(tup[0]) for tup in w]
tarfile.py: return map(lambda m: m.name, self.infolist())
     return [m.name for m in self.infolist()]
tarfile.py: return filter(lambda m: m.type in REGULAR_TYPES,
                                  self.tarfile.getmembers())
     return [m for m in self.tarfile.getmembers()
             if m.type in REGULAR_TYPES]
urllib2.py: return map(lambda x: x.strip(), list)
     return [x.strip() for x in list]
webbrowser.py: _tryorder = filter(lambda x: x.lower() in _browsers
                                       or x.find("%s") > -1, _tryorder
     _tryorder = [x for x in _tryorder
                  if x.lower() in _browsers or x.find("%s") > -1]

* Functions I don't know how to rewrite
Some functions I looked at, I couldn't figure out a way to rewrite them
without introducing a new name or adding new statements. (Warning: I
have trouble following code that uses 'reduce', so I only glossed over
lambdas in reduce calls.)

calendar.py: _months.insert(0, lambda x: "")
cgitb.py: inspect.formatargvalues(args, varargs, varkw, locals,
                  formatvalue=lambda value: '=' + pydoc.html.repr(value))
cgitb.py: inspect.formatargvalues(args, varargs, varkw, locals,
                  formatvalue=lambda value: '=' + pydoc.text.repr(value))
csv.py: quotechar = reduce(lambda a, b, quotes = quotes:
                       (quotes[a] > quotes[b]) and a or b, quotes.keys())
csv.py: delim = reduce(lambda a, b, delims = delims:
                       (delims[a] > delims[b]) and a or b, delims.keys())
difflib.py: matches = reduce(lambda sum, triple: sum + triple[-1],
                                    self.get_matching_blocks(), 0)
gettext.py: return eval('lambda n: int(%s)' % plural)
gettext.py: self.plural = lambda n: int(n != 1)
inspect.py: classes.sort(key=lambda c: (c.__module__, c.__name__))
inspect.py: def formatargspec(args, varargs=None, varkw=None,
                   ...
                   formatvarargs=lambda name: '*' + name,
                   formatvarkw=lambda name: '**' + name,
                   formatvalue=lambda value: '=' + repr(value),
inspect.py: def formatargvalues(args, varargs, varkw, locals,
                     ...
                     formatvarargs=lambda name: '*' + name,
                     formatvarkw=lambda name: '**' + name,
                     formatvalue=lambda value: '=' + repr(value),
pyclbr.py: objs.sort(lambda a, b: cmp(getattr(a, 'lineno', 0),
                                          getattr(b, 'lineno', 0)))
SimpleHTTPServer.py: list.sort(key=lambda a: a.lower())
subprocess.py: p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
symtable.py: self.__params = self.__idents_matching(lambda x:
                                                           x & DEF_PARAM)
symtable.py: self.__locals = self.__idents_matching(lambda x:
                                                           x & DEF_BOUND)
symtable.py: self.__globals = self.__idents_matching(lambda x:
                                                            x & glob)
urllib2.py:setattr(self, '%s_open' % type,
                    lambda r, proxy=url, type=type, meth=self.proxy_open:
                    meth(r, proxy, type))
xdrlib.py: unpacktest = [
         (up.unpack_uint, (), lambda x: x == 9),
         (up.unpack_bool, (), lambda x: not x),
         (up.unpack_bool, (), lambda x: x),
         (up.unpack_uhyper, (), lambda x: x == 45L),
         (up.unpack_float, (), lambda x: 1.89 < x < 1.91),
         (up.unpack_double, (), lambda x: 1.89 < x < 1.91),
         (up.unpack_string, (), lambda x: x == 'hello world'),
         (up.unpack_list, (up.unpack_uint,), lambda x: x == range(5)),
         (up.unpack_array, (up.unpack_string,),
          lambda x: x == ['what', 'is', 'hapnin', 'doctor']),
         ]

Of the functions that I don't know how to rewrite, I think there are a
few interesting cases:

(1) lambda x: ""
This is the kind of parameter adaptation that I think Jeff Shannon was
talking about in another lambda thread. Using the ignoreargs function I
suggested there[1], you could rewrite this as:
     ignoreargs(str, 1)

(2) lambda a: a.lower()
My first thought here was to use str.lower instead of the lambda, but of
course that doesn't work if 'a' is a unicode object:

py> str.lower(u'a')
Traceback (most recent call last):
   File "<interactive input>", line 1, in ?
TypeError: descriptor 'lower' requires a 'str' object but received a
'unicode'

It's too bad I can't do something like:
     basestring.lower

(3) self.plural = lambda n: int(n != 1)
Note that this is *almost* writable with def syntax. If only we could do:
     def self.plural(n):
         int(n != 1)

(4) objs.sort(lambda a, b: cmp(getattr(a, 'lineno', 0),
                                getattr(b, 'lineno', 0)))
My first intuition here was to try something like:
     objs.sort(key=operator.attrgetter('lineno'))
but this doesn't work because then we don't get the default value of 0
if the attribute doesn't exist. I wonder if operator.attrgetter should
take an optional "default" parameter like getattr does:
     Help on built-in function getattr in module __builtin__:

     getattr(...)
         getattr(object, name[, default]) -> value

(5) lambda x: x & DEF_PARAM
This could probably be written as:
    functional.partial(operator.and_, DEF_PARAM)
if PEP 309[2] was accepted, thought I'm not claiming that's any clearer...

So, those are my thoughts on how lambdas are "really" used. If others
out there have real-life code that uses lambdas in interesting ways,
feel free to share them here!

Steve

[1]http://mail.python.org/pipermail/python-list/2004-December/257982.html
[2]http://python.fyxm.net/peps/pep-0309.html

Relevant Pages

  • Re: Default parameters
    ... The default parameter value bindings are made a def time. ... The comparable def code to your lambda would be ... >One might argue that this could could benefit from resolving ...
    (comp.lang.python)
  • Re: whats wrong with "lambda x : print x/60,x%60"
    ... >> One perhaps needs to be a little more careful with instance variables, ... > def callback(): ... But this seems to defeat the main argument for removing lambda ...
    (comp.lang.python)
  • Re: Anonymous methods, blocks etc. (Cont. default block params)
    ... > room was that either using defor lambda() was generally felt to be ... Using def(), however, means that "def" is ... non-closure is at all more efficient that's agood thing. ... > much magic punctuation in Ruby as it stands now. ...
    (comp.lang.ruby)
  • Re: How to create functors?
    ... variables into the functions the lambda is wrapping. ... def MyFunction: ... CreateTask(lambda: SomeOtherFunction(localVariable)) # CreateTask ... >> It looks like in your version of Python "print" isn't a function. ...
    (comp.lang.python)
  • Re: Why anonymity? [was Re: map/filter/reduce/lambda opinions and background unscientific mini-s
    ... > Because it obfuscates your code for no benefit. ... expression above is within a gnat's whisker of the def equivalent, ... I don't see why lambda puts people off. ... Traceback: ...
    (comp.lang.python)