Re: ILC2005: McCarthy denounces Common Lisp, "Lisp", XML, and Rahul

Ray Dillinger <bear@xxxxxxxxx> writes:

> Kent M Pitman wrote:
>
>
> > Politically, &QUOTE was in Zetalisp because the Lispm designers have a
> > core design principle that said that anything they needed to implement
> > the language should be available to users. CL takes an opposite point
> > of view and says that you sometimes need a finite number of special
> > forms to bootstrap, but that after that you're better off with just
> > macros. So far, I've seen little to refute the CL claim.
>
> Hmmm. I think the LispM guys had a compelling point and may have
> been right. One of the fundamental issues holding CL back, as
> far as I'm concerned, is its lack of downward extensibility.

Well, I don't know. I would say I saw a substantial number of things
where people used this technology to firmly wed what they were doing
to the LispM rather than writing abstract portable code, so I'm just
not sure I agree with you.

In general, just about every time in my career that I've written
programs that used very low-level tools specific to that
implementation, those pieces of code were dead-ended when the hardware
or operating system was obsoleted.

Now, you might be thinking that a feature like &QUOTE was not tying
you to the operating system, so maybe you think we're not in
disagreement on this. If so, I'd agree there's a blurry line there.
But those features, for example, were tied to the compiler, and most
other compilers didn't have them. And so by tying to the compiler,
you tied to the platform indirectly.

In the design of Common Lisp, I asked Dave Moon (one of the architects
of the LispM's "new error system" (NES), and also Symbolics' lead
technical representative to X3J13 for a long time) what his opinion
was on including the type tree for errors into CL's design. They had
a very heavily articulated type tree. I expected him to say "yes,
sure, put it all into the CL Condition System proposal, the more the
better", but he surprised me and said to go conservatively. He said
that experience had shown that there were lots of things that were
overspecified in the doc, and that while customers were happy to use
them, they were just writing bad code or becoming overly dependent on
details that might change. Almost certainly we erred by not including
a few things like directory-not-found (although it was for technical
reasons that it wasn't included, not for reasons of oversight), but
largely I think he was right that a lot of things were too low level.

Now, there are good kinds of low-level, like the way that floats are
picked apart in CL. That shows a fair bit of abstraction that mirrors
the design space and seems to allow people to write low-level code
fairly portably.

But there are bad kinds of low-level, and that's mostly stuff that
exposes a low-level piece of a mechanism where the mechanism itself
has not been bought into. So if all of CL doesn't agree on a compiler
model, or on a certain model of optimizers, or of type propagation,
and then you provide accessors into these facilities, you're putting
the cart before the horse sometimes by providing hooks into these.

The MOP, for example, derives its coolness not from the fact that it
exposes low-levels, but that it exposes underlying agreement. Likewise
the pretty printer.

And there are places like LOOP where people disagree in what I'd bet is
a useless dispute because I think LOOP, while complicated, is not so
complicated that we couldn't force implementations to agree _enough_
that the ability to define loop paths could be portably achieved without
infringing the part of LOOP that should _not_ be exposed, which is what
is the right and most efficient expansion for any given implementation in
order to benchmark well. The more you try to pin down that latter part,
the more you just require Common Lisp to be slow. Some parts are
meant to be kept abstract.

> You have upward extensibility to the sky with macros and HOF's
> and means of abstraction: but downward extensibility, where you
> do things with arbitrary bits and bytes on hardware I/O ports
> or with interrupts or native threads or the binary interfaces of
> other programs, has always been a bugaboo for Common Lisp;

IMO, this is a place where we have a fundamental disagreement. Common
Lisp is, IMO, fundamentally not about data layout. For sake of making
this a lively discussion, I'll even go out on a limb and allege the
extreme position that all of the Lisp Language Family (even to include
Scheme) [and I rarely make generalizations about that large a class]
is not about implementing these details.

[If anyone who regularly corresponds with John McCarthy on this wants
to poll him on whether I'm "blaspheming" here, I'd be interested in knowing.]

Some languages are about laying out data, what I call "implementation
languages". In the abstract, I claim these are fundamentally built on
micro-managing, and this obsession with data layout seems to me to be
the siren's call that keeps you from ever diggig your way out of thinking
about the machine issues.

Lisp is a differnet way of thinking, not about implementation, but about
abstract capability. It's a way of thinking about big problems. The
president of any huge corporation does not get there by overspecifying the
small details of each cashier's cash drawer, etc. In fact, I think it's
assumed that there will be minor inefficiencies in the name of not making
the cost of tracking/preventing those inefficiencies larger than the lost
time they actually lead to.

Even mathemeticians understand this when they talk about big-O
notation, which effectively teach you that you shouldn't get lost in
the constant factors or you'll never get to the grand truths.

In Lisp, you don't say "I'm going to make an array that is indirect so
that I can adjust its size later" you say "give me an adjustable array".
If there is one error in recent design of CL, it's not the failure to
provide low-level access to these details, it's the failure to add more
tools for saying "give me a graphical image" or "give me an XML document"
just as easily. That is, the base should be moving up where you appear
to be saying it should move down.

> you
> have to go outside the language to do it,

or decide not to do it at all. Are you so sure that this is what stands
between you and the creation of a commercially successful program?
Or, if you're not in commerce, any program that will stand the test of time?

I allege, and you'll just have to find a way to prove me conclusively
wrong, that this is not what is holding you back. It may sound good
to the naive new employee or young child to say that "lack of
flexibility" is what's keeping them from getting their homework or job
requirements done. ("If only you let me do things my way, it would be
better...") Once in a while, smart people _can_ do things another
way. And once in a really rare time, smart people can _only_ do
things another way. But mostly, smart people can figure out how to
work within a structured system, and not-so-smart people can't work
except within a structured system.

And I'm not talking huge amounts of structure here. Lisp is one of
the least structured languages you can find. It allows more
flexibility of work style than most languages. But one of the ways it
does this is to preserve a certain degree of low-level abstraction
that keeps people from sabotaging themselves.

You might even be write about particular "downward extensibility",
whatever that means, but I will not easily cede the general field. If
you speak specifically, maybe there are particular details to be had.
But I don't think the general concept of barriers erected in that direction
is wrong. As such, back to the original point, I don't think the LispM
philosophy of providing all that they used is good.

It, for example, led to them exposing the bit patterns of the words, the
ability to pick apart pointers, the ability to make new pointers that were
in violation of GC protocols, etc. In effect, it allowed people to make
the GC as unreliable in the same way that pointer casting would make Java
unreliable.

> which puts CL (and
> Scheme too)

Ah, nice to see you're making your arguments nice and broad over
much of the language family, too. :)

> in the unenviable position of a mere scripting
> language.

Quoting myself from Slashdot in a remark that the community has
seemed to like (since parts of it show up in people's sigs from
time to time):

Both Scheme and Common Lisp have been and continue to be used in
real-world applications that might surprise you. These include (but
are certainly not limited to) applications in airline scheduling,
commercial database manipulation, computer operating systems,
bioengineering, web services, document format translation, and, yes,
even space exploration. Franz, Inc. has created quite a nice page of
Lisp success stories[1] that I think expand on this much better than I
could in the space

[1] http://www.franz.com/success/customer_apps/scheduling/nasa.php3

Personally, I doubt that all of these applications were best characterized
as "mere scripts", but I guess that's a subjective judgment on my part.
If they were scripts, they were certainly among the more useful scripts
I've heard of in my career, I'll just question your use of "mere" here.

[Or maybe you meant "Mir scripts" ...? Lisp has certainly been used for
spacecraft, but I'm not sure it's been on Mir.]

> C is rare among programming languages in that it
> takes downward extensibility as a serious goal and does it well,

And why would Lisp want to challenge that. [You should really read the
entirety of my slashdot interview, since it covers this point as well.]
I don't begrudge C doing this well any more than I begrudge working in a
team of experts where other people are experts in things I am not. It
detracts from neither them nor me when we each have a skill and can
collaborate to make something bigger than ourselves.

C is, IMO, a second-generation assembly language, higher level than
traditional assembly languages, but lower-level than Java, which I think
is also an assembly language. These languages provide useful power,
but they are not what I would choose to program in at the surface level.

> and C and its descendants practically rule the earth these days.
> I think this is not a coincidence.

I suspect C is used today because of the accident of history that it
happened to be the implementation language for Unix at the time it
took off.

Lisp, by contrast, has been deliberately beaten by people intending to
inflict death upon it for many years, and has survived because it contains
things that will not be denied.

In the modern civil rights movement for Gays, you sometimes hear people
say "Choice? You think I would choose this kind of abuse?" I think
Lisp's lot is often the same.

We don't cling to Lisp out of quaint nostalgia. We cling to it
because the features it offers aren't available in other languages,
and we need them. We are mindful of the fact that others don't value
them, but it makes those features all the more precious to us.

> This is a design point that is in common with lots of languages
> that had no way to specify such things and no escape; if you wanted
> to downward extend Pascal, for example, you had the same problem
> with access to machine primitives. Lisp has survived on the
> strength of its upward extensibility, even though every Lisp
> hacker has to use some other language for downward extensions
> and downward extensions aren't portable between implementations.
> That's a heavy burden for a language to carry in terms of
> acceptance and general utility, IMO.

I don't worry at all about the Siren's Call of gaining acceptance.
Acceptance doesn't make useful applications. Useful applications are
made by programs doing their job. I worry about impressing people
with what I build, not how I built it. And I don't feel nearly as
hampered about building tools in Lisp as I do in other languages.

"It's a poor craftsman who blames his tools." That doesn't keep you
from picking a good tool, but Lisp is already enough better than the
other tools that I'm happy to pick it. And I have strong reason to
believe that it achieves some of that betterness at the expense of
some of what you're asking for. You never get anything for free.
You're welcome to argue otherwise, of course. I'm just one opinion.
I'm not even saying I won't change my mind. I'm saying you haven't
presented any evidence that would even begin to change my mind.
You're welcome to try. (But personally, I'd rather see you build
something serious and then come back and tell us a concrete reason
that Lisp is standing in the way of your success, since in my experience
there are usually much simpler ways around THOSE problems than to change
the underlying philosophy of the language.)
.

Relevant Pages

  • Re: An Acceptable Lisp
    ... Programming languages are tools. ... So each tool (which a Common Lisp) has a specific ... of computer science students with a minimal language. ... Paul Graham has some specific design goals. ...
    (comp.lang.lisp)
  • Re: Where does the drive to syntax come from?
    ... No, Lisp is different again. ... I used to be interested in language design. ... the cases when we can automate the generation of source code ...
    (comp.lang.lisp)
  • Re: An Acceptable Lisp
    ... So each tool (which a Common Lisp) has a specific ... of computer science students with a minimal language. ... Paul Graham has some specific design goals. ... he works for Google, which uses mostly zero Lisp, AFAIK. ...
    (comp.lang.lisp)
  • Re: Can I write a better OS than Windows XP in Lisp ???
    ... Modern compilers give you a language that is just like C, but prettier to look at and with a couple of language features that require heap support. ... You could write one in Lisp. ... they cancelled the design of their PRISM OS. ... One of the reason's NT is slower is that they started at a high ...
    (comp.lang.lisp)
  • Job Opening in Munich for Lisp Programmer
    ... the position with someone who has good Lisp experience. ... Virtual CAD (VCAD) is a group within Cadence providing EDA (Electronic ... Design Automation) related services. ... are scripted with the SKILL language, ...
    (comp.lang.lisp)