Re: Very poor Lisp performance

Michael Sullivan wrote:
> Förster vom Silberwald <chain_lube@xxxxxxxxxxx> wrote:
>> When I was younger (now I am 31) I also believed benchmarking is a
>> must. However, in the meantime it is even this: if your ray-tracer in
>> OCaml were 100000 times faster than the Bigloo-Scheme version I would
>> not opt for OCaml quickly. Perfromance is important, no question, but
>> in a code project it is a rather tiny part.
>
> Depends on the code project, doesn't it? If you're doing primarily
> heavy duty (but medium complexity) mathematical manipulations, then
> speed is a big factor.

Yes, absolutely. I am primarily interested in the middle ground between slow
but hugely expressive languages like Mathematica and "fast" but archaic
languages like Fortran. Functional languages are great for this middle
ground and they are improving at such a rate that they've been eating into
the remits of Mathematica and Fortran a lot.

> I agree that in most problem domains, constant time speed factors are
> pretty much irrelevant for 95% of code, and we should be very skeptical
> about judging "language speed" by simple benchmarks like this, because
> an easier to program language, often leads to better algorithms, as well
> as a lot less programming time (and less downside variation in program
> speed -- cf: Erann's lisp-java-c experiment: the fastest C programs just
> beat the fastest lisp programs, but many of the C programs were *much*
> slower than the slowest lisp programs).

This is exactly why I measure LOC. From my results, the LOC measurements
indicate that code size will become a limiting factor much more quickly for
Java than for OCaml, for example. Although this is a very crude measure
(there are a huge number of language features, like a decent type system,
that aid productivity), my quantitative result reflect my experience.

However, LOC overly penalises Lisp and Scheme, IMHO. Specifically, Lisp and
Scheme programs are virtually unreadable unless the parentheses are
staggered by spreading expressions over several lines and using an
automatic indenter. So if I were to put a Lisp implementation of the ray
tracer on my site then I'd either state that, or I'd give results using
some other measure of verbosity, like characters. I do think Lisp deserves
to be somewhat penalised in this way, but LOC goes too far.

> But those tradeoffs aren't the same for all problem domains. In those
> rare problem domains where there's always lots time to optimize relative
> to the runtime available, bare speed matters.

I think it is also important to measure the performance of "natural" code.
In Lisp, natural code is often understood very generically by the compiler.
In most other languages, you have to work to get that generality.

> The problem is that it's still hard to tell the difference between a
> faster language and better writers of optimized code, as the almabench
> thread demonstrates. How much time do you spend on optimization before
> you abandon the project and say "It's easier to make this fast in X".
> And obviously some things will be easier to optimize in language Y than
> others. All this makes any given benchmark or set of benchmarks suspect
> as an absolute measure of speed, but it doesn't completely invalidate
> their potential usefulness, as long as one understands what they mean
> and what they do not mean.

Yes, absolutely. I do think my ray tracer is a suboptimal benchmark, of
course, but I was astonished that so many people put so much effort into
that almabench benchmark when (IMHO) it was an appalingly narrow and badly
coded test that isn't representative of any real programs, in my experience
as a computational physicist. I'm glad that this hasn't descended into a
"Lisp sux0rz" conversation though.

In contrast, my ray tracer involves integer and floating point arithmetic,
3D vectors, trees, tagged unions, recursion/iteration, naturally functional
(e.g. vector ops and ray_sphere) and imperative (e.g. for loops) style. So
it tests quite a lot using very little code and it is fun to play with.
This diversity also makes the collection of ray tracers a better
educational tool - people can glance at the code and see how different
things are implemented in different languages.

--
Dr Jon D Harrop, Flying Frog Consultancy
http://www.ffconsultancy.com
.

Relevant Pages

  • Re: a vast difference?
    ... the Boyer benchmark from Gabriel sounded interesting. ... source is 500 lines long in Lisp and 900 lines long in OCaml. ... been applied to the elegant and intuitive Lisp solution that Andre ... programmers would want to write Nathan's code. ...
    (comp.lang.lisp)
  • Re: Lisp and OCaml, why?
    ... > interpreted languages as possible over time, if not to use them at ... > lisp and OCaml though - why would someone use them in preference to ... Strong, static typing catches type errors at compile time, reducing ...
    (comp.programming)
  • Re: Program compression
    ... Haskell, SML, OCaml, Mathematica, F# and Scala all allow real ... problems to be solved much more concisely than with Lisp. ... for simple lists those three languages equal Lisp in conciseness. ... and thus there's no need for concurrent GC. ...
    (comp.programming)
  • Re: Alternatives 4
    ... This does show something interesting about lisp, ... And the OCaml code demonstrates how other languages can provide both ... performance and brevity at the same time, which is the power of OCaml. ... Emacs is the best there is as far as an IDE for OCaml goes ...
    (comp.lang.lisp)
  • Re: a LISP raytracer
    ... > None of the free Lisp compilers come close to the capabilities of Stalin. ... > OCaml does has camlp4 macros. ... My languages professor right now is a huge fan ... on OCaML, and not so much with LISP. ...
    (comp.graphics.rendering.raytracing)