Re: Clojure vs Java speed

Jon Harrop wrote:
Oxide Scrubber wrote:
Jon Harrop wrote:
fft1976 wrote:
On Jul 28, 9:46 pm, Oxide Scrubber <jharri...@xxxxxxxxx> wrote:
fft1976 wrote:
Java: 1.5x slower than C as a rule of thumb.
I think it can achieve parity.
I disagree. I don't think the JIT can do much about the memory layout
of the data structures. Compare a vector of complex numbers or of 3D
vectors (NOT of pointers to them) in C/C++ and Java. Run some tests. I
did and I looked at others'.
Right. Writing such algorithms generically incurs huge performance
degradation in Java and Clojure.
The JIT can't, but the coder can. If you want a vector of N
double-precision complex numbers in Java that is contiguous in memory,
for example, you could use a Java array of 2xN doubles and index into it
appropriately.

That is not generic.

With Clojure multimethods and macros, you can make it generic.

You cannot write code that is both that short and that efficient in Clojure.

You can write code that is that efficient in Clojure, and the algorithm can be expressed as compactly, but you may need some support macros. Those, however, need only be written once for each novel type you wish to support.

Also, note that this same sort function can be used to sort value types such
as (unboxed) complex numbers with no alterations whatsoever. There is no
need to use syntactic abstractions to workaround any deficiencies, no need
to write custom index functions than handle special indexing for custom
types to work around the lack of value types.

And there isn't in Clojure either if someone else (or you on an earlier project) has done that work already.

Your "value types" are overrated anyway. Locality of memory access patterns is pretty good with the new "garbage first" collector for separate objects allocated close together in time.

For me, 1.5x is a good trade-off for safety though.
The JIT can't, but the coder can. If you want a vector of N
double-precision complex numbers in Java that is contiguous in memory,
for example, you could use a Java array of 2xN doubles and index into
it appropriately.
That workaround sucks because you've either lost polymorphism
With a language lacking syntactic abstractions like Java, perhaps.

You are on the JVM to interoperate with Java. Syntactic abstraction in
Clojure might help by allowing you to work around these fundamental
deficiencies in the JVM and Java but it is no panacea: the deficiencies
already damaged most of the code you are trying to interoperate with.

If you want high-performance code you might want the high-performance parts not to make calls into Java, or else to be JNI calls, one or the other.

One example where Clojure will do just fine is when you want to do numerics and also present a nice user interface. The numerics code can be very fast (but may not use Java interop) while the GUI code only needs to be fast enough for the UI to feel responsive (and will use Java Swing methods and objects extensively). Your "deficiencies" may have "already damaged" the Swing code, but that's not the CPU-bound part of the system anyway.

or, if you rewrite the entire compiler to use whole program optimizations
and change the calling convention globally, you've lost incremental
compilation and dynamic loading.
Such optimizations and dynamic loading are mutually exclusive in
languages without "eval" or similar capabilities available at run-time,
yes.

i.e. not Java.

But Clojure is another story.

This is fanboy fantasy, not reality.
Yes.
Stop attacking me.

I am not attacking you.

Sure you are. I wrote truth, and in response to that I got dumped on with a steaming load of ad hominems.

Clojure has some nice features but its most serious deficiencies are
inherited from the JVM and there is nothing Clojure can do about it, e.g.
value types and TCO. Moreover, according to Sun employees on the JVM
languages group this is never likely to be fixed.
Clojure has TCO; you just have to make your use of it explicit (and then
the compiler alerts you if it's not really in tail position).

Clojure's recur is only a special case, as I described here:

http://flyingfrogblog.blogspot.com/2009/04/when-celebrity-programmers-attack-guido.html

Clojure offers trampolines to cover more general cases of indirect tail
calls between different functions but it is much slower than real tail call
elimination.

Balderdash.

I'm not sure what you mean by "value types".

http://msdn.microsoft.com/en-us/library/s1ax56ch.aspx

I'm sorry, but I am asking you, not some web site (and especially not some Microsoft web site), what you mean. If you don't want to actually have a constructive dialog, feel free to STFU. Otherwise, actually say something.

If you mean non-pointer types, Clojure has access to the full range of
Java non-pointer types: boolean, byte, short, int, long, float, double,
and char.

The ability to define custom value types is extremely valuable, most notably
in the context of technical computing where you want 32- and 64-bit complex
numbers, 2d vectors, 3d vectors in homogeneous coordinates, quaternions and
dozens of other types to be representable without the overhead of boxing,
heap allocating, stressing the garbage collector and so on.

First of all, I explained in previous posts that you can do that in Java, and with full data abstraction in Clojure.

Second, if you define classes like Complex and Vect in Java with fields of type double, the doubles don't get individually boxed.

Third, the overhead of "heap allocating, stressing the garbage collector and so on" in a modern JVM is about the same as the overhead of stack allocating, on an amortized per-object basis: a pointer bump and whatever construction is done to write values to memory.

Learn how modern garbage collectors work, please.

It can't currently pass them across function call boundaries without
boxing and unboxing, but a) if the function call is consistent, the JIT
can inline it and optimize away the boxing and unboxing, and b) Clojure
has macros, so it's possible to inline a function not being itself
passed across function calls without losing the function-call syntax.

Only if your code is written entirely in Clojure which will not be the case

Why won't it?

you are on the JVM to reuse existing Java libraries

Laughable. You may be on the JVM to get portability, or a top-notch GC, or any combination of things. And if you are on the JVM to use existing Java libraries, that usage might not be in the numeric parts of the code; say file I/O, networking, or GUI code using Java libraries and pure-Clojure numeric code.

Last, but certainly not least, the Java libraries you're using might be existing high-performance numerics libraries for Java! Or Java wrappers around GMP or a similar native-code library. Then the performance of numeric Clojure code is irrelevant.

but those libraries expose these deficiencies of the VM by providing
non-generic interfaces.

A bit of Clojure magic could wrap these in much nicer interfaces even so -- and, using macros or definline judiciously, without any run-time overhead.

As an aside, the JVM even screwed up generics by using type erasure instead
of monomorphization and per-type instantiation at JIT => when the JVM fails
to optimize, generics are extremely slow.

That's why if you need performance you use -server. Besides, Clojure doesn't care about Java's generics making them irrelevant.

Just make a macro version of the function, or even a macro that can take
a function's source code and spit out a macro version of that function.

That only works if the entire source is written in Clojure.

It works if that particular function's source is.

Anyway, the upshot of all this is that "Clojure has some nice features
but its most serious deficiencies are inherited from the JVM and there
is nothing Clojure can do about it" is simply false, because Clojure has
syntactic abstraction and definline and features like that which enable
a user to create seamless workarounds for those so-called "serious
deficiencies". (I especially like the ability to write macros that write
macros!)

Sure. Macros can be nice. But it is not feasible to work around deficiencies
in the VM that have already infected the Java code that you are trying to
interoperate with.

I see that I am arguing not against a rational man, but against some heretofore undiscovered species of religious zealot, from some mystery sect whose articles of faith involve "deficiencies" and "infections" not apparent to the infidel.

I shall probably quit now, because it is generally futile to argue against religious beliefs, however irrational and unfounded in actual logic and evidence.

Perhaps I should have realized sooner that you were not a rational person; when I referenced actual results of actual benchmarks proving Clojure able to hold its own in a numeric-speed pissing contest and just got back an "it's 10x slower" mantra even after disproving that claim.

Oh, well.

Perhaps the best solution for technical users would be
for languages like Scala and Clojure to target MLVM instead of the JVM
What is MLVM?

http://openjdk.java.net/projects/mlvm/

Are you having difficulty actually saying things in your own words? Pointer chasing doesn't interest me much; that's why I gave up on C and started programming in Lisps.

I suggest you read up on functional programming as well.

I am the author of the world's most profitable book on functional
programming.

Let me guess: "Functional Programming for Dummies" or something on that level?

I'll repeat: I suggest you read up on functional programming, and more specifically, on functional programming methods for number-crunching.

If Scheme can be used for heavy number crunching, and it can, then
Clojure can too.

Scheme is not at all common for number crunching.

Your Google-fu is not strong. Google says otherwise. (Results 1 - 10 of about 46,500 for scheme "number crunching". (0.20 seconds))

Dynamic typing makes performance too unpredictable and makes
optimization too tedious. The vast majority of number crunching
applications are written in statically typed languages because they
are predictably fast.

Horsefeathers. One of the hits in that same Google search says "Scheme programs can beat equivalent C number crunching programs." Another says "Nothing in Scheme precludes fast number crunching." And both of those are visible in the excerpts on the SERP, without even clicking any links.

Many of the "problems" you cite go away with aggressive JIT optimization, which you get if you use the -server flag when you launch the JVM.
.

Relevant Pages

  • Re: Clojure vs Java speed
    ... vectors in C/C++ and Java. ... The JIT can't, but the coder can. ... You cannot write code that is both that short and that efficient in Clojure. ... You are on the JVM to interoperate with Java. ...
    (comp.programming)
  • Re: Making Lisp popular - can it be done?
    ... Common Lisp could do the same as well in principle, ... to Java libraries. ... tons of libs available in the JVM (I think more than for anything ... can mix Clojure code with pure Java which can compete with and often ...
    (comp.lang.lisp)
  • Re: Making Lisp popular - can it be done?
    ... Lisp conference/meeting. ... My Clojure program took around 105 msecs and the Java code did the job ... I have seen the JVM outperforming two different native Lisp compilers. ...
    (comp.lang.lisp)
  • Re: Road to Clojure Survey
    ... I am also occasionally following the progress of the Java platform, and there seem to be no serious efforts to overcome its weaknesses. ... but your productivity can still be hundreds of times greater if you do ... When we ported all our code to Clojure, ... The JVM has a nice performance. ...
    (comp.lang.lisp)
  • Re: Java question
    ... Thats correct, however as I stated, you loose all the standard stuff that comes with a JVM and you loose the capability to have a runtime that behaves similar and can be tuned and parameterized similar to other platforms. ... Or when classes are used among multiple JVMs, in addition to the shared classloader cache having a shared JIT cache would likely have its benefits. ... Subject: Java question ... expect this to be the very same version of the compiler, ...
    (bit.listserv.ibm-main)