Re: Any Clojure users here?

On Thursday, February 10, 2011 8:34:03 AM UTC+1, Deeyana wrote:
On Tue, 01 Feb 2011 09:07:53 -0800, Alessio Stalla wrote:

On Monday, January 31, 2011 8:16:17 PM UTC+1, Deeyana wrote:
On Sun, 30 Jan 2011 03:42:04 -0800, Alessio Stalla wrote:

I didn't want to step into this discussion, but since it's touching
ABCL and I've been contributing to it for a while (though since some
time ago much less than I'd like to), I feel I have to set some
things straight.

On 30 Gen, 04:35, Deeyana <d.awlb...@xxxxxxxxxxxxxxx> wrote

That doesn't make sense. You say you feel you have to "set some things
straight", but then it transpires that the post you're quoting is one
of mine; but nothing I wrote would need to be "set straight". So, I'm
somewhat mystified here.

I mainly wanted to correct your claim

Well, there's your error right there.

that with ABCL you either benefit
from the CL standard or from the Java API, but not from both.

Not in a single project, you don't. You can benefit from one in project A
(uses JVM) and from the other in project B (easy portability to other
CLs) and thus from both across multiple projects, though.

So to clarify my meaning, I meant a single project, rather than a single
programmer, cannot derive both benefits.

They're not mutually exclusive, you know; if you have a 100% standard CL program and add a single call to a Java library, it doesn't immediately become completely unportable. There's a degree of portability, and having a big standard like CL helps to reach a higher degree. Of course if your program is 50% calls to implementation-specific libraries, porting will be hard, perhaps impossibly so.

If you intend to port your code to other Common Lisps, that will not
be easy unless you avoid using any JVM interop features of ABCL to
call Java libraries, and avoid using any ABCL-specific features, and
then you lose the benefit of Java's huge standard library and a lot
of the benefit of the JVM. It becomes just another VM-hosted Common
Lisp, albeit one whose VM is better tested, better optimized, and
more widely ported than most.

Porting might not be easy but at least it's doable; see below.

Is it worth doing, though?

Well, duh, it depends. In certain cases it might be worth, in others it
might not be. But you have the option, and that is one benefit of the CL
standard. Of course if you don't need that benefit, it has no value for
you. Isn't it you that's citing portability as a key feature over and
over?

Yes, but with non-JVM-tied CL programs, you do the work to port your app
to each new platform;

No, you confuse platform with language implementation! Many CL implementations run on multiple platforms, just like the JVM, and unless you used OS-specific features, you don't need any porting. You may need porting if you want your program to run on multiple CL implementations.

with Clojure (or a Java interop using ABCL
program), the JVM and Java standard library developers do (most of) the
work to port your app to each new platform. :)

Yes, just like the Allegro or LispWorks or CCL etc. developers do :)

I agree that using ABCL if you don't need any JVM library or
infrastructure (e.g. servlet containers, or Java WebStart, etc.)
probably makes little sense.

That was my main point.

Your point - if I have understood it clearly - is that once you use Java
interop, you've lost the benefit of Common Lisp, the standard. I don't
agree with that. When I started studying CL and writing some code, I
despised everything that wasn't portable, because I didn't want to
commit myself to a single implementation. With the growth of my
experience, I understood that my reasoning was backwards: you start by
picking a CL implementation that has the features you need; in my case,
that's often ABCL. Most CL implementations run on the most common
platforms - Windows, Linux and OS X - so that's not a problem if you
don't access specific OS features. Then if you later want to port your
code to another implementation, for whatever reason, you can leverage
the CL standard and only port the implementation-specific bits, if you
have structured your code well enough.

The same can be said about any large body of C code. The devil is in the
details though, and the bits that need porting are often fairly involved.
With a JVM-reliant program the bits that need porting across platforms
range from small to trivial to non-existent even for apps that use
threads, sockets, mouse input, asynchronous I/O, *and* graphics.

The same can be said for a Lisp application running on Allegro or LispWorks.. And free implementations like Clisp provide threads, sockets and other stuff portably across OSes. The JVM certainly has a lot of libraries and that's a good thing; CL is not at the same level for obvious reasons (much less money involved); still, you can do a lot of stuff portably with CL + portable libraries - many more things than you think.

Pardon? ABCL cannot directly call into C. Any C it calls will have to
be written as a native library designed to be called from Java, then
have a JNI wrapper written for it, and then be called into from ABCL
through that.

That means that though it will be possible for ABCL code to call C
code, it won't be possible for an existing CL library with C components
it calls via FFI to be used with ABCL without a lot of complex
modifications to both halves of that library.

In practice, you probably simply won't be able to use that
FFI-dependent CL library unless you're willing to do all of that work
yourself.

See the other replies. You can use CFFI-based libraries in ABCL without
modifications

ABCL might generate some magic around CFFI calls that wraps the Java side
of JNI's protocol,

No, it uses JNA, which is a different library, but you don't seem to accept it. Everything you said about ABCL and JNI does not make sense and I elided it.

With Clojure + JVM you could, in principle, isolate the bits that
directly use Java interop and port to ClojureCLR by rewriting just
those bits.

Yes, but without a Clojure standard, it's hard to know what the "core"
Clojure is

I'd presume the language core and clojure.core functions documented at
clojure.org.

I'd presume, too. But that's not enough. Are there clojure/core compatibility tests?

and the fact that Clojure uses JVM types for type hints

Type hint using functions would have to be inside the compatibility-layer
part.

Which doesn't exist currently.

and that there's no equivalent of CL read conditionals

#_ suppresses the following sexp (atom or list).

I assume you want more, though, e.g. ifdef-like conditional compilation.

Exactly.

For that you'll need to use macros like

(defmacro defcond [name test expr1 expr2]
`(def ~name ~(if (eval test) expr1 expr2)))

and then make a portable abstraction around a platform-dependent thing
like this:

(defcond max-int (is-CLR?) Int32/MaxValue Integer/MAX_VALUE)

This is slightly evil, because defcond uses eval to evaluate the test at
macroexpansion time. So, the is-CLR function needs to a) work on both
platforms, b) return true on CLR and false otherwise, and c) work at
macroexpansion time.

You also need defncond, defprotocol-cond, defrecord-cond, etc. Also, if you want it to be future-proof you can't have a boolean test, you need multiple alternatives. And having them at the top-level definition level only will cause a lot of code duplication.


Something like

(defn is-CLR? []
(try
(eval '(Int32/MaxValue))
(catch clojure.lang.Compiler$CompilerException _)))

ought to work. This also uses eval, mind you -- in fact, it just attempts
to compile Int32/MaxValue which will throw a CompilerException on Clojure
JVM and work on ClojureCLR, then catches the exception if it comes. If
that happens, the catch clause lacking a body evaluates to nil and that's
logical-false; otherwise, Int32/MaxValue is returned and that's logical-
true.

Fortunately, Clojure's eval is available post-deployment and not just
during REPL development.

Of course, it's a Lisp ;)

Note, this isn't exactly a read conditional; it's a macroexpansion time
conditional. That's as close as you can get in Clojure, but it seems to
be close enough to build finer-grained CLR-vs.-JVM conditional
compilation than manually substituting different versions of whole source
files. Here the granularity is global defines, but sexp-by-sexp is
possible:

(defmacro ifdef [test expr1 expr2]
(if (eval test) expr1 expr2))

...

(map #(some-fn % (ifdef (is-CLR?) clr-arg2 jvm-arg2) 42) some-seq)

probably makes porting significantly harder than with CL.

It's a Lisp. You might have to do a bit of extra work to e.g. create
defcond and is-CLR? and similarly, but you only have to do it once and
you can use macros and other abstractions to abstract over it in higher
layers. It seems generally the case that Lisps reduce linear added work
from boilerplate, alternate versions of code, etc. to logarithmic added
work if you have good enough macro-fu.

Sure. It's still harder than CL.

But here's something else: because of how widely ported the JVM is,
there's probably no *need* to ever port your Clojure program to
anything else. Or your ABCL program, for that matter. So it's all
rather academic. :)

The JVM is widely ported, but less widely ported than you think.

No, it's not.

For example, on 64bit Linux, no recent "client" JVM exists yet.

That'd be because on 64bit Linux the "server" JVM works just fine for
both client and server applications. The "client" vs. "server" thing is
really a case of poor naming; one is optimized for smaller-memory
machines, so older desktops, and the other for larger-memory machines, so
older servers and any reasonably recent hardware, including any 64-bit
hardware.

No, the "client" VM is optimized for fast startup too, and not having it available on 64bit Linux is hurting dynamic languages (which already have long startup times by themselves).


Also until not so long ago, in the OS X world, the JVM version was
closely tied to the OS version.

Anything from 1.5 on will run Clojure and 1.5 is quite old now.

But what if a library needed 1.6?

However, you just can't say that if you use ABCL and its Java FFI,
you lose everything the CL standard provides, because that's simply
not true.

I didn't say that. I said you lose ready portability to other CLs,

only if you heavily depend on Java libraries, which is not necessarily
the case.

A nontrivial modern application is going to need something from the set
of things that the HyperSpec (and C and C++ standards) don't provide
portably: networking, threading, graphics, mouse input, asynch I/O of any
kind (including nonblocking event-driven response to user keypresses),
and detecting things like isolated presses of the shift key without any
other key (that won't produce a character at stdin, but games for
instance commonly let you bind keyup and keydown events like that to
"fire missiles"; you can't implement such a game feature portably).

You can do all those things with a CL implementation which is portable across Windows, Linux, and OS X, 32 and 64 bit.

It can get those things from the JVM and Java standard library, and still
be widely portable across systems without much effort by you; or it can
get these things from a third-party Common Lisp library (or a C library
called via FFI) -- assume for argument's sake that there *aren't* hairy
problems with trying to use arbitrary C libraries from a JVM-hosted app
-- and then it requires more work to port to different systems, depending
on how widely the third-party library is ported.

But a typical modern app will need at least one of those things.

Now if you get that thing from Java, porting to other CLs becomes a pain;
if you get it from CL/C, porting anywhere becomes a pain.

You get to pick one or the other poison but not duck out of both unless
all you're writing is a console app with a rudimentary (or completely
absent) user-interface and no threading, nonblocking I/O, or networking.

If ABCL is a conforming Common
Lisp then it will necessarily provide what the CL standard provides,
which, like with the C standard, is not much (when it comes to host
interop, graphics, networking, etc., at any rate).

It's much, much more than the C standard. Macros, CLOS, the condition
system, the type system, a full numeric tower... just to name a few
things.

The things I'm talking about, listed above, are all missing from both,
though. They're all host-interop facilities (even threading is a form of
OS scheduler interop). The things you name -- macros, CLOS, conditions,
etc. -- are abstraction facilities instead.

I *did* say (when it comes to host interop, graphics, networking, etc.,
at any rate) as a qualifier when comparing CL's standard with C's in
terms of what it provides.

You don't gain *everything* you'd gain by using Clojure, though. You
trade away Clojure's uniform vector/map syntax,

CL has vector syntax, as someone pointed out upthread.

But it has an *icky* vector syntax. (Then again, Clojure has an equally
icky set syntax.)

Well, #(...) is not much more icky than [...].

True. On the other hand, if Clojure used that for vectors it couldn't use
#(...) as a shorthand for anonymous lambdas.

And you can write a portable reader macro for map syntax if you want
to.

And who will be familiar with it that reads your code? There's a big
benefit to standardizing these things.

#{:foo "bar" :baz "quux"} is not that hard to read as a map, and of
course there's always documentation.

Clojure's documentation. That's Clojure's map syntax.

Thing is, every Clojure coder will be familiar with it. Use reader macros
to add it to CL for your project, and will every CL coder than might read
your code quickly grok it, though?

Maybe that trade-off is worth it to you, but if so I'd be curious
which CL libraries you think make it worth it.

I can speak for my experience: most of my work with ABCL has been to
use it as a DSL on top of some Java library (most notably the Spring
framework). However, in a library I've used at least CLOS, some of
its MOP, and Cells, which are specific to CL.

Actually, there's a Clojure cells. :)

Is it as mature as the CL one? ;)

Be fair -- Clojure is a much newer language. That means

Con: Some things are less mature for it than for CL
Pro: It also has virtually no legacy cruft complicating everything

:)

No, I just need the host OS to get information about files.

Which you can't access portably from Common Lisp.

What is Common Lisp? It's a standard. It's text on paper, or in
files. You can't *do* anything from Common Lisp. You can do many
things from an *implementation* of Common Lisp, though - including
accessing the OS.

Differently in each case, for some tasks, it seems. Including querying
whether a filesystem object is a directory or a normal file.

Or you can sweep the portability problem under the rug in exchange for
growing your number of third-party dependencies, and thus the number of
third parties that can cause you headaches by changing something. :)

Or you can stick to a single implementation of CL - just like you stick
to a single implementation of Clojure - and only worry about porting
when you absolutely need to, knowing that in that case, the stuff you'll
need to port yourself won't be much.

And if it's, say, a Windows/x86 implementation of CL you can't stick to
the one implementation and run your app on Linux or MacOS;

There's not a single CL implementation running only on Windows/x86. There was Corman Lisp once, but I think it's dead.

but you can use the one JVM implementation of Clojure and run on any hardware/OS
combination with a 1.5-or-later JVM, which is a considerably larger set
than just Windows/x86.

Same with Clisp, LispWorks, Allegro, and probably others (SBCL runs on Windows but it's still experimental, I think).

I don't. I just *don't* ignore the fact that as soon as you step
outside of the standard you tie yourself to a single implementation.

If that implementation provides features you need, that's not bad.

Until you need to run outside Windows/x86 (or whatever).

Now, when you use Clojure you also more or less do so. But Clojure (and
ABCL) run on the JVM, which in turn runs almost everywhere. If you use,
say, Allegro CL specific features, that's much more limiting than if
you use ABCL-specific features or use Clojure.

Why? On which platform does the JVM (I mean the full J2SE) work where
Allegro or LispWorks don't?

Probably plenty, including the very widely used among developers platform
called "I vastly prefer open source toolchains to crippleware with
intentionally limited/missing features". ;)

Are you kidding? At least 6 Common Lisp implementations can run on Linux, not counting ABCL.


[...making system-dependent stuff like symlinks...]
Probably I'd just use the Unix symlink method on that platform rather
than implement a third method.

You have all those options with CL, as well as the option of using the
native POSIX facilities most implementations have.

POSIX facilities are themselves Unix-specific, for the most part, and CL
facilities for these things will require porting effort rather than "just
work" on every platform, so CL gains no advantage over Clojure there.

Except that it can make a symlink and Clojure can't? Of course a symlink is OS-dependent.

1) Portability is generally a good thing, but portability for
portability's sake is not needed in a lot of cases.

These days, gaining portability by using the JVM doesn't cost you much
of anything -- not speed, not access to system specific facilities
(since there's JNI), and not library functionality (the standard Java
library is huge, and there's tons of third-party libraries for various
purposes). The main issue is that going native is awkward, and if there
are layers in there between the user's code and the native call it
becomes downright difficult and unlikely (as with the hypothetical case
of CL code calls CL library calls C function, when the CL code is
ported to ABCL: the library will need to be ported with it, and the C
function modified to be callable via JNI, and a Java wrapper that uses
JNI created that the ported CL library calls through ABCL's Java FFI).

The point is, you don't gain much portability with the JVM over CL.

You gain some, and on the platforms where it counts, too (judging by
market share).

Apart if you're targeting Android.

A major and growing platform.

That requires you to port non-trivial Java code. Swing does not work on Android, it has its own GUI libraries.
.

Relevant Pages

  • Re: Any Clojure users here?
    ... Java libraries, and avoid using any ABCL-specific features, and then ... ABCL cannot directly call into C. ... Clojure and lose almost *everything* you'd lose by using Clojure, ...
    (comp.lang.lisp)
  • Re: Any Clojure users here?
    ... I mainly wanted to correct your claim that with ABCL you either benefit from the CL standard or from the Java API, ... Java libraries, and avoid using any ABCL-specific features, and then ... Clojure and lose almost *everything* you'd lose by using Clojure, ...
    (comp.lang.lisp)
  • Re: C89, size_t, and long
    ... libraries which have useful stuff). ... So because others here can't think of a good reason to use long instead of size_t and you can't prove there is never such a reason, we must all assume there could be a good reason to not use the mechanisms provided by the standard to deal with a problem? ... If you are claiming there is a reason to avoid the mechanisms the standard provides to allow portability then it is up to *you* to prove your point, not up to others to disprove it. ... but you can hardly blame the consequences on the Standard headers. ...
    (comp.lang.c)
  • Re: C89, size_t, and long
    ... libraries which have useful stuff). ... So because others here can't think of a good reason to use long instead of size_t and you can't prove there is never such a reason, we must all assume there could be a good reason to not use the mechanisms provided by the standard to deal with a problem? ... want to have standard headers included in their own headers which ...
    (comp.lang.c)
  • Re: comparing doubles for equality
    ... AFAIK there is no ANSI/ISO standard for it. ... Fortran 95 seem to have some built-in operators but it's not clear ... LIA,GIA,ICE libraries for interval methods in C++ from Delisoft ... Actually I've been for quite some months proposing getting together ...
    (comp.programming)