Re: apparently undefined function called by macro

Greg Bacon wrote:

;(eval-when (:load-toplevel :compile-toplevel :execute)
(defun id (x) x)
;)

(defmacro define-foo-class (name slots)
`(defclass ,name ()
,(mapcar #'id slots)))

(define-foo-class bar
((baz :initarg :baz :accessor baz)
(quux :initarg :quux :accessor quux)))

If I try to load it in a fresh image, clisp complains about undefined id:

[1]> (asdf:oos 'asdf:load-op :foo)
; loading system definition from foo.asd into #<PACKAGE ASDF0>
;; Loading file foo.asd ...
; registering #<SYSTEM FOO #x102A2839> as FOO
;; Loaded file foo.asd
;; Compiling file /tmp/asdf/packages.lisp ...
;; Wrote file /tmp/asdf/packages.fas
;; Loading file /tmp/asdf/packages.fas ...
;; Loaded file /tmp/asdf/packages.fas
;; Compiling file /tmp/asdf/foo.lisp ...
*** - FUNCTION: undefined function ID
[...]

Loading by hand, i.e., (progn (load "packages.lisp") (load "foo.lisp")),
is no problem, and as you can guess from the commented lines, wrapping
ID in EVAL-WHEN is also a workaround.

Well, hmm, my misunderstanding is more basic than ASDF or even the
reader. After removing the initial IN-PACKAGE form from foo.lisp,
COMPILE-FILE still complains about undefined ID.

This isn't related to packages or systems (only in the sense that systems can help you to solve this problem - see below).

Why is ID undefined? Please help me correct the error in my mental
model.

Common Lisp distinguishes between interpretation and compilation. When code is interpreted, each top-level form is interpreted after another, which means that each form can rely on all effects being "correctly" produced by the previous forms. That is, a defun defines a function, defmacro defines a macro, a defvar defines a variable, and so on.

When code is compiled, this is not necessarily the case. It is the case that each form is processed by the compiler, and for example it is guaranteed that all macros are completely expanded and thus "removed" from the code, but it is not necessarily the case that the effects are produced. Typically, the effects will only be produced at runtime. So, for example, a defun only announces the presence of a function at compile-time (so that other parts of the code can be checked, for example whether they pass the right number of arguments, or whether you accidentally redefine a function somewhere else), but the function itself will not be available. Likewise, a defvar only announces the presence of a variable, but the binding and its value will not be available at compile time.

However, as almost always in Common Lisp ;), there are exceptions to this rules: Some top-level forms do indeed have effects at compile time. For example, a defmacro definition _will_ be fully available at compile time. The reason is that you typically want to base subsequent code on your macro definitions, and the compiler must be able to completely macro-expand away these macro definitions as well. But this in turn means that all the functions that a macro uses to produce the expansion must also be available at compile time - but for function definitions, this is typically not the case, as I explained above.

The (eval-when ...) form is there to exactly provide you with the a way to tell the compiler that, say, a global function definition should be available at compile time as well (or even to tell the compiler that it is _only_ available at compile time, etc.).

However, using eval-when all over the place is ugly and can be hard to deal with. A better way to organize your code is to take advantage of systems. A system definition allows you to declare that one file of lisp code depends on some other file of lisp code. By default, a system definition will process a file by compiling _and_ loading it before proceeding to the next file, so if a macro definition depends on the presence of some functions at compile time, it is a good to put all the support code in another file and declare in the system definition that your macros depend on that support code. Then you can safely forget about (eval-when ...) in 95% of the time.

And this also aligns well with another rule of thumb when implementing macros (at least more complex ones): It is a good idea to provide a functional layer of what you want to express at one level, i.e., purely by use of defuns, and then to provide macros to ease using the functional layer at a higher level. Just put the functional layer in one file (or set of files), and the macro / syntactic layer in another file (or set of files), and declare the correct dependencies in a system definition - done.

I hope this helps.


Pascal

--
3rd European Lisp Workshop
July 3 - Nantes, France - co-located with ECOOP 2006
http://lisp-ecoop06.bknr.net/
.

Relevant Pages

  • Re: macro functionality in embedded languages.
    ... One advantage of the macro style is efficiency - the call expands at ... compile time and can be compiled with full optimization. ...
    (comp.lang.lisp)
  • Re: Please review macro
    ... >>compile time about into what code the particular macro invocation should ... Mind you, at the repl I guess you get both ... dramatic results unless you compile the macro first. ...
    (comp.lang.lisp)
  • Re: Question about compilation/evaluation environments
    ... compile time." ... "If a defmacro form appears as a top level form, ... store the macro definition at compile time, ... the macro later on in the file can be expanded correctly. ...
    (comp.lang.lisp)
  • Re: Trouble with offsetof
    ... The offsetofmacro requires two arguments, ... It is evaluated completely at compile time based on names, ... run time and can never involve a pointer. ...
    (comp.lang.cpp)
  • Re: modifying array access syntax
    ... I published recent speculation on the ARRAY ... > are in fact remnants of the property lists of these Lisp-N ... these that were meaningful at compile time (FEXPR, FSUBR, and MACRO ...
    (comp.lang.lisp)