Re: MOP/Macroexpansion

Andreas Thiele wrote: 
Hi,

the following code snippet works in my application if def-class is used at
top-level:

(defmacro def-methods (name)
  `(progn ,@(class-methods name)))

(defmacro def-class (name parents &body body)
  `(progn
     (defclass ,name ,(if parents parents '(db-root))
       ,@(transform-source name body)
       (:metaclass db-class))
     (def-methods ,name)))

class-methods computes a list of defmethod forms and needs to call
find-class on the created class. Thus defclass needs to be executed before
def-methods is expanded. This does not work, if I use def-class within a
function (not at top-level). In this case - I think - everything gets
expanded and thus def-methods gets expanded before defclass is executed.
class-method will fail on calling find-class of course.

At the moment I am more astonished that the macro works at top-level.

ANSI Common Lisp is very specific about what should happen at the top level in this case. From the Hyperspec: "If a defclass form appears as a top level form, the compiler must make the class name be recognized as a valid type name in subsequent declarations (as for deftype) and be recognized as a valid class name for defmethod parameter specializers and for use as the :metaclass option of a subsequent defclass. The compiler must make the class definition available to be returned by find-class when its environment argument is a value received as the environment parameter of a macro."

Note that PROGN doesn't invalidate the "top-level-ness" of enclosed forms.

For non-top-level defclass forms, this is not specified because a) the defclass form may not be executed at all and b) the initforms for slots may close over the lexical environment of the defclass form which is only available at runtime.

The idea would be to not rely on the defmethod macro to create your methods at runtime, but to programmatically create those methods. Then you are able to refer to the run-time-generated class directly. A portable way to do this is by using EVAL:

(let ((my-class (defclass ...)))
  (eval `(defmethod my-method ((object ,my-class) ...)
           ...)))

Note that ANSI Common Lisp explicitly allows you to use class objects as specializers instead of class names, so this is indeed portable.

In some implementations, the use of EVAL may lead to inefficiencies because the evaluated form is not compiled but interpreted, including the method body of the generated method! (!!!)

You can circumvent this by doing the following:

(let ((my-class (defclass ...)))
  (funcall
    (compile nil
      `(lambda ()
         (defmethod my-method ((object ,my-class) ...)
           ...)))))

This indeed also compiles the generated method body.

The clean but only semi-portable way is to use the idiom for programmatically creating methods suggested by AMOP:

(let ((my-class (defclass ...)))
  (multiple-value-bind
      (method-lambda method-args)
      (make-method-lambda
        gf (class-prototype (generic-function-method-class gf))
        '(lambda (...) ...) nil)
    (let ((method (apply #'make-instance
                    (generic-function-method-class gf)
                    :qualifiers ()
                    :lambda-list '(...)
                    :specializers (list my-class ...)
                    :function (compile nil method-lambda)
                    method-args)))
      (add-method gf method))))

This only works in CLOS implementations that provide a correct implementation of make-method-lambda. Of the ones that I have seen, only CMUCL and SBCL provide correct versions, and the one in LispWorks is nearly correct (modulo argument list and generated parameter passing convention for the method function). Allegro, clisp, MCL and OpenMCL don't provide make-method-lambda.

Note that none of these approaches allow you to close over the lexical environment of the class- and method-generating code. If you delay generating methods to runtime, you don't have a chance in this regard because ANSI Common Lisp doesn't require the lexical environments to be available at runtime. This is different when you directly say (defmethod ....) instead of one the solutions I have proposed above because the macroexpansion of defmethod can call make-method-lambda at macroexpansion time and thus close over the (then-accessible) lexical environment.

(I am not aware of any Common Lisp that gives you first-class access to lexical environments at runtime. OpenLisp does - see http://www.eligis.com/ - as apparently do some Scheme implementations. Whether you can take advantage of this in some CLOS-style object system + a working MOP for it is left as an exercise to the reader. ;)

Another workaround for your problem could be to define the respective class globally. The non-global defclass forms would then just redefine the global class which may be sufficient for your purposes.

I needed to be able to create methods programmatically in my code as well, therefore I have added a utility function ENSURE-METHOD to my Closer to MOP compatibility library. It's not part of the public version yet, but will be in the next release.


One more thing: You take care of superclass defaulting in your def-class macro. However, the "MOP-way" to do this is in methods specialized on initialize-instance and reinitialize-instance, as follows:

(defmethod initialize-instance :around
  ((class my-standard-class)
   &rest initargs
   &key direct-superclasses
   &allow-other-keys)
  (declare (dynamic-extent initargs))
  (if (loop for super in direct-superclasses
            thereis (subtypep super 'my-standard-object))
     (call-next-method)
     (apply #'call-next-method
            class
            :direct-superclasses
            (append direct-superclasses
                    (list (find-class 'my-standard-object)))
            initargs)))

(defmethod reinitialize-instance :around
  ((class my-standard-class)
   &rest initargs
   &key (direct-superclasses () direct-superclasses-p)
   &allow-other-keys)
  (declare (dynamic-extent initargs))
  (if (or (not direct-superclasses-p)
          (loop for super in direct-superclasses
                thereis (subtypep super 'my-standard-object)))
     (call-next-method)
     (apply #'call-next-method
            ... ; as in initialize-instance
                 )))

This is more robust than to do it in a macro because now you are sure that your superclass is added even when someone programmatically manipulates your classes.

That idiom doesn't work in some CLOS implementations though, because they do the defaulting of standard-object too early (at macroexpansion time instead of class object initialization time). Fortunately, Closer to MOP fixes this so that this idiom becomes indeed portable across most CLOS MOPs.

See http://common-lisp.net/project/closer/ for more details on Closer to MOP.


Pascal

--
2nd European Lisp and Scheme Workshop
July 26 - Glasgow, Scotland - co-located with ECOOP 2005
http://lisp-ecoop05.bknr.net/
.