Re: Order of macroexpansion

Peter Hildebrandt <peter.hildebrandt@xxxxxxxxx> writes:

Is it specified, in which order outer and inner are expanded?

It's outer that is reliably expanded first. This follows from an
understanding that eval itself is defined as an operation on an
expression (not a subpart of it) and that operation is described
recursively from the outside in:

| 3.1.2 The Evaluation Model
| http://www.lispworks.com/documentation/HyperSpec/Body/03_ab.htm
|
| A Common Lisp system evaluates forms with respect to lexical,
| dynamic, and global environments. The following sections describe
| the components of the Common Lisp evaluation model.
|
| 3.1.2.1 Form Evaluation
| http://www.lispworks.com/documentation/HyperSpec/Body/03_aba.htm
|
| Forms fall into three categories: symbols, conses, and self-evaluating
| objects.
|
| 3.1.2.1.2 Conses as Forms
| http://www.lispworks.com/documentation/HyperSpec/Body/03_abab.htm
|
| A cons that is used as a form is called a compound form.
|
| If the car of that compound form is a symbol, that symbol is the name
| of an operator, and the form is either a special form, a macro form,
| or a function form, depending on the function binding of the operator
| in the current lexical environment.
|
| 3.1.2.1.2.2 Macro Forms
| http://www.lispworks.com/documentation/HyperSpec/Body/03_ababb.htm
|
| If the operator names a macro, its associated macro function is
| applied to the entire form and the result of that application is used
| in place of the original form.

But if you did not know this, you could mostly have inferred it from
design constraints:

The fact that outer is a macro means a program. Lisp cannot know what
this program will return without running it. Hence, as a consequence,
Lisp cannot know whether inner will still even be in the expansion or,
importantly, whether if it is it will be quoted. In both cases
(omission and quotation), you would not want to run the inner macro.
And, even beyond that, macros take as an argument a representation of
the lexical environment in which they are being expanded [see
&environment], and there is no such environment available until
expansion has been done. Consequently, all of these factors mean that
the design is necessarily such that outer must be expanded to reach
inner.

[1] In the most general case, it can actually be both, since the
expansion can inject the identical (i.e., EQ) expression into both
evaluated and non-evaluated contexts.

Is it specified, in which order outer and inner are expanded? I see
that in sbcl outer is expanded before inner, and I can do the
following:

(let (store)
(defmacro outer (val &body body) (setf store val) `(progn ,@body))
(defmacro inner (expr) `(,expr ,store)))

That is, store is first setf'd by outer, then read by inner.

Is this implementation specific, or can I rely on this?

You can rely on the notion that outer expansion will occur before inner
expansion. However, I would not rely on the other things you're doing here.

The only thing you can be sure of is that the inner cannot be lexically
analyzed until the outer has been executed.

Note well that you cannot be sure that the compiler has not, for
whatever reason, called some other expansion of outer in the interim.
You don't show your calling code here, but for example [2]:
(outer 3 (outer 4 (inner identity)) (inner identity))
will probably have not so good effects for you (assuming I got the uses
of the args right at all).

Also, there are cases where the value may surprise you because the
expansion is allowed to be incremental/lazily in non-compiled code
(there are some additional constraints in compiled code, but they
still may not be firm enough for your needs--I didn't think them
through carefully--see "minimal compilation") and that means that,
since the avenue of communication you're using assumes a definite
left-to-right and top-to-bottom UNINTERRUPTED nature (not in the
process interrupt sense, but in the sense of contiguous action even as
a synchronous activity), you're likely to be surprised if the code
executes one branch of an IF within the OUTER and expands things one
way (leaving the other branch unexecuted and hence unexpanded) and
then later, on a different call to the function executes the other
branch, there will be [potentially] a different dynamically previous
call to OUTER such that within the expression, the expansions won't
even appear consistent.

Even in compiled code, I can think of various ways in which there are
issues I'd want to think hard about before relying on this strategy,
and I'd say you are leaning too heavily on things not guaranteed to
work for my comfort. The easiest example to see is that if you have a
multi-tasking Lisp, you haven't made your code threadsafe, so two
compilations co-occurring can clobber one another. (An example
involving an actual interrupt which happens to compile a call to the
same macro at an inopportune time even in a non-multitasking Lisp is
theoretically possible to construct, but is not likely to ever happen,
so I won't lead with that.)

Basically, although you've done the necessary bow to the god of
lexical scoping by putting a let around that, s/he isn't going to
protect you because you've not done the work to make there be more
than one store, nor to assure that these macros get USED in lockstep.
Left-to-right/inner-outer is not your enemy here... the enemy is that
this is only a partial order and that other programs have other needs
that will not perturb the partial order but could do damage to your
storage medium.

The correct general shape for a properly cooperative program of the
kind I think you want to write is (I think [2]):

(defmacro outer (val &body body)
(macrolet ((inner (expr) `(,expr ',val)))
,@body))


[2] I'm in a hurry so am just dashing this all off rather quickly, and
didn't test any of it. Caveat emptor. At least it gives you something
to think about. :)

.

Relevant Pages

  • Re: Order of macroexpansion
    ... Kent just sent me the following via email and asked me to post it as a correction to his former post. ... (defmacro outer (val &body body) ... | of an operator, and the form is either a special form, a macro form, ... Lisp cannot know whether inner will still even be in the expansion or, ...
    (comp.lang.lisp)
  • Re: Order of macroexpansion
    ... This syntax isn't a macro call. ... That is, store is first setf'd by outer, then read by inner. ... So when OUTER is being expanded the above expander function allocates ...
    (comp.lang.lisp)
  • Re: Macro questions
    ... the cause of the expansion order. ... you are 'stuck' with the very first 'outer' macro. ...
    (comp.lang.lisp)
  • Re: Nested inline counter arithmetic
    ... expansion and execution. ... This macro still exist :-) ... \edef expands tokens. ... which expands to the #1th character of #2 (which I'm using as an array) ...
    (comp.text.tex)
  • Re: Fast expanding macros /Was: How to declare recursive factorial function?
    ... I've been thinking a moment about the following: given a macro ... that works purely by expansion, in which cases is it possible to make it ... The macro is a test which expands either one or the other of its ... arithmetics without e-TeX/for arithmetics with large numbers. ...
    (comp.text.tex)