Re: System construction not recursive ?

"jurgen_defurne" <jurgen.defurne@xxxxxxxxxx> writes:
By trying to write this post, I seem to get the understanding that CL
compilation is just a translation of a representation in source code,
to a representation in byte-code or assembler, and that the resulting
(.fas, .fasl) files only contain the things that where stated in the
source files.

More or less, but it's more sophisticated.

You have "phases" that are more or less independent. In each phase,
you have the opportunity to have code executed to produce the data
(the "source") used by the phase.

read time
---------

At read time, the data consumed is the characters in the source
stream. It is parsed according to the lisp reader algorithm
described chapter 2 and 23. Reading (with CL:READ) produces ONE
lisp object, possibly compound, like a list of sublists and atoms.
This involves reader macros, either standard ones, or used defined
reader macros. These macros may or may not consume characters from
the input stream, and may or may not return one lisp object to be
considered as read. In doing so, these reader macro may do any
kind of processing, having access to a whole Common Lisp
environment.


macro-expansion time
--------------------

At macro expansion time, the input data is a form, whose CAR is the
name of the macro, and the macro can do whatever it wants with this
form and should produce a form as result, to be used in place of
the original form. You can also have access to the
macro-expansion time thru the MACRO-EXPANSION-HOOK, which allows a
user defined function to be called for each macro expansion. In
doing so, these macros and macro-expansion hooks may do any kind of
processing, having access to a whole Common Lisp environment.


compilation time
----------------

At compilation time, the input is forms, and the output is anything
adequat for the implementation. It might be a simplier forms, if
only the minimal compilation is implemented, or any kind of
"processor" code. You can define compiler-macros that are called
to substitute an _function_ call form by another form, before the
object code is generated by the compiler. In doing so, these
compiler macros may do any kind of processing, having access to a
whole Common Lisp environment.


execution time
--------------

At execution time, well, your program is executing with whatever
input it wants, and produces any output it wants. In doing so,
your functions may do any kind of processing, having access to a
whole Common Lisp environment.



Now the real time when these phases are executed can overlap, and in
the case of macro-expansion time, usually overlap either compilation
time or execution time.

Notice how I write "A whole Common Lisp environment". This is not
necessarily the same CL environment used in the various phases. If
you execute these different phases in the same lisp image, usually
it's the same CL environment. But if you read and compile a file in
one CL image, and then read and execute the .fasl file in another CL
image, then you'll have two distinct CL environments, and changes you
made in the CL environment while compiling won't be preserved in the
CL environment while executing, if you don't do anything special to
this effect.

The definitions of the CL:DEF* operators ensure that the changes
_they_ make to the CL environment will be in effect at execution time,
but usually not necessarily at compilation or macro-expansion time,
much less at read time. As mentionned, in other answers, you can use
EVAL-WHEN to have these effect taken into account in a different
phase.


I probably should see them more as object files, where the unresolved
dependencies must be resolved by the programmer, by using an
overarching system construction file which loads the necessary files
and then calls from the file which is the application, the main
routine.

Is this reasoning correct ?

Not really. You could rather see the .fasl files as scripts built by
the compilation phase to produce in the execution environment the
changes specified by the "source". It's already a running program,
used to build the eventual program.

(See for example the description of LOAD-TIME-VALUE in CLHS).


Are there other sources than the CLHS
about system construction ?

I don't know. I'd have to use google. I'll let you do so, there may
be some paper on this subject...


And now, for some fun:

C/USER5[57]> (cat "/tmp/p.lisp")

(eval-when (:compile-toplevel :load-toplevel :execute)
(defparameter *counter* 0))

(set-macro-character #\^ (lambda (stream char)
(declare (ignore stream char))
(incf *counter*)
(format *trace-output*
"~&Reading the caret number ~A~%"
*counter*)
*counter*))

(defvar *fun* (list (quote ^) (quote ^)))

(defmacro m ()
(format *trace-output* "~&At macro expansion time~%")
(when ^
`(progn
(format *trace-output* "~&At run time from (m)~%")
^)))


(print (m))


(defun f ()
(format *trace-output* "~&At run time from (f)~%")
(format *trace-output* "~&Read: ~S~%" (read-from-string "(^ ^)"))
(list (m) ^))

(print (f))


C/USER5[58]> (load"/tmp/p.lisp")
;; Loading file /tmp/p.lisp ...
Reading the caret number 1
Reading the caret number 2
Reading the caret number 3
Reading the caret number 4
At macro expansion time
At run time from (m)

4
Reading the caret number 5
At macro expansion time
At run time from (f)
Reading the caret number 6
Reading the caret number 7
Read: (6 7)
At run time from (m)

(4 5)
;; Loaded file /tmp/p.lisp
T
C/USER5[59]> (compile-file "/tmp/p.lisp")
;; Compiling file /tmp/p.lisp ...
Reading the caret number 1
Reading the caret number 2
Reading the caret number 3
Reading the caret number 4
At macro expansion time
Reading the caret number 5
At macro expansion time
;; Wrote file /tmp/p.fas
0 errors, 0 warnings
#P"/tmp/p.fas" ;
NIL ;
NIL
C/USER5[60]> (load"/tmp/p.fas")
;; Loading file /tmp/p.fas ...
At run time from (m)

4
At run time from (f)
Reading the caret number 1 ; here you can see that we have a different
Reading the caret number 2 ; environment, since the *counter* have been
Read: (1 2) ; reset by loading the .fas (defparameter)
At run time from (m) ; but the object read by ^ in the macro
; and function were built in another
(4 5) ; environment where it was already 4 and 5.
;; Loaded file /tmp/p.fas
T
C/USER5[61]>


--
__Pascal Bourguignon__ http://www.informatimago.com/
Un chat errant
se soulage
dans le jardin d'hiver
Shiki
.

Relevant Pages

  • Re: How can this work?
    ... Macro expansion time occurs either at compilation time, or at run time, ... The macro PRINT-STUFF calls the value of *TEMP* at macro expansion time. ...
    (comp.lang.lisp)
  • Re: Driver sample with *.asm source
    ... source and for it's compilation is used MASM? ... I remember reading an article for asm driver but can't find it now, ...
    (microsoft.public.development.device.drivers)
  • Re: Buddhism in Japan
    ... A lot of reading there. ... I'm really looking for something more of a compilation. ... A compact ... "beginners guide to Japanese buddhism" book, ...
    (talk.religion.buddhism)
  • Re: TrackMouseEvent - undeclared identifier--??
    ... You should not need to explicitly include winuser.h, since the documentation clearly ... In reading winuser.h, note that it is under a conditional compilation. ... MVP Tips: http://www.flounder.com/mvp_tips.htm ...
    (microsoft.public.vc.mfc)
  • Re: Very poor Lisp performance
    ... > languages like Python have too many dynamic features, ... > designed to allow efficient compilation. ...
    (comp.lang.lisp)