Re: 8051 assembler in Common Lisp

Peter Seibel <peter@xxxxxxxxxxxxxxx> writes:

> Greg Menke <gregm-xyzpdq@xxxxxxxxxxxx> writes:
>
> > Peter Seibel <peter@xxxxxxxxxxxxxxx> writes:
> >
> >
> Okay. Meanwhile, if you're interested I have some comments on what you
> show here. See below.

I am- thanks! My responses are interspersed.


> > Some terminology;
> >
> > sym is always bound to a top-level symbol
> >
> > syminst is the compiler state data struct recorded in each top-level
> > symbol's property list
> >
> > (sym-labels) accessor that gets/sets the list of labels contained in
> > a top-level symbol
> >
> > (sym-data) accessor that gets/sets the non-evaled list of sexps
> > forming the code in a top-level symbol
> >
> > (sym-compiled) accessor that gets/sets the list of eval'ed sexps of
> > a top-level symbol.
> >
> > (sym-linked) accessor for the output of the compiler; a sequence of
> > bytes comprising the compiled and linked code
> >
> >
> >
> > These macros handle creating and interning the top-level symbols at
> > read time;
>
> This sentence doesn't really make sense. Macros operate *after* read
> time. So it doesn't really make sense to talk about macros doing stuff
> at read time.

OK, so in pass 1 I read each top-level sexp and eval it, which creates,
interns and initializes all top-level symbols.


> > (defmacro with-sym-setup ((name opts symtype rest) &body body)
> > `(let ((sym (intern (string ',name)))
>
> The previous line is likely not having any effect at all. Note that:
>
> (eql (intern (string symbol)) symbol) ==> T
>
> as long as (eql (symbol-package symbol) *package*). In other words, if
> *package* hasn't changed value between the time the whose value is in
> NAME was read and the macro-expansion of WITH-SYM-SETUP happens,
> (intern (string ',name)) is just going to return the value of
> NAME. You might as well have said:
>
> (let ((sym name)) ...)
>
> or for that matter:
>
> (defmacro with-sym-setup ((sym opts symtype rest) &body body) ...)


Right- removing intern doesn't change behavior, thanks! Make it
simpler.


>
> > (syminst (make-instance ,symtype )))
> >
> > (proclaim '(special ,name))
> >
> > ;; symbol-value is the symbol's linked address
> > (setf (symbol-value sym) nil)
>
> The previous two lines seem hinky to me. For one thing the
> proclemation has a global effect so is not very friendly. And it's
> hard to imagine that it's actually necessary. (Though it may be
> necessary given the way the rest of the current code works.)

Lispworks grumbles about special variables if the proclaim is missing.
The symbol-value setf is superfluous. The global proclaim is
intentional as this is a symbol that I want available globally to all
the other assembly code.



> > ;; and init the syminst fields
> > (setf (sym-name syminst) (symbol-name sym))
> > (setf (sym-address syminst) nil)
> >
> > (setf (sym-org syminst) (getf ',opts :org nil))
> > (setf (sym-align syminst) (getf ',opts :align nil))
> >
> > (setf (sym-data syminst) ',rest)
>
> This is all fine though it might be more obvious what's going on if
> you'd define :initargs for these slots in the classes you may be
> instantiating. Then you could write this:
>
> `(let ((syminst (make-instance ,symtype
> :name (symbol-name sym)
> :address nil
> :org (getf ',opts :org nil)
> :align (getf ',opts :align nil)
> :data ',rest)))
> ..)
>
> and get rid of the SETFs.


Noted, however the setfs have come and gone and I've not gotten around
to formalizing their initargs.


> > (setf asm51::*cursymbol* syminst)
>
> I'm wondering why you're seting this rather than binding it.

Because I want to set it while processing the symbol so I can print
diagnostics later if something asserts during the pass 1 top-level
evals.



> > ,@body ))
> >
> > (defmacro deftext (name opts &rest rest)
> > `(asm51::with-sym-setup (,name ,opts '_textsym ,rest)
> > (push sym asm51::*textsyms*)))
>
> I wonder why you're package-qualifying the symbols with-sym-setup and
> *textsyms*. Aren't these macros defined in a file with an (in-package
> :asm51) form? (I point this out because it's a may be a symptom of
> another flavor of confusion about the relation between read time and
> macroexpand time and how they both relate to packages.)


*textsyms* is not imported into the compilation package, its intended to
be a symbol internal to the asm51 package, accumulating top-level
symbols. Macroexpand is not in question here, it would be the same if
everything was defun.


> > (defmacro defdata (name opts &rest rest)
> > `(asm51::with-sym-setup (,name ,opts '_datasym ,rest)
> > (push sym asm51::*datasyms*)))
> >
> > (defmacro defbss (name opts &rest rest)
> > `(asm51::with-sym-setup (,name ,opts '_bsssym ,rest)
> > (push sym asm51::*bsssyms*)))
> >
> >
> > via this loop in pass #1 (read pass)
> >
> > (loop for e = (read str nil nil)
> > while e
> > do
> > (eval e))
>
> I'm confused what the relationship between this loop and the macros
> shown above is? Are you using this loop to read a series of forms from
> a file where the top-level-forms read are likely to be DEFTEXT,
> DEFDATA, and DEFBSS forms? If so, why don't you just use LOAD to load
> the file? Then you could also use COMPILE-FILE to compile the files
> down to something that already has the macros expanded, etc. and will
> LOAD faster. (BTW, this use of EVAL, if I've understood things
> correctly, is fine since it's the level of evaluation the LOAD would
> normally do for you.)

The stream may not be a file.


>
> > In pass #2 (top-level symbol sexp-by-sexp code eval), I identify and
> > record each label symbol but do not intern it;
>
> So the interning is still a red-herring. What do you think interning
> does?

Top-level intern is removed as per above.

>
> > (loop for se in (sym-data syminst)
> > for e = nil
> > with rv = nil
> > do
> > ;;
> > ;; evaluate the sexp, accumulate non-nil results in rv
> > ;;
> > (cond ((symbolp se)
> > ;; se is symbol, make a label out of it
> > (setf e (make-label se))
> > ;; save it in the labels list
> > (push e (sym-labels syminst)) )
> >
> > (t
> > ;; not a symbol, eval it
> > (setf e (eval se)) ) )
>
> This seems, if you'll pardon me being blunt, like pretty much a
> canonical example of misuse of EVAL in a macro. The value of se was
> some code that was in the body of, say, a DEFTEXT form. If you want
> that code to be evaluated, you should arrange for it to be put into
> the expansion of DEFTEXT so it will be evaluated. Of course in your
> case that's not a simple change because you've, as far as I can tell,
> turned the whole macro-expansion machinery inside out.

Could you explain the last sentence a little more? At this stage in the
code I don't care about macros- all I'm doing is identifying and
recording label symbols and eval'ing each sexp element of the top-level
symbol, producing an intermediate object for each instruction. I might
be able to change this eval to apply since I'm assuming the only
elements in the sexp stream are label symbols or assembly instructions-
eval lets me cheat.


>
> > Now later on in pass #5 (link pass), I intern & set the value of the
> > label sybols within each top-level symbol.
>
> Again, what do you think you're accomplishing by interning these
> symbols. They're already interned. They've been interned ever since
> they were read. And even if they weren't interned, you could still do
> everything with them that you're doing anyway.

This intern (and its unintern complement) are also removed now. Thanks!

>
> To provide some food for thought, here's a sketch of part of an
> assembler that works somewhat like I think yours ought to. Basically I
> define a macro DEFTEXT which can be used to define an association
> between a name and a snippet of machine code. The idea (which I'm sort
> of guessing at from the code you've shown us) is that such as snippet
> is then combined with other snippets into a final assembly. This
> sketch doesn't include the ability to include arbitrary Common Lisp
> code within a DEFTEXT body but only because I'm not sure how you're
> envisioning that being used. At anyrate, note how there's no interning
> or uninterning of symbols and no calls to EVAL. After a DEFTEXT form
> is evaluated you can use the function GETTEXT to get at the machine
> code. For instance you can EVAL the following form (or load a file
> containing the form or whatever):
>
> (deftext foo
> (nop)
> (goto label2)
> (nop)
> label1 (nop)
> (push 128)
> (push 255)
> (goto label1)
> label2 (nop))
>
> then:
>
> ASM51> (gettext 'foo)
> #(0 1 11 0 0 2 128 2 255 1 4 0)
>
> Hope this gives you some ideas.

Below is the level of syntax I want to support. Your gettext example
cannot work because all symbols referenced by the assembly within it
must be defined before binary code can be produced. The gettext above
has to emit intermediate code that captures the symbols, which is
finally executed once all symbols have addresses.


(defproject testproj
(:text-base #x100
:data-base nil
:text-align 16
:data-align 8))

(defmacro xyzpdq (parm)
`(add acc ,parm))

(defparameter +RESERVE-DATA-LEN+ 100)


(defdata tst-data2 (:org #x20)
(dw 'tst-code))

(defbss tst-bss ()
(reserve +RESERVE-DATA-LEN+))

(defdata tst-data ()
(dw 0 1 2 3 'tst-code 'tst-bss)
(db 4 5 6 7 '(lobyte tst-data2))
(db \"hello, world\")
supercat2
(filldata +RESERVE-DATA-LEN+)
(filldata 10 #\\x)
(filldata 2 '(let ((x 1) (y 2) (z 3))
(list x y z))) )

(deftext tst-code ()
(nop)
(xyzpdq 10)
(nop)
supercat
(addc a 1)
(anl acc 15)
(inc dptr)
(cjne acc 3 'tst-data)
(clr a)
;;(ajmp 'supercat2)
(acall 'supercat)
(acall 'tst-data)
(acall '(+ tst-code 2)))



Gregm
.