Re: Macro functionality at runtime?

Pascal Bourguignon wrote:
> "senator" <senatorZergling@xxxxxxxxx> writes:
> > This, it seems, is because macro-expansion time occurs before runtime,
> > and at that time, we don't know what the subtree is yet.
>
> This is a big clue you don't need a macro here. A function would be enough.
>
> (defun apply-rule (tokens sub-tree new-tree)
> (let ((bindings (params-list tokens sub-tree)))
> (mapcar (lambda (k) (cdr (assoc k bindings))) (cdr new-tree))))

Yes, that is what I had in mind with the hash-tables, though a simple
associative list is good enough here (better, since it's shorter).

> > Side-note:
> > params-list is a function that takes tokens and sub-tree and returns
> > the form that "let" requires for variable binding.
> > Inputs to params-list:
> > tokens -> ("kkk" h i j)
> > sub-tree -> ("kkk" "aaa" "bbb" "ccc")
> > Output:
> > ((h "aaa")
> > (i "bbb")
> > (j "ccc"))
>
> It's shorter to just give us this params-list function than
> paraphrase. I don't understand why you do that...

I was trying to avoid clutter. Here you go, in all its full ugliness:
By the way, what's the more aesthetic way to indent multi-line
comments? With deep indents, sometimes, my comment looks like: (hope
the indents show up ok)

(blah blah blah (blah blah blah (defun blah ()
"comment here
here
here
and here"
func body)....)
Or is this just incompetence on my part (what Slime function can I use
here?)? Anyway, back to the function:

(defun params-list (tokens subtree)
"Returns a list of tokens matched to subtree.
Strings in tokens are ignored (assumed to match
subtree). Eg: Given tokens = ('match' a b c),
and subtree = ('match' 'hello' 'bye' (inside))
the list ((a 'hello') (b 'bye') (c '(inside)))
is returned. Note that ' is used to denote a
string quote (except for the lone quote for (inside))"
(cond ((null tokens)
'())
((stringp (car tokens))
(params-list (cdr tokens) (cdr subtree)))
(t (cons (list (car tokens) (if (consp (car subtree))
(list 'quote (car subtree))
(car subtree)))
(params-list (cdr tokens) (cdr subtree))))))

There's a couple of other things in there I thought wouldn't be
relevant to the discussion. This looks reasonable for now(not the best
I'm sure), but it will quite likely be modified/deleted as I play
around a bit more. There's sub-lists to consider, and who knows what
else (I'm still exploring the tree syntax from my front-end).

> Then:
>
> (apply-rule
> '("method" name type param)
> '("method" "method-Name" "type" "one-parameter")
> '("method" name param))
> --> ("method-Name" "one-parameter")

Yeah, this works too, and I think I played around like this at one
stage... I wanted macro at first as syntactic sugar (get rid of the
quotes as in set <-> setq). This later got bastardised into the form I
presented before.

**snip**

> Now, if what you needed was to generate optimized functions (because
> otherwise, I don't see the need for an anonymous function here
> either), You could just write this:
>
> (defun create-rule (tokens new-tree)
> (compile nil (eval `(lambda (sub-tree)
> (destructuring-bind ,(cdr tokens) (cdr sub-tree)
> (declare (ignorable ,@(cdr tokens)))
> (list ,@(cdr new-tree)))))))
> CREATE-RULE
> [61]> (create-rule
> '("method" name type param)
> '("method" name param))
> #<COMPILED-FUNCTION NIL> ;
> NIL ;
> NIL
> [62]> (funcall (create-rule
> '("method" name type param)
> '("method" name param))
> '("method" "method-Name" "type" "one-parameter"))
> ("method-Name" "one-parameter")

Now this looks interesting. At this stage (initial exploration), I
won't worry about speed just yet (I'm writing code more often, and that
is taking me much much longer than program compilation/execution
time...). What's with the eval? I've heard all the lore about it's
"badness", but have yet to understand why yet (I'll get there someday,
it's somewhere on my reading list (so many books, so little time...)).

> The optimizing is in the use of DESTRUCTURING-BIND and LIST, which can
> be optimized out by the lisp compiler. It should be O(n) instead of
> O(n²) for ASSOC.

Where is this specified? CLtL? Or does the Hyperspec have anything
about the standard compiler?

>
> You can use MACROEXPAND and MACROEXPAND-1, but the problem of the
> lexical scopes remains: you don't have them anymore at runtime!
>
>
> > If we have first class macros, what can we do? What sort of powers
> > will we get (I cannot fathom this)?
>
> Macros are nothing more than functions taking a sexp as input and
> giving a sexp aas output. You don't need to declare this kind of
> function as macros for your own sexp->sexp functions. You only need
> to declare such a function as macro when you want the _*LISP*_
> compiler to ue them to generate your _program_, not your _data_.

I just might need macros. For now, there doesn't seem to be any reason
other than syntactic "beauty", but I think as I move on to extended
rule forms (EBNF maybe...) or something more, this could start to
require more firepower...

> A compiler (a translator) is a meta program: it's data is program too.
> But when you translate Java to Python, these programs are purely data
> for the lisp system. Only your translator is program. So you don't
> need macros to transform Java to Python. You only need macros if you
> want to transform Lisp to Lisp. (High level Lisp to low level Lisp).

There's an initial stage where I read in the tree-transformation rules
-- I'm still playing with that. This seems similar to the next level
down, where we write the lexical analyser + parser using regexps and
EBNF. The EBNF feeds into the compiler-compiler, which generates (eg C)
code for doing the actual work. Perhaps this could probably happen here
too. Tree-transformation rules in, lisp code out.... I'm an Electrical
Engineer, not Computer Science, so I have no prior experience here
aside from reading some texts. Anyway, I have jumped the gun with that
previous question, so I'll head back to my exploration, and try to
reinvent the wheel, fire, and the arch.

And thanks Adrian,
> What you are doing is called "pattern matching" and there are
> books, tutorials, etc. that cover the implementation in
> detail. I think "On Lisp" includes an implementation of a
> simple pattern matcher, as does SICP. Check out Lecture 4a
> here:
> http://swiss.csail.mit.edu/classes/6.001/abelson-sussman-lectures/

I'm at about Chptr 2 :(, so I'll be there soon :). Hopefully, I'll have
something working by the time I read about it, hehe.

Cool stuff, all this.

.