Re: Future of LISP. Alternative to XML. Web 3.0?

From: "Juan R." <juanrgonzal...@xxxxxxxxxxxxxxxxxxxx>
Did you check [1]?
[1] http://theory.lcs.mit.edu/~rivest/sexp.txt.

Just now, in preparation for responding to your posting.
It has some rough edges. For example, when I saw /<carriage-return>
I didn't realize it meant literally the cr character within the
message, I thought the textual string <carriage-return> literally
appeared, and I hoped it would be later explained *which* carriage
returns (later) would be ignored and when the ignoring of them
would be turned back off. Then the examples just had nothing at
all, just a new-line situation, with no explanation what was
supposed to be there. Apparently what's there is the option of the
local implementation, and regardless of which of the four choices
applies, a backslant at the end of the line causes that line to be
directly adjacent to the start of the next line ignoring whichever
of the four types of newline happened to be present. That ought to
be explained a little, instead of leaving it to the reader to guess
what is meant.
Also the BNF has semicolons at the ends of some lines but not
others, and I have no idea what those semicolons denote. I'm pretty
sure they aren't supposed to be literally present in the syntax
being defined?

It seems to me we need to think of some useful service that
somebody might want to use, and then implement it with s-expr
instead of XML representation for queries and responses.
S-expr are more concise and easy to parse than XML. You would note
improvements when dealing with high volume traffic sites or
manipulating large files.

That's not the point I was making. If somebody implements a server
that merely duplicates an already-available service, except for
using s-expressions instead of XML, nobody is going to use it,
because it's cheaper to keep the existing XML software and continue
paying for extra bandwidth rather than rewrite all the software
just to save a fraction on the bandwidth.

We need a brand-new service that somebody might want to use, which
isn't already provided via XML, so somebody would have incentive
play with it manually to get a feel for the service offered and to
*see* how incredibly more legible and significantly more efficient
it is compared to XML, and then having seen that they might be
willing to download our new s-expression plugin for
distributed-computing API so that their software could then use XML
or s-expressions equally. Note that our server could provide output
in three modes, fully prettyprinted, typical-compact prettyprinted,
and ultra-compact, making it easy to switch between
debugging/explanatory mode and maximal-efficient mode.

As an alternative to a brand-new service, one hack I have been
thinking about from time to time is to get permission to use
Google's XML interface to their search engine, then write an
interface that maps between s-expressions and XML, so somebody
could connect to my CGI application, submit Google searches in
s-expression format, and get back RMI objects/references in
s-expression format, and then browse the RMI objects remotely via
my service. As a distributed application, for my own personal use,
I could write an s-expression client that submitted a Google search
via my s-expression interface, then browsed the resultset to
organize it better, then finally presents the nicely re-organized
index of search results as a tree of virtual Web pages for me to
browse from a regular Web browser.

One specific application I urgently need is a smart way to find
where people have posted followups to what I posted, collecting
them all in tree format, and checking off when I've already looked
at some article and decided I'm done with it. I could do that
directly locally by writing a HTML parser and parsing the results
from Google Groups search, or I could write the HTML parser coupled
to an s-expression interface, and then do all the rest of the work
with s-expression RPC, or I could get permission to use their XML
interface, write XML parser and s-expression interface etc.

1) Still now, there is not standard s-expr representation for <tag
attribute='value'>content</tag>. Each LISP project is reinventing
the wheel and that is not good. A standard s-expr for encoding XML
(HTML) format *is* needed.

Do *any* of those LISP projects have a (HTTP-accessible) server I
can play with, to see their various encodings of XML, to compare
them? My position is that first we let a thousand flowers bloom,
then we compare them, then we pick the prettiest. If the flowers
already have bloomed, I want to see them now. If they haven't yet
bloomed, then what's taking them so long?

It would be more cool if Mozilla run directly the LISP code ...

Note: I'm more interested in s-expressions replacing the XML as
used in distributed computing, rather than replacing the XHTML used
to replace SGML-HTML for Web pages. So Mozilla running the LISP
code seems moot. Normally Mozilla would connect to a CGI or JSP
application, as the toplevel user interface for some distributed
application, and it's only within that application that connections
would be made to other InterNet hosts and XML or s-expressions
would be exchanged.

The only reason I suggest running s-expressions on top of HTTP is
to make it easy to get access to such services for interactive play
just by using an existing Web browser instead of having to write a
whole new client program. I.e. I would pretend to be part of the
distributed application, calling some HTTP-accessible service as if
I were making a RPC/RMI call. Even later if I have written my own
client to make such calls, still for debugging it'd be nice to be
able to switch back to manual calls from a regular Web browser.

I agree with Graham that LISP is asking for an update. Some
modifications I am interested or working on include

Do you really mean modification (incompatible changes), or merely
enhancements/additions?

3) Documentation. XML was designed for documents, S-expr for data.

But documents *are* (one kind of) data!
S-expressions have already been used to represent lots of different
kinds of data. Surely they can also represent documents.

There are several documentation systems based in s-expr (e.g. Scribe),
but failed to be popular when compared with acceptance of HTML or TeX.

Both HTML and TeX suck when it comes to expressing mathematics.
Compare these two ways of expressing the same thing:
(expt (arrayelement x 5) 2)
x superscript 2 backspace subscript 5 endelement
Can you see that s-expressions clearly show the nesting of
mathematical notation, whereas TeX acts like a manual typewriter
moving head up or down with no reference to what the mathematics is
supposed to really mean?
2
x
5
Is that even possible to express in HTML?

By the way, way back in 1975 I wrote a crude text-layout program
(MRPP3 a.k.a. POX, successor to something (MRPPP and MRPP2) I had
written in 1969-70), which had overlays (composible pieces of
picture which could then be combined in various geometric
arrangements to make larger pictures), and macros (syntax vaguely
like Algol/MacSyma when called, defined usually to build overlays
out of pieces). Bill Gosper wrote a dandy set of macros for his own
use, enabling him to use my program to lay out immense continued
fractions and matrices and hypergeometric series etc., not as
pretty font-wise as Knuth's carefully tuned Tex, but properly
structured per the nested mathematical expressions.

Later, using PSL (Portable Standard Lisp), I re-wrote the program
(now MRPP4) to work directly from Lisp nested-lists, basically
using the same internal representation as Tony Hearn's "Reduce",
producing output in several formats (ASCII-art, and IBM CRTs
attached to VM/CMS system, and Unix workstations using X),
including a visual interactive structure editor where the cursor
covered an entire sub-expression, shown as that part of the display
highlighted, and could be zoomed in or out and panned up or down
the current level of structure, and then a command could be
executed on the current sub-expression as embedded in the larger
expression to perform algebraic manipulations.

MacSyma used a vaguely similar internal representation to Reduce, a
bit more ugly in my opinion. But in any case, the basic idea is
that internal Lisp structures were used to represent mathematical
documents, and then of course PRINT could externalize such data and
READ could get stuff back in, and MRPP4 could render the data
structures in the usual mathematical-notation format. (MacSyma also
had a math-format display mode, which it normally used for all
output in the ordinary Read-MathEvaluate-Print loop. AFAIK MacSyma
display output provided *only* ASCII-art mode, but I'm probably
mistaken.)

Anyway, one application I'm thinking of writing is a port of MRPP4
to CGI/CMUCL, whereby s-expressions from the client get
internalized to Lisp structures and then displayed as
MRPP4/MacSyma-style ASCII art, using <pre>...</pre> of course to
make sure things line up in all browsers. Of course I'd also have
an output mode for box-diagrams showing how the internal data is
structured. The math-display output would work only for valid math
structures, whereas the box-diagrams would work for all Lisp data.

OK, I've gotten to the part where you mentionned [3], but I had
skipped the section that mentionned [2], so now I went back and
found:
XUL will be fundamental piece for rich web applications based in
Mozilla clients [2].
[2] http://xul.sourceforge.net/compact.html

Yeah, I skipped that because I'm not interested in details of
Mozilla-specific addons/plugins/whatever. But now I looked at that
Web page, and it seems to have some good ideas. I think all the
options except Lisp suck, but it's probably nice to offer the other
formats for people who are stubborn and refuse to use s-expression
notation, so their stuff and our stuff can be easily exchanged in a
standard way. It would be nice if that author would write a
description of an algorithm for converting between XML and the
other notations. Maybe all that is needed is a tree-diagram showing
how XML forms a tree and then how to express that same tree in each
of the other formats being proposed. Using a visual tree as an
intermediary, instead of a pure BNF description, seems best.
(For definition, use a separate visual tree for each BNF
production, showing under the tree the XML and four alternative
notations for that same tree. Then as an example build a nested
tree that uses several of the various productions within it.)

Now back to where I was reading when I saw reference to [3]:
IMHO, we would modify the syntax for success on documentation
oriented tasks. I propose something like [3]
[3] See CanonML listed already in four place in XML alternatives
site (http://www.pault.com/xmlalternatives.html).
CanonML CanonML is a canonical language trying to offer the best of
different formal, scripting and markup languages. As a formal language
CanonML is based in a modification of S-expressions of SXML.
-> <http://canonicalscience.blogspot.com/2006/04/
canonml-markup-language-beyond-tex-xml.html>
If XML is so fine, why do
we see many alternatives (order of 100) improving its defects and
limitations?
Indeed! With 100 proposed, I would expect to see live
Web-accessible demos of at least twenty of them already.
Do you know of any such?
SXML provides a representation for XML documents in the Scheme
programming language -a kind of dialect of LISP-.
Is that really true, presumably some specialized Scheme notation
that is incompatible with Common Lisp?

I reserved the sequence :: for the tags because is "never" used in
real practice.

Except in its own documentation of course! As a followup to MRPP3,
I envisioned a flexible syntax for quoting strings, where *any*
character could appear verbatum inside a string providing that
portion of the string was delimited by something other than the
particular characters within it. For example, if we allow all of
these ways to quote text: <> [] {} "" '' || %%
then we can include that line of text in verbatim pieces such as:
"these ways to quote text: <> [] {}"[ "" '' || %%]
Note it's broken into two pieces, one quoted by "" but not
containing any internal double-quote marks, and one quoted by []
but not containing any square-brackets. So if some author never
used a particular character (or its mirror image) in an entire
document, he could quote the entire document as one piece by
choosing that particular string-delimiting character pair. I
dislike the idea of a priori decision that "nobody ever" uses a
particular character or character-sequence so it's fine to reserve
it for syntax and require special quoting if that character or
sequence were ever to occur within a document.

OK, enough of that!

At one point you said that [...] is more legible than (...), or
that's what you seemed to say. I would have no objection to a
syntax for s-expressions that made that substitution. In that case,
per my proposed extension from MRPP3 for quoting strings, () would
then be available instead of [] as one of the several available
ways to quote strings. No big deal.
.

Relevant Pages