keyboard EOF handling [Re: (warn) isn't doing what I expect it to]

Kent M Pitman <pitman@xxxxxxxxxxx> writes:
> Lars Brinkhoff <lars.spam@xxxxxxxxxx> writes:
> > Kent M Pitman <pitman@xxxxxxxxxxx> writes:
> > > Lars Brinkhoff
> > > > Kent M Pitman
> > > > > Additionally, there were some systems where you pressed Enter
> > > > > (a key distinct from Return or Linefeed) and this meant not to add
> > > > > a newline character but to activate immediately, submitting what
> > > > > had been typed but not any termination key.
> > > > That's ^D in Unix (except ^D on an empty line will usually be
> > > > interpreted as EOF).
> > > And i think some *nix based Lisp systems will close the input stream
> > > when they see a ^D, which isn't so good if you're doing a REPL on
> > > that stream.
> > They don't see ^D, they see the current input line,
> > without any terminting character. (If the line is empty,
> > there will be zero characters, interpreted as EOF.)
>
> Btw, for me, on both LispWorks 4.4.5 and Allegro 7.0 on Linux,
> control-D on a teletype (i.e., in a REPL in a shell, not in some
> GUI/IDE thing) neither echos nor has any apparent effect.
>
> Typed to Emacs with either of these lisps beneath it by using
> control-Q to quote it into the input stream, it just gets ignored,
> too. That is (if you assume my notation ^D below is replaced by an
> actual control-D (ascii 4), then: 3^D4 => 34 (+ 3 ^D 4) => 7
>
> I'm not challenging your claim--just offering information about how
> other implementations handle it. (I assume. I guess it's possible
> it's the Linux or my terminal settings.)

It is the tty driver on Unix that interprets special characters
such as ^D (to set the EOF flag on the input stream, rather than
passing it as input); it does this according to modes that have
been set on the device. It only does the EOF effect when
the ICANON flag is set. (You can change which character
will be the EOF command, by the way.)

Programs that need to read special characters use the tcsetattr()
function to turn off ICANON. Programs like Emacs also turn off
the ISIG bit, which normally causes interrupt signals to be sent
to the program when you type characters such as ^Z and ^C, since
Emacs doesn't consider those characters to be anything special.

When Emacs opens a pseudo-terminal for an inferior shell
there is undoubtedly some frobulation with the control bits.
Emacs creates and prepares the ptty that it will pass
to the inferior program. Emacs disables echo, turns off the
ICANON flag and some others, and frobs signal characters,
up/down-case conversion (from the days when ttys were TTYs),
tabs, NL->CR mapping, 8th bit handling, line and character
rubout handling, etc.

The command [c-c c-D] will signal EOF on inferiors
with using the function process-send-eof.
If the target is a pipe, this sends a ^D down the pipe.
If the target is a pty, it closes the pty.

(Yeah, that's more than I wanted to know, too,
but I hadn't read any C code all week, so...)

By the way, the terminal driver and job control features
of Unix are relatively new. It used to be that you could
only run one program at a time. The reason that you can
type ^Z and interrupt a job and start another one, is a
feature copied from ITS (which had it for decades prior).
A guy from MIT's Project MAC was visiting out at Berkeley
and put that stuff in. ^Z was recognized by the ITS
terminal driver (for ASCII terminals) as being the
interrupt character. On the ITS Knight TV keyboards,
there was a dedicated interupt key ("CALL") instead of ^Z.

The Lisp Machine, of course, had a key labeled ABORT.
It also had a key labeled END. The keyboard was driven
by an asynchronous process which read characters from
the hardware, and processed special ones -- for example,
by arranging through the scheduler to deliver an SI:ABORT
condition to the appropriate process.

(I don't really remember exactly how this worked in detail,
and am going to gloss over it a little bit...)

I think the INTERACTIVE-STREAM methods were also involved
in intercepting special characters (or some of them, anyway)
and could control which characters would be intercepted.
The BREAK or SUSPEND key (depending on which keyboard
you had), and the bucky (control/meta) versions of these
characters were also known to the interception protocol.

But the END key, not being an interrupt character, was handled
at a different level -- it was never specially intercepted.
END was just an ordinary character processed by the application's
stream's methods. END was just an activation character (like the
RETURN key), known to programs like the rubout handler used in
the "Listener" (interactive REP loop). Whether END or any other
character would normally be seen by the user of stream depended
only on the semantics of the stream.

These interactive streams that I am talking about are
also what implements the fancy rubout handling that
Kent was talking about.

The analogous pieces on Unix would be an interactive streams
library (something like "readline" I guess) that most programs
would use, which could handle a ^D character, versus the
kernel's tty driver intercepting that character (depending
on the tty object's flags).

On the gripping hand, another specially intercepted character
was the FUNCTION key. I don't think you could override it.
The sequence [FUNCTION END] would insert an out-of-band
EOF mark into the IO buffer of the input stream currently
associated with the console. (This is like ^D on Unix.)

Other FUNCTION key sequences were for things like clearing
buffered typeahead, arresting (not aborting) the processes,
frobbing various standard IO flags in the stream currently
using the console (such as automatic "more" processing and
window attention notifications) and other various window
system operations (such as rotating windows, clearing or
smashing window system locks in case of horrible bugs).
These functions were run in the system's Keyboard
handler process.

Another intercepted character that you could not override
was the SELECT key; it also read another character to
decide what to do. Its general purpose was launching
programs ("activities") and switching focus between them
(sort of like the START menu and alt-TAB in MS Windows).

You could add your own functions to the FUNCTION and
SELECT key dispatch tables, and you could make your
handler spin off another process to run an arbitrary
application.

(tv:add-function-key #\c-Page #'com-clear-output-history
"Clear the output history of the selected window.")

Some other random FUNCTION commands were: display
a list of logged-in users on other Lisp Machines
and their locations (aka "Finger"), also useful
for locating idle or free machines; buzz open the
lobby door on your floor; summon the annoyingly slow
elevator to your floor so that you would not have to
stand around waiting for it in the lobby; displaying
a memory usage indicator ("GC Thermometer"); putting
a news ticker into the system wholine; or playing audio
clips of the HAL 9000 and then shutting down the machine.

Here's an example from my login init file of frobbing
some keys in the TELNET program. (Apparently the program
was not properly modularized for this purpose and I didn't
bother to fix that; it should have had a handler method that
would let me map different keys for different flavors of
remote hosts.) Anyway, you can see from the #\Abort that
the TELNET program must have used the interupt character
interception protocol to have the Keyboard driver process lay off.

(setf (aref telnet:*telnet-keys* (char-code #\End)) (ascii-code #\)) ;c-D
(setf (aref telnet:*telnet-keys* (char-code #\Clear-input)) (ascii-code #\)) ;c-U
(setf (aref telnet:*telnet-keys* (char-code #\Help)) (ascii-code #\?)) ;c-H
(setf (aref telnet:*telnet-keys* (char-code #\Suspend)) (ascii-code #\)) ;c-Z
(setf (aref telnet:*telnet-keys* (char-code #\Abort)) (ascii-code #\))

There was also Left-control-Right-control-Left-meta-Right-meta-Rubout,
which would reboot the machine. Also, double-bucky-Return to warm boot.
These characters were never seen by Lisp at any level. They were handled
by the front-end processor. I don't remember the original machine having
any FEP - I think the keyboard circuitry actually drove the RESET line
on the machine! On the 3600, the FEP was a 68000 programmed in a static
language of Lisp syntax (cross-compiled, with Lisp macros). The FEP
didn't do much during normal operation -- it implemented the bootloader.
Both Lisp and the FEP had code to access a low-level disk file system,
which are a lot liked what we call "disk partitions" on systems today.
But it was much more flexible than "partitions", and it was a full-up
user-accessible file system. For example, if you found yourself in a
dire lurch where your normal file system (contained within a file on
this FEPFS) was broken or something, and you didn't have a working
network, you could write your Emacs buffers out to FEPFS files.
(And if you think having a 68000 and all this is overkill, yeah,
and a Z80 was used as the controller in the keyboard/CRT console head.
It's main job was to implement the brightness knob.) On the Ivory chip,
the "FEP" was just a mode that the machine could go into for booting.

Lars Brinkhoff <lars.spam@xxxxxxxxxx> writes:
> (This is no longer very Lisp-related, so I'll
> just write this additional article.)

It didn't seem too far off-topic or non-Lisp to me,
but then I like to pretend I'm Richard Burke or something.

We should have changed the subject line a while back.
I'll change it here so that people might look back upthread
and find Kent's description of the nifty rubout handler.
.

Relevant Pages

  • Re: "Read stuff from a file and chop it up to do stuff" code advice wanted.
    ... ;; This function returns TRUE if any character ... (if (char< char #\!) ... a stream and an array to hold characters in temp memory. ... ;; resulting string. ...
    (comp.lang.lisp)
  • Re: How to make Word STOP rewriting your copy
    ... The "character stream" that I was referring to is the GUI stream and not a physical storage stream (although I still believe the paragraph marker originally had it's roots there). ... the "power" of a tool can only be measured by how much PROODUCTIVE work can be done AT A GIVEN COST. ...
    (microsoft.public.mac.office.word)
  • Re: Read only last line-
    ... stream with a value that does not correspond to a position in the file ... example is putcon Windows, where one character ... That's the mapping you have to support. ... and fgetpos/fsetpos. ...
    (comp.lang.c)
  • Re: How do I stop a Winsock from buffering characters?
    ... it's applied at the OS level to the socket. ... Stream s = client.GetStream; ... from the client code and have the server see it right away. ... first character of the client send. ...
    (microsoft.public.windowsce.embedded)
  • Re: reading csv files
    ... >> stream, so, after the first name is extracted, the subsequent ... delimiter character, and not provide a complete solution to reading a CSV ... The next such call extracts 'yyyy', and ',', and the ...
    (alt.comp.lang.learn.c-cpp)