Re: LimpM, why the umbilical and the storage arrangement with the PDP10?

Barry Margolin <barmar@xxxxxxxxxxxx> writes:

In article <cs9r6pwcsda.fsf@xxxxxxxxxxxxxxxxxxxxxx>,
Andreas Davour <ante@xxxxxxxxxxxx> wrote:

cstacy@xxxxxxxxxxxxx (Christopher C. Stacy) writes:

Andreas Davour <ante@xxxxxxxxxxxx> writes:
A related question concerns the storage model. I've read Richard
Greenblatt's paper about the features of the CONS and he mentions
storage on another machine as a design feature. I'm not sure I
follow. Why is that good? Is there some excellent insight to that design
that I'm missing? I only know lisp, and no electrical and digital
engineering and could possibly miss a lot.

The same reason that people use network storage appliances now.
Also, most of the files that people wanted to access were
already on some other computer (a PDP-10) on the network.

The last I can understand, that it was a nice feature. But, I don't
understand why people use network storage appliances now!

Maybe you know to what extent this networked filesystem was used? Was it
like the ITS filesystem just for utility or was the local disc used for
storage of anything but the world image and the system sources?

There were a couple of reasons why networked filesystems were used:

I totally agree with Barry's observations here and wanted to add a few things
he reminded me of.

1) LispM's were too expensive for every engineer to have their own.
When I was at MIT (early 80's), I think there were about 4 in the EE
department lab, and another half dozen or so in the AI lab. Most files
were kept on fileservers so you could use any available machine in the
lab, rather than having to wait for the one with your files on it to
become free.

The importance of this cannot be understated. To say this more
plainly, most people had on their desk a VT52 (a 24x80 crt screen,
connected to a mainframe) that they did most of their work on. We
often shared access to Lisp Machines. A few people had them
dedicated, but the rest of us (and as an undergrad, I was *definitely*
one) had to move into others' offices at night after they'd gone home
and borrow them until daylight. The machines got nearly 24 hour use
that way, but one didn't always know which machine they would use
next. This culture of physical openness was enhanced because,
valuable as these machines were (and they were extraordinarily so),
there was no place to steal them to. No one could have carried them
out the door without being noticed, and no one would have wanted them
other than the people who already had access. But still, people left
their doors open and expected others to borrow their offices in a way
that is simply not done any longer most places I've seen in the modern
world. So Barry's point about using any available machine was really
a central issue.

2) Disk storage was really expensive in those days. So it would have
been prohibitively expensive to put much disk space on each machine.
There were economies of scale available by putting large disks on
central minicomputers and mainframes.

Building again on this, one issue is that these disks on the LispM were
INCREDIBLY large by some accounts. A 450MB disk was not uncommon, though
often it was less ... but no matter how you did it, in some ways, it was
luxurious, since there were about a half dozen mainframes at the Lab for
Computer Science and AI Lab combined, with less than a gigabyte per machine,
if I recall correctly (seems like the MIT-MC machine, which had a lot, had
two 270 MB disks and maybe a 450 MB disk ... and later maybe a couple more
disks of a size I don't recall, but comparable), shared between probably
50 or so people who were "real" users and maybe a few hundred or a thousand
users who were "tourist" users... so not much disk space at all for all
projects combined. And then along comes the Lisp Machine, where the address
space was way larger (suddenly 29 bits instead of 18 bits), and though the
disk seemed huge, it all went to "world loads". The software took up a
gigantic amount of that disk. So you might have 250MB on your own machine,
which seemed large, but then when a "world load" (a saved image of the entire
operating system with your code loaded) took up 50 or 100MB, you realized
that suddenly you could have maybe a distribution load and maybe one or two
other world loads and that was it. So storing a "file system" partition
on the same machine was a way to make you NOT have enough room for a world,
and to risk that someone needing to dump a world would delete your file system
to make room. Not a pretty picture. (These machines had no file security,
so anyone could delete anything. The network file systems had file security
only if the remote system had security. MIT ITS had none. TOPS-20 did. Unix
claimed to, but it was originally implemented by assuming that the remote
machine would ask for the password, so the lispm had a variable called
*check-passwords-for-show* that would prompt you for a password to pretend that
your unix had local file security, when in fact the client was responsible
for passwording the user and the lispm client was willing not to... So file
security was just managed differently than now, and the fear that your
stuff would go away was very tangible.)

3) Even in environments where everyone has their own personal machine,
it's common to work on group projects. Or you write a program on one
machine and then want someone else to be able to try it on theirs.
Shared filesystems foster this type of collaboration.

Indeed. The right model of this would be like URL sharing. It seems
like such a modern idea, unless you realize that in the old days, in
those environments, we could already just pass filenames around, so we
didn't need URLs. In 1985, when the Macintosh was just coming on the
market and PC meant DOS, the Lisp Machine had an interface where you
got command completion on the copy file command and its filename
prompting, and where you could type a "copy file" command to the
command shell and it would prompt for filenames and give you filename
completion for any file on the arpanet, in any filename
syntax. (Admittedly, it had a list of all servers on the arpanet and
knew which filename syntax each used, and it had to go out over FTP to
get the filenames to complete over, but it did so invisibly and didn't
tell you that was what it was doing.) So networked files were as
casual as URLs. "Where did you put the data?" "Oh, it's on MIT-MC at
MY;FILE >" or "Oh, it's on PARCFTP.XEROX.COM at /foo/bar". People
sometimes bought Lisp Machines for the single purpose of "connectivity
server" because they could talk CHAOS, TCP, DNA, SNA, and other
network protocols and link together machines that otherwise didn't
talk to one another.

4) It simplified the design of the OS by leaving out a fancy file
system. The network servers already had existing file systems, it was
easy to take advantage of them. Early Lispms just had a simple
partitioning scheme -- one partition for the firmware, another for the
Lisp image, another for paging, that's it. Eventually native file
systems were implemented, but there was no need for this in the early
days when they could count on the existence of mainframes.

Oh, and importantly, related to this, backup media was scarce and
expensive, and backup software not sophisticated. So one could have
centralized servers that were backed up. One might have a single
"cart tape" ("cartridge tape", kind of like a huge cassette tape) that
they occasionally backed up a set of files onto, but ordinarily files
that were going to get backed up wanted to be on a server. i found
cart tapes too much pain in a lot of cases and was glad the mainframes
were backed up.
.

Relevant Pages

  • Re: Database Performance Problem between 3:00PM and 4:00PM
    ... where by database becomes extremely slow between 3:00PM and 4:00PM. ... When one looks at CPU Usage, Memory Usage and Disk Usage nothing ... FRom the beginning we suspected Network to be an issue, ... what hardware do you use for storage? ...
    (comp.databases.oracle.server)
  • Re: intrepid qemu broken?
    ... -snapshot write to temporary files instead of disk image files ... Network options: ... connect the user mode network stack to VLAN 'n' and send ... -serial dev redirect the serial port to char device 'dev' ...
    (Ubuntu)
  • intrepid qemu broken?
    ... -snapshot write to temporary files instead of disk image files ... Network options: ... connect the user mode network stack to VLAN 'n' and send ... -serial dev redirect the serial port to char device 'dev' ...
    (Ubuntu)
  • Re: Why does MAIL DIR go slow during mail receive?
    ... Do you have more than one storage disk? ... > I have a question about an observed slowness in VMS MAIL. ... > directory while a different user sat in MAIL doing the DIR command. ...
    (comp.os.vms)
  • RE: Pull request for FS-Cache, including NFS patches
    ... Local disk cache was great for AFS back around 1992. ... desktop box and are connected by sufficiently fast network and you have ...
    (Linux-Kernel)