Re: Alternative to C for system programming

From: "W Karas" <wka...@xxxxxxxxx>
Modula-3 is designed as a modern (OO, GC, etc) system programming language.
Does Modula-3 have two kinds of pointers, one that points to
dynamically allocated objects, and another that is ignored by
garbage collection, and is covertable to/from an intergal type?

IMO you've prejudged possible solutions by stating the one and only
solution at the start instead of considering alternatives. For
example, instead of two kinds of pointers (GC/nonGC), there could
be blocks of memory that are GC and nonGC, with a table that tells
which is which, and a configuration file to determine the initial
state of the block-table early in booting, and system calls to
change this table later during booting and during normal operation
including hot updates to the OS.

When I hear "systems programming language with GC" I can't help
but be suspicious of a bait-and-switch. Seems like a tall order to
design a kernel that has GC running yet can still meet tight
minimal response times to system events. That is to say,
if a language has GC, but GC is typically disabled when doing
systems programming with the language, it's not very meaningful
to call it a systems programming language with GC.

It seems to me that so long as allocation/dealloction aren't done
right in the middle of a time-dependent event, such as serving an
interrupt on a device such as a rotating disk, there shouldn't be
any problem. Just make sure your disk buffer or whatever is
allocated *before* you start the disk-read operation, so by the
time an interrupt happens the buffer is just sitting there
instantly available as needed. Most other tasks don't really suffer
horribly if there's an occasional long pause (on the order of one
second) in the middle of an operation. For example, user's program
wants to write a new file, but directory needs GC before the system
can be sure whether expanding the directory will really be needed
to fit in one more file handle, so user's program experiences a
brief pause, no longer than more common pauses due to system gotten
really busy with fifty other users or a burst of spam coming into
the SMTP server.

So what's my answer to a sudden burst of incoming TCP/IP traffic,
where you might need to suddenly allocate a lot more buffer space
to avoid lost data? Triage is one idea: TCP/IP incoming-traffic
handler can ask memory allocation "can I get another megabyte
instantly, if not how much can I get instantly", if yes megabyte no
problem, if no the use the lesser amount that was gotten but
prioritize the incoming data, for example if a burst of spam is
most of the problem the throttle all traffic from all the IP blocks
involved in it, giving priority to all other incoming traffic
within the limited buffer space availble. Also, pre-allocate a
reasonable amount more than you already need, so you can already
handle any reasonable-size burst. Also, allow asynchronous/callback
allocation of memory: I/O module thinks it will need more memory
soon, but can't get it yet, so ask memory allocation module to
please give me three megabytes as soon as possible, with a callback
to notify me as soon as it's available after a major GC has
finished. The callback would then turn off triage mode, allowing
all incoming data to be buffered until buffers are nearly full
again and can't get more memory instantly, when triage is turned
back on while waiting for more memory via callback.

By the way, sometimes during lisp programming I wish there were a
way to make a CONS call such that if a GC would be needed to get
the needed memory then it'd return NIL immediately instead of
pausing for a long time then possibily dying due to no more memory
available. For example, I had an application that performed a sort
on a very large file. I wanted to load memory completely full with
a batch of data from the file, until there was no more room
available without doing a GC, *then* do a GC (to recover some slop
I had deliberately introduced, plus anything else that was no
longer referenced), then use the recovered space to perform an
in-memory sort on that already-loaded batch, and write to disk, and
then do a GC to recover all the space used by that batch. Instead,
the hack I ended up programming was: Introduce lots and lots of
non-accessible slop as I loaded a batch, both enough to give me
working space (as above) plus enough to hold the largest record in
the file (estimate from experience), wait until a GC had already
occurred before stopping loading a batch (which required calling
the when-last-GC function after *every* record of input), then the
rest of the algorithm basically as above.
.

Loading