Re: modularity... (was: Re: Looking for real world examples to explain the difference between procedural (structured?) programming and OOP)


"H. S. Lahman" <h.lahman@xxxxxxxxxxx> wrote in message
news:qLSVl.257$tr5.79@xxxxxxxxxxxxxxxxxxxxxxx
Responding to cr88192...

Thus a customer does not have to be in an IT domain. (I haven't worked
in IT since the '70s.) But wherever the customer lives there is some
domain infrastructure for their daily work.


yep...

70s, well before my time...

all I will say is that I have existed since during the production run of
the 286 but before the start of the production run for the 386...

so, the 286 is older than I am but the 386 is younger...

<sigh> The industry today is full of young whippersnappers.


maybe...

but, then again, I guess it could count as a failing that I am well into my
20's now but still not "made it big" yet (or, for that matter, have not
achieved much of anything "significant"...).

time escapes every minute and every second, and years go by on an annual
basis...


<apocryphal aside>
Many years ago I was at a New Year's Eve party talking with an old
classmate ('59) of mine. It was about 3 AM and we were commiserating about
the sad state of software development. He had been a contractor to JPL
working on the Apollo Lunar Lander. At one point he said something like,
"These young kids have it too *&%$#@! easy today. They have no idea what
Real Programming is. They have MHz processors, 3GLs, hard disks, and
megabytes of memory. We landed that *&%$#@! thing and got it back with a
*&%$#@! KHz processor and a *&%$#@! 50 register stack!" O.H. was always a
silver-tongued guy.
</apocryphal aside>


yep...

this reminds me of the NES, which I guess managed to do all the stuff it did
with only like 2kB of available RAM (and, officially, only 48kB of ROM,
although I guess "banking" was typically used to get around this, as well as
like a 1.47MHz processor...).

it is actually fairly impressive in some ways all the stuff typical NES
games could pull off within these restrictions...

I guess desktops were the opposite extreme, with 1MB of address space (and
512 or 640kB RAM), and an 8MHz processor, but with only CGA or EGA graphics
(point being that the NES had better graphics), well, that and the PC
speaker being terrible...

then again, an NES was much cheaper than an IBM PC though...


I guarantee you that no one in those industries is going to try to shove
a new protocol into that diagram without thinking a whole lot about how
to accommodate it. They will find a place to integrate it that will have
the least disruption. So if your mongo networking support system
faithfully reflects the structure of that diagram, it should be
<relatively> easy to introduce that new protocol into the software.


yes, ok.


this more reflects the world I interface with, only replace "protocols"
with "calling conventions" and "languages" and "libraries" and similar.

That's why god invented wrappers. B-) If the pot pourri of technologies de
jour don't fit, wrap them up to look like something else.


most VMs do this, but it leads to all sorts of integration issues...


BTW, one thing I don't think I mentioned in my ramblings was state
machines. Long before OO the R-T/E people discovered that state machines
where an excellent tool for synchronizing things, hence one cannot talk
about networking without talking about protocols.

As it happens, if one were tasked with defining a construct that would
*enforce* good OOA/D practice, the object state machine would be it. The
underlying rules of FSAs are very much in keeping with OO concepts. For
example, a state cannot know what the last state was or what the next
state will be. That makes it impossible to define state actions (object
methods) whose implementations depend on external context, particularly
what other object methods will do.

Object state machines also allow one to capture problem space rules and
policies in static structure. That is, the valid transitions define
constraints on the sequencing of invoking the object's behaviors. Then the
handshaking of interacting object state machines ensure that behaviors get
executed in the right sequence to solve the problem in hand. Consequently,
the translation methodologies, where rigor is needed to support full code
generation, all *require* that object state machines by used to describe
any object with behavior responsibilities.

You've indicated that you use event-based processing. I would suggest that
you look for ways to use more of it. It is difficult to get state machines
to interact properly, but once they are working they tend to be very
robust.


yes, ok.



I may as well mention that I have messed around with Erlang and similar
before, and Erlang had influenced some of my practices...

only that directly using this style in C (via threads, ...) turned out to be
very expensive at the time (not very efficient, high latency, ...).
eventually, this modified itself into making use of work queues (becomming
something essentially similar to the "LINDA" model...).

some of my interpreters had worked this way, as well as some of my early JIT
engines. typically, this involved reworking code into CPS form and using
queues to drive execution, ...

of course, I don't usually do this in C, as it is an awkward style.
similarly, having a compiler compile C code in CPS form would not likely
perform the same as non-CPS code...

however, this does allow certain kinds of features not possible with non-CPS
code, such as fine-grained concurrency, ability to stop and resume things,
automatic load balancing, ... as well as interesting language features, such
as 'async' calls, continuations, ...


actually, in one of my earlier projects (which did use an essentially
CPS-based interpreter, for Scheme, thing being back in 2003 or so), I was
able to have a full-3D world be shared between 2 computers over a LAN (with
no explicit client or server, or any specific networking machinery built
into the 3D engine). it also worked with the GUI, which was also written in
Scheme... (I could see the world on both computers, and it stayed in sync in
real-time...).

as can be noted, it was also possible for both threads and code to move
easily between any cooperating computers, ...

this is partly because the interpreter made use of special network
protocols, and because these protocols allowed pretty much the entire VM
heap to be treated as a sort of DSM (though, it was based on
semantic-synchronization, rather than physical-memory synchronization).

so, whenever one computer would change something, it would jump over the
network and the other versions would be updated. similarly, methods were
typically copied over and executed locally (rather than the more traditional
RPC-style client/server model...).

(as noted, even killing the original server, the world would continue
running unaffected...).

as noted, the protocol itself was fairly simple, as basically it was a
glorified stack machine (sort of like PostScript, but binary...). this way
the protocol was fairly easy to extend, as there were no specific "message
types" as in most other protocols (just a small fixed set of opcodes...).

also, subtly interesting: the protocol also had the I/you/they distinction
(as in natural language...).
the protocol itself did not know about code migration/... rather, the code
would "silently" flatten itself into a stream of opcodes and re-appear (it
would re-assemble itself) on the other end (and a few more opcodes would
come over the stream to get it where it needed to go, cause it to start
running, ... in turn, this could pull over more code and data, ...).

the opcodes themselves were typically simple data manipulation ops, such as
car/cdr/cons/...


none of my later technologies have generally been capable of this feat...

as the great cost, it required pretty much everything to be written on top
of the interpreter (including the physics and most of the renderer).
whereas, since then, most of my stuff has been written in C, and I have not
exactly managed to pull off making C "network transparent"...


making something like this possible again would require both making a
CPS-capable codegen backend (general purpose enough to handle C and
friends), as well as fully implementing (once again) DSM capabilities (this
is partly done via my compiler's "wide pointer" facilities, which utilize a
128 bit virtual address space...).

if such implicit code-migration were made possible again, likely it would be
done via migrating IL (and not machine code), and forcing all pointers to be
"wide", as well as via careful coding practices (non-careful code would
likely blow up...).

or such...



basically, I had decided, rather than implement a "typical" VM (AKA: with
all the usual issues with interfacing with the outside world), I would
work to try to keep external integration as a fairly high priority (and
try to keep things as modular as was reasonable).

but, alas, this has all complicated things somewhat, and in the case of
x86-64 had led to an ugly hack:
rather than being able to directly use the external calling conventions,
a custom one is used internally, and the dynamic linker automatically
stubs things to interface with the external world.

note that "XCall" (the internal calling convention) involves 2 major
components:
first is the calling convention itself (which on x86-64 combines elements
of x86 cdecl, Win64, and SysV/AMD64);
second is a name-mangling scheme, which is itself critical to the proper
operation of the calling convention (actually, "xcall" may be used on
x86, but in this case is just the name-mangling and a slight tweak of
cdecl, and so serves a role similar to C++ and the "thiscall" calling
convention...).


however, it is not as bad, as otherwise I would have had to implement 2
essentially different x86-64 targets: one for Linux and friends; and
another for Win64. as-is, I can't directly use either calling convention,
but the stub machinery works for both of them...

of course, it does lead to a "worst case" when using 2 internal function
pointers, as code may need to perform 2 transitions: XCall->SysV, then
SysV->XCall...



note that each transition is a chunk of automatically generated machine
code...

a partial solution is possible in my case:
void *(__xcall *foo)();

where the '__xcall' keyword allows the function to be called natively
(and thus faster), but will blow up if one uses it with a SysV or Win64
function pointer (or just casually assigns a function pointer between one
and the other...).

void *(__xcall *foo_f)();
void *(*bar_f)();

void *foo() //internally, xcall is used
{
}

...
bar_f=foo; //OK, bar_f gets a "wrapped" function pointer
foo_f=bar_f; //BAD

foo_f=foo; //OK, raw function pointer
bar_f=foo_f; //BAD

one "trick" here could be to involve internal conversion machinery, which
could make the operation safe but expensive...


alternatively, my planned alterations to the compiler core should be able
to make the actual calling convention part unnecessary (the compiler
would be flexible enough to handle both Win64 and SysV, and choose the
one most correct for the target), however, the name mangling scheme would
continue being useful.

similarly, if I am right my calling convention "should" perform on
average faster than SysV for typical coding practices. but, if I am
correct, than a similar pattern should likely appear:
pure-code benchmarks (not using system calls) should be slightly faster
on Win64 than on Linux x86-64. the reason here is that I suspect the SysV
calling convention, due to likely increase in register shuffling for many
tasks, may actually be slower than passing values on the stack (Win64
uses far fewer regs for arg passing, so there should be a difference).

No sage advice here. The last Assembly I was involved with was for PDP11s
and uVAXes. By the time we started using Wintel in our systems the
hardware I was on a different side of the house. We also never used VMs
because of the overhead; the closest we came was some DSP code that was
burned into an ASIC and ran embedded.


yes, ok.


it doesn't work as well for compilers, where I guess the better
approach is to study existing implementations...
Depends on the compiler. Generally the compiler for a specific LALR
language is not a good application area for the OO paradigm and its
problem space abstraction. That's because it is very linear where each
statement is pretty much independent of every other statement to be
parsed.


'LALR':
the parser is only a small part of the process (as far as complexity
goes).

all of the internal machinery and machine code generation tends to be far
more complex IME...

But the LALR structure is what provides the linearity. One can write the
code with at most three tokens in hand. That vastly simplifies the
compilation itself.

While no compiler is trivial, I don't see that much complexity.
Essentially the Syntax Table becomes a jump table to snippets of code that
write code fragments for the tokens in hand. Those code snippets may get
complicated, depending on what one is compiling, but they are
self-contained.


typically the parser produces an AST;
this AST is compiled to an IL;
this IL is parsed into an internal representation (in my case, a specialized
bytecode);
manipulations are performed, ...;
register allocation is done and the codegen begins writing out code.

a great problem in my case is RPN, which basically is nearly impossible to
re-organize by itself, so a different model is needed for the low-level (in
this case, a variable-based model), mostly so that I can get away from RPN's
rigid ordering issues...

the most terrible problem I have found though for RPN, is RPN... namely, it
is when trying to interface 2 different RPN-based models, but where the
ordering is, different...

reorganizing stack-machine -> stack-machine transforms are a pain (or, at
least, once the reorganizations are much of anything more complex than a dup
or exch here and there...).


RPN is very powerful and convinient IME, just it is working on such a
representation which is a problem... (reorganizations and abstract
transforms are terrible...).

so, the idea is to get into an easier to work with form:
SSA (Static Single Assignment) and TAC (Three Address Code).


apparently, it is capable of many almost magical things (even things as far
reaching as me using it to drive speech-synthesis and
natural-language-processing tasks), just it is not ideal for internal
compiler stuff (or, at least, once the compiler begins to become
"non-trivial").


For example, one of the languages we had to process specified the content
of a binary file. A record in that file was a very complex data structure
with dozens of internal offset tables to groups of different data and each
group had its own offset table to individual entries. The compiler's job
was to write the records to the file. So one might have LALR syntax like

PORT name PINS count TYPE type ...

That statement defined a PORT entry in a port group in a file record.
Parsing out the entry into binary values was easy. The tricky part was
putting the entry in the right place in the file record. So a snippet of
code was executed as soon as the opening PORT keyword was encountered.
That snippet "walked" the offset tables for stuff already defined to find
the next available spot for a PORT entry. That address was then available
to the rest of the parse so that each value was plugged into the right
place in the entry. Thus the intelligence was in the snippets of code
invoked from the syntax table.

[Obviously it was a bit more complicated than that because one had to
update the offset tables as well and one wanted a contiguous file record.
So the offset tables and groups were held in memory until one got to the
end of the file definition. At that point another code snippet cobbled
them together and actually did the file write. IOW, the actual compilation
was not LALR and the code snippets did extra cute stuff with offset tables
and whatnot. Nonetheless, that was easily handled in code snippets that
were unambiguously invoked from the syntax table. Moreover, because of the
granularity of the parse the code snippets weren't all that complicated
because they had very limited and specific things to do at that point in
the lexical analysis.]

Now doing a VM interpreter is not the same as writing a file record. But
in principle I don't see a lot of difference. The real work for LALR
syntax gets done in actions that have limited things to do. Those actions
are unambiguously dispatched from the syntax table. There will be some
infrastructure to keep track of things like data definitions and what's
currently in registers that the actions will need to reference so
individual actions may be somewhat complicated, but that is well-contained
and highly focused code.

So if one looks at all the actions and infrastructure taken together,
there may be substantial complexity. But I think is it *managed* in a
pretty straight forward and cookbook fashion. IOW, the complexity involves
more tedium than creativity.



my project is not an interpreter...

it produces both x86 and x86-64 machine code...

now, the tool was damn-near broke forcing x86-64 into the mix.
things get even more broken trying to force Java and C# into the mix...
it is a small mountain of awkward hacks...


hence, the need to redesign the lower end of the compiler to use a more
flexible design (AKA: not structuring the compiler internals as a big
complicated stack machine...).

aparently, a PostScript-based design is not the ideal basis for a "general
purpose" compiler (note that RPN is not just the input, but the actual
organizational super-structure for most of the internal machinery as
well...).

everything inside, almost all the way down to the ASM code fragments being
spit out, is all so many pushes and pops along a big stack...

actually, this was evidenced at one point by me using a simple hack, and
effectively utilizing the internal code optimizer as a full-on
interpreter... essentially my codegen/optimizer is itself turing-complete...
(the changes to pull this off were trivial...).

but, alas, there just seems to be something a little out of place in the
ability to get turing completeness out of something never actually designed
to "run" in the first place (just process input and spit out bits of
ASM...).

it would be similar to a situation where, if one noted that with the right
combination of #defines' and #ifdef's, the C preprocessor would suddenly
gain the ability to start running on its own and start autonomously
#include'ing arbitrary files and producing "novel" output... maybe it might
be ammusing, but in a way almost disturbing as well...



so, internally, for a while, I had been gradually erroding this stack-based
superstructure, gradually replacing it with a new model: registers and
general purpose operations... (AKA: directly adding 2 registers and storing
the result in a 3rd, rather than popping 2 values and pushing the
result...).

however, now even though a lot of the lower-level machinery is liberated,
the actual codegen superstructure itself needs to be liberated.

then, what of RPN?...
it can either be redirected to the new superstructure, or maybe abandoned, I
don't know yet...



--
Life is the only flaw in an otherwise perfect nonexistence
-- Schopenhauer

H. S. Lahman
H.lahman@xxxxxxxxxxx
software blog: http://pathfinderpeople.blogs.com/hslahman/index.html


.

Relevant Pages

  • Re: What does 32 bit application mean?
    ... Does this means that the compiler is ... Thus it corresponds to the register ... > assembler for 32 bit architecture. ... > [On all the modern machines I know, the register size and the address ...
    (comp.compilers)
  • Re: Atmel AVR assembler
    ... so I decided to write my own assembler. ... >> extensive testing on various machines and operating environments. ... where I may not be able to legally restore the compiler toolset. ... supporting tools where the code was developed using Lattice C around ...
    (comp.arch.embedded)
  • Re: Win XP no longer able to access LAN with win 98SE machines
    ... You only need to use this protocol if you have ... Right click on the adapter connection you wish to add the protocol and ... the other Windows machines as well. ... the computers on the different LAN segments all have static IP ...
    (microsoft.public.windowsxp.network_web)
  • Re: New system - dual-core processor??
    ... I made some tests with crafty only with different kind of machines, ... The time recorded is the one displayed by crafty in its analysis tree when showing Bxh6 as the new best move. ... Under Linux, the compiler used by default was gcc 4.0.3 ... Under windows, I used the executables available on the net, but I strongly suspect that they were compiled using the intel compiler. ...
    (rec.games.chess.computer)
  • Re: <ctype.h> toLower()
    ... > That depends on how the compiler arranges things. ... But you just said that string literals are constant so how can you assign ... In this context a pointer to an array of *constant* characters is passed. ... On Non-Windows machines this is different. ...
    (alt.comp.lang.learn.c-cpp)