Portability, blindly

T.M. Sommers wrote:
> Beth wrote:
> > Ah, more "deep irony" for you...after just criticising Microsoft for
having
> > no good ideas ever, I'm now about to praise one of their ideas from
> > VC++...basically, the idea of simply putting "debugging info" into a
> > _different_ file that you just pop in the same directory (and the
debugger
> > knows to look for it, loads it up and uses it :)...this is probably the
> > better idea overall...it doesn't need to "pollute" the actual
executable
> > (and no need to do daft things like compile with debug info and then
> > compile without debug info and then compile with debug info to switch
it
> > "on" and "off" :)
>
> Since in your development code you are using lots of asserts and
> other debugging scaffolding (you are, aren't you?),

Yes, I demonstrated before - regards the topic of "conditional assembly",
at that time - how to create such "debugging scaffolding" that could be
easily turned on and off by defining or not defining a symbol on the
command line...which, though I didn't show it in the example at the time,
could be equally used for "asserts" and other such stuff too...

A simple enough mechanism, in fact, that's very flexible:

-------------------------------

#ifdef DEBUGGING

#macro Assert

; stuff for assert goes in here

#endmacro

#macro DebugString

; stuff for printing debugging strings goes here

#endmacro

#else

; delibrately empty
;
#macro Assert
#endmacro

; delibrately empty
;
#macro DebugString
#endmacro

#endif

-------------------------------

....then, in the code, you can litter "asserts" and "debugging" stuff all
over the code, as needed...but this equates to absolutely nothing at all,
unless "DEBUGGING" is defined, when you compile the code...

> you have to
> recompile your production release anyway, so that is no big deal.
> And you can always use strip to get rid of the symbols if you
> want, so using -s gives no benefit and removed the possibility of
> using the symbols in debugging if necessary.

Yes; But the whole point is that you can simply NOT put the "debugging
information" into the executable at all in the first place...instead, you
have a separate "debug" file sitting alongside the already "stripped"
program...

There's nothing "wrong", of course, with the other way...but it introduces
an unneeded "extra step" of using "strip"...if you're "instrumenting" your
code, then, yes, you'd need to recompile anyway...this is true whatever
method is employed (inherent in the concept of "instrumentation"
itself)...that's a kind of "red herring" here, as this would be no
different _whichever_ method was used...

Basically, it boils down to this: There is "debugging information" for the
program...it's just a block of data that the debugger comprehends that
tells it "additional information" about the program (symbol names, types
and so forth :)...now, you can put that information into a "section" in the
executable program itself...or you can put it into its own separate file...

The advantage of putting it into the executable itself is that it's
_guaranteed_ to be distributed with the program itself because it's
_inside_ that program...the disadvantage is the need to "strip" the program
of this for release...

The advantage of putting it into a separate file is that the information is
never even put into the executable at all (no need to worry about
"stripping")...the disadvantage is that you need to keep the file alongside
for the debugger to find it...

Code instrumentation - "asserts" and "debugging scaffolding" - is actually
a separate matter and requires recompilation for release, _irrespective_ of
which method is used for "debug info" above (indeed, though both are used
for "debugging" purposes, they are quite, quite different things)...it's a
bit of a "red herring" to even mention this, in this context...

But, in the end, there's _very little_ to distinguish the two
methods...neither will prove particularly difficult or contravertial,
really...yeah, "strip" isn't that much of a big deal...but then neither
would it be a big deal for a debugger to grab the "debug info" out of a
separate file, instead of inside a program "section" in the
executable...once loaded in RAM, it's not going to make a blind bit of
difference what file it was stored in...

Neither is much of a "big deal"...and my preference for the "separate file"
scheme is just that it removes an "extra step" of running "strip" or
whatever...you know, it's never added into a program, anyway...so it's one
less worry...you can't "forget" to do it or anything...you would be likely
recompiling, anyway, true enough...but there's no need for any "-s" switch
or "strip" program...

But, yes, NEITHER is really any "big deal"...a trivial difference...I just
personally like the "separate file" scheme, simply because it never does
add in any "debug info" to the executable file itself, so it's one less
worry about "removing" it...it was never there in the first place, so it
needs no "removing"...if the "debugging" file is present, the debugger
loads it and uses it...that's it...nice and simple stuff...

The difference is trivial, though...it saves one "extra step"...which,
admittedly, was never a "big deal" of an "extra step", anyway...but, you
know, why have that "extra step" when you can do things perfectly well,
just as equally, without it? That's the very trivial difference on which
I'm basing the idea of preferring the "separate file" scheme...you know,
why take "extra steps" when they aren't strictly needed...that's it...

In practice, the two are near identical in use that it really doesn't
matter...I'm suggesting "separate file" because it's, basically, a small
bit "cleaner", in my view...

> > Again, it's Linux's "fatal flaw"...all the stuff is there but the
> > "documentation" is all a bit crap that it might be there but no-one
knows
> > about it to use it!
>
> Don't forget that most generic Unix documentation and books also
> will apply to Linux. The exceptions would be the Linux-specific
> things like, unfortunately for your present purposes, the dynamic
> linker. Even so, much can be learned from the documentation of
> this things of other Unixes. You also have available the best
> documentation possible: the source.

Yes; All true...

But, as you note, the "Linux specific" stuff is excluded from this...and,
in this particular context, the "documentation" being spoke of was
_ASSEMBLY LANGUAGE_ documentation...

Now, of course, assembly language can also call the "C level
functions"...so it's not necessarily implied that because you're using
assembly language, you're necessary using the "Linux specific" stuff of
"int 80h" or messing around with the "dynamic linker" or whatever...

But, you know, this is the "low-level" end of things and one purpose of
using assembly language is to be able to _access_ that "low-level"...using
ASM does not necessarily mean you'll be using Linux in a "low-level access"
way, that's true...but for those who _ARE_ using the "Linux specific /
low-level access" stuff, they'd likely be using assembly language for that
very purpose (because accessing such things is easier and more natural with
ASM :)...

I think you get my point, Hopefully...the specific context of
"documentation" I was talking about was mostly in the "Linux specific"
bracket because it's dealing with the "low-level" and "Linux specific"
stuff...hence, this is exactly the "documentation" which is in the
"excluded" bracket you've just specified...

In a sense, that's kind of the point...there's plenty of material about the
UNIX stuff...but what's missing is the more "Linux specific"
details...indeed, it's largely missing, undoubtedly, because many want to
write UNIX software (generic) rather than "Linux specific" software...

LuxAsm is "Linux specific", though...so the kinds of documentation for this
are, indeed, what's typically "not present" at the moment...but, yeah, we
don't really need to cater for the UNIX level of explaining the "write"
call because the "man / info" pages and many UNIX books cover this in
sufficient detail...it's the "other bits" that are the problem...

And, as for having the source code available...this, again, is quite
true...but, well, it's simply a matter of practicality...unravelling lots
of the kernel source code, just to discover one parameter or one structure?
Seems a lot of work to ask people to do...wouldn't it make sense for one
person to look at the source code, derive a simple "table" of the various
details and then people can use the table...saves a lot of duplicated
effort of "decoding" lots of kernel source code...note that, of course, the
source code itself does not "disappear" for the creation of such a table,
so it would _complement_ the source code, which is _STILL_ perfectly
available for people to look through...

The point of such a "table of information" would, of course, simply be
"time saving"...but, well, what's wrong with that? They could still at any
point look at the source code...it's GPL...it'll always remain "open" and
"available": The GPL's whole purpose is to guarantee that...

What you say is all completely true; But what we're talking about is simply
"making it easier" and "time saving" for the application programmer to have
the information already put into simple "tables" and so forth for
them...and which, for once, is "Linux specific" so it doesn't exclude such
details either...

This in no way, of course, suddenly makes those UNIX books or the Linux
source code "disappear"...they are still fully available options...this is
_ADDITIONAL_ to that for those programmers who want the more "Linux
specific" details and would rather the "convenience" of a "table" listing
the details in a simple and convenient format, rather than trawling through
all the source code every time to locate one detail or another...

> > Undoubtedly why that "Linux documentation project"
> > started to attempt to "rectify" the problem...annoying, really...it's a
> > good system...easy to program, clear in design, powerful in
capabilities,
> > etc....but all a bit useless while you can't find any good
"documentation"
> > to tell you how to use the darn thing! Even more so the case with
> > ASM-related stuff, as it's mostly non-existent
>
> That is because assembly is not much used in the Unix world. The
> Unix philosophy promotes portability, which pretty much excludes
> assembly.

Yes; I'm aware of this...but UNIXen, such as Linux, can also be programmed
in assembly language too...

Now, we could long debate whether it's a "good idea" or not to do so...

But my personal "philosophy" on the matter is that it's the programmer
themselves who decides such things...they know whether their application
would or would not benefit from "portability"...simply, it is they - and
only they - who are best qualified to make that choice...to (blindly)
dictate "portability" from an "ivory tower" as something "always
appropriate" for all programs and all programmers is where I _DO_ differ
from the "UNIX philosophy"...

I _DO_ believe that "portability" should be fully available...and in the
UNIX "singular purpose" philosophy for tools and utilities...and with the
"everything is a file" unification policy...these are all good...

The problem comes from "ivory tower diktats"..."portability" _SHOULD_ be
there...100% without disagreement...it _SHOULD_ clearly be possible to use
Linux and other UNIX in a "portable" manner...that, with correct coding and
the use of "portability" constructs like standardised C calls to
standardised system calls, you _CAN_, if you so desire, create "portable
source code"...it _SHOULD_ be available...it _SHOULD_ be there...

But it should be _OPTIONAL_...the decision to use or not to use should be
the _programmer's_ choice, as ONLY they are qualified to know whether it is
or isn't a desirable attribute for their code...

And, in a sense, most UNIXen do support that...Linux has the "INT 80h" /
"SYSENTER" interface available to applications...you can "socketcall" or
"pipe" to X directly...and these are "additional" to the more "portable"
layer...so, the programmer _can_ choose what they feel is most
appropriate...

If you like, "UNIX philosophy" is to be "portable"...BUT, with Linux, you
can _choose_...it is "UNIX compatible", so is "compatible" with this
"philosophy"...but Linux also can be coded more "specifically" too...

And, simply, you've already explained things for me, in a sense...yes,
_UNIX_ is "portable" and _UNIX_ is supported by a _WEALTH_ of books,
example code and other materials..._UNIX_ has that...but "Linux specific"
doesn't...that "other option" is relatively unsupported in comparison...

Hence, what we're doing with LuxAsm is supporting that "other option"...if
a programmer wants "UNIX philosophy", they are already supported well...C
would be the language of choice...and use of the "portable" system calls
would be the choice...the "support" for that is already there...and it's
very good...and, simply, we could not do better...so, we "defer" to their
expertise on the matter...if you want "portable", there is nothing we could
really do to improve on what already exists with a C compiler, the man
pages and the UNIX books and other support...indeed, that's already
there...that's already good enough...what would be the point in us
duplicating that effort and, likely, not being able to do as good a job?

Our purpose here is to cater for the "niche" that isn't well supported...to
make that "other option" also well-supported (we Hope), so that the
decision of "to be or not to be" portable can be made by the programmer, as
is appropriate to their situation, with, if you like, the "full facts", the
"full support" and such...a choice is NOT a "choice" unless it is an
_INFORMED_ choice (hence, "documentation")...also, a choice isn't really a
choice if you can't choose it (for example, Saddam's "election" where he
was the only candidate listed...that isn't a choice...so, it wasn't an
actual "election" at all)...and, if "unsupported", then this does effect
the "availability" of making that choice...

> > (well, not totally - there's
> > "linuxassembly.org" and such - but still not really enough...and
certainly
> > "embarassing" in comparison to Windows...okay, Windows has 90+% of the
> > "market" so this is hardly surprising...but still...very annoying,
>
> How much documentation of the Windows dynamic linker comes with a
> Windows distribution?

It's not solely quantity but the quality too...and the ease of
availability...

And, anyway, you are "changing the context" here...I was speaking here of
the _general_ situation of all kinds of documentation in general, not
specifically only the "dynamic linker"...and, it's quality, to be fair, is
usually that it's easy to read...and its availability is very good because
Microsoft often almost fall over themselves to provide
"documentation"...simply as they realise that the success of their OS does
depend a lot on the wealth of software for it...and, hence, it's in their
interest to help developers develop as much as they can...

> > And, basically, if you can connect to X by creating a "pipe" to it and
then
> > just normal "writing / reading to / from a file" to send / receive
data,
> > this'll probably be faster when it comes to "local connection"...BUT,
well,
> > can you even do a "remote pipe"? Probably not...so, when the "target"
is a
> > "remote connection" then, for sure, the "UNIX domain sockets" comes
into
> > effect, as that'll be best over "remote connection" (indeed, probably
the
> > _ONLY_ option XFree86 provides for "remote connection"...makes sense:
When
> > "remote", you're dealing with other machines, so some "standardisation"
to
> > keep them all talking to each other makes sense :)...
>
> Unix Domain Sockets work only on the local machine. For a remote
> connection you need TCP sockets. They are different animals.

Actually, when running Cygwin on Windows, XFree86 triggers the firewall on
TCP grounds...so, unless they altered that as part of the "translation"
(possible; I don't know for sure, as I've not directly "investigated"),
then it probably is TCP sockets...

I only say "UNIX domain sockets" because, amusingly enough, when I said the
other before, someone - it had better not have been you - complained that I
should say "UNIX domain sockets" instead...so, I was merely responding to
that "complaint"...damned if you do, damned if you don't...great, just
great...

Either way, the basic principle holds: "local" should not be incurring
unnecessary "overhead" for its IPC...that is, you _could_, for example,
send "local" via TCP to 127.0.0.1 (the "loopback address") and then
effectively do everything via the same TCP / IP mechanism, regardless of
"destination"...

Though possible, this is not a good thing because it'll make "local" jump
through the extra overhead, that "remote" needs, _unnecessarily_..."remote"
_needs_ the extra for delivering "packets" across the network...headers and
protocol and the like..."local" doesn't need it because the "destination"
is "right next door", so to speak...more direct "local" IPC should be used
that does not incur the "overhead"...

"Portability" has its uses...but "optimal" lies in the _SPECIFIC_
domain..."portable" is almost always at a _COST_ of "performance"
(generally the case because most "portability" derives from "indirection"
and "layering" and so forth...and every extra "layer" and "indirection"
eats into the performance because, if you like, it "increases the distance
to travel" while the CPU's speed remains constant...and, simply, "time =
distance / speed"...the CPU's speed is constant, so the time taken will
vary according to the "distance" it is asked to travel...and, generally,
"portability" is achieved by "indirection", "decoupling", "layers" and many
other devices that inherently _increase_ that "distance"...leading to, as I
say, a general point that "portability" is generally inversely proportional
to "performance"...that it is a "tug of war" / "trade-off" issue
inherently)...

And, in the "local" context, "portability" is a secondary issue
inherently...for instance, it is 100% guaranteed that a machine is going to
be "portable" to itself...that is logical enough, isn't it? The
"endianness" is _guaranteed_ to be the same (even for machines that can
"switch" endianness...it is still guaranteed that the "destination" machine
supports that endianness because, for pity's sake, _IT'S THE SAME
MACHINE_!! ;)...

This, indeed, is the ultimate demonstration of how "portability, blindly"
attitudes lead to quite illogical redundency...it's _THE SAME MACHINE_, so
it _MUST_ be "portable" to itself...it could NOT be any other way...it is
"by definition" always true...hence, all "portability constructs" placed
between the machine and itself are nothing but pure, unneeded redundency...

For example, when opening a connection to an X server, the X protocol
states that a message is sent which specifies "endianness"...in the local
context, this is slightly actually absurd...the machine is telling _itself_
what endianness it is?!? It must already know what that is, or, of course,
the machine couldn't tell itself what it is, if it didn't know (that
sentence even reads absurdly because it _should_ be such an unnecessary
statement of the obvious ;)...in this context, this message is only sent
once at connection start and also "doubles" as a "connection test" too, so
it can be forgiven...but, when you take much of this "portability" and
consider it in the actual _SPECIFIC_ context that it will actually function
(and it always will be "specific" when actually running...ignoring this "in
the abstract" does not cease to make this the actual case)...there's plenty
which ends up being redundent, illogical and even a touch absurd...a
machine telling itself what "endianness" it is? This NEVER makes sense in
the "local" context...but that is exactly what the X protocol is specifying
because it's being "portable"...this particular case can be forgiven on the
grounds that it makes a useful "connection check" by having one "standard
message" sent on all connections being made, by which the sever can tell
that things are actually working properly...

A machine is inherently "portable" to itself (this is even true should it
start sending raw machine code! It cannot not be the case, by definition
;)...so, in the "local" context, an awful lot of "portability constructs"
are automatically and inherently useless...

Hence my general principle: "local" and "remote" should be considered
separate cases...the X protocol delibrately does not mandate that
"connections" should use any particular method for "delivery"...what many
aren't realising is that the lack of specification of the "transport" is a
_DELIBRATE_ measure to allow such "local" and "remote" differences to
exist...for the choice of "connection transport" to be made
_intelligently_, according to the situation...this "hint" is just not
picked up by many implementations and then end up using "portability,
blindly" on the connection, making the whole thing slower for the sake of
that which is inherently redundent and illogical...

> If you are going to be doing much Unix programming, you need
> Stevens's _Advanced Programming in the Unix Environment_, and his
> _Unix Network Programming_, vol. 2 of which is about IPC.

If you are going to be doing much criticism, you need to first understand
the points and the persons to whom you are offering your critique...

Never mind, you _look_ "clever" for doing it, yes? And that's all you care
about, in the end, isn't it?

So, "mission accomplished"...we are now perfectly aware of all your "fancy
book learning", thanks to clever use of "name dropping" there at the
end...and how you're so "superior" because you eat kernel source code for
breakfast (how laughably "wimpy" the idea of making a "convenient table"
when you program with a soldering iron and eat silicon for dinner! :)...

Great...very impressive stuff...anything else _useful_ you might want to
add here? Or is "elitism" the only "trick" you do?

Truth is useless without the wisdom to realise what it is you're in
possession of...a "badge of honour"? No, no...that's not what it's properly
used for...try again...

Beth :)


.