Re: Why RosAsm Breaks on a large number of symbols

From: f0dder (f0dder_spicedham_at_flork.dk)
Date: 07/04/04 

Date: Sun, 4 Jul 2004 06:49:39 +0200

> If you wonder why I take no notice of you any longer, it has been
> years of PHUKUPS like this sold on your status of being a leading
> guru in the assembler market who does not write assembler code and
> promotes C compilers.
>
I never claimed to be a guru, neither with assembly nor anything else.
But I do admit that, any day, I would promote a C compiler over badly
written assembly code. Not true that I don't write assembly code,
though, I just limit it to where it's useful or fun for me.

>> Of course not, polling generally kills performance.
>
> Go back to the drawing board and learn how to write a polling loop, a
> Windows message loop is a good starting point. It is bland
> generalisations like this that give me good reason to take no notice
> of you.
>
Yes, it's a generalization, and I think I stated that pretty explicitly.
Along with giving a number of concrete situations where polling is BAD.
Oh, and GetMessage suspends your thread until a message arrives, so if
you want to compare polling performance, do a PeekMessage based loop
without any Sleep calls.

> Interestingly enough, the only restriction on the use of "Sleep" seems
> to be with you, it does exactly what I require with running an
> external process which is a seperate EXE file, it yields the time
> slice to a higher priority process. These types of statements are why
> I no longer take any notice of you.
>
The masm32v8sp1 (whatever) installer does Sleep(0) + GetExitCodeProcess
in a loop, and it managed to make my system sluggish while waiting for
the library building to finish (and this was while the library building
was waiting for the initial keypress before doing any work.) - this was
probably because I had a lot of stuff running, including winamp.
Running taskmgr shows 100 CPU usage, install taking it all, with around
70-80% being spent in kernel-mode (Sleep needs a kernel-mode detour,
I haven't bothered checking if GetExitCodeProcess does but since it
involves getting data from another process, it most likely does.)

But of course this is not a genuine bug report, when it comes from me :)

> I do in fact get some very good high level advice and assistance
> for the library in the project and it will keep expanding as more
> work is done on it but it take a lot more than hot air and platitudes
> to keep it going.
>
I suggest you fix bugs before expanding, there's a couple of them and
they have been causing other people than me trouble. There's a double
free bug in the BitmapFromMemory.asm file at line 113 (because the
fDeleteOnRelease parameter is set to true on the CreateStreamOnHGlobal
call on line 85), Shell doesn't check whether CreateProcess succeeded
and it still leaks handles because you don't CloseHandle hThread and
hProcess in the pr_info structure.

Might be other bugs as well. Using GetExitCodeProcess isn't a bug as
such, it's just an inefficient hack.

>> I didn't say I don't care about Win9x or do not test thing I release
>> under 9x, just that I don't particularly care about the internal
>> implementation, as 9x is one majorly broken piece of software.
>
> Problem is here you cannot have it both ways, sure win95 had its
> problems as every operating system has but its works on a scale that
> much older hardware can handle where the vast majority of computers
> running around the world cannot run the later bloated versions. In a
> world that has an average computer power of a 133 Pentium, such
> platitudes are nonsense.
>
And as I said: I didn't say I don't care about Win9x or do not test
for it - just that I haven't studied and don't care much about the
internal implementation of things, as long as they work. I do tend
to study NT, though.

> This is what you have confused, it has to be a bug to fix it, not a
> sugestion to do a task another way.
>
Well then, at least fix the double free bug in BitmapFromMemory and
the handle leak bug in shell.

> The fundamental design I use with shell is sound, start an external
> process and poll for the exit value. The method you are proposing works
> on at least some windows version but it does nothing better and is not
> faster.
>
The method I propose, I daresay, on all windows versions. WaitForSingleObj
has been there at least since 95, it's designed to put your threads in a
0-CPU-usage wait-state on various objects, including threads and processes.
Read the microsoft documentation, you'll see :)
As for nothing better / not faster, you might want to listen a bit to
Randall Hyde - an example that shows success proves nothing, and example
that shows a problem shows, well, a problem. (The way he phrases it sounds
a lot more elegant, forgive my clumsy English.) Your way of doing it gave
me sluggish behaviour while installing the latest masm32, and I am pretty
sure I can reproduce the behaviour. The 100% CPU usage with 70-80% spent
in kernelmode is 100% reproducable.

> What you have continued to confuse is the target of such a procedure,
> "shell" is a DOS era design, not a multitasking, multithreading design
> and it needs to be written because the operating system does not have
> such a function available.
>
I'm not confusing anything. You need to start an external process, and
you need to wait until it's done executing. This should of course be
done in the simplest and most efficient way - which you aren't doing.
Well, one thing that _does_ confuse me is why you keep talking about
multithreading etc. in this context - apart from, of course, the fact
that creating a new process creates at least one new thread in the
system :-)

> This is because Windows is fundamentally an asynchronous multitasking
> environment that is badly suited for synchronous operations.
>
Async multitasking environment yes, but it has a lot of facilities
when you need synchronous operations, whether it's intra- or inter-
process. This doesn't have much to do with the issue of your shell(),
though.

> When you make reference to "hacks" you are in fact referring to how
> Window handles synchronous operations. The only safe place to perform
> synchronous operations is within a single binary file as the system
> must be able to run binary files in that manner.
>
Nah, there's plenty of hacks in windows, but I'm referring to your
hack right now :). When you say "binary file" you probably mean a
process. You can sync between processes too, but it can be tricky to
do right if you need anything fancy (like syncing access to a memory
mapped file). Waiting for process termination is trivial though, and
doesn't require anything but a WaitForSingleObject call :)

> The ignorance here is also underwhelming, the bottom line is if you
> don't need a linker option, turn it off. At the time I posted a test
> that showed you the mistake you were making.
>
Your 'test' showed nothing - however I posted a sample that you failed to
comprehend :). It's pretty easy, repeat a PROTO definition and it will
cause masm to output an external reference. With /OPT:NOREF this makes the
linker include that DLL even if you don't call (or otherwise reference)
the proto. With /OPT:REF the linker detects the symbol isn't referenced,
throws a warning (not an error), and doesn't add the DLL to your import
table.
The easiest way to show this problem is to build an application that does
nothing but a call to ExitProcess, link it with /OPT:NOREF, and check the
import table. The comment out the wsprintf definition from windows.inc,
build again, and test the import table. If you don't see it then, I will
give up convincing you - but of course keep on telling other people about
this inefficiency.

> Now with you persistent misleading advice about the register
> preservation convention for Windows programing, we were in fact NOT
> saying the same thing. You have kept parrotting that do the
> preservation in a CALLBACK when its very bad and unreliable code
> design to do this.
>
I said the place where windows requires preservation is in callbacks
(including but not limited to wndproc, dlgproc, WNDENUMPROC, etc.).
That 'preserve' means "push/pop if you use them", not "push/pop blindly".
That the registers you don't have to preserve should be considered
trashed if you call external code (like the API).

> You ALWAYS preserve register on the basis of need according to the
> existing convention.
>
Duh?

> When someone takes your advice and does the preservation at the callback
> level, they do not catch register prservation errors in procedure higher
> up so the code they write above the callback will
>
You need to preserve register in callbacks - that's the only _rule_ about
register preservation in windows. For your internal code, you're free to
use whatever calling convention and register preservation you want. You've
said this yourself, too, on multiple occurances (and, thankfully, usually
with a warning of one form or another.)

I can't remember the register-preservation thread exactly, but I haven't
changed my mind since then. It's about what windows requires of you -
register preservation in callbacks. I generally advise people to use the
standard calling convention for their own procs, unless there's substantial
benenift from using a custom scheme, so your fine list of crash synonyms
is unwarranted.

> You should leave Scali out of you vendetta, he is at least an
> honourable enemy that will knife you front on and tell you why and I
> have never seen him do anything stupid like supporting virus coding,
> he primarily write graphics code and at least manages to be useful.
>
I've told you about your bugs and I've reported them in public. If that's
not up front, I don't know what it is. Would be interesting to know which
kind of backstabbing you feel I've done, as I thought I've always addressed
things in public. I have never supported virus writing, and I have never
released any such code - so I suggest you stop your libel.
I also find it interesting you have to turn to such personal attacks, when
I'm just pointing out bugs and flaws in your code.

> .... but nowhere is there a place for the Gospel according to f0dder.
>
Heh, "Gospel according to f0dder."
Perhaps you have a bit more respect for people like Jibz, Donkey, _Stone,
and Jeremy Collake - all of them pretty fine programmers. Perhaps you
should have a look at how they do certain things...

> Years of treachery to old friends <snip>
>
Treachery? Pointing out bugs? I think that's somewhat less treacherous
than your constant patronizing, and libel along the lines of calling me
a virus writer.

If you get the time, you might want to read the PlatformSDK chapter on
synchronization, and perhaps study "Inside Windows 2000" - both are
interesting, and will lead you to writing better code.