Re: TelCo "programming challenge" (update)

From: Bill Turner, WB4ALM (Abracadabra-magic-wb4alm_at_arrl.net)
Date: 05/14/04 

Date: Fri, 14 May 2004 19:57:40 GMT

Mike Cowlishaw wrote:
> David Frank wrote:
>
> [sigh there seems to be an almost identical threads in c.l.fortran]
>
>
>>Well to chime in for my fav language Fortran, below is a scorecard
>>that needs more inputs from fav language adherents..
>>
>>1. Fortran 25 statements 3 sec to execute
>
>
> [so here's an almost identical answer. sorry, readers...]
>
> Hi .. yes, by using integers and manual scaling, it is easy to
> write a special-purpose piece of code for this that is very
> short and fast.
>
> But that's missing the point of the benchmark The benchmark
> is intended to estimate the overhead of the calculations when
> they are done in a straightforward and maintainable way.
>
> The total speed is uninteresting, as is the size of the code
> (and for readability Rexx probably wins). As someone pointed
> out, the full application would be more complex and work
> on much larger quantities of data. What the benchmark
> does appear to do is match the behaviour of those large
> real applications in the one respect it is intended to measure.
>
> Mike
>
>

"...The benchmark is intended to estimate the overhead of the
  calculations when they are done in a straightforward and
  maintainable way."

"...What the benchmark does appear to do is match the behaviour of those
large real applications in the one respect it is intended to measure."

- - -

I like your statements, Mike - Over the years or my employment, I have
observed many people not understanding what a benchmark is or does...
...often with a frightening misuse of the results.

As part of a "challenge" of why you MUST understand what the benchmark
is measuring, I allowed a vender once (many years ago) to make changes
to a major benchmarking suite or programs that my employer used to
verify the "actual" processing speed of mainframe systems handling OUR
processing job mix.

The change was simple and did not change any of the processing modules
in the test suite. (The file blocksizes were mearly upped from the
previous standard of 8k blocking to a newer one of 27k (half-track
blocking) --- which just happened to match our guideline for optimium
disk file blocksizes.)

As a result of this minor external change, a non-IBM monolithic machine
with one processing engine seemingly out-performed the 8-engine IBM
top-of-the-line system that we were using at that time. In fact, the
Non-IBM system was rated at 18 to 20 times faster than our fastest IBM
systems --- something that we knew just couldn't be true...

As it turned out, the seemingly "innocent" change to the benchmark JCL
caused some cache memory and an "instruction pipeline" to be constantly
flushed on the IBM system, while the single engine system gained a hugh
advantage because of the way it used its smaller cache memory with it's
one processing engine.

Ever since then, I have been very careful before beleiving "my thingy
is bigger (or faster) then your thingy" type claims - especially in the
Computer Industry.

Many computer hardware nuts are aware of the fisco that occurred a few
years back, when one of the PC Video Card manufacturers modified the
hardware on their Video Card to exploit the "standard" benchmark test,
by storing more of the "polygon" requests onboard while deciding how
to best perform the requested operations. The modified card did run
the benchmark tests far faster then all of it's competitors, but if I
recall correctly, it did not really perform very well when doing real
world gaming graphics....

As has been said often "Ceteris paribus - Caveat venditor".

/s/ Bill Turner, wb4alm 

-- 
*******  -..-..-.. *** -..-..-.. *** -..-..-.. ***   -. * .  *******
***  Please remove the dashes and abracadabra magic to email me. ***
***  /s/ Bill Turner, Wb4alm                                     ***
*******  -..- * -... * ....- * .- * .-.. * --  ****  .- -.-   ******

Relevant Pages

  • Re: Limiting closed port RST response from XXX to 200...
    ... Mike Silbersack wrote: ... |> On a server I'm benchmark testing, via local host, I'm getting Limiting ... What you should be asking is - why are 7500 junk packets per second ...
    (freebsd-hackers)
  • Re: Fortran decimal anyone?
    ... >> Mike, ... >> perhaps you can supply missing info in below scorecard? ... > But that's missing the point of the benchmark The benchmark ... PLUS statements to invoke rounding required. ...
    (comp.lang.fortran)
  • Re: Which method would be faster?
    ... [MIKE D] ... any benchmark or other test is going to be affected by ... the "power" of the PC, as well as other factors (network connection, et ...
    (microsoft.public.vb.general.discussion)
  • Re: Fortran decimal anyone?
    ... > I will chime in for my fav language Fortran, ... > perhaps you can supply missing info in below scorecard? ... But that's missing the point of the benchmark The benchmark ... Mike ...
    (comp.lang.fortran)