Re: Fastcode CompareMem B&V

Hi Dennis,

>> 1) Arrays to be compared are not initialized, so the number of compare
>> iterations needed to hit the first mismatch is down to luck. This badly
>> affects the scoring depending in which order the entries are benchmarked.
> SetLength initializes all data to zero?

Only if the elements are reference counted (e.g. array of strings, array of
interfaces, etc.).

>> 2) The largest compare is up to 1000 bytes. There should be a benchmark
> with a larger range.
> Can you argue that case further. We need to know more about compare sizes
> in
> the real world

We'll never know the exact breakdown of the values of the size parameter
passed to CompareMem in programs out there, but surely there will be
programs using a size parameter greater than 1000. The weight such a
benchmark should carry can be debated, but I don't see how there can be an
argument against inclusion of such a benchmark.

>> 4) The number of bytes to compare (the third parameter) is monotonically
>> increased in each benchmark run, thus putting no pressure on the branch
>> predictor in places where the CompareMem routine needs to branch
>> depending
>> on the number of bytes to compare.
> At which compare lengths would you branch and how much is the cost af a
> mispredicted branch versus the number of clocks necessary to compare those
> bytes?

I don't know, and with the benchmarks as it is now I cannot tell. I did run
my entries in a profiler though, and all the compare size checks and
branches are essentially free - which surely shouldn't be the case.

>> 5) The data to compare is exactly the same for all compares in a given
>> benchmark (of which there are only 2), all that differs is the compare
>> length. There should be greater variety in the input data.
> Why? A Compare is not depending on which data to compare.

There should be enough variety to ensure that the routines get tested with
all kinds of data (e.g. identical, very-similar, very-dissimilar, etc.).
Comparing the same data thousands of times in a loop does not ensure that.

>> 6) There should be benchmarks with purposely constructed long sequences
>> of
>> data that are the same with the exception of a single mismatch somewhere.
> Why?

So there are long-running compares - see point 2. With completely random
input the chances of of the the CompareMem routine needing to compare more
than 1 dword is 1 in 4 billion.

Regards,
Pierre


.

Relevant Pages

  • Re: [fbsd] Re: (Another) simple benchmark
    ... default worker type for windows. ... While I agree this kind of benchmark is worth doing in order to compare ... performances of the Apache package and the underlying OS. ...
    (freebsd-performance)
  • Re: benchmarking FC1
    ... You want to compare different hardware plattforms with FC1 installed or ... don't think there are benchmark kits out for testing Gnome/KDE ... Fedora GNU/Linux Core 1 on Athlon CPU kernel 2.4.22-1.2149.nptl ...
    (Fedora)
  • Re: Fastcode CompareMem B&V
    ... > increased in each benchmark run, thus putting no pressure on the branch ... > on the number of bytes to compare. ... A compare should be terminated on matches and mismatches with 50% ... Deadline for 2005 B&V's was passed at 12/12. ...
    (borland.public.delphi.language.basm)
  • Re: My own small benchmark
    ... Didn't send it twice, sorry! ... Any benchmark is testing and comparing the systems on its overall ... I wanted to compare them by the efficiency, ... In some computer-magazins there are comparisions by needed ...
    (comp.lang.pascal.delphi.misc)
  • My own small benchmark
    ... Any benchmark is testing and comparing the systems on its overall ... I wanted to compare them by the efficiency, ... In some computer-magazins there are comparisions by needed ... power or TDP per GHz or something but tact per time? ...
    (borland.public.delphi.non-technical)