Some MM challenge musings

From: Pierre le Riche (pleriche_at_hotmail.com)
Date: 02/10/05

  • Next message: Eric Grange: "Re: Some MM challenge musings" 
    Date: Thu, 10 Feb 2005 08:17:52 +0200

    Hi,

    A few musings/suggestions I would like to make regarding the MM challenge.

    1) FastMove.pas should always be in the uses clause of the .dpr. MMs don't
    have to use it if they don't want to, but at least it would level the
    playing field.

    2) The FastMove.pas included with the current B&V crashes on WinXP SP2 on an
    AMD64 with DEP enabled. Probably easy to fix - haven't checked.

    3) Optimizing MM entries for the B&V tool: I'm sure this will become a very
    contentious issue. Certainly the winner should be the MM that will be the
    fastest in real world applications, however since the B&V tool is probably
    easiest to test with our MMs could well end up being fast in the B&V tool
    but not so great outside it. With so few benchmarks at the moment, and only
    1 remotely "real world", it's easy to exploit the weaknesses in the B&V tool
    by making dubious "optimizations" that would be unacceptable in real world
    usage. I'm talking excessive block upsizing in reallocmem calls, reluctance
    to downsize blocks, and very coarse block size granularity resulting in
    excessive overhead when allocating many small objects. These are just some
    examples. However, I think placing a ban on the types of optimizations is
    not the solution. What we should do is fix the B&V tool. My suggestion is
    the following: If you see a weakness or dubious optimization in a particular
    MM, write a benchmark that punishes it for that behaviour. If everyone tries
    to nail their opponent's entries, the winner should really be the best
    entry.

    4) Benchmark weighting. Some benchmarks are trivial and others are much more
    "real world". How do we weigh the different benchmarks?

    I'm busy fixing some bugs in the B&V and I will also be adding some
    benchmarks. Expect it later today.

    Regards,
    Pierre

  • Next message: Eric Grange: "Re: Some MM challenge musings"

    Relevant Pages

    • Re: Linux Software RAID 5 Performance Optimizations: 2.6.19.1: (211MB/s read & 195MB/s write)
      ... Justin Piszcz wrote: ... stripe_cache+read_ahead were the main optimizations that made everything ... I have individual bonnie++ benchmarks of ...
      (Linux-Kernel)
    • Re: ruby-ole-1.2.3 released!
      ... Initial property set support. ... Optimizations from benchmarks and profiling, ...
      (comp.lang.ruby)
    • Re: MM ideas
      ... Well, I've had some experiences with MMs that were very fast in benchmarks, ... but slowed things down at runtime in real apps. ...
      (borland.public.delphi.language.basm)
    • Re: Fastcode MM B&V 0.56
      ... >> Most MM's run through all Tests like a sharp knife through cheese, ... >> but some of them didn't complete, for various reasons. ... you mentioned that all validations passed for all MMs ... a number of Benchmarks didn't work for all MMs. ...
      (borland.public.delphi.language.basm)
    • Re: Some MM challenge musings
      ... Certainly the winner should be the MM that will be the ... I'm talking excessive block upsizing in reallocmem calls, ... The experiments done in real world applications by some of you are of great ... but make many different benchmarks and more real world ...
      (borland.public.delphi.language.basm)