Re: export

From: David Abrahams (dave_at_boost-consulting.com)
Date: 09/03/04 

Date: 2 Sep 2004 22:23:12 -0400

"Walter" <walter@digitalmars.nospamm.com> writes:

> "David Abrahams" <dave@boost-consulting.com> wrote in message
> news:upt56cq3g.fsf@boost-consulting.com...
> > "Walter" <walter@digitalmars.nospamm.com> writes:
> > > I'll start by saying that DMC++ is the fastest C++ compiler ever
> > > built.
> > That kind of assertion is part of why some people don't trust your
> > claims. It's meaningless on its face, because what's "fastest"
> > depends on which programs you try to compile. I also note from
> > http://biolpc22.york.ac.uk/wx/wxhatch/wxMSW_Compiler_choice.html that
> > you don't seem to have speed testing some significant contenders.
> > Metrowerks Codewarrior is one of the fastest compilers in some of my
> > tests. So it's hard to see how you can claim to have the "fastest
> > C++ compiler ever built."
>
> I didn't write that speed benchmark or the web page it's on. You're
> welcome to post some benchmarks showing MWC (or any other compiler)
> is faster.

I'm not claiming some other compiler is faster. I'm saying that _you_
claim DMC++ is "the fastest" on the basis of substantially incomplete
data.

> If so, I'll retract the statement.

If you want people to trust your claims, go to some lengths to make
sure they're backed up by substantially complete tests. I don't feel
a need to prove you wrong -- who knows, you might even be right -- but
there isn't enough data to say yet. I'm content to sit on the
sideline pointing out the flimsy foundation for your bold claims ;-)

> The wxWindows makes an
> excellent compile speed benchmark because:
>
> 1) it is not contrived in any way to be a compile speed benchmark
> 2) it is real, widely used, mainstream C++ code
> 3) it is very large
> 4) it is freely available for anyone to verify the results for themselves
> 5) it has been ported to a very large number of compilers
> 6) I didn't write wxWindows, didn't run the benchmarks, and have no control
> over the web page it's posted on, lest I be accused of bias.

Sure, if you're compiling GUI code written in the style of wxWindows,
it's a good benchmark. If you're doing high-performance scientific
computing it might be completely inappropriate.

> But of course I get accused of it anyway <g>.

Actually I didn't accuse you of bias. Everyone expects you to be
biased (at least I do) towards something you wrote. I also expect
claims to be fair and supportable, which I don't think yours are, in
this case.

> > > I've spent a lot of time working with a profiler on it, going
> > > through everything in the critical path. The critical path
> > > turns out to be lexing speed. How fast can it deal with the raw
> > > characters in the source file?
> >
> > That's commonly the bottleneck in the parsing process of many
> > languages...
>
> Yes, but it is worth verifying, since profiling can sometimes produce
> surprising results.

Yes.

>
> > > Unfortunately, the phases of translation required in C++ means that
> > > each character must be dealt with multiple times. D is designed so
> > > that the source characters only need to be examined once.
> > ...but if that's your bottleneck, you obviously haven't tested it
> > on many template metaprograms. Template instantiation speed can
> > be a real bottleneck in contemporary C++ code.
>
> You might be right, but I don't see typical C++ code's dependency on
> massive quantities of .h files going away anytime in the foreseeable
> future.

For a certain class of project, that is indeed an important bottleneck

> Just #include'ing windows.h, and all the headers it #include's, is a
> big bottleneck. Last time I checked, it defined 10,000+ macros. STL
> is another huge source of text that needs to be swallowed.
>
> > I happen to know that DMC++
> > can't yet compile a great deal of Boost, so maybe it's no coincidence.
>
> Since Boost contains workarounds for non-compliance in many
> compilers, but such work was not done for DMC++, that is an unfair
> remark.

It's not an unfair remark. Compilers that require fewer workarounds
get ported much more quickly. It seems logical that you haven't speed
tested DMC++ against many template metaprograms if DMC++ can't compile
Boost, for whatever reason.

> Currently, David James is doing some excellent work in adapting
> Boost for DMC++, and he's been very helpful in identifying some
> problems that need fixing.

I know.

> So far, none of them have any influence on compile speed, and I
> don't expect they will. The only C++ feature that did was ADL -
> which works correctly in DMC++.

...which is more than I can say for some other compilers. So, Bravo!

> > It's easy to be fastest if you don't conform,
>
> I don't know of any correlation between compiler performance and
> conformity across compilers

Well, I can tell you that the front-end widely acknowledged to be the
most conformant is also the slowest in many of our metaprogram
compilation tests. Coincidence?

> nor do I know of any technical basis for such a correlation.

I'm not drawing any correlation, though -- you need the 2nd half of
that sentence in order to retain the original intention.

> > and you only benchmark the features you've implemented.
>
> How could enormous (and very complicated) libraries like wxWindows,
> STL and STLsoft work with DMC++ if somehow only a carefully selected
> subset picked for fast compiling of the language is implemented?
> I can't even conceive how to build such a contrived implementation.

I'm not claiming it's intentional. Of course you've optimized the
features that you test, and as for the features that don't work, well,
you can't rightly claim your compiler is faster on those than any
compiler that *does* implement them.

> > If you'd like, I'll privately forward you the performance appendix of
> > http://www.boost-consulting.com/tmpbook, which contains some very
> > simple benchmarks and graphs showing performance for a few other
> > compilers. Maybe you can use that as a way to think about optimizing
> > template instantiation.
>
> Sure, I'd love to see it. But simple benchmarks imply they are of a small
> size.

Yes. They measure specific effects in the template machinery that
become significant in complex template metaprograms.

> Small size programs can be great for benchmarking optimizer
> performance, but typically are not representative benchmarks for
> compile speed

True. They only have some relevance to template instantiation
speed. However, some programs' compile times are indeed dominated by
template instantiations.

> because when compile speed matters is when you're trying to stuff
> 300,000 lines of header files down the compiler's maw.

No, compile speed matters when it takes more time that you're willing
to wait, for whatever reason. Saying it only matters when you have
lots of program text is circular reasoning. 

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Relevant Pages

  • Re: 7.0 CPU and Memory Performance
    ... Before doing this, I wanted to take some benchmarks to see how the scripts that I would run would fare between the two versions, and the results are somewhat confusing... ... Statically compile the 5.4 binary, and run the same binary on both ... Robert N M Watson ... Now, I really don't know exactly what the ubench program is doing, but I think the description says that it is doing random integer and floating point operations for the CPU tests, and random memory allocation and copying for the memory test. ...
    (freebsd-performance)
  • Re: 7.0 CPU and Memory Performance
    ... I wanted to take some benchmarks to see how the scripts ... Statically compile the 5.4 binary, and run the same binary on both ... And a rerun of the FreeBSD 7.0 ubench making sure there is absolutely no activity on the box ... The author's description of his benchmark doesn't inspire confidence: it does "rather senseless memory allocation and memory to memory copying operations for another 3 mins concurrently using several processes". ...
    (freebsd-performance)
  • Re: eForth 64bit trouble
    ... This 'token-re-interpreter' is now working, ... compile the benchmarks. ... mode / for the AMD X2 when writing data that is 'too close' to code. ...
    (comp.lang.forth)
  • Re: A Symbol Table Benchmark
    ... Benchmarks are generally how people say one ... > definitly not what makes an assembler fast, ... Nevertheless RosAsm 3x recent speed_up was because ... the various Assemblers compile a no end Labels List. ...
    (alt.lang.asm)
  • Re: export
    ... C++ compiler ever built." ... That's commonly the bottleneck in the parsing process of many ... can't yet compile a great deal of Boost, ... template instantiation. ...
    (comp.lang.cpp)