Re: forall and do loop

highegg wrote:

For OpenMP, I just code up a "normal" (serial) program, barely
considering the future parallelism, debug, then throw directives here
and there, and it normally works on the first try.
Running on 2 CPUs usually gives 2x speed-up, running on 8 CPUs only
rarely approaches 8x speed-up, but it scales at least to some extent
with very few extra work.

I've used both... while I think that OpenMP is simpler, especially if
you already have a working serial code, I would expect MPI to give
better performance. Put simply, OpenMP relies more on a smart compiler,
while MPI relies more on the programmer thinking very carefully about
what they want to do.

I think that the above comment illustrates the power of OpenMP - take a
working serial code, put in a few directives, and you have a parallel
code while will scale well to four processors or so (my personal record
for an OpenMP parallelisation was about ten seconds, and the code took
longer to recompile - it even used a FORALL :-) However, that was a very
simple program!).

With MPI, you have to think a lot more in advance about how the data
will be divided up. But because humans are (often ;-) ) smarter than
compilers, this will ultimately lead to better performance. For example,
in MPI, there's certainly local and remote data, akin to the global and
thread data of OpenMP programs. However, because accessing the remote
data is so expensive, you think a lot about how to avoid doing so, in a
way you don't have to with OpenMP. Ultimately, this will lead to a
faster prorgam. Plus, getting thousands of processors into a shared
memory machine tends to involve throwing large amounts of money in the
direction of SGI/Cray/etc., moreso than a stack of blade servers.

Richard
.

Quantcast