Re: Parallelization on Dell Workstations

Sebastian Hanigk wrote:
Tim Prince <timothyprince@xxxxxxxxxxxxx> writes:

There is a lot of argument about whether hybrid MPI with OpenMP or MPI
alone is ideal for such machines. The optimum, again, depends on
factors which you haven't divulged.

Implicit hybrid programming has been useful on Europe's currently
largest supercomputer, the Mare Nostrum, because the node interconnect
consists of only one Myrinet interface - which caused bandwidth problems
in our code if four MPI processes were to run on each node. The usage of
the multithreaded LAPACK-library with only one MPI-process per node was
in that case faster (which makes it quite obvious that sheer FLOPS count
is worthless without good interconnects).
If the processes on a single node send messages among themselves on
shared memory, and don't involve more messages to other nodes than in a
hybrid model, there is nothing to be gained in savings on communication.
A random mapping of processes to nodes, evidently, will saturate the
intra-node connection quickly.
We face similar issues in attempting to get good performance with 8-core
nodes, when performance per core is not as high as in 4-core nodes,
additional message traffic among nodes will limit cluster performance at
a smaller number of nodes.
.