Re: Next Generation of Language

George Neuner wrote:


I have considerable experience with manual staging (on DSPs) and I can
tell you that it is a royal PITA to schedule several functional units
and keep them going full blast using software alone.

I bet it is!



Non Uniform Memory Access simply means different memories have
different access times - that describes just about every machine made
today. The NUMA model distinguishes between "near" and "far" memories
in terms of access time, but does not distinguish by how the memories
are connected - a system with fast cache and slower main memory fits
the model just as well as one with a butterfly network between CPU and
memory.

I agree with this in theory (and of course, all (well, nearly, there
have been recent cacheless designs which aimed to hide latency by
heavily multithreaded processors) machines are NUMA in that sense. But
I think the conventional use for the term is for multiprocessors where
all memory is "more local" (in time terms) to some processors than it
is to others, and that was the sense in which I was using it. You can
think of these kinds of machines as systems where there is only cache
memory. It seems to me inevitable that all large machines will become
NUMA, if they are not all already. And the nonuniformity will increase
over time.

My argument is that physically, these machines actually are distributed
memory systems, but their programming model is that of a shared memory
system. And this illusion is maintained by a combination of hardware
(route requests to non-local memory over the interconnect, deal with
cache-coherency etc) and system-level software (arrange life so that
memory is local to the threads which are using it where that is
possible etc).

Of course these machines typically are not MPP systems, and are also
typically not HPC-oriented. Though I think SGI made NUMA systems with
really quite large numbers of processors, and a Sun E25K can have 144
cores (72 2-core processors), though I think it would be quite unusual
to run a configuration like that as a single domain.

--tim

.