Large RAM sizes in embedded systems

I'd be curious to hear from anyone who has worked on embedded systems
with relatively large amounts of directly-addressable RAM.

I'm discussing with a colleague/client an application that would work
on a nominally 740GB data set. The nature of the data and the required
processing is such that it's a "must-have" performance improvement to
hold the entire data set in RAM rather than swapping in from some
secondary storage mechanism. It's perfectly acceptable for the machine
to take an entire week to cold-boot. It's not acceptable, once booted,
for it to wait several seconds to page in data from a hard disk :)

Hence I would say my RAM requirement would be speced at 1TB of
error-correcting RAM. The hardware interfaces I would require are
gigabit Ethernet, SATA for the boot media, and a means for connecting
to an ASIC that does all the real processing work. The interface for
that latter is not yet defined, but would quite likely be PCI Express.
All this suggests a PC-style architecture as the way to go.

I'm really not finding much (read: anything) in the way of monolithic
computer modules that can address 1TB. I've found mention of server
clusters that have that much in aggregate, but it's spread across
several computers. OS support for such large RAM sizes also appears to
be problematic, but I could work around this.

Is anyone else dealing with similar problems? This is strictly a
theoretical investigation for me now - more of a feasibility review
than anything else - but it's quite an intriguing project. Maybe the
right approach is to build a massively parallel engine with identical
modules handling manageable (8GB?) slices of the data set. However this
would be very expensive in terms of power and additional support
circuitry.

.

Relevant Pages

  • Re: Large RAM sizes in embedded systems
    ... with relatively large amounts of directly-addressable RAM. ... on a nominally 740GB data set. ... OS support for such large RAM sizes also appears to ... Wescott Design Services ...
    (comp.arch.embedded)
  • Re: Large RAM sizes in embedded systems
    ... embedded solution to a where a network cluster would be more ... with relatively large amounts of directly-addressable RAM. ... on a nominally 740GB data set. ...
    (comp.arch.embedded)
  • Re: Faster Marshaling?
    ... and 2 seconds to load a 9MB sized Marshal ... there's a way around this without extensive modification to ruby. ... loaded into RAM. ... entire data set in RAM all at once, because any one final data table ...
    (comp.lang.ruby)
  • Re: Large RAM sizes in embedded systems
    ... with relatively large amounts of directly-addressable RAM. ... on a nominally 740GB data set. ... OS support for such large RAM sizes also appears to ... It may even be an effective way to prototype a system that you can come back to later and replace all the SBC's with something that's been stripped down to the bare essentials necessary to boot up and host your gargantuan blocks of memory. ...
    (comp.arch.embedded)
  • Re: Large RAM sizes in embedded systems
    ... with relatively large amounts of directly-addressable RAM. ... on a nominally 740GB data set. ... The hardware interfaces I would require are ... computer modules that can address 1TB. ...
    (comp.arch.embedded)