Re: Large RAM sizes in embedded systems
- From: David Lundquist <nobody@xxxxxxxxxxx>
- Date: Tue, 26 Dec 2006 21:12:58 -0500
Tim Wescott wrote:
larwe wrote:Have you proven you can't get the performance you need from a hard drive based system? I would think with the proper indexing (I hope you don't expect to sort though a terabyte looking for you data, even with high speed DDR2 memory that is going to take a while!) that you could retrieve even a megabyte or two 50 or 100mS? Probably less. The key here of course is to know where the data you need is located. You may need some sort of home grown file system tweaked to your requirements. Keep the "file allocation table" in local RAM. Just how fast does it need to be?
I'd be curious to hear from anyone who has worked on embedded systemsPerhaps the right approach is to _buy_ as many PC-architecture boards as you need to hold that much memory, each with appropriate RTOS support (Linux with RTAI?), and rope them together on a bus?
with relatively large amounts of directly-addressable RAM.
I'm discussing with a colleague/client an application that would work
on a nominally 740GB data set. The nature of the data and the required
processing is such that it's a "must-have" performance improvement to
hold the entire data set in RAM rather than swapping in from some
secondary storage mechanism. It's perfectly acceptable for the machine
to take an entire week to cold-boot. It's not acceptable, once booted,
for it to wait several seconds to page in data from a hard disk :)
Hence I would say my RAM requirement would be speced at 1TB of
error-correcting RAM. The hardware interfaces I would require are
gigabit Ethernet, SATA for the boot media, and a means for connecting
to an ASIC that does all the real processing work. The interface for
that latter is not yet defined, but would quite likely be PCI Express.
All this suggests a PC-style architecture as the way to go.
I'm really not finding much (read: anything) in the way of monolithic
computer modules that can address 1TB. I've found mention of server
clusters that have that much in aggregate, but it's spread across
several computers. OS support for such large RAM sizes also appears to
be problematic, but I could work around this.
Is anyone else dealing with similar problems? This is strictly a
theoretical investigation for me now - more of a feasibility review
than anything else - but it's quite an intriguing project. Maybe the
right approach is to build a massively parallel engine with identical
modules handling manageable (8GB?) slices of the data set. However this
would be very expensive in terms of power and additional support
circuitry.
I don't know if the best way to realize this would be to use your GB ethernet, or to try to get them talking nicely on one Compact-PCI bus, or VME, or PC-104, or something stranger.
I do think that if you have the space (and power supplies) this may be the least-engineering way to develop the hardware. It may even be an effective way to prototype a system that you can come back to later and replace all the SBC's with something that's been stripped down to the bare essentials necessary to boot up and host your gargantuan blocks of memory.
Dave
.
- References:
- Large RAM sizes in embedded systems
- From: larwe
- Re: Large RAM sizes in embedded systems
- From: Tim Wescott
- Large RAM sizes in embedded systems
- Prev by Date: Re: Large RAM sizes in embedded systems
- Next by Date: Re: Large RAM sizes in embedded systems
- Previous by thread: Re: Large RAM sizes in embedded systems
- Next by thread: Re: Large RAM sizes in embedded systems
- Index(es):
Relevant Pages
|
