Re: Advice needed for memory intensive app

From: Gerry Quinn (gerryq_at_indigo.ie)
Date: 11/24/03 

Date: Mon, 24 Nov 2003 16:23:42 GMT

In article <Xns943CE2FBDC8B0newspubwuggyorg@217.32.252.50>, Ian Woods <newspub2@wuggyNOCAPS.org> wrote:
>"BCC" <a@b.c> wrote in
 
>> I have an application in c++ which is basically a glorified cellular
>> automaton. The ca is a 3D array of objects, and we are trying to
>> figure out a way to lighten things up because we are running out of
>> memory.
>>
>> I pulled out everything that I could that did not actually store a
>> value and placed it in a 'helper' class for both CCellBase and CValue,
>> but this doesn't seem to be enough.
>>
>> For example, if we want a 100x100x100 matrix, this is 1,000,000 Units.
>> If we want to have everything initialized at some point, all the
>> classes combined will sum up to several million objects.
>>
>> I have studied the flyweight design pattern and concluded that it does
>> not apply to this model- my units only contain non-duplicate
>> information at this point.
>>
>
>The lower bound (without using things like compression) is simply
>answerable:
>
>a) take the number of things you need to store
>b) find out how big each of them is...
>c) add them all together
>
>No matter what you do you'll have to store that much data /somewhere/. If
>that has to be main memory, and main memory isn't big enough, you're
>stuck. Well, not quite stuck - you now have to deal with off-line data.

Like he says.

When you say the data is non-duplicated, do you mean that it is
*different* for each cell? If the cells are mostly empty, you could
just store the filled ones.

Other than that, the obvious things to do are:

1. Buy more memory

2. Design it so that you can page efficiently to and from disk. For a
3D cellular automaton, chopping it into cuboids might be a good idea.

Gerry Quinn 

-- 
http://bindweed.com 
Screensavers and Games for Windows
Download free trial versions
New arcade-puzzler just out - "Volcano"

Relevant Pages

  • Re: why is there 256 columns on excel?
    ... It was based on memory bytes: ... 16 binary bits, or two bytes, can store the integers 0-65535 (i.e,. ... each cell address had a representation that requires ... Prev by Date: ...
    (microsoft.public.excel.misc)
  • Re: Player memory, visibility limitations and fog of war - how do YOU do it?
    ... Even if your levels are 100x100 and you store 100 bytes per cell, ... Even if there are 100 levels, it's just 100Mb of memory. ...
    (rec.games.roguelike.development)
  • Re: Exchange 2003 Private Store Corruption
    ... Once you've pulled the faulty RAM, create the new mailbox store and move ... then you can start working on the problem mailboxes. ... have DIMM's installed in pairs in this server? ... Online Diagnostic Utility says "Correctable Memory Threshold Exceeded" ...
    (microsoft.public.exchange.admin)
  • Re: Can operant conditioning account for all learning?
    ... are not replaying the raw sensory data form that event. ... problems your high level abstractions didn't recognize. ... I would suggest it uses the same amount of memory more ... Most likely the brain doesn't store things in ...
    (comp.ai.philosophy)
  • mm/ hacks -- Generic memory store?
    ... I want to do a memory management hack for generic memory stores. ... "store" for excess memory before OOM. ... RAM swap ... store would be capable of providing compression or encryption. ...
    (Linux-Kernel)