Re: writing at the end of the lines



Only one small comment I have to add.

Kamaraju Kusumanchi <kk288@xxxxxxxxxxx> wrote:

> Now due to the size of the arrays and the large number of time steps
> required etc., it is not an option to store all the data

I urge you to think a little more along this line to make sure that you
have accurately evaluated whether this is an option. Memory tends to be
larger than it used to be (by quite a lot), and virtual memory helps
too. I have seen several people spend a lot of time worrying about
amounts of memory that aren't worth the trouble. While one certainly
does want to think about resource requirements, one also needs to
have a realistic evaluation of teh cost of the resource versus, for
example, the cost of programmer time.

You mentioned what I interpreted as on the order of 1000 points of x. If
you have only about 1000 points of time, that would be only an array of
about a million elements. You could plausibly go quite a bit larger than
that without running into a lot of trouble on todays machines.

It may be that you are correct that it isn't an option for you, but I
just wanted to make sure that you hadn't dismissed the possibility too
rapidly, without thinking through the numbers - I've seen people do
that.

And if you can't store the whole file, another possibility is to block
the problem. Store as big a chunk as you reasonably can, which
would certainly be a lot more than one time frame, even if it is less
than all of them.

....
> When I write the output of this program on to a file the output looks
> something like this:
>
> time x_1 x_2 x_3 x_4 ..... x_m
> t_0
> t_1
> t_2
> .
> .

That's real typical organization for time series data.

--
Richard Maine | Good judgement comes from experience;
email: last name at domain . net | experience comes from bad judgement.
domain: summertriangle | -- Mark Twain
.



Relevant Pages

  • Re: Xml Data Storage
    ... If the number is quite large then you may want to cache portions in memory, ... If the number is small then just store the contents in memory. ... I need to use theese informations in my ping method. ... arrays to store IP Adresses and SiteNames.The arrays was created at runtime just when the ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Store large number .
    ... The suugestion to use arrays is flawed since large arrays (for example, ... arrays) do not handle memory allocation well. ... The advantage of the string class is that stores only one byte for each char ... > store large number. ...
    (alt.comp.lang.learn.c-cpp)
  • Re: Is this a memory problem?
    ... Alternatively, split large arrays ... into multiple smaller ones. ... sets, is to store the data on disk, and load manageable chunks into ... memory for processing. ...
    (comp.os.linux.misc)
  • Re: Exchange 2003 Private Store Corruption
    ... Once you've pulled the faulty RAM, create the new mailbox store and move ... then you can start working on the problem mailboxes. ... have DIMM's installed in pairs in this server? ... Online Diagnostic Utility says "Correctable Memory Threshold Exceeded" ...
    (microsoft.public.exchange.admin)
  • memory leak in (?)... (redux)
    ... the mem leaks in our long-running tcl daemons. ... when actively exercised the app grows continuously. ... the mystery is that while our audit code does show us a couple of arrays ... and memory are checked and meanwhile externally, ...
    (comp.lang.tcl)

Loading