Re: Vector problem? - How can I make the faster?

On Fri, 26 Oct 2007 22:49:00 -0700, Chun <chun.tse@xxxxxxxxx> wrote:

Hello,

I've written some code in C++ to decode a binary file. First I read
the contents of the file and then create an index to each record and
field in the file. Then I have method to dump to stdio or I used
populate TCL structures for use in tcl scripts.

When I run on windows then it decodes a large file faily quickly.
However, when I run on Sun solaris then it "stalls" every now before
finally completing. Someone said that this is to do with me using
vectors and the "stall" was the vector algorithm finding more memory
to increase it's size.

I've attached my code. Could someone comment on it and where it could
be improved? I'm particular interested in performance "tricks".

Thanks in advance
Michael

[snip code]

Two comments:

1 Do you really need to read the whole file into memory in one piece?
Can you rewrite your code to read just one record at a time?

current_record <- read next record from file
while (current_record != null)
index current record
index fields of current record
current_record <- read next record from file
endwhile

This only uses enough memory for a single record, not for the whole
file.


2 If you do have to read the entire file, then a C++ vector is
probably not your best choice. A vector guarantees that it is in
contiguous memory so whenever it needs to grab more memory, there is a
chance that the whole vector will have to be moved - very time
consuming.

You can avoid this by either a) allocating enough memory for the
vector initially so it never needs to grab more memory or b) using
another container, such as a deque, which can allocate separate
non-contiguous memory chunks when it needs to grow.

rossum

.

Relevant Pages

  • Re: 128 BIT 32x4 PROMS?
    ... A memory decode prom on ... is on CPU card) has blown and was running hot. ... A15 bank select on the 48k card but hit a race problem. ...
    (sci.electronics.repair)
  • Re: 128 BIT 32x4 PROMS?
    ... A memory decode prom on ... is on CPU card) has blown and was running hot. ... A15 bank select on the 48k card but hit a race problem. ...
    (sci.electronics.repair)
  • Re: Vector problem? - How can I make the faster?
    ... I've written some code in C++ to decode a binary file. ... vectors and the "stall" was the vector algorithm finding more memory ... there could be a problems with populatng the TCL structures. ... memory footprint; it sounds as though your program has a large ...
    (comp.programming)
  • 128 BIT 32x4 PROMS?
    ... Proms are Harris semiconductor but I've lost the orignal chip. ... A memory decode prom on ... is on CPU card) has blown and was running hot. ...
    (sci.electronics.repair)
  • Re: Large image
    ... to run the device out of memory long before it's fully in memory. ... Correct, that's why I said that he should write a "read and decode" method, ... essentially the equivalent of "new Bitmap ". ... But is it possible to load for instance the upper left 1000*1000 ...
    (microsoft.public.dotnet.framework.compactframework)