Re: Buffer or Realloc?
- From: Eric Sosman <Eric.Sosman@xxxxxxx>
- Date: Mon, 28 Aug 2006 10:52:04 -0400
bwaichu@xxxxxxxxx wrote On 08/27/06 18:37,:
Is it generally better to set-up a buffer (fixed sized array) and read
and write to that
buffer even if it is larger than what is being written to it? Or is it
better to allocate memory and realloc it for the size of the what is
being written each time? In other words, what is the decision factor
between deciding to use a fixed size buffer or allocating memory
space and reallocing the size?
There's no hard-and-fast rule, just as there's no
hard-and-fast rule for choosing between arrays and trees.
Each has its virtues and its drawbacks.
Some considerations for allocating "buffers," where
the somewhat vague term is understood to mean "places to
store varying amounts of stuff:"
- If you know an upper bound on the size of the stuff
(or if you're willing to report failure if the stuff
is larger), a fixed-size buffer is attractive because
of its simplicity.
- If the upper bound is large, it may be a good idea to
avoid declaring the buffer as an `auto' variable.
Many systems have less `auto' memory than dynamic or
static memory.
- A statically-allocated buffer may well be the ultimate
in simplicity, but it has some disadvantages. First,
it exists and occupies memory even when it's not being
used; for a large buffer this may be too wasteful.
Second (like any static object), it creates difficulties
if you want to write functions that are re-entrant or
can be called recursively.
- How many instances of "stuff" must the program keep track
of simultaneously? If there will be many of them, it is
often a good idea to put each in a dynamic allocation of
the minimum size, possibly by using realloc() on a buffer
obtained from malloc(), or possibly by copying from the
initial buffer to a new buffer specially allocated for
each instance.
- If an instance of "stuff" can be very large, consider
breaking it up into smaller pieces and storing those
pieces in another data structure. It may be easier to
manage a linked list of a thousand 1-megabyte chunks
than to find storage for a gigabyte array.
- If you decide to malloc() a buffer and then expand it as
needed with realloc(), it is usually better to expand by
a "magnification factor" than by a fixed amount. That is,
it is usually better to increase the buffer size by 50%
or 100% than to add 100 or 1000 bytes at every expansion.
- A popular technique used by many malloc() implementations
(but not by all) is to add about 8 bytes of bookkeeping
data to each allocation, and to round the total up to a
multiple of 8. (Or perhaps the magic number is 4 bytes,
or maybe 16 -- it depends.) Consequence: If you make a
large number of small allocations, a sizeable fraction of
the total memory will be expended in "overhead" rather
than in "payload." Corollary: If you've got a 200-byte
buffer that holds only 199 bytes of "stuff," shrinking
it with realloc() probably won't do much.
- Some malloc() implementations add their 8 bytes (or so)
of overhead but then round up to the next power of two.
If you request 1024 bytes, you'll wind up reserving 2048
for an "efficiency" of only 50%. To guard against this
effect, avoid power-of-two requests; if it makes sense
in terms of the rest of the program, aim for allocations
that are (nominally) a little less than a power of two
in size.
--
Eric.Sosman@xxxxxxx
.
- References:
- Buffer or Realloc?
- From: bwaichu@xxxxxxxxx
- Buffer or Realloc?
- Prev by Date: Re: int main()?
- Next by Date: Re: int main()?
- Previous by thread: Re: Buffer or Realloc?
- Next by thread: Re: Buffer or Realloc?
- Index(es):
Relevant Pages
|
