Re: Buffer or Realloc?

Michael Mair wrote:
Flash Gordon schrieb:
websnarf@xxxxxxxxx wrote:
bwaichu@xxxxxxxxx wrote:

Is it generally better to set-up a buffer (fixed sized array) and read
and write to that
buffer even if it is larger than what is being written to it? Or is it
better to allocate memory and realloc it for the size of the what is
being written each time? In other words, what is the decision factor
between deciding to use a fixed size buffer or allocating memory
space and reallocing the size?

Fixed size arrays are good for things that whose contents *CANNOT*
exceed the size of that array. A string usually is never that kind of
thing.

So usually the decision is quite simple: if you are dealing with
mutable strings *NEVER* use a fixed size array (this is a definitive
mark of an amateur and is usually accompanied by either buffer overflow
errors or ruthless truncation),

OK, so for the string I've got to prepare as part of a message to the UK Government gateway where the specification says the string has a maximum length of 10 characters I should not use a fixed size buffer but a reallocating buffer? Presumable so that I don't immediately allocate the maximum size I should start at 1 byte and increase the size 1 byte at a time so avoid wasting memory?

You should not say to *NEVER* do something as invariably someone can come up with a *real* example where that rule does not apply, such as the one I just mentioned. I've got plenty of other interfaces where I am either preparing a string (and yes, I do want a C string for simplicity of manipulation such as using sprintf to create it) which has either a *fixed* final length or a small but definite upper bound.

In such cases, there often is some sort of inbetween:
Think of a piece of code that generates C identifiers which can be
restricted to a certain length, say 6 or 31 characters.
I'd generate them full-length, with the realloc() approach -- obviously
starting with a sufficiently large initial buffer -- and then shorten
them by a well-defined algorithm to the required length while retaining
uniqueness within a set of "visible" identifiers.
It is even possible that the final, _immutable_ version is copied to
a fixed-length array.

Since malloced blocks have a certain amount of overhead in terms of <OT>heap management structures</OT> and overhead in terms of calling malloc/free plus the extra work (however small) of ensuring you don't leak memory and handle malloc failures I find it far better to use a fixed length buffer when there is something putting a definite and not unreasonable limit on the size. Also your suggestion can still lead to memory fragmentation.

BTW: The "1 byte" remark IMO is insulting the intelligence of the
involved participants and was rather unnecessary -- even though
"*NEVER*" and "*ALWAYS*" make my teeth ache...

Possibly I should have started at two bytes so you could fit more than an empty string ;-)

> but instead use the reallocing buffer

strategy. There is a common tendency in C programmers to conflate char
arrays and strings -- the two are different things conceptually and
this has implications for what you are actually doing.

True. Life would be much easier if C had a true string type that automatically resized, but it doesn't, and when you know a reasonable and definite upper bound on size a fixed length array is perfectly adequate to the job.

Which is different how from what Paul wrote?
If there is a definite upper bound, you typically do not do
"anything wild" with the arrays/strings.

In part I was agreeing with Paul (it can happen), but also pointing out that he was ignoring the fact that in many situations there are definite and reasonably small upper bounds, in which case a fixed sized buffer is the easiest thing to get right.

<snip>
malloc, calloc and realloc are fairly slow function calls. So one
thing you should generally do is try to minimize the number of times
you call such functions. When reusing a buffer like you are doing
above, you only want to realloc more space if you need it.

In addition you can hit memory fragmentation issues.

Indeed. A "reasonably" large starting buffer with the option for
reallocation often is one of the better ways to go. Determining
"reasonable" of course may be a hard problem subject to changes
throughout the programme's lifetime :-)

This depends very much on the likely patter of memory usage and the lifetime of the program but is definitely the right way to go in some situations.
--
Flash Gordon
.

Relevant Pages

  • Re: Discovering variable types...
    ... >- but I suppose MS expect us to use wrappers ... memory allocations for your variables from disk as well. ... >They most certainly are of fixed size, changing the size of a String ... >>me to keep buffer size and current postion right in the memory block. ...
    (comp.lang.pascal.delphi.misc)
  • Re: Is this string input function safe?
    ... return a pointer to mallocated memory holding one input string, ... complains about use of deallocated pointers, ... mallocating an appropriate amount of memory. ... the contents of the buffer are indeterminate (for different ...
    (comp.lang.c)
  • Re: strange behaviour
    ... Pointer to the buffer to receive the null-terminated string containing ... if the Windows directory is named Windows ... string copied to the buffer, not including the terminating null character. ... You must supply memory for windows to put the directory into. ...
    (comp.lang.pascal.delphi.misc)
  • Re: Something wrong in my program
    ... what becomes of the memory block starting at this address is no ... our text buffer can contain 15 characters ... a string is a char array *terminated ...
    (comp.lang.c)
  • Re: strtok and strtok_r
    ... Other Posix utility functions are ... Why do you cast the result of realloc? ... Your function invokes undefined behaviour when running out of memory, ... first part of the string is not copied properly. ...
    (comp.lang.c)