Re: resizing an array, is there a better way?

Jens.Toerring_at_physik.fu-berlin.de
Date: 08/05/04


Date: 4 Aug 2004 22:58:06 GMT

Keith Thompson <kst-u@mib.org> wrote:
> mwojcik@newsguy.com (Michael Wojcik) writes:
>> In article <cem5in$4pe$2@news6.svr.pol.co.uk>, "Malcolm"
>> <malcolm@55bank.freeserve.co.uk> writes:
>> >
>> > "Jack Klein" <jackklein@spamcop.net> wrote in message
>> > >
>> > > > However this is not really much improvement, because
>> > > > internally realloc() usually allocates a new area of memory
>> > > > and then copies.
>> > >
>> > > Maybe it does, maybe it does not. In any case, even if it does
>> > > require copying it is quite possible that it will use a mechanism more
>> > > efficient than the OP's for loop.
>> > >
>> > In practise realloc() will usually be obliged to perform a copy.
>>
>> That's a dubious assumption. There are allocation schemes which
>> over-allocate areas (typically rounding them up to a power of two);
>> realloc under such a scheme might well be able to simply extend the
>> "in use" portion of the block in many cases.
>>
>> In some implementations, allocated blocks might be scattered through
>> a large virtual memory space such that they can be trivially extended.
>>
>> In short, you *don't know* what realloc does.

> I just did an experiment. I wrote a small program that initially sets
> a pointer to the result of malloc(1), then repeatedly sets to the
> result of realloc with successively larger values (1, 2, 3, ...).

> When I iterate from 1 to 1000000, the pointer value only changes a few
> times for small values. On one implementation, the value changes only
> when realloc() is called with a size of 13; on another, it changes at
> 9, 17, 25, and 33. For all other sizes, all the way up to a megabyte,
> it just extends the existing region of memory; presumably no copying
> is done.

> If I add a call to malloc(1) before each realloc() (with no
> corresponding free()), the pointer value changes much more often on
> one implementation, but not on the other. Probably the small
> allocation is (at least sometimes) done just after the existing block,
> preventing it from being expanded; on the other implementation, it
> looks like the small allocation happens to be done just *before* the
> existing block, allowing it to expand.

> All this is, of course, extremely system-specific. Michael is quite
> correct: you *don't know* what realloc does.

System-specific or not, would you be prepared to tell which implemen-
tations you were testing that with?

Using your program on gcc (on Linux) exhibits the interesting property
that as the number of bytes you realloc() increases the less often it
needs to move the block. And when it gets slightly above 128 kB it does
not move the block anymore - that's probably due to, as I learned just
recently, its switching the method for memory allocation above 128 kB
(from sbrk() to mmap()).

> Here's the program. One caveat: the "ptr != prev" comparison probably
> invokes undefined behavior when realloc() moves the memory block and
> ptr becomes invalid.

As far as I understand the constraints, comparing pointers for equality
or inequality is absolutely ok, only trying to establish a relation like
that one is larger than the other gets you into muddy waters.

                                      Regards, Jens

-- 
  \   Jens Thoms Toerring  ___  Jens.Toerring@physik.fu-berlin.de
   \__________________________  http://www.toerring.de


Relevant Pages

  • Re: malloc
    ... If I use both the calls ie. malloc and realloc together, how the allocated memory will be ... Again, an old pointer value, valid for the first allocation of p, will ...
    (comp.lang.c)
  • Re: realloc, copy and VM
    ... > When you realloc for more size, it may be necessary to allocate a new block, ... > copy memory and deallocate the old, for example if there is not enough free ... That *is* extending the original block. ... via lazy allocation and copy-on-write.) ...
    (comp.lang.c)
  • Re: when can realloc fail?
    ... will the allocation of ptr be freed? ... about what the Standard requires/allows. ... memory pointed to by p, basing the claim on the deallocation ... Don't call realloc() with the second ...
    (comp.lang.c)
  • Re: appending elements to an ALLOCATABLE array
    ... > That's not a realloc. ... this) of addressing more than one pre-allocated memory locations using ... > As a proposed solution, I don't think what he asked for works. ... > For pointer allocation, you only need one copy today. ...
    (comp.lang.fortran)
  • Re: sizeof([ALLOCATED MEMORY])
    ... power-of-two sizes, so if you do many reallocs with a size of current+N, ... Power of two only allocation leads ... to a huge memory waste, since we would waste the half of the memory in ... number of calls to realloc(), not the amount of real work done by the user's ...
    (comp.lang.c)