Re: Storing the size of an array in the structure itself

From: Wynand Winterbach (wynand_at_realtimerodeo.net)
Date: 07/21/04 

Date: 20 Jul 2004 16:16:07 -0700

"Mike Wahler" <mkwahler@mkwahler.net> wrote in message news:<dRFGc.7333$oD3.6129@newsread1.news.pas.earthlink.net>...
> "Wynand Winterbach" <wynand@realtimerodeo.net> wrote in message
> news:7b396160.0407061256.44414026@posting.google.com...
> > I think every C programmer can relate to the frustrations that malloc
> > allocated arrays bring.
>
> I don't find them frustrating at all.
>
>
> > In particular, I've always found the fact that
> > the size of an array must be stored separately to be a nightmare.
>
> Why? Why is it any more 'frustrating' than keeping track
> of any other piece of information in a program?

Anything that decouples information in this way makes the life
of the programmer harder, and the code harder to read! Array size
is an integral part of the concept of an array. Would you have
an employee struct, only to store an employee's birth date separately?
I doubt it.

Also note
> that memory obtained with 'malloc()' isn't inherently an 'array',
> it's just a 'chunk' of memory, which you might or might not use
> to store an array.

All I'm going to say here is DUH. Geesh, I cannot think of a single
mammal capable of C programming that would assume a malloc chunk
is anything more than just that - a chunk of memory. I never even
once intimated to any such suggestion.

>
> > There are of course many solutions, but they all end up forcing you to
> > abandon the array syntax in favour of macros or functions.
>
> Not at all. The 'best' solution imo is to simply save the
> size you allocate. And again, allocated memory isn't required
> to be used as an array.
>
>
> > Now I have two questions - one is historical, and the other practical.
> >
> > 1.) Surely malloc (and friends) must store the size allocated for a
> > particular memory allocation, if malloc is to know how much to
> > deallocate when a free() occurs?
>
> An implementation of 'malloc()' must of course keep 'housekeeping'
> information. But each implementation is free to implement 'malloc()'
> with whatever method is most appropriate for the target platform.
> The language standard only dictates the *behavior* of 'malloc()',
> not how it is to be implemented.
>
>
> > Thus, why was the C library designed
> > in such a fashion as not to make this information available?
>
> Think about it. WHen you call 'malloc()', you *have* this information.
> Otherwise you couldn't tell it how much to allocate.

Yes, but again you're missing the point. Of course you have it! But
this is not my principle problem.

> Also, if you're willing to go nonstandard and platform-specific,
> many implementations do provide a function to give the information
> you're after. Check your documentation.
>
> >Or I am
> > seriously missing something here?.
>
> I think you're just being lazy. :-)

There is no virtue in the calvinistic discipline wrought onto the
programmer who has to keep track of so many details of the language
at the expense of the problem s/he is solving.

This is why Python is so great - the ultimate language for the lazy
programmer. You also happen to produce pretty solid code with it,
due to the concentration on the problem at hand. And when you need
speed, use SWIG and C to build a module.

> >
> > 2.) Why not store the size of the array in its first four bytes (or
> > first sizeof( size_t ) bytes ), and then shift the pointer to the
> > array on by four bytes? Thus one has:
> >
> > first 4 bytes everything else
> > [ size ][ data ]
> > /\
> > void * blah ---'
>
> This might indeed be the way it is done for some implementations,
> but it's not required. Perhaps for a given architecture it's
> simply not possible or too inefficient.
>
> > Then it should behave as a "normal" array,
>
> IMO you need to stop automatically thinking of allocated memory as
> an 'array'. It's simply allocated memory, to be used as desired.
>
>
> >with the added advantage of
> > knowing its size.
>
> You allocated it, you already know its size. Also note that
> the requirement for 'malloc()' is that it allocate *at least*
> the number of requested bytes, but it's allowed to allocate more
> (would typically be done in the interest of meeting the target
> platform's alignment requirements and/or of efficiency).
>
> > The reason I have doubts here, is that if this was
> > such a good idea, I'm sure it would already have been widely used.
>
> It would unnecessarily restrict implementors and possibly which
> platforms the C standard library could be implemented for.
>
>
> >Any
> > compelling reason for avoiding this? This is a bit of hackery, but the
> > hackery will be confined to the functions for allocating, resizing and
> > checking the size of the array.
>
> Right, it's 'hackery'. Keep It Simple. Just Remember The Size.
> (Pass it to any functions that need it).
>
>
> > The code could work as follows:
> >
> > void*
> > malloc_array( size_t element_size, size_t items )
> > {
> > size_t sz = element_size * items;
> > void* result = malloc( sz + sizeof( size_t ) ); /* allocate memory
> > for array and for size chunk */
> > *((size_t*) result) = items; /* assign the size to
> > the first few bytes */
> > return sz + sizeof( size_t ); /* return a pointer
> > to the array pointing just beyond size chunk */
> > }
> >
> > size_t
> > sizeof_array( void *array )
> > {
> > return *( (size_t*) (array - sizeof( size_t )) );
> > }
>
> If you want to go to all that trouble, be my guest. But I wouldn't
> bother.
>
>
> > This technique of course could also be used to store the byte size of
> > the elements in the array.
>
> But the memory allocated by 'malloc()' needn't necessarily be
> used as an array.
>
>
> >Oh yes, and in order to detect whether the
> > size value was corrupted by accidentally writing over it, one could
> > use a magic number (which would again be added by the same technique),
> > which could be consulted with debugging code.
>
> Perhaps some implementations do this. But again, they're not
> required to.
>
> -Mike

Relevant Pages

  • Re: Another C# marshaling question
    ... // Get the pointer of the location in memory. ... > public static extern int FF_Function( ... > // Allocate the array. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: INTENT(WORKSPACE) as a new Fortran feature?
    ... >> the array once in the caller than each time within the subroutine. ... > the stack pointer is being changed to allocate local variables in any ... "actual" memory would never be touched at all. ... If, on a particular machine, the compiler knows that it would be ...
    (comp.lang.fortran)
  • Re: Creating a logical matrix in a mex-file
    ... Then use mxMalloc to allocate the memory ... Use mxSetData to set the data of the array to the ... This would avoid the initialization of the memory. ...
    (comp.soft-sys.matlab)
  • Re: The problems arrayed before me...
    ... > When an array is grown, how does Delphi manage this cleanly? ... default memory manager, the first thing it tries to do, when it receives ... When you ask for a certain amount of memory, the memory manager is free ... to allocate more than you ask for, and it keeps track of that. ...
    (alt.comp.lang.borland-delphi)
  • Re: Cons cell archaic!?
    ... from s-expression or XML or other syntax you keep the bloated array ... For using vectors to emulate lists that ... Allocate 2, move 1 element: ... What do you think of that algorithm? ...
    (comp.lang.lisp)