Re: problem with memcpy and pointers/arrays confusion - again

Michael Mair wrote:
Martin Jørgensen schrieb:

-snip-

[Understood and agreed everything till now]

printf("Total memory occupation is: %li bytes",
(long) total_mem_used);


The same situation. This is uncritical on most modern
host machines but try to keep conversion specifiers and
passed argument types consistent.


Ok. size_t I guess...


Note: unsigned long may be more easy to use. If you want to
printf() size_t, you always have to cast to unsigned long in
C89, as there is no conversion specifier for size_t (in C99,
you have 'z', i.e. %zu will do).

Is size_t always unsigned long or can it on some system "just be" unsigned int too? Because my book is a little unclear whether I have to use %u or %lu on a system where %zu doesn't work?

I would prefer the unsigned long type, just to be sure it can handle large memory numbers too...

free(int_array);
free(double_array);

exit(0);
}


void *allocate_mem (size_t size, char *filename,
int line, unsigned long *total_mem)
{
int *void_ptr = malloc(size);


Calling a pointer to int a void_ptr is not exactly a good
idea. In addition, you want to be able to use the memory
allocation for arbitrary types.
Use a void *.


Oops, that was a mistake that came because I was considering how I solved the problem with making a function that could both be used for malloc'ing int * and double * data types (guess it also works for int **, double ** types and perhaps also with different float-types although I seldom use these). I started with two functions (int_ and double_ prefix) and forgot the remove this int *-thing from the code...


Happens often enough in real life, too.
It is often easier to write a function anew instead of trying to
adapt it.

Yeah, probably if you're as experienced as you I guess :-)

I still have to look at something (examples or old code), when it comes to dealing with casts + pointers + malloc + free-stuff :-)

if (void_ptr == NULL)
{
fprintf(stderr, "%s: line %d, malloc(%ld) failed.\n",
filename, line, (long) size);
exit(EXIT_FAILURE);
}

void_ptr[size/sizeof *void_ptr - 1] = size;


Here, you originally had [N]. This is a bad idea at best.
Even if you passed the number of elements separately,
you would here access the N+1st element of an array of
int -- which might not be at all the location of the
Nth element of the type of array you are allocating memory
for.


Why not? I thought malloc'ing space for N+1 made memory available so I could access element [N] like I normally could access anything between [0 < (N+1)].


Yes. Barry Schwarz already answered this one:
void_ptr[N] is effectively the same as
*( (int *) ( ( (char *) void_ptr) + N*sizeof *void_ptr) )

So one can write:

*( (int *) ( ( (char *) void_ptr) + N*sizeof *void_ptr) ) = size; ???

That's a long pointer address... But this explanation is nice to look at I think, although it has a lot of confusing parentheses... I had to fill in space, as you can see to understand it better...

This calculation goes wrong as soon as you are not working with
ints but with doubles. If sizeof (double) > sizeof (int) which is

I see that...

very likely, then you are accessing someplace in the middle of
the array and store the size representation there. Even worse,
if you allocate for char with sizeof (char) < sizeof (int), you
are accessing a place way beyond your allocated memory.

Let me make sure I understood this. Lets say N=3: 3 * sizeof (char) = 3 bytes. And sizeof(int) is 4 bytes, AFAIR, right?

So, looking at:

*( (int *) ( ( (char *) void_ptr) + N*sizeof *void_ptr) ) = size;

The problem would be that *( (int *) (.... + 3 bytes) ) would be casted to.... Hmm... would that be 0 or 1 due to the (int *)? My guess is that would be the first element of void_ptr because it's probably like dividing the offset 3 bytes by sizeof(int) = 4 bytes, and throwing the remainder away?

So the problem is that the 3 first bytes (first N=3 char elements) would be overwritten? Did I understand it? :-)

Note: What I did above is nothing better -- it only is a replacement
for N.

Hm.... I can see there is a problem... I'll have to look at your proposal.

This is the reason why you need the element size and number,
so you can calculate the correct address where you can store
the size.

Ok, I see.

/* save memory allocated for this pointer */
*total_mem += size; /* update total memory allocated untill now */

return void_ptr;
}
- - - - - - - - - - - - - - - - - - - -

I changed a little from my post yesterday because now I just made the return pointer of allocate_mem() void * so I hope it can handle double * as well as int * pointers at the same time... Not sure although...


No, it cannot.
You _could_ try to salvage the idea with something like

void *alloc_mem (size_t num_elems, size_t elem_size,
char *filename, int line,
size_t *total_mem)
{
void *mem;
size_t size = num_elems*elem_size;
size += (sizeof (size_t) <= elem_size) ? elem_size
: sizeof (size_t);


Hmmm. A stupid question, but isn't there something wrong there? sizeof (size_t) less than or equal to elem_size gives size += elem_size if true and sizeof (size_t) else.

According to my C-book, size_t is just a data type and not a specific number. My book says that size_t is an unsigned integer type and: "Instead, like the portable types (in32_t and so on), it is defined in terms of the standard types".


Yes. I allocate enough memory to store a size_t in the "element"
beyond the last "official" element of the array.

Ok, having looked at the code for very long I think it looks okay...

Obviously, we need at least size_t bytes in case the element size
of the allocated storage is less than sizeof (size_t) -- otherwise
we would store our size outside the allocated storage.

Yep.

To be on the safe side, I also allocated enough storage for at
least one more element, so that one cannot access storage that does
not belong to the allocated storage via the "last+1st" array element.
This is not strictly necessary but wastes only a few bytes.

Where did you do that?

From my point of view it looks like the allocated space is exactly what is needed... I probably overlooked that...

mem = malloc(size);

if (!mem)


It that the same as testing for if (mem == NULL) ?


Yes, sorry for obscurity.

That's okay. Nice to learn something new.

{
fprintf(stderr, "%s: line %d, malloc(%lu) failed.\n",
filename, line, (unsigned long) size);
exit(EXIT_FAILURE);
}

/* save memory allocated for this pointer */
memcpy(((char *)mem)+num_elems*elem_size,
&size, sizeof size);


Ok, on second thought, perhaps this is connected to the size += (sizeof (size_t) <= elem_size) ? elem_size...-stuff, since you're copying the address of &size to location ((char *)mem)+num_elems*elem_size ?


Yes. It is perfectly possible that the element size is smaller than
the alignment required to store a size_t. The only safe way to store
and retrieve something not a char type[*] to or from an arbitrary
address is via bytewise copy. This is done via memcpy() or memmove().
This is the reason why we need at least sizeof size == sizeof(size_t)
extra bytes.

I think I understand that now... That is really great coding... I would never have figured that out myself, but it is actually very logical now I'm reading your explanation...

And I feel I'm taking a giant leap towards learning "pointer acrobatics", by watching that code :-)

In order to "ward" against the own stupidity (accessing array element
"N+1" even if you are not supposed to -- for this there is an access
function!} I made sure that we also have enough memory for a full
array element.

And here you're talking about the "retrieve_memsize"-function? That is nice... Probably saved me for some trouble :-)

[*] i.e. signed char, unsigned char, or char

*total_mem += size; /* update total memory allocated untill now */

return mem;
}

size_t retrieve_memsize (void *mem, size_t num_elems,
size_t elem_size)
{
size_t size = 0;
if (mem) {
memcpy(&size, ((char *)mem)+num_elems*elem_size,


And (char *)mem does cast the mem-pointer to pointer to char???


Yes. You cannot perform pointer arithmetics on void *. As we really
have to calculate the byte number via the product as the element
sizes may vary, converting to char * is natural.
If we used another pointer type, then we could not necessarily access
each address and accessing the address becomes more difficult due
to pointer arithmetics.

That is because you know that char is exactly 1 byte, right (is it that way on all systems???) ?

That was exactly what I needed to understand this, I think...

You could also write
&(((char*)mem)[num_elems*elem_size])
where I inserted gratuitous parentheses for clarity.

So you're saying that instead of:

( (char *)mem ) + num_elems*elem_size

I could write:

&( ( (char*)mem ) [num_elems*elem_size] )

Then I can see you're converting mem-pointer to "pointer-to-char" and in both cases you're adding what corresponds to element number [N]. I think I understand that too now...

In C99, you could cast mem to an appropriate variable length array
pointer type ((*char)[elem_size]) and get the address of its
num_elems element.

Not understood this part about variable length array, but it's luckily not that important I guess. You would still need something with: [num_elems*elem_size] right?

sizeof size);
}
return size;
}

<snip!>

A better approach, if you need the size, is making the
information explicit:

struct attributed_mem {
size_t size;
void *mem;
};


That was a good idea.

This means you only have to pass and recieve
struct attributed_mem * arguments.

I'll still have to get back to this later...

But why don't take this struct-thing a step further like here:

struct attributed_mem {
void *mem;
size_t num_elems,
size_t elem_size
size_t size;

};

?

Then in the end of the week, where I can play with this again, I could change the program so one of prototypes would become:

size_t retrieve_memsize(attributed_mem attributes);
{
(code that gets the properties from struct type attributed_mem, variable: attributes and does whatever)
}

-snip-

If you have questions, you are of course welcome to ask.

Thanks, I just have these comments in this post...



Best regards / Med venlig hilsen
Martin Jørgensen

--
---------------------------------------------------------------------------
Home of Martin Jørgensen - http://www.martinjoergensen.dk
.

Relevant Pages

  • SSPI Kerberos for delegation
    ... const char *tokenSource, const char *name = NULL, ... DWORD bufsiz = sizeof buf; ... int n = ib.cbBuffer; ... // wserr() displays winsock errors and aborts. ...
    (microsoft.public.dotnet.framework.remoting)
  • Re: Memory Structure Pointer Problems
    ... typedef struct sta { ... char* name; ... int num_cmpnds; ... A pointer to a struct cmp is almost ...
    (comp.lang.c)
  • Re: Insufficient guarantees for null pointers?
    ... will the compiler know what the bounds are after converting that char * ... to an int *, if it could point to either of two arrays which happen to ... compares equal to the original pointer. ...
    (comp.std.c)
  • Re: Request critique of first program
    ... Returns a pointer to an asplit_result struct (unless unable to ... Success is indicated by SUCCESS, ... const char *out_file_b, ... const long int num_lines); ...
    (comp.lang.c)
  • Re: OT: Requesting C advice
    ... int and long usually only have two distinct sizes. ... Note that what you suggest works because sizeof(.) for integer ... that char be at least 32 bits. ... Cray compiler that had 64-bit chars. ...
    (Fedora)