Re: malloc under linux

Keith Thompson wrote:

santosh <santosh.k83@xxxxxxxxx> writes:
Keith Thompson wrote:
santosh <santosh.k83@xxxxxxxxx> writes:
jacob navia wrote:
[...]
This makes any serious usage of malloc completely impossible!

Theoretically, but practically this is not much of a problem for most
programs.

It always return true!

No. It means what it says, i.e., even if malloc returns a non-null
pointer, memory might not actually be allocated. Linux attempts to do
so, when the memory is actually written to.
[...]

jacob's statement isn't an unreasonable inference, though. Given the
description, it's plausible that malloc() will always return a
non-null result (which can be interpreted as "true").

An experiment on my own Linux box shows that it doesn't *quite* work
that way; malloc() does return a null pointer for very large arguments
(above about 2.1 gigabytes on a system with 1 gigabyte of physical
memory).

Yes, that's why I raised that point. I too have observed malloc failing
for amounts much larger than the system's virtual memory.

Incidentally I wonder how far Lew Pitcher's recommendation to use calloc
is a good workaround? To me it seems that it would merely trigger the
"process culling" a bit earlier, with potentially the process being
killed even before calloc returned.

Anyway we should probably stop. This is getting very OT.

There are some very topical questions here, though. Does lazy
allocation satisfy the C standard's requirements for malloc()?

Seemingly it satisfies the Standard's requirements _as long as_ the actual
storage can be allocated and used in it's entirety. It would satisfy the
Standard even when some other process is killed and sufficient memory is
reclaimed to allow our process to continue, since Standard C has no concept
of multiple interacting processes anyway. Of course a silent forced
termination of the program when it tries to use it's allocated storage is
against the intent of the Standard. The Standard clearly says that a
successfull return from malloc implies that the requested storage was
allocated and is available for use in it's entirety.

Is it actually non-conforming behaviour. Possibly, depending on how exactly
you chose to look at malloc and the host environment, either as a part of
the C implementation, and hence answerable to the Standard, or outside the
implementation and hence outside the requirements of the Standard entirely.

Personally I consider such a setup broken, but I wouldn't blame the C
implementation though.

<snip>

Either way, it's obviously a difficult question; should the standard
be updated to clarify this point one way or the other? (That's a
question for comp.std.c.)

Is the issue really important enough in actual practice to justify it's
mention and clarification in the Standard, since only a fraction of the
existing C implementations actually exhibit this problem?

If the standard doesn't permit lazy allocation, then there's not a
whole lot we can say about it here; it just means that a lot of
systems are non-conforming.

I can't see how the Standard does not permit lazy allocation, since it says
nothing about the details of how *alloc's storage is actually allocated.

If a program can be arbitrarily terminated due to some condition in
the operating environment, that doesn't necessarily make the C
implementation non-conforming. If my program fails to complete
execution because I pulled the power plug, I'm not going to complain
to my C compiler vendor. Programs can die due to all sorts of
resource constraints. Having *my* program die because *your* program
allocated a lot of memory and the OS didn't complain at the time of
allocation is certainly unpleasant, but it's far from clear that it's
non-conforming behavior.

Yes, the issue is certainly ambiguous. I would consider that it, (the lazy
allocation feature), doesn't violate the letter of the Standard, but
certainly goes against it's intent. In any case the Standard has very
little control over this and making a statement one way or the other is not
going to affect the actual practice or the potential problem.


.

Relevant Pages

  • Re: Simple question about headers and malloc!
    ... Therefore I am making all of its declarations ... memory (using malloc) and then exit back to main. ... allocation, I get data strored from the second allocation... ...
    (microsoft.public.vc.language)
  • Re: A solution for the allocation failures problem
    ... Can you find anything in the standard which *allows* this behaviour? ... it defines how your C program is supposed to behave. ... previously allocated memory available for further allocation. ... says the memory will be made available for further allocation. ...
    (comp.lang.c)
  • Re: Simple question about headers and malloc!
    ... Therefore I am making all of its declarations ... memory (using malloc) and then exit back to main. ... allocation, I get data strored from the second allocation... ...
    (microsoft.public.vc.language)
  • Re: close fd while select/poll/epoll
    ... out of memory other than malloc returning NULL. ... POSIX *cannot* make this illegal. ... my proposal becomes part of the standard. ...
    (comp.os.linux.development.system)
  • Re: Freeing memory - will it be available immediately
    ... Your interpretation of "as the standard defines it" is ... Your interpretation of "for further allocation" as "for futher ... Except that this ignores the very reason for the standard, ... return the memory back to the OS in the manner you describe. ...
    (comp.lang.c)