Re: malloc under linux

jacob navia wrote:

Hi

I was working under linux today when I discovered this:

Did you just now discover this feature? Haven't we had discussions of this
several times in this group, the last one, IIRC, a month or two ago?

man malloc

<quote>
[snip pages]

BUGS
By default, Linux follows an optimistic memory allocation strategy.
This means that when malloc() returns non-NULL there is no guarantee
that the memory really is available. This is a really bad bug.
In case it turns out that the system is out of memory, one or more
processes will be killed by the infamous OOM killer. In case Linux is
employed under circumstances where it would be less desirable to
suddenly lose some randomly picked processes, and moreover the kernel
version is sufficiently recent, one can switch off this
overcommitting behavior using a command like
# echo 2 > /proc/sys/vm/overcommit_memory
<end quote>

I just can't believe my eyes.

This makes any serious usage of malloc completely impossible!

Theoretically, but practically this is not much of a problem for most
programs.

It always return true!

No. It means what it says, i.e., even if malloc returns a non-null pointer,
memory might not actually be allocated. Linux attempts to do so, when the
memory is actually written to.

And if the system discovers it promised more than it actually has,
then it will start killing processes at random.

This means that you are compiling a big piece of code, and then another
process or service gets killed because the compiler requested too
much memory!

Besides, the "work around" proposed supposes OF COURSE that you
are root!!!!

How is it possible to screw things up like this?
Has anyone here observed this behavior?

Yes, once, and that too because of an artificial program I created to mimic
a DOS attack. It ate up memory until it was killed. In my experience, the
offending process was always killed, not someother random one.

What is the work around? I mean at the C level.

Nothing, except to try to allocate memory only when you need it. Off-loading
some objects to a disk file would be a possible mitigating strategy, but
it's a big chore for a bug in some other program.

How can you avoid using malloc() ???

If you don't mind a slight loss of portability, Linux specific APIs like brk
and getrlimit/setrlimit can help. Also consider mmap. With plain vanilla
malloc, there is nothing much you can do, except be as sparing as possible
with your memory allocation.

.

Relevant Pages

  • Linux ELF loader vulnerabilities
    ... Numerous bugs have been found in the Linux ELF binary loader while ... Internally the Linux kernel uses a binary format loader layer to ... and the position of the memory map header in the binary image and ... An user may try to execute such a malicious binary with an unterminated ...
    (Bugtraq)
  • [Full-Disclosure] Linux ELF loader vulnerabilities
    ... Numerous bugs have been found in the Linux ELF binary loader while ... Internally the Linux kernel uses a binary format loader layer to ... and the position of the memory map header in the binary image and ... An user may try to execute such a malicious binary with an unterminated ...
    (Full-Disclosure)
  • Linux ELF loader vulnerabilities
    ... Numerous bugs have been found in the Linux ELF binary loader while ... Internally the Linux kernel uses a binary format loader layer to ... and the position of the memory map header in the binary image and ... An user may try to execute such a malicious binary with an unterminated ...
    (Full-Disclosure)
  • malloc under linux
    ... Linux follows an optimistic memory allocation strategy. ... This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. ... jacob navia ...
    (comp.lang.c)
  • Re: How to set up a Linux machine that occupies the minimum memory footprint ?
    ... Of course, if you use 8GB of RAM, or more precisely if you ... Are you saying that if I have a 4GB RAM memory, ... Linux will somehow "know" that it doesn't have ... You've said in another message that your app is ...
    (comp.os.linux.setup)