Re: [OT]Re: malloc


Random832 wrote:

2006-12-13 <1166034660.875671.203580@xxxxxxxxxxxxxxxxxxxxxxxxxxx>,
goose wrote:

Random832 wrote:

2006-12-13 <1166013671.850548.228900@xxxxxxxxxxxxxxxxxxxxxxxxxxx>,
goose wrote:
Random832 wrote:
2006-12-12 <1165956252.845272.153500@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>,
goose wrote:

Not *all* typical desktops perform the same; ISTR that FreeBSD
will happily overcommit knowing that all the memory (virtual + real)
will never ever be needed at the same time. When all processes
*do* attempt to run concurrently

Um... run concurrently? you mean as opposed to being in swap? Because
that's where the "virtual" part normally comes in.


Don't attempt to be condescending; you aren't any good at it.

Was I wrong?

You weren't even in the same context. It's impossible to be
wrong if you don't even understand the question.

Regardless of everything else, your use of the term "run concurrently"
was misleading - a program can be blocking/sleeping for hours, entirely
swapped out, and it _still_ ties up every single page of virtual memory
it has ever used; you seemed to suggest this was not the case. If this
wasn't the context you were talking about, your words were wrong for the
context you DID mean.

I apologise; I stupidly assumed that everyone reading would be
aware of what "concurrently" meant in the context of overcommits.

Rereading it, I see that I was maybe a tad ambigous.


See my reply to Barry Schwarz elsethread.

Your only reply to him that I can see is the one that I, in turn replied
to. Right above this one.

goose,
It's more common than you think.

What is? A program never "releases" memory without ever using it, that'd
be silly.

I've provided links to the threaded mailing-list discussion of this.

You have not. At least, not that I've been able to find. Perhaps
providing these links in an actual reply to one of my messages would be
more useful?

They were in reply to Barry earlier. Nevertheless, if you really wanted
to you could have googled for (without the quotes) "FreeBSD overcommit
malloc"
and taken the second link to read the entire discussion (a pretty big
one, IIRC).

The first link will allow one to find out exactly *how* common this
technique
is. Like I said in a previous post, this is learnt by undergraduates
all over
the world every year - it was mentioned as a technique in my
undergraduate
OS design course.


So basically it's gambling that program A will either
"release" some memory or die before program B attempts to write the
memory *that it just allocated* most likely in the next few seconds,
thus allowing it to thwart any code that might be in place for program
B to sanely handle the case where not as much memory as it wanted was
available.

Yes. I think you get the picture now.

When making such a bet, how often does the system win? I'd guess the
odds are probably pretty slim.

Well, if you bet that way, you might lose your money. FreeBSD has
always
(since I remember, anyway) had this, as well as all non System V
unixes.
Since FreeBSD (and other unixes) have had quite a good reputation for
robustness under load, I suspect that this works better than intuition
would
suggest.

The reason might have something to do with the fact that processes
allocate blocks bigger than they need *right now*, and only fill it up
to capacity later, if at all. A common method of using a dynamic
array is to double the size each time you reallocate it - you usually
end
up never accessing the last two-thirds of it under this method. More
than
onje regular here has suggested this method in the past.

goose,
Enough now, this is off-topic as it is, no need to drag it out
further.

.

Relevant Pages

  • Re: [OT]Re: malloc
    ... goose wrote: ... the first umpteen calls will succeed and all subsequent calls will ... Not *all* typical desktops perform the same; ISTR that FreeBSD ... so all malloc calls *always* return a pointer to memory. ...
    (comp.lang.c)
  • NVIDIA FreeBSD kernel feature requests
    ... NVIDIA has been looking at ways to improve its graphics driver for the ... FreeBSD i386 platform, as well as investigating the possibility of adding ... and user mappings of I/O memory, ...
    (freebsd-hackers)
  • RE: FreeBSD 4.9 freezes
    ... Copyright 1992-2003 The FreeBSD Project. ... Timecounter "TSC" frequency 2793012088 Hz ... Hyperthreading: 2 logical CPUs ... avail memory = 1546452992 ...
    (freebsd-questions)
  • Re: Question about top values on memory usage
    ... Another question is that is httpd uses threads (as provided by FreeBSD) ... SIZE is the total amount of virtual memory that a process has allocated. ... RES is the amount of physical memory in use by the process and will only include that part of a process's virtual memory space which is currently allocated in physical memory. ... The amount of swap space will place a hard upper bound on the number of httpd processes you can run before sbrk start failing. ...
    (freebsd-stable)
  • Re: Question about top values on memory usage
    ... Maybe someone with deeper knowledge of the internals of FreeBSD ... As you see, 'size' is the same for all processes, while RES varies. ... the real memory taken by a process is RES and SIZE ... RES is the amount of physical memory in use by the process and will ...
    (freebsd-stable)