Re: GC performance - GC fragility

You normally will end with a heap looking like a swiss cheese.

It won't interestingly enough. Though the GC heap will look like one before compaction.

AFAIK FastMM tries to fight this with granularity and passing different allocation sizes to different "heap blocks", but it can't be IMHO generally be prevented.

Well the point you're missing here is that memory is the MM operates on virtual memory. If you only have macro-fragmentation in the virtual memory space, you can operate with limited fragmentation-induced waste (ie. below a constant amount) in the allocated memory space.
In practice you're looking at negligible amounts of memory waste compared to a GC (as all test cases so far have illustrated).

The only fragmentation that remains in FastMM is that of virtual memory when allocating large blocks. Though in 64bit, with virtually unlimited virtual memory that wouldn't be a problem at all (and in 64bit there are virtual memory solutions to reallocating large blocks to arbitrary sizes without a copy).

It's a big problem of long running applications.

FastMM and NexusDB (which uses a similar tactic) have been used in long running servers, and they are known to work without any stalls... unlike GC-based alternatives, where the GC stability is so poor that MS doesn't recommand the use of the concurrent GC in servers, but use of the blocking GC to maintain stability... that's because the concurrent GC can be gotten into a cycle of never ending GC compaction if your allocation rate exceeds or equals the compaction rate, a condition easy to reach when part of your allocated pool is swapped out.

how a native heap can (generally) prevent fragmentation,
> without compacting the memory ?

I suggest looking up the basm ng archive in google, there were quite a few discussions on the subject (look for memory manager challenge threads).

Eric
.

Relevant Pages

  • Re: Windows array allocation problem
    ... I've just written a program which does nothing except call GlobalMemoryStatus to get the available virtual memory, then allocates an array to that size and in decreasing steps until STAT returns zero. ... The maximum allocation is consistently 146 MB less than the reported available virtual memory. ... So the phenomenon isn't due only to how Windows handles virtual memory but how the compiler handles it. ... The test shows that even for the simplest program, the Intel compiler doesn't allow allocating as much of the application virtual memory as the Compaq compiler does, at least not to one large array. ...
    (comp.lang.fortran)
  • Re: Windows array allocation problem
    ... If the required size exceeds the possible size, the program has internal virtual memory management to solve the problem in pieces, storing intermediate pieces on the disk. ... much is available for an actual allocation. ... since the array dimension and the determination of whether to use the virtual memory manager are presently being determined by the calling program and passed in. ...
    (comp.lang.fortran)
  • Re: Windows array allocation problem
    ... By "available virtual memory" I meant the amount that's actually available for use at the moment that the array allocation is made. ... Functions that report the total amount ...
    (comp.lang.fortran)
  • Re: Windows array allocation problem
    ... If I won't be able to allocate nearly all the available virtual memory, how can I determine in advance how much I *can* safely allocate? ... By "available virtual memory" I meant the amount that's actually available for use at the moment that the array allocation is made. ... My program has provision for approximately predicting the additional memory which will be used later in the program, and adjusts the allocation accordingly. ...
    (comp.lang.fortran)
  • [PATCH 7/7] Compact memory directly by a process when a high-order allocation fails
    ... Ordinarily when a high-order allocation fails, direct reclaim is entered to ... Compaction by moving pages in memory is considerably cheaper than ...
    (Linux-Kernel)