Re: using mmap on large (> 2 Gig) files


Martin v. Löwis wrote:

What architecture are you on? On a 32-bit architecture, it's likely
impossible to map in 2GiB, anyway (since it likely won't fit into the
available address space).

Indeed. But why does Python's memory mapping need to be flushed? And
why doesn't Python's mmap take an offset argument to handle large
files? Is Python actually memory mapping with mmap or just faking it
with fseek? If Python isn't memory mapping, there would be no limit
imposed by the 32 bit address space.

.

Relevant Pages

  • Re: using mmap on large (> 2 Gig) files
    ... But the question remains whether Python's "mmap" qualifies as a "memory ... Memory mapping a file means that the file is "mapped" ... I haven't looked at the source, but I'd be surprised if Python actually ... mapped file, flushing would obviously not be required. ...
    (comp.lang.python)
  • Re: using mmap on large (> 2 Gig) files
    ... "memory mapping" a file by means of fseekis probably more ... efficient than using UNIX' mmapor Windows' ... Both the Unix mmap and the Win32 MapViewOfFile allow a starting byte ...
    (comp.lang.python)
  • Memory-Mapping with LFS
    ... What are the current file size limits for memory mapping via glibc's ... - for a native 32-Bit System not using LFS? ... it is possible to mmap files orders of magnitude larger ... "There is a limit per process in the kernel vm that prevent from ...
    (Linux-Kernel)
  • Re: read vs. mmap (or io vs. page faults)
    ... resident pages associated with the a memory mapping, ... assume that for every mmap(). ... any processing at all and measure the time it takes to pre-fault the ... :page-faulting code do a better job using all this knowledge, ...
    (freebsd-questions)
  • Re: using mmap on large (> 2 Gig) files
    ... mmap), so you always map from the beginning of the file. ... you have ever worked with memory mapping files in C, ... require I/O, but there is no way around that in any case. ...
    (comp.lang.python)