Re: hard disk faster than main memory?

From: Markus Langlotz BTR Poegl Koma Tutor (langlotz_at_in.tum.de)
Date: 08/19/04 

Date: Thu, 19 Aug 2004 17:45:29 +0200

Nick Landsberg schrieb:
> Markus Langlotz BTR Poegl Koma Tutor wrote:
>
>> Hallo *,
>>
>> I'm afraid we're a little bit off-topic, but perhaps someone could give
>> us an explanation or at least a hint concerning a very strange
>> performance behaviour in our scientific code. We're are working on a
>> sophisticated numerical algorithm working on stacks. Normally the stacks
>> are implemented using arrays (fixed size). For sometimes our stack size
>> exceeds main memory, we wrote a stack implementation writing to hard
>> disk directly (using small buffers 1-32MB). So here's the strange
>> behaviour: The stack implemenation
>> writing everything to hard disk is faster than the stack implementation
>> working on main memory. Is there any explanation for this behaviour or
>> has someone made similar experience?
>>
>> Thanks a lot for any hint, explanation or idea
>>
>> Tobias & Markus
>>
>> http://www5.in.tum.de/forschung/peanoag/ (german)
>>
>
> (You did not say which O/S you are using so the following
> is speculation based on Unix/Linux based behavior
> of some systems. Nor did you say by how much
> the behaviors differed.)
>
> There are several second and third order effects which
> may be coming into play here.
>
> Since you are exceeding main memory within your single
> application, some portion of that application's "pages"
> will be written to disk by the O/S onto the swap partition
> or swap file. This is usuallly done via an LRU algorithm
> (least recently used). So, in your case, you are using
> the disk for paging even though it's hidden from you by
> the O/S. Writes to the swap partition are usuallly done
> in 4096 byte chunks. Whether these pages are ever read
> in again is dependent on your algorithm and how much
> the algorithm jumps around between various arrays
> in your "stack." If there is other disk activity
> on the same physical drive as the swap partition
> you are contending between the paging and the other
> activity. A random I/O operation to disk is usually
> is dominated by disk seek time (on the order of 7-10
> ms. per random read. YMMV).
>
> By taking it upon yourself to read in the data
> from a disk file you have eliminated some of the
> seeming non-determinism in the paging algorithm
> (at least non-determinism from the application's
> standpoint), and moved the I/O activity to being
> more under your control. If you are reading
> 32K at a time, and you just happened to write
> that data in a contiguous sector when you
> created it, you will be getting all 32K with
> a single I/O operation rather than possibly
> 8 I/O operations (if it had to get paged in
> from the swap partition). If this data is on a physical
> device of it's own, you may have any contention for
> that device, and the seek times may not play as
> much of a role as during random I/O's. (This
> presumes your disk-based algorithm is reading 32K,
> operating on it for however long is necessary,
> possibly writing it out, and then reading in the
> next batch of 32K.) Given that conjecture, it
> is entirely possible that doing your own I/O
> management designed specifically for your
> unique application, is more efficient than
> the general purpose paging algorithm used by
> the O/S.
>
> Not a definitive answer, just guesswork
> at this point.
>
> NPL
>
> P.S. - This was common practice in the days
> of 16-bit machines. There was no choice
> except to do it that way.
>

We are working on a P4 Xeon and LINUX (don't know the distribution).
We forgot to write some important point. The comparisions were done
using data amounts, which fitted into the main memory (no swapping - see
other posting). But even then the hard disk version of the code was
about 10% faster.

Markus & Toby

Relevant Pages

  • Re: hard disk faster than main memory?
    ... > sophisticated numerical algorithm working on stacks. ... For sometimes our stack size ... > writing everything to hard disk is faster than the stack implementation ... A random I/O operation to disk is usually ...
    (comp.programming)
  • Re: hacker challenge - traverse a list
    ... that stack to a long integer and packing its values, ... Your algorithm, in abstract, is either bounded (and thus does not solve ... "Platonism" has a precise meaning. ... The problem here is that thugs think they can discount intentions. ...
    (comp.programming)
  • Re: Profiling tools/how to discover hot-spots in my code?
    ... so the real hotspots will show up independent of the disk access patterns. ... Program hotspots are caused by poor algorithm design. ... the internal performance counters in the allocator. ...
    (microsoft.public.vc.mfc)
  • Re: Transpose a large file(>2GB)
    ... What is the fast and ruby way to transpose a large file?? ... I can't read the whole file into memory due to the file sizes.. ... length is a multiple of the disk block size. ... From the rest of the algorithm, you assume that the OP wants to ...
    (comp.lang.ruby)
  • Re: coerce for arbitrary types
    ... the algorithm itself is essentially last-in first-out, ... via some sort of recursion? ... WaitStack <= new stack containing topnode ... As nested lists, there's only one number in a proper position. ...
    (comp.lang.lisp)