Re: hard disk faster than main memory?

From: Nick Landsberg (SPAMhukolauTRAP_at_SPAMworldnetTRAP.att.net)
Date: 08/19/04 

Date: Thu, 19 Aug 2004 15:23:05 GMT

Markus Langlotz BTR Poegl Koma Tutor wrote:
> Hallo *,
>
> I'm afraid we're a little bit off-topic, but perhaps someone could give
> us an explanation or at least a hint concerning a very strange
> performance behaviour in our scientific code. We're are working on a
> sophisticated numerical algorithm working on stacks. Normally the stacks
> are implemented using arrays (fixed size). For sometimes our stack size
> exceeds main memory, we wrote a stack implementation writing to hard
> disk directly (using small buffers 1-32MB). So here's the strange
> behaviour: The stack implemenation
> writing everything to hard disk is faster than the stack implementation
> working on main memory. Is there any explanation for this behaviour or
> has someone made similar experience?
>
> Thanks a lot for any hint, explanation or idea
>
> Tobias & Markus
>
> http://www5.in.tum.de/forschung/peanoag/ (german)
>

(You did not say which O/S you are using so the following
is speculation based on Unix/Linux based behavior
of some systems. Nor did you say by how much
the behaviors differed.)

There are several second and third order effects which
may be coming into play here.

Since you are exceeding main memory within your single
application, some portion of that application's "pages"
will be written to disk by the O/S onto the swap partition
or swap file. This is usuallly done via an LRU algorithm
(least recently used). So, in your case, you are using
the disk for paging even though it's hidden from you by
the O/S. Writes to the swap partition are usuallly done
in 4096 byte chunks. Whether these pages are ever read
in again is dependent on your algorithm and how much
the algorithm jumps around between various arrays
in your "stack." If there is other disk activity
on the same physical drive as the swap partition
you are contending between the paging and the other
activity. A random I/O operation to disk is usually
is dominated by disk seek time (on the order of 7-10
ms. per random read. YMMV).

By taking it upon yourself to read in the data
from a disk file you have eliminated some of the
seeming non-determinism in the paging algorithm
(at least non-determinism from the application's
standpoint), and moved the I/O activity to being
more under your control. If you are reading
32K at a time, and you just happened to write
that data in a contiguous sector when you
created it, you will be getting all 32K with
a single I/O operation rather than possibly
8 I/O operations (if it had to get paged in
from the swap partition). If this data is on a physical
device of it's own, you may have any contention for
that device, and the seek times may not play as
much of a role as during random I/O's. (This
presumes your disk-based algorithm is reading 32K,
operating on it for however long is necessary,
possibly writing it out, and then reading in the
next batch of 32K.) Given that conjecture, it
is entirely possible that doing your own I/O
management designed specifically for your
unique application, is more efficient than
the general purpose paging algorithm used by
the O/S.

Not a definitive answer, just guesswork
at this point.

NPL

P.S. - This was common practice in the days
of 16-bit machines. There was no choice
except to do it that way. 

-- 
"It is impossible to make anything foolproof
because fools are so ingenious"
  - A. Bloch

Relevant Pages

  • Re: hard disk faster than main memory?
    ... >> sophisticated numerical algorithm working on stacks. ... For sometimes our stack size ... >> writing everything to hard disk is faster than the stack implementation ... Writes to the swap partition are usuallly done ...
    (comp.programming)
  • Re: hacker challenge - traverse a list
    ... that stack to a long integer and packing its values, ... Your algorithm, in abstract, is either bounded (and thus does not solve ... "Platonism" has a precise meaning. ... The problem here is that thugs think they can discount intentions. ...
    (comp.programming)
  • Re: Profiling tools/how to discover hot-spots in my code?
    ... so the real hotspots will show up independent of the disk access patterns. ... Program hotspots are caused by poor algorithm design. ... the internal performance counters in the allocator. ...
    (microsoft.public.vc.mfc)
  • Re: Transpose a large file(>2GB)
    ... What is the fast and ruby way to transpose a large file?? ... I can't read the whole file into memory due to the file sizes.. ... length is a multiple of the disk block size. ... From the rest of the algorithm, you assume that the OP wants to ...
    (comp.lang.ruby)
  • Re: coerce for arbitrary types
    ... the algorithm itself is essentially last-in first-out, ... via some sort of recursion? ... WaitStack <= new stack containing topnode ... As nested lists, there's only one number in a proper position. ...
    (comp.lang.lisp)