Re: Most efficient way to write a large matrix to disk



glen herrmannsfeldt wrote:
[...]
For reasonable N, I would not expect a large difference.
Also, some systems might want a buffer big enough for a single
WRITE, which would give CBC a big advantage...
(snip)

Ok, good to know :-)



Yes, it sounds like you are filling the disk cache with
the first case, and then for the second it actually waits
until it is written to disk.

I did a few more tests and yes, the (software, I presume) disk
cache / write behind makes my rudimentary benchmark strategy
almost useless...
For very big runs clocked times increase a lot probably because
all the memory is used to store the matrix and there is no memory
left for the disk cache.


Most of the work is in library routines that don't change
with optimization level. Disk drive speeds don't change,
either.


Yes, I thought as much too.

Thank you,

Lorenzo


.



Relevant Pages

  • Re: Exploiting the Perl
    ... CO> snip ... This is what shared memory is for. ... asked about shared ram that stays resident outside of processes. ... CO> resort to slow disk if you need fast memory. ...
    (perl.beginners)
  • Re: Hard disk speed - Maybe OT
    ... This will of course all be disk cache. ... The current capture uses ... I dont think RAM will help all that much. ...
    (alt.os.linux.suse)
  • Re: [opensuse] Raid5/LVM2/XFS alignment
    ... I did not think about the on disk cache when I set the count. ... There may even be a way from user space to tell the drives to flush ... dd will invoke a kernel write call for each ...
    (SuSE)
  • Re: dual processor machine
    ... Steve> letting the machine use that memory for disk cache. ...
    (comp.os.linux.hardware)
  • Re: memory cache timeout
    ... You can divide the memory a process uses into "anonymous" working ... and that which consists of pages from disk that have been ... and are considered as part of disk cache. ... and how likely it is to reclaim disk cache that *isn't* ...
    (Fedora)