Re: Pi program and hard disk usage

On Sep 27, 12:39 pm, mike3 <mike4...@xxxxxxxxx> wrote:
On Sep 27, 12:50 pm, user923005 <dcor...@xxxxxxxxx> wrote:





On Sep 27, 12:14 am, mike3 <mike4...@xxxxxxxxx> wrote:

On Sep 24, 12:57 pm, user923005 <dcor...@xxxxxxxxx> wrote:

On Sep 22, 4:45 pm,mike3<mike4...@xxxxxxxxx> wrote:
<snip>
I think Mr. Harter's (IIRC) suggestion is going to be pretty near to
optimal. You might improve it a bit with LRU cacheing or something,
but no matter what you do you will be waiting on the disk. I think
that also is plainly obvious.

However there must, must be some way that those really fast
pi programs like PiFast, QuickPi, etc. manage to do huge runs
that are too big to fit in RAM so fast. How do they do it? Any
ideas?

I guess that they figured out a way to do it so everything they
operate on fits in RAM.
If you use disk, you slow down by a factor of 300,000. Since they
don't seem horribly slow, we can assume that they are not using disk
for RAM.

Maybe you have more temporary variables than they do.

However it seems that, at least with PiFast, disk memory
is used. But they must have found a way to economize the
amount of access to the disk.

The problem I'm having with my program is that I cannot
fit a large enough Fast Fourier Transform (FFT) (actually
"Number Theoretic Transform" or NTT) to do the entire
128 megs of base 26 digits multiplication in RAM, all at
once. So I'm using a Karatsuba "divide and conquer"
O(n^1.585) approach to break up the large multiplication
into smaller ones that will fit in RAM. How could one
implement this as efficiently as possible when the numbers
are stored on disk, ie. minimize the amount of disk
access as much as possible?

Kratsuba is actually a good idea for disk based use, because it
operates on blocks of the numbers (you would not have to load the
whole number into memory -- just the pieces you are working on).

Because disk will be so dominant, it might even be better to use
*schoolbook*.

.

Relevant Pages

  • Re: OT/drift: when is a RAMdisk an appropriate solution
    ... include the "ram disk" component in your project. ... Sometimes, a physical RAM ... only to wind up going directly to regular old ordinary memory, ... testing the file system software. ...
    (comp.lang.c)
  • Re: teaching a child - console or GUI
    ... >> possible to set up some pretty fancy 'accelerators' ... disk under a file name that is ... ... or are they really a bunch of pointers ... You mean that you had to be convinced of the 'CD in RAM' approach? ...
    (comp.lang.pascal.delphi.misc)
  • Re: Future Linux on Bistable Storage
    ... One major difference between disk and RAM is the tradeoffs between size, ... resume the system -- except perhaps for I/O initialisation. ... Writing all of RAM to disk burns more power than powering RAM for several ...
    (Linux-Kernel)
  • Re: Hard disk speed - Maybe OT
    ... This will of course all be disk cache. ... The current capture uses ... I dont think RAM will help all that much. ...
    (alt.os.linux.suse)
  • Re: AppleWorks 3 + RamFactor + RamWorks 3 = what?
    ... I honestly don't remember if AppleWorks 3 needed a Ram Driver to ... If it does need a Ram driver, the ones on ProSel 8 are ... Once the boot disk is set up and booted from a 3.5" disk drive, ...
    (comp.sys.apple2)