Re: Most efficient way to write a large matrix to disk

Arjan wrote:
In my particular case, they are the nuclear-motion
wave functions for a triatomic molecule.
[...]

This sounds like a PhD-project, am I right?

Well, you are right in thinking I'm a PhD student (fortunately,
towards the end). However, it's not I who wrote this program
(actually, it's a suite of programs). What I am doing is
maintenance, fixing bugs and adding a few functionalities.
The original program suite is called DVR3D and was
written over a period of 20 years in my research group,
firstly but my supervisor and then by many others.
For the record, the full program suite can be found here:
http://www.tampa.phys.ucl.ac.uk/ftp/vr/cpc03/
The file cpc.pdf contains the paper describing the program
(it was later published in 2004 in Computer Physics
Communications). If you have a look at this file you'll perhaps
get an idea of the level of complexity of this whole thing...


I'll continue to play the devil's advocate (still with all due
respect!):


Please, do :-)

(in some coordinate system)

What's the grid? Which base do you take? It must be a very inefficient
base if you need
that many nodes!

Ok, in fact I simplified a little. The program is not "grid based" in
the "naive" way, it used a numerical technique know as DVR,
discrete variable representation, which also involves generating
a grid. The grid points are the Gaussian quadrature points associated
with the basis functions chosen for the three coordinates. The coordinate
system used in my case are Radau coordinates... If you're really interested
you can have a look at the paper, unfortunately I'm afraid
the details are far too involved to be explained here on the newsgroup...
I can only say that for "ordinary" runs the grid size is much smaller,
about 45x45x35 but in the particular calculations at hand
we have to use a much larger one (we are investigated very
high energy states close to dissociation which are very delocalised).

Also, even if a cleverer way of dealing with this particular situation
existed, writing an ad hoc modification for this is not an option,
it'd be way to complicated and time consusiming for me to do now.

In e.g. a Proper Orthogonal Decomposition, you'll get
the relevant eigenstates.
The ones with relatively very small eigenvalues are insignificant.
Therefore, their transitions are irrelevant.
Throw them away and reduce the space required for your problem!
Storing 100*100*50 nodes in double precision cannot be meaningful.
This is not a programming/computer issue, but a lack of insight in the
physics!
Talk to someone who knows about the physics, not to a programmer!

I think you're underestimating the difficulty of the problem at hand.
A three-body system may look small but it can still be very, very difficult
in some cases. Many people on my group work on the molecular
ion H3+ and even with present-day computers/ computer clusters
computations are heavily limited by computational limitations...

Lorenzo


.