Re: java based supercomputer

On Feb 15, 12:54 pm, "sav" <mrobe...@xxxxxxx> wrote:
On 15 Feb, 20:39, "Daniel Pitts" <googlegrou...@xxxxxxxxxxxxx> wrote:



On Feb 15, 11:16 am, "sav" <mrobe...@xxxxxxx> wrote:

On 14 Feb, 22:24, "Daniel Pitts" <googlegrou...@xxxxxxxxxxxxx> wrote:

On Feb 14, 1:56 pm, "sav" <mrobe...@xxxxxxx> wrote:

I am trying an ambitous project of trying to adapt a piece of code
that i came accross on an IBM website. It uses psuedo remote threads
and is a way of utilising RMI, multithreading and will develop a java
based supercomuting technique. I wish to use a similar method for
checking the correlation beteween an array of data and another array
comprised of combining 8 other arrays of data, so by using a
straightforward for next loop will take forever but by utilising this
java psuedo remote threads will take a considerrably less time.
Howeve, i am relatively new to java but i learn fast. I need someont
to help me achieve this, i am willing to pay. I hacve the code
available for review

This first question, which isn't even Java specific, is:
Does your algorithm lend itself well to paralellization? You could
find out by running it with normal Thread's on a multiprocessor
system. If this improves performance, then you can proceed to the next
questions.

The second question, still not Java specific:
When you split the computation between N threads, how much
communication is there between those threads? If the communication
between threads has higher latency or limited bandwidth, how does that
effect your algorithms throughput?

Java specific question: Have you profiled your existing "slow" program
to see what is taking the most time? It might not be what you
expect.

Also, have you considered finding a less naive algorithm? Without
more details about your goal I can only speculate, but it might be
that your approach is an O(N^3) algorithm, when in fact there may
exist an O(n*logn) algorithm.

So, before trying to cross machine boundaries, or even thread
boundaries, it might be worth looking into other algorithms that speed
up your process.

Hi i did send you a full email to your email address did you get it?

No. Three reasons. The first is that my home computer has been
disabled for a few weeks.
The second reason is that the given e-mail address gets so much spam
that I usually just delete it with out looking.
The third reason is that I didn't invite you to e-mail me, please keep
the discussion here unless invited to otherwise.

As far as I know, no one here gets paid to answer questions on this
newsgroup. On the other hand, many of us DO get paid to answer
questions one-on-one. If you want free advice, post here; if you want
to pay for solutions, find a contractor.

ok cool, weel first to answer your questions, i am pretty sure that
the algorithm will be quite easy to parralelise if that is the correct
term, the only bottleneck i can see is checking the correlation value
against the highest rank, ie the algorithm basically is a for next
loop BUT with billions of calcs to perform, the bottlenck will
obviously be when the calcs need to check for a higher correlation
against the current highest rank, however with the java program i have
seen it seesm to perform many of the loop functions simoultaneously,
ie i have 8 arrays of of which perform 10^3 calculations so many of
these calcs can be carried out concurrently.

The current system runs in VBA not in java so i havent run in java
yet.

Look forward to your reply

Okay, so if I understand correctly:

You have N number of arrays to test;
Testing array A doesn't interact with the testing of array B.
Testing any array results in an easy-to-compare score.
So, you could create up to N threads, each calculates and records the
score.
All of the scores can then be easily compared to one another.

Since there are only 8 arrays (hence 8 scores), the sorting isn't too
hard to do.

How do you score array A? What do you mean by correlation? longest-
common-subsequence? number of similar/same elements? Something else?

.

Relevant Pages

  • Re: A very basic question on relative cost of programming primitives.
    ... I'm writing a volume renderer in pure Java ... The volumes will typically be 512 Mega voxel, ... Java though will trap out-of-bounds array accesses in the runtime. ... I'm aware that choice of the volume rendering algorithm will also ...
    (comp.programming)
  • Re: java based supercomputer
    ... checking the correlation beteween an array of data and another array ... java psuedo remote threads will take a considerrably less time. ... Does your algorithm lend itself well to paralellization? ...
    (comp.lang.java.programmer)
  • Re: java based supercomputer
    ... checking the correlation beteween an array of data and another array ... java psuedo remote threads will take a considerrably less time. ... Does your algorithm lend itself well to paralellization? ... the only bottleneck i can see is checking the correlation value ...
    (comp.lang.java.programmer)
  • Re: Java on Big Memory Machines
    ... My algorithm has already ... My computer has a hundred gigabyte memory, ... >>> it would be perfectly happy with a 16 gigabyte array. ... Java seems to be designed by conservatives, who also didn't see the expansion ...
    (comp.lang.java.programmer)
  • Re: Generating a Matrix of Random values with a specified Correlation
    ... Example algorithm: ... Create a 3rd array, where array B is sorted so that array ... A and array C have a correlation of X. ... so the correlation of A and B is rho = c/sqrt. ...
    (sci.math)