Re: java based supercomputer

On 16 Feb, 00:17, "Daniel Pitts" <googlegrou...@xxxxxxxxxxxxx> wrote:
On Feb 15, 12:54 pm, "sav" <mrobe...@xxxxxxx> wrote:



On 15 Feb, 20:39, "Daniel Pitts" <googlegrou...@xxxxxxxxxxxxx> wrote:

On Feb 15, 11:16 am, "sav" <mrobe...@xxxxxxx> wrote:

On 14 Feb, 22:24, "Daniel Pitts" <googlegrou...@xxxxxxxxxxxxx> wrote:

On Feb 14, 1:56 pm, "sav" <mrobe...@xxxxxxx> wrote:

I am trying an ambitous project of trying to adapt a piece of code
that i came accross on an IBM website. It uses psuedo remote threads
and is a way of utilising RMI, multithreading and will develop a java
based supercomuting technique. I wish to use a similar method for
checking the correlation beteween an array of data and another array
comprised of combining 8 other arrays of data, so by using a
straightforward for next loop will take forever but by utilising this
java psuedo remote threads will take a considerrably less time.
Howeve, i am relatively new to java but i learn fast. I need someont
to help me achieve this, i am willing to pay. I hacve the code
available for review

This first question, which isn't even Java specific, is:
Does your algorithm lend itself well to paralellization? You could
find out by running it with normal Thread's on a multiprocessor
system. If this improves performance, then you can proceed to the next
questions.

The second question, still not Java specific:
When you split the computation between N threads, how much
communication is there between those threads? If the communication
between threads has higher latency or limited bandwidth, how does that
effect your algorithms throughput?

Java specific question: Have you profiled your existing "slow" program
to see what is taking the most time? It might not be what you
expect.

Also, have you considered finding a less naive algorithm? Without
more details about your goal I can only speculate, but it might be
that your approach is an O(N^3) algorithm, when in fact there may
exist an O(n*logn) algorithm.

So, before trying to cross machine boundaries, or even thread
boundaries, it might be worth looking into other algorithms that speed
up your process.

Hi i did send you a full email to your email address did you get it?

No. Three reasons. The first is that my home computer has been
disabled for a few weeks.
The second reason is that the given e-mail address gets so much spam
that I usually just delete it with out looking.
The third reason is that I didn't invite you to e-mail me, please keep
the discussion here unless invited to otherwise.

As far as I know, no one here gets paid to answer questions on this
newsgroup. On the other hand, many of us DO get paid to answer
questions one-on-one. If you want free advice, post here; if you want
to pay for solutions, find a contractor.

ok cool, weel first to answer your questions, i am pretty sure that
the algorithm will be quite easy to parralelise if that is the correct
term, the only bottleneck i can see is checking the correlation value
against the highest rank, ie the algorithm basically is a for next
loop BUT with billions of calcs to perform, the bottlenck will
obviously be when the calcs need to check for a higher correlation
against the current highest rank, however with the java program i have
seen it seesm to perform many of the loop functions simoultaneously,
ie i have 8 arrays of of which perform 10^3 calculations so many of
these calcs can be carried out concurrently.

The current system runs in VBA not in java so i havent run in java
yet.

Look forward to your reply

Okay, so if I understand correctly:

You have N number of arrays to test;
Testing array A doesn't interact with the testing of array B.
Testing any array results in an easy-to-compare score.
So, you could create up to N threads, each calculates and records the
score.
All of the scores can then be easily compared to one another.

Since there are only 8 arrays (hence 8 scores), the sorting isn't too
hard to do.

How do you score array A? What do you mean by correlation? longest-
common-subsequence? number of similar/same elements? Something else?

Nok you are quite close its a little more compliacted than that but
you have the basic idea, Array A values dont change they are fixed
upon loaded from en excel file, i have written the code for that,
array b is made up of values based upon the values from 8 other arrays
each array is a calculation of (10^3)*N calcs, ie array 1 does a clac
and is then added to the values of the other 8 arrays, the addition or
the product of these arrays makes up Array B, array A and array B are
then checked for correlation, we can then check to se if the rank
value breaks the current highest rank. Its basically a very simple
algorithm, i have already have a working version in VBA and it works
perfectly but the amount of time it would take to complete is is the
tens of years so remote psuedo threads seems the way forward i have
two models i have downloaded from sourceforge but i dont fully
understand how they work yet, hence the need for someone to help me,
and as i said i am willing to pay.

.

Relevant Pages

  • Re: java based supercomputer
    ... checking the correlation beteween an array of data and another array ... java psuedo remote threads will take a considerrably less time. ... Does your algorithm lend itself well to paralellization? ... the only bottleneck i can see is checking the correlation value ...
    (comp.lang.java.programmer)
  • Re: A very basic question on relative cost of programming primitives.
    ... I'm writing a volume renderer in pure Java ... The volumes will typically be 512 Mega voxel, ... Java though will trap out-of-bounds array accesses in the runtime. ... I'm aware that choice of the volume rendering algorithm will also ...
    (comp.programming)
  • Re: regex question - trying to find ".mp3" in a SELECT box
    ... But `Array' is a Function object reference. ... Establish a new execution context using F's FormalParameterList, ... Since is not a primitive value, the exception should be thrown. ... | 5.2 Algorithm Conventions ...
    (comp.lang.javascript)
  • Re: Mergesort Vs Quicksort
    ... I might not have correctly remembered my algorithm of months ago ... for sorting records in an array using one auxilary of the ... on how things turn out from lower levels of recursion, ... whether the number of records in the array segment to be sorted is ...
    (comp.programming)
  • Re: java based supercomputer
    ... checking the correlation beteween an array of data and another array ... java psuedo remote threads will take a considerrably less time. ... Does your algorithm lend itself well to paralellization? ...
    (comp.lang.java.programmer)