Re: How to store a large amount of 3D data points in Java?

From: Dimitri Maziuk (dima_at_127.0.0.1)
Date: 06/30/04


Date: Wed, 30 Jun 2004 18:17:25 +0000 (UTC)

Roedy Green sez:
> On Tue, 29 Jun 2004 23:58:06 +0000 (UTC), Dimitri Maziuk
><dima@127.0.0.1> wrote or quoted :
>
>>None of which would be a problem if you could store ints in a Java
>>collection, like you can in C++.
>
> With 1.5 autoboxing you can, even if it is implemented with objects.
>
> Further, you can easily write analogs of ArrayList, HashMap, and
> HashSet that hold primitives. People imagine there must be some black
> magic that makes these classes work. The guts are very simple and you
> can study the source in src.jar/src.zip for the object version. In the
> old days everyone wrote these classes for themselves. Read
> http://mindprod.com/jgloss/hashtable.html for example to help
> understand how hashed lookup works.
>
> You keep coming across like some helpless newbie who imagines if the
> class does not exist prewritten there is some insurmountable obstacle.
>
> C++ has a slight advantage in writing such code because you can, for
> example, store pairs of ints at each slot in an array. With Java, you
> need two separate arrays indexed the same, or you need to store a
> reference to an object containing a pair at each slot. But you
> strongly exaggerate when you state that Java can't tackle the same
> problems quite efficiently. The java code is slightly less convenient
> to write but it runs more efficiently since indexing does not require
> multiplication, just shifts which can sometime be done free.

You keep coming across like someone with reading comprehension
skills of a doorpost and attention span of an amoeba.

ArrayList does array copy for resizing. Whether I look at src.zip
or not, it still does array copy.

You said it yourself: that is unsuitable for large dataset/low
memory situations because it requires twice the memory.

C++ has a "slight advantage" in that you can use realloc(3) to grow
your array in situ, no array copy required.

This whole discussion started because someone suggested replacing
container objects with arrays of primitive types to store large
datasets.

(The keyword here is _primitive_types_, so 1.5 autoboxing is no use
_because_ it is implemented with objects, BTW.)

You said that right way of using arrays is to count the values first
and create array of the correct size. I added that this doesn't work
in a general case where number of elements is not known in advance;
in addition, retrieveing array elements based on value rather than
array subscript is inefficient, so you'd need to also write your own
lookup tables etfc.

"The Java code" to implement lookup tables, make sure your arrays
never get out of sync, etc., "is slightly less convenient to write"
only for very large values of "slightly".

C++ has a "slight" advantage in that you can use vectors to store
your datapoints regardless of their data type. (Ever heard the
phrase "first class citizens" applied to user-defined data types?
That's what OO languages other than Java do.)

The fact that you can store pairs in arrays is _not_ an advantage
of C++: you can store pairs in Java arrays too. The only difference
is that pair comes standard in C++; in Java you need to define it
yourself.

The problem, again, is having too many objects in your Java program.
That's why you want to store your pairs in two arrays instead of a
Pair []. (And then you need to make sure your two arrays never ever
get out of sync, and that makes your Java code "slightly less
convenient to write"... but I already said that.)

And by the way, array indexing in C++ does not necessarily
require multiplication: '*' in "*a + elt_num" is _not_
a multiplication sign, appearances notwithstanding .

Dima

-- 
Tlaloc: What was Elrond's second name?
Gruber: Hubbard                           -- <ahbou=3C69EB63.A7C431F4@last.com>


Relevant Pages

  • Re: Java Generic programming using subclassing
    ... Java nowhere guarantees that the elements of an array ... Now chances are that any Java ... > reference will be visible through the other. ... > eligible for garbage collection, and will in fact be collected before ...
    (comp.lang.java.programmer)
  • Re: java based supercomputer
    ... checking the correlation beteween an array of data and another array ... java psuedo remote threads will take a considerrably less time. ... Does your algorithm lend itself well to paralellization? ... the only bottleneck i can see is checking the correlation value ...
    (comp.lang.java.programmer)
  • Re: MinMax aus Integer-Array schnell ermitteln
    ... Trick "Loop unrolling" erreicht. ... Java sollte eigentlich an Delphi rankommen, ... Array UP ... private static void fillArray0() ...
    (de.comp.lang.java)
  • Re: MinMax aus Integer-Array schnell ermitteln
    ... Trick "Loop unrolling" erreicht. ... Java sollte eigentlich an Delphi rankommen, ... Array UP ... private static void fillArray0() ...
    (de.comp.lang.java)
  • Re: partition numbers in four terms
    ... I try to find a class able to store in an array or a collection or something ... I found a code on the net but in java:http://bytes.com/topic/c/answers/869529-print-all-combinations-number ... ... Simply that is an order partition ...
    (microsoft.public.dotnet.languages.vb)