Re: what's better way to store a million keys in mem?

John_Woo wrote:
Hi,

A application, needs to check whether a user already logined, by
looking at static var in memory.

I don't have good idea. what's doing is, use Hashmap <not sure
Hashtable is better interms of inserting/removing/finding> to store ID
<string> -- key and status <character> -- value.

question:
what the better way to implement this goal, it should support up to 1
million of keys, in terms of high inserting/removing/finding, and
happened more frequently.

If you *must* keep track of users by keeping user data in memory, this
is probably one of the quickest solutions. But it will eat up a lot of
memory.

If we assume an overhead of 8 bytes per object and 4 byte references, a
HashMap will use /at least/ 28 bytes per entry and that does not
include any data you put in. So if we assume your keys and values are
say 50 bytes on average, a HashMap will use /at least/ around 80
megabytes for 1 million entries. And that can probably be safely
rounded up to about 100 MB. If you run a 64-bit machine, you are even
worse off. While this may not sound too much, all of this memory is
memory that the main part of your application can't use.

HashMaps (by design) are also notorious for thrashing the CPU cache,
because of bad locality of data, and this is made even more painful in
java, because everything in the map is *also* a reference which
augments the problem.

I'd give a real database a try for this. There is of course a bit of
overhead, but you use a lot less memory and I guess it will also be
faster. Of course, you should benchmark and compare any solutions
suggested.

Regards,
Daniel Sjöblom

.

Relevant Pages

  • Re: Extreamly large Hashtable
    ... >>>I'm assuming that if it is in memory it will be faster then looking ... >>performance (unless it means that the keys' equals() and ... >>references to the objects, ... Even if the Map.Entry overhead is more ...
    (comp.lang.java.programmer)
  • Re: Handling OutOfMemory Error
    ... If you can work out ahead of time how big that StringBuffer eventually needs to be, or even put a useful upper bound on it, then you can create the StringBuffer with that much capacity in the first place, which will avoid the need to expand it, and might avoid that failure. ... In general, a great way to reduce memory use is to find ways of doing things incrementally, so you don't need to have all your data in memory at once. ... report the total ... Otherwise, whilst ArrayList is fine, the HashMap would worry me slightly in terms of space. ...
    (comp.lang.java.programmer)
  • iBatis Memory Leak Bug!Some Help me!
    ... iBatis Memory Leak Bug!Some Help me! ... "SqlMapExecutorDelegate.java" etc which used one or one more hashmap ... Map sqlIncludes = new HashMap; ... private Map cache = Collections.synchronizedMap); ...
    (comp.lang.java.help)
  • Re: Is there a Class like HashMap, but...
    ... ByteCoder wrote: ... >> just use a HashMap and set the value to null? ... It wouldn't use any more memory to set the value to the same object ... You should be kludging things like this anyway. ...
    (comp.lang.java.programmer)
  • Re: Need to Parse delimited File into DataStructures .....
    ... You need to think about how you will need to access the data, it may be that HashMap will be faster for lookups if you don't know NAME. ... If you do know NAME then HashTable will be faster but I think it uses more memory. ... Usually you should declare a variable as a Map and implement it as a HashMap or Hashtable or TreeMap or whatever, depending on the desired performance characteristics. ...
    (comp.lang.java.programmer)