Re: what's better way to store a million keys in mem?

dsjoblom@xxxxxx wrote:
John_Woo wrote:
Hi,

A application, needs to check whether a user already logined, by
looking at static var in memory.

I don't have good idea. what's doing is, use Hashmap <not sure
Hashtable is better interms of inserting/removing/finding> to store ID
<string> -- key and status <character> -- value.

question:
what the better way to implement this goal, it should support up to 1
million of keys, in terms of high inserting/removing/finding, and
happened more frequently.

If you *must* keep track of users by keeping user data in memory, this
is probably one of the quickest solutions. But it will eat up a lot of
memory.

And it won't be stored in memory anyway. Every system now-a-days uses virtual memory. Which means that memory that isn't used much is off-load to disk, and only loaded back when needed. So your data will be stored on disk anyway.

A database is a good idea. Maybe check out file locking as an alternative. For example, have a list of users and passwords in a file. Maybe call it /etc/passwd or something. Each time a user logs in, lock the first byte of his/her entry in the /etc/passwd file. This may take more memory and time than a Java hash map but who knows, sometimes these low level system operations are highly optimized so, on the other hand, you also might come out ahead.

Off the top of my head, I think the other way to do this is to write a very efficient program in C, then let Java talk to it over a socket or something.

Final thoughts: one million users is A LOT. I mean, REALLY A LOT. There are only 40,000 or so user ports on a Unix system with TCP/IP. Have you very carefully investigated the requirements of this system you are building? I'm starting to seem more than just "a database" or "use C." I think you'll need to build out some kind of load balancing array, and that's non-trivial. Even a couple thousand users logged in at any given time is going to stress out a single system greatly. Better think this through carefully if it's important...
.

Relevant Pages

  • Re: Out of memory?
    ... Extensive disk testing needs to read / write to the disk and aren't ... Having so many database problems ... is read into memory but it can exceed the available memory. ... I suspect your AD got corrupt in a bad way (I recall your previous ...
    (microsoft.public.windows.server.sbs)
  • Re: Modeling events that occur in a game world
    ... A good DB only goes to disk when RAM is not available. ... Which DBs can you imbed completely in memory like this? ... In biz apps a database is usually available and assumed available. ...
    (comp.object)
  • Re: dataset Performence Issue
    ... Microsoft that a DataSet is okay to abuse as a DataBase. ... Managed Code can never be as fast and as optimized ... very good for 90% of the situations i.e. normal memory usage, ... Merge/GetChanges - and oh lets not forget keeping your disconnected cache ...
    (microsoft.public.dotnet.framework.adonet)
  • Creating A DataSet Programmatically
    ... disk, that's actually quite a loaded question. ... DataSet in memory whenever you like, ... As for reading into a database, there are a number of ways ... access the data that it stores (presently it stores only ...
    (microsoft.public.dotnet.framework.aspnet)
  • Re: Out of memory?
    ... Diags are usually pretty good about memory and cpu test and the disk ... Having so many database problems ... I suspect your AD got corrupt in a bad way (I recall your previous ...
    (microsoft.public.windows.server.sbs)