Re: Why sorting binary data is so slow?

On Oct 17, 11:17 am, ladayaros...@xxxxxxxxx wrote:
On 17 ÏËÔ, 12:44, Alexandre Ferrieux <alexandre.ferri...@xxxxxxxxx>
wrote:

What makes you think so ? Do you believe dichotomy is the only way of
looking up a needle in a haystack ?

No, but I'm trying this approach now...

If I were to do this, I'd replace

š šlappend AllRecs $rec

with

š šset AllRecs($rec) $something ;# <- you didn't say what output was
sought

All 'somethings' total size is over 1Gb ;)

Then this array must be made persistent (to be used by script that
actually makes
searches). It's one indexing -- many searches scheme. Total index
files
(there are many of them for one data file) size is over 4Gb, and I
need to search in all of them.



Then at search time the lookup would be

š šset found $AlLRecs($key).

And I must read whole array from disk to make it work...

If you need a database, why not use a database? Storing it all in an
SQLite DB will provide persistency and allow fast searching.

Mark
.

Relevant Pages

  • Re: Need Help deleting record from text file
    ... Who said you need to read an entire array into memory just to access a small portion of it? ... If you write the data in a suitable format you can later open the data file by reading just a relatively small header chunk and then read in the elements or the portion of data you require. ... How is using a database going to speed up that operation! ...
    (microsoft.public.vb.general.discussion)
  • Re: KirbyBase
    ... creating objects from the database records was much easier. ... Hal, I don't know if you have had a chance to take a look at the beta yet, but I basically tried to implement a uniform way to specify one-to-one links, one-to-many links, and calculated fields in the ... I suppose it would in effect be embedding an array where all the ... My first couple of attempts at adding more complexity to KirbyBase did not honor this concept. ...
    (comp.lang.ruby)
  • Re: Issues with CHANGE_TRACKING AUTO after initial index population
    ... There's a problem with 64 bit SQL 2005 server. ... Director of Text Mining and Database Strategy ... started the FT initial index population when our site activity is low. ... High activity on the data file. ...
    (microsoft.public.sqlserver.fulltext)
  • Re: KirbyBase
    ... I'm an Object Guy and Jamey is a Database Guy. ... That requires the table name and key field name to be specified ... I suppose it would in effect be embedding an array where all the ... that is) might compress by a factor of 10 or more. ...
    (comp.lang.ruby)
  • Re: using BLOB objects and ...
    ... while saving the array object to the database which is ok but while ... convert it back to object of 4 dimention boolean array which will be ... will let you store your 4 dimensional boolean array in it. ... You may have to modify my suggestion ...
    (comp.lang.java.programmer)