RE: DBI to BerkeleyDB?



ManKyu Han wrote:
Hi. Again.
I did benchmark using BerkeleyDB (Hash) module (random select) and
the number I got was around 10,000 / sec.
I also did similar benchmark using DBD::DBM (with BerkeleyDB Hash
support). But this time, the number was too low. (less than 200
/sec).

I moved "prepare" outside loop, so DBD::DBM (BerkeleyDB) works almost
twice faster (100 -> 200 /sec) but compare to BerkeleyDB, it is still
slower. Since DBD::DBM is now using BerkeleyDB, shouldn't it perform
as well as BerkeleyDB ?? (or at least not as bad as what I got?)

You're comparing the speed of a very low-level dbm file interface to the
speed of a high-level rdbms interface that is implemented in pure Perl.
With BerkeleyDB there is almost no overhead -- you're essentially
calling the C library directly. With DBD::DBM, though, you're going
through several layers of abstraction -- at least DBI, a pure-perl DBD,
and a pure-perl SQL engine.

I don't have much experience in DBD::DBM, but this speed difference
doesn't really seem unreasonable.

If you have the option to use BerkeleyDB (specifically, if you only want
to store an index of name-value pairs), then you should probably use
that directly. It's one of the fastest (if not THE fastest) ways to
persist a hash. If you might need multi-column support in the future
and you need an in-process database, try DBD::SQLite. It's relatively
robust and is implemented in C.

Regards,
Philip
.



Relevant Pages

  • Re: BerkeleyDB
    ... DB_File to BerkeleyDB. ... untie attempted while 1 inner references still exist at ./dbm_test.pl line ... untie %hash; ...
    (perl.beginners)
  • Re: BerkeleyDB
    ... DB_File to BerkeleyDB. ... untie attempted while 1 inner references still exist at ./dbm_test.pl line ... warning. ... untie %hash; ...
    (perl.beginners)
  • Re: multiuser hash (BerkelyDB?)
    ... I need to rewrite so multiple processes will ... access the hash at the same time. ... I was told by a guy from sleepy cat that the Berkeley database will (by ... BerkeleyDB it seamed reasonable to believe him. ...
    (comp.lang.perl.modules)
  • BerkeleyDB
    ... I was trying out some jobs with the Berkeley DB and decided to move up from DB_File to BerkeleyDB. ... I don't need a lot of features, ... I shouldn't get this warning. ... untie %hash; ...
    (perl.beginners)