DBI & memcached (was: Google SoC: Fame and Fortune ($4500) await you!)

On Wed, Mar 19, 2008 at 03:12:30AM -0000, Greg Sabino Mullane wrote:

I'm thinking about creating a merge between DBI and memcached. So cache
would be handled automaticly and integration would be seamless. This idea
occured to me a few times, since i was writing web app's which took
advantege of memcached. I was always getting tired of manual cache
management, and if you are using some ORM, it even harder to manage cache
propely. With plain SQL you could do something like $cache->set($key ,
$value), and then $cache->get($key), where $key would be your SQL statement,
which work fine in most cases.

I'm actually writing something very similar at the moment, and it's a lot
tricker than that, as you have to account for placeholders, as well as writing
a way to do cache invalidation. I'd be happy to have you take my work (which
is very Postgres-specific) and expand it into a generic DBI caching mechanism,
of which memcached would just be one method to use.

I'll also throw in a mention of DBD::Gofer, which now supports caching
using any class that implements get($key) and set($key, $value) methods.

In fact you can enable basic caching for idempotent queries in *any*
application, with no code changes, just by setting an env var:

DBI_AUTOPROXY='dbi:Gofer:transport=null;cache=1'

Tim.
.

Relevant Pages

  • Re: Google SoC: Fame and Fortune ($4500) await you!
    ... I was always getting tired of manual cache ... With plain SQL you could do something like $cache->set($key, ... I'm actually writing something very similar at the moment, ... PGP Key: 0x14964AC8 200803182311 ...
    (perl.dbi.users)
  • Re: efficacy of Linux w/o swap
    ... | Caching is supposed to happen, and this does not affect performance. ... that the impact starts to degrade the writing process itself. ... | Writing directly to disk will substantially decrease performance. ... Performance for the first 3/4 of an IDE disk ...
    (comp.os.linux.development.system)
  • RE: Disk vs Tape scenario
    ... With the storage available for me to test it is hard to eliminate cache ... Theoretically a volume using RAID-10 on 8x10K RPM disks can handle ... A write SSCH rate of 475/sec would push the RAID-10 parity group to 950 ... I just tested with two IEBDG jobs writing 20 million 80 byte records ...
    (bit.listserv.ibm-main)
  • Re: Cache questions
    ... > sets and the MMX and XMM registers where possible. ... > I'm writing a routine which will need to read from a source bitmap ... especially when it causes cache misses. ... > some stack space, and I could write one of my local variables with, ...
    (comp.lang.asm.x86)
  • Re: need fastest way to write 2gig array to disk file
    ... Eric Taylor wrote: ... > writes seem to go directly to the cache and so a 2 gig output ... > On another similar system, with scsi disks, once the program ... (when writing to the cache, ...
    (comp.os.linux.development.apps)