Re: Progress report

From: Erann Gat (gNOSPAMat_at_flownet.com)
Date: 03/28/04 

Date: Sun, 28 Mar 2004 11:25:18 -0800

In article <87zna0oqee.fsf@nyct.net>, Rahul Jain <rjain@nyct.net> wrote:

> gNOSPAMat@flownet.com (Erann Gat) writes:
>
> > It turns out I didn't even need any fancy tricks like a STORABLE mixin or
> > a before method on slot-value. SXHASH was all I needed to manage the
> > cache.
>
> That's because you require an explicit STORE operation on the whole
> object after mutation instead of extending (SETF S-V-U-C) to do the
> STORE implicitly on the mutated slot-value.
>
> But now that I think about it, there's really no difference in the
> programmer effort involved.

Yes, that's the conclusion I came to. The programmer has to tell the
system somehow what they do and do not want stored.

> The only "problem" is that the persistence
> layer needs to _recursively_ check _all_ slot-values to see if they've
> changed.

No, this is the really cool thing. Once you've stored an object, as long
as that object hasn't be GC'd it's in the cache along with its sxhash the
last time it was stored. So to update everything in the DB you just walk
through the cache and compare the cached stored sxhash with the current
actual sxhash and if they differ you need to re-store the object. So the
programmer only has to STORE things once. Thereafter they can just call
SYNC and everything that is STOREd that needs to be updated will be.

I didn't demonstrate this because I hadn't implemented it, but it's trivial:

(define-method (sync (db odb hash-cache))
  (loop for o being the hash-keys of hash-cache do
        (unless (= (ref hash-cache o) (sxhash o)) (store db o))))

and now I can demonstrate it:

? (sync db)
NIL
? x
(A B C D)
? (setf (car x) 'z)
Z
? (sync db)
1898: replace CONS(id,hash,CAR,CDR) values(1324,378158984,1319,1321)
NIL
?

That took me literally less than three minutes.

(I just realized that sxhash doesn't quite do the right thing. It will
generally cause you to store more stuff than is really necessary. But
it's easy enough to write a new hash function that does the Right Thing.)

> Have you looked at (un)CommonSQL, btw? It's not quite the same, in that
> it's not intended to be a proper object store, but rather a way of
> building instances based on relational data -- slots can be defined to
> be joins, e.g.

Yes, I have. It's much more complex and sophisticated than what I'm
doing. My project is designed to be lightweight. Also, (un)CommonSQL
doesn't work on MCL.

E.

Relevant Pages

  • Re: Hung task - sync - 2.6.33-rc7 w/md6 multicore rebuild in process
    ... It seemed as though relcaim of fs cache became ... sync took a long time, and was reported by the kernel as a hung ... Running with sky2 dma patches (in rc8) and increased the audit name ...
    (Linux-Kernel)
  • Re: Rows returned are out of sync with the request.
    ... ('driver_connection' from cache) ... <- STORE= 1 at SQLRelay.pm line 145 ... However I suspect the signal was not delivered until the execute completed. ... Can $sth ever be already defined here? ...
    (perl.dbi.users)
  • Re: MMIO and gcc re-ordering issue
    ... It's quite gross at the moment, it has a sync before the access ... cacheable store is meaningful eg. for retaining io operations within ... It's meaningless in the sense that nothing guarantees that the writel ...
    (Linux-Kernel)
  • Re: [PATCH resend 5/9] MIPS: sync after cacheflush
    ... R4k shall pull data from its store buffer on ... There's no mention of uncached loads to have SYNC properties. ...
    (Linux-Kernel)
  • Re: issue in sending mail by exchange account
    ... So, if it goes though the cache, it will stack up in the outbox. ... Can you please tell me how can I send by exchange account in online mode through an existing profile in which exchange account is not the default one and neither its store is default? ...
    (microsoft.public.win32.programmer.messaging)