Re: cgi concurrency approaches?

From: Alan Kennedy (alanmk_at_hotmail.com)
Date: 01/23/04 

Date: Fri, 23 Jan 2004 15:30:35 +0000

[replying to someone else's reply because I missed the original]

[Paul Rubin]
> I'm wondering if folks here have favorite lightweight ways of dealing
> with concurrency in cgi's,
[snip]
> but of course there's the obvious race
> condition if two people hit the
> cgi at the same time.
>
> Fancier solutions include running an transactional database in another
> process and connecting to it, setting up a daemon that remembers the
> counter value in memory and serializes access through a socket that
> the cgi opens, using a file-system specific hack like linking to
> counter file to lock it, having a timeout/retry if the counter is
> locked, with a possible hangup if a lock file gets left around by
> accident, etc. Each is a big pain in the neck.
> Anyone have some simpler approaches?

I don't know about a platform independent solution, but if you were
willing to limit yourself to certain varieties of Unix, e.g. Linux, it
would probably be relatively easy to implement a persistent counter
using something like System V message queues, or shared memory +
semaphores. This obviously will work only on systems which support
System V IPC.

Here is a brief page about System V IPC

http://www.science.unitn.it/~fiorella/guidelinux/tlk/node56.html

And here's a module, which I've never used, which purports to provide
a python interface to System V IPC facilities, if available.

http://www.theory.org/~mac4/software/

Given the above facilities, there are two ways I would approach the
problem of a persistent counter.

1. Use a dedicated Queue, and hold the counter value inside a single
message which "lives" on that queue. Those incrementing the counter
read the single message from the queue, increment it and put it back
on the queue. Any processes awaiting access to the "counter message"
would then do a blocking read on the queue. Since all puts and gets of
messages are atomic, you are guaranteed only atomic updates to your
counter. However, you could lock your system up if one of your
accessing CGI processes did not put the counter back again! You can
use timeouts as well, if my memory serves (it's been a long time since
I used System V IPC). You can also get fancy, with priorities, etc.

2. Store the value in pre-created shared memory partition, protected
by a semaphore. Since there are shared memory python modules for
several platforms, you might have a better chance with this approach
on non-unix platforms.

The Python Object Sharing (POSH) module might provide wrappers to some
useful functionality, although the module itself might be a little
heavyweight for providing a simple persistent counter.

http://poshmodule.sourceforge.net/posh/html/posh.html

HTH, 

-- 
alan kennedy
------------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan:              http://xhaus.com/contact/alan

Relevant Pages

  • Re: Threads vs Processes
    ... queue for example. ... You can even pass many Python objects ... Shared memory means the same memory appears in multiple processes, ...
    (comp.lang.python)
  • Re: Threads vs Processes
    ... queue for example. ... I actually do use pickle, could you elaborate on the safety issue? ... "shared memory" mean something more low-level like some bits that don't ... necessarily mean anything to python but might mean something to your ...
    (comp.lang.python)
  • "System V IPC is a botch?"
    ... immediately used to access a message queue, semaphore set, or shared memory ... With Sys V IPC, B can ... Does not fit comfortably within the "traditional unix filesystem ... > threaded process to block on multiple message queues. ...
    (comp.unix.programmer)
  • Re: Swapping and IPC
    ... > a swapping mechanism. ... Associated with each process that had IPC enabled was a queue ... IIRC, the sending process could arrange for the s.p. ...
    (comp.os.linux.misc)
  • Re: Will python ever have signalhandlers in threads?
    ... explains why Python requires so little of platform ... >> I don't fault the current Queue implementation. ... >> threads ask for the lock continuosly while the timeout threads only ... >> ask for the lock periodically. ...
    (comp.lang.python)