Re: tcl and threads: installing and other info

susara schrieb:
Cameron Laird wrote:

Just for practice, then, say a few words about your
concurrency requirements. There's a fair probability that
we can suggest a Tcl-based approach that does everything you
need withOUT you having to mess with configuration options,
download new sources, or so on.

Firstly thanks VERY much for the doc pointers, I'll go read up. Then,
my reasons for going off on a thread...

I've got a process that connects to a postgres db on the one hand, and
on the other accepts socket connections from subscription processes. It
waits on database notification events. On notify it has to do some
intensive processing and forward the results to the subscribed
processes. The processing will be pretty time consuming, and I'm pretty
sure those db notifications are going to come in faster than what I'm
going to be able to process.

So I'd like to parallelise the notification processing. The master
process will listen for the db notifications; fire off slave threads to
handle them, get the results back from each thread as it completes
processing and send it to the subscribed processes.

Sounds like the correct architecture, a event based master process using
a recent Tcl postgreSQL interface and event based socket stuff to manage
the IO things.
But if your on linux you could use processes instead of threads for the
slaves, started with open | listening on a pipe, much easier and much
more stable in case one of the slaves crashes or does nasty things. If
in a thread it would kill you process, in another process you can just
close the pipe, kill the child and restart a new one.

As your using PostgreSQL you could even create much of the master
process as a stored procedure inside the database, called by triggers.

I'll have to protect some common data against concurrent access. I
could be lazy and use the database's locking mechanisms to coordinate
the slaves, but for performance I'd prefer to keep as much of this in
memory and somehow share it between them, using mutexes where
necessary.
If you don't have enough memory for the database to do its
locking/concurrency control in memory your into swapping anyway.

Tcls thread model is message passing based, so you don't have shared
memory and mutex stuff usually, unless you explicitly request something
like it, see the thread package documentation for details.

This in particular holds for the results that have to be
communicated back to the master.

If there is an elegant way of handling this without using threads I'd
be very happy. I really don't feel like messing around with installing
manually built libraries on a debian box, especially not a live system
that runs 24/7...

As i said above. Use a master process that spawns new slaves as
processes when needed. Works perfectly well. I had such a system for
managing a distributed web crawler, where each agent-controller process
managed from 0-50 Perl processes on pipes, doing job control, recovering
from crashes of the perl scripts etc.

Btw. Tclsh on debian is threaded by default, see the entry in the
tcl_platform array by doing:

tclsh
% parray tcl_platform

Michael


.

Relevant Pages

  • Re: Buffered IO and pipes
    ... > master with as little waiting around as possible, ... > slaves can get back to doing their computation. ... need to use setvbufto make a larger-than default buffer. ... > memory. ...
    (comp.unix.solaris)
  • Re: [RFC/PATCH 0/22] W1: sysfs, lifetime and other fixes
    ... defined in family driver. ... Bus master driver - is like NIC driver, it does not know about the rest ... It has it's logical slaves, it has it's own attributes and features, ... This is wrong - netlink notification is used and will be moved to ...
    (Linux-Kernel)
  • Re: [RFC/PATCH 0/22] W1: sysfs, lifetime and other fixes
    ... > Why do you need separate files? ... >> link between private data and master and set useable master's name. ... but slave count and slaves attributes itself requires that mutex. ... >> structure and have family drivers create necessary attributes ...
    (Linux-Kernel)
  • Re: Conversion of RS232 serial data from Fire Alarm to MODBUS
    ... Master kept showing "Invalid response from Slave" and "Message Time ... are usually the channel out from the master to all of the slaves. ... In RS485 2 wire you devise a mechanism to change the direction of the line ... the master to tell the master computer line driver whether the ...
    (sci.engr.control)
  • Re: IP2IP_Addr in IPIF
    ... The docs do not explain too well what they mean by "local target ... the Master PLB IPIF comes with a slave ... the master IPIF is not a pure master. ... internal memory, the idea is that you use the IP2IPBus_Addr address ...
    (comp.arch.fpga)