Re: Problem: 'Threads' in Python?

From: Francis Avila (francisgavila_at_yahoo.com)
Date: 01/05/04 

Date: Mon, 5 Jan 2004 16:42:33 -0500

Ralph Sluiters wrote in message ...
>Hi,
>i've got a small problem with my python-script. It is a cgi-script, which
is
>called regulary (e.g. every 5 minutes) and returns a xml-data-structure.
>This script calls a very slow function, with a duration of 10-40 seconds.
To
>avoid delays, i inserted a cache for the data. So, if the script is called,
>it returns the last caculated data-structure and then the function is
called
>again and the new data is stored in the cache. (There is no problem to use
>older but faster data)
>
>My problem is, that the client (A Java program (or browser, command line))
>waits, until the whole script has ended and so the cache is worthless. How
>can I tell the client/browser/... that after the last print line there is
no
>more data and it can proceed? Or how can I tell the python script, that
>everything after the return of the data (the retieval of the new data and
>the storage in a file) can be done in an other thread or in the background?

Wouldn't a better approach be to decouple the cache mechanism from the cgi
script? Have a long-running Python process act as a memoizing cache and
delegate requests to the slow function. The cgi scripts then connect to
this cache process (via your favorite IPC mechanism). If the cache process
has a record of the call/request, it returns the previous value immediately,
and updates its cache in the meantime. If it doesn't have a record, then it
blocks the cgi script until it gets a result.

How can threading help you if the cgi-process dies after each request unless
you store the value somewhere else? And if you store the value somewhere,
why not have another process manage that storage? If it's possible to
output a complete page before the cgi script terminates (I don't know if the
server blocks until the script terminates), then you could do the cache
updating afterwards. In this case I guess you could use a pickled
dictionary or something as your cache, and you don't need a separate
process. But even here you wouldn't necessarily use threads.

Threads are up there with regexps: powerful, but avoid as much as possible. 

--
Francis Avila

Relevant Pages

  • Re: IIS Template Cache and Script Engine Cache
    ... Ok, I went and asked the ASP Dev about it, so I will share what I've ... The two concepts you are asking about is the "Template Cache" and "Script ... The "Template Cache" is a cache of the intermediate form of the particular ...
    (microsoft.public.inetserver.iis)
  • Re: IIS Template Cache and Script Engine Cache
    ... the templates in the template cache containing "other ... By this do you mean that the HTML that is in the asp page ... Cache" and "Script ...
    (microsoft.public.inetserver.iis)
  • Re: query parameters at the end of the JavaScript file
    ... CGI to serve static script files. ... information in the query is a version number. ... Browsers are not supposed to cache GET requests with queries AFAIK. ... There is no "query tacked onto it", the request is part of the URI ...
    (comp.lang.javascript)
  • Re: query parameters at the end of the JavaScript file
    ... CGI to serve static script files. ... Browsers are not supposed to cache GET requests with queries AFAIK. ... If the recs suggested that you don't cache a URI simply because it was a GET request then every search engine on the web would become crippled within an hour. ...
    (comp.lang.javascript)
  • Re: Caching via URL
    ... > I need to implement a caching mechanism for my script URLs. ... This way the browser will cache the page. ... You can have a script named browse. ... Apache to handle it as PHP script. ...
    (alt.php)