Re: Advice on long running processes

commander_coder@xxxxxxxxxxx wrote:

Hello,

I write a lot of CGI scripts, in Python of course. Now I need to
convert some to long-running processes. I'm having trouble finding
resources about the best practices to do that.

I've found a lot of email discussions that say something like, "You
need to educate yourself about the differences when you have long-
running processes" but I've not had a lot of luck with finding things
that explain the differences.

The biggest differences between run-and-exit vs. long running processes are
resource management and error recovery. Let's take them one at a time.

Resource management. In a short-lived process, you really don't have to
worry about this at all. Snarf as much memory as you need, open as many
files as you want, and when you exit, the operating system cleans it all up
for you. With a long running process, you have to worry about stuff like
that.

In Python, you're isolate from the low-level details of memory management,
but still need to think about it a bit. Imagine you had code that looked
like this in your main loop:

for request in getNextRequest():
requestList.append (request)
processRequest(request)

requestList is going to keep growing without bounds and eventually will eat
up all available memory in the system and your process will crash.
Everything you store, you also need to delete when you're done with it.

Same with files. In a short-lived process, you can generally open as many
files as you want and never worry about closing them. It unlikely you will
ever run out of file descriptors. In a long running process, that's not
the case. If you open a new file each time you get a request and never
close it, after a few hundred requests (depending on the operating system,
maybe even a few thousand), you'll run out of file descriptors.

The other big thing is error recovery. In a short lived process, if
something fails, you print an error message and exit. In a long running
process, you need to somehow recover from the error and keep going as best
you can. This can be tricky.
.

Relevant Pages

  • Re: Workgroup Messaging Message Configuration
    ... Project MVP ... Visit http://project.mvps.org/ for the FAQs and additional information ... The first time I did that, a status request was displayed ... I sent the request to the resource at ...
    (microsoft.public.project)
  • Re: Resource reservation problem
    ... Are you sure the client is using the "Make a Meeting" to send the request? ... double-clicking ON IT trying to create a meeting directly on the resource. ...
    (microsoft.public.outlook.calendaring)
  • Re: if-modified-since question (protocol problem?)
    ... request, but in response you are interested only if your request ... To base the decision on whether there is something 'new to show' would require knowledge about something 'old' - a cached resource. ... No Last-Modified header is sent in the response, ...
    (comp.infosystems.www.authoring.html)
  • Re: Browsers Resolving Symbolic Link (e.g. Windows Shortcuts)
    ... That path also contains a subdirectory 'style' in the same directory, and the subdirectory 'style' contains the file 'std.css' which is a Windows shortcut file. ... The only connection to HTML or CSS is via the http server, the program that interprets a request from your browser and "serves" a resource. ... Secondly a windows "shortcut" is not a symbolic link. ...
    (alt.html)
  • Re: Get running process by volume
    ... drivers receive IRP_MJ_CREATE request in context of a thread that has ... process which are reading/writing or querying informtion on that particular ... To be precise the answer i want is to get the list of running processes at ...
    (microsoft.public.win32.programmer.kernel)