Re: python logging multiple processes to one file (via socket server)



On Oct 27, 6:09 am, Gelonida N <gelon...@xxxxxxxxx> wrote:
Hi,

I have a rather 'simple' problem.
Logging from multiple processes to the same file AND be sure, that no
log message is lost,

1.) Log multiple processes to one file:
------------------------------------------

I have a python program, which I want to log, but which forks several times.

Due to the forking logging to files with the default logging.FileHandler
seems out of question.

It seems, that I could use a SocketHandler, which collects data from all
different processes and logs then to one file.

Does anybody have a working example.

2.) Ensure, that no log message is lost.
------------------------------------------
If I understood the doc of the SocketHandler, then
it will drop messages if the socket handler is not available.

However for my current activity I would prefer, that it aborts if it
cannot connect to the socket and that it blocks if the log server
doesn't handle the sent data fast enough.

Is this possible.

Thanlks a lot in advance.

What I found so far:http://docs.python.org/howto/logging-cookbook.html#logging-to-a-singl...

It states:
"The following section documents this approach in more detail and
includes a working socket receiver which can be used as a starting point
for you to adapt in your own applications."

Somehow I have a mental block though and fail to see the 'following
section'.

I also foundhttp://code.google.com/p/python-loggingserver/and ran
first tests.

However it seems, that this server stops logging my application after
about 128 log entries. (the number varies and is not necessarily exactly
128), whereas the console loggier continues logging.

I'm not really show why and would prefer a simpler example first.

Thanks in advance for any code example, idea, link, comment.


You might want to check out the SIMPL toolkit (http://
www.icanprogram.com/06py/main.html). A SIMPL receiver will
"naturally" queue and serialize messages from multiple senders.

bob
.



Relevant Pages

  • python logging multiple processes to one file (via socket server)
    ... Logging from multiple processes to the same file AND be sure, ... Due to the forking logging to files with the default logging.FileHandler ... that no log message is lost. ... it will drop messages if the socket handler is not available. ...
    (comp.lang.python)
  • Re: any use case of logging.config.listen()?
    ... design a socket interface only to expose logging configuration. ... temporarily turn "up" the logging to watch the new plug-in emit some ...
    (comp.lang.python)
  • Re: Error Logging Framework
    ... > - It allows different modes of logging like Deep and Shallow. ... different parts of the enterprise will likely ... While you'd want to support alerts, especially email alerts, keep in mind ... include the session id and perhaps user id of the client in the log message, ...
    (comp.object)
  • Re: Looking for good Logger for VB
    ... For logging to a shared file, either from multiple processes or from ... >> Dim hFile as Integer ...
    (microsoft.public.vb.general.discussion)
  • Re: MAC Framework, Socket information
    ... want to do is log all TCP/UDP outgoing connections that are being made. ... I can't help but wonder: would you be better-served by using the kernel's audit facilities to track events like socket connection? ... Are you blending access control and logging in your module, or is this really just about logging? ... struct socket *socket, struct label *socketlabel, ...
    (freebsd-hackers)