Re: line duplication using logging to file

Hi,

Thanks for the help.
Meanwhile I have written the logging function from scratch and it works
without the multiple lines.

This means that the multiple line copy is not due to the multiple
processes (or thread) trying to access the log file but to something
else.

Thanks.
Sebastien.

the new function :


import logging
import time

def write_log(level, message):
nom_logger="main_log_file.txt"
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(levelname)s %(message)s',
filename=nom_logger,
filemode='w')

if str(level).lower() == "info" :
logging.info(str(message))
elif str(level).lower() =="error":
logging.error(str(message))
elif str(level).lower()=="warning" :
logging.warning(str(message))
elif str(level).lower() =="critical":
logging.critical(str(message))
elif str(level).lower() == "exception":
logging.exception(str(message))
else :
logging.INFO(str(message))

return







Paul McGuire a écrit :
"seb" <sebastien.thur@xxxxxxxxxxx> wrote in message
news:1168329975.491514.282000@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.

Seb.


A quick tally of log messages by timestamp and comment gives this data:

('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
('2007-01-08 18:26:19,578', '___run____thread lance') : 2
('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
('2007-01-09 08:51:26,733', '___run____thread lance') : 2
('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1

Does this suggest anything to you?

-- Paul


(BTW, here is the pyparsing program I used to do this analysis)

from pyparsing import
Word,nums,Combine,alphas,oneOf,Literal,SkipTo,restOfLine

# create pyparsing grammar definition for a log line
date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word(nums,exact=2)+ \
','+Word(nums,exact=3)
timestamp = Combine(date + ' ' + time)
severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this
data
backslash = Literal("\\")
fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") +
".py")
logline = ( timestamp.setResultsName("timestamp") + "-" +
severity.setResultsName("severity") + "-" +
fileref.setResultsName("fileref") +
restOfLine.setResultsName("comment") )

# create list of ParseResults, with addressable log line elements by results
name
logEntries = [ logline.parseString(line) for line in logdata ]

# tally up log lines by timestamp and comment
tallyByTimestamp = {}
for entry in logEntries:
tallyKey = (entry.timestamp, entry.comment)
tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1

for ts in sorted( tallyByTimestamp.items() ):
print "%s : %d" % ts

.

Relevant Pages

  • Re: Future ISPF directions (was: Re: How Was Share?)
    ... Shutting off propagation of the Enqueue for PROFILE would only ... apply to logging on to multiple systems in the sysplex. ... LOG might be a bigger issue if you wanted multiple concurrent ... For IBM-MAIN subscribe / signoff / archive access instructions, ...
    (bit.listserv.ibm-main)
  • Re: lew? 2 qq
    ... Everyone writes to the same channel, and the logging overhead to stderr is incurred with every call; that can have an adverse impact on performance. ... you have to invent your own formats, disentangle all the output from multiple sources, disentangle important output from less so, and you can't selectively suppress some log output and not other. ... All less critical messages ("WARNING", "DEBUG", etc.) not only are not emitted, they have negligible impact on production performance. ... Logging is configurable at deployment time via an external resource file, so you don't even have to hard-code all that control. ...
    (comp.lang.java.help)
  • Re: Logging 0.7.0 (Quixotic Starfish)
    ... Multiple Rails processes can ... You can also use the Syslog appender in Logging and dump to the syslog ... This is good for collecting logs from multiple machines. ...
    (comp.lang.ruby)
  • Re: Trace debugging in run-time (re-direct Debug.Print to a file?)
    ... You could use a file logging class that is safe to use with multiple ... Note that I'm not redirecting you to a file-based logging as it has some ... overhead and possible issues as Windows doesn't update the log file ...
    (microsoft.public.vb.general.discussion)
  • Re: logging producing redundant entries
    ... within Zope Extensions. ... I'm using this simple logging setup at the top of a zope Extension module: ... You seem to be executing the code given above multiple ...
    (comp.lang.python)
Loading