Re: Simple text parsing gets difficult when line continues to next line

John Machin wrote:
Jacob Rael wrote:
Hello,

I have a simple script to parse a text file (a visual basic program)
and convert key parts to tcl. Since I am only working on specific
sections and I need it quick, I decided not to learn/try a full blown
parsing module. My simple script works well until it runs into
functions that straddle multiple lines. For example:

Call mass_write(&H0, &HF, &H4, &H0, &H5, &H0, &H6, &H0, &H7, &H0,
&H8, &H0, _
&H9, &H0, &HA, &H0, &HB, &H0, &HC, &H0, &HD, &H0, &HE,
&H0, &HF, &H0, -1)


I read in each line with:

for line in open(fileName).readlines():

I would line to identify if a line continues (if line.endswith('_'))
and concate with the next line:

line = line + nextLine

How can I get the next line when I am in a for loop using readlines?

Don't do that. I'm rather dubious about approaches that try to grab the
next line on the fly e.g. fp.next(). Here's a function that takes a
list of lines and returns another with all trailing whitespace removed
and the continued lines glued together. It uses a simple state machine
approach.

I agree that mixing the line assembly and parsing is probably a mistake although using next explicitly is fine as long as your careful with it. For instance, I would be wary to use the mixed for-loop, next strategy that some of the previous posts suggested. Here's a different, generator-based implementation of the same idea that, for better or for worse is considerably less verbose:

def continue_join_2(linesin):
getline = iter(linesin).next
while True:
buffer = getline().rstrip()
try:
while buffer.endswith('_'):
buffer = buffer[:-1] + getline().rstrip()
except StopIteration:
raise ValueError("last line is continued: %r" % line)
yield buffer

-tim

[SNIP]

.

Relevant Pages