File handling: The easy and the hard way

From: Hans-Joachim Widmaier (hjwidmaier_at_web.de)
Date: 09/30/04 

Date: Thu, 30 Sep 2004 15:56:53 +0200

Hi all.

Handling files is an extremely frequent task in programming, so most
programming languages have an abstraction of the basic files offered by
the underlying operating system. This is indeed also true for our language
of choice, Python. Its file type allows some extraordinary convenient
access like:

    for line in open("blah"):
        handle_line(line)

While this is very handy for interactive usage or throw-away scripts, I'd
consider it a serious bug in a "production quality" software. Tracebacks
are fine for programmers, but end users really never should see any.
Especially not when the error is not in the program itself, but rather
just a mistyped filename. (Most of my helper scripts that we use to
develop software handle files this way. And even my co-workers don't
recognize 'file or directory not found' for what it is.) End users are
entitled to error messages they can easily understand like "I could not
open 'blaah' because there is no such file". Graceful error handling is
even more important when a program isn't just run on a command line but
with a GUI.

Which means? Which means that all this convenient file handling that
Python offers really should not be used in programs you give away. When I
asked for a canonical file access pattern some months ago, this was the
result:
http://groups.google.com/groups?hl=de&lr=&ie=UTF-8&threadm=pan.2003.12.30.21.32.37.195763%40web.de&rnum=1&prev=/groups%3Fhl%3Dde%26lr%3D%26ie%3DUTF-8%26q%3Dfile%2Bpattern%2Bcanonical%26btnG%3DSuche%26meta%3D

Now I have some programs that read and write several files at once. And
the reading and writing is done all over the place. If I really wanted to
do it "right", my once clear and readily understandable code turns into a
nightmare. This doesn't look like the language I love for its clarity and
expressivness any more. Python, being a very high level language, needs a
higher level file type, IMHO. This is, of course, much easier said than
done. And renown dimwits like me aren't expected to come up with solutions.
I've thought about subclassing file, but to me it looks like it wouldn't
help much. With all this try/except framing you need to insert a call
level anyway (wondering if this new decorator stuff might help?). The best
I've come up so far is a vague idea for an error callback (if there isn't
one, the well known exceptions might be raised) that gets called for
whatever error occured, like:

class File:
    ...
    def write(self, data):
        while True:
            try:
                self._write(data)
            except IOError, e:
                if self.errorcallback:
                    ret, dat = self.errorcallback(self, F_WRITE, e, data)
                    if ret == F_RETURN:
                        return dat
                else:
                    raise

The callback could then write a nice error message, abort the program,
maybe retry the operation (that's what the 'while True'-loop is for) or
return whatever value to the original caller. Although the callback
function will usually be more than a few lines, it can be reused. It can
even be packed into your own file-error-handling module, something the
original usage pattern can't.

If you still bear with me, you might as well sacrifice a few more seconds
and tell me what you think about my rant. Is everything just fine as it is
now? Or do I have a point? I always felt it most important to handle all
errors a program may encounter gracefully, and the easier this is to do,
the less likely it is a programmer will just sneak around the issue and
let the interpreter/run time system/operating system handle it. (And yes,
I'm guilty of not obeying it myself, as it can double or triple the time
needed to write the whole program; just because its so cumbersome.)

Waiting-for-you-to-jump-on-me'ly yours,
Hans-Joachim

Relevant Pages

  • Re: Some number theory
    ... it's only "recreational" in a sad way. ... it can be boiled-down to HSJ's handling of language; ...
    (sci.math)
  • Re: Was not making tail recursion elmination a mistake?
    ... Not the language; the implementation techniques. ... there is a myth out there saying "don't use exception ... > handling, for it's dog slow". ...
    (comp.lang.lisp)
  • Re: COBOL Transactions?
    ... normal COBOL thing is to supply the BEGIN TRANSACTION keywords, ... not required in the language spec for COBOL 85. ... Indexed file handling is in the PL/I semantics. ...
    (comp.os.vms)
  • Re: HOWTO: Re-raise an error?
    ... Well, if you look at any of the scripts Microsoft ships with IIS, error handling is done by just checking Err.Number anywhere they expect something to have failed. ... From that perspective, examples of "real world exception handling" are documented, but there are more practical ways of doing it. ... Ideally, if you know what you're looking for in terms of another language, you could find out whether anyone's thought of a good way to simulate it in VBScript. ... ActiveState O'Reilly Python cookbook code samples ratings review ...
    (microsoft.public.scripting.vbscript)
  • Re: Read a SQLite database? please help
    ... > I understand that if you know the language that's easy for you. ... If you don't know callback mechanism, you are not prepared enough to do ... any programming in C/C++ for Windows platforms. ... Mateusz Łoskot ...
    (microsoft.public.pocketpc.developer)