Error handling with Compress::Zlib

From: Mark Seger (Mark.Seger_at_hp.com)
Date: 10/21/04 

Date: Thu, 21 Oct 2004 11:26:48 -0400

I have a script that periodically copies a number of files from /proc to
a zipped file using this library. It's been working fine for over a
year and I've recently encountered a corrupted file. My first question
has to do with how the thing got corrupted in the first place and the
second question is why don't I get an immediate error when hitting the
corrputed data.

Now for some details.

Writing the file I simply do a

   $file=Compress::Zlib::gzopen($filename,'ab') or error-handling-here
   $file->gzwrite($data) or error-handling-here
   periodically I flush the buffer with
   $file->gzflush(2);

I don't report any errors when writing the file, but I also just noticed
on closer inspection that I'm not checking the return status from flush.
  Could that be my problem? groan...

If I try to gunzip the file, it fails with a message that the file is
corrupted. If I zcat the file and redirect the output to another file,
I get a couple errors that the file is corrupted but it does continue
processing. However, the resultant file is clearly corruputed midway
through. If I read the file with gzreadline it eventually returns an
error, but not before returning a whole pile of corrupted records. Does
this mean that the data buffer was corrupted (maybe by gzflush) but
successfully compressed?

Is corruption related to the way I'm doing the flushing and if so what
could I have done to prevent this? Assuming I did trap the error (which
I will definately add to my code) is there a clean way to recover? My
current thought is I should just close the file and reopen a newer version?

-mark

Relevant Pages

  • Re: Current tools for finding memory overwrites
    ... Fair enough, the whole problem with memory corruptions is that you can only detect a corruption in the area around a memory allocation, not the allocation itself (because it may be a valid write - the only people that can detect a corruption of data in an object are the people that wrote the object, by writing a function for integrity testing and calling it at regular intervals). ... Plus if you know where the data is getting knobbled, the analysis tab coupled with the Allocation History settings can be useful for finding references to your object that also reference parts of previously deleted objects. ...
    (microsoft.public.win32.programmer.tools)
  • Re: Low-bandwith ADSL, or something? (mostly off-topic)
    ... the following was guaranteed to cause corruption: ... >- writing to small files ... IIRC NTFS stores small files directly in the MFT, ... Prev by Date: ...
    (uk.comp.os.linux)
  • Re: Windows 2000 NTFS
    ... Peter T. Breuer wrote: ... > to ntfs partitions DOES cause corruption. ... Writing from linux, that is. ...
    (comp.os.linux.portable)
  • Re: gtkpod and Filesystem
    ... Did you try to reformat or dosfsck it? ... the filesystem code should handle corrupt filesystems more ... what's "more gracefully" in the light of fs corruption? ... blocked write access to avoid further damage caused by writing to an ...
    (Linux-Kernel)
  • Re: Developer warning: file handles can become un-usable after power OFF/ON
    ... possible corruption issues, you might try flushing every write. ... corruption can happen even if you flush every write. ... but when in fact it is not the same (e.g. because the storage card was ...
    (microsoft.public.pocketpc.developer)