Re: detecting corrupt/stuffed files (how to?)

From: Jole (no_spam_at_nospam.com)
Date: 05/14/04 

Date: Fri, 14 May 2004 01:00:34 GMT

Hi
I'm a little unsure of what happens to files being copied when the OS
crashes. What does happen? does the filesystem not report the file's
correct size? It's still readable right?

My application consisits of 2 parts. It has one simple program running on
one pc which creates a file with data in it. This first program creates
this file on a floppy. Then that disk is transferred to another pc, which
reads that newly copied file. I need to make sure that the file copy on
the floppy isn't damaged in any way.

What if the first program copied it to the disk, and then renamed it. That
way, the other application which reads from the disk, looks for a file with
the different name. If it finds one, then we know the OS didn't previously
crash when copying the file.

here is my algorithm:

first program:
create file schedule.dat on floppy
rename file to schedule_new.dat (if OS crashes during copying, the file
wont be renamed)

client program:
look for file on floppy (schedule_new.dat) with the renamed name. (if it
finds this file it means that the OS didn't crash while creating the file)

i realize now that, as you were saying, this doesn't mean the file is the
same file. ie, it's possible to copy the file but it wont be the same copy
(have the same checksum). I'll be looking into this. Perhaps i can get
away without worrying about file integrity, so long as it doesn't happen
too often.

thanks

Thomas Weidenfeller wrote:

> Followup ignored. comp.lang.java is not a valid newsgroup.
>
> Jole wrote:
>> I'm writing a program that needs to read from a file. In order for the
>> program to be robust, it should somehow check that the file isn't
>> corrupt,
>> or stuffed in any way. For example, that file may have been created but
>> a crash occurred
>
> What type of crash? The OS crashing? Your application crashing?
>
>> at that point in time (while it was being created), damaging
>> the file.
>
> If the file system got damaged, the OS has to care about this (once it
> gets aware of it). Java has no specific API to find about this. Most
> general-purpose operating systems don't provide such information to an
> application. Well, a read attempt might return an error at some
> unexpected point.
>
> If you need some extra protection on the file system level, consider
> using a journaling file system.
>
> If just the data in the file is corrupt , and not the file system, it
> appears as a normal file to your application. So read on:
>
>> Now, my program which needs to read from this file, should first
>> check that it's in good condition, and that it hasn't been stuffed up in
>> any way. What is the normal way of doing this?
>
> A common way to ensure integrity is to run a checksum over the file, and
> append it to the end of the file. This doesn't protect from malicious
> attempts, but is a good protection for accidents.
>
> There are a few tricks, e.g. appending the checksum data in a way that
> when running the checksum algorithm again over the combined data, the
> resulting second checksum is 0. This simplifies the verification of the
> data a little bit: Blindly running the checksum algorithm over the
> combined data must return 0, or the file is corrupt.
>
> If you need to have protection from malicious tampering, look into
> digital signatures.
>
>> I'm using Java and am aware of some of those File.XX methods. perhaps
>> the
>> File.isReadable() methods will fail if the files have been damaged or
>> corrupted?
>
> No, not at all. Read the API documentation.
>
> /Thomas

Relevant Pages

  • Re: detecting corrupt/stuffed files (how to?)
    ... If the file system got damaged, the OS has to care about this (once it ... If you need some extra protection on the file system level, ... If just the data in the file is corrupt, and not the file system, it ... A common way to ensure integrity is to run a checksum over the file, ...
    (comp.lang.java.programmer)
  • Re: First-timer buying AS/400 and have questions
    ... License keys are how IBM controls everything. ... I can solve a lot of performance problems with memory. ... Checksum was released with OS/400 V1R0M0. ... In the root file system ...
    (comp.sys.ibm.as400.misc)
  • Fiber Channel Devices and Alternative Pathing
    ... Current partition table: ... topslnbap001:~> apconfig -D ... checksum: 2348860931 ... corrupt: No ...
    (SunManagers)
  • Re: XL driver checksum producing corrupted but checksum-correct packets
    ... you might want to try hooking up the UDP checksum drop case to ... :I may have misread: my reading was that the if_xl card marks the packet as ... the checksum check on my server on reception is succeeding). ... is because the client is no longer sending a corrupt packet. ...
    (freebsd-hackers)
  • Re: protecting a Web server by signing every URL
    ... If you have a static checksum function ... > request going through a controller firt, ... > in use on a production server. ... > But, still, it's a nice additionnal layer of protection. ...
    (comp.security.misc)