Re: reading files with error
- From: Maurice Ling <mauriceling@xxxxxxx>
- Date: Sun, 18 Sep 2005 14:15:00 +1000
jepler@xxxxxxxxxxxxxx wrote:
I have written a program to do something similar. My strategy is: * use os.read() to read 512 bytes at a time * when os.read fails, seek to the next multiple of 512 bytes and write '\0' * 512 to the output I notice this code doesn't deal properly with short reads, but in my experience they never happen (because the disk error makes an entire block unreadable, and a block is never less than 512 bytes)
I use this code on a unix-style system.
def dd(src, target, bs=512): src = os.open(src, os.O_RDONLY) if os.path.exists(target): target = os.open(target, os.O_WRONLY | os.O_APPEND, 0600) existing = os.lseek(target, 0, SEEK_END) else: existing = 0 target = os.open(target, os.O_WRONLY | os.O_CREAT, 0600)
total = os.lseek(src, 0, SEEK_END) / bs os.lseek(src, existing, SEEK_SET) os.lseek(target, existing, SEEK_SET)
if existing: print "starting at", existing i = existing / bs f = 0 lastrem = -1
last = start = time.time()
while 1:
try:
block = os.read(src, bs)
except os.error, detail:
if detail.errno == errno.EIO:
block = "\0" * bs
os.lseek(src, (i+1) * bs, SEEK_SET)
f = f + 1
else:
raise if block == "": break
i = i + 1 os.write(target, block)
now = time.time() if i % 1000 or now - last < 1: continue last = now
frac = i * 1. / total
rem = int((now-start) * (1-frac) / frac)
if rem < 60 or abs(rem - lastrem) > .5:
rm, rs = divmod(rem, 60)
lastrem = rem
spd = i * 512. / (now - start) / 1024 / 1024
sys.stderr.write("%8d %8d %8d %3.1f%% %6d:%02d %6.1fMB/s\r"
% (i, f, i-f, i * 100. / total, rm, rs, spd))
sys.stderr.write("\n")
Sorry but what are SEEK_END and SEEK_SET?
Maurice
--
Maurice Han Tong LING, BSc(Hons)(Biochem), AdvDipComp, CPT, SSN, FIFA, MASBMB, MAMBIS, MACM
Doctor of Philosophy (Science) Candidate, The University of Melbourne
mobile: +61 4 22781753, +65 96669233
mailing address: Department of Zoology, The University of Melbourne
Royal Parade, Parkville, Victoria 3010, Australia
residence: 9/41 Dover Street, Flemington, Victoria 3031, Australia
resume: http://maurice.vodien.com/maurice_resume.pdf
www: http://www.geocities.com/beldin79/
begin:vcard
fn:Maurice Ling
n:Ling;Maurice
org:The University of Melbourne;Department of Zoology
adr:;;Gate 12, Genetics Lane;Parkville;Victoria;3010;Australia
email;internet:mauriceling@xxxxxxx
title:Ph.D. Candidate
tel;cell:+61 4 22781753
x-mozilla-html:FALSE
url:http://www.geocities.com/beldin79/
version:2.1
end:vcard
- Follow-Ups:
- Re: reading files with error
- From: Christian Stapfer
- Re: reading files with error
- References:
- reading files with error
- From: Maurice LING
- reading files with error
- Prev by Date: Re: socket.gethostbyaddr problem
- Next by Date: Re: complex data types?
- Previous by thread: Re: reading files with error
- Next by thread: Re: reading files with error
- Index(es):
Relevant Pages
|
