Re: diff type operations in TCL

MH wrote:
Well.. That depends.. I guess doing a chunk in binary mode would be
reasonably fast. Does Tcl actually DO a memcmp on the results though, or
does it first convert them to some other representation in order to do a
binary comparison?

As for speed, well.. Doing Tcl_Gets on a channel (inside a C++ program) versus
fdreopen and fgets, for a 1 million line file adds a ~1.5 second penalty to
my parsing program.

I was thinking about handling the files as binary (so no character set
conversion, leading to the channel system managing its buffers using
memcpy instead of slower operations), using [read] (so fixed size chunks
which again encourage fast data handling), and using [string compare]
(which really does do memcmp; I've checked!) If the chunks are fairly
large (a few megabytes is reasonable on a modern machine) the overall
comparison should be quick. This is considerably at odds with what you
were proposing...

Indeed, what I'm proposing is exactly this:

proc filesEqual {file1 file2 {chunk 8388608}} {
# Test the "Duh!" case first
if {[file size $file1] != [file size $file2]} {
return 0
}

# Written to use 8.5a4 features...
set f1 [open $file1 rb]; set f2 [open $file2 rb]
# Otherwise, use [fconfigure $f1 -translation binary] here

while {![eof $f1]} {
# NB: the 'ne' operator is currently slower for this task
if {[string compare [read $f1 $chunk] [read $f2 $chunk]]} {
close $f1; close $f2
return 0
}
}
close $f1; close $f2
return 1
}

FWIW, that produces the following bytecode.

ByteCode 0x819b8c8, refCt 1, epoch 0, interp 0x815a130 (epoch 0)
Source "\n # Test the \"Duh!\" case first\n if {[file size $f"
Cmds 20, src 463, inst 163, litObjs 10, aux 0, stkDepth 4, code/src 0.93
Code 432 = header 100+inst 163+litObj 40+exc 28+aux 0+cmdMap 92
Proc 0x81b2348, refCt 1, args 3, compiled locals 5
slot 0, scalar, arg, "file1"
slot 1, scalar, arg, "file2"
slot 2, scalar, arg, "chunk"
slot 3, scalar, "f1"
slot 4, scalar, "f2"
Exception ranges 1, depth 1:
0: level 1, loop, pc 71-126, continue 128, break 137
Commands 20:
1: pc 0-31, src 38-106 2: pc 0-7, src 43-58
3: pc 8-15, src 65-80 4: pc 19-26, src 93-100
5: pc 32-47, src 152-174 6: pc 37-44, src 160-173
7: pc 48-63, src 177-199 8: pc 53-60, src 185-198
9: pc 64-139, src 206-422 10: pc 71-126, src 299-416
11: pc 76-97, src 304-353 12: pc 81-88, src 320-334
13: pc 89-96, src 338-352 14: pc 101-107, src 369-377
15: pc 108-114, src 380-388 16: pc 115-122, src 400-407
17: pc 128-133, src 215-221 18: pc 140-146, src 428-436
19: pc 147-153, src 439-447 20: pc 154-161, src 453-460
Command 1: "if {[file size $file1] != [file size $file2]} {\n "
Command 2: "file size $file1"
(0) push1 0 # "file"
(2) push1 1 # "size"
(4) loadScalar1 %v0 # var "file1"
(6) invokeStk1 3
Command 3: "file size $file2"
(8) push1 0 # "file"
(10) push1 1 # "size"
(12) loadScalar1 %v1 # var "file2"
(14) invokeStk1 3
(16) neq
(17) jumpFalse1 +12 # pc 29
Command 4: "return 0"
(19) startCommand 8 # next cmd at pc 27
(24) push1 2 # "0"
(26) done
(27) jump1 +4 # pc 31
(29) push1 3 # ""
(31) pop
Command 5: "set f1 [open $file1 rb]"
(32) startCommand 15 # next cmd at pc 47
Command 6: "open $file1 rb"
(37) push1 4 # "open"
(39) loadScalar1 %v0 # var "file1"
(41) push1 5 # "rb"
(43) invokeStk1 3
(45) storeScalar1 %v3 # var "f1"
(47) pop
Command 7: "set f2 [open $file2 rb]"
(48) startCommand 15 # next cmd at pc 63
Command 8: "open $file2 rb"
(53) push1 4 # "open"
(55) loadScalar1 %v1 # var "file2"
(57) push1 5 # "rb"
(59) invokeStk1 3
(61) storeScalar1 %v4 # var "f2"
(63) pop
Command 9: "while {![eof $f1]} {\n # NB: the 'ne' operator is "
(64) startCommand 75 # next cmd at pc 139
(69) jump1 +59 # pc 128
Command 10: "if {[string compare [read $f1 $chunk] [read $f2 $chunk]"
(71) startCommand 56 # next cmd at pc 127
Command 11: "string compare [read $f1 $chunk] [read $f2 $chunk]"
(76) startCommand 22 # next cmd at pc 98
Command 12: "read $f1 $chunk"
(81) push1 6 # "read"
(83) loadScalar1 %v3 # var "f1"
(85) loadScalar1 %v2 # var "chunk"
(87) invokeStk1 3
Command 13: "read $f2 $chunk"
(89) push1 6 # "read"
(91) loadScalar1 %v4 # var "f2"
(93) loadScalar1 %v2 # var "chunk"
(95) invokeStk1 3
(97) strcmp
(98) tryCvtToNumeric
(99) jumpFalse1 +26 # pc 125
Command 14: "close $f1"
(101) push1 7 # "close"
(103) loadScalar1 %v3 # var "f1"
(105) invokeStk1 2
(107) pop
Command 15: "close $f2"
(108) push1 7 # "close"
(110) loadScalar1 %v4 # var "f2"
(112) invokeStk1 2
(114) pop
Command 16: "return 0"
(115) startCommand 8 # next cmd at pc 123
(120) push1 2 # "0"
(122) done
(123) jump1 +4 # pc 127
(125) push1 3 # ""
(127) pop
Command 17: "eof $f1"
(128) push1 8 # "eof"
(130) loadScalar1 %v3 # var "f1"
(132) invokeStk1 2
(134) not
(135) jumpTrue1 -64 # pc 71
(137) push1 3 # ""
(139) pop
Command 18: "close $f1"
(140) push1 7 # "close"
(142) loadScalar1 %v3 # var "f1"
(144) invokeStk1 2
(146) pop
Command 19: "close $f2"
(147) push1 7 # "close"
(149) loadScalar1 %v4 # var "f2"
(151) invokeStk1 2
(153) pop
Command 20: "return 1"
(154) startCommand 8 # next cmd at pc 162
(159) push1 9 # "1"
(161) done
(162) done

Donal.
.

Relevant Pages