Re: process a BIG string

I won't hit the memory limit because i only read about 10 to 64Kb at a
tim from the file, process it and output it to the other one.
I'm not sure that increasing the execution time will be the solution,
this thing should be able to process files of about 100-200Mb, and if i
need 30seconds for a 2-3Mb file, i can't imagine how long it will take
to process a 100Mb file. during this time the user won't get any
feedback...
I'm goint to take your solution into consideration, longer exection
time and a FAST server might do the trick...

Erwin Moller a scris:
quamis@xxxxxxxxx wrote:

Hy,
i need to process every character in a file, so i open the file read
in buffers of about 8192bytes and process each buffer, then i write
the output to another file.

the problem is that with large files(>8Mb) i get a script error(Fatal
error: Maximum execution time of 30 seconds exceeded ).
i acess every character in the buffer with
$chr=ord($bufferIn{$i}); (where $i=0...8192)
seems like all he time the script consumes is in the for loop and the
chr/ord functions.

can i do something to speed things up?
is there any other way of acessing a single characher except
$bufferIn{$i} ?

Hi,

I am not sure if you can speed things up in your script, but why not simply
increase the max_execution time?

This can be done with
ini_set("max_execution_time",60);
for 60 seconds.

Have a look here for more options:
http://nl2.php.net/manual/en/ini.php#ini.list

You might also hit the roof of your memory usage if you handle and process
very large file. In that case look at: memory_limit (defaults to 8MB)

Hope that helps.

Regards,
Erwin Moller

.

Relevant Pages

  • Re: for each file loop question on reading variables
    ... Is the thing that you want to do based on filename a control lfow thing or is it a data thing. ... In the Control Flow you may want to use the Script task. ... If what you want to do is simply make a FF Connection Manager point to this new file then you need to look at property expressions. ... Highlight the Conection Manager, hit F4, ind the Expressions item and hit the ellipses. ...
    (microsoft.public.sqlserver.dts)
  • Re: import magic
    ... Then I run this Funduc script to tidy it up. ... Before Hit=< ... Display Replace String=0 ... Min Date Filter= ...
    (comp.lang.java.programmer)
  • Re: Persistent failure of break statement ( novice )
    ... After you hit the zero key, do you hit the Enter key? ... script as a file, and run it from the command line. ... print('Value:', num) ...
    (comp.lang.python)
  • Re: Does fileinput.input() read STDIN all at once?
    ... all the lines, then wait for another Ctl-D, and so on (until I pressed ... There is probably a 1024 byte buffer. ... get something before you hit CTRL-D. ... Note that terminals (what you get if you are typing from the command ...
    (comp.lang.python)
  • BASH vs Expect
    ... me in charge of the unix boxes. ... and the internal server room. ... having said that - I must admit I'm script stupid. ... I've got a monitoring server that's internal that will hit the ...
    (comp.unix.shell)