Re: Best method for seeking to n lines before end of file [ implementation of tail ]

graemenewlands@xxxxxxxxx wrote:
Hi, thanks for your answers,

The criteria we have is speed - a routine maybe called several thousand
times a day on files that are between 1kB and 100MB in size.

I can pipe-open an external "tail" on a 100MB file, and read from it,
about 500 times a second. So I don't think speed is going to be a problem
with that method. (Of course, the size of the file is irrelevant,
as long as it is seekable. It is the size of the last 10
lines that matters, in this case 1670 bytes.) Of course, this was on a
modern machine with Linux and YMMV.

$ time perl -le 'foreach (1..1_000) {open my $fh, "-|", "tail foo.txt" \
or die $!; my @x= <$fh>}'

0.374u 1.278s 0:02.21 74.2% 0+0k 0+0io 0pf+0w


One of our limitations is that we're using ActiveState perl, version
5.005_03 - our servers have "stable" builds that fortunately includes
the mks toolkit that has an implementation of tail. Unfortunately, this
means that the modules Tie:File and the File: modules are not
implemented.

You don't want Tie::File anyway, it will be slow if you only want the end
of a large file. I got File::ReadBackwards to work on 5.004. All I had to
do was hack it to manually define SEEK_SET and SEEK_END.


I guess another method would be to seek backwards byte by byte and look
of 0D 0A pairs, just in case anyone's written a whole bunch of spaces
(or tabs etc) at EOL?

I don't quite understand this question. Do your people have a known habit
of writing a whole bunch of spaces or tabs at (before?) EOL? We certainly
can't answer that question for you. If so, what's to keep them from
writing any other kind of gibberish to the file?

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB
.

Relevant Pages

  • Re: NaNoWriMo
    ... Tabs, fortunately, are ASCII ... Deadlines - when I manage to convince myself of their necessity - work ... when I get back to fiction writing, I need to invent a new one. ... I write best when I've got a nice game to distract me. ...
    (rec.arts.sf.composition)
  • Re: quick question - yes or no?
    ... invested, say, three years in writing something, they might be left ... The big money goes to the ... interest in writing bunch of crap novels. ...
    (misc.writing)
  • Re: quick question - yes or no?
    ... invested, say, three years in writing something, they might be left ... The big money goes to the ... interest in writing bunch of crap novels. ...
    (misc.writing)
  • Re: quick question - yes or no?
    ... invested, say, three years in writing something, they might be left ... The big money goes to the ... interest in writing bunch of crap novels. ...
    (misc.writing)
  • Re: quick question - yes or no?
    ... invested, say, three years in writing something, they might be left ... the idea of investing time and effort on something that may ... interest in writing bunch of crap novels. ...
    (misc.writing)