Re: opening a big file



beyhan wrote:
The key is :

use Tie::File

"Tie::File" represents a regular text file as a Perl array. Each ele‐
ment in the array corresponds to a record in the file. The first
line
of the file is element 0 of the array; the second line is element 1,
and so on.

The file is not loaded into memory, so this will work even for
gigantic
files.

Changes to the array are reflected in the file immediately.


Richard Lee wrote:
I am trying to open a big file and go through line by line while
limiting the resource on the system.
What is the best way to do it?

Does below read the entire file and store them in memory(not good if
that's the case)..

open(SOURCE, "/tmp/file") || die "not there: $!\n";
while (<SOURCE>) {
## do something
}

sometime ago I saw somewhere it had something like below which look
like it was reading them and going through line by line without
storing them all in memory.
I just cannot remember the syntax exactly.

open(SOURCE, " /tmp/file |") || die "not there: $!\n";
while (<>) {
## do something

}




What you guys are all throwing at me is bit beyond my skillz... I will read them and get back to you guys.
.



Relevant Pages

  • Re: Fast string operations
    ... Looping: I thought looping over arrays in managed code was "slow" ... array handling and such. ... The problem with TrimHelper is that it always returns a new string instance. ... The customer perceives this as a memory leak. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: High Memory Consumption of Classes and Arrays
    ... Only the array itself has overhead. ... memory as a reference type. ... > least consume 40 bytes of memory. ...
    (microsoft.public.dotnet.framework.performance)
  • Re: OOP and C++ and C (was Re: Dennis Ritchie -- An Appreciation)
    ... different layout in memory. ... memory is one large array like structure. ... That's the fundamental abstraction that our hardware attempts to ...
    (comp.lang.c)
  • Re: Fast linked list
    ... > amounts of memory rather than huge chunks of it. ... random insertions into a vector/dynamic array are not as slow ... to cause a cache miss. ... some hard numbers on speed differences between lists and arrays. ...
    (microsoft.public.vc.mfc)
  • Re: Fast linked list
    ... > amounts of memory rather than huge chunks of it. ... random insertions into a vector/dynamic array are not as slow ... to cause a cache miss. ... some hard numbers on speed differences between lists and arrays. ...
    (microsoft.public.vc.language)