Re: XML to inMemory Hash

Hi,

Yes its saving the memory.

But now am into a new problem :(

Its taking lot of time to parse a file.

Is there any way to create chunks so that it wont parse the entire file.

my requirement is to read the chunks based on an attribute.

I tried

my $twig = XML::Twig->new(twig_roots => {'node[@id="'.$ID.'"]' => \&chunks});
$twig->parse($showtimeString);

sub chunks{
my( $twig, $reOrganize)= @_;
$chunkedIXMLG = $_->sprint;
$twig->purge;
}
sub myprocess
{
print $chunkedIXMLG;
}

Also

my $twig = XML::Twig->new(twig_handlers => {'node[@id="'.$ID.'"]' => \&chunks});
$twig->parse($showtimeString);

sub chunks{
my( $twig, $reOrganize)= @_;
$chunkedIXMLG = $_->sprint;
$twig->purge;
}

Since it is reading the whole file each time to parse the node based on the ID.It is taking lot of time.I need to read a 100MB file each time to take the node.

Thanks in advance,
Prabu


Jenda Krynicky <Jenda@xxxxxxxxxxx> wrote:
From: Prabu Ayyappan

Thanks for the valuable inputs,

I am planning to solve my memory issue by making the XML into Chunks
using Xml::Twig and using simplify( ) in XML::Twig to convert that
chunks back into the Hash data structure.

Is that advicable method to proceed to reduce my Memory size.So that
after processing that chunk i will purge my memory (purge in
XML::Twig)

Thanks in advance,
Prabu

Yes, this should reduce the memory footprint while keeping the
necessary changes to the script small. I think in your case it's the
best solution.

Jenda
===== Jenda@xxxxxxxxxxx === http://Jenda.Krynicky.cz =====
When it comes to wine, women and song, wizards are allowed
to get drunk and croon as much as they like.
-- Terry Pratchett in Sourcery


--
To unsubscribe, e-mail: beginners-unsubscribe@xxxxxxxx
For additional commands, e-mail: beginners-help@xxxxxxxx
http://learn.perl.org/





---------------------------------
Ready for the edge of your seat? Check out tonight's top picks on Yahoo! TV.

Relevant Pages

  • Re: XML to inMemory Hash
    ... Is there any way to create chunks so that it wont parse the entire file. ... I am planning to solve my memory issue by making the XML into Chunks ...
    (perl.beginners)
  • Re: random writing access to a file in Python
    ... the size of available memory for the sorting operation (making it ... possible to work on larger chunks of data in memory) has less impact ... In the second run the size of processed chunks between reading/writing was in the order of up to tenths of Megabytes, where in the first run in order of up to hundreds Megabytes. ... decision about the size of buffers for merge sorting the chunks into the final file, so that they all fit into the 300 MByte of used memory ...
    (comp.lang.python)
  • Re: open a very big file
    ... Suppose the file had 50 rows and you had to find the sum of the numbers. ... to load data in chunks which fit in the memory ". ... If you still run into memory issues with the above, ...
    (comp.soft-sys.matlab)
  • Re: Efficient algorithm for finding duplicates in 56-bit number
    ... |) processor time) it would take to sort millions of those values ... and that'a a heck of a big IF (if it fits in memory). ... | chunk fits in memory) and then merges the chunks. ...
    (comp.programming)
  • block move clarification - System.Move()
    ... time, with chunks capable of being expressed in 8 bit, 16 bit and 32 bit ... I've been working on pooling memory and working out ways to make the memory ... each buffer already in existence. ... But I do not know ASM, which is why I am here. ...
    (borland.public.delphi.language.basm)