Re: Massive Memory Structures
- From: ruu@xxxxxxxxx
- Date: Fri, 15 Jun 2007 15:59:40 -0700
On Jun 15, 2:28 pm, jgra...@xxxxxx wrote:
r...@xxxxxxxxx writes:
No-one knows? Not even a "if you have to use more than 4G of memory,
you don't know what you are doing" reply?
Dave
On Jun 13, 4:44 pm, r...@xxxxxxxxx wrote:
OK - I was under the impression that perl running on a 64 bit OS,
compiled for said 64 bit OS would be able to handle data structures in
memory greater than 4G in size. I've tried this with a couple of OS's
(Solaris on sparc, Solaris on x86_64) with both packaged and compiled
versions of perl with identical results - as soon as I hit 4G with my
script, I get an out of memory error and the script dies.
I'm pretty sure I don't have a ulimit issue, since more or less
everything is unlimited, and other programs run by the same user can
access over 4G of ram quite happily.
Two questions:
1) Is it possible to access a structure over 4G with perl running on
Solaris?
2) If so, what options do I need to compile in to make this happen.
Dave
Possibly if you follow the steps in the posting guidelines,
such as posting a runnable short program that demonstrates the problem,
your post will both overcome the work-threashold-limit
and also pass the "sounds interesting" threshold
so that one of the few people here who have not only
the knowledge, ability, experience, but also the resource (4G solaris machine)
to check it out on their system, and if it works, to post their configuration.
Our workgroup has been moving away from Solaris boxes
and toward Linux running on AMD Opterons, for 4G/8G/16G memory setups,
so even if you posted an example, I wouldn't have the ability to try it on
a suitable Solaris machine.
--
Joel
OK. Sounds fair. If you feel like running this on a Linux system, I
would be interested to know if it works or not, even if it isnt
Solaris.
Below is a short script that works under Solaris (and probably
anything that has mkfile). It will create a 5g test file, and then
attempt to pull the whole thing into $bigvar. You will need to have
enough space somewhere to create the 5G file, and at least 8G of RAM
to attempt. Please, anyone reading this, DO NOT RUN THIS SCRIPT IF
YOU ARENT SURE WHAT IT WILL DO, OR ON A PRODUCTION SYSTEM - there is a
reasonable chance that your OS may fail in exciting ways if it uses up
all of the memory. Further rules:
1) Please dont run the script if you manage the safety systems of a
nuclear power station.
2) Please dont run the script on anything labelled "life support"
3) If you work in a lab, have played "Far Cry", and thought "This
looks familiar" at any point during the game, please dont run the
script.
#!/usr/bin/perl
$filename = shift || die("Need to be passed a test file.\n");
`/usr/sbin/mkfile 5g $filename`;
unless (-s "$filename") {
die("Failed to create testfile $filename.\n");
}
open(FILE,"$filename");
$bigvar = <FILE>;
close FILE;
print "I successfully read in the test file.\n";
unlink($filename) || die("Failed to remove test file $filename.\n");
Dave
.
- Follow-Ups:
- Re: Massive Memory Structures
- From: Ilya Zakharevich
- Re: Massive Memory Structures
- From: Dr.Ruud
- Re: Massive Memory Structures
- References:
- Massive Memory Structures
- From: ruu
- Re: Massive Memory Structures
- From: ruu
- Re: Massive Memory Structures
- From: jgraber
- Massive Memory Structures
- Prev by Date: Re: Perl script to identify corrupt mbox messages?
- Next by Date: Re: Massive Memory Structures
- Previous by thread: Re: Massive Memory Structures
- Next by thread: Re: Massive Memory Structures
- Index(es):
Relevant Pages
|
