Re: ZoneView or ways of dealing with large objects?


"Mark Space" <markspace@xxxxxxxxxxxxx> wrote in message news:44A372C3.4040201@xxxxxxxxxxxxxxxx
Oliver Wong wrote:

"Mark Space" <markspace@xxxxxxxxxxxxx> wrote in message news:FUAng.28746$VE1.18652@xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Hi all.

I've got a program which displays potentially very large files. Files larger than 250k or so cause Java to run out of memory. I'm building a bigger view based on the file (converting binary to hex) so this view object just over one megabyte when the JVM poops out.

Is there some standard way of dealing with this? I'm looking at ZoneView, but there are really no instructions or examples available. Does anyone have a pointer to one? What about any kind of View, I haven't been able to locate any of those either.

I'm probably going to implement this manually because I only have one scrollbar to worry about, but I'd like to extend my knowledge of Java so any help you can give would be great.

Are you sure this is where the problem lies? I've written Java programs which load text files in the hundreds of megabytes into memory without too much trouble (assuming I give Java enough memory, e.g. with "-vmargs -Xms256m -Xmx512m"). Are you developing on a memory constrained device?



Well, not 100% sure, but it works for small files, and generates a OOM error for large ones. I'm working on a PC, under Netbeans 5.x, and it's not particularily memory constrained. I only have 256Mb of memory, but I don't think that's holding the JVM back.

By memory-constrained device, I meant like embedded processors with 256Kb of RAM.


I'm also doing a LOT of string manipulation. I'm building a JTextArea object basically one and two characters at a time. So the heap is likely about as full of garbage as it is likely to get.

Are you using the StringBuilder or StringBuffer classes for your string manipulations?


I thought about asking the JVM to increase it's memory size, but that didn't seem like a scalable solution. Plus I might have to ask users to do the same. So I'm looking for something scalable, something that'll work even when I point it at a 10 Gb file.

Have you considered memory-mapped files?

- Oliver

.

Relevant Pages

  • Re: ZoneView or ways of dealing with large objects?
    ... Are you using the StringBuilder or StringBuffer classes for your string manipulations? ... But I'd still like a more scalable solution than relying on memory manipulation. ... What happens when I try to display a file that's bigger than main memory + swap file? ... I plan on doing some heavy graphics work in the future so I'm interested in solutions for very large file sizes. ...
    (comp.lang.java.help)
  • Re: forth and virtual memory
    ... too, maybe even the same order, so ordering the blocks by allocation ... on systems with too little memory ... What Java is known for, and what it actually does, are two distinct ... My measurements indicate that some of the benchmarks (from SpecJVM98, ...
    (comp.lang.forth)
  • Re: What is the fastest method of parsing scheme?
    ... These issues can be eliminated by the use of custom memory allocators ... Any other ideas why Scheme would be faster than C++ and Java for heap ... For example, in my compiler, the procedure ) ...
    (comp.lang.scheme)
  • Re: JVM/Java memory footprint
    ... I found that if I use Java for developing the CLI ... application I will be exhausting the memory of our Application Server ... Just to mention the architecture,users use ... what WHAT specific REASON do you have to make ...
    (comp.lang.java.programmer)
  • Re: Function pointers (Callback functions) in Java ?
    ... > allocate a memory pool from which I can request memory slices as and ... Java however, does not support structures like C does. ... This is a data centric solution, ... So for a callback, you cannot use anything other than ...
    (comp.lang.java.programmer)