Re: Out of memory with file streams



On Mon, 17 Mar 2008 15:13:44 -0400, Mark Space <markspace@xxxxxxxxxxxxxx> wrote:

Hendrik Maryns wrote:
The API could have been more informing over the memory implications of this backreferencing mechanism. The memory footprint is not even mentioned in the Javadoc of the reset() method.

"Reset" is also a terrible name for a method of an output stream. "Reset" normally means something totally different when talking about IO streams.

From collections, "clear" probably would have been better. Maybe "cleanUpMemory" would have been even better. And maybe using weak references would have been even better still. Don't make the user deal with the internal memory of an object.


Well, "reset" does reset the ObjectOutputStream back to the state it was initialized to: all objects are new, no class descriptors have been written, etc. Using reset() will decrease your memory overhead, but at the cost of increasing your data size (since the class descriptors have to be re-serialized).

Weak / Soft references would be slick, but there is an extra "quirk" that would have to be addressed. reset() will put a RESET marker in the stream in order for ObjectInputStream to recognize that the stream has been reset (at which point it will dump it's references). Even while ObjectOutputStream could use soft references to determine that an object is no longer referenced, you also have to notify ObjectInputStream whent it's safe to clear it's references to objects that will not be subsequently re-referenced later in the stream.

For simple objects you can get around all of this by using

ObjectOutputStream.writeUnshared() / ObjectInputStream.readUnshared(), but those will still graph references for the nested object you'll get in a tree structure :/

Anyway, hope that was interesting food for thought,

-Zig
.



Relevant Pages

  • Re: off screen plain
    ... created a surface (soon to maybe be a memory stream) of 1024x14000. ... reset wouldn't let me lock the texture anymore. ... What if the device is lost for sometime. ...
    (microsoft.public.win32.programmer.directx.managed)
  • Re: "cant find project or library" error has started appearing in VBA running on Word 2003
    ... Have you tried clicking on Reset on the Run menu in the VBE ... Is there any way to reset the references if the REFERENCES item is ... into the immediate window all is OK ... but this does not resolve the issue- Hide quoted text - ...
    (microsoft.public.word.vba.general)
  • Debugger not working
    ... I had to rebuild my A2K2 FE by exporting to an empty db. ... Reset all ... references. ... VBA Debugger doesn't seem to work now. ...
    (microsoft.public.access.developers.toolkitode)
  • Re: 2.6.22-rc3-mm1 - pppd hanging in netdev_run_todo while holding mutex
    ... Under 22-rc2-mm1, if my VPN connection got reset, ppp0 just quietly went away. ... it seems to end up wedged and waiting for references to ...
    (Linux-Kernel)
  • How to reset handheld programmatically?
    ... I need to perform reset on the handheld. ... Any references to how to accomplish this task. ...
    (microsoft.public.dotnet.framework.compactframework)