Dynamically-sized memory buffer?

From: Robert Mischke (rm888_at_yahoo.com)
Date: 01/16/05 

Date: Sat, 15 Jan 2005 23:21:54 GMT

I'm curious: Is there a common "best practice" to store binary data in
memory, when the amount of data is not known in advance? For example,
think of retrieving data from an URL, and there is nothing like a http
"content-length" field, and you don't want to use a temporary file.

My current idea is providing a container class (let's call it
DynamicMemBuffer) that uses a linked list of fixed-size byte[] buffers
internally. So a new DynamicMemBuffer starts with a single internal
buffer of size n. When n bytes have been added/written to the
DynamicMemBuffer, a new internal buffer is created, added to the
linked list, and writing of new bytes is directed into this buffer.
>From the outside, the DynamicMemBuffer looks like a continuous byte
stream held in memory. As an interface, standard getInputStream() and
getOutputStream() methods could be provided.

While all of this is quite simple to implement, I'm reluctant to do so
because I can't imagine this problem hasn't been solved a thousand
times before.

So are there better approaches and/or ready-to-use classes out there?

Thanks,
Robert

Relevant Pages

  • Re: ReadFile and Buffersize
    ... Imagine what happens if the buffer size is ... imagine that the maximal amount of memory the ... memory, necessary for reading a second megabyte from the file, write it ...
    (microsoft.public.win32.programmer.kernel)
  • Re: Continuously concatenating binary data
    ... If your user have small amount of memory or record large ... It would be no differents than stream approach. ... In fact stream give you better control on amount of memory ... >> beginning how large this buffer would need to be. ...
    (comp.lang.cpp)
  • Re: Help - Memory Leak Issue with SCSI Stack on Linux
    ... >the amount of used memory. ... disk buffer cache--shown in the "buffers" column. ... The disk buffer cache tends to grow soon after starting Linux up. ...
    (comp.os.linux.development.system)
  • Re: Discovering variable types...
    ... >- but I suppose MS expect us to use wrappers ... memory allocations for your variables from disk as well. ... >They most certainly are of fixed size, changing the size of a String ... >>me to keep buffer size and current postion right in the memory block. ...
    (comp.lang.pascal.delphi.misc)
  • Re: Discovering variable types...
    ... >memory it points to is on the heap. ... sequentially reading data, if one is randomly reading records, then a ... >project is what's prompting me to improve disk access. ... from a memory buffer I can do it in about a second. ...
    (comp.lang.pascal.delphi.misc)