Re: Streams

Thanks for the help, although I have a question or three.

Right now, I'm writing from an Array directly to a Flash Drive.
So, by writing the array to a stream, compressing from the stream straight
to disk saves a lot of writing to the flash, by not writing the 6-10 megs to
a text file, then zipping, then deleting. The 6 meg file is less than 300K
when compressed. So I now have a single 300Kbyte file written.

Big Benefits, however, looking at the code below, it appears I have 4 times
the memory being used then I need.
The Original Array, Working Copy of the Array, String List, and then the
Stream.

The working copy of the Array is a snap shot of the original array. This
allows a background thread to process the copy and then allows the original
to keep on working and collecting data.

I feel after adding the Copy of the Array to the String List, then I need to
Free the Copy of the Array, then after adding the String List to the Stream,
Free the String List, then after compressing the file, I could free the
stream and compression component. This only keeps a couple of copies in
memory at any given time (Plus the original).

Now, with all this said, heck, instead of using this huge dynamic array with
86,000 records, Each Record containing about 100 Fields of Integers,
shouldn't I simply use a String List to begin with, since it will all be
string data later when written? Is a String List effecient enough to do
large amounts of data like that? I guess it is, we're using it above.

Or, Would it be better to use a stream from the get go? I could add a
record to a MemoryStream 1 at a time then process it when I need to.

Speedwise, and safest, which is the preferred method.

Richard


"Nicholas Sherlock" <n_sherlock@xxxxxxxxxxx> wrote in message
news:devplv$eii$1@xxxxxxxxxxxxxxxxxx
> Richard wrote:
>> I have a typical text file that I have been writing to disk. Afterwards,
>> I zip it up and delete the original.txt file.
>> Would I be better off writing it to a stream, then using compression
>> routines that support compressing streams then writing to disk. Seems it
>> would speed things up somewhat. We are talking 6 meg text files here.
>
> It would be much tidier, but not necessarily faster - Windows may never
> actually dump the file to disk during your current method.
>
>> Are there any simple examples how to write a text file to a Stream?
>
> That depends on how you want to build the file. I'd:
>
> list:tstringlist;
> stream:tmemorystream;
>
> stream:=tmemorystream.create;
> try
> list:=tstringlist.create;
> try
> list.add('Hello!');
> list.add('Line 2');
> list.savetostream(stream);
> stream.seek(0,sofrombeginning); //rewind before compressing
> docompress(stream);
> finally
> list.free;
> end;
> finally
> stream.free;
> end;
>
> Cheers,
> Nicholas Sherlock



----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----
.

Relevant Pages

  • Re: clearerr(stdin) not clear
    ... > float, short, double, etc. files, writing them to an array in sections ... and them writing them out. ... > processing, now I want to evaluate all the pixel information, and then ... alter where you're reading from in the stream. ...
    (comp.lang.c)
  • Re: Streams
    ... I'm writing from an Array directly to a Flash Drive. ... >So, by writing the array to a stream, compressing from the stream straight ...
    (alt.comp.lang.borland-delphi)
  • Re: File Streams
    ... >one line at a time to a disk file and making it comma delimeted at the same ... > I know I could store pointers for the positon I'm at in the array, ... >working array then dump it it may be faster and less hassles, or a stream. ... Generally I have found that the optimum speed is obtained by writing ...
    (alt.comp.lang.borland-delphi)
  • Re: Streams
    ... Would I be better off writing it to a stream, then using compression routines that support compressing streams then writing to disk. ...
    (alt.comp.lang.borland-delphi)
  • Re: Problem with a script
    ... a loop there becomes impractical. ... You still have them as uniquely named array indexes... ... writing the code twice will only ... reading your entire code and parsing it in their head, ...
    (comp.lang.php)