Re: Best way to deal with long loops?

• Previous message: Raptor: "Re: Best way to deal with long loops?"
• In reply to: Raptor: "Re: Best way to deal with long loops?"
• Next in thread: Raptor: "Re: Best way to deal with long loops?"
• Reply: Raptor: "Re: Best way to deal with long loops?"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Date: Sat, 15 Jan 2005 13:37:15 +0000 (UTC)

On Sat, 15 Jan 2005 01:29:50 -0800, "Raptor" <bogus@none.com> wrote:

<snip>

>> Try pumping it out to a TMemoryStream
>> - use SetSize first to about 5mb
>> - that should minimize mucking around extending memory
>
>I'm doing
>
>aFileStream := TFileStream.Create(aFile, fmCreate or fmShareDenyWrite);
>aFileStream.Write(TreeSettings, SizeOf(TTreeSettings));
>
>then looping throught the nodes with
>
>aFileStream.Write(NodeSettings, SizeOf(TNodeSettings));
>WriteStringToStream(aFileStream, aNode.Text);
>WriteStringToStream(aFileStream, aNode.Font.Name);
>
>> That way you will be able to figure out how much time is wasted on
>> thousands of disk writes
>
>That shouldn't be a problem for my own routine, as I write 50,000 nodes in
>1-2 seconds.
>
>You think I could speed that up even more? Jeeze Laweeze.

Yes - just try replacing the TFileStream with a TMemoryStream
- you'll get an idea
- all disk access is 'expensive'

>> Note: TFileStream is /not/ buffered
>
>How would you handle my code differently?

Same as you do it, but with a smarter descendant of TFileStream
- or maybe just bang it out to a TMemoryStream and then write that out
to a TFileStream

>> More below
>>
>> >I'm writing the following record, then the fontname and node
>> >text:
>> >
>> >type
>> > TNodeSettings = record // R/W for every node.
>> > nsLevel : integer;
>> > nsImageIndex : integer;
>> > nsOverlayIndex : integer;
>> > nsSelectedIndex : integer;
>> > nsStateIndex : integer;
>> > nsChecked : boolean;
>> > nsExpanded : boolean;
>> > nsSelected : boolean;
>> > nsAllowGrayed : boolean;
>> > nsReadOnly : boolean;
>> > nsData : pointer;
>> > nsNodeType : TDCNodeType;
>> > nsColor : TColor;
>> > nsState : TCheckBoxState;
>> > nsFontColor : TColor;
>> > nsFontStyle : TFontStyles;
>> >end;
>> >
>> >I seem to be writing a bunch of normal-sized blocks, with
>> >text and fontname and this record taking up about as much
>> >space as I'd expect, then a long line of apparent trash is
>> >thrown in. Yet I'm reading it perfectly, with all the node
>> >attributes working great.
>>
>> I am not sure about the 'trash'
>
>It was a figment of opening the file in WordPad. When I opened in in a hex
>vewrer I accounted for all the extra space.

Ha ! - never examine a binary file with a Text editor !
- so easy to get confused
I always use an ancient DOS 'sort of' Hex editor
- saves panic attacks

>> - my hunch is that it is the 'Inherited' savetostream
>> - not sure at all
>
>No inheritance. Writing these out from scratch. Very fast... I was just
>wondering why my file was 2+ times larger... now I know.

Reckon it is the Packed stuff ?
How interesting

>> Are you sure that your routine is really like theirs ?
>> I would have expected them not to store 'default' values
>
>Must not be. Theirs is 10 times faster than the stock TTreeView, but 1/10th
>as fast as mine, and I'm I'm storing ALL the no9de parameters.

I expect the stock TTreeView is storing data rather differently

>> Also :
>> TNodeSettings = PACKED record
>
>That may account for much or all of the difference apart from my writing
>font name out very node. Dunno. Forgot that PACKED directive.

Easily done

>> When writing records to file it is a bad idea not to specify 'packed'
>> as you land up writing filler bytes of garbage between valid record
>> fields
>> - a waste of space
>> - and very confusing when examining the data on disk.
>
>Thanks for the reminder. I think I have a handle on this now, but any
>suggestions for even more speed or fast space savings surely appreciated,
>JF.

I'll be really interested in hearing how you get on with a pre-sized
TMemoryStream

I wonder whether the records tend to be 'duplicated'
You could convert the Booleans to Bits although that will only save 4
bytes per record

nsData : pointer;
That should not really be stored - I think
- as it means nothing on disk

Realistically you should not need to store any data that is the same
as the last record, which could save you quite a lot of space.

• Previous message: Raptor: "Re: Best way to deal with long loops?"
• In reply to: Raptor: "Re: Best way to deal with long loops?"
• Next in thread: Raptor: "Re: Best way to deal with long loops?"
• Reply: Raptor: "Re: Best way to deal with long loops?"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]