Re: Delphi 8 and future of EXE

From: Marco van de Voort (marcov_at_stack.nl)
Date: 05/04/04 

Date: Tue, 4 May 2004 09:11:22 +0000 (UTC)

On 2004-05-04, Bjørge Sæther <bjorge@hahaha_itte.no> wrote:

>>
>> Well, in my case, keeping a copy of a base corporate database in
>> memory, for immediate statistics to boss and customers, trend
>> identification etc.
>>
>> In memory delphi objects perform the query in a split second. Query in
>> Oracle DB takes tens of minutes.
>
> Well, you could do a kind of "foreseeing reporting", like: "Ok, it's 10:50
> and user AB is logged on. There is a 42% chance he will issue this query
> within 30 minutes. Let's run it now".

One can't. It is a datawarehouse kind of thing, people can pointy click
the query. Moreover, the memory is actually cheaper than the Oracle db,
so there is no other disadvantage than going out of memory.

There are a lot of approaches that could ease the pain a bit, but all are
more involved and thus more costly.

The memory solution is right through the middle, cheap, easy to build,
enhance and scalable (as in adding other kinds of queries and processing)

So we chose it, and invested some time in optimizing memory use, so that we
can add data another 1-1.5 years without breaking the 32-bit mem border without
compromise, and maybe a bit longer without. (for the core data)

If temporary memory (for queries etc) becomes a problem, then we will
already have to duplicate the box, and make a separation in query types.

However this is possible, so the 32-bits border problem is not accute, but
close.

> BTW, what kind of data and reporting is this about ?

Household trashcontainer emptying data. The logs are from the trashtruck's
board computer. Nothing spectacular, just everyday small business data
processing.

>>> A person typing 200 chars/min for 8 hours/day 365 days/year in 70
>>> years would fill some 2.5GB of plain ASCII text files. All Pascal
>>> code I've written within 10 years goes into my 128MB USB memory
>>> device.
>>
>> The data above _comes_ from plain ascii files. However they are
>> machine logfiles. And machines produce a lot of logs.
>
> Ain't that the problem with logs - either it's too little, or it's too
> much....;-)

Only data needed for the queries is absorbed and stored quite efficiently.

Relevant Pages

  • Re: Help! Difficulty understanding DB -> Object mapping
    ... > Optimizing things like SQL queries depends upon the specific RDB schemas ... > Virtual memory is cheap. ... > Prefer single complex queries to multiple simple queries. ... in the past you had to have a pretty good knowledge of how the query was ...
    (comp.object)
  • Re: FTS Performance in SQL 2005
    ... Can you post you query plans and the output of statistics IO ... SQL Server MVP ... Because i've set the MAX sql-server memory to 3.5 GB instead of 4.0 GB ... cost relative to the whole batch, ...
    (microsoft.public.sqlserver.fulltext)
  • Re: Help! Difficulty understanding DB -> Object mapping
    ... The bottleneck is getting it into memory as datasets; extracting particular information from the datasets in memory is the fast part. ... [I would add that whoever writes the subsystem needs to understand the real trade-offs, ... Most DBMS's have been able to give good performance with joins for some time, although in the past you had to have a pretty good knowledge of how the query was executed in order to achieve that performance. ... "Compiled" queries, say in stored procedures, are typically tokenized on the first invocation, and a query plan is computed based on the passed parameters for that invocation. ...
    (comp.object)
  • Re: FTS Performance in SQL 2005
    ... Because i've set the MAX sql-server memory to 3.5 GB instead of 4.0 GB ... its the pipe between the CPU and Memory that could be the bottleneck. ... and that you have a covering index on the columns in the query. ... Plan window will then give you a percentage query cost relative ...
    (microsoft.public.sqlserver.fulltext)
  • Re: FTS Performance in SQL 2005
    ... Now if I do a query directly to the field I would theoretically need: ... SQL Server MVP ... Because i've set the MAX sql-server memory to 3.5 GB instead of 4.0 GB ... cost relative to the whole batch, ...
    (microsoft.public.sqlserver.fulltext)