Re: teaching a child - console or GUI

From: Marco van de Voort (marcov_at_stack.nl)
Date: 07/28/04 

Date: Wed, 28 Jul 2004 08:52:49 +0000 (UTC)

On 2004-07-27, J French <erewhon@nowhere.com> wrote:

><marcov@stack.nl> wrote:
>
>>> Could you replicate such an interface for your 'pet' ?
>>> (ANS: undoubtedly)
>>
>>No. It doesn't perform. And it is not a pet.

> Surely, you are dealing with a lot of historical data, so it is
> possible to set up some pretty fancy 'accelerators'
>
> What bit of it does not perform

Unfinished sentence on my part, I think. I think I meant that the
application doesn't perform int 21h's, or any other legacy technique.

> - or rather what is it that makes it slow if you keep most of the data
> on disk ?

Filters/queries over 6 million objects (in 8 entities/tables) must be in an
acceptable time. Multiple users might run queries at the same time, but not
many (say 1-4 users)

Also note that 6 million objects are a lot more lines in a RDBMS, because you
have all kind of coupling tables for 1 to many relations.

The use of indexes is limited, or you really need a lot of indexes.

>>> Try it and kick the data set up to 8Gb and see what happens
>>
>>That happens in 2020. 300MB/year
>
> Will they be interested in 2001 data in 2020 ?

No, 5 years max. And even that is unlikely. They probably could do with 2,3
years real data, and global stats from the other 2,3 years. But it is not
worthwhile to code that.

However that 300MB/year figure and the five year figure is the
current situation.

We didn't know the exact sizes and date ranges yet when we made the
decision. At the time we were afraid of hitting the Delphi limit of 2-3 GB.
It was more 780MB/year in the initial version, and application memory
on top of that. Improved indexing, and packing some data decreased the
size of the data.

>>> Try abstraction, if only to prove it does not work.
>>
>>Been there, done that, got the t-shirt, and it is already pale.
>
> Yes, well as we get older, we get craftier
> Maybe it is time to look at it again

I had to be convinced too. (by my collegue). But now I have done a few projects with the mentality,
I wonder why I never saw it myself.

We are still thinking in a DOS way about memory I think. Memory as precious
resource. It is a commodity now.

> Something like File Mapping ?

Nope.

> Realistically I have things that should really be re-written, but I
> defend my not doing so on the grounds of
> - why upset it
> - its working so the users will not benefit
> - I'm idle
> - I'm not interested
>
> I can seldom point to anything and say that it cannot be improved, or
> rather 'future proofed'.

I would not try to retrofit such system to a RDBMS based solution, unless
you have severe (as in magnitudes) performance problems. The way one works
is too different. It is more meant for new development.

Relevant Pages

  • Re: teaching a child - console or GUI
    ... I use a RDBMS to reuse existing code and optimizations. ... >>And loading them in a system under load (with constant disk io) is worse ... quality stuff), and quality power supply, quality memory etc. ... > lists of pointers on disk, extract files of frequent search fields in ...
    (comp.lang.pascal.delphi.misc)
  • Re: Populating a dictionary, fast
    ... Starting at Sun Nov 11 10:46:37 2007 ... Completed deletion at Sun Nov 11 11:16:23 2007 ... it's a memory issue, or maybe a garbage collection issue, not ... I've tried three disk-based implementations already (BerkeleyDB, cdb, and an RDBMS) ...
    (comp.lang.python)
  • Re: OO and RDB war : in the advent of MRAM
    ... (Magnetoresistive Random Access Memory, non-volatile computer main ... Note that (at least in in the RDBMS jargon) integrity has nothing to do ... "customer" table. ... navigational because it appears to have no or too few consistent rules ...
    (comp.object)
  • Re: Three cat fambly..
    ... fancy a pint on saturday in memory of ... old friends or summat? ...
    (uk.rec.sheds)
  • Re: Application logic and Business logic
    ... The BTree, for example, became widely used in the implementation ... >already a lot of research on alternative implementations that can take ... >implementation of an RDBMS, not about the services that an RDBMS ... Main memory RDBMS's are a hot camp. ...
    (comp.object)