Re: DBMS and lisp, etc.

From: Chris C apel (chris_at_ibanktech.net)
Date: 05/21/04 

Date: 20 May 2004 17:30:47 -0700

"Will Hartung" <willh@msoft.com> wrote in message news:<2gv1ucF7c544U1@uni-berlin.de>...
> "Chris C apel" <chris@ibanktech.net> wrote in message
> news:69d0f981.0405180947.8f32f4d@posting.google.com...
> > 1) Data access migration. I think this is simple, though, as I can
> > write a data access layer specific to my application and implement it
> > for a RDBMS after I have a clear idea of my data structure. Are there
> > packages to generate such a layer based on CLOS classes, perhaps?
>
> The other problem is essentially enforing the Relational view of your data
> in a decidedly non-relational system. The reason this is important is that
> if you don't take the discipline now to view your data relationally for
> eventual storage in a DBMS, then you'll have more headache later when you
> only want to change the query and persistence layer.
>
> Also, try and think of "big" queries in your system, getting as much as
> practical in a single query rather than grabbing the nodes of your tree, and
> then lazily filling them in with queries to the child bits. That's fast on
> in memory systems, but will destroy performance on a DBMS. Round trips to
> the DBMS are Evil. SQL Joins are your friend.

That's a great point. I'm imagining a sort of data access layer where
I have around one function for every different time I access my data,
tailored to that specific instance. Otherwise, it's too easy to write
a bunch of data access functions that do small pieces and then try to
combine multiple calls to these methods. That's, as you say, a *bad
thing* when dealing with RDBMS's. But with (almost) one function for
each data access point, I force myself to think about how I can get my
data over the line in the most efficient way possible. Is this a good
design principle for DAL's in general?

> You don't need a "generic OO -> Relational" mapping system. Just a layer
> that seperates your data from the back end, and unless you have 1000 tables,
> you can write this yourself, by hand. By hand you can specialize your
> queries with a lot less complexity than using an OR mapper.

Agreed. I do have a question about data representation, though. While
it's tempting to just have references to related objects as sublists
in my data structure, in the DBMS they're going to be keyed and ID'd.
Is it OK, practical, and good practice to hide this in the data layer,
with the application code treating the references as transparent? I
would tend to think so, but, as I've hinted, either SLIME or SBCL in
combination with TBNL (and maybe Emacs, too) seem to have problems
with my circular references (regardless of *print-circle*, it tries at
random to print out my structure at the SLIME REPL). The alternative
would be to use a hash table and ID generator for my "tables".

This train of thought leads me to believe it would be difficult to
write a /generic/ data access layer that would be significantly less
complicated than the equivalent SQL in any case, except for
marshalling issues, and still perform acceptably under most real-world
conditions.

And all that seems like a shame, because data access seems, on its
face, to be something that could be macro-ized and library-ized to a
huge extent. For instance, in my day job, I find myself dealing with a
huge program (for what it is) whose job is simply to manipulate the
contents of five simple tables. Though there's probably a better way,
(especially in C#, with its datasets and SqlBuilders and whatnot) I
find myself writing a function that manually generates the SQL for
each changed row. And it's broken, too, because the INSERTs need to go
in order of table A, B, C, D, E, while the deletes need to go in E, D,
C, B, A, something I didn't realize until today. That means iterating
twice over each table-cache. I don't know that /I/ could do much
better if I were faced with the same task in Lisp. But faced with such
large amounts of repetition, one really does have to wonder how much
of it would be necessary with an optimal database library.

> Finally, you'll need some way to filter and order your data. As you can
> imagine, it's fairly easy to simply use hash tables to get basic references,
> but one of the features of the relational model is being able to query and
> order on arbitrary data elements, including those that are not indexed at
> all. Now, you may not feel you'll need to do that at all, but it's a shame
> to toss out a real benefit of the DBMS and design around it at this point.
> Kind of depends on your data model.

So you're saying what here? That my data access layer should include a
way to order the data that translates well into SQL?

> [..] Many [lisps] have the ability to dump, essentially, FASL files, which are
> normally used to store compiled code, but some Lisps can also save and load
> FASLs of any data structure. These load and save MUCH faster than "print" and
> "read",

Sounds like a cleaner way to do it, too.

Thanks for the points,

Chris Capel

Relevant Pages

  • Re: Circular dependency
    ... the business logic layer gets a request for an object. ... instantiates the object and passes it down to the data access layer. ... I expected to seperate each layer into different projects and reference ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: How to use typed datasets in 3-tier environment
    ... Data Access class and passing typed dataset to it. ... The typed DataSet objects don't need to know anything about the ... Check out the MSDN written Data Access Layer class, ...
    (microsoft.public.dotnet.framework.adonet)
  • Re: Circular dependency
    ... Then both the Data Access Layer and the Business Logic Layer ... I expected to seperate each layer into different projects and reference ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Do you use the new datasource controls?
    ... It's fine for my dad's website, but not for the enterprise architecture I'm building, which will eventually need to have a web service. ... The code in the button click should be quite deeper, it should be encapsulated in a data access layer and exposed via a rich business layer. ... You don't actually need a DAL anymore, you just set properties of the data source and the framework does the dirty work for you. ...
    (microsoft.public.dotnet.framework.aspnet)
  • Re: Wanted: TDataset Sybase SQL Anywhere connectivity, no dbExpress
    ... method of accessing Sybase ASA as well? ... And not all DBMS features may be represented by ODBC API. ... that data access components must "know" DBMS to work with it really well. ... but probably it offers only high level API to ODBC API. ...
    (borland.public.delphi.thirdpartytools.general)