Re: Transaction Oriented Architecture (TOA)

H. S. Lahman wrote:
Responding to Gagne...

<snip>

I sense, perhaps incorrectly, a disparaging view of CRUD/USER applications. You comments in both this thread and the thread "Relational Database & OO" seem to indicate an opinion that CRUD/USER applications are too simple to be representative of sophisticated OO designs.

Not disparaging. CRUD/USER processing is a major segment of IT and it is going to be around for as long as /people/ analyze gobs of data through pattern recognition. But CRUD/USER processing is not a good application for the OO paradigm because ...
<snip>

I guess I'm thinking we should design as simply as possible. For OO to shine does it really require something more flashy than CRUD/USER apps? If those apps are so simple to write then we should all strive to make all our apps CRUD/USER apps.

I can imagine darn few applications that aren't CRUD/USER. But perhaps that's because I think of everything as being a transaction. Before you do anything to /my/ database you had better have logged in first and had a valid session. Want to query the DB? You'd better have a valid session so I can record what you did and that query had better have a stored procedure to implement it. Want to change something? You'd better have a valid session so I can record that, too, with a procedure to match. What are you trying to do? It had better be a transaction I already know about so I can record it properly and make sure your session gets credit (or blame).

When my auditors show up they love me. I can tell them who did what when. My users love me 'cause I can tell them what something looked like before they made the change, last week, last year, or from epoch. If they need to correct something it requires another transaction--also attributed to the user that made the fix, what they did to fix it, and when they fixed it. If there isn't a transaction to fix what got screwed up we create one ('cause that won't be the last time that happened) or on rare occasions (and critical) my programmers fix it (but leave an audit trail in our bug-tracking system).

Our batch programs use the same transactions as our users do. My security system is implemented at the transaction level. A transaction processor cares little where a transaction comes from. It can be batch, web, ATM, phone, fat client, web service, or script. Doesn't matter.

Embedded applications may not fit this model, but I'm unsure 'cause I've never written one. Maybe they do and I'm thinking too provincially.

I submit that encapsulation of the database mechanisms is always necessary except for trivial applications. It is basic separation of concerns. The problem solution doesn't care if the data is stored in flat files, an RDB, an OODB, or clay tablets. The problem solution should not have to know about mechanisms like SQL query construction, optimizations like anticipatory caches, or encoding/decoding of dataset formats.
My goal is to make every program as trivial as possible--but not too trivial. <http://blogs.in-streamco.com/anything.php?title=programming_rules>

That may or may not be significant. Perhaps you can describe a non-trivial example?

<snip>
I am advocating inserting an interface that separates the problem solution from the persistence access mechanisms. There is nothing new in this. The existing RAD layered models already do exactly what I am advocating. Consider the classic model:

Presentation
--------------
Business
--------------
Data

In the RAD world the Data Layer actually has two pieces; one on the client side and the rest on the server side. That division just gets hidden because all the networking is hidden. At a minimum one has to link in a bunch of infrastructure modules into the application for the Business objects to be able to talk to the server. When the computing environment technologies change, one just links in a new set of infrastructure modules.

Our goal is to have something like this:

Presentation (where)
-------------
Business (why)
-------------
Transaction server/Security (who & what)
-------------
Stored Procedures (how)
-------------
DB (don't know what to put here)

Now some layered models take this a step further and have more layers. In such models the client-side and server-side Data Layers are explicitly separated. In that case the client-side Data Layer has the responsibility for mundane tasks like forming SQL queries and encoding/decoding SQL datasets. What I am talking about is analogous to that layer that decouples the Business Layer through an interface so the Business objects don't have to know about the specific mechanisms.
That's where I won't go with you. We had a little bit of SQL that snuck-through into production and it assuredly bit us in the *** as soon as we needed to make a minor change to the DB and discovered I need to ship a new program along with it because its SQL require changing.

For topmind, that's too-tight coupling.

Life has improved since that SQL was replaced with a stored procedure call whose implementation can change independent of the programs that use it.

<snip>

If I'm right in understanding what you're saying (and what countless vendors, analysts, and pundits sell, present, and report), this is exactly where our designs part company and where Transaction Oriented Processing (or TOA--but I dislike the word architecture when used with software) proposes an alternative, thinner, and simpler model.

It's thinner and simpler because it doesn't separate concerns. Failing to separate concerns means that one must touch the solution logic whenever something changes in the persistence realm and that is a potential maintainability problem.
You're not including your abstraction layer as part of your "solution logic." I'm curious if your abstraction layer is independent enough that when you change tables or column names in the DB there's nothing outside the DB to change?

[Though I would bet that the total executable code size will be less if one does encapsulate the persistence access concerns because of encoding invariants. That's because the same executable code works for all queries. So the more tables and joins needed, the more embedded code is littered throughout the solution if one does queries on a context-by-context basis.]

One needs to design the interface to the problem solution's needs. Thus it is the interface that needs to be replaced during large scale reuse (think: Facade pattern). To be successful the interface needs to be a pure message-based interface so that each side can map the message ID and data packet into its own unique view. That, in turn, means a consistent mapping of identity on each side of the interface.

The belief that there needs to be symmetrical mappings on either side of the interface assumes their needs to be a mapping in the first place. That's surprisingly similar to petitio principii, the fallacy of assumption more commonly known as "begging the question." TOA/TOP proposes (and I know I haven't gotten that far in the article) the database and its application domain stored procedures are the only persistence mechanism necessary, and that the benefits of a focused, single, data-permeable gateway between application and database far exceed the benefits of O/R mappings--regardless of abstraction--and that its lightweight appearance shouldn't be dismissed as missing heavyweight kick.

Sorry, but I don't follow this. What other persistence mechanism do you think is necessary in the approach I advocate? All I am doing is decoupling the problem solution from the persistence mechanism de jour.
We're both trying to do that, aren't we?

When I wrote that paragraph it made perfect sense to me. :-)

Weren't you advocating a 1:1 mapping between the DB and the application (even if through another layer)? That's the symmetry I was referring to. Believing that mapping between the DB and any abstraction layer be at or near 1:1 assumes first that a mapping exist and second that a 1:1 mapping is some kind of goal. The second goal assumes that first, and I believe the first is ill-founded.

Fortunately, that is usually easy for data, especially for paradigms like the RDM that are designed to provide generic identity. Half the work is done if the DBMS schema is available to the application. So one just needs a bunch of table lookups to map data packet elements into Table/Field identifiers to construct SQL strings. Those lookup tables get defined from external configuration data that maps the interface messages into the RDB schema.

The price of this is encode/decode of message data packets on each side of the interface. For a SQL RDB, that effectively means duplicated dataset encode/decode. Fortunately DB access is in milliseconds while encoding/decoding data packets is in microseconds so nobody is likely to notice anything except the developer's extra keystrokes, which don't count in the overall scheme of things.

It may be early in the AM (for me) but I'm not following what your talking about above.

Which paragraph is the problem? (I don't want to elaborate on both if only one is a problem.)
Actually, both (sorry). I don't understand what DB identity has to do with half the work or look-up tables. Going from that to encoding/decoding messages left me in the dust.

<snip>

At another level, I don't see how you can advise not embedding SQL in the problem solution without providing the indirection of a message-based interface.

Use stored procedures. No mapping necessary.

Alas, this is a Major Hot Button for me. I think stored procedures are one of the most abused mechanisms in IT. They are a maintenance nightmare if they are triggered by the DBMS or call one another. (I assume in your case they are triggered only by the application, but I think there are still potential problems.)
True--they are called by the application. Our system has only a single trigger. I don't like them--not explicit enough.

It depends on what is in your stored procedures. If they are devoted solely to accessing the database, then fine. The stored procedures are effectively providing a generic API to the persistence mechanisms. IOW, when your application invokes the stored procedure via a method call, it is just sending a message to the Data Layer (or my DB access subsystem).
Our procedures (I'm unsure if you'll think this is good or bad) are the lowest-level of our business logic. To update an account a procedure must be called with your current session ID, an account number, a transaction type (ie deposit, withdrawal), and a few other params. The procedure knows how to update the account table, the daily account balance table, the transaction history tables, validate the tran type, and a few other miscellaneous housekeeping functions. Things that any application that may want to execute an account transaction would have to do--regardless where they came from or what language they came from.

Our procedures, our about /how/ things happen--not why.

--
Visit <http://blogs.instreamfinancial.com/anything.php> to read my rants on technology and the finance industry.
.