Testing and OO was Re: Is it possible to use the value of the PROGRAM ID within the source code?

From: Clark F. Morris, Jr. (cfmtech_at_istar.ca)
Date: 06/25/04 

Date: Fri, 25 Jun 2004 13:49:39 -0300

Joe Zitzelberger wrote:
> In article <cbenk8$nt2$1@peabody.colorado.edu>,
> "Howard Brazee" <howard@brazee.net> wrote:
>
>
>>On 24-Jun-2004, riplin@Azonic.co.nz (Richard) wrote:
>>
>>
>>>So, how do you think OO overcomes this objection ? Why would these
>>>shops suddenly say: "we won't have to retest all invokers" ?
>>
>>A "feature" of OO is that once a shop moves to OO, it is impossible to do the
>>same rigorous testing that were the earlier standards, so the standards get
>>changed.
>
>
> Not impossible, unnecessary...

Given that testing in all too many shops is less than adequate now, I am
skeptical that testing will become unnecessary when things are
structured by OO. I suspect that the problem that fouled up my account
at the Royal Bank of Canada for a week was due to a misunderstanding of
the scope of the effect of a change that ran for the first time on
Monday, May 31. This made national news and you can see their
statements at www.royalbank.ca. Note, I suspect this particular change
was to a batch non-OO program. The concern I have is program one in a
processing stream works as expected but program twenty gives unexpected
results because it didn't provide for the change in data caused by
program one. If an object is changed for performance reasons and all
inputs and outputs are kept the same (exactly), then testing can be
confined to the object. If it is changed to accept new inputs leading
to new outputs or to provide changed outputs, then testing should be
done to test the effects of the change on most if not all invokers.
Indeed a running of the entire processing cycle in which the changes
occur may be necessary. An input that on day one causes a rejection or
error flagging can cause interesting problems when it becomes legal on
day two. User changes to control tables used in packages can have
similar effects so the problem is not unique to traditional or OO code.
  There is a reason that many server environments be they IBM mainframe,
other mainframe, Unix, Windows, etc. phase in operating system and
middleware changes through systems test, test, and systems assurance
before putting them in production. As the number of interactions
increase the probability of having an unexpected side effect of a change
increases.

This does not say that OO or CALL prototypes have no effect on
reliability. The enforcement of expectations being met is of great
value in catching sometimes hidden errors. The provision of
repositories along with better documentation can cut down on the errors
in the first place. Indeed, moving control and modification of the
system to end users may pose a greater threat than doing less testing of
invokable OO modules.

It is interesting that the last three shops I have been in used the
DYNAMIC bind option in batch which meant that subroutines were bound at
run time. In CICS subroutines are either bound to the caller when
called by literal or at run time when called by data-name. CICS has a
large amount of interaction between separately compiled modules
involving data areas and transient data that have varying degrees of
complexity. In many cases the testing may not be adequate because of
people not understanding the variety of control flows.

I am grateful that the Royal Bank had controls in place that told them
something went wrong. As environments become more fluid, provision of
appropriate controls and checks becomes more vital. The use of OO can
be helpful but we have to understand the full scope of a change.
Managing change and verification of reliability has not been given
proper respect in many shops. Indeed one of the largest areas of
vulnerability is user control tables. There many times is very little
consistency checking of the values entered and the tools available to
those responsible for the tables are meager at best. Documentation may
be less than perfect. OO can provide benefits that more than offset the
execution cycle overhead just as Windows XP provides significant
benefits over Windows 98 in terms of reliability at the cost of more
cycles being spent verifying the validity of system requests. Note you
can use z/OS versus MVS/XA etc. as the comparison because what we have
seen is reliability and security slowly added to our environments at the
cost of validity checking at the boundaries.

Relevant Pages

  • Re: A question about graphic I/O
    ... Is it the reluctance of vendors who sit at standards ... Cobol will control conversions well even if it is less ...
    (comp.lang.fortran)
  • Re: status of Programming by Contract (PEP 316)?
    ... transport aircraft or jet fighters? ... such control systems that run on aircraft themselves, ... implied aspersions against Python's reliability. ... career was leading and coordinating pilot projects in Eiffel for one ...
    (comp.lang.python)
  • Re: type 1
    ... have stated, control is the key; a cure isn't in the cards, yet. ... Accuracy, reliability, and biocompatibility haven't gone together so far. ... Which ADA presented technolgy did you see? ... Not wanting to throw money down ratholes is reasonable: throwing money at it hasn't worked yet, ...
    (misc.health.diabetes)
  • Re: Cirrus Death Trap?
    ... bonkers at night over the Atlantic in IMC during a ferry flight to ... and not fly anything that gives glass avionics ... disconnecting that control. ... does not impact the reliability of aviation electronics. ...
    (rec.aviation.piloting)
  • Re: Ignoring the ASP.NET conventions...
    ... MS appears to have opted to effectively shaft web standards with this ... have full control over it. ... the purview of a technology. ... as "desktop applications") when compared with x/html elements. ...
    (microsoft.public.dotnet.framework.aspnet)