Re: Software testing

From: Jason Chapman (jason_at_nospamjac2.co.uk)
Date: 10/23/04 

Date: Sat, 23 Oct 2004 08:31:43 +0100

"Jonathan Neve" <jonathan@!nospam!microtec.fr> wrote in message
news:4177eedd@newsgroups.borland.com...
> Hi everyone,
>
> I'm looking for a tool that will in some way facilitate / automate
> functionnal application testing. What would be your suggestions? What I
> really need is something that can understand something of the buisness
> logic, and make sure everything works as designed.
>
> I work in the realm of database application programming, so unit testing
> is to me of limited use. It's not so much the procedures, functions and
> classes I need to test, as the actual outcome, the GUI, etc. If a label is
> too short to display all the data, this is a bug. If a grid contains
> erroneous data, or lacks data, this is a bug. It's not just a matter of
> error messages (since this is usually the easiest kind of bug to detect).
>
> What I need, is a tool in which one can define all the buisness logic of
> the application, telling it exactly what should happen in each situation,
> and that can then try out every combination, in random order, a random
> number of times, coming at each problem from every possible angle, etc.
>
> What would you suggest? Is there a tool that at least comes close to my
> above specifications?

Jonathan,

Hello again - hope you are well.

I have looked into this a couple of times, both for my own developments
(similar to you 2 developers data centric apps on IB & Firebird) and for a
client I was mentoring (Larger DB apps with 100's of concurrent users).
>From my experience, teams want better quality, look around, evaluate a
couple of tools, then descend back to manual testing :-(, the learning curve
and set up time of putting auto testing around a 200 form application is
just too much to comprehend when there are dealines looming. From this I
think people look at auto testing too late in the development lifecycle -
anyway I digress.

I found that Mercury Load Runner et all and most of the tools are excellent
at testing things like web apps (for functionality and stress test), but I
was especially impressed by TestComplete. It has been top of my list of
"things I must purchase and use every day", for a while. It allows you to
write modular, delphi-esc, data driven scripts. I started writing modules
to log into my application, goto every client records etc. very easily. I
didn't (because you can't in the eval copy), push it to its limits (the app
I wanted to test has 100's of forms / frames), so I don't know how it fares
with test scripts made up of 50K lines and running for 8 hours.

One comment - I think that random testing will not be very useful to you, I
think that you want tests to be repeatable and that just by doing full
regression tests and being able to easily add new tests will yield best
results.

Issues I came up with were:
1) For an application that grows (we deliver a new build monthly), it is
difficult to maintain a test database (the actual data that the application,
is looking at), that is relevant and populated, without breaking tests, e.g.
if you use a copy of the live data and mrs migens changes her name and you
build a script around her, you are goosed.
2) (1) Leads you to create your own data for the test, e.g. your test will
centre around mrs migens, who you just inserted - which limits tests which
require a large volume of historic data.
3) The design, specifically to do with the modular nature of the test
scripts seems as fundamental as the design of your application, e.g. I have
a navigation tool in one of my apps which uses a treeview, this is fine, but
I am thinking of ditching it as the number of options per client is growing
too much. If I don't design my script to modularise the navigation, then
every script would have to be re-recorded if I changed to another navigation
metaphor.
4) The time to design the tests was going to be massive to be complete,
although I felt that I would get a good ROI on simple tests. We often add
functionality that will appear on every screen, which may break something
somewhere.

Testcomplete is still up there waiting for me to buy it and spend a month or
so designing / writing and running tests on one application that has taken
multi-man years to write. I think the ease of recording and then editing of
tests is great. If you eval it and bought it I would be interested in
pooling thoughts about it as this would reduce both our learning curve costs
and may deliver better scripts.

Just my 2c

Cheers,

JAC.

Relevant Pages

  • Re: Language improvement: Add scope to class member fields
    ... every small subset of functionality into seperate classes. ... MyMethodwould check this flag first within a lock and only run the method ... currently stands there is no protection. ... Allowing classes to become more complex is not a terrific design goal, ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: news of the week
    ... And as market leader, that company does not have the time to listen to ... When I designs a website, I constantly have to think about why ... didn't design the website for me. ... offers the same functionality, yet. ...
    (rec.games.computer.ultima.dragons)
  • Re: Is it possible to use the value of the PROGRAM ID within the source code?
    ... Mr Wagner, structured design, as I understand it, is functional ... object oriented design is closer to the idea of functional cohesion, ... If functionality is likely to change, ...
    (comp.lang.cobol)
  • Late Show Fashion #2510
    ... "I have little interest in design. ... I hate placemats that ... Okay all good design is based upon functionality. ... aesthetics, functionality, and comfort. ...
    (alt.fan.letterman)
  • Re: Open architecture automobiles
    ... >> machine by the functional design specs of the parts. ... So, how many kinds of car do we actually need, ... > which we may select our version of optimum? ... cost, functionality, status, life, reliability, maintenance costs, etc. ...
    (sci.physics)