Re: Code correctness, and testing strategies

On Jun 3, 12:35 am, ja...@xxxxxxxxxxxxxx (Jacob Hallen) wrote:
In article <mailman.1539.1211660262.12834.python-l...@xxxxxxxxxx>,

David <wizza...@xxxxxxxxx> wrote:
[...]
That's why you have human testing & QA. Unit tests can help, but they
are a poor substitute. If the customer is happy with the first
version, you can improve it, fix bugs, and add more unit tests later.

The most important aspect of usnit testing is actually that it makes the code testable.
This may sound lik an oxymoron but it is actually a really important property. Testable
code has to have a level of modularity as well as simplicity and clarity in its
interfaces that you will not achieve in code that lacks automated unit tests.

I don't agree. To me the most important aspect of unit testing is that
it is automated. That is, the time you spend writing new tests you
reap each time you run your test suite.

The fact that writing automated tests has a beneficial effect on the
modularity of your code is a pleasant though hard to measure side
effect.

Automation should also be the most convincing argument for the OP. The
most evident liability of the approach he described is the need to re-
instrument his code all over again each time. Writing an equivalent
set of automated tests is likely to take him more, but the time taken
to write each test is time he doesn't need to spend anymore.

[...]
Another aspect that you are raising is the use of human testing and QA. I agree that
these are important, but every bug they discover is a failure of the developers and
their tests. Our testers can sail through a testbed in 30 minutes if there are no bugs.
Every single bug adds 30-60 minutes of testers time in order to pinpoint the bug
and supply the developers with enough information to locate and fix it. Add some
10 minutes to testing time on the next testbed to verify that the bug is actually
fixed. In my end of the world, tester time is not cheaper than developer time. It
is also a scarcer resource than developer time.

Moreover, hand performed testing takes the same amount of time each
time it is performed and doesn't enjoy the incremental aspect of
automated testing.

Cheers,
Nicola Musatti

.

Relevant Pages

  • Re: Possible Word 2003 bug
    ... provides it's Word object model specifically for the purpose of automation. ... consider this bug, because it does not work as it is supposed to. ... > G'day Garrett Manahan, ... > Steve Hudson - Word Heretic ...
    (microsoft.public.word.vba.general)
  • Re: Linux 2.6.21
    ... But that doesn't make bugzilla "the One Choice". ... If there is no automation, manual tracking is still better than ... We get everything that comes in emailed to us and we can respond by email and RT recognizes the responses being in the same thread and lumps them into the same bug ). ... It also has some cool features like "extract this into the FAQ" and there is a "FAQ" in RT that contains an autogenerated FAQ from what people have pulled out in that way. ...
    (Linux-Kernel)
  • Re: lots of quick tests that have very low cost but uncertain value
    ... It means "we don't know whether this test will reveal a bug or not". ... Here's a great example of a cheap test, arrived at through automation. ... the cost of the test has gone up from 6 ...
    (comp.software.testing)
  • Re: Found bug in SUM?
    ... Is this a know bug? ... Enrique Vidal Sanchez ... SUM makes the computation. ...
    (comp.soft-sys.matlab)
  • Re: dynamic type checking - a pauline conversion?
    ... > significant difference in bug production rate. ... Odd that your tests would never find a bug. ... > that I go faster when I write these unit tests because I spend less ... > time debugging and less time system testing. ...
    (comp.object)