Re: how many bugs do you find and correct during TDD?

From: Steve Jorgensen (nospam_at_nospam.nospam)
Date: 12/31/04


Date: Thu, 30 Dec 2004 23:23:44 GMT

On Thu, 30 Dec 2004 14:47:33 -0800, Tammy <qqq@ppppp.com> wrote:

>
>
>David Lightstone wrote:
>
>> Not without simulataneously keeping accurate records and being a very good
>> observer. Funny thing about that task, it is very very difficult to
>> simultaneously do something and monitor the process by which something is
>> done. The task exceeds my capacity and I suspect those who are reading this
>> newsgroup.
>
>Get a piece of paper and a pencil. For every bug mark a tick on the paper.
>Count them at the end of the day. I am sure that is in the capacity
>of someone who argued so well for incapacity :-)

No, probably not, particularly when doing TDD.

So, let's say the avergage test/fail/pass cycle is 10 minutes. In that time,
first you write a test you expect to fail. Perhaps, it does fail, and perhaps
it doesn't. If it is not as expected, is that a bug, an imporperly written
test, or is that a misunderstanding of what the code's current untested
capabilities are. A little bit of digging is done, and it is found that the
test was not written properly. Is that a bug? Next, the test is corrected,
and now fails as expected. That's not a bug because we expected it, right?
OK, now we do the simplest thing that could possibly work to make the test
pass, and the test still fails (or another test fails). Is that a bug? I
dunno - we didn't know whether to expect the test to pass or fail because we
were experimenting to see if the simplest thing actually did work. Now, we
look and see why that code failed the test, and we try something else that
turns out to work, and this small cycle is complete.

As you can see from this pattern, TDD is not so much about finding bugs before
they go into production as setting up tests that ensure code that passes them
functions as designed, then making the code meet the tests. I would say that
a bug, in this context is defined as something that got through our net either
becuase we forgot a test, wrote an inadequate test, we misunderstood the
request, or the request itself was the result of a misunderstanding of the
problem. How many bugs does TDD catch then? The number is the difference
between how many bugs are in the system you would have built without TDD and
the number of bugs in the system you wrote -with- TDD. Since we never wrote
the one without TDD (code that would look vastly different, by the way) and
counted the number of bugs detected in pre-production testing and in
production, we can never know that number.



Relevant Pages

  • coreutils-6.11 released
    ... 300 in gnulib. ... you can run this command ... -fR to copy a fifo or "special" file onto an existing file would fail ... [bug introduced in coreutils-5.90] ...
    (gnu.announce)
  • Re: embedded in strings
    ... This looks like a bug. ... out why the index is causing this to fail. ... | queries with vaues longer than the column being queried. ... |> | query analyzer if you want to see what I'm talking about. ...
    (microsoft.public.sqlserver.ce)
  • Re: TDD: Test-Driven Design or Test-Driven Development?
    ... > If you intent to get tests pass you'll succeed. ... > software with intention to make it fail you'll succeed again. ... "If you are curious, or code does something unexpected, or you receive a bug ... If they write a test that fails due to missing abilities, ...
    (comp.object)
  • Re: how many bugs do you find and correct during TDD?
    ... > first you write a test you expect to fail. ... If it is not as expected, is that a bug, an imporperly written ... > request, or the request itself was the result of a misunderstanding of the ...
    (comp.object)
  • Re: Possible shared mapping bug in 2.4.23 (at least MIPS/Sparc)
    ... > Don't blame the kernel - the kernel is only doing what the user asked it ... Surely it's better to fail the mmapon other archs ... It's a bug either way ... ... send the line "unsubscribe linux-kernel" in ...
    (Linux-Kernel)