Re: how many bugs do you find and correct during TDD?

From: Steve Jorgensen (nospam_at_nospam.nospam)
Date: 12/31/04 

Date: Thu, 30 Dec 2004 23:54:28 GMT

On Thu, 30 Dec 2004 15:37:35 -0800, Tammy <qqq@ppppp.com> wrote:

>
>
>Steve Jorgensen wrote:
>
>> So, let's say the avergage test/fail/pass cycle is 10 minutes. In that time,
>> first you write a test you expect to fail. Perhaps, it does fail, and perhaps
>> it doesn't. If it is not as expected, is that a bug, an imporperly written
>> test, or is that a misunderstanding of what the code's current untested
>> capabilities are. A little bit of digging is done, and it is found that the
>> test was not written properly. Is that a bug?
>
>Yes.
>
>> Next, the test is corrected,
>> and now fails as expected. That's not a bug because we expected it, right?
>
>Agreed.
>
>> OK, now we do the simplest thing that could possibly work to make the test
>> pass, and the test still fails (or another test fails). Is that a bug? I
>> dunno - we didn't know whether to expect the test to pass or fail because we
>> were experimenting to see if the simplest thing actually did work.
>
>If you expected it to pass and it didn't then that is a bug.

Then you must define "expected". We weren't sure what to expect, we were
trying something that might work, and using the test (and our existing suite
of prior tests) to answer the question of whether it did or didn't.

>> As you can see from this pattern, TDD is not so much about finding bugs before
>> they go into production as setting up tests that ensure code that passes them
>> functions as designed, then making the code meet the tests. I would say that
>> a bug, in this context is defined as something that got through our net either
>> becuase we forgot a test, wrote an inadequate test, we misunderstood the
>> request, or the request itself was the result of a misunderstanding of the
>> problem. How many bugs does TDD catch then?
>
>I am trying to figure out the error rate of a human programmer as they
>work. So only classifying a bug as something that slips the net doesn't
>accomplish that goal.

The concept of "error" is so manied and various that I don't see how it's even
meaningful. In the case above, you said that when the test passes and
shouldn't, that's a bug. In my mind, though, when you type a few lines,
something's wrong, and you see it and fix it that fast, that's simply the
process of thinking and writing code. A bug is something that's wrong after
you passed the point where you declared it to be "right".

Making and fixing errors while coding can happen so fast you don't even think
about it. If someone is more careful and avoids that bug, but it takses them
10 minutes to do vs my 5 seconds to try the test and find out, which is
better? TDD allows me to do more experimenting and less analysis, so I may in
fact type more "bugs", but write more reliable code faster. So - what is it
you hope your measurement will tell you?

Relevant Pages

  • Weekly Python Patch/Bug Summary
    ... Patch / Bug Summary ... new exit and quit objects ... Python memory allocator: Free memory ... tkinter Dialog fails when more than four buttons are used ...
    (comp.lang.python)
  • Weekly Python Patch/Bug Summary
    ... Patch / Bug Summary ... http://python.org/sf/1126187 opened by Philippe Kirsanov ... Fix to allow urllib2 digest auth to talk to livejournal.com ... gensuitemodule.processfile fails ...
    (comp.lang.python)
  • Weekly Python Patch/Bug Summary
    ... Patch / Bug Summary ... http://python.org/sf/1457736 closed by loewis ... http://python.org/sf/1580726 reopened by gbrandl ... Bulding source with VC6 fails due to missing files ...
    (comp.lang.python)
  • Re: Exception raised when drawing many circles
    ... at Microsoft.DirectX.Direct3D.Line.DrawTransform(Vector3vertexList, Matrix transform, Int32 color) ... Direct3D9::Vertex buffer must be unlocked during DrawPrimitive call ... I don't quite follow your response to Olivier, but I fail to see how this behavior could be anything but a bug in the Line class. ... I'd guess what's going on is that the Line class or something it wraps makes some assumption about a maximum buffer size and fails to reallocate properly when a request is made to draw lines that require a larger buffer. ...
    (microsoft.public.win32.programmer.directx.managed)
  • Re: how many bugs do you find and correct during TDD?
    ... If it is not as expected, is that a bug, an imporperly ... No as we haven't finished the cycle. ... and the test still fails. ... How many bugs does TDD catch then? ...
    (comp.object)