Re: Art of Unit Testing

phil hunt wrote:
On Tue, 02 Aug 2005 21:26:28 +0200, Christoph Zwerschke <cito@xxxxxxxxx> wrote:
According to the "extreme programming" paradigm, testing should be done several times a day. So a requirement for extreme programm is that tests are fast enough. If the testing needs too much time, people are discouraged to test often.

Indeed. Running the tests should ideally take less than a few seconds. Any longer, and people won't use them so often.

That's probably too general a statement to be very useful, and I don't think it's probably helpful to a newcomer who is seeking guidance.

If one makes the XP-style distinction between "unit" and "acceptance" tests, then the acceptance test *suite* should take no more than some number of minutes. I believe ten minutes absolute max is a commonly offered rule of thumb, but I've found even eight minutes a little too long. Five seems to be just fine, and less if of course wonderful.

For unit tests, the entire suite should take only a minute or two, max, if possible, and preferably well under a minute. If you're doing test-driven development (TDD), you will want to be able to run unit tests (though maybe not all of them) sometimes a couple of times per minute! If the overhead is fifteen seconds, you can't type much new code in between running the tests.

Still, in a large project (and especially one written in Python, with the overhead of interpreter startup and the cost of executing bytecode) the suite can get fairly long if you have many hundreds of tests. In that case, there's nothing that says you can't run individual test files, or even individual test cases, when you are focused on one small area of the code. That should let you run tests in only a couple of seconds, as phil just recommended, in almost any case.

Having your code organized nicely into packages can help as well. If you have a helpful "test runner" utility which scans subdirectories for test cases, you can easily run all the tests in a given package independently of the rest of the app while you are working on that one package, reducing the risk inherent in running only a subset of your full test suite.

-Peter
.

Relevant Pages

  • Re: sun acquires MySQL
    ... competition to MS Office as a full suite. ... Only if it is packaged to package everything in a database (data, ... ..the Sin of Ignorance, ...
    (Fedora)
  • Re: Will Microsoft Works run...
    ... Microsoft word and works are 2 different things. ... Works is a package that includes several programs where word is ONE ... both the WORKS suite and the OFFICE suite. ... which is the Word selection and the Works ...
    (microsoft.public.windowsxp.basics)
  • Re: word 2003
    ... of Works Suite 2003 or 2004, you use the code from the Works Suite package. ... > versions of microsoft word, ... > place i bought it cant find the package the worksuite ...
    (microsoft.public.word.application.errors)
  • Re: Latest Dolphin patches have broken Unit Test running of individual suites
    ... It appears to me that there are two issues: inherited tests went missing ... a separate suite. ... There was a bug in the original D6 release in that the SUnit Browser shown ... support browsing tests by package (i.e. to allow a TestCase to have loose ...
    (comp.lang.smalltalk.dolphin)