Re: How to have unittest tests to be executed in the order they appear?

Matthew Woodcraft <mattheww@xxxxxxxxxxxxxxxxxxxxxx> writes:

Ben Finney <bignose+hates-spam@xxxxxxxxxxxxxxx> wrote:
Your test cases should *not* depend on any state from other test
cases; they should function equally well when executed in any
arbitrary sequence. Dependencies between separate test cases (e.g.
"they only work correctly when run in a specific sequence") means
you're not isolating them properly.

So a mode to randomise the test sequence would be nice to have.

Twisted has a "trial" framework that allows just such a mode
<URL:http://buildbot.net/repos/release/docs/reference/buildbot.steps.python_twisted.Trial-class.html>.

Unittest's behaviour (using alphabetical order) doesn't really help
to detect undesired dependencies (which might be bugs in the test
suite or bugs in the underlying code).

Agreed. It's just a matter of making a custom unittest.TestRunner,
though. Yes, a Small Matter of Programming which I haven't actually
done, but unittest doesn't make it difficult to do that.

But running tests in the order they appear is often helpful: you can
put the tests for basic stuff before the tests for advanced stuff,
and then if you suddenly get seventeen failing tests, you know that
the first failure is the best bet to investigate first.

Surely, since "suddenly" implies you changed one small area of the
code, that area of the code is the best place to look for what caused
the failure.

--
\ "I planted some bird seed. A bird came up. Now I don't know |
`\ what to feed it." -- Steven Wright |
_o__) |
Ben Finney
.

Relevant Pages

  • Re: [opensuse] runlevel daemon startup shutdown sequence
    ... Default-Stop are ignored in SuSE Linux, ... that one cannot dictate 'stop dependencies' using the LSB ... headers of the init scripts in SLES10* ... How does sles set the sequence? ...
    (SuSE)
  • Re: Task Hopping
    ... In Project they describe a physical relationship between the tasks such that the timing of the predecessor task either drives or controls the timing of the successor task. ... While your software developers may jump back and forth between tasks, there surely are some of those dependencies in the Project - you have to write a program module before you can test it, for example. ... Think not about how they want to sequence their work, instead approach it from the way they HAVE to sequence their work according to the project process itself - if you flowchart the process, diagramming the flow of materials and information through the project from start to end, the result will be a network diagram where each arrow represents a dependency link. ...
    (microsoft.public.project)
  • Re: Resource vs. Task Dependency
    ... > But a resource dependency is saying the that order and timing of the ... > sequence is determined by the availability of the required resource and ... > process logic - then leveling coupled with leveling priorities will ... > versus on driven by resource dependencies but I don't see how it could be ...
    (microsoft.public.project)
  • Re: Resource vs. Task Dependency
    ... but I'm still trying to grasp the advantage of a>> resource "dependency" the functions like a task dependency does. ... >> Obviously they must be sequenced but the kicker in my mind is WHAT> sequence? ... >> You say one would be able to see timelines driven by task dependencies>> versus on driven by resource dependencies but I don't see how it could be>> possible that they are different. ... Critical path scheduling got it's big break from>>> the ...
    (microsoft.public.project)
  • Re: How to have unittest tests to be executed in the order they appear?
    ... Dependencies between separate test cases (e.g. ... "they only work correctly when run in a specific sequence") means ... bugs in the underlying code). ... first failure is the best bet to investigate first. ...
    (comp.lang.python)