Re: OO Software Project Entropy Question.

From: Phlip (phlip_cpp_at_yahoo.com)
Date: 08/10/04 

Date: Tue, 10 Aug 2004 06:18:08 GMT

AndyW wrote:

> Software Project Entropy often occurs when the rate of 'work' is high
> but nothing is being produced. When this occurs at the extreme level -
> the software project stalls - often resulting in a failed project.

Such a project should show trouble signs long before reaching that extreme.
I'm aware some corporate cultures enforce such oblivion, though...

> One of the reasons [the one I want to focus on here] is that software
> projects may focus on fixing defects rather than producing new code.
> This can occur when process and/or requirements are ill defined and
> bad code is produced.

Uh, there are those who think this happens when unit tests don't accompany
new code. With tests, ill defined requirements are less of a problem. Just
implement the defined ones, and rely on the tests to assist changes.

Development follows two nested cycles. Writing source code to implement
features is the inner cycle, and deploying versions to users is the outer
cycle. If feedback from tests drives the inner cycle, then feedback from
users can steer the outer cycle more frequently and accurately.

> Alternatively the software development method being used on the
> project may have a focus on mini-cycles where development is followed
> by brief periods of bug-fixing. This is performed usually in short
> incremental cycles.

Some software teams report _no_ bug fixing.

> If the code being produced has a low level of quality in that the
> number of defects being found during the micro test phases is high -
> the project may tend to spend more and more time fixing bugs. As this
> time being spent gets longer and longer, the time available to
> procuding new code gets shorter. Thus entropy is occuring on the
> project with the result being an eventual development stall.

Right. Now turn it around. Suppose you run tests after the fewest possible
edits - say 10 at the most - and you simply hit Undo when tests fail. Now no
code change can decrease code's ability to be tested.

> Questions
>
> I have 3 basic questions that i'd like to ask, but welcome any
> disussion on any other ideas/experiences people may have related to
> this topic.
>
>
> 1). As entropy occurs on the project, so the cost of the project
> rises. If the entropy level increases, then the percentage of the cost
> of the project that is incurred due to wasted effort also rises.
>
> Do people monitor this - either using metrics or some other technique
> - if so how (comment ?].

If a team implements features in small batches, such as 5 features a week,
and if over a few weeks that number goes down a little, the team and
features are probably maturing. But if it goes down a lot, and if many of
the features requests are really bug fixes, then something is wrong.

Sorting features in order of business priority goes a long way towards
ensuring the value of those feature are high during the start of a project,
when entropy may be high too.

> 2. The introduction of the newer more modern techniques for software
> development (i'm thinking of Agile/Extreme etc] means that the
> mini-cycles as mentioned above are more likely to occur.

Not as mentioned above. No bugs. ;-)

> What effect do these methods have on the levels of entropy in the
> software project - is it reduced or increased or is there no focus
> given to it - why ?

There are those who decide that "refactoring" is re-work. Suppose you plan
to refactor, and your plan succeeds. Suppose you are surprised how soon your
design stabilizes, and needs no new refactors. Suppose refactoring
statistically decreased the odds of current and future bugs.

Bugs are errors in behavior. Design Smells are errors in design - not bugs.
When you use Test-Driven Development, you write tests and then make code
pass them. That forces behavior to stabilize _first_. No bugs. Then you
refactor, while chronically passing tests, to remove the design smells.

In traditional programming, one designs structures, once, and then develops
until the behavior stabilizes. Here, one writes behavior twice (tests and
code), and then develops until the structure stabilizes.

Unstable behavior is bugs. With wall-to-wall tests, unstable structural
design is only a pleasant challenge.

> 3. Following on to question two - given the older techniques/methods
> that were in use before these methods came about [became mainstream] -
> is there a comparison that can be made.

Ask Ed Berard if there was agile before Agile.

You describe "non-Agile processes". Without Agile practices, software
engineering projects occupy a spectrum, from excessive planning to
undisciplined hacking. Agile projects rise above that spectrum. We apply
discipline to the good parts of planning and hacking. We carefully plan to
write many tests, until new features are as easy as hacking-without the low
design quality and high bug rate. Projects become useful early, and then add
value incrementally.

> I do not want to create a debate on the methodologies mentioned above
> - I just want to use them since many people are familiar with those
> that i have mentioned.

What's to debate? it's all just Software Engineering...

> Also, my background is in large software development (millions of
> dollars, thousands of people) - so if anyone else is familiar with
> that environment would like to comment that would be great. I
> suspect that it would be more relevant to looking at small and medium
> sized projects as I feel that most people come from that invironment.
> So for the purposes of this discussion - i shall choose the smaller
> scale projects.

The point of Agile processes is scalability. Anyone can write a recursive
descent parser off the top of their head in a day or two. What matters is
whether writing that module will cost a project, over the long term, more
than its value provides. If a new module is a bug breeder, the project's
quality will slowly go down over time.

I just wrote the following, a couple days ago, before your post arrived:

"When code experiences a series of changes, each leading directly to a
passing test, no change can reduce the code's ability to be tested.
Refactoring helps entropy favor robustness. New test cases become easier,
and each makes the code's features less likely to break during a change." 

-- 
  Phlip
  http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces

Relevant Pages

  • Re: SproutMethod.pdf
    ... Design Paterns book) (i.e. through Factories, ... Refactoring and the other techniques that I write about are in fact ... security and other essential features were lost on the ... > I know how to write good programs sometimes, using mathematics, but as you point ...
    (comp.object)
  • Re: creating methods for objects
    ... then converting it to a lower level language. ... As the Ruby code grew, I attended to its design, and refactored to ... While refactoring, I ran all the tests every ... only bugs I found were in the immediate feature I worked on. ...
    (comp.lang.cpp)
  • An engineering comment was: Re: Commentary: Challenging Darwin
    ... Christians devoted to promoting balanced science education in ... I observed ancient Greek statues I saw design. ... Then that machine, presented with a detailed description of itself, should be able to distinguish bugs from features in its own design. ...
    (talk.origins)
  • Re: AOP in PHP - Was: OO in PHP
    ... Now the amount of refactoring we do can in no way be called gold plating ... We don't get those kind of bugs anymore. ... left are design issues and requirements mismatch. ... > I am not saying that refactoring is bad, I am just saying that it should ...
    (comp.lang.php)
  • Re: Refactoring not happening
    ... Have you watched my BDNtv episode on refactoring? ... John Kaster http://blogs.borland.com/johnk ... Features and bugs: http://qc.borland.com ...
    (borland.public.delphi.non-technical)
Loading