Re: Perplexed in a SW group..

From: H. S. Lahman (h.lahman_at_verizon.net)
Date: 01/25/04 

Date: Sun, 25 Jan 2004 22:15:59 GMT

Responding to Whitney...

> This is the group's first foray into object-oriented programming. One
> incredibly salient observation made by one poster was that he saw
> similarities in my discussion to one from a group "mired in COM
> components," and that is *precisely* our group. We've been a COM group
> for years now, and those observations were spot-on, down to the
> cobbling together of fixes. We're moving into OO under .NET and C#,
> away from VB.

OK, this seems to be an education problem; more precisely, a
re-education problem. (It may also be a People Problem; more later.)
Alas, getting budget for such things is often sticky. One thing you
might consider as a step-stone is an external audit.

Basically this is just hiring an external consultant to come in and
critique the OOness of the operation. It is important to make it clear
to the consultant that one is not looking for solutions; only
identifying problems. Linking those problems to concrete manifestations
would be helpful. From your description, even a cursory evaluation will
result in a poor report card. When tied to productivity and
reliability, that can be used as ammunition to justify more training and
mentoring.

[Caution: the consultant will have a conflict of interest if there is a
potential for follow-up training and mentoring. (Dilbert's "I'm going
to write me a mini van" syndrome.) This will be obvious to management
as well. To be more salable to management you will probably have to
make it clear to the evaluator that you will use someone else for follow-up]

BTW, where I worked before retiring we had our act together pretty well.
  Yet we still had an external audit every few years as a basic process
monitoring tool. So you can use that notion as a selling point.
Management is very nervous about software development because they
usually don't understand it. The idea of an external audit is likely to
be very appealing.

[Anecdote on Management's view of software. Alex D'Abeloff, who was CEO
of Teradyne and later President of MIT, once described his view of
software to me. He drew a circle on a blackboard (this was quite a few
years ago) and filled it in with the side of the chalk. He then drew
several arrows pointing towards it, labeling each with a '$'. Being a
bit dense I looked at him quizzically and he clarified, "It's a black
hole. Money goes in and nothing comes out."]

>
> For what its worth, our departmental culture is transitioning away
> from an "island" mentality in code development. We've been building
> ASP pages and VB components and applications for *years*, so our
> heritage is certainly not one of object-oriented design. Some of them
> barely understand what "interface" or "abstract base class" means.
> That's why I perceive this to be a great opportunity. We really are,
> in a sense, starting fresh; fresh libraries, new applications, and a
> chance for a fresh vision -- all in a time when we're also being
> required to establish CMM 2 certification.
>
> There's no question that legacy heavily influences the discussion I
> related; they've never been trained or compelled to think in terms of
> reusable units, so the notion is, in no unrealistic way, alien to
> them. The world to which they've become accustomed is a non-reuse
> world; a program is a monolithic thing in that world, so the notion of
> "sharing" out parts of it indiscriminantly would inherently be a
> reckless notion. That's the mindset that has to be reinvented. It
> takes more up-front analysis and design for everything that's written
> to accomplish that, and that kind of analysis really hasn't been done
> in the past. It is starting now.

Such legacy code is going to be a long-term problem for you. In general
it tends to be difficult to make well-formed OO code interact with
legacy code because entirely different construction paradigms were
employed. It is also difficult to replace legacy code directly because
the applications are large and monolithic (i.e., the entire development
staff can't disappear for a couple of years to do the replacement).

That leaves piecemeal replacement. The traditional approach is the
minimum cut-set technique where one draws a line across the call graph
that cuts the fewest call-graph links. That is a lousy way to do it if
one moving to OO solutions because the call graph can be very deceptive.
  A better approach is the following:

(1) Figure out how an ideal new OO application would be partitioned.
Basically this is just a UML Package Diagram block diagram view. [Good
partitioning, though, is based upon OO principles like problem space
abstraction, encapsulation, implementation hiding, etc. IOW, a
subsystem is a class on steroids.]

(2) Select a subsystem from the new application to implement.

(3) Implement it in proper OO fashion and encapsulate it behind a
disciplined interface. Ignore the legacy code; build to business
requirements for the subject matter that the subsystem encapsulates.

(4) Test the new subsystem functionally against its requirements. With
a good interface, this is usually pretty easy.

(5) Identify the legacy code that has the same functionality. This will
not be easy, but it is manageable.

(6) Insert the same interface from (4) into the legacy code around the
functionality to be removed (identified in (5)). This will be the
toughest thing to do because some surgery in the surrounding code will
be inevitable. Note though, that this is also manageable because the
legacy code is not removed; all you are doing is inserting a new
interface to it.

(7) Test the old system with the new interface. This ensures that the
legacy code to be excised has been properly identified. It also ensures
  that the new subsystem can be substituted.

(8) Excise the legacy code and the implementation of the interface.
Replace it with the new subsystem and the implementation of the new
interface. This should be easy because the remaining legacy code is
still talking to the interface inserted in (6). That is, the hole in
the legacy code is already there for the insertion.

(9) Test the old system with the new subsystem installed. This should
pretty much Just Work because the functionality of the new subsystem was
tested in (4) and (8) should not be a problem because the remaining
legacy code was not touched since it was tested in (7).

I mention this because you will face a similar problem anytime you need
to add anything reusable to the legacy code. As long as steps (2)
through (9) are done, you should be successful. This process is handy
because it minimizes risk by separating the concerns of replacement into
  manageable chunks that are independently verifiable.

>
> Some have asked about management commitment...that's an interesting
> thing. We have a *primarily* non-technical manager - and MIS
> background moreso than CS, but with development background/experience;
> an excellent (in fact, outstanding) detail-oriented, people-concerned
> manager who wants to let the technical people solve the technical
> problems. The hard decisions will be made if necessary. But the
> current manager may rotate out of the position in a few months.
> Neither the current manager nor the prospective replacement has much
> of a hard, if any, foundation in OO, to my knowledge or experience.

You can sell a lot of stuff to management if you know what buttons to
push. The big one is data about cost / benefit. (IMO, any manager
dealing with software developers should wear a large lapel button
emblazoned with, "Show Me The Data!".) If you can quantify cost and
benefits, you can sell it. Uncertainty is not a bar if your estimates
are defensible and you can build a logical case for Best Case/ Worst
Case bounds. [Contrary to common developer perception, managers
understand uncertainty; they just don't like it.]

Corollary. Assuming you do get some funding for training and mentoring,
don't stop collecting data. You want to be able to build credibility
with Management for the future by demonstrating the benefits. Perhaps
more important, you need to sell other developers that the change is
worthwhile.

Speaking of selling the developers, do not underestimate the need to do
that. There are going to be several who want the status quo and they
won't be very cooperative. You need to prove to them with hard data
that there is benefit to changing the way things are done.

IME (I spent most of the last five years before retirement on process
work), there are two critical rules for introducing changes within
software groups:

(1) Start small to build credibility. Don't attempt sea changes. Pick
a target change where you will be able to quickly and easily demonstrate
a benefit. Look for stuff to fix that can be demonstrated next month,
not next year. Pick a soft target. There will always be something that
is so obvious it can't help but improve with the change in a big way.
Look for the big step function benefits initially and leave the
incremental nibbles until there is general buy-in. Credibility is
absolutely essential.

[If you bring in a consultant, do so on a long-term, incremental basis.
  Don't look to fix everything at once. Have he consultant focus on
specific things like application partitioning or proper state machine
design that coincide with your soft targets.]

(2) Measure what you do. Decide ahead of time on some metric that will
demonstrate success and then collect data for it before and after the
change. Whenever possible choose a metric that is highly focused on the
change. Don't measure total defect rate if your change only effects
interface errors. Alas, there is a trade-off. The data collection
can't be perceived as invasive. Developers will buy temporary data
collection as part of an experiment, but don't push it. Keep it simple,
even if the metric is less focused.

IME, developers hate to collect data initially. However, once the data
is out there they usually reverse themselves. Being basically
analytics, they get caught up in the implications. They are also almost
always surprised by the results to some extent. Very quickly they see
the data is insufficient to answer other questions. Before you know it
they are off developing data collection tools on their own time to
answer the new questions. (Not to mention the usual, "I can do this
better" ego trip.) It usually only takes a few cycles of process
improvement to build a data collection culture because the value of hard
data becomes apparent very quickly to people who are analytic as part of
the job description.

In the long-term, though, it is important to make the data collection
highly automated and largely transparent to developers. Budgeting time
for building good tools is an excellent investment. (It is also a great
task for summer interns.) It is also important to get feedback from the
developers and pay attention to it. Often it is not the actual time
that data collection takes, but the developer's perception of its
invasiveness that is important.

As an example, Humphrey's TSP book comes with an Excel spread*** for
doing data collection and project tracking. Objectively, it is a great
tool and developers should be able to track their time to the accuracy
of the system clock with no more than five minutes of their time per
week. However, it is a horrible design from a software developer's
viewpoint. For one thing, it uses the wrong tool; Access would have
been better for data collection than Excel. But the big problem is that
it does too much. There are a gazillion tabs for different project
management views. Alas, the developer is only interested in three of
those tabs on a daily basis and one has to scroll to reach them.

The presence of those tabs and the need to scroll to get to the
"important" tabs was perceived as invasive. One developer said, "This
is a worthless piece of crap and a waste of my time!" It didn't matter
that tracking time only took 3-4 keystrokes and couldn't possibly amount
to more the 5 minutes a week for the developer. The perception was that
it was too big and too complicated to deal with.

>
> Our group is a talented one, with strong opinions and thoughts.
> Sometimes we don't all get along as well as we might, and
> unfortunately that focuses on a very very small minority which has
> been an issue meritorious of its own thread, but I'd prefer not to go
> into that here. Suffice it to say its been an issue, and will probably
> continue to be one. Its a source of great stress for everyone
> involved, not just me.

Right. Software development has traditionally been an ego trip. In
40-odd years in the business I have never met a developer who did not
feel they were at least better than the average. The dufuses are always
the guys who moved on to a different job and left their code behind.

Fortunately if they are talented, it is usually possible to introduce
change. They will be smart enough to realize that one can resolve
disputes by running experiments and collecting data. When introducing a
change argue, "Let's try this and see if it works," rather than, "This
is clearly the best way to do it because...". Everyone resists change
when it is imposed upon them or they are not convinced of its value.
But the smart ones aren't afraid to cave when he results of applying the
scientific method goes against them (i.e., they know which battles to
fight).

Another piece of the pie is building a professional culture. People
will tend to be more willing to change their position if holding it will
appear unprofessional. That happens when two conditions prevail: it is
accepted that nobody is always right and it is accepted that nobody
always thinks of everything. Then one can reach consensus in a rational
fashion. Getting that far is the easy part.

The other aspect of a professional culture is that it is blameless. No
one is blamed for mistakes. Instead, mistakes trigger process
improvements to eliminate the mistakes in the future. Nor is anyone.
blamed for an inferior idea when a better one appears. In a blameless
culture, everyone is focused on fixing problems rather than assessing
blame. Alas, this is much harder to achieve.

My favorite anecdote in this vein was from nearly two decades ago when
we were doing procedural development. We worked from very detailed
specifications; essentially one line of spec converted into 1-3 of C.
(All of the thinking went into the specs and their reviews; the actual
coding was a rote exercise that probably could have been done with a
reasonable smart orangutan and a large bag of bananas.) We collected
data on everything and did militant process improvement.

During one increment of improvement we focused on a particular error.
It seems that some lines in the spec we never encoded at all. Every
developer on the team averaged between 8 and 15 of these per thousand
spec lines except one who /always/ has 0. It didn't take a lot of data
analysis to figure out we should ask the woman with 0 errors what she
did different. It turned out that she used a highlighter to mark the
spec lines after she encoded them. As a result we were able to
eliminate an entire class of errors for everyone on the team by making a
highlighter a "standard".

The point here is that in a lot of shops suggesting that level of data
collection on individual developers would cause open rebellion. In our
shop, though, it didn't matter who averaged the 15 errors. (It happened
to be me.) It was just another data point along the road to preventing
defects. Unfortunately, it took us several years to get to that level
of professionalism and the path was bumpy.

*************
There is nothing wrong with me that could
not be cured by a capful of Drano.

H. S. Lahman
hsl@pathfindermda.com
Pathfinder Solutions -- Put MDA to Work
http://www.pathfindermda.com
(888)-OOA-PATH

Quantcast