Re: XP and Pair Programming

Nick Malik [Microsoft] wrote:
> Thank you, Isaac,
>
> I need to read as carefully as you do before I post a link. You caught some
> troubling things that makes me think this article is fluff and not science.
>
> "Isaac Gouy" <igouy@xxxxxxxxx> wrote in message
> news:1124907006.852667.215990@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> >> Along the way, I found an interesting article. The military conducted an
> >> experiment of using pair programming, and measured the result.
> >> http://www.stsc.hill.af.mil/crosstalk/2003/03/jensen.html
> >
> > Where does it say "The military conducted an experiment"?
> > I see "a large software organization", 1975, Fortran.
>
> He is not clear about the year in which the experiment takes place, but it
> is easy to infer 1975 from his intro. He does say Fortran (a point which I
> missed). It also appears that the person conducting the experiment was not
> working for the defense department at the time, and later joined the
> civilian STSC before publishing the paper last year. In this sense, this is
> an old study.
>
> I had not seen this experiment published before, however, nor can I find
> other references to it. Is it possible that this old study is previously
> unpublished? If that is the case, it stands as empirical evidence of pair
> programming at least.

I like empirical evidence!
I'm less keen on anecdotal evidence :-)


> >> Excerpts
> >> "We hoped for a productivity gain of anything greater than 0 percent. Any
> >> small gain would have compensated for the two programmers loading on each
> >> task. The 127 percent gain achieved was phenomenal and a cause for
> >> celebration."
> >
> > Another excerpt:
> > "Project individuals could not directly obtain a productivity and error
> > baseline for the project"
> >
> > Where these even the same programmers working under the same manager?
>
> That is not clear. However, he did state that the productivity and error
> rates were baselined as that company prior to this experiment and that the
> numbers were nearly constant across projects (fairly typical). I am trained
> in measuring software productivity, and in my experience, this is typically
> true of an organization: projects using different programmers with different
> managers tend to have nearly identical productivity rates when they all work
> in the same overall environment (cultural, regional, educational, political,
> organizational).
>
> Therefore, although an individual baseline could not be ascertained for this
> particular team, statistics for the organization in general were available.
> I would state that this is still a valid experiment, if only to show that
> more studies are needed.

Isn't that extraordinary when we also know that productivity varies
between programmers by orders of magnitude?


> > Another excerpt:
> > "The individual teams appeared to be more focused in their activities.
> > The highly visible aspect of this attribute was that programmers took
> > fewer breaks for restrooms, coffee, outside discussions, etc."
> >
> > In other studies of pair programming is any increased productivity
> > simply explained by increased work-effort?
> >
>
> You have unfairly excerpted a specific sentence. The author, through
> analysis, details out the reasons why he believes that pairs were more
> productive in this case. Taking a single sentence out of context is not a
> good measure of an open mind.

Humbly suggest that your original post selectively excerpted specific
sentences ;-)


> The reasons that the author details include Brainstorming, continuous design
> walkthrough, focused energy, mentor, motivation, and problem isolation. The
> negatives included counter-productivity due to bad pairings ("two prima
> donnas") and the need for better colocation of the team in a war-room
> environment (which was lacking in this study).
>
> In my personal experience with pair programming, I admit that I spent less
> time on breaks, but that was not because I was being worked harder... it was
> because I was enjoying my work more. I cannot say but I can guess that some
> of the same feelings were being felt by the teams in the quoted study.

Shouldn't we use the simplest explanations first?
Shouldn't we measure how much work-effort increased and then see if
there's still a productivity difference that requires explanation?

They could have sold pair-programming to management on the basis that
each one of the pair acted as a supervisor that kept the other working
harder :-)


> >> "The error analysis showed the project had achieved an error rate that
> >> was
> >> three orders of magnitude less than normal for the organization.
> >> Integration
> >> of the first two components (approximately 10,000 source lines) was
> >> completed with only two coding errors and one design error. "
>
> I do not find this surprising. I would also assume that a modern team,
> using modern debugging tools, would exhibit a far lesser "gain" in reducing
> errors because we should be generating fewer errors to begin with. That
> said, I do not, in the least, find it unusual that error rates would fall
> this much. There was a lot more room to fall in 1975.
>
> >
> > Another excerpt:
> > "The remaining three components had more errors, but the number of
> > errors for these components was significantly less than normal."
>
> The finding is important considering the success criteria. In this case,
> the author tells us that the experiment would be considered successful if
> the productivity and error rates broke even. The fact that they fell
> substantially on one component, and fell significantly on others is still a
> useful finding, if only to inspire further research. It is not, in itself,
> scientifically valuable since no numbers are exhibited.
>
> >
> > Why aren't we told the error rate for the remaining three components?
>
> I'm guessing it is because the author is a true believer and wanted to skew
> the paper towards Pair Programming, yet the teams working on these
> components didn't do a good job of measuring their observed error rates, so
> he was forced to rely on empirical data. (e.g. He asked them if they were
> seeing fewer errors, and they said "significantly less").

afaik "empirical" doesn't exclude measurement. (I had the notion that
most of Science was empirical.)


> It is an interesting paper. I have seen papers like this published in
> journals in the past, but I don't consider the findings to be the result of
> careful scientific study. I would chalk it up as a fluff piece, but still
> interesting enough to hopefully inspire actual scientific study of the area.
>
> That said, I haven't found a single study or paper, even at this level,
> where someone experimented with pair vs. solo programming and found solo
> programming to be measurably more productive.

Shouldn't we be comparing pair programming with partner programming? Is
collaborating to get the project done more abnormal than working in
splendid isolation?

(Am I suffering from a very bad case of rhetorical questions? Yes.
Sorry.)


> --
> --- Nick Malik [Microsoft]
> MCSD, CFPS, Certified Scrummaster
> http://blogs.msdn.com/nickmalik
>
> Disclaimer: Opinions expressed in this forum are my own, and not
> representative of my employer.
> I do not answer questions on behalf of my employer. I'm just a
> programmer helping programmers.
> --

.

Relevant Pages

  • Re: XP and Pair Programming
    ... >>> In other studies of pair programming is any increased productivity ... correct in stating that any excerpt runs the risk of being interpreted out ... >> In my personal experience with pair programming, ...
    (comp.object)
  • Re: Academic research aimed at improving programmer productivity?
    ... Well, to accomplish significant results, people need ... > productivity in the shorter hours. ... Hm, people have pitched Extreme programming in various ways to me before, ... > resemblance to the way in which MIS managers accomplish their goals. ...
    (comp.programming)
  • Re: XP and Pair Programming
    ... > correct in stating that any excerpt runs the risk of being interpreted out ... >>> In my personal experience with pair programming, ... > input to the productivity formula. ... > conducting Pair Programming research. ...
    (comp.object)
  • Re: Academic research aimed at improving programmer productivity?
    ... > programming as a team activity is one of the major problems of industry. ... >> should be concerned with programmer productivity. ... > Marxism accomplishes that purpose, so I don't see why we need a Marxism ... Soviet managers fulfilled their Five Year Plans you discover a strange ...
    (comp.programming)
  • Re: Anybody running unigraphics nx5?
    ... thinking about what took so long and what else Feature Based Chaining ... You think the real time saved is in programming? ... 10% of the machining time it will cover ALL the programming time times 10. ... Flexability is the key to productivity. ...
    (alt.machines.cnc)