Re: Static vs. Dynamic typing (big advantage or not)---WAS: c.programming: OOP and memory management

From: Robert Strandh (strandh_at_labri.fr)
Date: 07/15/04 

Date: 15 Jul 2004 08:34:10 +0200

"Jörn W. Janneck" <jwjanneck at yahoo dot com> writes:

> even *if* you were right, this would only explain the lack of success of an
> allegedly superior technology if you would also postulate an inordinate
> amount of uniformity. because, you see, irrespective of whatever reason
> people use to choose their tools, say even if people made completely random
> tool choices, the *market* favors them (the choosers) for their market
> success.

I am not sure I follow exactly what you mean, but you seem to be
saying that even if a good tool is chosen from time to time, the
market will realize this and prefer it to the others.

Not so. First, the uniformity exists, partly because software is a
"winner takes all" market where a slight initial advantage rapidly
translates to market domination. And, contrary to what people think,
the market is not rational. In software, it is dominated by rumors,
personal preferences, inertia, marketing hypes, etc. Combine that
will the extraordinary psychological barriers against learning
something new, and you have a pretty good picture of what is going on.

Another contributing factor is that it is hard to measure
productivity. Today, nobody can produce hard statistics to prove that
a tool is superior to another. And unfortunately, the PDGs of today's
software companies seem to want hard statistics. I claim that it is
the job of a PDG to make decision based on intuition; otherwise he or
she could be replaced by a spread*** program. But the PDGs in the
software industry today usually do not have any intuition about
productivity, simply because they have usually never worked in
software themselves. Therefore, they must use other ways of making
decisions, and you get the above list: rumors, personal preference,
etc.

> iow, the correlation between success and productivity is independent of the
> competence of the choosers.

No, if most choosers are ignorant of better technology, then most will
choose inferior technology, and if the market is incapable of fixing
that, then it will remain that way.

> otoh, if you look at a single individual who is
> successfully participating in the market for some time, it seems appropriate
> to assume that the choices he or she made are not completely random.

The choices are not random, but not the best either. Again, you trust
the market to find the best tools, but unfortunately the market is not
rational.

> at least the reasoning i proposed was this: if something is superior, then
> this would imply that it makes its users (assuming sufficient variation in
> the market for it to have *any* users at all) more successful market
> participants,

This is true to some extent, but not completely, since the market
chooses from other criteria.

> thus presumably increasing the demand for it etc.

This does not follow. Even in the case of individuals and companies
that are successful because of superior technology, there is a
tendency to reject that technology if it requires learning something
unfamiliar. Take the case of Viaweb (Paul Graham) who managed to sell
his system to Yahoo because of superior technology. The developers at
Yahoo then (if I understood the situation right) spent many
programmer-years to rewrite the system in an inferior programming
language that they happened to know about. Pretty typical market
behavior, actually.

> history is full of examples where a technology that multiplied productivity
> for some industry by a factor of 2 or more was developed and prouctized but
> not bought and used for a quarter of a century? after the industrial
> revolution? or even in the it industry?

Superiority is not just a matter of productivity. But yes, history is
full of examples of better technology that lost because of marketing
hype, inertia, rumors, etc. Betamax is the favorite example.

> but how does that account for the fact that people learn new and (hence)
> unfamiliar things in our industry all the time?

Even performance-oriented people obviously have to learn something new
from time to time, or else they will not be employed. And even though
the barriers are strong, they can be overcome when the
performance-oriented person realizes that learning is the only way
out. But it takes extraordinary psychological effort to question
something that already seem to work pretty well, in favor of something
that might work better.

> but if a company using a language that makes them twice or three times as
> productive, improves their product quality, whatever, zooms past them in the
> market, and their choice would be to queue up for unemployment benefits or
> click on the smalltalk book at amazon and zoom along, i think they might
> consider whether their personal preferences are the right guide for making
> that choice.

This reasoning assumes that outsiders realize (or are willing to
consider) the reason for the better productivity. That is not the
case in general.

> i think the basic issue i have with all these lines of argument is that they
> treat programming language choice as an exclusively personal matter driven
> by nothing but individual taste. i don't think this is correct.

That was not my intent. Such choices can be made on the company level
by a well-informed PDG.

> perhaps the reason why it appears to be a decision made based on pure fancy
> is because we only consider it a choice among already a highly pre-filtered
> selection of possibilities. for instance, most people would probably agree
> (and note that this only statistically implies that they would be correct)
> that, say, haskell is not the right choice when your job is: write an office
> suite to compete with ms office. i suppose you would agree that the decision
> against this candidate language is more than a matter of personal taste, it
> is a matter of whether you believe you can meet the basic performance
> requirements of the product using that language, based on whatever it is you
> know about it. okay then, let's take assembly language---we can certainly
> meet any performance requirement that can be met using assembly? yeah, but i
> suppose again we agree that this would be a poor choice, even if we liked
> assembly a lot, because of factors such as portability and productivity.

I do not know Haskell well enough to agree that it would be
unsuitable. But what you brought up is VERY INTERESTING because, as
you wrote, decisions are made "based on whatever you know about it".
The trouble here, is that people who are in a situation to make such
choices are not necessarily well informed. Your Haskell example was
interesting because I often hear people rejecting Lisp (I use Lisp as
an example, because that is the example I am most familiar with)
because of things they think they know, and that are completely false.
Their psychological barriers make it so much more comfortable to
reject something without being well informed. If you have to learn
enough to make an informed decision, you might discover something
better, that you would then have to learn. It suffices to look at
some posts in this group about dynamic languages or about garbage
collection to see this phenomenon. How much simpler to use
disparaging arguments, or to ridicule some totally irrelevant aspect
of some technology ("too many parentheses") than to learn about it.

> it may be that *inside such a prefiltered selection,* the choice of
> programming language seems arbitrary, and perhaps the reason is that, to a
> first approximation and from a purely commercial perspective, it actually
> is. if indeed the differences among contenders in that set of reasonable
> choices are very much smaller than the productivity differences between good
> and bad programmers, then it might even be the rational thing to do for a
> company to just go along with what their "good" programmers say, if only to
> keep them happy and inside the organization.

First, many instances of better technology never make it into the
prefiltered selection, because they are rejected for silly or
downright incorrect reasons. But it is true that the final choice is
often made by some of the best programmers in a company. The trouble
is that even the best programmers in a company do not often know to
choose better technology that they have to learn first. In my
experience, better languages never make it past the filter, usually
because nobody in the company knows it. But rejecting some better
choice for that reason is making implicit assumptions such as:

  * training the programmers is more expensive than the gain in
    productivity.

  * hiring new programmers for the new task is more expensive than the
    gain in productivity.

And that is exactly what the problem is. Better technology is
systematically rejected for non-rational reasons.

> but the expected value of such a bet diminishes with the increasing
> differences between contending tools. i find it unlikely that an it shop
> would leave >2x productivity gain on the table for several decades, waiting
> for some competitor to pick it up and run with it.

Such reasoning is what creates the current state of the market. After
all, it suffices to look at some large software company and check out
what THEY did. Since THEY obviously made a rational choice, then WE
should make the same.

Your position is psychologically very comfortable. As long as you
believe that it is unlikely that an IT shop would let that happen, you
can pretty safely assume that your current choice is not too bad.
This saves you a lot of time investigating something that might be
better. The trouble is, most others do the same so nothing ever
changes. 

-- 
Robert Strandh
---------------------------------------------------------------------
Greenspun's Tenth Rule of Programming: any sufficiently complicated C
or Fortran program contains an ad hoc informally-specified bug-ridden
slow implementation of half of Common Lisp.
---------------------------------------------------------------------
Quantcast