Re: Lisp's future

From: Erik Naggum (erik_at_naggum.no)
Date: 01/31/04

  • Next message: Edi Weitz: "Re: Cello Rising" 
    Date: 31 Jan 2004 01:48:29 +0000

    * Erann Gat
    | What this debate is really about, it seems to me, is no different
    | from the old deliberative vs. reactive debate that took place in AI
    | robotics during the 80s.

      This debate is really about whether the act of programming produces or
      consumes information. Of course it does both.

      This odd discussion is similar to the age-old discussion about how to
      write an essay. Some people believe that you should think carefully
      about everything, construct a skeleton and a plan and proceed to flesh
      it out, in the belief that you should have all the information you
      need to write the essay up front and that the writing itself should
      not produce more information, indeed that thinking of something you
      had not thought of to begin with is some sort of failure condition,
      like lack of planning or lack of concentration on the task at hand.
      Other people believe that formulating an idea in words /creates/ the
      idea and that you should only define a purpose, a goal condition, and
      then proceed to write your ideas down while monitoring that you are on
      the right course. (Monitoring is how consciousness works to begin
      with and with some training becomes automatic and emotional: you feel
      good when you go in the right direction and bad when you go in the
      wrong direction. People who plan compulsively are usually the kind
      who think that emotions have no business in directing attention or
      focusing effort.) Creative people also argue that premature planning
      solidifies unformed ideas that should be expressed fully if they are
      allowed to form at all, because almost all ideas appear good until put
      to the test, and the best test is not to think about what you would
      do, but to actually do it. (The word «essay» itself is curious -- it
      originally meant an attempt at expressing something, but the written
      attempt has become such a standard model that the word has come to
      mean a particular style of exposition and the history of the word is
      all but forgotten.)

      Most of the arguments in favor of one or the other of these positions
      (when both are clearly necessary and correct) are made for the sole
      purpose of pushing a pendulum that has swung too far in one direction
      back in the other direction, so the context of each argument is that
      the antithetical position has become too strong, for whatever reason
      and by whatever measure. We have not arrived at the stable position
      that is satisfactory to all parties, so we keep pushing the pendulum
      too much in our direction when we perceive that it has been pushed too
      much in the other direction by others. In all likelihood, the stable
      position is one where different people exhibit different capacities
      and tendencies and where the belief in one answer has been replaced by
      a more pluralistic view. In my view, the planners are not only wrong,
      but so averse to feeling bad that they prevent themselves from feeling
      good about what they do and therefore have no direct experience of the
      pleasure of thinking properly and so resort to their plans to avoid
      the pain of realizing that they do not think well. Herbert Dreyfus
      has argued that it is extremely important to be emotionally attached
      to the success of one's work if one is to progress beyond advanced
      beginner to competent or even proficient, and point out that students
      who work very hard to avoid feeling the pain of failure never become
      competent enough to feel the joy of competence and success. It is no
      accident that those who recoil from the pain of failure are those who
      are most likely to ask for «rules» or reach for the «answer book».

      Some people are very good at predicting what will happen and plan for
      both the likely and the unlikely, but then become nervous wrecks when
      their planning turns out to be faulty and they have to re-plan, which
      they resist until it is already too late. Others are not so good at
      prediction and planning, but are very good at setting goals, and they
      try things out, compare it with their desired results, learn from each
      failure, accept that neither other people nor the world in general are
      obliged to do their bidding, and they never become nervous wrecks in
      the face of hardship, even disasters, but frequently set new goals as
      they find that some goals were unreachable at a reasonable cost.

      In the real world, it takes all kinds to make a good society, and the
      only track record we know from people who believe that only one kind
      of person should exist is incredibly murderous tyrannies. A lot of
      (dysfunctional) people believe that all those who do not obey every
      rule should be punished or killed, and they spend their time blocking
      other views, with such phraseology as «unconditionally bad» about them
      while they misrepresent them in order to destroy the ability of others
      to present them in a rational discussion. Typical behavioral patterns
      with these people include hostility towards any violation of their
      plans and rules, and when they see some person doing something they do
      not like, they believe that the proper course of action is to do what
      they do not like on purpose and in some demented idea of retaliation
      and retribution, even though they are never directly affected, instead
      of working to rectify or improve the global situation. Breaking rules
      and upsetting plans cannot be mistakes to these people, only willful
      acts of destruction, which only severe punishment can cure, and they
      are never sufficiently humble to even consider the possibility that
      they are the first to break any rules on purpose. People who plan too
      much are also unable to enjoy or cope intelligently with surprises,
      for any surprise is evidence of their insufficient planning.

      The important distinction between planners and creators is whether an
      action taken for a given purpose does or does not provide feedback to
      the decision machinery that determines the next action. Listening to
      feedback is not an option if you want to retain your mental health,
      but listening to feedback will frequently mean that any plans you had
      must be discarded. The more energy you have poured into planning, the
      less likely you are to listen to feedback that it is junk. If you
      make up your mind that the world is different than it is, that it
      «ought» to be different, you will run into psychological problems, the
      larger the more you insist. A lot of people believe that standing
      fast in their beliefs is the hallmark of integrity and that it is the
      duty of all people of true belief to force other people to obey their
      beliefs. These days, we call them fundamentalists and terrorists, but
      in the world of software development, we have no such useful labels
      for those of deranged psychology and highly irrational beliefs, but
      they are nonetheless present in the industry. The most effective test
      they can be put to is whether /all/ actions taken by programmers are
      able to provide feedback to the planning and scheduling of the whole
      project, or only /some/ actions (of the selection of which they demand
      to be the final arbiter).

      If /all/ actions taken by programmers are used for feedback, the best
      way to view the act of programming is that of producing information
      about the problem to be solved. Some of it may be wrong, some of it
      may be of no use, some of it may be unexpectedly true and need work to
      be proven, but all of it is used as feedback. This is obviously not
      the dichotomous relationship with planning that some people think it
      is -- if you are both intelligent and of sound mental health, you do
      not plan your actions, you plan your results and the acceptable costs,
      and you trust both your own competence and that of others in actually
      accomplishing those goals without detailed instructions. Some of the
      time, however, an action may produce information that will affect the
      results and the costs, and it is at this point that a rational manager
      listens and an irrational manager ignores the available corrective
      feedback. When all actions are used for feedback, the consequence of
      making mistakes will be a corrective and appropriate response, because
      everything everybody does /matters/.

      If only /some/ actions taken by a programmer are used for feedback,
      the result will necessarily be that programmers will perform actions
      that are not used for feedback with reckless abandon. In this regime,
      the already tyrannical manager will write ever more detailed plans to
      enforce his model of reality, the stronger the more wrong it is, just
      like people who have to be taken care of by society in general because
      they have stopped accepting input from the real world. In business,
      however, those who keep pushing for something they believe in tend to
      be remembered when they have had a string of successes, but there is
      now a growing literature on the stupidity of the highly intelligent in
      both managing their lives and their money. (E.g., Why Smart People
      Can Be So Stupid, edited by Robert J. Sternberg, ISBN 0-300-09033-1.)
      It is quite possible for people who watch others who have succeeded by
      pushing for something they believed in to believe that it was pushing
      for something they believed in that produced the success, but in logic
      this is known as the post hoc ergo propter hoc fallacy. In the real
      world, pushing for something you believe in generally causes massive
      and lasting failure, but if you accidentally get it right, pushing may
      magnify the success. This is like performing an objectively insane
      action which by sheer accident happens to associate itself in the mind
      of the deranged with some random success condition that has nothing
      whatsoever to do with the action. You could have worn the same tie
      the last three times you won a contract or did a good presentation,
      and then suddenly it becomes your «lucky tie». Lots of people do
      this, and such things as astrology and numerology or the importance of
      winning New Hampshire because past presidents did, find a receptive
      audience in the highly unscientific population. The more /selective/
      you are in the feedback you accept, the more insane your reasoning
      will become as you will necessarily reject corrective feedback that
      would have led to better reasoning.

      The point at which it becomes rational and productive to impose one's
      will on reality or other people is perhaps the most interesting aspect
      of the philosophy of ethics. It is not understood to any significant
      degree how to determine this point in advance, so ethical thinking is
      best thought of as a systematic approach to respond to objections, but
      this imposes a requirement to object, and to do so rationally and only
      constructively, which is not as well-defined as it ought to be. The
      history of modern societies provide us with the evidence that we have
      swerved aimlessly in this regard and are both unsystematic and highly
      irrational in our objections and the expression thereof, especially in
      cases where the severely misguided notions of «good» and «evil» are
      involved, and people conjure up feelings of suffering «injustice»
      instead of coping intelligently with the situation at hand. There are
      people so devoid of rational ethical thinking that when they observe
      something they object to, they do exactly the same thing on purpose in
      order to express their objections, and then fail to understand that
      they have done something objectionable. Human rights, for instance,
      is an expression of a fuzzy line of demarcation around an area where
      people are not allowed to impose their will on any other person for
      any reason whatsoever. The U.S. Constitution was also thought to have
      protected certain rights even under serious threats, but turned out to
      be completely irrelevant to the American people. Social security is
      an attempt to draw up an area where every citizen must accept the cost
      of sustaining other citizens who are unwilling or unable to sustain
      themselves, and those who object to this system tend to ignore the
      fact that the absence creates slavery and people so desperate that no
      law can bind them. The political debates are mostly about which areas
      of public and personal life must yield to the imposition of the will
      of others. This is far from the obvious and well-known that the mark
      of the highly irrational and tyrannical pretends it is: /Nothing/ is
      «unconditionally bad»; the will of some people who have made up their
      mind about both reality and people /must not/ be imposed or accepted
      to be imposed on others to the detriment of the ability to object both
      rationally and constructively.

      Being intelligent and reasonable human beings with the particular kind
      of brain that we possess, we are still not only fallible, but very far
      from omniscient. We are also naturally inclined to commit a fallacy
      known as conformance bias, which makes us want to prove that what we
      already believed was true and to believe faint affirmations even in
      the presence of rejection. Another dangerous problem with puny human
      brains is that counter-information tends to be rejected because opening
      every decision up for re-evaluation would consume infinite resources,
      and it also takes too much time and energy to re-evaluate enough to be
      able to know whether some counter-information is important or not. For
      these reasons, we have set up the extremely inhuman methodology known
      as «science»: Figuring out that a particular causal relationship is
      for real independent of our desire for it to be is really hard work.
      Most people much prefer to believe that what they would like to be
      true is also actually true than to do all the hard work to determine
      what is and what is not actually true, not the least because most of
      what people believe is actually dramatically false. Given what we
      know about human fallibility and society-wide myths and beliefs, it
      should be obvious that we simply cannot afford to be selective in what
      feedback we are willing to accept. Every single relevant, corrective
      feedback we refuse to consider will come back to harm us. So, too,
      with every rational, constructive objection to what we have done, but
      those who are least able to accept corrections are unsurprisingly also
      those who are least able to express corrections and objections in a
      way that other people can make use of, and they are the first to jump
      to conclusions about other people's motives and rush to accusations
      and all sorts of counterproductive idiocy. Instead of thinking when
      they meet corrective feedback, they defend themselves and stick to
      their plans precisely /because/ those plans are in danger of proving
      insufficient or wrong, and this is much more likely to happen if the
      plans have been very costly to produce.

      To strike a balance between planning and creativity, we need to zoom
      out and look at the available measurements of the success we have with
      our various approaches. To do this, we need to be aware of the goals
      we have for our work at a sufficiently abstract level. The more we
      plan, the more concrete our goals become, however, and the danger of
      planning to avoid costly mistakes in achieving our goals is that we no
      longer remember what we were originally heading out to accomplish.

      The most productive plan, then, must be the plan that is not concrete,
      but abstract, yet has definite test criteria. The distinction between
      a plan and a vision is therefore only in the availability of the tests
      we can employ to know if we have gotten where we wanted to get. This
      is the kind of plan that is incommunicable to computers at present, so
      we are in no danger of confusing a program with its intention, just as
      we are never in danger of confusing a test harness with the objective
      of the whole endeavor.

      The notion that planning at the detail level is a necessary product of
      good planning at the abstract goal level is itself a product of the
      kind of personality disorder that prevent creative people and experts
      from producing the desirable results. This kind of planning is only a
      good thing if you plan to hire monkeys who respond well to electric
      shocks when they deviate from their instructions. Human beings are
      not computers or physical devices and planning for intelligent people
      is different from planning for morons. Most planners also believe
      that they are vastly more intelligent than the people who are going to
      implement the plan in practice. More often than not, however, they
      are not noticeably more intelligent and the belief that one person can
      plan what another person should do is a very good way of making the
      person who plans unable to accept negative feedback from the stupider
      person who implements it and the person who performs the planned steps
      is unlikely to invoke their intelligence when the plans are bad, so
      excessive planning is a very good way of destroying productivity and
      /avoiding/ the abstract goals of an enterprise entirely.

      However, we must take the planners' word for it -- they could never
      employ their creativity and intelligence in the face of unexpected
      situations and their fear of failure is probably well-founded, so if a
      person demands detailed plans, we should feel a profound pity for
      their damaged psychology and inferior brainpower and help them write
      their plans and ensure that they measure only irrelevant properties of
      the system being developed so they are kept busy without messing up
      for anyone who have do to deal work. In this regard, we should go
      talk to the social science people, because they educate a lot of the
      bureaucrats who prevent politicians from doing more damage than they
      absolutely have to with their misguided micro-controlled plans. 

    -- 
Erik Naggum | Oslo, Norway                                      2004-031
Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.
  • Next message: Edi Weitz: "Re: Cello Rising"