Re: [EGN] Re: turing completeness
From: Michael N. Christoff (mchristoff_at_sympatico.caREMOVETHIS)
Date: Wed, 18 Feb 2004 22:11:51 -0500
"Gerry Quinn" <email@example.com> wrote in message
> In article <rqoYb.firstname.lastname@example.org>, "Michael N.
Christoff" <email@example.comREMOVETHIS> wrote:
> >"Gerry Quinn" <firstname.lastname@example.org> wrote in message
> >> >>>
> >> >>>An artifical neural network can distuinguish scents better than any
> >> >>>human. And artificial neural networks are purely mathematical
> >> >>
> >> >> I do not believe you - it seems to me that physical components such
> >> >> wires and chips would be needed to make an artifical neural network.
> >> >>
> >> >Fair enough. But the process can be described perfectly by
> >> I don't think so, unless you invented lots and lots of axioms
> >> the system, the stimuli given, and the response.
> >That doesn't seem right. Inputs are digitized and the neural nets I've
> >dealt with are simulated in software, and hence must be formalizable. Do
> >you mean that the output of a neural net may not be calculable without
> >actually running it (ie: not analytically solvable)? Why would a
> >description of a neural net require information about its responses? Its
> >seems like saying that the description of a function is not only a rule
> >calculating the mapping of inputs to outputs, but must also include
> >information about its responses to certain inputs.
> What I mean is that to train a NN to do anything useful, you have to
> provide lots of stimuli and responses according to whether it reacts
> correctly. To express this mathematically would require many terms.
> You can describe a useless untrained NN in terms of simple mathematics,
> but to describe either a trained one, or a process sufficient to go from
> an untrained to a trained one, you require lots and lots of data which
> in mathematics correspond to a complicated system of axioms.
> A 100x100 array of neurons linked orthogonally, and a description of the
> properties of each, might be a simple mathematical system. But if you
> describe the weighting of each neuron, that is instantly 10000 extra
> axioms. Describing how it was trained to get this weighting would be
> just as difficult.
> We could, I suppose, say "The 100x100 array that is best [by some
> criterion] at recognising a chess knight", and that is a simple-ish
> description that in principle defines a particular array. If there
> are finitely many possible weightings, linkages, and inputs, the problem
> is certainly decidable. But we are going far away from simulating a
> single network in mathematics here.
A mathematical description may contain raw data, like lists of weights. I
agree that describing a single NN is not the same as providing a general
mathematical system for creating NNs that can perform specific tasks.
However, if I remember correctly, the point was that one could give a
mathematical description of something (in this case an NN) that could smell
better than humans. To do this, one need only provide a single NN that does
this. A sort of 'proof by construction'. Also I don't see why each weight
would require an axiom. A set of input data and an algorithm for processing
the data constitutes a mathematical description. A closed-form solution is
not required. In fact, I'd be surprised if there was a closed form
mathematical description of any but the most rudimentary NNs. Although an
algorithm and list of weights may not really help us understand "how" or
"why" the NN is successful, its still a mathematical description. But maybe
thats not what you are getting at.
In terms of having to describe how its weights were attained: Couldn't one
also use that argument for arbitrary programs? ie: One may say that in
order to fully describe Dijkstra's shortest path algorithm that one must
include a description of the mental process he employed to discover it. In
the case of NNs, if empirical evidence agrees that an NN, with weights etc.
applied, is able to distinguish certain scents better than a human, then it
should be enough to present that description without mention of the process
by which it was developed.
However I may have completely misinterpreted and misunderstood you're point.
l8r, Mike N. Christoff