Re: Help on choosing a valid pattern: composite or not?

[ Are you doing machine learning too? ]

On Mon, 28 Nov 2005 22:03:41 +0100, CarmineM wrote:

> *** The (short) story ***
>
> A while ago, I wrote a sort of toolkit to play with neural
> networks (back prop.).
> As of today I've realized that I wrote it using objects but
> not quite in an object oriented way, so after reading
> some documentations, I thought it could be an interesting
> exercise writing it from scratch, this time with
> a different point of view.
>
> *** The (rough) idea ***

I have started with a more general case: a classifier. A network of nodes
or an acyclic graph of nodes is a case of classifier. Classifier is an
object which gets an example to classify (values of the features) and
returns a classification (a distribution of measures over the classes.)

> Implement a toolkit that allows to:
> - Build "Nodes" with run-time pluggable behaviours that,
> given some numerical inputs, will produce one numerical output.

In general case a node returns a classification of the given example. This
is not numerical. The implementation of the node may evaluate some numeric
weights and recursively descend to the children but that is in the
implementation.

> - Build "Nets" of such nodes that, given some numerical
> inputs, will produce one or more numerical outpus.
> "Nets" are partitioned in "Layers".

Building nodes would be learning in general case. As for layers, you might
wish not to expose them, for example, if you plan to add feature selection
at some time. Typically a node deals with one feature. Nodes related to one
feature comprise a layer. This invariant breaks if you change the order of
features as you train. Features relevance may depend on the path in the
graph, etc. It is a difficult thing, I am puzzling over it for a
considerable time.

> - Allow a "Net" to be a "Node" of another "Net" in such a way
> that the "Parent-Net" doesn't have to be aware of its subnets.
> It should treat them just as "Nodes".

It is for free if the graph is acyclic. For a parent the children would be
partial classifiers. For graphs with cycles one can pretend that it is so,
provided, you are sure that recursive calls would converge to some
distribution of weights. Once weights in all nodes have stabilized, you
stop.

> - Extend the "Net" so that it could be persisted to some sort
> of storage (just to make things a little more funny, let's
> say that the storage medium can be chosen via a plugin interface).
> (O/R Mapping tools to use and/or implement?!?)

It caused no problems in my case. Each node is derived from an abstract
persistent object type. The persistency methods refer to an abstract
persistent storage class. So it automatically supports whatever medium is
given. So far I used ODBC, APQ and plain files as mediums.

> - Extend the "Net" and the "Node" so that it could be graphically
> represented and manipulated.

I didn't do it, but I'm considering it. I don't think that it should be
directly node's responsibility.

> *** And now back to what I started writing in the original post ***
>
> I came to the point of designing what the "Node" and the "Net" are (
> yes, I've just stared bothering... :) ),
> and thought "Composite Pattern" could be of some help in unifying
> the interface from a client's point of view.
>
> At its minimum the interface shared by "Node" and "Net" is:
>
> - Execute
> . A "Net" can be told to execute itself, and that would
> mean to call each "Node"'s Execute
> . A "Node" can be told to execute itself, and that would
> mean to fetch its input and apply some calculation in order
> to produce its output.

= Classify

> - Add
> . A "Node" can be added to a "Net", and that would mean to
> become the destination of some other "Node"'s output, and/or
> the input of another one (or one of the "Net"'s outputs...)

This should be a private method used in training. Consider also a public
(more restricted variant, but maintaining the invariants) for manual
composing. This can be useful for GUI.

> . A "Net" can be added to another "Net", and that would
> mean to become one or more inputs of a "Node" and/or one
> of the parent "Net" outputs.
>
> - Remove
> . A "Node" can be removed from a "Net", breaking all it's
> forward and backward links to other "Net"'s elements.

No, removing should leave the classifier's network operating. That means
that its children should become immediate children of its parents. The
resulting network should classify as the initial one with the feature value
(of the node being deleted) set to "unknown".

> This means that "Node" and "Net" will contain other things aside of
> this shared interface.
>
> E.g.: "Node" have some parameters that can be adjusted so that the
> computing strategy can produce "better" outputs (whatever
> better means...)

I moved that into the training and classification contexts. Nodes
themselves stay immutable.

> That's what made me concerned about the "Composite Pattern".
> Would the differences between "Node" and "Net" suggest there is
> a suitable pattern than Composite to apply? (I thought of
> Flyweight because sharing "Node" could be an interesting
> variation point...)

I didn't distinguish them on the implementation level. Internally nodes are
persistent objects so they are accessed via handles (smart pointers)
anyway. These handles carry the public interface. A classifier handle is a
what you call "Net". But it just refer to the start node. There is also
node handle type which is more specialized and provides composition
interface.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de
.

Relevant Pages

  • Re: Help on choosing a valid pattern: composite or not?
    ... - A manual published by some italian authors "Manuale ... Classifier is an object which gets an example to classify (values of the features) and returns a classification ... removing should leave the classifier's network operating. ...
    (comp.object)
  • Re: Seperation function generated by NN toolkit
    ... plot of the classifier generated during the training? ... rate the plot of the function created internally by the network. ... Class-conditional PCA planes ...
    (comp.soft-sys.matlab)
  • Re: Help, a problem about Boost algorithm
    ... hard for the neural network; it can only achieve an error rate of 50%. ... Show that if there are no restrictions on which 50% of the patterns the ... Boosting is actually a way to "assign" only one class (or classifier) with the responsibility of decision ...
    (comp.ai.genetic)
  • How to design a weak classifier to classify histogram
    ... I am designing a boosting classifier to classify face ... image, the feature of image is the histogram of gray value, but now I ...
    (sci.image.processing)
  • How to design a weak classifier to classify histogram
    ... I am designing a boosting classifier to classify face ... image, the feature of image is the histogram of gray value, but now I ...
    (comp.graphics.algorithms)