Re: Help on choosing a valid pattern: composite or not?

Hello Dmitry,

Dmitry A. Kazakov ha scritto:
[ Are you doing machine learning too? ] 

Well, not quite. I've always been fascinated by
Neural networks, GA and such things, so a couple
of years ago I bought two books:

- Neurocomputing by Hecht Nielsen
- A manual published by some italian authors "Manuale
  sulle reti neurali".

This gave me the start to write the toy application
I've described in my previous post.

In other words, it's just not work or research stuff,
just curiosity.

I have started with a more general case: a classifier. A network of nodes
or an acyclic graph of nodes is a case of classifier. Classifier is an
object which gets an example to classify (values of the features) and
returns a classification (a distribution of measures over the classes.)

My focus was on building the tools (classes, infrastructure) not a particular kind of Neural network and, to some degree, I've succeeded. :)

Implement a toolkit that allows to:
- Build "Nodes" with run-time pluggable behaviours that,
  given some numerical inputs, will produce one numerical output.



In general case a node returns a classification of the given example. This
is not numerical. The implementation of the node may evaluate some numeric
weights and recursively descend to the children but that is in the
implementation.


- Build "Nets" of such nodes that, given some numerical
  inputs, will produce one or more numerical outpus.
  "Nets" are partitioned in "Layers".



Building nodes would be learning in general case. As for layers, you might
wish not to expose them, for example, if you plan to add feature selection
at some time. Typically a node deals with one feature. Nodes related to one

Layers were not intented as "feature packaging", they were just "containers" of nodes. Maybe layer is not the correct term to use.

feature comprise a layer. This invariant breaks if you change the order of
features as you train. Features relevance may depend on the path in the
graph, etc. It is a difficult thing, I am puzzling over it for a
considerable time.

I see you wrote that a node "deals with one feature", what do you mean with "features"?

- Allow a "Net" to be a "Node" of another "Net" in such a way
  that the "Parent-Net" doesn't have to be aware of its subnets.
  It should treat them just as "Nodes".



It is for free if the graph is acyclic. For a parent the children would be
partial classifiers. For graphs with cycles one can pretend that it is so,
provided, you are sure that recursive calls would converge to some
distribution of weights. Once weights in all nodes have stabilized, you
stop.


My intention was to put subnets already trained into another net.
Maybe this was my idea of "feature packaging".
I mean, let's say we have found a net able to give high values of
confidence about a phenomenon, and that we know two or more phenomena
that depends on the former.
The former net could be used to as a subnet for the one evaluating
other dependant phenomena, without taking a part in the learning
process of its owner.

I was just playing with my imagination, perhaps this would be of no
particula usefullness, or maybe it's just another viewpoint of some
well consolidated theory unknown to me.

- Extend the "Net" so that it could be persisted to some sort
  of storage (just to make things a little more funny, let's
  say that the storage medium can be chosen via a plugin interface).
  (O/R Mapping tools to use and/or implement?!?)



It caused no problems in my case. Each node is derived from an abstract
persistent object type. The persistency methods refer to an abstract
persistent storage class. So it automatically supports whatever medium is
given. So far I used ODBC, APQ and plain files as mediums.

In the previous implementation, there was no separation at all between
the classes and the persistence abstraction.
The classes implemented some dump and building facilities which I used to store and retrieve their state. Just plain simple text files.

This time I'm aiming to low coupling and put features and services
where they belong.

- Extend the "Net" and the "Node" so that it could be graphically
  represented and manipulated.



I didn't do it, but I'm considering it. I don't think that it should be
directly node's responsibility.

Indeed, the previous implementation, although not perfect from an OO viewpoint, used different classes for graphical representation.
Perhaps I used some sort of "Decorator Pattern".

There was a "Node" class and a "GraphicNode" one. The latter having
a pointer to its "Node".

- Add
	. A "Node" can be added to a "Net", and that would mean to
	become the destination of some other "Node"'s output, and/or
	the input of another one (or one of the "Net"'s outputs...)



This should be a private method used in training. Consider also a public
(more restricted variant, but maintaining the invariants) for manual
composing. This can be useful for GUI.

I meant "Add" to be just as simple as linking nodes each other.
In the past, training was performed by the net itself which was presented with proper training sets and limit to reach in order
to consider training done.

- Remove
	. A "Node" can be removed from a "Net", breaking all it's
	forward and backward links to other "Net"'s elements.



No, removing should leave the classifier's network operating. That means
that its children should become immediate children of its parents. The
resulting network should classify as the initial one with the feature value
(of the node being deleted) set to "unknown".


Interesting, didn't thought about this option. After raw removal of
nodes, I simply set the state of the network to "untrained".

[...Node have properties that net doesn't...] 
I moved that into the training and classification contexts. Nodes
themselves stay immutable.

If nodes are immutable, what about their weights? I feel I'm missing something here.

That's what made me concerned about the "Composite Pattern".
Would the differences between "Node" and "Net" suggest there is
a suitable pattern than Composite to apply? (I thought of
Flyweight because sharing "Node" could be an interesting
variation point...)



I didn't distinguish them on the implementation level. Internally nodes are
persistent objects so they are accessed via handles (smart pointers)
anyway. These handles carry the public interface. A classifier handle is a
what you call "Net". But it just refer to the start node. There is also
node handle type which is more specialized and provides composition
interface.

Is that use of smart pointers that allows you to say that nodes are immutable?
Are nodes containers of pointers to interfaces which implement behaviours and can be trained?

Regards,
Carmine
.

Relevant Pages

  • Reliable File Transfer Multicast Software for Satellites, VPNs, LANs--Beta Testers Wanted
    ... Milestone Technologies Inc., located in Raleigh NC, is searching for a limited number of beta testers for the next generation of its data communications software, featuring reliable multicast file transfer protocols. ... Variants of this program could even be used to allow bandwidth sharing on a satellite network between multiple customers or for multiple applications on the same network. ... Other variants could be used for monitoring network reliability, bandwidth quality and site outages. ... The strengths of this product are the features for its intended use, which is to provide tools for the sending of files to a large number of PCs, and to allow unlimited groups, easy group management, data link quality logs, and detailed transaction logs for troubleshooting. ...
    (comp.protocols.misc)
  • Re: bash: WHY???
    ... features, features, and all the dangerous ones until very recently [and ... > Some free software is undoubtably good, ... But an interoperable network filesystem is a real ... network code and filesystem code, and needs to give full attention to ...
    (comp.unix.solaris)
  • Re: Help on choosing a valid pattern: composite or not?
    ... I have started with a more general case: a classifier. ... Typically a node deals with one feature. ... removing should leave the classifier's network operating. ... > That's what made me concerned about the "Composite Pattern". ...
    (comp.object)
  • Re: Help on choosing a valid pattern: composite or not?
    ... when you have a set of features x1,x2,..,xN, a set of classes ... As I was thinking of subnets a question arose about component's interface in composite pattern. ... Both being part of Component's interface, the "Add" will have a void default behaviour for Nodes and "Link" will be doing something in either cases. ... The resulting network should classify as the initial one with the feature value set to "unknown". ...
    (comp.object)
  • Domain and Cached Credentials (read the FAQ)
    ... but I happen to be working with Microsoft's corporate network this ... cannot join a Windows XP Media Center 2005 PC to the domain*. ... Professional features, specifically Domain Join and Cached Credentials ...
    (microsoft.public.windows.mediacenter)