Re: Help on choosing a valid pattern: composite or not?

Hi Lahman,

H. S. Lahman ha scritto:
Ah, a little context always helps. B-) Basically I agree with Kazakov so my comments below just provide a somewhat different spin... 

Just finished reading Dmitry's reply.

*** The (rough) idea ***
[...cut of my rough idea...]
All this structure is can be captured in relationships. Since a neural net is inherently hierarchical the GoF Composite pattern would work fine. Just make the substitutions for the GoF structure:

Component := Kazakov's "classifier".
Composite := Net
Leaf := Node

Yes, I already did the substitutions on the GoF structure and came to yours (your and Kazakov's) conclusion.

The tricky part is that the Node has pluggable behaviors. You could do that by tacking on a GoF Strategy or State pattern. That is, the "links" you proposed would just instantiate the relationship from a given Node to the right behavior role.

IOW, when one "walks" the net for training (assigning "links") or execution, it is the Composite's structure one navigates but the grunt work is done in the State/Strategy.


Interesting.
I've always relegated learning to weight balancing, not to linking
nodes each other.
There are a few literature's sections I've skimmed only, which
talk about "growing" networks and "adapting" ones.
Those are, if I'm not wrong, the ones in which learning involves
modification in linking between nodes and sprouting of new nodes.
Too complex at that time and for a toy app. :)

Strategy came to my mind because each node has a different
transfer function (each transfer function relies on node's
weight's and other properties).


[...Extend the net so that it can persist...]
Different problem. Figure out how to solve the neural net problem and then worry about how to store the nets. IOW, that's a job for another subsystem that is encapsulated by an interface that is designed for the neural net view. The subsystem then performs the transformation to whatever the persistence view of the data is (RDB, OODB, clay tablets, etc.). 

I do agree with you.

- Extend the "Net" and the "Node" so that it could be graphically
  represented and manipulated.

That is just a matter ot "walking" the composite structure to add/remove nodes and instantiate/deinstantiate relationships.

Why I think it "could be interesting" to write from scratch?

- Practice with MVC (or its variation) to decouple the
  model (Net, Nodes, Subnets...) from its graphical representation.


I wouldn't dream of using MVC for this. B-( The CRUD/USER layered models are designed for pipeline applications between a UI and an RDB where the "business" layer doesn't really do anything except provide data integrity validation and an intermediate step stone for converting one view to the other. Your neural net's "business" layer is far too complex for that. IOW, the UI and DB are both peripheral to the real problem rather than dominant aspects of it.

[However, if there was a handy MVC infrastructure around to make talking to the UI or the RDB easier, I would not be averse to using it. B-)) But when doing so it would be encapsulated in UI and persistence access subsystems so that it was entirely transparent to the neural net solution.] 

Then it's me not understanding the MVC infrastructure.

I thought it this way:

- View: Some classes with following capabilities
- GfxNode class:
. Can paint a node on screen
. Can paint a node's links to it's children
. Can intercept UI events (double clicks pops up a window with information on a node's state...)

....

- Controller: Some intermediate (Mediator, Adapter, Façade?!?) class which are responsible to fetch and set model objects states (e.g.: changing node's links, properties, weights...)

- Model: Just the net and nodes as described previously.

Is this idea of MVC an ill one? :)
This could explain the bad feeling I have whan thinking about using MVC in my current applications... (complex model with lots of business rules).

Again, what are the options when one wishes to decouple UIs from complex models and business logics?

- Practice with Application layering (UI, Business, Storage...)

I suspect you will have multiple subsystems just for the neural net solution (e.g., the basic net configuration/navigation, training logic, and execution logic could be quite different subject matters). Throw in persistence, a UI, and a fancy graphics editor and application partitioning starts to get interesting. [I have a category devoted to this on my blog that may be of interest.] 

I'll pay a visit as soon!

At its minimum the interface shared by "Node" and "Net" is:

- Execute
    . A "Net" can be told to execute itself, and that would
    mean to call each "Node"'s Execute
    . A "Node" can be told to execute itself, and that would
    mean to fetch its input and apply some calculation in order
    to produce its output.


I see this as a job for someone else who "walks" the neural net structure in a predefined order. Whenever they get to a Node, they invoke the relevant behavior(s) via the "links" to the Strategy/State. (It is in the nature of Strategy/State that the specific behavior is hidden from whoever is "walking" the net so every node is "executed" in exactly the same way.) 

Visitor Pattern could help here.

BTW, I would see training as a different mode of operation for the neural net. However, I suspect it can be expressed as a superset of execution. That is, the net is executed exactly the same way but someone else evaluates the results to modify the "links" between Node and Strategy/State. 

Indeed, it is a different operation mode which:

- Execute the net just as if it was already trained
- Compares the results with the expected ones and
	- If they meet expected standards then training is over
	- else keep training until standards are met or user
	  realize that the net won't converge...

The training happens by means of node's weights adjusting
via back propagation.

- Remove
    . A "Node" can be removed from a "Net", breaking all it's
    forward and backward links to other "Net"'s elements.


I share Kazaov's skepticism on the implications of removal. But I haven't done a whole lot of neural net work. B-)

As I wrote in reply to Kazakov, I didn't thought that "holes" produced by raw removal could be harmful to the net's training.
Remove, just like Add, was just a means to actually build the net's
structure.


I do have appreciated your posts.
Thanks!

Regards,
Carmine
.

Relevant Pages

  • Re: Help on choosing a valid pattern: composite or not?
    ... is a different Strategy for execution and weight balancing. ... Even if you aren't doing something so elaborate initially, it won't hurt to decouple through Strategy/State initially with a view to expansion. ... it would get initialized there using data from an interface method like displayNodewhere the content was encoded from a Node object in the neural net view. ... [The cost of the decoupling is the encode/decode of the data packet across the subsystem interface. ...
    (comp.object)
  • Re: HOw to introduce a proactive approach on DB use?
    ... At every end of the month, endusers are running reports (business ... objects) and Peoplesoft batches. ... We determined our highest CPU ... we may average 16 milliseconds of CPU per execution to ...
    (comp.databases.oracle.server)
  • Re: Where to store SQL statement
    ... Less scalable than business logic in the business layer. ... how do you rate the optimization of the execution? ... logic in the middle tier under massive parallel loads? ...
    (comp.lang.java.databases)
  • Re: Isnt in the local news.
    ... serve, than on business problems. ... execution of a good business plan are more ... So when are you going to supply all us readers ... the mind reading capabilities of your readers? ...
    (soc.culture.hawaii)
  • Re: Isnt in the local news.
    ... serve, than on business problems. ... about disincentives when the incentives and ... execution of a good business plan are more ...
    (soc.culture.hawaii)