In the end, the PhD didn’t work out; to give the briefest possible explanation, my supervisor and I wanted to pull it in too-different directions. I may well return to the research I was doing in my spare time, but my thought at the moment is that I will start using this blog for general research notes and ponderings, probably on a range of topics including physics, maths and computer graphics.
A lot has happened since my last post, but it’s never really felt constructive to post about here. I’m currently looking at diffusion as a mechanism for temporal memory formation.
This is a round-up of work by Principe, Euliano et al on dynamic topographic maps and temporal self-organisation.
- Principe, Euliano and Garani (2002), Principles and networks for self-organization in space-time
Applying R-D-type dynamics to SOMs (SOMTAD – Self-Organising Maps with Temporal Activity Diffusion) and GasNets to achieve spatiotemporal memories. Talks about the possible role of NO in memory formation, includes a review of relevant work in this area.
The basic approach of TAD is for winning neurons to send out waves of activity making neurons close to recent winners more likely to fire next. This enables the network to form noise-resistant memories of temporal sequences. Positive results are obtained here for robot navigation and (with the GASTAD) speech classification.
Leaves open further theoretical work on R-D type systems: ‘Although there are clear links to the R-D paradigm (the generation of traveling waves), we were unable to derive a cost function and learning rule from first principles’. Also suggests exploring the use of diffusion to train other sorts of networks, such as MLPs and RBFs.
- Neil R. Euliano (1998), Temporal Self-Organization for Neural Networks (PhD thesis)
Principe’s supervisee and collaborator explores three related approaches for temporal self-organisation in ANNs. SOTPAR and SOTPAR2 seem to correspond to SOMTAD and GASTAD in the above paper. Additional results are reported on phoneme sequences with SOTPAR and time series prediction with SOTPAR2. The third approach explored is ‘Dynamic Subgrouping of RTRL [real-time recurrent learning] in Recurrent Neural Networks’, an approach to supervised dynamic learning.
- Neil R. Euliano, Jose C. Principe (1996), Spatio-temporal self organizing feature maps
Some of the work leading up to Euliano’s thesis, above.
- Neil R. Euliano, Jose C. Principe (2000), Dynamic Subgrouping in RTRL Provides a Faster O(N2) Algorithm
Further work on RTRL.
- Jose C. Principe, Neil R. Euliano and W.Curt Lefebvre (1999) Neural and Adaptive Systems: fundamentals through simulations
A 672-page book which I have not yet had a chance to see, though the National Library of Scotland has a copy.
- Cho et al (inc. Principe) (2007), Self-organizing maps with dynamic learning for signal reconstruction
Signal reconstruction for brain-machine interfaces, compressing and reconstructing neural spike data.