Loading...
*  Recurrent neural network
ISBN 0-471-49517-4. RNNSharp CRFs based on recurrent neural networks (C#, .NET) Recurrent Neural Networks with over 60 RNN ... discrete time recurrent neural networks can be viewed as continuous-time recurrent neural networks where the differential ... "Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling". arXiv:1412.3555 [cs.NE]. "Recurrent Neural ... A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle ...
*  Bidirectional recurrent neural networks
Translation Handwritten Recognition Protein Structure Prediction Artificial neural network Recurrent neural networks Long short ... Standard recurrent neural network (RNNs) also have restrictions as the future input information cannot be reached from the ... Bidirectional Recurrent Neural Networks (BRNN) were invented in 1997 by Schuster and Paliwal. BRNNs were introduced to increase ... "Bidirectional recurrent neural networks." Signal Processing, IEEE Transactions on 45.11 (1997): 2673-2681.2. Awni Hannun, Carl ...
*  Geoffrey Hinton
Sutskever, Ilya (2013). Training Recurrent Neural Networks. proquest.com (PhD thesis). University of Toronto. OCLC 889910425. ... His other contributions to neural network research include distributed representations, time delay neural network, mixtures of ... He is the director of the program on "Neural Computation and Adaptive Perception" which is funded by the Canadian Institute for ... Hinton taught a free online course on Neural Networks on the education platform Coursera in 2012. Hinton joined Google in March ...
*  Almeida-Pineda recurrent backpropagation
A recurrent neural network for this algorithm consists of some input units, some output units and eventually some hidden units ... Pineda, Fernando (9 November 1987). ""Generalization of Back-Propagation to Recurrent Neural Networks". Physical Review Letters ... Almeida-Pineda recurrent backpropagation is an extension to the backpropagation algorithm that is applicable to recurrent ... IEEE First International Conference on Neural Networks. San Diego, CA, USA. pp. 608-18. ...
*  Hava Siegelmann
Cabessa, J.; Siegelmann, H. T. (2012). "The Computational Power of Interactive Recurrent Neural Networks". Neural Computation. ... In the early 1990s, she and Eduardo D. Sontag proposed a new computational model, the Artificial Recurrent Neural Network (ARNN ... For her lifetime contribution to the field of Neural Networks she is the recipient of the 2016 Donald Hebb Award. She earned ... She is on the governing board of the International Neural Networks Society, and an editor in the Frontiers on Computational ...
*  Evolutionary acquisition of neural topologies
An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5:54-65, 1994. [1] ... The encoding has important properties that makes it suitable for evolving neural networks: It is complete in that it is able to ... Evolutionary acquisition of neural topologies (EANT/EANT2) is an evolutionary reinforcement learning method that evolves both ... For evolving the structure and weights of neural networks, an evolutionary process is used, where the exploration of structures ...
*  Intelligent maintenance system
Heimes, F. O. (6-9 October 2008). "Recurrent neural networks for remaining useful life estimation". International Conference on ...
*  Spectral radius
To avoid the vanishing gradient problem and the exploding gradient problem in a recurrent neural network (RNN), it is desired ... "On the difficulty of training recurrent neural networks". Proceedings of the 30th International Conference on Machine Learning ...
*  Hypercomputation
"The Computational Power of Interactive Recurrent Neural Networks" (PDF). Neural Computation. 24 (4): 996-1019. doi:10.1162/neco ... Similarly, a neural net that somehow had Chaitin's constant exactly embedded in its weight function would be able to solve the ... Hava Siegelmann; Eduardo Sontag (1994). "Analog Computation via Neural Networks". Theoretical Computer Science. 131 (2): 331- ... "Analog Computation via Neural Networks" (PDF). Theoretical Computer Science. 131: 331-360. doi:10.1016/0304-3975(94)90178-3. ...
*  Liquid state machine
Echo state network: similar concept in recurrent neural network. Reservoir computing: the conceptual framework. Self-organizing ... a new framework for neural computation based on perturbations" (PDF), Neural Comput, 14 (11): 2531-60, doi:10.1162/ ... The recurrent nature of the connections turns the time varying input into a spatio-temporal pattern of activations in the ... A liquid state machine (LSM) is a particular kind of spiking neural network. An LSM consists of a large collection of units ( ...
*  Paul Werbos
He also was a pioneer of recurrent neural networks. Werbos was one of the original three two-year Presidents of the ... he was awarded the IEEE Neural Network Pioneer Award for the discovery of backpropagation and other basic neural network ... which first described the process of training artificial neural networks through backpropagation of errors. The thesis, and ...
*  Backpropagation through time
With recurrent neural networks, local optima are a much more significant problem than with feed-forward neural networks. The ... Consider an example of a neural network that contains a recurrent layer f {\displaystyle f} and a feedforward layer g {\ ... The algorithm was independently derived by numerous researchers The training data for a recurrent neural network is an ordered ... BPTT begins by unfolding a recurrent neural network in time. The unfolded network contains k {\displaystyle k} inputs and ...
*  Ronald J. Williams
A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1, 270-280. R. J. Williams ... He also made fundamental contributions to the fields of recurrent neural networks and reinforcement learning. David E. ... Ronald J. Williams is professor of computer science at Northeastern University, and one of the pioneers of neural networks. He ... Gradient-based learning algorithms for recurrent networks and their computational complexity. In Back-propagation: Theory, ...
*  Gain-field encoding
Muscle memory Neural coding Encoding (Memory) Recurrent neural network Gain Polack, Pierre-Olivier; Friedman, Jonathan; ... The primary process by which this interaction can take place is speculated to be recurrent neural networks where neural ... This multiplicative property is an effect of recurrent neural circuitry. A target neuron that takes only two types of direct ... For example, neural activity for the interaction between gaze direction and retinal image location is almost exactly ...
*  Alex Graves (computer scientist)
In 2009, his CTC-trained LSTM was the first recurrent neural network to win pattern recognition contests, winning several ... Santiago Fernandez, Alex Graves, and Jürgen Schmidhuber (2007). An application of recurrent neural networks to discriminative ... Graves, Alex; and Schmidhuber, Jürgen; Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks, in ... Graves is also the creator of neural Turing machines and of the closely related differentiable neural computer. "Alex Graves - ...
*  Memetic algorithm
Ku K. W. C. and Mak M. W. and Siu W. C. (2000). "A study of the Lamarckian evolution of recurrent neural networks". IEEE ... Ichimura, T.; Kuriyama, Y. (1998). Learning of neural networks with parallel hybrid GA using a royal road function. IEEE ... More recent applications include (but are not limited to) training of artificial neural networks, pattern recognition, robotic ... Aguilar, J.; Colmenares, A. (1998). "Resolution of pattern recognition problems using a hybrid genetic/random neural network ...
*  Stochastic gradient descent
Sutskever, Ilya (2013). Training recurrent neural networks (PDF) (Ph.D.). University of Toronto. p. 74. Zeiler, Matthew D. ( ... on Neural Networks (IJCNN). IEEE. Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (8 October 1986). "Learning ... COURSERA: Neural Networks for Machine Learning Hinton, Geoffrey. "Overview of mini-batch gradient descent" (PDF). pp. 27-29. ... "Efficient backprop." Neural networks: Tricks of the trade. Springer Berlin Heidelberg, 2012. 9-48 Díaz, Esteban and Guitton, ...
*  Gated recurrent unit
"Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling". arXiv:1412.3555 [cs.NE]. "Recurrent Neural ... Gated recurrent units (GRUs) are a gating mechanism in recurrent neural networks, introduced in 2014 by Kyunghyun Cho et al.. ...
*  Jeffrey Elman
In 1990, he introduced the simple recurrent neural network (SRNN; aka 'Elman network'), which is a widely used recurrent neural ... He is a psycholinguist and pioneer in the field of neural networks. In 1990, he introduced the simple recurrent neural network ...
*  Lee Giles
Earlier research was concerned with recurrent neural networks and optical computing. His research interests are in intelligent ... Giles' work on neural networks showed that fundamental computational structures such as regular grammars and finite state ... machines could be theoretically represented in recurrent neural networks. Another contribution was the Neural Network Pushdown ... Giles is a Fellow of the Association for Computing Machinery (ACM), IEEE and International Neural Networks Society, INNS. He ...
*  Handwriting recognition
... the recurrent neural networks and deep feedforward neural networks developed in the research group of Jürgen Schmidhuber at the ... Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks, in Bengio, Yoshua; Schuurmans, Dale; Lafferty ... Neural networks are quick to set up; however, they can be inaccurate if they learn properties that are not important in the ... Each neural network uniquely learns the properties that differentiate training images. It then looks for similar properties in ...
*  Bidirectional associative memory
... (BAM) is a type of recurrent neural network. BAM was introduced by Bart Kosko in 1988. There ...
*  Connectionist temporal classification
... for training recurrent neural networks (RNNs) such as LSTM networks to tackle sequence problems where the timing is variable. ... Labelling unsegmented sequence data with recurrent neural networks". In Proceedings of the International Conference on Machine ... CTC refers to the outputs and scoring, and is independent of the underlying neural network structure. It was introduced in 2006 ... CTC scores can then be used with the back-propagation algorithm to update the neural network weights. Alternative approaches to ...
*  Gene regulatory network
This model is formally closer to a higher order recurrent neural network. The same model has also been used to mimic the ... Also, artificial neural networks omit using a hidden layer so that they can be interpreted, losing the ability to model higher ... Formally most of these approaches are similar to an artificial neural network, as inputs to a node are summed up and the result ... Some other recent work has used artificial neural networks with a hidden layer. Body plan Cis-regulatory module Genenetwork ( ...
*  Sepp Hochreiter
Unlike NNs, recurrent neural networks (RNNs) can use their internal memory to process arbitrary sequences of inputs. If data ... LSTM overcomes the problem of recurrent neural networks (RNNs) and deep networks to forget information over time or, ... Sharing of Very Short IBD Segments between Humans, Neandertals, and Denisovans Recurrent Neural Networks and LSTM LSTM Home ... Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. A Field Guide to Dynamical Recurrent ...