Recurrent Neural Network and its Variants — Linking great past to grand future

Getting Notalgic with RNN

Many machine learning models have been tried in literature to find meaningful trends out of the data collected. Various models and algorithms have evolved in past to solve problems as simple as finding “Price of apartment in Bay Area” to something as complex as dealing with varied signals i.e. Photos, Videos, Speech and Natural language text. Of course the more complex data and their representations(dimensions) are more varied are the techniques to find trends from them.

Without a doubt, Optimization techniques of form either convex or non-convex have made learning tasks possible. Simpler and not much scalable models used linear algebra, matrix operations and probablility theorems. Neural Networks have revolutionised the whole space of models with their own version of methodologies falling under branch of deep learning working with production scale datasets. CNN, RNN, GAN are some of basic models that worked pretty well and now their variants and specialized forms are creating new piece of Research litrature.

Convolutional neural networks(CNN) are simply neural networks that use convolution in place of general matrix multiplication in at least one of their layers and are specialised for processing data that has a known, grid-like topology. Convolution leverages three important ideas…