Neural networks are a series of algorithms that identify underlying relationships in a set of data. These algorithms are heavily based on the way a human brain operates. These networks can adapt to changing input and generate the best result without the requirement to redesign the output criteria. In a way, these neural networks are similar to the systems of biological neurons.
Deep learning is an important part of machine learning, and the deep learning algorithms are based on neural networks. There are several neural network architectures with different features, suited best for particular applications. Here, we are going to explore some of the most prominent architectures, particularly in context to deep learning.
Multilayer Perceptrons
Multilayer Perceptron (MLP) is a class of feed-forward artificial neural networks. The term perceptron particularly refers to a single neuron model that is a precursor to a larger neural network.
An MLP consists of three main layers of nodes — an input layer, a hidden layer, and an output layer. In the hidden and the output layer, every node is considered as a neuron that uses a nonlinear activation function. MLP uses a supervised learning technique called backpropagation for training. When a neural network is initialised, weights are set for each neuron. Backpropagation helps in adjusting the weights of the neurons to obtain output closer to the expected.
MLPs are most ideal for projects involving tabular datasets, classification prediction problems, and regression prediction problems.
Convolution Neural Network
Convolution neural network (CNN) model processes data that has a grid pattern such as images. It is designed to learn spatial hierarchies of features automatically. CNN typically comprises three types of layers, also referred to as blocks — convolution, pooling, and fully-connected layers.
The convolution and pooling layers perform feature extraction, and these extracted features are mapped into the final output by the fully connected layer. CNN is best suited for image processing.
Some of the applications areas of CNN are in image recognition, image classification, object detection, and face recognition.
Recurrent Neural Networks
In Recurrent Neural Networks (RNN), the output from the previous step is fed back as input to the current step. The hidden layer in the RNN enables this feedback system. This hidden state can store some information about the previous steps in a sequence.
The ‘memory’ in RNN helps the model in remembering all the information that has been calculated. It, in turn, uses these same parameters for each of the inputs to produce the output, thereby reducing the complexity of parameters.
RNN is one of the most widely used types of neural networks, primarily because of its greater learning capacity and its ability to perform complex tasks such as learning handwritings or in language recognition. Some of the other fields where RNN finds application is — prediction problems, machine translation, video tagging, text summarisation, and even music composition.
Deep Belief Network
The Deep Belief Networks (DBN) use probabilities and unsupervised learning to generate the output. DBNs consist of binary latent variables, undirected layers, and directed layers. DBNs are unlike other models as every layer is regulated in order, and each one of them learns the entire input.
In DBNs, each sub-network’s hidden layer is a visible layer for the next one. This composition enables quick layer-by-layer unsupervised training procedure where contrastive divergence is applied to each sub-network, starting with the lowest visible layer. Greedy learning algorithms are used to train DBNs. The learning takes one layer at a time. Hence, each layer receives different versions of the data, and each layer uses the output from the previous layer as its input.
DBNs find major applications in image recognition, video recognition, and motion capture data.
Restricted Boltzmann Machine
The Boltzmann Machine (RBM) is a generative and non-deterministic (stochastic) neural network that learns probability distribution over its set of inputs. RBMs are shallow, two-layer neural networks that constitute the building blocks of deep-belief networks. The first layer in an RBM is called the visible or the input layer, and the second one is called the hidden layer. It consists of a neuron-like unit called node; the nodes are connected to each other across layers but not within the same layer.
RBMs are generally used in building applications such as dimensionality reduction, recommender systems, and topic modelling. However, in recent years generative adversarial networks are slowly replacing RBMs.
