Introduction to neural networks

Neural networks have been here for quite some time, with initial papers arriving more than a few decades ago. The recent popularity is due to the availability of better software for algorithms and proper hardware to run them. Initially, neural networks were motivated by how humans perceive the world and were modeled according to biological neuron functions. This was continuously modified over the course of time and has since been evolving to get a better performance. 

A simple neural network

A simple neural net consists of a node which takes in an input or a list of inputs and performs a transformation. An example is shown in the following figure:

Mathematically, it takes the inputs x and applies a transformation W to get the output y:

The input x can be a vector or multi-dimensional array. Based on the transformation matrix W we get the output y to be a vector or multi-dimensional array. This structure is further modified by also including the non-linear transformation...