of the network. One approach comprises of one full training cycle on the training set.
Validation Set Approach
A set of examples used to tune the parameters [i.e., architecture] of the network. For example to choose the number of hidden units in a Neural Network.
Making Test Set
A set of examples used only to assess the performance [generalization] of a fully specified network or to apply successfully in predicting output whose input is known.
Five Algorithms to Train a Neural Network
- Hebbian learning Rule
- Self – Organizing Kohonen Rule
- Hopfield Network Law
- LMS algorithm (Least Mean Square)
- Competitive Learning
Artificial Neural Network Architecture
A typical Neural Network contains a large number of artificial neurons called units arranged in a series of layers.
In typical Artificial Neural Network, comprises different layers –
- Input layer – It contains those units (Artificial Neurons) which receive input from the outside world on which network will learn, recognize about or otherwise process.
- Output layer – It contains units that respond to the information about how it’s learned any task.
- Hidden layer – These units are in between input and output layers. The job of the hidden layer is to transform the input into something that output unit can use in some way.
Most Neural Networks are fully connected that means to say each hidden neuron is fully linked to every neuron in its previous layer(input) and to the next layer (output) layer.
Learning Techniques in Neural Networks
In supervised learning, the training data is input to the network, and the desired output is known weights are adjusted until production yields desired value.
The input data is used to train the network whose output is known. The network classifies the input data and adjusts the weight by feature extraction in input data.
Here the value of the output is unknown, but the network provides the feedback on whether the output is right or wrong. It is Semi-Supervised Learning.
The adjustment of the weight vector and threshold is made only after all the training set is presented to the network. It is also called Batch Learning.
The adjustment of the weight and threshold is made after presenting each training sample to the network.
Learning and Development in Neural Networks
Learning occurs when the weights inside the network get updated after many iterations.
For example – Suppose we have inputs in the form of patterns for two different class of patterns – I & 0 as shown and b -bias and y as the desired output.
We want to classify input patterns into either pattern ‘I’ & ‘O.’
Following are the steps performed:
- Nine inputs from x1 – x9 along with bias b (input having weight value 1) is fed to the network for the first pattern.
- Initially, weights are initialized to zero.
- Then weights are updated for each neuron using the formulae: Δ wi = xi y for i = 1 to 9 (Hebb’s Rule)
- Finally, new weights are found using the formulae:
- wi(new) = wi(old) + Δwi
- Wi(new) = [111-11-1 1111]
- The second pattern is input to the network. This time, weights are not initialized to zero. The initial weights used here are the final weights obtained after presenting the first pattern. By doing so, the network
- The steps from 1 – 4 are repeated for second inputs.
- The new weights are Wi(new) = [0 0 0 -2 -2 -2 000]
So, these weights correspond to the learning ability of the network to classify the input patterns successfully.
4 Different Techniques of Neural Networks
Classification Neural Network
A Neural Network can be trained to classify given pattern or dataset into predefined class. It uses Feedforward Networks.
Prediction Neural Network
A Neural Network can be trained to produce outputs that are expected from a given input. E.g., – Stock market prediction.
Clustering Neural Network
The Neural network can be used to identify a unique feature of the data and classify them into different categories without any prior knowledge of the data.
Following networks are used for clustering –
- Competitive networks
- Adaptive Resonance Theory Networks
- Kohonen Self-Organizing Maps.
Association Neural Network
A Neural Network can be trained to remember the particular pattern so that when the noise pattern is presented to the network, the network associates it with the closest one in the memory or discard it. E.g. , Hopfield Networks which performs recognition, classification, and clustering, etc.
Neural Networks for Pattern Recognition
Pattern recognition is the study of how machines can observe the environment, learn to distinguish patterns of interest from their background, and make sound and reasonable decisions about the categories of the patterns.
Some examples of the pattern are – fingerprint image, a handwritten word, human face or speech signal.
Given an input pattern, its recognition involves the following task –
- Supervised classification – Given the input pattern is identified as the member of a predefined class.
- Unsupervised classification – Pattern is assigned to a hitherto unknown class.
So, the recognition problem here is essentially a classification or categorized task.
The design of pattern recognition systems usually involves the following three aspects-
- Data acquisition and preprocessing
- Data representation
- Decision Making
Approaches For Pattern Recognition
- Template Matching
- Syntactic Matching
- Artificial Neural Networks
Following Neural Network architectures used for Pattern Recognition –
- Multilayer Perceptron
- Kohonen SOM (Self Organizing Map)
- Radial Basis Function Network (RBF)
Neural Network for Deep Learning
Following Neural Network, architectures are used in Deep Learning
- Feed-forward neural networks
- Recurrent neural network
- Multi-layer perceptrons (MLP)
- Convolutional neural networks
- Recursive neural networks
- Deep belief networks
- Convolutional deep belief networks
- Self-Organizing Maps
- Deep Boltzmann machines
- Stacked de-noising auto-encoders
Neural Networks and Fuzzy Logic
Fuzzy logic refers to the logic developed to express the degree of truthiness by assigning values in between 0 and 1, unlike traditional boolean logic that represents 0 and 1.
Fuzzy logic and Neural networks have one thing in common. They can be used to solve problems of pattern recognition and others that do not involve any mathematical model.
Systems combining both fuzzy logic and neural networks are neuro-fuzzy systems.
These systems (Hybrid) can combine advantages of both neural networks and fuzzy logic to perform in a better way.
Fuzzy logic and Neural Networks have been integrated to use in the following applications –
- Automotive engineering
- Applicant screening of jobs
- Control of crane
- Monitoring of glaucoma
In a hybrid (neuro-fuzzy) model, Neural Networks Learning Algorithms are fused with the fuzzy reasoning of fuzzy logic.
The neural network determines the values of parameters, while if-then rules are handled by fuzzy logic.
Neural Network for Machine Learning
- Multilayer Perceptron (supervised classification)
- Back Propagation Network (supervised classification)
- Hopfield Network (for pattern association)
- Deep Neural Networks (unsupervised clustering)
Applications of Neural Networks
Neural networks have been successfully applied to the broad spectrum of data-intensive applications, such as:
|Application||Architecture / Algorithm||Activation Function|
|Process modeling and control||Radial Basis Network||Radial Basis|
|Machine Diagnostics||Multilayer Perceptron||Tan- Sigmoid Function|
|Portfolio Management||Classification Supervised Algorithm||Tan- Sigmoid Function|
|Target Recognition||Modular Neural Network||Tan- Sigmoid Function|
|Medical Diagnosis||Multilayer Perceptron||Tan- Sigmoid Function|
|Credit Rating||Logistic Discriminant Analysis with ANN, Support Vector Machine||Logistic function|
|Targeted Marketing||Back Propagation Algorithm||Logistic function|
|Voice recognition||Multilayer Perceptron, Deep Neural Networks( Convolutional Neural Networks)||Logistic function|
|Financial Forecasting||Backpropagation Algorithm||Logistic function|
|Intelligent searching||Deep Neural Network||Logistic function|
|Fraud detection||Gradient – Descent Algorithm and Least Mean Square (LMS) algorithm.||Logistic function|
Advantages of Neural Networks
- A neural network can perform tasks that a linear program can not.
- When an element of the neural network fails, it can continue without any problem by their parallel nature.
- A neural network learns and does not need to be reprogrammed.
- It can be implemented in any application.
- It can be performed without any problem.
Limitations of Neural Networks
- The neural network needs the training to operate.
- The architecture of a neural network is different from the architecture of microprocessors, therefore, needs to be emulated.
- Requires high processing time for large neural networks.
Face Recognition Using Artificial Neural Networks
Face recognition entails comparing an image with a database of saved faces to identify the person in that input picture. Face detection mechanism involves dividing images into two parts; one containing targets (faces) and one providing the background.
The associated assignment of face detection has direct relevance to the fact that images need to be analyzed and faces identified, earlier than they can be recognized.
Learning Rules in Neural Network
The learning rule is a type of mathematical logic. It encourages a Neural Network to gain from the present conditions and upgrade its efficiency and performance.
The learning procedure of the brain modifies its neural structure. The expanding or diminishing quality of its synaptic associations rely upon their activity.
Learning rules in the Neural network:
- Hebbian learning rule ; It determines, how to customize the weights of nodes of a system.
- Perceptron learning rule; Network starts its learning by assigning a random value to each load.
- Delta learning rule; Modification in sympatric weight of a node is equal to the multiplication of error and the input.
- Correlation learning rule; It is similar to supervised learning.
How Can XenonStack Help You?
XenonStack can help you develop and deploy your model solutions based on Neural Networks. Whatever kind of problem you face – Prediction, Classification or Pattern Recognition – XenonStack has a solution for you.
Unlock the real value of your data to make smarter decisions. Embrace Machine Learning Services in your Enterprise to solve complex Business Problems. XenonStack offers Data Cleansing and Extraction Services, Data Modeling Services, and Algorithm Development.
Fraud Detection & Prevention Services
XenonStack Fraud Detection Services offers real-time fraud analysis to increase profitability. Data Mining is used to quickly detect fraud and search for spot patterns and detect fraudulent transactions. Data Mining Tools like Machine Learning, Neural Networks, Cluster Analysis are used to generate Predictive Models to prevent fraud losses.
Data Modeling Services
XenonStack offers Data Modelling using Neural Networks, Machine Learning, and Deep Learning. Data Modelling services help Enterprises to create a conceptual model based on the analysis of data objects. Deploy your Data Models on leading Cloud Service Providers like Google Cloud, Microsoft Azure, AWS or on the container environment – Kubernetes & Docker.