A Neural Network: What is it?

Artificial Neural Networks (ANNs) are a type of machine learning algorithm, also a method in Artificial Intelligence (AI) inspired by the structure and function of biological neurons. ANNs are capable of learning and adapting to complex data, making them useful in various applications like image classification, natural language processing, speech recognition, and more.

Neural Networks are like the workhorses of Deep learning. With enough data and computational power, they can be used to solve most of the problems in deep learning. It is very easy to use a Python or R library to create a neural network and train it on any dataset and get great accuracy.

ANNs consist of multiple layers of interconnected nodes that process information and make predictions based on patterns in the data provided. They have been used in fields such as finance, healthcare, and marketing to improve decision-making and automate tasks. One of the advantages of ANNs is their ability to learn from large datasets and generalize to new data. 

However, their performance can be affected by factors such as the quality and quantity of training data, the complexity of the network architecture, and the choice of hyperparameters. 

In this guide, we will create our own neural network in Python from scratch. We will build a simple feedforward neural network with backpropagation and use it to classify handwritten digits from the MNIST dataset.

Step 1: Install Required Libraries

We will use the following Python libraries:

• NumPy – for numerical computations.
• Matplotlib – for visualizing the data.
• scikit-learn – for loading and preprocessing the MNIST dataset.

To install these libraries, open your terminal and type the following commands:

Python Code 

pip install numpy pip install matplotlib pip install scikit-learn

Step 2: Load and Preprocess the Data

We will make use of the 60,000 training and 10,000 test handwritten digit images from the MNIST dataset. The images are 28×28 pixels in size and are grayscale. 

The built-in function of scikit-learn will be used to load and preprocess the data.

Step 3: Define the Neural Network Architecture

The architecture will consist of an input layer, one or more hidden layers, and an output layer. Each layer will have a specific number of neurons and will be connected to the next layer through weighted connections.

More precisely our neural network will have the following architecture: 

• Input layer with 784 neurons (28×28 pixels).
• Hidden layer with 64 neurons and ReLU activation function.
• Output layer with 10 neurons and softmax activation function.

Step 4: Compute the gradients

For those who don’t know what gradients are, a gradient measures how all weights have changed in relation to the change in error.You can also think of a gradient as the slope of a function. The gradient descent algorithm uses this slope to iteratively adjust the weights in order to minimize the error and improve the accuracy of the model. This process is repeated until the algorithm converges on a set of weights that produce the lowest possible error. Just keep in mind it cannot be zero, zero means that the model has stopped learning.

Step 5: Create a neural network object

Now you have to declare parameters to create your neural network by using the command lines below:  

input_size = 784

hidden_size = 64

output_size = 10

learning_rate = 0.1

num_epochs = 10

nn = NeuralNetwork(input_size, hidden_size, output_size)

By using the above lines you will be able to initialize a neural network with 784 input features, 64 hidden neurons, and 10 output neurons. And  run the backpropagation algorithm for 10 epochs with a learning rate of 0.1.

Step 6 : Train the neural network

Once the neural network object is created, it can now be trained using the dataset and then used to make predictions on new data. It is important to choose the appropriate architecture and hyperparameters for the neural network to achieve optimal performance.

nn.train(X_train, y_train, learning_rate, num_epochs)

Step 7: Evaluate the neural network

We will evaluate the performance of the network by comparing the predicted output with the actual output and calculating a metric such as accuracy. Additionally, we may also analyze any patterns or trends in the errors made by the network to identify areas for improvement.

train_accuracy = np.mean(nn.predict(X_train) == y_train)

test_accuracy = np.mean(nn.predict(X_test) == y_test)

print(‘Train accuracy:’, train_accuracy)

print(‘Test accuracy:’, test_accuracy)

Step 4: Evaluate the Neural Network We will evaluate our neural network by computing the accuracy on the training and test sets. “`python # Evaluate the neural network train_accuracy = np.mean(nn.predict(X_train) == y_train) test_accuracy = np.mean(nn.predict(X_test) == y_test) print(‘Train accuracy:’, train_accuracy) print(‘Test accuracy:’, test_accuracy)

Output:

yaml

Train accuracy: 0.9752083333333333 Test accuracy: 0.9655

We can see that our neural network is able to correctly predict the outputs for the new data with a very high training and testing accuracy.

Conclusion:

In this guide, we created a simple neural network in Python from scratch. We used the MNIST dataset to train and evaluate the neural network. Our neural network achieved an accuracy of 96.55% on the test set. With further optimizations and modifications, we can improve the performance of our neural network.

It is important to note that neural networks are a powerful tool in machine learning and can be applied to various tasks beyond image classification. As you continue to explore and learn about neural networks, you can experiment with different architectures, activation functions, and optimization algorithms to improve their performance on different datasets and tasks.

FAQ’s

Is it hard to build a neural network from scratch?

It is very simple to create a neural network, train it on any dataset, and get excellent accuracy by using a Python library and modules. However, building a neural network from scratch requires a strong understanding of the underlying mathematical concepts and algorithms involved, which can be challenging for beginners. It also requires significant time and effort to fine-tune the network’s architecture and parameters for optimal performance.

Can you build a neural network without using any library?

Using the NumPy library can significantly simplify the process of building neural networks, as it provides efficient numerical operations and array handling capabilities. However, if you prefer not to use NumPy, you can still build neural networks using other libraries or by implementing the necessary operations from scratch. In this case, we will be using the matplotlib library to visualize and plot the loss as we train our model for a specific number of epochs. This will allow us to monitor the progress of our neural network and make any necessary adjustments to improve its performance.

Can a neural network learn itself?

A neural network can, in fact, learn on its own. This process is known as reinforcement learning, in which the network learns on a hit-and-miss basis. During reinforcement learning, the neural network receives feedback from its environment and adjusts its parameters accordingly to maximize a reward signal. This allows the network to learn and improve its performance without explicit instruction from a programmer.