By ShilpBot Team · 8 min read · Updated
A neural network is a mathematical function made of many small units called neurons, loosely inspired by the brain. By adjusting millions of numbers inside it, the network learns to map inputs, like image pixels, to outputs, like the label "cat".
The building blocks
Neurons and weights
Each artificial neuron takes several numbers as input, multiplies each by a weight that says how important that input is, adds them up, adds a bias, and passes the result through an activation function. The activation function introduces non-linearity, which is what allows a network to learn curved, complex patterns rather than only straight-line relationships.
Layers
Neurons are arranged in layers:
- The input layer receives the raw data, such as pixel values.
- Hidden layers transform the data step by step. Networks with many hidden layers are called deep networks.
- The output layer gives the final answer, such as probabilities for each class.
Learning in four steps
Training repeats the same cycle many times.
- Forward pass: feed an example through the network and get a prediction.
- Loss calculation: a loss function measures how wrong the prediction was. A bigger number means a worse guess.
- Backpropagation: the network works backwards to calculate how much each weight contributed to the error.
- Weight update: gradient descent nudges every weight slightly in the direction that reduces the loss.
Gradient descent in one picture
Imagine standing on a foggy hillside and trying to reach the valley floor. You cannot see the whole landscape, but you can feel which way the ground slopes under your feet. You take a small step downhill, feel again, and repeat. The size of each step is the learning rate. Steps that are too large overshoot the valley, and steps that are too small take forever.
Key training terms
| Term | Meaning |
|---|---|
| Epoch | One full pass through the training dataset |
| Batch | A small group of examples processed together |
| Learning rate | How big each weight update is |
| Loss | A number measuring how wrong the model is |
| Parameter | A learned number, such as a weight or bias |
Overfitting and how to reduce it
A network with enough capacity can memorise its training examples, including their noise, and then fail on new data. Signs of overfitting are low training error with much higher test error. Common remedies include:
- Getting more and more varied training data
- Using a simpler model
- Dropout, which randomly switches off neurons during training
- Early stopping, which halts training when test performance stops improving
Different networks for different data
- Convolutional neural networks (CNNs) specialise in images by scanning for local patterns like edges and shapes.
- Recurrent networks and transformers handle sequences such as text, audio and sensor streams.
Robots commonly use CNN-based models to interpret camera images, which we explore next in Computer Vision for Robots.
Key takeaways
- A network is layers of weighted sums plus activation functions.
- Learning means adjusting weights to reduce a loss, using backpropagation and gradient descent.
- Always judge a model on data it has not seen during training.
Frequently asked questions
What does backpropagation do?
Backpropagation calculates, efficiently and layer by layer from the output back to the input, how much each weight contributed to the prediction error. Gradient descent then uses that information to update the weights.
How many layers make a network deep?
There is no strict cutoff. Networks with more than a couple of hidden layers are usually called deep, and modern models can have dozens or hundreds of layers.
Why do neural networks need so much data?
They contain a very large number of adjustable weights. Without enough varied examples they tend to memorise the training set instead of learning general patterns.
