Training a neural network is the process of gradually adjusting its internal weights so that its predictions become more accurate. While the underlying maths can get complex, the overall training loop follows a consistent, repeatable pattern.
Step 1: Forward Pass
A batch of training examples is fed through the network from the input layer to the output layer, with each layer applying its current weights to produce a prediction. Early in training, since the weights start out essentially random, these predictions are usually far from correct.
Step 2: Measuring the Error
A loss function compares the network's prediction to the actual correct answer and produces a single number representing how wrong the prediction was. The entire goal of training is to make this loss number as small as possible across all training examples.
Step 3: Backpropagation and Weight Updates
Using a technique called backpropagation, the network calculates how much each individual weight contributed to the error, then nudges every weight slightly in the direction that would reduce that error, a process guided by an optimisation method called gradient descent. This entire cycle, forward pass, loss calculation, and weight update, repeats for many batches of data and multiple passes, called epochs, until performance stops improving meaningfully.
Once training completes, the network's weights are frozen and it moves into evaluation, where it is tested on separate data it never saw during training to confirm it has genuinely learned useful patterns rather than just memorised the training set.