Backpropagation
ELI5 — The Vibe Check
Backpropagation is how errors flow backwards through a neural network during training. The model makes a prediction, computes how wrong it was, and then sends that 'wrongness signal' backwards through all the layers, assigning blame to each weight. Then gradient descent uses those blame scores to fix the weights. It's the entire reason neural networks can learn.
Real Talk
Backpropagation is the algorithm for computing gradients of the loss function with respect to all model parameters. Using the chain rule of calculus, it propagates the error signal backward from the output layer through all intermediate layers. These gradients are then used by the optimizer (e.g., Adam) to update parameters.
When You'll Hear This
"Backpropagation through time is tricky for sequential models." / "The exploding gradient problem breaks backpropagation."
Related Terms
Gradient Descent
Gradient Descent is how an AI learns — it's the algorithm that nudges the model's weights in the right direction after each mistake.
Loss Function
The loss function is the AI's score of how badly it's doing.
Neural Network
A neural network is a system loosely inspired by the human brain — lots of little math nodes connected together, passing numbers to each other.
Training
Training is the long, expensive process where an AI learns from data.
Weights
Weights are the numbers inside a neural network that determine what it knows and how it behaves — they're the AI's 'brain cells.