Gradient Descent
ELI5 — The Vibe Check
Gradient Descent is how an AI learns — it's the algorithm that nudges the model's weights in the right direction after each mistake. Imagine you're blindfolded on a hilly landscape and you want to reach the lowest point. You feel which way is downhill and take a small step. Repeat millions of times and you reach the bottom. That 'downhill direction' is the gradient.
Real Talk
Gradient descent is the optimization algorithm used to minimize a loss function by iteratively updating model parameters in the direction opposite to the gradient of the loss with respect to those parameters. Variants include SGD, Adam, AdaGrad, and RMSProp. The learning rate controls step size. Modern deep learning uses mini-batch gradient descent.
When You'll Hear This
"The model optimizes via gradient descent." / "Adam is a variant of gradient descent with adaptive learning rates."
Related Terms
Backpropagation
Backpropagation is how errors flow backwards through a neural network during training.
Loss Function
The loss function is the AI's score of how badly it's doing.
Training
Training is the long, expensive process where an AI learns from data.
Weights
Weights are the numbers inside a neural network that determine what it knows and how it behaves — they're the AI's 'brain cells.