Top-p
ELI5 — The Vibe Check
Top-p (also called nucleus sampling) is another dial that controls how an AI picks its next word. Instead of choosing from ALL possible words, it only considers words that together add up to P% of the probability. Set it to 0.9 and it only considers the most likely words until they total 90% of the probability. It's like letting the AI only pick from the 'reasonable' options.
Real Talk
Top-p (nucleus sampling) is a token sampling strategy where the model only samples from the smallest set of tokens whose cumulative probability exceeds the threshold p. This dynamically adjusts the candidate vocabulary, avoiding both tail tokens and overly restrictive greedy decoding. It is often used in combination with temperature.
When You'll Hear This
"Use top-p of 0.95 to avoid incoherent outputs." / "Top-p and temperature work together."
Related Terms
Inference
Inference is when the AI actually runs and generates output — as opposed to training, which is when it's learning.
LLM (Large Language Model)
An LLM is a humongous AI that read basically the entire internet and learned to predict what words come next, really really well.
Temperature
Temperature controls how creative (or chaotic) an AI's responses are. Low temperature (like 0.1) makes it boring, safe, and predictable — great for code.
Token
In AI-land, a token is a chunk of text — roughly 3/4 of a word.
Top-k
Top-k limits the AI's word choices to the K most likely options. If K is 50, the AI only picks from the top 50 most probable words for each step.