Top-k
ELI5 — The Vibe Check
Top-k limits the AI's word choices to the K most likely options. If K is 50, the AI only picks from the top 50 most probable words for each step. It's like telling someone they can only order from the top 50 items on the menu — still lots of choice, but you won't accidentally end up with the weird exotic stuff nobody orders.
Real Talk
Top-k sampling restricts the token sampling pool to the K highest-probability tokens at each generation step. Unlike top-p, the number of candidates is fixed regardless of the probability distribution shape. Lower K values produce more focused output; higher K values allow more diversity. Often combined with temperature and top-p.
When You'll Hear This
"Set top-k to 40 for more coherent generation." / "Top-k and top-p can be used together."
Related Terms
Inference
Inference is when the AI actually runs and generates output — as opposed to training, which is when it's learning.
Temperature
Temperature controls how creative (or chaotic) an AI's responses are. Low temperature (like 0.1) makes it boring, safe, and predictable — great for code.
Token
In AI-land, a token is a chunk of text — roughly 3/4 of a word.
Top-p
Top-p (also called nucleus sampling) is another dial that controls how an AI picks its next word.