← AI Engineering Studio
Concept Explainer · AI Engineering

Temperature vs. Top-P Sampling

Two different dials that both get described as controlling "randomness" — but they act on the token distribution in completely different ways, at completely different stages.

Before either setting ever gets involved, the model has already done its job: for the current context, it has computed a probability for every single token in its vocabulary — tens of thousands of candidates, each assigned a likelihood of being the next token. Temperature and top-p don't change that underlying computation at all. They're two independent, optional adjustments applied to that already-finished distribution, right before a token gets randomly sampled from it. People routinely treat them as two names for the same "randomness knob." They're not — one reshapes, the other filters — and mixing them up leads to genuinely wrong intuitions about what a given setting will do.