Min-p and typical sampling

Min-p#

Nguyen et al., Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs, ICLR 2025 oral (arxiv.org/abs/2407.01082) define the kept set as

$$ V_{\text{min-p}} = \left{ x \in V : P(x) \ge p_{\text{base}} \cdot \max_{y \in V} P(y) \right} $$

with $p_{\text{base}} \in (0, 1]$, typically $0.02$ to $0.1$.

The scale-relative threshold is the whole design. Under temperature scaling $z \mapsto z/T$, the ratio $P(x)/\max_y P(y)$ transforms as

$$ \frac{P_T(x)}{\max_y P_T(y)} = \exp!\left(\frac{z_x - z_{\max}}{T}\right) = \left(\frac{P_1(x)}{\max_y P_1(y)}\right)^{1/T} $$

so the surviving set is a monotone function of $T$, with a far gentler slope than top-p's. Top-p's cutoff depends on the cumulative mass, and therefore on the entire tail. The empirical table above shows both the benefit and its limit.

The dispute, which is instructive#

The original paper reported gains on GPQA, GSM8K and AlpacaEval Creative Writing across Mistral and Llama 3 at 1B to 123B, especially at high temperature.

Min-p, Max Exaggeration: A Critical Analysis of Min-p Sampling in Language Models, 2025 (arxiv.org/abs/2506.13681) re-examines all four lines of evidence. It reports that the benchmark gains do not reproduce under matched baselines. It finds confounds in the human evaluation, and argues that community adoption was mis-cited as endorsement.

Two things to take from this beyond the specific question. First, decoding-method papers are unusually easy to over-claim, because a baseline evaluated at one hyperparameter setting and a proposed method tuned across many is not a comparison. Second, a method can be mechanically sound and still not be an improvement on the benchmarks used to argue for it. Min-p is now widely implemented — in transformers, vLLM, SGLang and llama.cpp — which is a fact about adoption, not about quality.

Locally typical sampling#

Meister et al., Locally Typical Sampling, TACL 2023 (arxiv.org/abs/2202.00666) start from information theory rather than from tail-cutting.

The conditional entropy at step $t$ is

$$ H_t = -\sum_{x \in V} P(x \mid x_{<t}) \log P(x \mid x_{<t}) $$

and a token's information content is $-\log P(x \mid x_{<t})$. The typical set is tokens whose information content lies near $H_t$:

$$ V_{\text{typ}} = \arg\min_{V' \subseteq V} \sum_{x \in V'} \Big| {-\log P(x)} - H_t \Big| \quad \text{subject to} \quad \sum_{x \in V'} P(x) \ge \tau $$

Implementations approximate this greedily: sort by $\big|{-\log P(x)} - H_t\big|$ ascending, take a prefix until cumulative mass reaches $\tau$.

The motivating hypothesis is that human language is efficient in Shannon's sense — speakers spread information evenly rather than always choosing the most predictable word. Text that is too predictable is as unhuman as text that is too surprising, which is the observation Holtzman et al. made empirically about beam search.

Top-h, the newest member#

Top-H Decoding: Adapting the Creativity and Coherence with Bounded Entropy in Text Generation, NeurIPS 2025 (arxiv.org/abs/2509.02510) formulates truncation as an entropy-constrained optimisation. Tokens are added in descending probability order while cumulative entropy stays under $\tau = \alpha H$, with $\alpha \in (0, 1]$ the single parameter. It is available as top_h in transformers 5.x.

Choosing between them#

There is no decisive published comparison, and the min-p dispute is a warning against trusting one. Defensible practice:

• Greedy for anything with a correct answer: extraction, classification, tool arguments, code completion where determinism matters more than variety.
• Top-p at 0.9 to 0.95 with $T \le 1$ as the boring default. Enormously deployed, well understood.
• Min-p at 0.02 to 0.1 with $T > 1$ when you deliberately want a hot distribution — creative writing, generating diverse candidates for reranking.
• Typical at 0.2 to 0.95 where repetitiveness is the specific complaint. It targets that failure directly.

And evaluate on your own task. Sampling settings interact with a model's calibration, which is a property of its post-training, not of the decoding rule.

Papers#

• Nguyen et al., Turning Up the Heat: Min-p Sampling, ICLR 2025 — arxiv.org/abs/2407.01082
• Min-p, Max Exaggeration: A Critical Analysis of Min-p Sampling, 2025 — arxiv.org/abs/2506.13681
• Meister et al., Locally Typical Sampling, TACL 2023 — arxiv.org/abs/2202.00666
• Hewitt et al., Truncation Sampling as Language Model Desmoothing, 2022 — arxiv.org/abs/2210.15191
• Top-H Decoding, NeurIPS 2025 — arxiv.org/abs/2509.02510

What to learn next#

• Repetition and frequency penalties — the other family of logit edits, aimed at a different failure.
• Temperature and sampling — the knob these methods were designed to survive.
• Reading logprobs — measuring the confidence these rules are relative to.