vLLM extends adaptive verification to draft models without confidence heads

vLLM has extended Adaptive Verification to draft-model-based speculative decoders that do not ship with a trained confidence head, using a lightweight estimator fitted during serving. The change broadens a performance optimization that had previously been limited largely to DSpark configurations with a confidence head, according to the project's engineering write-up.

What changed

Adaptive Verification reduces work by trimming draft tokens that are unlikely to be accepted before the target model verifies them. That requires a calibrated per-token estimate of acceptance probability. Draft models such as EAGLE, MTP and DFlash, along with some DSpark checkpoints, lack the dedicated confidence head previously used to produce that estimate; adding one would require retraining the drafter.

vLLM's new approach instead applies a small logistic-regression model to the log-odds of the drafter's most likely token. It uses one slope shared across draft positions and a separate intercept for each position. The coefficients are continually refitted from accept-or-reject outcomes that the target model already produces during normal verification, allowing the estimator to adapt to a particular model pair and incoming workload without changing the checkpoint.

The implementation landed in vLLM pull request 52228. Operators enable it through enable_adaptive_verification in the speculative configuration; when a compatible draft model lacks a confidence head, vLLM selects the online estimator automatically.

What the measurements show

The project tested the estimator with MiMo-V2.5-Pro-FP4 plus DFlash, Inkling-NVFP4 plus MTP, Muse-Glimmer-30B plus DFlash2, and DeepSeek-V4-Flash plus DSpark. At concurrency 64, Adaptive Verification with the estimator improved output throughput over fixed verification at the same speculative length across all tested datasets.

On DeepSeek-V4-Flash plus DSpark, the estimator and a trained confidence head were roughly even across the measured concurrency levels. For MiMo-V2.5-Pro-FP4 plus DFlash, a configuration that lacked a confidence head, Adaptive Verification landed on the throughput-versus-interactivity frontier at five of seven tested concurrencies.

Those results do not make the estimator universally optimal. vLLM notes that a shorter fixed speculative length can still beat Adaptive Verification at the maximum speculative length for a particular concurrency. The target model's attention backend must also support variable-length verification, and the project found smaller benefits for autoregressive MTP and EAGLE speculators than for DFlash and DSpark.

The practical change is narrower but useful: existing draft checkpoints can now use workload-adaptive token trimming without a separate confidence-head training step. The work builds on Adaptive Verification contributions by Red Hat's Lucas Wilkinson and NVIDIA's Benjamin Chislett.