live wire
▸PLATFORM · Clang 23 bytecode interpreter cuts allocations and expands constant-expression supportRed Hat Developer▸PLATFORM · RHEL becomes first-party on OCI; OpenShift subscriptions reach Oracle MarketplaceRed Hat▸AI · vLLM-Omni merges native MOSS-Image serving with parallelism and offloadvLLM▸AI · vLLM adds System One endpoint for Decision 3.0 multimodal routing models (Oct. 15)vLLM▸SECURITY · Red Hat rates OpenShift AI 3.4.5 update Critical; advisory lists hundreds of CVEsRed Hat Product Security▸SECURITY · Red Hat’s first Security Signals report shifts focus from CVE counts to risk paths (Oct. 15)Red Hat▸PLATFORM · IBM adds non-destructive FlexVolume-to-CSI migration tool for OpenShift object storageIBM Cloud▸vLLM tests agent-built Kimi K3 decode path at up to 1.9× speedup; PR remains openvLLM▸AI · LeaderWorkerSet 1.0 changes rollout defaults for disaggregated AI workloadsKubernetes SIGs▸SECURITY · Red Hat flags active exploitation of kernel flaw affecting OpenShift CoreOS; no fix listedRed Hat Product Security▸AI · Red Hat sets Dec. 9 showcase for AI 3.6 agent governance and model servicesRed Hat▸vLLM 0.32.0 drops Python 3.10, restricts CORS defaults and adds security fixesvLLM▸SECURITY · Red Hat tracks Moderate Hono auth bypass across OpenShift AI, Dev Spaces and AnsibleRed Hat Product Security▸SECURITY · Red Hat Hardened Images fixes Moderate libpng denial-of-service flawRed Hat▸PLATFORM · Clang 23 bytecode interpreter cuts allocations and expands constant-expression supportRed Hat Developer▸PLATFORM · RHEL becomes first-party on OCI; OpenShift subscriptions reach Oracle MarketplaceRed Hat▸AI · vLLM-Omni merges native MOSS-Image serving with parallelism and offloadvLLM▸AI · vLLM adds System One endpoint for Decision 3.0 multimodal routing models (Oct. 15)vLLM▸SECURITY · Red Hat rates OpenShift AI 3.4.5 update Critical; advisory lists hundreds of CVEsRed Hat Product Security▸SECURITY · Red Hat’s first Security Signals report shifts focus from CVE counts to risk paths (Oct. 15)Red Hat▸PLATFORM · IBM adds non-destructive FlexVolume-to-CSI migration tool for OpenShift object storageIBM Cloud▸vLLM tests agent-built Kimi K3 decode path at up to 1.9× speedup; PR remains openvLLM▸AI · LeaderWorkerSet 1.0 changes rollout defaults for disaggregated AI workloadsKubernetes SIGs▸SECURITY · Red Hat flags active exploitation of kernel flaw affecting OpenShift CoreOS; no fix listedRed Hat Product Security▸AI · Red Hat sets Dec. 9 showcase for AI 3.6 agent governance and model servicesRed Hat▸vLLM 0.32.0 drops Python 3.10, restricts CORS defaults and adds security fixesvLLM▸SECURITY · Red Hat tracks Moderate Hono auth bypass across OpenShift AI, Dev Spaces and AnsibleRed Hat Product Security▸SECURITY · Red Hat Hardened Images fixes Moderate libpng denial-of-service flawRed Hat
upstreambeat.ai
newsAI

vLLM gives Semantic Router’s Decision 3.0 models a native serving path

A merged model-integration patch and six new recipes put typed multimodal decisions behind vLLM’s System One API, although users need a nightly build for now.

vLLM outperforms checkpoint runtime on Decision 3.0 throughput at 64 clients.
AI-generated illustration
By The News Desk· Oct 15, 2026the quick take — two AI hosts go live when you do

vLLM has merged native support for the Decision 3.0 model family used by vLLM Semantic Router and published deployment recipes for all six checkpoints. The change turns those models into a directly served classification workload instead of requiring their checkpoint-specific runtime.

The merged integration detects the models’ decision configuration and serves them through vLLM’s /v1/systemone endpoint. The accompanying recipe collection covers D3, D3 Flash, Mini, Nano, Lite and Edge, ranging from roughly 600 million to 26 billion parameters.

Decisions without generated text

Decision 3.0 is designed for typed questions over a supplied state. A caller can ask the model to choose among options, return a yes-or-no probability, or score an input on a scale. The state can contain text or JSON and can also include images or video. Rather than generate an explanation token by token, the model returns probabilities for the available answers in one forward pass.

That makes this a routing and control-plane capability, not another chat endpoint. A platform team could use the same API shape for request classification, policy decisions or model-selection signals while choosing among six model sizes. The 27B recipe includes launch commands, hardware selectors and client examples for the System One API.

The implementation loads the published checkpoints without converting them. It maps their Qwen 3.5 backbone plus a separate answer-code readout into a vLLM sequence-classification model. The merged pull request also adds handling for the models’ bidirectional full-attention layers and their image and video limits.

The throughput gain shows up under concurrency

The project compared the 27B, 4B and 0.6B variants with their checkpoint runtime on one AMD Instinct MI325X. Across 416 questions per model, vLLM changed no decisions for D3, one for D3 Mini and two for D3 Edge; the changed cases were reported as ties or near-ties.

The more meaningful serving difference appeared under load. At 64 concurrent clients, the project measured 8.5 requests per second for D3 through vLLM versus 4.6 for the checkpoint server. D3 Mini reached 45.0 versus 18.3 requests per second, while D3 Edge reached 110 versus 41.7. Those are project-supplied results from one AMD test environment, not an independent benchmark, and the pull request says CUDA, tensor parallelism and three of the six variants were not measured in that validation.

Nightly builds are the entry point

The support landed on vLLM’s main branch on October 14. The recipes state that it is not present in the current v0.31.0 release, so deployments need a nightly build dated October 15 or later. The 27B recipe calls for at least 63 GB to load its BF16 variant and notes that reducing the maximum model length can shorten cold-start time.

For teams evaluating the feature, the practical sequence is to select a recipe that fits the available accelerator memory, install the appropriate CUDA or ROCm nightly, and test the /v1/systemone behavior against their own routing data. Production adoption should wait for normal compatibility and workload validation rather than treating the project’s single-node measurements as a capacity promise.

sources

Filed by The News Desk. Corrections: desk@upstreambeat.ai · Our standards →

comments · 0

    Comments are moderated before they appear. Your email is used once to confirm it is you — never shown, never sold. Corrections and questions get an answer from the desk when we have one.

    more from the desk