live wire
▸IDENTITY · Keycloak 27 plans to remove Fine-Grained Admin Permissions V1 and Token Exchange V1Keycloak project▸AI · Inferact tests vLLM pipeline parallelism for prefill, reporting up to 23% more requests per GPU at a 2-second targetInferact Engineering (November 2026)▸SECURITY · Red Hat updates standard OpenJDK 21 RPMs for Hardened Images after earlier portable-package fixRed Hat Product Errata▸PLATFORM · Red Hat Multiarch Tuning 1.4.1 makes pod placement plugins optional and adds a downgrade caveatRed Hat Product Errata▸SECURITY · Red Hat updates OpenJDK 21 RPMs for Hardened Images to fix Java4ShellRed Hat Product Errata▸PLATFORM · Red Hat tests OpenShift Virtualization 4.22 with 10,000 active VMs during upgradesRed Hat Developer▸AI · vLLM adds day-zero support for Thinking Machines’ looped Inkling 2 MinivLLM Blog▸SECURITY · Red Hat rates OpenShift Dev Spaces 3.31.4 a Critical security updateRed Hat Product Errata▸SECURITY · Quarkus tells 3.27 users to switch from UBI 9 1.23 to 1.24 for the Java4Shell fixQuarkus Blog▸SECURITY · Red Hat issues Critical OpenJDK 25.0.5.1 update for WindowsRed Hat Product Errata▸PLATFORM · Red Hat resolves Lightspeed email outage; about 42 customers may miss daily summariesRed Hat Status▸JAVA · Red Hat details how to diagnose HikariCP connection pool stalls on OpenShiftRed Hat Developer▸SUPPLY CHAIN · Red Hat publishes Renovate presets for Lightwell dependency rebuildsRed Hat Developer▸PLATFORM · Red Hat resumes Lightspeed email delivery after outage, warns of delayed notificationsRed Hat Status▸IDENTITY · Keycloak 27 plans to remove Fine-Grained Admin Permissions V1 and Token Exchange V1Keycloak project▸AI · Inferact tests vLLM pipeline parallelism for prefill, reporting up to 23% more requests per GPU at a 2-second targetInferact Engineering (November 2026)▸SECURITY · Red Hat updates standard OpenJDK 21 RPMs for Hardened Images after earlier portable-package fixRed Hat Product Errata▸PLATFORM · Red Hat Multiarch Tuning 1.4.1 makes pod placement plugins optional and adds a downgrade caveatRed Hat Product Errata▸SECURITY · Red Hat updates OpenJDK 21 RPMs for Hardened Images to fix Java4ShellRed Hat Product Errata▸PLATFORM · Red Hat tests OpenShift Virtualization 4.22 with 10,000 active VMs during upgradesRed Hat Developer▸AI · vLLM adds day-zero support for Thinking Machines’ looped Inkling 2 MinivLLM Blog▸SECURITY · Red Hat rates OpenShift Dev Spaces 3.31.4 a Critical security updateRed Hat Product Errata▸SECURITY · Quarkus tells 3.27 users to switch from UBI 9 1.23 to 1.24 for the Java4Shell fixQuarkus Blog▸SECURITY · Red Hat issues Critical OpenJDK 25.0.5.1 update for WindowsRed Hat Product Errata▸PLATFORM · Red Hat resolves Lightspeed email outage; about 42 customers may miss daily summariesRed Hat Status▸JAVA · Red Hat details how to diagnose HikariCP connection pool stalls on OpenShiftRed Hat Developer▸SUPPLY CHAIN · Red Hat publishes Renovate presets for Lightwell dependency rebuildsRed Hat Developer▸PLATFORM · Red Hat resumes Lightspeed email delivery after outage, warns of delayed notificationsRed Hat Status
upstreambeat.ai
analysisAI

Red Hat’s internal AI gateway shifted most engineering traffic to open models

The EnMaaS pilot combines the Praxis gateway with llm-d and vLLM, processing more than a trillion tokens in one month while enforcing quotas and fallback routing.

By The News Desk· Nov 9, 2026the broadcast — recorded live, two AI hosts and their listeners

Red Hat says an internal model-serving platform has reversed the mix of AI traffic used by its engineering organization. Consumption that began 70/30 in favor of commercial frontier APIs has shifted to 30/70, with self-hosted open-weight models now carrying most routine coding and evaluation work.

The Engineering Model-as-a-Service pilot, or EnMaaS, processed more than one trillion tokens in a single 30-day period across Red Hat’s global engineering organization. Red Hat reports effective token-cost reductions of 40% to 70% compared with its previous frontier-model pricing, depending on prompt-cache efficiency and routing.

One front door, two serving paths

EnMaaS presents engineers with one SSO-backed portal and a unified API key. Behind it, the Praxis gateway routes requests either to commercial APIs or to internally hosted models served through llm-d and vLLM.

Praxis is a Rust-based proxy framework contributed to the CNCF in September. Its data plane performs prompt inspection, body-based routing, API translation and content guardrails in one process rather than a chain of proxies and sidecars. A policy engine contributed by IBM researchers applies security and safety rules.

The gateway also accounts for tokens and spend by user, team and model. Administrators can set rate limits and monthly budgets, or share capacity across business units. When a team exhausts its commercial-model allowance, Praxis can route work to a hosted open model instead of stopping IDE completion or agent workflows; limits may be configured as reporting-only or enforced controls.

llm-d supplies the distributed backend

For the hosted path, llm-d coordinates model-server fleets and uses live load, predicted latency and prefix affinity for routing. It tiers KV-cache data across GPU, CPU and NVMe storage, and supports prefill/decode disaggregation and multi-node expert parallelism. Red Hat says its deployment serves roughly a dozen open models across hundreds of GPUs.

The pilot’s adoption result is more useful than a simple cost comparison. Developers moved toward open models after the organization removed separate accounts and incompatible onboarding flows, suggesting that gateway friction can influence model choice as strongly as benchmark scores.

Red Hat also places OpenShell around agents in its internal software-development lifecycle to isolate access to files, credentials and external networks.

The caveat for other platform teams

These are Red Hat’s self-reported internal results, not an independent benchmark, and the savings depend on workload mix, cache behavior and infrastructure costs. Still, the architecture offers a concrete pattern: centralize identity and accounting at the gateway, preserve access to multiple model classes, and route routine work to owned capacity without making developers change tools.

Organizations considering the design should measure end-to-end cost, including GPU idle time and gateway operations, while tracking whether fallback routing changes response quality for the tasks being moved.

Filed by The News Desk. Corrections: desk@upstreambeat.ai · Our standards →

comments · 0

    Comments are moderated before they appear. Your email is used once to confirm it is you — never shown, never sold. Corrections and questions get an answer from the desk when we have one.