IBM’s chip-design agents make inference economics part of the engineering method

IBM Research’s latest chip-design experiment makes a narrower—and more useful—claim than “AI designs chips.” It shows that the economics of inference can materially change how much verified hardware exploration an engineering team can afford.

The researchers used an “agent factory” to optimize a classical decoder for quantum error correction. A coordinator assigned optimization hypotheses to four parallel workers, which modified a Relay-BP decoder implementation. Correctness checks and high-level synthesis (HLS) results then fed back into subsequent rounds. The agents were not merely producing code: they were searching architectural choices, including memory organization and tree-structured reductions, and measuring the resulting latency and hardware-resource tradeoffs.

Under a budget of 100 hypotheses, the all-open configuration produced a 52× HLS speedup over the baseline for about $26 in model-inference cost. IBM says the all-proprietary configuration reached a higher 204× speedup, but cost $299—11.5 times as much—and the open search found the smallest-area design. The open configuration consumed more tokens, about 241 million versus 216 million, so its cost advantage came from serving economics rather than using less inference.

That distinction is the important result. Agentic engineering loops repeatedly send large amounts of shared context: source code, constraints, prior changes and synthesis reports. IBM ran the open model on premises with llm-d, whose cache-aware routing and separation of prompt processing from token generation are designed to reuse that context efficiently. For this workload, IBM reports an approximately 98% prefix-cache hit rate and a blended serving cost near $0.11 per million tokens.

The experiment therefore frames inference infrastructure as part of the engineering method, not an interchangeable backend. If each hypothesis involves reasoning, coding, debugging and tool output, reducing the marginal cost of another hypothesis can expand the feasible search space. It can also keep design artifacts on premises.

The limits matter. Each of the three model configurations—open, proprietary and hybrid—was evaluated in only one search run. IBM explicitly says the observations do not establish a universal model ranking. The reported costs cover model inference, not the full hardware-development workflow. And doubling the all-open campaign from 100 to 200 hypotheses improved speedup only from 52× to 60×, evidence of diminishing returns.

Even with those caveats, the demonstration points to a practical benchmark for agent systems: not just the best result they produce, but the number of correct, synthesized alternatives they can explore per dollar. On IBM’s quantum-decoder task, open models plus llm-d traded peak observed speed for dramatically cheaper verified exploration. That may be the more consequential metric for organizations deciding whether agent factories can move from demos into sustained engineering work.

Source: IBM Research, “Making AI-driven chip design affordable with open agent factories and distributed inference at scale”, Oct. 13, 2026.