OpenAI's Custom ASIC Chip Surpasses Nvidia Blackwell in Just 9 Months?

Original author: Dong Jing

Original source: Wallstreetcn

OpenAI's first self-developed chip, Jalapeño, has burst onto the scene, upending the existing landscape of the AI chip industry at a breathtaking pace—this is not just a product launch, but a signal: AI is reshaping the very way chips are designed.

According to a Wallstreetcn article, Bloomberg reported on Aug. 25 that OpenAI stated Jalapeño leads Nvidia's GB300 in two key metrics: AI workload processed per unit of power and response speed. The chip, developed in collaboration with Broadcom, is designed specifically for the AI inference stage and is expected to be put into practical use as early as later this year. Richard Ho, head of OpenAI's chip division, said Jalapeño delivers strong performance at a low power consumption of 700 watts, helping to significantly reduce data center electricity costs.

According to semiconductor research firm SemiAnalysis, the chip took only about 16 months from design initiation to tape-out, with the critical CoWoS packaging tape-out node completed in November 2025—just 9 months ago, far shorter than the industry's typical 18 to 36-month cycle.

SemiAnalysis researchers visited OpenAI's lab in person and tested Jalapeño using its proprietary benchmark suite InferenceX. The conclusion was direct: the chip beat all Nvidia, AMD, and Google chips they had previously tested on performance per watt (perf/W).

Even more noteworthy is that this result was achieved without Jalapeño enabling speculative decoding or prefill-decode disaggregation (PDD), while the other chips compared were all running their optimal configurations.

No wonder renowned semiconductor analyst Dylan Patel bluntly stated, "It's not just Blackwell that got toppled—even Nvidia's Rubin chip has been surpassed"!

Analysts believe this result poses a direct challenge to Nvidia's market position and forces the market to reassess the competitive landscape of AI chips.

Benchmark Data: Jalapeño Crushes Blackwell on Multiple Key Metrics

SemiAnalysis's test results show that Jalapeño's advantage in inference efficiency is quite significant.

On the GPT-OSS 120B model, Jalapeño outputs approximately 1,459 tokens per second, while Nvidia's GB200 only achieves 535. In terms of end-to-end latency, Jalapeño completes a task in just 1.65 seconds, while GB300 takes nearly 6 seconds—a 3.6x difference. In high-interactivity scenarios, when GB300 is pushed to its fastest decoding speed (169 tokens per second), Jalapeño's throughput is 104.3 times higher.

On tokens per megawatt per second—a core metric for data center energy efficiency—Jalapeño achieves approximately 53 million tokens/MW/s on the GPT-OSS model, while GB200 NVL72 only reaches about 10 million.

SemiAnalysis points out that this metric is essentially equivalent to tokens produced per joule, directly determining the revenue ceiling of a data center—an advantage that is particularly critical in an era where computing power is constrained by electricity.

From a system-level cost perspective, after SemiAnalysis factored in power supply, cooling, and networking, Jalapeño's total cost of ownership (TCO) per chip per hour is approximately $1.56, nearly on par with the H100's $1.55, while Nvidia's Vera Rubin is as high as $3.61.

It is worth noting that SemiAnalysis also raised several important caveats.

First, the models used in testing are not the most cutting-edge currently available. Nvidia and AMD have published results based on the AgentX suite on larger-scale models (such as DeepSeek V4 Pro and Kimi K3), while Jalapeño has not yet completed AgentX testing—a suite that better reflects performance in real production scenarios with multi-turn and long-context workloads.

Second, a fairer comparison would be against Nvidia's Vera Rubin, which also uses HBM4, rather than Blackwell. Vera Rubin's performance per watt is about 5.4x that of GB200 NVL72, and compared with Jalapeño, the two are nearly equal in total cost of ownership (TCO) per token.

Third, Jalapeño is still in the engineering sample stage, with mass production expected to ramp up gradually only by 2027.

AI-Designed Chips: The Flywheel Effect of GPT-Astra and Codex

The core reason Jalapeño was developed at such astonishing speed is the deep involvement of AI tools. According to SemiAnalysis, OpenAI extensively used internal AI models in the chip design process, including the widely watched GPT-Astra.

Social platform X user Andrew Curran, citing OpenAI information, pointed out that GPT-Astra was deeply involved throughout Jalapeño's R&D:

"The team used Codex and GPT-Astra to tune three open-source models that were not originally in Jalapeño's mass production plan to high performance within two months."

This means AI is not only accelerating chip design itself but also rapidly expanding the range of models the chip can support.

SemiAnalysis's data further quantifies this contribution:

AI-assisted design reduced SIMD unit area by 8% and matrix engine area by 10%, while also outperforming the initial version in timing and power consumption.

At the software level, OpenAI used an internally extended version of Codex to write Jalapeño kernels, with some kernel code reaching about 3,000 lines. On the most performance-critical attention mechanisms and MoE modules, AI-generated code was 1.5 to 1.8 times faster than implementations by top human engineers.

SemiAnalysis's assessment is sharp: OpenAI models running on Nvidia GPUs (such as GPT-5.6 Sol) are being used to design a chip that poses a real threat to the CUDA moat—"Nvidia's own GPUs are breeding their potential successor in real time."

Architecture Analysis: Why Is "General-Purpose" Faster?

The outside world generally assumed Jalapeño is a chip deeply customized for OpenAI's own models, but SemiAnalysis's conclusion is the opposite: Jalapeño is a general-purpose chip for AI inference, capable of running various models and workloads, even including the game Doom ported with Codex.

At the architectural level, Jalapeño's core design philosophy is "eliminating fixed latency." Unlike GPUs that rely on complex memory hierarchies, Jalapeño directly binds compute cores to HBM slices, with cores synchronized through a dedicated high-bandwidth aggregation network, significantly reducing memory access latency.

Additionally, Jalapeño uses out-of-order execution (OoO) cores paired with L1 cache, rather than the software-managed scratchpads commonly used in other accelerators, allowing it to approach hardware theoretical peaks in small-batch, low-latency scenarios.

In terms of memory bandwidth, Jalapeño uses HBM4, achieving 15.4TB/s of single-package memory bandwidth, with HBM bandwidth per watt of 22, compared to Nvidia Rubin's 11.1 and GB300's mere 5.71.

SemiAnalysis notes that Jalapeño's HBM4 pin speed reaches 10Gbps, slightly higher than Nvidia Rubin's 9.6Gbps, and the HBM supplier may be Samsung.

It is worth mentioning that Jalapeño chose not to implement prefill-decode disaggregation (PDD). SemiAnalysis provided a detailed explanation:

In real production environments, parameters such as input-output ratio, concurrency, and cache hit rate continuously change. Fixed pooling leads to lower global utilization, whereas a homogeneous resource pool can flexibly respond to traffic changes and dynamically allocate between latency-sensitive requests and high-throughput batch processing.

CUDA Moat: Cracks Already Showing?

SemiAnalysis's report makes a weighty judgment: the CUDA moat may already be dead.

The basis lies in the comparison of software ramp-up speed. Nvidia Rubin's CoWoS tape-out was completed one month earlier than Jalapeño's, but so far, the only public benchmark data for Rubin comes from CoreWeave's engineering samples. Nvidia has not opened its labs to third parties for free testing as OpenAI has. SemiAnalysis believes this reflects not a hardware gap but a software maturity gap.

Building a software stack from scratch allowed OpenAI to shed historical baggage and make cleaner architectural decisions. OpenAI uses its proprietary kernel programming language Gluon (built on Triton) and an internal inference engine called "Teacup," and leverages Codex to rapidly tune kernels. SemiAnalysis observed that in less than two weeks, Jalapeño's throughput at specific interaction speeds improved by more than 2x; within 8 days, the team scaled tensor parallelism from TP8 to TP32, achieving full-rack deployment across racks.

However, SemiAnalysis also clearly points out that current testing only covers the relatively simple 8k1k workload and has not yet completed AgentX multi-turn long-context testing. Under more complex agent workloads, the performance of components such as routers and prefix caches will become new challenges.

Next Steps: B0 Already in Fab, 10GW Roadmap Unfolding

Jalapeño's story is far from over.

According to SemiAnalysis, all publicly tested samples are A0 stepping, while B0 stepping has already entered the fab and is expected to improve performance per watt by about 25% over A0—B0's single compute die will deliver 13.4 PFLOPs of MXFP4 compute with TDP maintained at 700W.

At the system level, OpenAI collaborated with Celestica to design a complete rack solution: each ASIC cabinet contains 128 Jalapeño chips, with a dual-cabinet system total power consumption of about 160kW, comparable to Nvidia's GB300 dual-wide cabinet.

For large-scale deployment, a single scale-out network domain can connect up to 2,048 Jalapeño XPUs across 16 racks. On mass production, SemiAnalysis expects a gradual ramp-up in 2027, with the next milestone being 100MW deployment scale.

The larger strategic picture is that OpenAI and Broadcom have signed a 10GW custom accelerator cooperation agreement, a scale roughly equivalent to the full output of ten nuclear power units.

The Wallstreetcn article writes that Richard Ho, head of OpenAI's chip division, said the company has reached a power and cost level that can effectively reduce infrastructure costs, and "this is just the first step." He also emphasized that Nvidia remains an important partner, and OpenAI's demand for computing power is enormous, so it will not abandon existing suppliers in the short term.

Analysts point out that Jalapeño's significance may not lie in how many Nvidia chips it can replace today, but in proving something previously widely doubted: an AI company, using AI tools, in record time, built a truly competitive chip. This flywheel has begun to turn.