OpenAI builds its first chip with Broadcom

Key Takeaways

• OpenAI's Jalapeño chip was designed from scratch for LLM inference, not adapted from general-purpose accelerators
• The chip went from design to production in nine months, with OpenAI using its own AI models to accelerate development
• Deployment at gigawatt scale with Microsoft and other data center partners begins in 2026

OpenAI has entered the chip business. The company announced Jalapeño, its first custom accelerator designed specifically for running large language models. Built in partnership with Broadcom, the chip represents OpenAI's bet that controlling its own silicon will make AI faster, cheaper, and more reliable at scale.

The chip is already running ML workloads in the lab, including GPT-5.3-Codex-Spark. Early testing shows performance per watt "substantially better" than current hardware, though OpenAI hasn't released specific benchmarks. A technical report with detailed numbers will follow in the coming months.

Why OpenAI is building its own silicon

The economics are straightforward. OpenAI reportedly spends hundreds of millions monthly on compute, primarily NVIDIA GPUs. A custom chip optimized for inference, the task of running trained models rather than training them, could cut costs significantly while improving response times for ChatGPT's hundreds of millions of users.

OpenAI President Greg Brockman framed it as infrastructure strategy. "By designing more of the stack ourselves, we can serve more intelligence with greater efficiency and keep pushing advanced AI toward broader access," he said.

The company is following a path Google paved with TPUs, Amazon with Trainium and Inferentia, and Meta with MTIA chips. The difference is OpenAI's singular focus on LLM inference. Jalapeño isn't a general-purpose accelerator adapted for language models. It was built for them from the start.

What makes Jalapeño different from NVIDIA GPUs?

Richard Ho, who leads OpenAI's hardware program, described the design philosophy: "We optimized the architecture around the kernels, memory movement, networking, and serving patterns that matter most for frontier AI models."

Translation: LLM inference has specific bottlenecks. Moving data between memory and compute is expensive. General-purpose GPUs waste resources on capabilities that don't matter for token generation. Jalapeño's architecture reduces data movement and balances compute, memory, and networking to achieve utilization closer to theoretical peak performance.

OpenAI claims this lets them combine the throughput of leading AI accelerators with latency closer to specialized inference systems. For interactive products like ChatGPT, lower latency means faster responses. For the API business, better throughput means serving more customers per chip.

Nine months from design to production

The development timeline is notable. OpenAI went from design to engineering samples in nine months, a pace the company attributes partly to using its own AI models during the process. This is one of the first high-profile examples of AI accelerating its own hardware development.

Broadcom handled chip implementation, board design, rack system integration, and high-performance networking. Celestica contributed to production systems. Broadcom's Tomahawk networking silicon connects the chips at scale.

Broadcom CEO Hock Tan called it "just the beginning of a multi-generation roadmap." The companies plan to deploy the chips at gigawatt scale, an unusual measurement that signals data center ambitions. For context, one gigawatt powers roughly 750,000 homes.

The full-stack play

OpenAI now controls models, products, and chips. This vertical integration creates what the company calls a "flywheel": better infrastructure improves efficiency, better efficiency reduces costs, lower costs enable broader access, broader access funds more infrastructure investment.

The chip is designed for flexibility beyond OpenAI's own models. The company says Jalapeño works with "all LLMs" based on their understanding of inference needs across the industry. Whether competitors will actually run their models on OpenAI silicon remains an open question.

“The world is moving to a compute-powered economy. Jalapeño is part of our long-term full-stack infrastructure strategy to make compute more abundant.”

— Greg Brockman, President and Co-Founder of OpenAI

What this means for NVIDIA

NVIDIA's data center revenue exceeded $47 billion in fiscal 2024, driven largely by AI training and inference demand. OpenAI building custom chips doesn't eliminate that relationship immediately. Training frontier models still requires massive GPU clusters. But inference represents a growing share of compute spending as deployments scale.

If Jalapeño delivers on its efficiency promises, OpenAI could shift inference workloads away from NVIDIA over time. The multi-generation roadmap suggests this isn't a one-off experiment but a sustained investment in proprietary silicon.

Frequently Asked Questions

When will OpenAI's Jalapeño chip be available?

Deployment begins in 2026 with Microsoft and other data center partners. OpenAI hasn't announced specific availability dates for API customers.

How does Jalapeño compare to NVIDIA H100 performance?

OpenAI claims substantially better performance per watt than current hardware but hasn't released specific benchmarks. A technical report with detailed comparisons is expected in the coming months.

Can other companies use OpenAI's inference chip?

OpenAI says Jalapeño is designed for all LLMs, not just their own models. Whether competitors will choose to run workloads on OpenAI silicon remains unclear.

Who manufactured the Jalapeño chip?

Broadcom handled chip implementation and silicon design. Celestica contributed to production systems. The fabrication partner wasn't disclosed in the announcement.

Will Jalapeño reduce OpenAI API pricing?

OpenAI hasn't announced pricing changes. If the chip delivers the claimed efficiency gains, lower inference costs could eventually translate to cheaper API access.

Source: OpenAI News