Google's new AI chip targets 6-10x efficiency gains for Gemini

Key Takeaways

• Google's 'Frozen v2' chip could generate 6-10x more tokens per watt than current hardware
• Release is targeted for 2028 as part of Google's $180-190 billion AI infrastructure investment
• The announcement lifted Alphabet stock 3%, signaling investor appetite for AI cost efficiency

Google is building a new server chip designed to make its Gemini models dramatically more efficient. Internally called 'Frozen v2,' the chip could generate six to ten times more tokens per watt than Google's current AI hardware, according to a report from The Information citing anonymous sources. The chip is slated for 2028.

Google didn't confirm or deny the report. A spokesperson told TechCrunch: 'Our teams are constantly researching and experimenting with new innovations to deliver maximum performance and efficiency for our users and customers. While not every project moves into production, this rigorous exploration is central to our full stack approach.'

Why is Google building custom AI chips?

The short answer: Nvidia dependency is expensive and risky. Nvidia dominates the AI chip market, and that dominance has left every major AI company, Google included, competing for the same limited hardware. Custom silicon sidesteps the supply crunch and, when done right, delivers better performance per dollar for specific workloads.

Google has been at this longer than most. The company launched its first Tensor Processing Unit (TPU) in 2016, years before the current AI boom. TPUs are purpose-built for the matrix multiplications that power neural networks. They're not versatile like Nvidia's GPUs, but for running Gemini at scale, versatility isn't the point. Efficiency is.

The timing matters. Google has committed to spending between $180 billion and $190 billion on AI infrastructure. Investors have openly worried about whether those expenditures will pay off. A chip that delivers 6-10x better efficiency directly addresses that concern. Unsurprisingly, Alphabet's stock climbed about 3% on Monday morning after the report landed.

Google isn't alone in this race

Every major AI player is now designing custom silicon. OpenAI announced its first chip in June, an inference processor called 'Jalapeño.' Anthropic is reportedly in discussions with Samsung about a chipmaking partnership. Amazon has its Trainium and Inferentia chips. Microsoft is building custom AI accelerators. Meta has its MTIA chips.

The logic is the same across the board. Running large language models is staggeringly expensive. OpenAI reportedly loses money on every ChatGPT query. Google runs Gemini across Search, Workspace, Cloud, and dozens of other products. Even modest efficiency gains compound into billions saved.

Jeff Dean, Google's Chief Scientist, has been blunt about this: 'Custom silicon is essential to delivering AI at scale. General-purpose chips simply can't match the efficiency we need.'

What does 6-10x efficiency actually mean?

The metric here is tokens generated per unit of power. A token is roughly three-quarters of a word. If Frozen v2 delivers on the reported efficiency gains, Google could either run Gemini at the same capacity for a fraction of the electricity bill, or scale up capacity dramatically without proportional cost increases.

Data centers already consume enormous amounts of energy. AI workloads are accelerating that trend. A chip that squeezes more useful work from each watt has implications beyond the balance sheet. It affects where Google can build data centers, how quickly it can expand, and how it answers growing scrutiny about AI's environmental footprint.

The 2028 timeline is a long way off

Three years is an eternity in AI. When Google first announced TPU in 2016, GPT-3 didn't exist. The models Frozen v2 will run in 2028 may look nothing like today's Gemini. Google's spokesperson acknowledged as much: 'not every project moves into production.'

Still, the stock market reacted. Investors aren't betting on the chip itself. They're betting on Google's ability to control costs as AI spending escalates. Frozen v2 is a signal that Google has a plan, not just a budget.

Frequently Asked Questions

When will Google's Frozen v2 chip be available?

According to The Information's report, the chip is slated for release sometime in 2028. Google has not officially confirmed the timeline.

How much more efficient is Frozen v2 compared to current Google AI chips?

The report claims Frozen v2 could be 6-10x more efficient, measured by tokens generated per unit of power consumed.

Will Frozen v2 replace Nvidia GPUs in Google's data centers?

Frozen v2 appears designed specifically for running Gemini models efficiently. Google will likely continue using a mix of TPUs, custom chips, and Nvidia hardware for different workloads.

How much is Google spending on AI infrastructure?

Google has announced plans to spend between $180 billion and $190 billion on AI infrastructure buildout.

Source: TechCrunch / Lucas Ropek