11 rules for cutting AI token costs in code assistants

Key Takeaways

• Start with balanced models like Gemini 3.5 Flash and scale up only when tasks fail or require complex reasoning
• Automate repetitive tasks with scripts and delegate output-heavy work to sub-agents to avoid context bloat
• Use the 'Elephants and Goldfish' approach: plan in high-reasoning sessions, execute in clean low-token sessions

Google Cloud has published a practical guide to AI token efficiency, laying out 11 principles for developers who want to keep their AI coding assistants fast, accurate, and cheap. The core argument: context bloat is the enemy. Stuffing too much information into prompts increases latency, burns through your token budget, and pushes models toward hallucination.

The guide, authored by Alex Astrum and Luke Schlangen from Google's developer relations teams, reframes the developer's role. You're not writing every line anymore. You're directing AI agents. And direction costs tokens.

Why token efficiency matters now

Token pricing varies by model, but the range sits between $0.01 and $0.06 per thousand tokens for most leading LLMs. That sounds cheap until you're running hundreds of queries daily across a development team. A 10,000-token prompt costs 5x more than a 2,000-token prompt, and the bloated version often confuses the model more than it helps.

Modern LLMs offer context windows of 128K to over 1 million tokens. But larger contexts mean higher latency and cost. The guide pushes developers to treat tokens like a constrained resource, not an infinite well.

The 11 principles, condensed

Google's framework splits into three themes: model selection, task structure, and context hygiene.

On model selection, the advice is simple. Start with Gemini 3.5 Flash at medium reasoning. Scale up only when a task fails, requires too many iterations, or demands complex design work. Most tasks don't need your largest, most expensive model.

For task structure, the guide introduces the "Elephants and Goldfish" approach, credited to David Rensin. Use a high-reasoning, long-context session (the "Elephant") to generate a detailed execution plan (the "Goldfish"). Then execute that plan in a clean, low-token session. Checkpoint progress with commits so you can restart from a clean state when context fills up.

Delegation matters. Output-heavy tasks like deep research or separating frontend from backend work should go to sub-agents. You reconcile final results, not full trajectories.

On context hygiene, the guide is blunt: stop explaining your workflow, testing rules, and environment in every prompt. Package reusable context into SKILL.md files and scripts. The agent triggers them automatically.

Specific tactics that save tokens

Several principles target common waste patterns:

• Automate with scripts and CLI tools. For repetitive chores like formatting files or extracting log data, have the agent create simple local tools. Run read-only commands to research the codebase before writing code.
• Shift verification left. Run local builds and unit tests before UI testing. Save expensive browser smoke tests for right before handoff.
• Undo when adrift. If the agent drifts and you know the fix, use the Undo button or revert files. Don't pile corrective prompts on a broken state. That poisons the context.
• Be specific, not micro-managing. Point the agent to the exact file, section, or error. Use inline comments like "// SHOULD BE X, NOT Y, FIX THIS" instead of sending it on a 10,000-line log quest.
• Iterate on rules. If you keep correcting the same behavior, update your global rules in AGENTS.md. Fix the instructions instead of prompting repeatedly.

Avoiding runaway loops

One principle deserves its own section: avoid uncontrolled loops. Supervisor loops that scan projects for pending work can find optimizations, but they can burn your entire token budget in minutes. If you run loops, set strict limits and stop conditions. High autonomy requires tighter guardrails.

The guide specifically warns against letting agents poll status in a loop. Use event-driven wakeups instead.

Session management

The final principle: start new sessions for each new topic. Continuing the same chat lets the agent reuse existing context, which helps. But if you're switching topics, that old context becomes noise. A fresh session pulls in only what it needs.

The bigger picture

Token efficiency isn't just about cost. It's about attention. Context bloat causes models to forget instructions, miss critical details, and hallucinate. The guide frames this as directing AI attention, not just managing a budget.

For engineering leaders, the implication is clear: token discipline is becoming a team skill. Just as code review caught bugs and style guides enforced consistency, prompt hygiene and context management will become part of standard practice.

Frequently Asked Questions

What is AI tokenomics in software engineering?

AI tokenomics refers to managing token consumption in AI coding assistants. Tokens are the units LLMs use to process text. Efficient token use reduces costs, improves response speed, and decreases hallucination rates.

How much do AI tokens cost for coding assistants?

Token pricing varies by model, typically ranging from $0.01 to $0.06 per 1,000 tokens for leading LLMs. Costs scale with prompt size and frequency of use.

What is the Elephants and Goldfish approach in AI coding?

A two-phase method where you use a high-reasoning, long-context session to create a detailed plan, then execute that plan in a clean, low-token session. It separates planning from execution to reduce context bloat.

How do I reduce context bloat in AI coding assistants?

Package reusable instructions into SKILL.md files, use specific inline comments instead of open-ended searches, start new sessions for new topics, and undo rather than correct when the agent drifts.

What model should I start with for AI-assisted coding?

Google's guide recommends starting with Gemini 3.5 Flash at medium reasoning, then scaling to larger models only when tasks fail or require complex design work.

Source: Cloud Blog