Optimizing API Costs: Strategic Use of Claude and Open Source Models

The first time I really paid attention to my AI API bill, I'd been running the ACP Agent for about two weeks. The number wasn't catastrophic, but it was on a trajectory that would've been catastrophic by month-end if I hadn't noticed.

The problem wasn't the project. The problem was that I'd been using the most expensive model for every task — including tasks a much cheaper model could handle perfectly. Categorizing logs, formatting strings, summarizing structured data: all of these were running through the premium tier when they didn't need to.

API costs are the silent leak in AI-native development. They don't break anything, they don't show up as errors, and they only become a problem once they're already a problem.

What This Post Covers

The strategies I use to keep AI API costs predictable across production projects: matching models to tasks, designing prompts that don't waste tokens, caching aggressively where it makes sense, and the safety rails that prevent runaway costs from a misbehaving agent. The goal isn't being cheap — it's being efficient enough that AI infrastructure doesn't burn the project's economics.

Match the Model to the Task

The biggest cost mistake I made early on was using the same model for everything. The instinct is reasonable: "Use the best model, get the best results." The reality is that "best" varies wildly by task type, and the price difference between models is enormous.

For complex reasoning — debugging gnarly issues, designing architecture, writing nuanced prose — Claude Sonnet earns its higher cost. The output quality is meaningfully better than cheaper alternatives.

For routine work — categorizing log entries, summarizing API responses, generating short structured outputs — Claude Haiku is fast, cheap, and entirely sufficient. The output quality difference between Haiku and Sonnet on these tasks is invisible. The cost difference is roughly 5x.

For the x402 Protocol's kr-sentiment endpoint, I use Haiku. It takes exchange data and news headlines, produces a 200-word English sentiment report. Haiku handles it for about $0.003 per call. Using Sonnet would cost roughly five times that for output a user couldn't tell apart. The architecture decisions in that endpoint — design, caching, fallbacks — happened in Sonnet during planning. The runtime work happens in Haiku.

The rule I follow: Use the strongest model when the cost of being wrong is high — debugging, architecture, code review, anything where bad output causes downstream problems. Use the cheapest model that still works when the task is structured and the output format is predictable. The decision per task takes about 10 seconds, and the savings compound across thousands of calls.

Cache Aggressively, but Cache Right

If the AI is going to produce the same output for the same input, paying for it twice is a waste. Caching is the cleanest way to cut costs without sacrificing functionality.

For the x402 Protocol's sentiment endpoint, I use a one-hour cache. When a request comes in, the API checks if there's a result from the last hour. If yes, return it instantly. If no, call Claude and cache the result. Korean crypto sentiment doesn't change minute-to-minute, so an hour of staleness is invisible to users and the cost reduction is dramatic. With a single user calling the endpoint repeatedly, costs would be unbounded. With caching, costs are capped at 24 calls per day per user, regardless of how many requests come in.

The pattern works for any task where the inputs are stable and the outputs don't need to be brand new every time:

# Bad: every request hits Claude def analyze_market(): return claude_call(prompt) # pays every time # Good: cache check first def analyze_market(): cached = get_cache("market_analysis") if cached and cached.age < 3600: return cached.value # free result = claude_call(prompt) set_cache("market_analysis", result, ttl=3600) return result

The caching layer doesn't have to be sophisticated. A simple in-memory dictionary works for solo projects. Redis works once you have multiple processes that need to share cache. The thing that matters is the pattern: check before you call.

The trap is caching things that shouldn't be cached. User-specific outputs need user-keyed caches. Time-sensitive data needs short TTLs. If you're not careful, you'll cache something stale and ship the wrong answer to a user who's expecting fresh data. The discipline is to cache only when staleness is genuinely acceptable.

Lean Prompts, Sharp Outputs

Every word in your prompt costs money. Every word in the response costs money. For a project doing a few API calls a day, this doesn't matter. For a service running thousands of calls daily, prompt design becomes a real lever.

The temptation is to give the AI massive context every time: "Here's the entire conversation history, here's the user profile, here's the documentation, please answer this small question." Each of those input tokens is paid for, and most of them aren't actually contributing to the answer.

The shift I made: trim the context to what's actually needed. Replace verbose instructions with concise ones. Specify the output format explicitly so the AI doesn't pad with explanations.

Compare these two prompts that produce the same useful output:

# Verbose (~80 tokens) "Please carefully review the following code and let me know if there are any errors. After identifying any issues, please suggest specific improvements that could make this code better. Be thorough in your analysis. Here is the code to review: [code]" # Lean (~20 tokens) "Review this code. List bugs and improvements. Be brief. [code]"

Both produce useful output. The lean version costs roughly a quarter of the input tokens. Multiply by thousands of calls and the difference shows up on the bill.

This isn't about being curt with the AI. It's about treating prompt design as a cost surface. When the prompt is clearer, the output is usually better too. Vague prompts produce padded outputs because the model is hedging against uncertainty. Sharp prompts produce sharp outputs.

The Safety Rails That Save You From Yourself

The worst API bill scenario isn't slow drift. It's an agent stuck in an infinite loop, hammering the API thousands of times before you notice. This has happened to me. The bill wasn't catastrophic because of platform-level limits, but the lesson stuck.

The fix is layered. None of these alone is sufficient; together they cap downside risk:

Hard usage limits at the platform level. Both Anthropic and OpenAI let you set monthly spending caps in the dashboard. Set them. Even if you set them generously, the cap is the difference between "annoying surprise" and "career-ending bill."

Per-task token budgets in your code. When you call Claude, set a max_tokens parameter. If a runaway response tries to stream forever, it gets cut off at your limit. The output might be incomplete, but the cost is bounded.

Logging every call with cost estimates. Every Claude call in the x402 Protocol writes a JSONL line with input tokens, output tokens, and estimated USD cost. If something starts hammering the API, the log shows it within seconds. A simple background job alerts me if hourly Claude spend crosses a threshold.

Circuit breakers for retry logic. If a Claude call fails, retry it — but not infinitely. After three failures, stop. The original failure was probably a transient issue; if the next three retries also fail, something is structurally wrong and continuing to retry just burns money.

Each of these is small. Together they make API costs predictable instead of mysterious.

What This Looks Like in Production

Across the x402 Protocol, the ACP Agent, the SpeedTap bot, and Weather Bot — everything I run that uses AI APIs — here's roughly what costs look like:

Daily AI spend, all projects combined: usually under $1, often $0 on quiet days.

Maximum theoretical daily spend if everything ran constantly: $5-10, capped by caching layers and usage budgets.

Actual peak day so far: about $2, on a day I was actively debugging the kr-sentiment endpoint and triggering more calls than usual.

This isn't a heroic optimization story. It's just basic cost discipline applied consistently. Pick the right model. Cache when stale data is acceptable. Write prompts that don't waste tokens. Set guardrails that prevent runaway costs. The strategies are obvious once you list them. The work is in actually applying them every time you build something new.

Where to Start

If you've never thought about API cost as a design dimension, here's the entry path. Pull up your current AI API spending dashboard. Look at what you spent last month. Estimate how many of those calls used the most expensive available model.

Pick one task in your codebase that hits the API frequently. Test whether the cheaper model handles it acceptably. If yes, switch. That single change typically cuts costs significantly with no functional impact.

From there, audit prompts for unnecessary verbosity and check whether your most-called endpoints are caching results. Each pass through the codebase reveals more efficiency. The cumulative effect is the difference between AI being a sustainable infrastructure cost and a project-killing burn rate.

What's Next

Cost optimization is a horizontal concern — it touches every project. The next post in this series goes vertical: comparing serverless platforms versus traditional VPS hosting for AI workloads, and which one I pick for which type of task. The decision isn't always obvious, and the wrong choice creates cost and complexity problems that compound over time.

← Previous: GitHub Workflow 101       Next: Serverless vs. Traditional VPS →

---

More posts in this series will cover the actual stack — deployment patterns, security, monitoring, and the workflows that hold everything together. If you're working on shipping something with AI tools and have questions, drop them in the comments — the more we share, the faster we all move.

Disclaimer: This blog documents practical development workflows based on personal experience. Nothing here is financial, legal, or professional advice.