How I Cut My Claude API Costs by 80% with Caching and Model Selection

Two weeks into running my first AI-powered API endpoint, I noticed the Claude bill climbing in a way that didn't match the traffic. Not catastrophic yet, but the trajectory was clear — if I didn't change something, the API cost would outpace any revenue the project could generate.

The fix wasn't one thing. It was three changes applied together: using cheaper models for routine tasks, caching responses so the same question doesn't get answered twice, and trimming prompts to stop paying for words that didn't improve the output. Combined, they cut my effective per-call cost by roughly 80%.

Here's each one, with the actual numbers.

The Problem: One Model for Everything

My initial setup was simple and expensive. Every AI call — whether it was generating a 200-word market sentiment report or categorizing a single log entry — went through Claude Sonnet. Sonnet is the strongest model. It's also the most expensive. Using it for a task that Haiku handles equally well is like hiring a lawyer to sort your mail.

The cost difference between models is significant:

# Approximate per-call cost for a typical task # (2,000 input tokens, 200 output tokens) Claude Sonnet: ~$0.015/call Claude Haiku: ~$0.003/call # Difference: roughly 5x # At 100 calls/day: # Sonnet: $1.50/day = $45/month # Haiku: $0.30/day = $9/month

For my kr-sentiment endpoint, the task is: take Korean exchange data (kimchi premium, volume surges, investment warnings) plus filtered news headlines, and produce a 200-word English sentiment report with a score from -1.0 to +1.0. Haiku handles this perfectly. The output quality difference between Haiku and Sonnet for this specific task is invisible to users.

I still use Sonnet for architecture decisions, complex debugging, and anything where bad output creates downstream problems. The split is simple: if the output format is predictable and the task is structured, Haiku. If the task requires nuanced reasoning or the cost of being wrong is high, Sonnet.

Caching: Pay Once, Serve Forever

The single biggest cost reduction came from not calling the AI at all.

My sentiment endpoint was calling Claude on every request. Ten users asking for Korean market sentiment in the same hour meant ten Claude calls producing ten nearly identical responses. Each one cost $0.003. The market sentiment doesn't change minute-to-minute — paying for essentially the same answer ten times was pure waste.

The fix: check the cache before calling Claude. If there's a response less than one hour old, return it instantly. If the cache is empty or expired, call Claude once, cache the result, serve it to everyone for the next hour.

async def get_sentiment(): # Check cache first cached = sentiment_cache.get("latest") if cached and cached["age"] < 3600: return cached["data"] # free, instant # Cache miss — call Claude result = await claude_haiku(prompt) sentiment_cache.set("latest", result, ttl=3600) return result
The economics: Without caching, 100 requests per hour = 100 Claude calls = $0.30/hour. With a 1-hour cache, 100 requests per hour = 1 Claude call + 99 cache hits = $0.003/hour. That's a 99% reduction in AI cost for that endpoint. The cache hit response time is under 1 millisecond. The cache miss takes 2-3 seconds (waiting for Claude). Users can't tell the difference between a fresh response and a 45-minute-old one because the underlying market conditions haven't changed meaningfully.

One critical detail: if ten requests arrive simultaneously when the cache is empty, you don't want ten parallel Claude calls. An async lock ensures only the first request triggers the AI call. The other nine wait for that result and share it. Without the lock, a burst of traffic at cache expiration multiplies your cost by the burst size.

Leaner Prompts, Same Output

This one sounds minor until you multiply it by thousands of calls. Every word in your prompt is a token. Every token costs money. The difference between a verbose prompt and a tight one is 3-4x in input tokens.

# Verbose prompt (~90 tokens) "Please carefully analyze the following Korean cryptocurrency market data and news headlines. Based on your analysis, provide a comprehensive sentiment assessment including a numerical score between -1.0 and +1.0, along with a detailed explanation of your reasoning. Here is the data:" # Lean prompt (~25 tokens) "Korean crypto market analysis. Score -1.0 to +1.0. Under 200 words. Data:"

Both prompts produce usable sentiment reports. The lean version costs about a quarter of the input tokens. Across thousands of calls, that's a meaningful number.

The counterintuitive part: leaner prompts often produce better output. The verbose version gives Claude room to hedge and pad. The lean version forces a direct answer. Less ambiguity in the instruction means less ambiguity in the response.

Safety Rails That Prevent Surprise Bills

Cost optimization only matters if you also prevent the worst case. Three guardrails I run on every project that uses AI APIs:

Platform spending caps. Both Anthropic and OpenAI let you set monthly limits in the dashboard. Set one. Even a generous cap is the difference between an unexpected $50 and an unexpected $500. Mine is set well above normal usage but well below "something went catastrophically wrong."

Max tokens per call. Every Claude call in my code includes a max_tokens parameter. If a response tries to run forever, it gets cut off. The output might be incomplete, but the cost is bounded.

Cost logging. Every Claude call writes a log entry with input tokens, output tokens, and estimated cost. A background process sums the hourly spend. If it exceeds a threshold, I get a Telegram alert. This caught a misconfigured retry loop early — it had fired 40 times in a minute before the alert stopped me from letting it continue.

What My Costs Actually Look Like

Real numbers from running multiple AI-powered services across my $0 infrastructure stack:

# Daily AI spend across all projects Typical quiet day: $0.00 (no sentiment requests, cache covers everything) Typical active day: $0.05 - $0.15 Busiest day so far: $0.47 (debugging session triggered extra calls) # Per-endpoint economics (kr-sentiment) Revenue per call: $0.05 Claude cost per call: $0.003 (on cache miss only) Cache hit rate: ~95% Effective AI cost: ~$0.00015 per request served Margin: >99%

The 95% cache hit rate is the number that makes the whole thing work. Only 1 in 20 requests actually triggers a Claude call. The other 19 get cached data at zero marginal cost. Revenue stays at $0.05 per request regardless of whether it's a cache hit or miss.

The Three Changes, Ranked by Impact

If you can only do one thing, do caching. It has the largest single effect on costs because it eliminates calls entirely rather than making them cheaper.

If you can do two things, add model selection. Switching routine tasks from Sonnet to Haiku is a 5x cost reduction on those calls with no quality loss for structured outputs.

Prompt trimming is the third priority. It matters at scale but the per-call savings are smaller than the other two. Do it after caching and model selection are in place.

Together, the three changes took my projected monthly AI cost from "unsustainable" to "rounding error." The infrastructure costs $0. The AI costs pennies per day. The margin on every paid API call is above 99%. That's the cost structure that lets a solo project survive long enough to find users.

Related guides:

Disclaimer: This blog documents practical workflows based on personal experience. Nothing here is financial, legal, or professional advice.

Comments

Post a Comment