What is prompt caching?
Prompt caching is a model API feature that stores the processed form of a repeated prompt prefix, so later requests that start with the same prefix are billed at a lower cached-input rate.
Checked
Agents resend a lot. Every step of a run repeats the system prompt, the tool definitions and often a long stretch of history or documents. Prompt caching lets the provider reuse its work on that repeated prefix, and bills those tokens at a fraction of the standard input price.
The saving depends on structure. A cache matches from the start of the prompt, so anything that changes early in the prompt breaks the match for everything after it.
How to structure prompts for caching
Stable content first
System prompt, tool definitions and reference documents go at the start, in a fixed order.
Variable content last
The user's message, timestamps and per-request data go at the end.
Append, do not rewrite
Anything that edits earlier context, such as compaction or clearing old tool results, changes the prefix, so later calls no longer match the cache. Plan those edits at set points.
Check the hit rate
Provider usage data reports cached tokens; track them per step to confirm the cache is working.
Where it shows up in lab guidance
Google's context engineering guidance lists keeping a stable prefix for caching as one of its core moves. Anthropic's Contextual Retrieval post notes that a knowledge base under about 200,000 tokens can go straight into the prompt, with prompt caching keeping the repeated cost down, before reaching for retrieval.
Providers implement caching differently. Gemini 4 Argon, for example, supports implicit caching only, while other APIs let developers mark cache points explicitly. Rates and rules change often, so they are worth checking on the provider's pricing page before a cost model is fixed.
How Motiontic's engineers apply it
Motiontic's engineers lay out prompts for caching from the first build, then read the cached token counts on real traffic. On agents with long system prompts and many tools, prompt order is often the cheapest cost fix available.
Standard and cached input prices per 1M tokens, as of 2026-10-31
| Model | Standard input | Cached input |
|---|---|---|
| Claude Opus 5.5 | $4.00 | $0.20 |
| Claude Sonnet 5.5 | $2.00 | $0.10 |
| Claude Haiku 5.5 (prompts up to 100K) | $0.10 | $0.01 |
| GPT-6.1 Sol | $2.00 | $0.10 |
Common questions
How much does prompt caching save?
It depends on the model and on how much of each prompt repeats. As of 2026-10-31, cached input on Claude Opus 5.5 and GPT-6.1 Sol is priced at one twentieth of the standard input rate, and only the repeated prefix gets that rate.
Why is my prompt cache not being hit?
Usually because something early in the prompt changes between calls, such as a timestamp, a reordered tool list or edited history. The cache matches from the start, so stable content must come first and stay identical.
Is prompt caching the same as semantic caching?
No. Prompt caching is done by the model provider on an identical prompt prefix, and the model still generates a new response. Semantic caching is an application layer that returns a stored answer when a new question is similar to an old one.
Related
Context engineering
Curating everything in the model's view, not just the wording of the prompt.
Context rot
Why agents get worse the longer they run, and what keeps context clean.
Agent memory
Session state for the task at hand, plus stored knowledge the agent recalls later.
AI product development
New products and major AI features, built from scratch alongside the client's team.
Running agents that have to hold up?
Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.