Prompt Caching
Prompt Caching is an API optimization architecture and cost-reduction mechanism provided by frontier LLM providers (notably anthropic) that enables API builders and harness developers to cache frequently reused, stable context prefixes (such as system prompts, tool schemas, and reference documentation) rather than paying full price to re-parse and re-ingest them on every turn.
Core Economics and Pricing
Prompt caching transforms the operational economics of agentic loops and multi-turn workflows:
- 90% Cost Reduction: Caching stable input tokens typically delivers up to a 90% discount on repeated input context across API requests.
- Opus Tier Example: For frontier models such as Claude Opus, standard input tokens cost approximately 0.50 per million tokens.
- Latency Improvements: Beyond monetary savings, cached tokens bypass redundant model pre-fill computations, significantly reducing time-to-first-token (TTFT) for multi-turn agent runs.
Implementation Targets for AI Builders
As emphasized by nate-b-jones in his 2026 builder guidance, prompt caching is no longer an optional or advanced feature—it is a baseline engineering requirement for production systems. Builders should audit their plugins and cache three primary stable layers:
- System Prompts & Personas: Core agent instructions, operational guidelines, and style guardrails that remain identical across conversations.
- Tool & MCP Definitions: Dynamic or static tool definitions, function schemas, and MCP plugin specifications (see harness-design).
- Reference Documents & Knowledge Context: Long-lived background documentation, static codebases, schemas, or memory indexes loaded into the context window.
Strategic Significance in Agentic Harnesses
Prompt caching directly enhances overall cost-per-accepted-result and makes sophisticated multi-step agent designs economically viable:
- Harness Auditing: Developers must audit their active plugins and harnesses to ensure that dynamic, frequently changing variables do not invalidate the cached prefix.
- Multi-Turn Workflows: Enables dense context architectures (such as extensive skill libraries and automated verification checks) to be kept persistently in-context without incurring prohibitive per-turn token churn.
- Synergy with Token-Efficient Models: Paired with streamlined token consumption in models like fable-5-1 (which uses ~45% fewer tokens per task), prompt caching makes continuous autonomous background workflows financially viable (see fable-5-1-is-quietly-45-percent-cheaper-to-run).