Concept: Reused Input & Token Compounding
Reused Input & Token Compounding refers to the structural phenomenon where conversational LLM interfaces simulate memory by re-transmitting the entire conversation transcript, system instructions, and tool definitions on every single user interaction.
Mechanics of Context Compounding
When interacting with AI models in thread-based interfaces (such as codex or claude-code):
- Turn 1: Cost = Initial Prompt + System Setup.
- Turn 2: Cost = Turn 1 Prompt + Response 1 + New Query.
- Turn 10: Cost = New Query + All 9 Previous Turns (Prompts + Responses + Tool Calls).
- Turn 30: The newly typed message becomes a microscopic rounding error (<1%) compared to the accumulated prompt payload.
In heavy development environments, this mechanism generates massive context inflation. In an operational audit of 143 codex threads in a single day, nate-b-jones recorded 3.77 billion tokens moved through his workspace, of which 3.59 billion tokens (96%) were reused input.
The Output Double-Tax
Output tokens carry a compound financial and context tax:
- Output tokens are billed at higher rates per token upon generation.
- Every output token generated automatically becomes part of the permanent input context for every future turn in that conversation, repeatedly incurring input token billing.
Structural Mitigation Framework
To defeat token compounding without waiting for frontier labs (who lack financial incentives to reduce user token burn), operators utilize three layers of defense:
- In-Place Editing: Editing prompt typos or unclear queries directly instead of adding corrective follow-up turns.
- Artifact Carriage: Extracting finished deliverables (e.g., research briefs or code specs) and passing only those artifacts into fresh threads, discarding intermediate debate history.
- Tool Definition Pruning: Restricting connected MCP tool servers. Connecting standard developer MCP tools (GitHub, Slack, Sentry, Grafana) loads roughly 55,000 tokens of tool definitions into context before the model executes any prompt logic.
- Automated Skill Guardrails: Deploying skills like token-saver-skill to pre-search files locally, truncate outputs, and enforce exact word bounds.
- Proxy Interception: Intercepting requests at the network proxy layer with tools like ringer to check local caches in open-brain-stack and enforce hard token limits.