Auditing a Prompt for Cache Hits: Prefix Order, Breakpoints and Minimum Lengths

What Anthropic, OpenAI and Google document about prompt caching, and a five-step audit for an existing agent request

*Image: Network equipment and cables organized in a server rack at a modern office environment during the afternoon by Shixart1985, licensed under CC BY 2.0.*

An agent that sends a 10,000-token system prompt on every turn pays to process the same tokens again each time, unless the provider can match the beginning of the request against a saved copy. All three providers examined here describe that match as a prefix match, and Anthropic and OpenAI list the conditions that break it. A request builder that puts a timestamp, a user name or a reordered tool list near the top can lose the discount without any error being returned. This article walks through the documented rules, one independent measurement, and an audit you can run against your own request payloads.

Documentation was read on 2026-11-02. Provider caching pages change often, so re-check the pages linked below before relying on a number or a model name.

What the three providers document

Anthropic. The prompt caching page says a cached prefix covers the whole prompt in a fixed order: tools, then system, then messages, up to and including the block marked with cache_control. A change at one level invalidates that level and every level after it. Changing a tool definition (names, descriptions, parameters) therefore invalidates the entire cache. The page documents two ways to place the breakpoint: automatic caching, where a single top-level cache_control field puts the breakpoint on the last cacheable block, and explicit breakpoints set on individual content blocks.

Two mechanics on the same page explain most surprising misses. First, a cache write happens only at the breakpoint, and the stored key is a hash of everything up to and including that block. Second, a read walks backward from the breakpoint looking for entries earlier requests already wrote, and it checks at most 20 positions. The page includes a worked case where static content fills blocks 1 through 5 and a timestamp plus the user message sit in block 6, with the breakpoint on block 6. The hash at the breakpoint changes on every request, the backward walk finds no earlier write, and the caller pays for a fresh cache write each time without ever getting a read. Its stated fix is to move cache_control to block 5, the last block that stays the same across requests. The page notes that automatic caching hits the same problem in this layout, because it places the breakpoint on the last cacheable block, which is the one that changes.

The default lifetime of an entry is 5 minutes, measured from the start of the request that wrote or read it. The page also offers a 1-hour lifetime at additional cost. Prompts shorter than the model's minimum cacheable length are processed without caching and no error is returned. The page says to check cache_creation_input_tokens and cache_read_input_tokens in the response usage: if both are 0, the prompt was not cached.

OpenAI. The prompt caching guide describes a reusable prefix as the unchanged tokens at the beginning of a prompt and says reuse requires the entire rendered prefix to match. It lists the request settings that alter the rendered prefix, including model, tools (names, descriptions, schemas and ordering), parallel_tool_calls, reasoning.effort and text.verbosity. It documents a prompt_cache_key field, reports hits through cached_tokens in the usage object, and states a 1,024-token minimum for its newest models, with earlier models varying by request settings. Retention depends on the model. Earlier models use a prompt_cache_retention setting whose supported values depend on the model: some accept in_memory or 24h, while GPT-5.5 and GPT-5.5 Pro support only 24h. For GPT-5.6 and later, the default minimum lifetime is 30 minutes, and the Responses API offers a 12-hour option.

Google. The Gemini context caching page says implicit caching is on by default for Gemini 2.5 and newer models and lists a minimum input token count per model, which was 4,096 for the first models in its table on the capture date. Explicit caching remains available for supported Gemini 3 and 2.5 models through the generateContent API, though not in the Interactions API or for Gemini 4 Argon onward. Its guidance for raising implicit hit rates is short: put large and common content at the beginning of the prompt, and send requests with a similar prefix within a short time. For multi-turn and agent loops on Gemini 4 (Argon and newer), the page adds that the full previous request is cached only if the new request adds at least 2,050 tokens, counting model output, thoughts and function results; for smaller additions, the last 2,050 tokens of the previous request are re-processed at the standard input rate.

All three pages point the same way on ordering: content that repeats across requests belongs first, and content that varies belongs last. Anthropic and OpenAI also state that a change in the repeated part ends reuse from that position onward; Google's page gives the ordering guidance without spelling out an invalidation rule. The providers differ in who places the breakpoint (automatic or explicit on Anthropic, implicit or explicit on OpenAI's newest models, implicit on Google with explicit caching still offered for supported Gemini 3 and 2.5 models), in lifetimes, and in minimum lengths.

What a measurement of agent workloads adds

Documentation states the mechanism but gives no figure for what an agent loop gains. The preprint Don't Break the Cache (arXiv 2601.06007) measured prompt caching across OpenAI, Anthropic and Google models on DeepResearch Bench, a benchmark where agents answer research questions through web search tool calls. The evaluation used over 500 agent sessions with 10,000-token system prompts, with 40 samples per condition for each model. It reports cost savings of 41% to 80% against a no-cache baseline, with statistically significant results for all four models tested.

The paper compared three strategies: caching the full context, caching only the system prompt, and caching everything except tool results. Its authors report that caching only the system prompt gave the most consistent benefits across cost and latency, and that cost savings were consistent across all three strategies. They also report that enabling full-context caching can increase latency, which they attribute to dynamic tool calls and results that may trigger cache writes for content that will not be reused across sessions.

The study's scope is narrow. The workload is one benchmark of web-search research agents, the system prompts were a fixed 10,000 tokens, the study is a preprint, and the paper itself notes that minimum lengths, lifetimes and prices vary by provider and change over time. A coding agent that edits files, or a support bot with short prompts, may land elsewhere. The savings range shows the mechanism matters for long-running agent sessions, and a specific deployment needs its own measurement.

A five-step audit of an existing request

The steps below combine the documented rules. They need only access to logged request payloads and the usage fields each provider returns.

1. Print the rendered request in order. Dump the exact payload sent for two consecutive turns of one session: tools, system text, then messages. Diff the two dumps. The first differing character marks the latest point a prefix cache can match, because the providers above describe matching from the start of the request.

1. Move variable text below stable text. Search the first diff hunk for timestamps, request IDs, user names, retrieved documents and feature flags. Move each below the last stable block. On Anthropic, set an explicit breakpoint on the last stable block rather than on the incoming user message. Keep tool lists in a fixed order, since OpenAI lists tool ordering among the settings that change the prefix and Anthropic invalidates the whole cache on any tool definition change.

1. Check the length against the minimum. Count the stable prefix in tokens with the provider's own counter. If it falls under the documented minimum for the model, the prompt is not cached; Anthropic's page adds that no error is returned in that case. Anthropic's page also suggests that expanding the cached content to reach the minimum is often worthwhile when a prompt is just short, which is a cost judgment to make with your own traffic numbers.

1. Compare request spacing with the lifetime. Anthropic's default 5-minute lifetime is measured from the start of the request, so a long streamed response shortens the usable window for the next turn. An agent that pauses for tool execution or human approval for longer than the lifetime reads nothing on the next call. Log the gap between consecutive requests of a session and compare it with the lifetime in the provider page. On Gemini 4, also log how many tokens each turn adds, since additions under 2,050 tokens do not get the full previous request cached.

1. Read the usage fields after every change. Log cache_creation_input_tokens and cache_read_input_tokens for Anthropic, cached_tokens for OpenAI, and usage.total_cached_tokens for Gemini. A healthy session shows a write on the first turn, then large reads on later turns along with small incremental writes for the content each turn appends. A session with large writes on every turn and no reads points first to a prefix that changes between requests. The sources document other causes too: gaps longer than the cache lifetime (step 4) and, on Anthropic, a breakpoint that a growing conversation has pushed 20 or more blocks past the last write.

A hypothetical example

The following layout is invented to illustrate the audit. It is not a measurement from any real system.

Before (request builder output, turn 1 and turn 2)
  tools      <12 tool definitions, fixed order>
  system[0]  "Current time: 2026-11-02T14:03:11Z"   <- differs each turn
  system[1]  <4,200 tokens of policy and style rules>
  messages   <conversation so far>

The diff in step 1 stops at the first line of system[0]. The tool definitions before it still match, but a breakpoint placed after the rules hashes the timestamp too, so every turn writes and none reads.

After
  tools      <12 tool definitions, fixed order>
  system[0]  <4,200 tokens of policy and style rules>   <- breakpoint here
  system[1]  "Current time: 2026-11-02T14:03:11Z"
  messages   <conversation so far>

With the timestamp moved after the breakpoint, the stable prefix is the tool definitions plus the rules. Under the Anthropic rules above, turn 2 can read the entry turn 1 wrote, provided the gap stays inside the lifetime and the prefix clears the model's minimum length. The reordering costs nothing if the model does not need the time at the top of the prompt, which is a quality question to test on your own tasks.

What remains unverified

Provider pages state prices, lifetimes and minimums that were current on the capture date, so the numbers quoted above may have changed. No source read here covers whether moving a time or user field lower in the prompt changes answer quality; test that on your own tasks.