Better prompt caching for GPT-6

GPT‑6 enables persistent agents to work for hours on complex tasks, from refactoring codebases to producing well-researched documents and presentations. The applications behind these agents make a series of API requests that build on one another, often carrying forward the same instructions, tool definitions, and context from earlier turns. OpenAI caches that shared context to reuse computation across requests, reducing response times and giving developers discounts of up to 90% on cached input tokens.

With the GPT‑6 family, we launched an improved prompt caching system that delivers higher cache hit rates by default. We now give cache discounts for eligible shared prefixes reused within a 30-minute window. We’re also introducing new tools to help developers monitor cache performance, diagnose misses, and choose how much of a prompt to cache.

The new Prompt Caching Dashboard ⁠ (opens in a new window) shows how much of your application’s input is served from cache. Track hit rates over time and use the input composition chart to compare cached and uncached tokens. These views help you spot drops in cache hits and evaluate how changes to your application impact caching performance.

When you see an unexpected cache miss, use the prompt caching diagnostics tool ⁠ (opens in a new window) to understand what happened. Compare a request with a recent response to identify changes to the model, tools, settings, or input that prevented reuse. The estimated number of affected tokens helps you assess the size of the impact and decide how you can optimize your integration to maximize cache hit rates.

Choose what to cache. Explicit cache breakpoints let you choose which prompt prefixes to reuse. The refreshed prompt caching guide ⁠ (opens in a new window) explains how to use them, how long cached prefixes remain eligible, and how changes to tools and inputs affect reuse.

Adjust reasoning effort without breaking cache. On GPT‑6 models, you can now change reasoning effort ⁠ (opens in a new window) between responses without breaking cache. Raise effort for a harder task or lower it for a routine follow-up by appending a configuration_update while leaving request-level reasoning effort unchanged. This lets you adjust how much reasoning a task needs while preserving reusable context.