Python: Support prompt cache breakpoints for GPT-5.6 models in OpenAI clients by Mordris · Pull Request #7163 · microsoft/agent-framework · GitHub

Motivation & Context

GPT-5.6 models support explicit prompt cache breakpoints, and cache writes are billed on these models, so controlling where a prefix is cached now matters for cost. Today the clients silently drop a prompt_cache_breakpoint set through Content.additional_properties, and prompt_cache_options is not part of the typed chat options, so neither knob is usable from the framework.

Description & Review Guide

What are the major changes?
- A prompt_cache_options field on OpenAIChatOptions and OpenAIChatCompletionOptions, typed with the openai SDK's own PromptCacheOptions params type for each API. _prepare_options already forwards options generically, so no extra plumbing was needed.
- A shared _attach_prompt_cache_breakpoint helper (private to the two chat clients) copies prompt_cache_breakpoint from Content.additional_properties onto the outgoing part, following the existing detail/file_id/filename pattern. It is applied to the blocks each API accepts: input_text/input_image/input_file for the Responses client, and text/image/audio/file parts for the Chat Completions client.
- A client_prompt_caching.py sample (per review feedback) demonstrating both knobs with the visible cached-token count.
- Chat Completions: _prepare_content_for_openai gets an explicit text case (text previously fell through to the to_dict fallback), and text parts that carry a breakpoint keep list-form content, because the plain-string flattening cannot hold one. The same applies to system/developer messages. Without a breakpoint, the existing string forms are preserved unchanged.
What is the impact of these changes?
- Opt-in only. Requests without the new option/metadata are unchanged; unit tests pin the exact part shapes for that case.
- Verified against the live API: with mode: "explicit" (which disables the implicit breakpoint), a repeated ~1.6k-token prefix reports cached_tokens ≈ 3100 on the second call across gpt-5.6-luna, gpt-5.6-sol and gpt-5.6-terra on the Responses API, and on gpt-5.6-luna via Chat Completions; the same setup without a breakpoint reports 0. Older models are unaffected; opting in on one surfaces the API's own 400 (prompt_cache_breakpoint is not supported on this model).
- The SDK's PromptCacheOptions types were introduced in openai 2.45.0, so the import is guarded with a local fallback and the dependency floor stays at 2.25.0; older openai versions keep working (verified against 2.25.0), and sending the option itself requires 2.45.0, as noted in the field docstrings.
What do you want reviewers to focus on?
- Whether additional_properties is the surface you want for the per-part breakpoint, or if you'd rather have a first-class field on Content (happy to rework it).
- The list-vs-string content decision for Chat Completions text parts that carry a breakpoint.

Related Issue

Fixes #7157

Contribution Checklist

The code builds clean without any errors or warnings
All unit tests pass, and I have added new tests where possible
The PR follows the Contribution Guidelines
This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Pull request overview

This PR adds Python OpenAI client support for GPT‑5.6 prompt caching controls by (a) exposing request-level prompt_cache_options in the typed options and (b) forwarding per-part prompt_cache_breakpoint metadata from Content.additional_properties into the outgoing request payloads, while preserving existing message-shape behavior when no breakpoint is present.

Changes:

Introduces PromptCacheOptions and a shared attach_prompt_cache_breakpoint(...) helper in _shared.py.
Adds prompt_cache_options to both Responses (OpenAIChatOptions) and Chat Completions (OpenAIChatCompletionOptions) typed option sets.
Updates Chat Completions content/message preparation to keep list-form text parts when a breakpoint is present, and adds unit tests covering breakpoint propagation and options passthrough.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
python/packages/openai/agent_framework_openai/_shared.py	Adds a typed PromptCacheOptions shape and a helper to copy prompt_cache_breakpoint from Content.additional_properties into outgoing parts.
python/packages/openai/agent_framework_openai/_chat_completion_client.py	Plumbs breakpoint copying into Chat Completions parts and adjusts string-vs-list flattening so breakpoints are not lost.
python/packages/openai/agent_framework_openai/_chat_client.py	Plumbs breakpoint copying into Responses API parts and adds typed support for prompt_cache_options.
python/packages/openai/tests/openai/test_openai_chat_completion_client.py	Adds unit tests ensuring breakpoints are preserved (list-form) and prompt_cache_options passes through.
python/packages/openai/tests/openai/test_openai_chat_client.py	Adds unit tests ensuring breakpoint propagation for Responses parts and prompt_cache_options passthrough.

Yunus Emre Gültepe (@Mordris) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
@microsoft-github-policy-service agree [company="{your company}"]
Options:

(default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"
Contributor License Agreement

@microsoft-github-policy-service agree

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/openai/agent_framework_openai
_chat_client.py	1299	123	90%	335, 348, 698–702, 710–713, 719–723, 773–780, 782–784, 791–793, 851, 859, 882, 1000, 1099, 1158, 1160, 1162, 1164, 1230, 1244, 1324, 1334, 1339, 1382, 1498–1499, 1514, 1751, 1756–1757, 1840, 1850, 1877, 1883, 1893, 1899, 1904, 1910, 1915–1916, 1996, 2040, 2043–2046, 2060, 2062, 2070–2071, 2083, 2125, 2190, 2207, 2210, 2237–2239, 2278, 2295, 2298, 2360, 2367, 2404–2405, 2440, 2478–2479, 2497–2498, 2541, 2668–2669, 2687, 2773–2781, 2811, 2957, 2972, 3021–3024, 3034–3036, 3063–3065, 3075–3076, 3082, 3097, 3219–3220
_chat_completion_client.py	388	18	95%	468, 564–565, 569, 815, 817, 822, 825, 926, 928, 945, 966, 974, 998, 1011, 1044, 1067, 1388
_shared.py	163	16	90%	243, 262–264, 276, 286, 298, 304, 326, 330, 364–365, 384, 403–404, 406
TOTAL	45144	5089	88%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
9232	33 💤	0 ❌	0 🔥	2m 27s ⏱️

would be good to have a sample for this as well

Added a sample: samples/02-agents/providers/openai/client_prompt_caching.py (plus a row in the folder README). It marks a stable prefix with prompt_cache_breakpoint under prompt_cache_options: {"mode": "explicit"} and prints cache_read_input_token_count per turn so the cache read is visible. Verified against gpt-5.6-luna: first turn writes (0 cached), the next one reads 1964 cached input tokens.

… clients Add request-level prompt_cache_options to OpenAIChatOptions and OpenAIChatCompletionOptions, and forward a per-part prompt_cache_breakpoint from Content.additional_properties onto the content blocks each API supports. Text parts that carry a breakpoint keep typed list content, since the plain-string form cannot hold one; without a breakpoint the existing string forms are unchanged.

Replace the custom PromptCacheOptions TypedDict with the openai SDK's own types for each API, which raises the openai floor to 2.45.0 where those types were introduced. Make the breakpoint helper private to the two chat clients. Add a prompt caching sample with a README entry, and unquote the helper's Content annotation so the pyupgrade hook passes.

The SDK's PromptCacheOptions types only exist in openai 2.45.0 and later, so each client falls back to a local mirror when the import fails and the dependency floor stays at 2.25.0. A TYPE_CHECKING-only import is not enough because the options classes are introspected with get_type_hints() at runtime. Verified against openai 2.25.0: the package imports, the fallback resolves, and part-level breakpoints still work; sending the option itself requires 2.45.0, which the field docstrings now note.

Assigning None instead, as suggested in review, trips pyright's reportInvalidTypeForm on the field annotation (the symbol becomes type | None after the try/except). An empty TypedDict gives the same effect for users on older openai versions: any content they put in prompt_cache_options is flagged by their type checker, since the option cannot be sent on those versions anyway, while get_type_hints() on the options classes keeps working at runtime.

…k type The empty-TypedDict fallback flagged valid `prompt_cache_options` usage under pyright on every openai version — including this PR's own `client_prompt_caching.py` sample (`poe check -S`) — because pyright resolves the try/except symbol to the fallback shape regardless of the installed openai, while mypy/ty resolve the failed import to `Any` and never warn. So a type-only "warn on old openai" signal is not achievable cleanly across type checkers. Restore the faithful fallback (mirrors the SDK's `mode`/`ttl` shape) so the option type-checks identically on every supported openai version, and add a runtime guard: setting `prompt_cache_options` on openai < 2.45 now raises a clear ChatClientInvalidRequestException instead of forwarding an unusable option to the SDK. This keeps the option non-silent for all users regardless of type checker, without forcing an openai upgrade. Adds tests covering the guard for both clients.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

The system/developer branch switched to list-form content whenever prompt_cache_breakpoint was set to any non-None value, but the option is only attached when the value is a mapping. A malformed value (e.g. a string) therefore changed the message shape without adding a breakpoint. Decide the shape from the built part instead, matching the user-role path.

Copilot AI review requested due to automatic review settings July 17, 2026 05:58

Giles Odigwe (giles17) added the python Usage: [Issues, PRs], Target: Python label Jul 17, 2026

Copilot started reviewing on behalf of Yunus Emre Gültepe (Mordris) July 17, 2026 05:58 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

Comment thread python/packages/openai/agent_framework_openai/_chat_completion_client.py Outdated

Eduard van Valkenburg (eavanvalkenburg) reviewed Jul 17, 2026

View reviewed changes

Comment thread python/packages/openai/agent_framework_openai/_shared.py Outdated

Comment thread python/packages/openai/agent_framework_openai/_shared.py Outdated

Giles Odigwe (giles17) added the documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs label Jul 17, 2026

Yunus Emre Gültepe (Mordris) requested a review from Eduard van Valkenburg (eavanvalkenburg) July 17, 2026 15:02

Eduard van Valkenburg (eavanvalkenburg) reviewed Jul 20, 2026

View reviewed changes

Comment thread python/packages/openai/agent_framework_openai/_chat_client.py Outdated

Yunus Emre Gültepe (Mordris) requested a review from Eduard van Valkenburg (eavanvalkenburg) July 20, 2026 13:49

Eduard van Valkenburg (eavanvalkenburg) reviewed Jul 21, 2026

View reviewed changes

Comment thread python/packages/openai/agent_framework_openai/_chat_completion_client.py

Yunus Emre Gültepe (Mordris) force-pushed the feat/openai-prompt-cache-breakpoints-7157 branch from 50f9932 to eafcc86 Compare July 22, 2026 06:54

Yunus Emre Gültepe (Mordris) temporarily deployed to github-app-auth July 22, 2026 06:54 — with GitHub Actions Inactive

Yunus Emre Gültepe (Mordris) force-pushed the feat/openai-prompt-cache-breakpoints-7157 branch from eafcc86 to 33863fa Compare July 22, 2026 07:11

Yunus Emre Gültepe (Mordris) temporarily deployed to github-app-auth July 22, 2026 07:11 — with GitHub Actions Inactive

Yunus Emre Gültepe (Mordris) and others added 6 commits July 22, 2026 10:22

Clarify system-message content-shape comment

4f9c14f

Yunus Emre Gültepe (Mordris) force-pushed the feat/openai-prompt-cache-breakpoints-7157 branch from 33863fa to 69f8e10 Compare July 22, 2026 07:22

Yunus Emre Gültepe (Mordris) temporarily deployed to github-app-auth July 22, 2026 07:23 — with GitHub Actions Inactive

Yunus Emre Gültepe (Mordris) requested review from Copilot and Eduard van Valkenburg (eavanvalkenburg) July 22, 2026 07:23

Copilot started reviewing on behalf of Yunus Emre Gültepe (Mordris) July 22, 2026 07:24 View session

Copilot AI reviewed Jul 22, 2026

View reviewed changes

Comment thread python/packages/openai/agent_framework_openai/_chat_completion_client.py Outdated

Python: Support prompt cache breakpoints for GPT-5.6 models in OpenAI clients by Mordris · Pull Request #7163 · microsoft/agent-framework · GitHub

Repository navigation

Python: Support prompt cache breakpoints for GPT-5.6 models in OpenAI clients - #7163

Python: Support prompt cache breakpoints for GPT-5.6 models in OpenAI clients#7163
Eduard van Valkenburg (eavanvalkenburg) merged 7 commits into
microsoft:mainfrom
Mordris:feat/openai-prompt-cache-breakpoints-7157

Yunus Emre Gültepe (Mordris) commented Jul 17, 2026 •
edited
Loading

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Yunus Emre Gültepe (Mordris) commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026 •
edited
Loading

Uh oh!

Uh oh!

Eduard van Valkenburg (eavanvalkenburg) left a comment

Uh oh!

Uh oh!

Uh oh!

Yunus Emre Gültepe (Mordris) commented Jul 17, 2026

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Python: Support prompt cache breakpoints for GPT-5.6 models in OpenAI clients by Mordris · Pull Request #7163 · microsoft/agent-framework · GitHub

Repository navigation

Python: Support prompt cache breakpoints for GPT-5.6 models in OpenAI clients - #7163

Conversation

Yunus Emre Gültepe (Mordris) commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation & Context

Description & Review Guide

Related Issue

Contribution Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Yunus Emre Gültepe (Mordris) commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Python Unit Test Overview

Uh oh!

Eduard van Valkenburg (eavanvalkenburg) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Yunus Emre Gültepe (Mordris) commented Jul 17, 2026

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Yunus Emre Gültepe (Mordris) commented Jul 17, 2026 •
edited
Loading

github-actions Bot commented Jul 17, 2026 •
edited
Loading