Fix prompt cache defeats: strip billing headers from instructions, use stable prompt_cache_key by Copilot · Pull Request #62 · CaddyGlow/ccproxy-api · GitHub

Every Anthropic→Codex request turn was a guaranteed cache miss due to two independent bugs: a per-request cch=<hash> in Claude Code's x-anthropic-billing-header system prefix producing unique instructions on every turn, and _apply_request_template overwriting any template-configured prompt_cache_key with a fresh uuid.uuid4().

Changes

ccproxy/llms/formatters/anthropic_to_openai/requests.py

Added _strip_nonsemantic_system_lines() which filters out x-anthropic-billing-header: lines (case-insensitive) before system text is forwarded as instructions
Applied to both the string and block array system paths

# Before: billing header forwarded verbatim → unique instructions every turn
joined = "".join(block.text for block in request.system if block.text)

# After: header stripped → byte-identical instructions across turns
cleaned_blocks = (
    _strip_nonsemantic_system_lines(block.text or "")
    for block in request.system if block.text
)
joined = "\n\n".join(b for b in cleaned_blocks if b)

ccproxy/plugins/codex/adapter.py

_apply_request_template now forwards the template's prompt_cache_key verbatim instead of generating a new UUID — caller-supplied keys are still preserved unchanged

# Before
merged["prompt_cache_key"] = str(uuid.uuid4())   # fresh UUID → always a cache miss

# After
merged["prompt_cache_key"] = prompt_cache_key     # stable → cache can hit

Tests

Updated test_prepare_provider_request_applies_codex_template_defaults to assert the corrected behavior (== "template-cache-key")
Added 5 formatter tests covering string/block system paths, case-insensitivity, header-only system, and cross-request stability
Added 2 adapter tests for stable key forwarding and caller-key precedence

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

astral.sh
- Triggering command: /usr/bin/curl curl -LsSf REDACTED (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

…ache_key Agent-Logs-Url: https://github.com/CaddyGlow/ccproxy-api/sessions/392aa95e-bd93-4b16-a9d7-8b7829631987 Co-authored-by: CaddyGlow <219252898+CaddyGlow@users.noreply.github.com>

Pull request overview

Fixes two independent causes of prompt cache misses in the Anthropic→OpenAI/Codex path by making both the forwarded system instructions and the prompt_cache_key deterministic across turns.

Changes:

Strip non-semantic x-anthropic-billing-header: lines from Anthropic system content before mapping to OpenAI instructions (string + block-list paths).
Preserve template-provided prompt_cache_key in the Codex adapter instead of replacing it with a fresh UUID (while still honoring caller-supplied keys).
Add/adjust unit tests to assert stable instructions and stable prompt_cache_key behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
ccproxy/llms/formatters/anthropic_to_openai/requests.py	Adds a helper to remove billing-header lines and applies it when building OpenAI Responses instructions.
ccproxy/plugins/codex/adapter.py	Stops randomizing prompt_cache_key when it comes from the request template.
tests/unit/llms/formatters/test_anthropic_to_openai_helpers.py	Adds tests covering header stripping behavior and instruction stability across requests.
tests/plugins/codex/unit/test_adapter.py	Updates/extends tests to verify stable template prompt_cache_key forwarding and caller precedence.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…ent (#63) Apply `_strip_nonsemantic_system_lines()` in the Anthropic->OpenAI Chat request converter so the per-request `x-anthropic-billing-header` (with the regenerating `cch=<hash>`) does not defeat upstream prompt caching on the chat-completions path. Mirrors the fix landed for the Responses API converter in #62. If the cleaned system content is empty, the system message is dropped entirely instead of injecting an empty system role. Adds 5 chat-converter tests mirroring the Responses-API coverage: string-system stripping, block-system stripping, case-insensitive matching, header-only-system drops the message, and cross-request stability when only the `cch=` hash differs.

Initial plan

950475f

Copilot AI assigned Copilot and CaddyGlow May 2, 2026

Copilot started work on behalf of CaddyGlow May 2, 2026 18:41 View session

Copilot AI linked an issue May 2, 2026 that may be closed by this pull request

System prompt forwarded with per-request x-anthropic-billing-header line — defeats upstream prompt cache #61

Closed

Fix prompt cache issues: strip billing headers and stabilize prompt_c…

ed21951

…ache_key Agent-Logs-Url: https://github.com/CaddyGlow/ccproxy-api/sessions/392aa95e-bd93-4b16-a9d7-8b7829631987 Co-authored-by: CaddyGlow <219252898+CaddyGlow@users.noreply.github.com>

Copilot AI changed the title [WIP] Fix issue with prompt cache in OpenAI Responses API payload conversion Fix prompt cache defeats: strip billing headers from instructions, use stable prompt_cache_key May 2, 2026

Copilot finished work on behalf of CaddyGlow May 2, 2026 18:48

Copilot AI requested a review from CaddyGlow May 2, 2026 18:48

CaddyGlow approved these changes May 2, 2026

View reviewed changes

CaddyGlow marked this pull request as ready for review May 2, 2026 18:56

Copilot AI review requested due to automatic review settings May 2, 2026 18:56

CaddyGlow merged commit 7f973ab into main May 2, 2026
14 of 15 checks passed

Copilot started reviewing on behalf of CaddyGlow May 2, 2026 18:57 View session

Copilot AI reviewed May 2, 2026

View reviewed changes

Comment thread ccproxy/llms/formatters/anthropic_to_openai/requests.py

if not line.strip().lower().startswith(_NONSEMANTIC_SYSTEM_PREFIXES)

).strip()

CaddyGlow mentioned this pull request May 2, 2026

fix(formatters): strip billing header from chat converter system content #63

Merged

4 tasks

Fix prompt cache defeats: strip billing headers from instructions, use stable prompt_cache_key by Copilot · Pull Request #62 · CaddyGlow/ccproxy-api · GitHub

Fix prompt cache defeats: strip billing headers from instructions, use stable prompt_cache_key - #62

Fix prompt cache defeats: strip billing headers from instructions, use stable prompt_cache_key#62
CaddyGlow merged 2 commits into
mainfrom
copilot/fix-prompt-cache-issue

Copilot AI commented May 2, 2026 •
edited
Loading

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		if not line.strip().lower().startswith(_NONSEMANTIC_SYSTEM_PREFIXES)
		).strip()

Fix prompt cache defeats: strip billing headers from instructions, use stable prompt_cache_key by Copilot · Pull Request #62 · CaddyGlow/ccproxy-api · GitHub

Fix prompt cache defeats: strip billing headers from instructions, use stable prompt_cache_key - #62

Fix prompt cache defeats: strip billing headers from instructions, use stable prompt_cache_key#62CaddyGlow merged 2 commits intomainCaddyGlow/ccproxy-api:mainfrom copilot/fix-prompt-cache-issueCaddyGlow/ccproxy-api:copilot/fix-prompt-cache-issueCopy head branch name to clipboard

Conversation

Copilot AI commented May 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

ccproxy/llms/formatters/anthropic_to_openai/requests.py

ccproxy/plugins/codex/adapter.py

Tests

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Fix prompt cache defeats: strip billing headers from instructions, use stable prompt_cache_key#62
CaddyGlow merged 2 commits into
mainfrom
copilot/fix-prompt-cache-issue

Copilot AI commented May 2, 2026 •
edited
Loading