feat(compaction): configurable threshold and model (global + per-model) - #11931
feat(compaction): configurable threshold and model (global + per-model)#11931tobiastornros wants to merge 2 commits into
Conversation
Sorry, something went wrong.
…model) Add compaction settings at both global and per-model levels: - threshold: trigger compaction at a percentage of context (1-100, default: 100) - model: use a different model for compaction summaries This allows users to: - Trigger compaction earlier for models that degrade at lower context usage - Use cheaper/faster models for compaction to save costs Closes anomalyco#11930 Relates to anomalyco#11314, anomalyco#8140, anomalyco#10017, anomalyco#8629
|
The following comment was made by an LLM, it may be inaccurate: Related PRs FoundPotential duplicates/related work:
These PRs suggest there's been ongoing work on compaction and context threshold features. PR #10123 appears most similar, implementing custom compaction thresholds. You may want to check if #11931 supersedes or builds upon any of these, particularly #10123. |
Sorry, something went wrong.
|
Future idea: we could add compaction_max_tokens on model level too, an absolute token cap regardless of percentage. Useful for cost control or latency-sensitive setups. Kept it simple for now, can extend later. |
Sorry, something went wrong.
|
Closing this pull request because it has had no updates for more than 60 days. If you plan to continue working on it, feel free to reopen or open a new PR. |
Sorry, something went wrong.
Ever noticed how Opus gets kinda dumb around 50-60% context? Yeah, me too. Built this to escape the dumbzone.
Adds compaction settings at both global and per-model levels:
Global config:
{ "compaction": { "threshold": 80, "model": "anthropic/claude-haiku-5-20260205" } }Per-model config:
{ "provider": { "anthropic": { "models": { "claude-opus-4-20250514": { "compaction_threshold": 50, "compaction_model": "zhipu/glm-4-plus" } } } } }Per-model overrides global. Default is 100 (current behavior).
Closes #11930
Relates to #11314, #8140, #10017, #8629