Context Overflow Recovery
When the API returns a “context too long” error, ChatCLI applies up to 3 recovery levels before giving up:- Level 1: Aggressive Budget
- Level 2: Emergency Truncation
- Level 3: Nuclear Truncation
First attempt: halves the budget limits and cleans up misalignments.Actions:
- Repairs tool result pairing (removes orphans, injects synthetics)
- Reduces
DefaultTurnBudgetCharsandDefaultPerResultMaxCharsto 50% of their original values - Applies budget enforcement with reduced limits
- Truncates long assistant messages to 5,000 chars
The original limits are restored after application. Only the current history is affected by the reduction.
Error Detection — model overflow
The system recognizes multiple forms of overflow errors:Corporate proxy / gateway recovery
Enterprise environments often sit behind a proxy or gateway that enforces a POST body size cap — typically 1-5 MB, completely independent of the model’s context window. You can be well within Anthropic’s 200K-token window (~800 KB) and still take a mysterious rejection from the proxy. Worse: many proxies don’t return a clean 413 — some send a WAF 403 (Cloudflare, Akamai, mod_security), 431 (header too large), or simply drop the TCP connection mid-POST, surfacing asEOF / connection reset on the client.
ChatCLI detects all three patterns and funnels them through the same recovery flow as context overflow.
Error Detection — proxy/gateway
WAF detection is conservative — a 403 without firewall signals continues to be treated as an auth error (OAuth refresh + retry). Only when a 403 carries specific proxy/WAF signals is it reclassified as a recoverable payload failure. This prevents invalidating valid OAuth credentials when the real problem is on the network layer.
The corporate Bedrock case: when the proxy/WAF intercepts the POST to Bedrock Runtime and returns an HTML block page with status 403, the AWS SDK tries to parse the body as JSON and fails with
"invalid character '<' looking for beginning of value" and an empty RequestID. That pattern is an unambiguous middlebox fingerprint (a real AWS 403 returns well-formed JSON) — ChatCLI reclassifies it as a recoverable payload failure and triggers the same recovery ladder.EOF / connection-reset detection applies a history-size threshold (500 KB) before suspecting payload. Small requests that hit EOF keep being treated as transient network failures (normal retry). Only when the history is already suspiciously large is EOF reclassified as a probable body cap.
Pre-flight check
Every agent turn, history is measured before the request goes out. Two paths: WithCHATCLI_MAX_PAYLOAD set:
If history crosses 85% of the cap, BudgetRatio is forced to 0.40 up front — aggressive preventive compaction. The user sees:
Adaptive learned cap
Bedrock providers annotate every transport error with the exact size of the request the middlebox rejected. When a payload rejection fires, ChatCLI derives the session cap directly from that observation — ¾ of the rejected size — instead of guessing:CHATCLI_MAX_PAYLOAD is already set below it, your value wins; a later, smaller rejection tightens it further.
When the rejected size is unknown (providers without size annotation), the previous behavior remains as fallback: assume 4 MB for the rest of the session.
Floor diagnosis — when compaction cannot help
The system prompt (agent charter, personas, skills, MCP tool docs) is never compacted. When it alone reaches the size the gateway just rejected, no amount of history compaction can produce an acceptable request — retrying identical payloads would only burn recovery attempts. ChatCLI detects this and fails fast with an actionable message:Content shrinking — Level 3 that actually works
Whole-message dropping is a no-op when the history is short — agent sessions often hold just a system prompt plus a handful of huge tool results, andMinKeepRecent keeps exactly the messages that carry the bulk. Emergency truncation therefore has a final pass: it shrinks the content of non-system messages, largest first, until the payload budget is met. System messages are never touched.
When the compression layer is active, each message is archived verbatim in the CCR store before its first cut and the shrunk content carries a <<ccr:KEY>> marker — the model can recover the original at any time with @recall. Nuclear truncation (level 3 of the recovery ladder) additionally hard-caps every kept non-system message at 4,000 chars.
System notice injected in history
After a payload-limit-triggered recovery, ChatCLI injects auser message before the retry instructing the model to prefer smaller reads going forward. This breaks the model’s loop of trying to re-read the same huge file that caused the 413 in the first place. The notice is injected at most once per session — recovery detects an existing copy anywhere in the history (even folded into a compaction summary) and never stacks a second one, since every extra byte works against the very limit being recovered from:
Max Output Token Escalation
When the model stops generating because it hit themax_tokens limit, ChatCLI can automatically escalate:
Continuation Message
When the model is interrupted by a token limit, ChatCLI injects a continuation message:Configuration
Live feedback during compaction
Since this release, the terminal never “freezes” during a long compaction anymore.HistoryCompactor emits status at each pipeline phase via SetStatusCallback:
Ctrl+C / ESC propagates correctly and aborts compaction without corrupting history (returns ctx.Err() instead of blindly falling through to emergency truncation).
Microcompact (pre-budget)
BeforeNeedsCompaction checks whether history exceeds budget, the agent loop applies ApplyMicrocompact — a pure-Go, no-LLM, no-network pass that progressively truncates/summarizes old tool results (2+ turns old → head+tail preview; 4+ turns old → one-line summary). In most cases this keeps history inside budget without triggering the (expensive) Level 2.
With the compression layer active, microcompact is lossless: the original tool result is archived in the CCR store before the cut and the preview/summary stub embeds a <<ccr:KEY>> marker. Markers carry forward across levels — when a truncated preview later degrades to a one-line summary, the marker survives on the summary — so the model can always expand the original with @recall.
Aggressive Budget Ratio
At level 1, the tool result budget limits are multiplied by0.5 (50%). This means:
Recovery Flow
Interaction with Other Systems
Context recovery works in conjunction with:Tool Result Budget
The result budget is the first line of defense. Recovery activates when the budget was not sufficient.
Microcompaction
Progressive compaction reduces context growth over time.
Conversation Control
The
/compact command is the proactive way to prevent overflow.Cost Tracking
Monitor context usage to anticipate when /compact will be needed.