Skip to content

fix(docs-sync): raise output token cap to 16384 for Sonnet 5.5 - #2071

Merged
soheimam merged 1 commit into
masterfrom
fix/docs-sync-token-limit
Oct 5, 2026
Merged

soheimam merged 1 commit into
masterfrom
fix/docs-sync-token-limit

Conversation

@soheimam

@soheimam soheimam commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Why

#2068 switched the docs sync to Sonnet 5.5. On the first real run (run 37331981837), the re-sync of base-std@3820cf0 dropped a page: seize-and-cancel-units.mdx (8,955 chars) hit the 4096-token output cap (stop_reason=max_tokens), and the truncation guard rejected it. That's why #2069 touches 8 files where #2054 touched 10. introduction.mdx used 4,093 of 4,096 tokens, so it nearly failed too.

Sonnet 5.5's tokenizer uses more tokens per char than 4.6, so the old 4096 budget (measured on 4.6 at 10.7k–14k chars) no longer fits pages the route table sends to the model.

Changes

  • llm/client.mjs: DEFAULT_MAX_TOKENS goes from 4096 to 16384. CLAUDE_MAX_TOKENS still overrides it.
  • release-utils.mjs: MAX_REGENERABLE_CHARS goes from 10000 to 32000 to match. This is the pre-call skip for pages too large to regenerate (~2.2 chars/token measured on Sonnet 5.5, with margin).
  • index.mjs: doc comment updated.

The truncation guard is unchanged. If a page still overflows, it is rejected rather than committed partially.

Cost and risk

  • Output is billed per generated token, not per cap. Short pages cost the same.
  • Calls are streamed, so a longer generation doesn't hit the gateway's ~90s buffered timeout.
  • I confirmed that the gateway accepts max_tokens: 16384 for claude-sonnet-5-5.
  • No workflow, permission, or secret changes.

Tests

npm --prefix scripts run test:base-std-sync: 166 pass, 1 fail. The failure (upstream docs tree routes to the pages the IA guidelines assign, an extra events.mdx route) also fails on master before this change and is unrelated.

Follow-up (separate PR)

Re-running a sync briefly resets its branch to master, which makes GitHub auto-close the open PR (that's how #2054 was closed and replaced by #2069). The fix is to commit on a temporary ref and move the PR branch in one step.

Each page call returns the full regenerated page. Under Sonnet 5.5 the
4096-token cap truncated seize-and-cancel-units.mdx (8,955 chars), so the
page was rejected and dropped from the sync PR (#2069 vs #2054).

- DEFAULT_MAX_TOKENS: 4096 -> 16384 (CLAUDE_MAX_TOKENS still overrides)
- MAX_REGENERABLE_CHARS: 10000 -> 32000 to match the new budget
  (~2.2 chars/token measured on Sonnet 5.5)

Co-authored-by: Toshi <toshi-noreply@coinbase.com>
@cb-heimdall

cb-heimdall commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Heimdall Review Status

Requirement Status More Info
Reviews ✅ 1/1
Denominator calculation
Show calculation
1 if user is bot 0
1 if user is external 0
2 if repo is sensitive 0
From .codeflow.yml 1
Additional review requirements
Show calculation
Max 0
0
From CODEOWNERS 0
Global minimum 0
Max 1
1
1 if commit is unverified 0
Sum 1

@soheimam
soheimam merged commit 5ceae24 into master Oct 5, 2026
13 checks passed
@soheimam
soheimam deleted the fix/docs-sync-token-limit branch October 5, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants