fix(ingest): read Spring AI's Anthropic and Bedrock Converse usage as cache-exclusive

[JeremyFunk]

Problem

Spring AI's native Anthropic client (AnthropicChatModel.getDefaultUsage and its streaming path) and its Bedrock Converse client (BedrockProxyChatModel) report gen_ai.usage.input_tokens without the cache, as the Anthropic Messages and Converse APIs do. The cache comes separately as gen_ai.usage.cache_read.input_tokens and gen_ai.usage.cache_creation.input_tokens.

input_excludes_cache() in apps/ingest/src/ai_session/usage.rs had no spring_ai arm, so ingest read these spans as cache-inclusive and subtracted the cache from the prompt. The cache > prompt guard only catches prompts smaller than their cache; once uncached input is at least as large as the cache, the session total came out too low.

Evidence from EU prod, using synthetic spans built from the Spring AI v2.0.1 source (gen_ai.system=anthropic, op chat, scope org.springframework.boot):

Session	Sent (uncached / cache read / cache write / out)	Maple total before	Real total
verify3-b-edge-conv	5000 / 4000 / 1000 / 200	5200	10200
verify3-b-bigprompt-conv	6000 / 1000 / 500 / 200	6200	7700
verify3-b-smallprompt-conv	300 / 4000 / 1000 / 200	5500 (correct, via the guard)	5500

Fix

Add a spring_ai arm: the prompt excludes the cache when gen_ai.system is anthropic or bedrock_converse. In Spring AI this attribute is set per ChatModel class (AiProvider enum), so it names the client that made the call, not the model's vendor. The OpenAI client, which is also what Spring AI apps use for OpenRouter, reports openai and stays inclusive.

Spring AI writes only gen_ai.system (AiObservationAttributes.AI_PROVIDER), not gen_ai.provider.name, in both v2.0.1 and main, so only that key is read. The lookup runs only for Spring AI model-call spans.

Tests

spring_ai_anthropic_and_converse_are_cache_exclusive in usage.rs covers the three sessions above, a Bedrock Converse span, and a Spring AI OpenAI-client span that stays inclusive. cargo test --lib ai_session: 83 passed.

Note

Spring AI 1.1.x's Anthropic client reports no cache attributes at all, so the cache is invisible on those spans and ingest cannot recover it.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

Summary by CodeRabbit

• Bug Fixes
  • Improved AI usage reporting for Spring AI: Anthropic and Bedrock Converse now exclude cached input from prompt totals, while OpenAI continues to include it.
  • Updated prompt usage calculations for other supported AI integrations to use the standard inclusive calculation, with the existing fallback applied when cached input exceeds prompt input.

[maple-review-bot]

Maple review

🟢 Confidence 4/5 · likely safe to merge
One match arm on gen_ai.system, guarded, with the three prod sessions and the inclusive OpenAI case tested; the Spring AI enum strings themselves are the author's reading of its source.
_{quality 100/100 · no findings · tests covered · risk medium}

Adds a spring_ai arm to the ingest cache-exclusivity rule, read per span from gen_ai.system, so Spring AI's Anthropic and Bedrock Converse usage stops losing the cache from the prompt. Contained and correct; safe to merge.

• input_excludes_cache takes the Span and branches on gen_ai.system for spring_ai
• Spring AI anthropic/bedrock_converse spans read cache-exclusive; openai stays inclusive
• New test spring_ai_anthropic_and_converse_are_cache_exclusive covers the three prod sessions

What was checked

• value_str returns "" for a non-string value, so the new matches! cannot panic (ai_session.rs:492)
• The cache > prompt guard is kept (usage.rs:291), so a spring_ai span without gen_ai.system is read as before
• gen_ai.system stays on the span until stamping: the read is per span, so one spring_ai trace can mix clients

_{163c28a · Updated on every push. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.}

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 056f8b2c-5fe7-4d40-bdba-d507efad7635

📥 Commits

Reviewing files that changed from the base of the PR and between 2e91012 and 163c28a.

📒 Files selected for processing (1)

• apps/ingest/src/ai_session/usage.rs

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 1 remain after this review.

---

📝 Walkthrough

Walkthrough

Cache-exclusivity selection now checks span attributes for Spring AI. The previous model-name-based rules for Strands, Google ADK, Agno, and Microsoft Agent Framework are removed. Tests cover Spring AI Anthropic, Bedrock Converse, and OpenAI usage.

Changes

AI usage cache accounting

Layer / File(s)	Summary
Span-based cache selection apps/ingest/src/ai_session/usage.rs	Usage::read passes the span to input_excludes_cache. The function treats Spring AI Anthropic and Bedrock Converse input as cache-exclusive. Spring AI OpenAI input remains inclusive. The prior model-name rules for other integrations are removed. Tests cover these cases and the existing prompt-smaller-than-cache fallback.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 163c2

This change fixes how cache tokens are counted for Spring AI Anthropic and Bedrock Converse spans. It is limited to ingest usage accounting and no merge-blocking risk was found.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: handling Spring AI Anthropic and Bedrock Converse usage as cache-exclusive.
Docstring Coverage	✅ Passed	Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 1 files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

Warning

Some tools did not complete. Review the errors below.

🔧 Clippy (1.98.1)

Clippy execution failed

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}