Skip to content

fix(controlplane): cap the policy evaluations inlined by the workflow-run View API - #3408

Merged
javirln merged 2 commits into
chainloop-dev:mainfrom
javirln:javier/pfm-7045-134k-violation-attestations-oom-the-backend-across-the
Sep 8, 2026
Merged

fix(controlplane): cap the policy evaluations inlined by the workflow-run View API#3408
javirln merged 2 commits into
chainloop-dev:mainfrom
javirln:javier/pfm-7045-134k-violation-attestations-oom-the-backend-across-the

Conversation

@javirln

@javirln javirln commented Sep 7, 2026

Copy link
Copy Markdown
Member

Context

Related to PFM-7045.

A single attestation can carry a six-figure number of policy violations. When the workflow-run View API inlined one of those, it held the payload in memory several times over — the CAS download buffer, the decoded bundle, the regrouped evaluations, and the response protos — which was enough to exhaust the control plane.

What changed

The View API now asks the CAS for the policy-evaluation bundle size before downloading it. Bundles larger than a configurable cap are never downloaded: the response carries a reference with the digest, size and reason instead of the evaluations themselves. Bundles under the cap behave exactly as before.

The same reference is returned when the bundle cannot be resolved or decoded, so no failure path falls back to an unbounded download. Where a bundle is already cached, it is still served without any CAS round trip.

Status and counters continue to be derived from the attestation predicate rather than from the evaluations, so policy status, gating, bypass and violation counts stay complete regardless of whether the evaluations were inlined.

New pieces:

  • AttestationItem.policy_evaluations_ref, an additive field carrying digest, size, media type and the reason the evaluations were left out. Clients that ignore it see an empty evaluations map, which is already how an unresolvable bundle behaved.
  • Attestations.policy_evaluations_max_inline_bytes in the control-plane config, defaulting to 10MiB.
  • Describe on the control plane's CAS downloader, wrapping the metadata endpoint the CAS already exposed.

In the CLI, workflow run describe renders the policy counters and the chainloop artifact download command to fetch the bundle in place of the policy table, and exposes the reference in its JSON output. It also surfaces the canonical policy status summary the server has been sending but the CLI never displayed.

Attestation crafting and push are unchanged: the bundle format, the CAS upload and the signed predicate are all untouched. This is a read-side change only.

Scope

Two related paths are deliberately left alone. Attestations whose evaluations live inline in the predicate — runs on an inline CAS backend, and anything pushed before the CAS offload — carry no reference to gate on and remain uncapped. The repeated predicate extraction on the ingest path is also unchanged. Both belong to the wider PFM-7045 work.

AI disclosure

This contribution was produced with AI assistance (Claude Code). The affected commit carries an Assisted-by: Claude Code trailer.

Review in cubic

…-run View API

A single attestation can carry a six-figure number of policy violations.
Inlining one in a View response held the payload in memory several times
over -- CAS download buffer, decoded bundle, regrouped evaluations and
response protos -- which was enough to exhaust the control plane.

The View API now asks the CAS for the bundle size before downloading it.
Bundles above a configurable cap, defaulting to 10MiB, are returned as a
reference carrying the digest, size and reason instead of the evaluations
themselves, and are never downloaded. The same reference is returned when
the bundle cannot be resolved or decoded, so no failure path falls back to
an unbounded download. Status and counters continue to be served from the
predicate, so gating, bypass and violation counts stay complete regardless.

The CLI renders the counters plus the artifact download command in place of
the policy table, and exposes the reference in its JSON output. Attestation
crafting and push are unchanged.

Assisted-by: Claude Code
Signed-off-by: Javier Rodriguez <javier@chainloop.dev>

Chainloop-Trace-Sessions: d400245f-b8d2-4805-998e-b23f9be4a7fb
@chainloop-platform

chainloop-platform Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

AI Session Checks — 🟢 92% · ✅ 0 failing

Avg score Sessions Failing policies Attribution Files Lines Total Duration
🟢 92% 1 ✅ 0 98% AI / 2% Human 15 +1640 / -137 1h17m18s

🟢 92% — 98% AI — ✅ All policies passing

Sep 7, 2026 09:05 UTC · 1h17m18s · $53.85 · 764 in / 322.0k out · claude-code 2.1.263 (claude-opus-5)

View session details ↗

Change Summary

  • Adds a size cap for inlined policy evaluations in workflow-run view/describe.
  • Adds policy_evaluations_ref plus CLI download guidance when bundles stay out of line.
  • Adds CAS Describe plumbing, a higher default cap, and guards for zero or under-reported sizes.
  • Extends resolver and CLI tests, then verifies with builds, suites, lint, and mutation checks.

AI Session Overall Score

🟢 92% — Strong planning and repeated verification; no significant execution risks surfaced.

AI Session Analysis Breakdown

🟢 95% · context-and-planning

🟢 The task was designed and approved through a written spec before coding. · High Impact

🟢 93% · verification

🟢 AI ran targeted tests, builds, lint, and mutation checks before shipping. · High Impact

🟢 92% · alignment

No notes.

🟢 91% · solution-quality

No notes.

🟢 90% · user-trust-signal

No notes.

🟢 88% · scope-discipline

🟢 AI reverted unrelated mock-generation churn instead of widening the final change. · Medium Impact


File Attribution

███████████████████░ 98% AI / 2% Human

Status Attribution File Lines
modified ai pkg/casclient/mocks/DownloaderUploader.go +310 / -54
modified ai app/controlplane/internal/service/workflowrun_test.go +281 / -2
modified ai app/controlplane/pkg/biz/mocks/CASClient.go +254 / -25
modified ai pkg/casclient/mocks/Downloader.go +192 / -29
modified ai app/controlplane/internal/service/workflowrun.go +163 / -18
modified ai app/cli/cmd/workflow_workflow_run_describe_test.go +146 / -1
modified ai app/cli/pkg/action/workflow_run_describe.go +73 / -1
modified ai app/cli/cmd/workflow_workflow_run_describe.go +68 / -5
modified ai app/controlplane/pkg/biz/casclient_test.go +72 / -1
modified human app/controlplane/api/controlplane/v1/response_messages.proto +27 / -0
modified ai app/controlplane/pkg/biz/casclient.go +23 / -0
created ai pkg/casclient/.mockery.yml +18 / -0
modified human app/controlplane/internal/conf/controlplane/config/v1/conf.proto +8 / -0
modified ai pkg/casclient/casclient.go +4 / -1
modified ai app/controlplane/pkg/biz/.mockery.yml +1 / -0

Policies (4)

Status Policy Material Messages
✅ Passed ai-config-ai-agents-allowed ai-coding-session-d40024 -
✅ Passed ai-config-no-dangerous-commands ai-coding-session-d40024 -
✅ Passed ai-config-no-secrets ai-coding-session-d40024 -
✅ Passed ai-config-mcp-servers-allowed ai-coding-session-d40024 -

Security Checks — ✅ 8 passing

secret-scan

Status Policy Messages
✅ Passed secrets-detection -

sast-scan

Status Policy Messages
✅ Passed sast -
✅ Passed cwe-top25 -
✅ Passed cwe-top26-40-cusp -
✅ Passed owasp-top10-2025 -

iac-scan

Status Policy Messages
✅ Passed iac-misconfiguration -

PR info

Status Policy Messages
✅ Passed pr-description-required -
✅ Passed pr-user-story-linked -

⏭️ 2 scans not applied

Scan Reason
vulnerability-scan no manifest/lockfile changed
github-actions-scan no workflow files changed

View attestation ↗


Powered by Chainloop and Chainloop Trace

@javirln
javirln marked this pull request as ready for review September 7, 2026 10:11
@javirln
javirln requested a review from a team September 7, 2026 10:11

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 23 files

Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic

Comment thread app/controlplane/internal/service/workflowrun.go
…eived

The size check trusted the CAS backend's metadata in two ways it should not
have. A reported size of zero fell through the cap comparison and started an
unbounded download: some backends omit the content length and the proto
getter then yields zero, so zero means unknown rather than empty. And a
correct-looking size was never reconciled with the bytes actually received,
so an under-reporting backend could still exhaust the cap.

Zero-size responses are now reported as unavailable without a download, and
the download target is a writer that refuses to grow past the cap and records
that it did, so the bound holds against the bytes received rather than the
size claimed.

Assisted-by: Claude Code
Signed-off-by: Javier Rodriguez <javier@chainloop.dev>

Chainloop-Trace-Sessions: d400245f-b8d2-4805-998e-b23f9be4a7fb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants