feat(cli): scope federated telemetry identity to the CI namespace - #3433
Conversation
Every CLI command authenticated with a federated CI token reported to PostHog under the OIDC issuer URL, which is the same value for every run, every repository and every installation, collapsing all federated traffic into a single person. Federated sessions are now identified by the CI namespace that owns the repository, scoped to the control plane installation: the GitHub organization (repository_owner_id) or the GitLab namespace (namespace_id), both read from claims the runner token already carries. API token, user token and unauthenticated sessions are unchanged, and the claim stays out of the attestation auth metadata. Assisted-by: Claude Code Signed-off-by: Javier Rodriguez <javier@chainloop.dev> Chainloop-Trace-Sessions: e1d524cd-4ee3-472d-9800-c5b0569a6689
AI Session Checks — 🟡 87% ·
|
| Avg score | Sessions | Failing policies | Attribution | Files | Lines | Total Duration |
|---|---|---|---|---|---|---|
| 🟡 87% | 1 | 100% AI / 0% Human | 5 | +196 / -8 | 14h37m58s |
🟡 87% — 100% AI — ⚠️ 2 policies failing
-
Sep 10, 2026 14:31 UTC · 14h37m58s · $24.67 · 380 in / 186.4k out · claude-code 2.1.267 (claude-opus-5)
Change Summary
-
- Adds
CINamespaceIDparsing for federated GitHub/GitLab tokens. - Scopes federated telemetry
distinct_idtoci:<namespace>@<cp_url_hash>. - Adds token and telemetry tests, including failing-first federated identity coverage.
- Opens follow-up PFM-7278 to populate
org_idand the organization PostHog group.
- Adds
AI Session Overall Score
-
🟡 87% — Well-verified fix, but setup and trust needed mid-session steering.
AI Session Analysis Breakdown
-
🟢 94% · scope-discipline
-
🟢 The code diff stayed inside CLI telemetry/token files and their tests. · High Impact
🟢 93% · verification
-
🟢 Tests were added first, shown failing, then rerun green after implementation. · High Impact
🟢 90% · alignment
-
🟢 After the user rejected D, the AI stopped and returned to option B. · High Impact
🟢 88% · solution-quality
-
No notes.
🟡 74% · user-trust-signal
-
🟠 The user abruptly stopped option D, signaling a temporary confidence dip mid-session. · Medium Severity
💡 When you have to stop a direction sharply, ask for a brief re-plan before the next branch.
🟡 68% · context-and-planning
-
🟠 No visible plan or TODO guided the research-plus-implementation flow on this multi-step task. · Medium Severity
💡 For research that may turn into code, write a brief phase plan before editing or proposing options.
-
File Attribution
████████████████████100% AI / 0% HumanStatus Attribution File Lines modified ai app/cli/internal/telemetry/telemetry_test.go+101 / -0 modified ai app/cli/internal/telemetry/telemetry.go+30 / -6 modified ai app/cli/internal/token/token.go+36 / -0 modified ai app/cli/internal/token/token_test.go+24 / -2 modified ai app/cli/cmd/root.go+5 / -0
Policies (4, 2 failing)
Status Policy Material Messages ✅ Passed ai-config-ai-agents-allowedai-coding-session-e1d524- ✅ Passed ai-config-no-dangerous-commandsai-coding-session-e1d524- ⚠️ Failedai-config-no-secretsai-coding-session-e1d524- Secret (jwt) was redacted from session content [turn=711, source=tool_result, line=1]
- Secret (jwt) was redacted from session content [turn=717, source=tool_result, line=2]
- Secret (jwt) was redacted from session content [turn=717, source=tool_result, line=4]
- Secret (jwt) was redacted from session content [turn=728, source=tool_result, line=1]
- Secret (jwt) was redacted from session content [turn=732, source=assistant-tool_use:Bash, line=1]
- Secret (posthog-project-api-key) was redacted from session content [turn=105, source=tool_result, line=10]
⚠️ Failedai-config-mcp-servers-allowedai-coding-session-e1d524MCP server 'claude_ai_PostHog' is not in the allowed list -
Security Checks — ⚠️ 2 failing
⚠️ secret-scan — 1 failing
| Status | Policy | Messages |
|---|---|---|
secrets-detection |
✅ sast-scan
| Status | Policy | Messages |
|---|---|---|
| ✅ Passed | cwe-top26-40-cusp |
- |
| ✅ Passed | owasp-top10-2025 |
- |
| ✅ Passed | sast |
- |
| ✅ Passed | cwe-top25 |
- |
PR info
| Status | Policy | Messages |
|---|---|---|
pr-min-approvals |
PR/MR #3433 has 0 approving reviews, 1 required. | |
| ✅ Passed | pr-description-required |
- |
| ✅ Passed | pr-user-story-linked |
- |
⏭️ 3 scans not applied
| Scan | Reason |
|---|---|
vulnerability-scan |
no manifest/lockfile changed |
github-actions-scan |
no workflow files changed |
iac-scan |
no IaC files changed |
Powered by Chainloop and Chainloop Trace
Assisted-by: Claude Code Signed-off-by: Javier Rodriguez <javier@chainloop.dev> Chainloop-Trace-Sessions: e1d524cd-4ee3-472d-9800-c5b0569a6689
There was a problem hiding this comment.
1 issue found across 5 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="app/cli/internal/token/token.go">
<violation number="1" location="app/cli/internal/token/token.go:162">
P2: When `repository_owner_id` or `namespace_id` is encoded as a non-integral or oversized JSON number, this conversion changes the namespace before telemetry identity construction. Reject non-integer and out-of-range values before converting, or preserve the original numeric representation.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| return v | ||
| } | ||
| case float64: | ||
| return strconv.FormatInt(int64(v), 10) |
There was a problem hiding this comment.
P2: When repository_owner_id or namespace_id is encoded as a non-integral or oversized JSON number, this conversion changes the namespace before telemetry identity construction. Reject non-integer and out-of-range values before converting, or preserve the original numeric representation.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At app/cli/internal/token/token.go, line 162:
<comment>When `repository_owner_id` or `namespace_id` is encoded as a non-integral or oversized JSON number, this conversion changes the namespace before telemetry identity construction. Reject non-integer and out-of-range values before converting, or preserve the original numeric representation.</comment>
<file context>
@@ -129,3 +145,23 @@ func Parse(token string) (*ParsedToken, error) {
+ return v
+ }
+ case float64:
+ return strconv.FormatInt(int64(v), 10)
+ }
+ }
</file context>
Every CLI command authenticated with a federated CI token reported to PostHog under the same
distinct_id: the OIDC issuer URL. For GitHub Actions that ishttps://token.actions.githubusercontent.com, identical for every run, every repository and every installation, so all federated CI traffic collapsed into a single person.Federated sessions are now identified by the CI namespace that owns the repository, scoped to the control plane installation:
The namespace comes from a claim the runner token already carries — GitHub's
repository_owner_idor GitLab'snamespace_id. Both are numeric and survive a rename, and no additional request or configuration is involved. A provider that emits neither claim falls back to the previous behaviour.API token, user token and unauthenticated sessions are unchanged. The claim is exposed on
ParsedTokenfor telemetry only and does not reach the attestation auth metadata, which stays keyed on the token ID.Populating
org_id, so PostHog'sorganizationgroup works for user and federated sessions, is tracked separately in PFM-7278.Closes PFM-7269
AI disclosure
Written with AI assistance (Claude Code). Every commit carries an
Assisted-by: Claude Codetrailer.