Skip to content

feat(cli): scope federated telemetry identity to the CI namespace - #3433

Open
javirln wants to merge 2 commits into
chainloop-dev:mainfrom
javirln:javier/pfm-7269-cli-telemetry-all-federated-ci-tokens-report-as-one-posthog
Open

feat(cli): scope federated telemetry identity to the CI namespace#3433
javirln wants to merge 2 commits into
chainloop-dev:mainfrom
javirln:javier/pfm-7269-cli-telemetry-all-federated-ci-tokens-report-as-one-posthog

Conversation

@javirln

@javirln javirln commented Sep 11, 2026

Copy link
Copy Markdown
Member

Every CLI command authenticated with a federated CI token reported to PostHog under the same distinct_id: the OIDC issuer URL. For GitHub Actions that is https://token.actions.githubusercontent.com, identical for every run, every repository and every installation, so all federated CI traffic collapsed into a single person.

Federated sessions are now identified by the CI namespace that owns the repository, scoped to the control plane installation:

ci:<namespace id>@<cp_url_hash>

The namespace comes from a claim the runner token already carries — GitHub's repository_owner_id or GitLab's namespace_id. Both are numeric and survive a rename, and no additional request or configuration is involved. A provider that emits neither claim falls back to the previous behaviour.

API token, user token and unauthenticated sessions are unchanged. The claim is exposed on ParsedToken for telemetry only and does not reach the attestation auth metadata, which stays keyed on the token ID.

Populating org_id, so PostHog's organization group works for user and federated sessions, is tracked separately in PFM-7278.

Closes PFM-7269

AI disclosure

Written with AI assistance (Claude Code). Every commit carries an Assisted-by: Claude Code trailer.

Review in cubic

Every CLI command authenticated with a federated CI token reported to
PostHog under the OIDC issuer URL, which is the same value for every run,
every repository and every installation, collapsing all federated traffic
into a single person.

Federated sessions are now identified by the CI namespace that owns the
repository, scoped to the control plane installation: the GitHub
organization (repository_owner_id) or the GitLab namespace (namespace_id),
both read from claims the runner token already carries.

API token, user token and unauthenticated sessions are unchanged, and the
claim stays out of the attestation auth metadata.

Assisted-by: Claude Code
Signed-off-by: Javier Rodriguez <javier@chainloop.dev>

Chainloop-Trace-Sessions: e1d524cd-4ee3-472d-9800-c5b0569a6689
@chainloop-platform

chainloop-platform Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

AI Session Checks — 🟡 87% · ⚠️ 2 failing

Avg score Sessions Failing policies Attribution Files Lines Total Duration
🟡 87% 1 ⚠️ 2 100% AI / 0% Human 5 +196 / -8 14h37m58s

🟡 87% — 100% AI — ⚠️ 2 policies failing

Sep 10, 2026 14:31 UTC · 14h37m58s · $24.67 · 380 in / 186.4k out · claude-code 2.1.267 (claude-opus-5)

View session details ↗

Change Summary

  • Adds CINamespaceID parsing for federated GitHub/GitLab tokens.
  • Scopes federated telemetry distinct_id to ci:<namespace>@<cp_url_hash>.
  • Adds token and telemetry tests, including failing-first federated identity coverage.
  • Opens follow-up PFM-7278 to populate org_id and the organization PostHog group.

AI Session Overall Score

🟡 87% — Well-verified fix, but setup and trust needed mid-session steering.

AI Session Analysis Breakdown

🟢 94% · scope-discipline

🟢 The code diff stayed inside CLI telemetry/token files and their tests. · High Impact

🟢 93% · verification

🟢 Tests were added first, shown failing, then rerun green after implementation. · High Impact

🟢 90% · alignment

🟢 After the user rejected D, the AI stopped and returned to option B. · High Impact

🟢 88% · solution-quality

No notes.

🟡 74% · user-trust-signal

🟠 The user abruptly stopped option D, signaling a temporary confidence dip mid-session. · Medium Severity

💡 When you have to stop a direction sharply, ask for a brief re-plan before the next branch.

🟡 68% · context-and-planning

🟠 No visible plan or TODO guided the research-plus-implementation flow on this multi-step task. · Medium Severity

💡 For research that may turn into code, write a brief phase plan before editing or proposing options.


File Attribution

████████████████████ 100% AI / 0% Human

Status Attribution File Lines
modified ai app/cli/internal/telemetry/telemetry_test.go +101 / -0
modified ai app/cli/internal/telemetry/telemetry.go +30 / -6
modified ai app/cli/internal/token/token.go +36 / -0
modified ai app/cli/internal/token/token_test.go +24 / -2
modified ai app/cli/cmd/root.go +5 / -0

Policies (4, 2 failing)

Status Policy Material Messages
✅ Passed ai-config-ai-agents-allowed ai-coding-session-e1d524 -
✅ Passed ai-config-no-dangerous-commands ai-coding-session-e1d524 -
⚠️ Failed ai-config-no-secrets ai-coding-session-e1d524
  • Secret (jwt) was redacted from session content [turn=711, source=tool_result, line=1]
  • Secret (jwt) was redacted from session content [turn=717, source=tool_result, line=2]
  • Secret (jwt) was redacted from session content [turn=717, source=tool_result, line=4]
  • Secret (jwt) was redacted from session content [turn=728, source=tool_result, line=1]
  • Secret (jwt) was redacted from session content [turn=732, source=assistant-tool_use:Bash, line=1]
  • Secret (posthog-project-api-key) was redacted from session content [turn=105, source=tool_result, line=10]
⚠️ Failed ai-config-mcp-servers-allowed ai-coding-session-e1d524 MCP server 'claude_ai_PostHog' is not in the allowed list

Security Checks — ⚠️ 2 failing

⚠️ secret-scan — 1 failing

Status Policy Messages
⚠️ Failed secrets-detection
  • Gitleaks secret detected d56ac29:app/cli/internal/token/token_test.go:jwt:67, secret: eyJhbGci...)
  • Gitleaks secret detected d56ac29:app/cli/internal/token/token_test.go:jwt:78, secret: eyJhbGci...)

sast-scan

Status Policy Messages
✅ Passed cwe-top26-40-cusp -
✅ Passed owasp-top10-2025 -
✅ Passed sast -
✅ Passed cwe-top25 -

PR info

Status Policy Messages
⚠️ Failed pr-min-approvals PR/MR #3433 has 0 approving reviews, 1 required.
✅ Passed pr-description-required -
✅ Passed pr-user-story-linked -

⏭️ 3 scans not applied

Scan Reason
vulnerability-scan no manifest/lockfile changed
github-actions-scan no workflow files changed
iac-scan no IaC files changed

View attestation ↗


Powered by Chainloop and Chainloop Trace

Assisted-by: Claude Code
Signed-off-by: Javier Rodriguez <javier@chainloop.dev>

Chainloop-Trace-Sessions: e1d524cd-4ee3-472d-9800-c5b0569a6689
@javirln
javirln marked this pull request as ready for review September 11, 2026 07:30

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 5 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="app/cli/internal/token/token.go">

<violation number="1" location="app/cli/internal/token/token.go:162">
P2: When `repository_owner_id` or `namespace_id` is encoded as a non-integral or oversized JSON number, this conversion changes the namespace before telemetry identity construction. Reject non-integer and out-of-range values before converting, or preserve the original numeric representation.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

return v
}
case float64:
return strconv.FormatInt(int64(v), 10)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When repository_owner_id or namespace_id is encoded as a non-integral or oversized JSON number, this conversion changes the namespace before telemetry identity construction. Reject non-integer and out-of-range values before converting, or preserve the original numeric representation.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At app/cli/internal/token/token.go, line 162:

<comment>When `repository_owner_id` or `namespace_id` is encoded as a non-integral or oversized JSON number, this conversion changes the namespace before telemetry identity construction. Reject non-integer and out-of-range values before converting, or preserve the original numeric representation.</comment>

<file context>
@@ -129,3 +145,23 @@ func Parse(token string) (*ParsedToken, error) {
+				return v
+			}
+		case float64:
+			return strconv.FormatInt(int64(v), 10)
+		}
+	}
</file context>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant