Skip to content

feat(analytics): an authored cube's measures.format and dimensions.granularities take effect on the query doors (#20282, stage 2) - #20635

Draft
objectstack-fleet[bot] wants to merge 8 commits into
mainfrom
claude/issue-20282-cube-format-granularities
Draft

objectstack-fleet[bot] wants to merge 8 commits into
mainfrom
claude/issue-20282-cube-format-granularities

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Part of #20282
Clause-②: yes (narrowing)

Stage 2 of #20282, under claim 5886559774 (session_014EJ1ED8X4MMrT18BhVx4tx, domain:spec seat 2). An AUTHORED analytics cube's measures.format and dimensions.granularities now reach the readers a compiled dataset already reaches. refreshKey is measured only, and nothing is built for it. The card stays open for stage 3 (descriptions) and for the refreshKey decision.

What changes

One Cube shape has three producers: authored cubes (AnalyticsServiceConfig.cubes, which the CLI threads from analyticsCubes), compiled datasets, and ad-hoc inference. Until now, both keys were read only on the compiled-dataset path.

  • measures.format
    • analytics-service.ts#withDeclaredMeasureFormats runs in queryIn, beside the SQL-echo gate.
    • Every measure column a query names now carries its cube measure's declared format as fields[].format. This holds for every strategy (NativeSQL, ObjectQL, the delegated fallback) and for both member spellings.
    • A measure that declares no format gets no key. A value already on the column is never replaced.
    • On the dataset door, the value read is the compiler's copy of the dataset measure's format, the same value enrichResultColumns writes anyway.
    • GET /analytics/meta is unchanged. See the premise checks below.
  • dimensions.granularities
    • analytics-service.ts#withDeclaredGranularityDefaults runs on query() and on the generateSql() dry run, before the source-field gates and strategy selection.
    • It reads dataset-executor.ts#declaredDefaultGranularity, a new function extracted from granularityOf, which now calls it too. Both producers are therefore read by one rule.
    • The rule, exactly the compiled-dataset path's:
      • a single-entry list is the default bucket for a time dimension the query groups by without stating a granularity;
      • a stated granularity always wins;
      • a granularity outside the list is not refused;
      • a list of two or more states no default;
      • a timeDimensions entry that carries only a dateRange, for a dimension the query does not group by, stays a filter.
  • Spec (packages/spec/src/data/analytics.zod.ts, after PR docs(spec): re-anchor the dead tracker citations in stack.zod.ts and data/analytics.zod.ts to the commits that decided them (stage 7) #20616 merged; origin/main merged first through scripts/pm/os-regen-merge.sh as d963f30e33)
    • MetricSchema.format and DimensionSchema.granularities gain describes that state the enforcement.
    • The metric's example values move from the names "currency" and "percent" to numeral patterns, the vocabulary the fields[].format slot documents.
    • content/docs/references/data/analytics.mdx is regenerated with gen:docs.
  • Ledger
    • Both rows in packages/spec/liveness/analytics_cube.json go dead → live. Each cites its readers as file#symbol and the CLI threading producer.
    • The file note's two sentences that named both keys as dead are rewritten.
    • state-counts/analytics_cube.md is regenerated with gen:liveness-counts: live/dead goes from 18/9 to 20/7.
    • The analytics_cube Notes cell in liveness/README.md, which listed both keys among the dead, is rewritten.
  • Changeset: @objectstack/spec minor and @objectstack/service-analytics minor, with a **BREAKING** sentence.

Clause-② (measured arm): yes (narrowing)

  • Widening: two authored keys take effect, and fields[].format is populated for authored cubes. The contract already declares that member.
  • Narrowing, measured:
    • Method: a throwaway service-seam probe at 958251b6ac, run once on head and once with the fill ablated through scripts/ablation-replace.mjs. The mutation landed (blob 9d77adcd798f → ee1b928e6f7e) and was restored to HEAD. The probe is not committed.
    • Cube: an authored cube whose placed_at declares granularities: ['month'] and whose shipped_at declares two intervals.
request fill ablated (= base behaviour) head
default composition (native SQL and engine aggregate): custom-SQL measure grouped by placed_at 200, one row per timestamp 400 INVALID_FIELD (the engine path's custom-SQL refusal)
host whose queryCapabilities offers raw SQL only (a hand override; AnalyticsServicePlugin wires both) 200 "No strategy can handle query"
count grouped by placed_at 200, raw timestamps 200, month buckets
control: custom-SQL measure grouped by shipped_at (two intervals) 200 200
  • The 400 is byte-identical (code, status and message) to what the same request with granularity: 'month' stated by hand already got. The pin is DECLARED NARROWING in the service test.
  • The changeset carries the **BREAKING** sentence (the class and its remedy) and the disposition not-required (no-migration-prescription).
  • check-adr-0087-registration: ✓ 1 declared-breaking changeset(s), each carrying an ADR-0087 disposition.
  • Whether this deserves a D3 semantic entry instead, as stage 1 got, is left to the seat. The reason it might is that the protocol-18 sub-day conversion can itself mint a single-entry list.

Premise checks, against origin/main 7510663c87

  • format on CubeMeta: not done, because the premise does not hold.
    • The dataset path surfaces format only through fields[]. A compiled dataset's getMeta projection is { name, type, title } too.
    • AnalyticsMetadataResponseSchema records the narrowing for this (#6442).
    • content/docs/api/data-api.mdx already sends clients to fields[] for format.
    • The spec contract files (contracts/analytics-service.ts, api/analytics.zod.ts) are outside this claim's surface.
  • granularities refusal: none invented.
    • The dataset path never compares a requested granularity against the list, so there is no refusal to mirror.
    • What a multi-entry list should mean ("Supported Granularities") is an open fork in the report.
  • refreshKey census (tree 958251b6ac; packages/services, packages/drivers, packages/rest, non-test):
    • refreshKey: 0 hits. The repo-wide control finds 9 files.
    • Pre-aggregation, materialized-view and rollup terms: 7 hits, all unrelated (automation subflow rollups, and a driver-sql built-in column flag).
    • ICacheService consumers: plugin-auth rate-limit and secondary storage, runtime inbound rate limit, dispatcher counter store, sms. None is in analytics.
    • service-analytics reads no job, cache or scheduler service. Its one cache is the request-scoped label map in dimension-labels.ts#withLabelFetchCache (the lit control, 1 hit).
    • A scheduler exists (service-job's IJobService: cron, interval and db adapters), and nothing in analytics uses it.
    • Nothing exists that could key on refreshKey, so building a cache is a separate card.

Verification

Final head d665865d5b, unless a line says otherwise. Builds and test runs went through os-verify-lock.sh. The check:* gates and eslint ran outside it, as the lock's scope prescribes. So did gen:docs, after two lock acquisitions for check:generated --fix timed out in the queue (exit 99).

  • Builds (①):
    • pnpm --filter '@objectstack/service-analytics...' build: exit 0.
    • turbo run build --filter='@objectstack/runtime^...': 29/29.
    • pnpm --filter @objectstack/spec build, after the describe edit: exit 0.
  • Tests (②):
suite files tests result
service-analytics, whole package 135 3178 pass
new service-door file — 13 included above
spec --project local 575 16917 (+1 todo) pass
spec --project repo 42 745 pass
runtime: the new REST pin plus the 2 sibling harness files that consume service-analytics 3 24 pass
rest: the 7 files that import service-analytics 7 86 pass (at 958251b6ac; service-analytics src is unchanged since)
  • Typecheck is clean for spec, service-analytics and runtime. Runtime's includes check:test-typecheck, and the new file adds no debt.
  • tsc --listFiles puts the new service test in service-analytics' program.
  • Two consumers were not run, and both are unaffected by construction because their cubes declare no format and no time dimension: packages/client analytics-automation-json-erasure.test.ts, and the dogfood analytics files, which declare neither key.
  • Ablations (predicted before each run; every leg restored to the HEAD blob with git diff HEAD empty):
mutation suite predicted observed at
format early-return (9d77adcd798f → 0d48cfde9469) service door 4 red 4 red, 8 green 9bf3b3b0b4
format early-return REST, through dist/ 1 red 1 red, 2 green 9bf3b3b0b4
granularity early-return (9d77adcd798f → 94b1bc63165a) service door 6 red 6 red, 7 green 958251b6ac
granularity early-return REST, through dist/ 2 red 2 red, 1 green 9bf3b3b0b4
  • Both REST legs were rebuilt, then checked with ablation-dist-preflight (marker present in 2 built files). Each restore leg was rebuilt again and checked --absent, with the tree clean.
  • Gates:
    • dispatch-gates --commands: 111 derived. --ran with exit codes: 111 run, 0 NOT MEASURED. 109 exit 0.
    • Two exit 1, both pre-existing. Their inputs are byte-identical to the merge base 1322cc72c:
      • check:platform-checklist: areas/identity-auth.json cites auth-plugin.ts#twoFactor, which is absent;
      • check:docs-transcript-drift: 4 CLI transcripts print "author-time rules (47)", and the count derives to 46.
    • check:liveness: analytics_cube 27 classified (live 20, dead 7), with the state-counts current.
    • check:generated: all 15 artifacts up to date.
  • Lint, narrowed:
    • eslint --no-inline-config --format json on the 4 changed .ts files: 4 files, 0 errors, 0 warnings.
    • The other 4 changed paths (.md / .json) are reported by eslint itself as "File ignored because no matching configuration".
    • Invariance: eslint.config.mjs never enables type-aware linting (no parserOptions.project), so this diff cannot move a verdict on an untouched file.

Acceptance notes

  • File-surface amendments to claim 5886559774, each forced by the claim's own items:
    • the analytics_cube Notes cell in packages/spec/liveness/README.md. It is the prose half of the state table whose shard the claim names, and it named both keys as dead.
    • packages/runtime/src/analytics-authored-cube-format-granularity.test.ts, the REST pin the claim asks for "where the analytics harness reaches". It is a new file, and the runtime harness drives the real dispatcher route.
    • the generated content/docs/references/data/analytics.mdx, which the describes regenerate.
  • packages/services/service-analytics/src/preview-evaluator.ts still open-codes the single-entry rule (dim.granularities?.length === 1) on the draft-preview path. That makes it a third spelling beside declaredDefaultGranularity. It is outside this surface; carrier: 承接者:无.
  • examples/app-showcase/src/data/analytics/showcase.cube.ts authors done_rate: { format: 'percent' }. That named style now reaches fields[].format verbatim, and a numeral-pattern renderer does not read it as a percentage. The spec describe now teaches the pattern vocabulary. The example's value is outside this surface and is reported to the seat.
  • Carried from stage 1 and unchanged here: the open-core os serve artifact-fallback boot threads no analyticsCubes.

Generated by Claude Code

…anularities reach the query doors (wip)

Claude-Session: https://claude.ai/code/session_014EJ1ED8X4MMrT18BhVx4tx
Co-authored-by: Claude <noreply@anthropic.com>
…ty at the query doors, with dataset controls

Claude-Session: https://claude.ai/code/session_014EJ1ED8X4MMrT18BhVx4tx
Co-authored-by: Claude <noreply@anthropic.com>
… over POST /analytics/query and /analytics/sql

Claude-Session: https://claude.ai/code/session_014EJ1ED8X4MMrT18BhVx4tx
Co-authored-by: Claude <noreply@anthropic.com>
…es are live; pin the declared narrowing; changeset

Claude-Session: https://claude.ai/code/session_014EJ1ED8X4MMrT18BhVx4tx
Co-authored-by: Claude <noreply@anthropic.com>
…cribe what the analytics service does with them

Claude-Session: https://claude.ai/code/session_014EJ1ED8X4MMrT18BhVx4tx
Co-authored-by: Claude <noreply@anthropic.com>
…be-format-granularities

# Conflicts:
#	packages/spec/liveness/README.md
…e-granularity-default-enforced

Claude-Session: https://claude.ai/code/session_014EJ1ED8X4MMrT18BhVx4tx
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/service-analytics, @objectstack/spec, touching 11 documentable anchor(s). ⚠️ 3 changed file(s) yielded no anchor (packages/spec/liveness/README.md, packages/spec/liveness/analytics_cube.json, packages/spec/liveness/state-counts/analytics_cube.md), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

1 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/api/data-api.mdx (via DimensionSchema (symbol, a top-level const), MetricSchema (symbol, a top-level const))

⛔ 3 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/v14.mdx (via generateSql (symbol, a method of class AnalyticsService))
  • content/docs/releases/v17/17-2.mdx (via MetricSchema (symbol, a top-level const))
  • content/docs/releases/v17/17-5.mdx (via generateSql (symbol, a method of class AnalyticsService))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • 3 changed file(s) yielded no anchor (packages/spec/liveness/README.md, packages/spec/liveness/analytics_cube.json, packages/spec/liveness/state-counts/analytics_cube.md) — pages documenting those are invisible to this run
  • 4 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 137 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 3f45b6cc13fb4646fab723a517225aa911cf17b8 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from c5adcea116fe2dd0005d0ab8efa1f39e8e36a4d6 — the merge of head 257ab1bc9287fb746279ea387a631eb389fb1a67 into base 3f45b6cc13fb4646fab723a517225aa911cf17b8, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin c5adcea116fe2dd0005d0ab8efa1f39e8e36a4d6 && git checkout c5adcea116fe2dd0005d0ab8efa1f39e8e36a4d6
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 3f45b6cc13fb4646fab723a517225aa911cf17b8 257ab1bc9287fb746279ea387a631eb389fb1a67 && git checkout -B drift-repro 3f45b6cc13fb4646fab723a517225aa911cf17b8 && git merge --no-ff 257ab1bc9287fb746279ea387a631eb389fb1a67

node scripts/docs-audit/affected-docs.mjs --json 3f45b6cc13fb4646fab723a517225aa911cf17b8

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 3f45b6cc13fb4646fab723a517225aa911cf17b8 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:data size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants