Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .changeset/20637-cube-refresh-key-retired.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
'@objectstack/spec': minor
---

feat(spec)!: retire an analytics cube's `refreshKey` — the refresh cadence and data-change probe nothing read (#20637)

**BREAKING** — `refreshKey` on an analytics cube (`CubeSchema`), with its `every` and `sql`, is now refused at parse: nothing ever read it, and no analytics result is cached, so a declared refresh cadence refreshed nothing. Delete the key. Every analytics query is computed when it is asked, as it always was. A refresh cadence is declared again when a result cache exists.

Clause-②: no (narrowing)

Measured before removal: `git grep refreshKey` over the non-test sources of `packages/services`, `packages/drivers` and `packages/rest` answered 0 lines (4 for the neighbouring `.public` in the same pathspec). `@objectstack/service-analytics` references no cache or job service; its one cache is request-scoped (dimension labels). The one in-repo author was the showcase app (`every: '1 hour'`), which no longer writes it.

**Removed rather than enforced** (ADR-0049 enforce-or-remove; the maintainer's ruling on the card, letter C): a result cache keyed by cube, query, read scope and tenant is a subsystem with its own design, and a key that does nothing until then is the residue ADR-0049 removes. `sql` also had no safe seam: raw SQL on a schedule, outside the read scope every other cube `sql` goes through.

## FROM → TO

| you wrote (17.5 and earlier) | write instead |
| --- | --- |
| `refreshKey: { every: '1 hour' }` | nothing — delete the key |
| `refreshKey: { sql: 'SELECT MAX(updated_at) FROM orders' }` | nothing — delete the key |
| `refreshKey: { every: '1 hour', sql: '…' }` | nothing — delete the key |

**The one-line fix:** delete `refreshKey` from every cube.

**What an author who still writes it sees.** `tsc` fails at the authoring site (`Cube` types the key `never`), and the parse — `defineCube()`, `defineStack({ analyticsCubes })`, `PUT /api/v1/meta/analytics_cube/:name` — refuses it at `refreshKey` with the prescription:

> `analytics_cube.refreshKey` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing read it: no analytics result is cached, so neither `every` nor `sql` ever refreshed anything. Delete the key; every analytics query is computed when it is asked. A refresh cadence is declared again when a result cache exists. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand.

`os migrate meta --from 17` lists the mechanical edits for existing sources; apply them by hand.

## The retirement kit

- **A `retiredKey()` tombstone** on `CubeSchema`, a `strictObject` — so the refusal carries the prescription rather than a bare unknown-key report, and `tsc` fails first. The nested `every` / `sql` shape is gone with it. `RETIRED_KEYS_BY_MAJOR[18]`: `data/Cube:refreshKey`. The key had no default, so no retired-default residue is owed.
- **The D2 conversion `cube-refresh-key-removed`** (protocol 18, retired from the load path) deletes the whole block from every `analyticsCubes[]` entry, one notice per cube, as a lossless delete. A built artifact or a stored `analytics_cube` row that carries it loads through the rehydration seams, which replay it.
- **The D3 entry `cube-refresh-key-retired`** asks the author whether anything they built assumed cube results were cached or refreshed on a schedule. They never were.
- **Ledger:** the `refreshKey.every` / `refreshKey.sql` rows collapse into one `dead` tombstone row.
- **No deprecation window**, per the project's startup-stage posture.

⚠️ **The out-of-repo consumer population is NOT MEASURED.** `@objectstack/spec` is published, so this is breaking for consumers no telemetry was consulted for.

<!-- adr-0087: registered cube-refresh-key-removed, cube-refresh-key-retired -->
9 changes: 1 addition & 8 deletions content/docs/references/data/analytics.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ Type: `[string, string]`
| **measures** | `Record<string, { label: string; description?: string; type: Enum<'count' \| 'sum' \| 'avg' \| 'min' \| 'max' \| 'count_distinct' \| 'number' \| 'string' \| 'boolean'>; sql: string; … }>` | ✅ | Quantitative metrics, keyed by metric name: the record key IS the metric's name, published and queried as `<cube>.<key>`. A metric declares no inner `name`. |
| **dimensions** | `Record<string, { label: string; description?: string; type: Enum<'string' \| 'number' \| 'boolean' \| 'time' \| 'geo'>; sql: string; … }>` | ✅ | Qualitative attributes, keyed by dimension name: the record key IS the dimension's name, published and queried as `<cube>.<key>`. A dimension declares no inner `name`. |
| **joins** | `Record<string, { name: string }>` | optional | |
| **refreshKey** | `{ every?: string; sql?: string }` | optional | |
| **refreshKey** | `never` | optional | [REMOVED] `analytics_cube.refreshKey` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing read it: no analytics result is cached, so neither `every` nor `sql` ever refreshed anything. Delete the key; every analytics query is computed when it is asked. A refresh cadence is declared again when a result cache exists. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. |
| **public** | `boolean` | optional (default: `true`) | Whether the analytics API exposes this cube. Default true (visible). false hides it from GET /analytics/meta and refuses POST /analytics/query and /analytics/sql for it (CUBE_NOT_FOUND). Visibility only: the underlying object's permissions and row-level security still govern its records on every door. |
| **_lock** | `Enum<'none' \| 'no-overlay' \| 'no-delete' \| 'full'>` | optional | Item-level lock — controls overlay & delete (ADR-0010). |
| **_lockReason** | `string` | optional | Human-readable reason shown when a write is refused by _lock. |
Expand Down Expand Up @@ -167,13 +167,6 @@ Type: `[string, string]`
| :--- | :--- | :--- | :--- |
| **name** | `string` | ✅ | Target cube name — the object this join reaches. The ON clause is DERIVED from the declared relationship between the two cubes' objects (a foreign-key equality) and is never authored. The KEY this join is declared under in the `joins` record is the FOREIGN-KEY FIELD on this cube's own object, not a second spelling of the object it reaches: the runtime emits `LEFT JOIN <name> <key> ON <base>.<key> = <key>.id` and resolves a member written `<key>.<field>` through that alias. A join keyed after the TARGET object joins on a column the base object does not have, so nothing resolves. |

### Nested Shape: `Cube.refreshKey`

| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **every** | `string` | optional | Refresh interval (e.g. "1 hour") |
| **sql** | `string` | optional | SQL to check for data changes |


---

Expand Down
10 changes: 5 additions & 5 deletions docs/audits/2026-07-unknown-key-strictness-ledger.counts/data.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The `strict` column is the one the campaign schedules against; it counts both th

| Dir | Sites | strict | passthrough | catchall | strip |
|---|---|---|---|---|---|
| `data/` | 159 | 76 | 1 | 0 | 82 |
| `data/` | 158 | 75 | 1 | 0 | 82 |

## `data/` — sites

Expand All @@ -31,7 +31,7 @@ classify and is not listed (it becomes reportable the day it grows its first sit

| File | Sites |
|---|---|
| `analytics.zod.ts` | 7 |
| `analytics.zod.ts` | 6 |
| `data-engine.zod.ts` | 15 |
| `datasource.zod.ts` | 6 |
| `document.zod.ts` | 8 |
Expand All @@ -56,15 +56,15 @@ classify and is not listed (it becomes reportable the day it grows its first sit
| `seed-loader.zod.ts` | 12 |
| `seed.zod.ts` | 1 |
| `validation.zod.ts` | 6 |
| **total** | **159** |
| **total** | **158** |

## `data/` — open

Per file, how many of its sites still silently discard unknown keys. The `Class`
column that decides the bucket split is hand-written in the ledger; the arithmetic
over it is here.

**82 strip of 159**, in 11 file(s).
**82 strip of 158**, in 11 file(s).

| File | Strip | Sites |
|---|---|---|
Expand All @@ -79,7 +79,7 @@ over it is here.
| `hook.zod.ts` | 5 | 7 |
| `query.zod.ts` | 4 | 5 |
| `seed-loader.zod.ts` | 12 | 12 |
| **total** | **82** | **159** |
| **total** | **82** | **158** |

| Bucket | Sites |
|---|---|
Expand Down
2 changes: 1 addition & 1 deletion docs/audits/2026-07-unknown-key-strictness-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -725,7 +725,7 @@ column does not move and the `strip` column falls by the count of what left.
| `driver/memory.zod.ts` / `driver/mongo.zod.ts` / `driver/postgres.zod.ts` | authorable | The per-driver shapes for the `config` slot — what an author actually writes under `datasource.config` (`host`, `port`, `filename`). **Undeclared here until the coverage walk went recursive** (see below): a subdirectory was invisible to the gate, so these sites sat outside the map while the map reported full coverage. **Strict as of #4410**, which is also what unblocked them: this row previously read "strictness here would enforce nothing" because nothing parsed `datasource.config` against these schemas and both `*DriverSpec.configSchema` literals were `{}`. Now `DatasourceSchema` parses `config` against them, and the same schemas project onto `configSchema` and onto the Studio connection form. (#4410 also ran the parse over each `readReplicas` entry; #4468 retired that key outright — see the row above.) `postgres.zod.ts` drops a site: its `ssl` was a `boolean | {ca, cert, key, …}` union, and the object arm is gone — certificates now live in the datasource-level `ssl` block (declared, strict, and until #4410 read by nobody), leaving `config.ssl` as the on/off shorthand. That narrowing is forced by the same projection: the Studio form renders anything that is not boolean/enum/number as a TEXT INPUT, so a union here would have produced a wizard whose every `ssl` value the new gate rejects. `memory.zod.ts` keeps 6 but loses two KEYS — `indexes` / `maxRecordsPerObject`, which `InMemoryDriverConfig` has no field for, removed under ADR-0049 rather than blessed by the new gate |
| `driver/turso.zod.ts` | authorable | The libSQL/Turso `config` contract, added by **#6345** — and the last driver on the platform whose `config` had no gate at all. It was not an oversight of #4410 but a consequence of turso not being a BUILTIN: its driver ships in the optional `@objectstack/driver-turso` package, so `resolveDriverId('turso')` returned `undefined` and `validateDriverConfig` answered `{ known: false }` — "nothing to check against" — while both boot hosts dispatched `turso` for real. A datasource carrying `{ token: … }` (the plausible spelling; the driver reads `authToken`) was therefore accepted in silence and then connected UNAUTHENTICATED, which is #4410's own failure mode surviving in the one driver #4410 could not see. Every site strict, same error factory as the rest of the campaign, including the nested `sync` block — a bare `z.object` there would have dropped `sync: { interval: 60 }` and synced on the default while the author believed otherwise, i.e. added a strip site to this map instead of closing one. The declared keys are drawn from what `TursoDriverConfig` actually READS, not from what libSQL supports, so closing this gap does not open an ADR-0049 one: `client` (a live `@libsql/client` instance — not authorable metadata), `pool` and `schemaMode`/`readOnly` (datasource-level, like every other driver) are deliberately absent |
| `driver/mysql.zod.ts` / `driver/sqlite.zod.ts` | authorable | The rest of the `config` contract, added by #4410. `mysql.zod.ts` and `sqlite.zod.ts` (sqlite + sqlite-wasm) are shapes that **never existed** — both driver ids were offered by the connection form and buildable by the shared factory, with no config contract anywhere, so `driver: 'sqlite'` + a misspelled `filename` was an ephemeral `:memory:` database reported as configured. All three sites strict, same error factory as the rest of the campaign. (Their sibling `driver/common.zod.ts` holds shared enums and prescription strings and has no `z.object(` site, so the coverage gate skips it) |
| `analytics.zod.ts` | authorable | **strict as of #4001 batch D — all 8 sites; the `mixed (p)` resolved to authorable on both halves.** The cube family (Metric + its `filters[]` item, Dimension, CubeJoin, Cube + `refreshKey`) has two live authoring doors, measured: `defineCube()` `.parse()`s an author literal (the showcase example authors through it) and `defineStack({ analyticsCubes })` carries every cube through `StackSchema.parse` — the whole family resolves REACHABLE in the batch-D BFS (positive/negative controls green in the same run). Pre-close probes: a cube's `publik`, a metric's `title`, a join's `relationshipp` (falling back to the `many_to_one` default — a different join shape under a successful parse), a `refreshKey.sqll` all parsed clean and vanished. The query half was subtler and is the batch's live behaviour change: `AnalyticsQuerySchema`'s TOP level was already gated at its one production door (`api/analytics.zod.ts`'s `AnalyticsQueryRequestSchema` is `.extend(…).strict()` since #3878), but **top-level strictness does not recurse** — measured on `main`, `timeDimensions: [{ dimension, granuarity: 'day' }]` rode through the strict wrapper with the typo silently stripped, bucketing the whole range as one group under an ordinary 200. Closing the base makes the posture hold at every door instead of only at the wrapper that re-applied it. ADR-0010 envelope deliberately NOT declared: no protected item re-parses here (`CubeRegistry.register` takes typed objects without a parse; `analytics_cube` resolves no `getMetadataTypeSchema` entry so `saveMetaItem` never parses it; artifact ingest parses the compiled definition BEFORE `applyProtection` stamps). Producer sweep over objectui/cloud: zero cube/analytics-vocabulary producers (objectui's `data-objectstack` sends only declared keys — `cube`/`measures`/`dimensions`/`where`) |
| `analytics.zod.ts` | authorable | **strict as of #4001 batch D — all 8 sites; the `mixed (p)` resolved to authorable on both halves.** **Two of the eight sites later left with their keys (ADR-0049 enforce-or-remove): the metric `filters[]` item (#10414) and the cube's `refreshKey` block (#20637, retired whole — nothing read `every` or `sql`, and no analytics result is cached). The batch-D verdicts for those two are superseded, not reopened; their pins in `analytics-strictness-batchd.test.ts` now assert the retirement prescriptions.** The cube family (Metric + its `filters[]` item, Dimension, CubeJoin, Cube + `refreshKey`) has two live authoring doors, measured: `defineCube()` `.parse()`s an author literal (the showcase example authors through it) and `defineStack({ analyticsCubes })` carries every cube through `StackSchema.parse` — the whole family resolves REACHABLE in the batch-D BFS (positive/negative controls green in the same run). Pre-close probes: a cube's `publik`, a metric's `title`, a join's `relationshipp` (falling back to the `many_to_one` default — a different join shape under a successful parse), a `refreshKey.sqll` all parsed clean and vanished. The query half was subtler and is the batch's live behaviour change: `AnalyticsQuerySchema`'s TOP level was already gated at its one production door (`api/analytics.zod.ts`'s `AnalyticsQueryRequestSchema` is `.extend(…).strict()` since #3878), but **top-level strictness does not recurse** — measured on `main`, `timeDimensions: [{ dimension, granuarity: 'day' }]` rode through the strict wrapper with the typo silently stripped, bucketing the whole range as one group under an ordinary 200. Closing the base makes the posture hold at every door instead of only at the wrapper that re-applied it. ADR-0010 envelope deliberately NOT declared: no protected item re-parses here (`CubeRegistry.register` takes typed objects without a parse; `analytics_cube` resolves no `getMetadataTypeSchema` entry so `saveMetaItem` never parses it; artifact ingest parses the compiled definition BEFORE `applyProtection` stamps). Producer sweep over objectui/cloud: zero cube/analytics-vocabulary producers (objectui's `data-objectstack` sends only declared keys — `cube`/`measures`/`dimensions`/`where`) |
| `document.zod.ts` | wire (p) | |
| `hook.zod.ts` / `hook-body.zod.ts` | mixed | **strict as of #4001 data step** for the AUTHORING shapes: `HookSchema` (+ `retryPolicy`) and both body branches (`ExpressionBodySchema` / `ScriptBodySchema`). `HookContextSchema` and its `session` / `provenance` / `user` blocks are the RUNTIME shape the engine hands a handler — they stay tolerant, and must: strictness there would make an engine-internal enrichment (as `provenance` was in #3712) a breaking change for anyone parsing a context they were given. The file's old blanket `authorable (p)` was too wide — verification split it |
| `mapping.zod.ts` | authorable (p) | |
Expand Down
7 changes: 4 additions & 3 deletions examples/app-showcase/src/data/analytics/showcase.cube.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,10 @@ export const DeliveryCube = defineCube({
name: 'showcase_project',
},
},
refreshKey: {
every: '1 hour',
},
// No refresh cadence: no analytics result is cached, so every query against
// this cube is computed when it is asked. The `every: '1 hour'` this file
// declared was read by nothing, and the key is retired.
//
// No `public` key: the cube is VISIBLE, the default. It is this app's
// demonstration of the `/api/v1/analytics/*` surface (src/coverage.ts marks
// `analyticsCubes` demonstrated, and the platform checklist's dashboards item
Expand Down
2 changes: 1 addition & 1 deletion packages/spec/authorable-surface/data.json
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@
"data/Cube:measures",
"data/Cube:name",
"data/Cube:public",
"data/Cube:refreshKey",
"data/Cube:refreshKey [RETIRED]",
"data/Cube:sql",
"data/Cube:title",
"data/CubeJoin:name",
Expand Down
Loading
Loading