diff --git a/.changeset/20919-spec-connector-source-live.md b/.changeset/20919-spec-connector-source-live.md index d08e61f48a8..68c7459d98f 100644 --- a/.changeset/20919-spec-connector-source-live.md +++ b/.changeset/20919-spec-connector-source-live.md @@ -11,8 +11,9 @@ states that the next pull's starting point is read from the TARGET field a `fieldMapping` entry copies it onto (an unmapped one is refused at pull time), and `watermark` states the one-response limit: the connector's paging is not followed, so a paged endpoint yields its first page only. The liveness ledger's -`connectorSource` rows are `live` and keep `authorWarn` (nothing schedules a pull -yet). The retired `connector.syncConfig` prescription and the -`connector-sync-keys-retired` upgrade entry say the same, and the entry's -acceptance criterion no longer claims the connector is validated at authoring. +`connectorSource` rows are `live`, with no author warning: that nothing schedules a +pull yet is said on the key's description. The retired `connector.syncConfig` +prescription and the `connector-sync-keys-retired` upgrade entry say the same, and +the entry's acceptance criterion no longer claims the connector is validated at +authoring. No key, value or default changed. diff --git a/.changeset/21127-connector-source-live-row.md b/.changeset/21127-connector-source-live-row.md new file mode 100644 index 00000000000..769a69ff361 --- /dev/null +++ b/.changeset/21127-connector-source-live-row.md @@ -0,0 +1,27 @@ +--- +'@objectstack/spec': patch +--- + +fix(spec): a stack whose mapping authors `connectorSource` validates and lints again — the liveness ledger's `live` row no longer carries an author warning + +Clause-②: no + +`os validate` and `os lint` exited 1 on any stack with a `mappings[]` entry that +authored `connectorSource`, and the only output was the liveness lint's internal +error `ledger entry has unrecognised status "live"`. The ledger graded the key +`live` (the connector sync executor reads every key of the binding) and still +asked the lint to warn whoever authored it; the lint has no warning for a key +that works, and stops on that inconsistency by design. The row carries no warning +now, so both commands judge the stack and exit 0 when nothing else is wrong. The +runtime metadata door no longer returns an `authoring-rule-threw` advisory for +the same mapping. + +The note the warning used to carry is on the key's description, where an author +reads it: a pull runs when a `job` drives it, nothing schedules one yet, so the +binding alone moves no rows. The retired `connector.syncConfig` prescription and +the `connector-sync-keys-retired` upgrade entry no longer say that authoring the +binding warns. + +`check:liveness` now refuses a `live` ledger row with `authorWarn: true` at any +depth, and prints how many rows opt into an author warning on every run. A +`planned` row with `authorWarn` still warns. No key, value or default changed. diff --git a/content/docs/references/data/mapping.mdx b/content/docs/references/data/mapping.mdx index 418b2a850e1..e3f4b0f6ee5 100644 --- a/content/docs/references/data/mapping.mdx +++ b/content/docs/references/data/mapping.mdx @@ -49,7 +49,7 @@ const result = ImportFieldMappingSchema.parse(data); | **fieldMapping** | `{ source: string \| string[]; target: string \| string[]; transform: Enum<'none' \| 'constant' \| 'lookup' \| 'split' \| 'join' \| 'javascript' \| 'map'>; params?: object }[]` | ✅ | | | **mode** | `Enum<'insert' \| 'update' \| 'upsert'>` | optional (default: `"insert"`) | | | **upsertKey** | `string[]` | optional | Fields to match for upsert (e.g. email) | -| **connectorSource** | `{ connector: string; action: string; input?: Record; recordsPath?: string; … }` | optional | Pull binding: the rest/openapi connector this mapping pulls rows from (one-way, full or timestamp-incremental; a `job` sets the cadence). Pulled when a job drives it; nothing schedules it yet | +| **connectorSource** | `{ connector: string; action: string; input?: Record; recordsPath?: string; … }` | optional | Pull binding: the rest/openapi connector this mapping pulls rows from (one-way, full or timestamp-incremental; a `job` sets the cadence). Pulled when a job drives it; nothing schedules it yet, so the binding alone moves no rows — schedule the pull with a `job` once a job can drive one | | **_lock** | `Enum<'none' \| 'no-overlay' \| 'no-delete' \| 'full'>` | optional | Item-level lock — controls overlay & delete (ADR-0010). | | **_lockReason** | `string` | optional | Human-readable reason shown when a write is refused by _lock. | | **_lockSource** | `Enum<'artifact' \| 'package' \| 'env-forced'>` | optional | Layer that set _lock (artifact \| package \| env-forced). | diff --git a/content/docs/references/integration/connector.mdx b/content/docs/references/integration/connector.mdx index b7d39b78345..858e7508cca 100644 --- a/content/docs/references/integration/connector.mdx +++ b/content/docs/references/integration/connector.mdx @@ -191,7 +191,7 @@ const result = ConnectorSchema.parse(data); | **auth** | `{ type: 'none' } \| { type: 'bearer'; credentialRef: string } \| { type: 'api-key'; credentialRef: string; headerName?: string; paramName?: string } \| { type: 'basic'; username: string; credentialRef: string }` | optional | Declarative instance auth — references credentials via `credentialRef` (resolved at boot), never inline secrets. Requires `provider` (ADR-0097). | | **actions** | `{ key: string; label: string; description?: string; inputSchema?: Record; … }[]` | optional | | | **triggers** | `never` | optional | [REMOVED] `connector.triggers` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — a connector trigger never started anything: `AutomationEngine.registerConnector` registers a connector's actions only, no polling loop read `intervalSeconds` (or the `interval` spelling it was renamed from), and no receiver was driven by a `webhook` trigger. Delete the key; the `ConnectorTrigger` shape leaves with it. To start work from an external system, write a flow that calls the connector's action in a `connector_action` node: for an external event, an `api` flow that the event's sender calls; for a scheduled pull, a `schedule` flow. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. | -| **syncConfig** | `never` | optional | [REMOVED] `connector.syncConfig` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no engine ever ran a connector-attached sync: nothing read `strategy`, `direction`, `realtimeSync`, `timestampField`, `conflictResolution`, `batchSize`, `deleteMode` or `filters`, so the `latest_wins` and `soft_delete` defaults resolved and deleted nothing. Delete the key; the `DataSyncConfig` shape leaves with it. A sync is defined on its TARGET: a `mapping` (`targetObject`, `fieldMapping`, `mode`, `upsertKey`) whose `connectorSource` names the `rest` or `openapi` connector it pulls from, the read action and an optional timestamp `watermark`, with a `job` for the cadence. Its pull runs when a `job` drives it; nothing schedules one yet, and authoring the binding warns until a job can. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. | +| **syncConfig** | `never` | optional | [REMOVED] `connector.syncConfig` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no engine ever ran a connector-attached sync: nothing read `strategy`, `direction`, `realtimeSync`, `timestampField`, `conflictResolution`, `batchSize`, `deleteMode` or `filters`, so the `latest_wins` and `soft_delete` defaults resolved and deleted nothing. Delete the key; the `DataSyncConfig` shape leaves with it. A sync is defined on its TARGET: a `mapping` (`targetObject`, `fieldMapping`, `mode`, `upsertKey`) whose `connectorSource` names the `rest` or `openapi` connector it pulls from, the read action and an optional timestamp `watermark`, with a `job` for the cadence. Its pull runs when a `job` drives it; nothing schedules one yet, so the binding alone moves no rows. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. | | **fieldMappings** | `never` | optional | [REMOVED] `connector.fieldMappings` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no engine ever moved a value through a connector field mapping: nothing read `source`, `target`, `defaultValue`, `dataType`, `required` or `syncMode`. Delete the key; the `ConnectorFieldMapping` shape leaves with it. Map fields on the sync's TARGET instead: a `mapping`'s `fieldMapping` (`source` → `target`, with a `transform` the import path executes), which its `connectorSource` pulls through. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. | | **webhooks** | `never` | optional | [REMOVED] `connector.webhooks` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — a webhook nested inside a connector was never registered as a `webhook` item, so it was never materialized into `sys_webhook` and never delivered, and nothing emits the connector events its `events` list could name (`sync.completed`, `auth.expired` and the rest). Delete the key; the nested shape leaves with it (`WebhookConfig`, `WebhookEvent`, `WebhookSignatureAlgorithm`). To have a webhook actually sent, declare it in the stack's top-level `webhooks:` collection, which is materialized into `sys_webhook` and delivered on record events — note that doing so STARTS deliveries this connector never made. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. | | **rateLimitConfig** | `never` | optional | [REMOVED] `connector.rateLimitConfig` was removed in @objectstack/spec 17.0.0 (ADR-0049 D2) — the entire shape is gone, not just this key: `ConnectorRateLimitConfig` and its `RateLimitStrategy` enum were removed with it, because no outbound rate-limiting engine ever existed. The platform's only token bucket (runtime `security/rate-limit.ts`) throttles INBOUND requests to us; nothing throttled the calls a connector makes out, so every knob here was inert while reading like a configured cap. Delete the key. Do NOT substitute `shared` `RateLimitConfig` — that is the inbound limiter and would cap the wrong direction; until an outbound throttle exists, rate-limit at the connector provider or upstream gateway. Run `os migrate meta --from 16` to list the mechanical edits for existing sources; apply them by hand. | @@ -488,7 +488,7 @@ Connector type | **auth** | `{ type: 'none' } \| { type: 'bearer'; credentialRef: string } \| { type: 'api-key'; credentialRef: string; headerName?: string; paramName?: string } \| { type: 'basic'; username: string; credentialRef: string }` | optional | Declarative instance auth — references credentials via `credentialRef` (resolved at boot), never inline secrets. Requires `provider` (ADR-0097). | | **actions** | `{ key: string; label: string; description?: string; inputSchema?: Record; … }[]` | optional | | | **triggers** | `never` | optional | [REMOVED] `connector.triggers` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — a connector trigger never started anything: `AutomationEngine.registerConnector` registers a connector's actions only, no polling loop read `intervalSeconds` (or the `interval` spelling it was renamed from), and no receiver was driven by a `webhook` trigger. Delete the key; the `ConnectorTrigger` shape leaves with it. To start work from an external system, write a flow that calls the connector's action in a `connector_action` node: for an external event, an `api` flow that the event's sender calls; for a scheduled pull, a `schedule` flow. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. | -| **syncConfig** | `never` | optional | [REMOVED] `connector.syncConfig` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no engine ever ran a connector-attached sync: nothing read `strategy`, `direction`, `realtimeSync`, `timestampField`, `conflictResolution`, `batchSize`, `deleteMode` or `filters`, so the `latest_wins` and `soft_delete` defaults resolved and deleted nothing. Delete the key; the `DataSyncConfig` shape leaves with it. A sync is defined on its TARGET: a `mapping` (`targetObject`, `fieldMapping`, `mode`, `upsertKey`) whose `connectorSource` names the `rest` or `openapi` connector it pulls from, the read action and an optional timestamp `watermark`, with a `job` for the cadence. Its pull runs when a `job` drives it; nothing schedules one yet, and authoring the binding warns until a job can. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. | +| **syncConfig** | `never` | optional | [REMOVED] `connector.syncConfig` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no engine ever ran a connector-attached sync: nothing read `strategy`, `direction`, `realtimeSync`, `timestampField`, `conflictResolution`, `batchSize`, `deleteMode` or `filters`, so the `latest_wins` and `soft_delete` defaults resolved and deleted nothing. Delete the key; the `DataSyncConfig` shape leaves with it. A sync is defined on its TARGET: a `mapping` (`targetObject`, `fieldMapping`, `mode`, `upsertKey`) whose `connectorSource` names the `rest` or `openapi` connector it pulls from, the read action and an optional timestamp `watermark`, with a `job` for the cadence. Its pull runs when a `job` drives it; nothing schedules one yet, so the binding alone moves no rows. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. | | **fieldMappings** | `never` | optional | [REMOVED] `connector.fieldMappings` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no engine ever moved a value through a connector field mapping: nothing read `source`, `target`, `defaultValue`, `dataType`, `required` or `syncMode`. Delete the key; the `ConnectorFieldMapping` shape leaves with it. Map fields on the sync's TARGET instead: a `mapping`'s `fieldMapping` (`source` → `target`, with a `transform` the import path executes), which its `connectorSource` pulls through. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. | | **webhooks** | `never` | optional | [REMOVED] `connector.webhooks` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — a webhook nested inside a connector was never registered as a `webhook` item, so it was never materialized into `sys_webhook` and never delivered, and nothing emits the connector events its `events` list could name (`sync.completed`, `auth.expired` and the rest). Delete the key; the nested shape leaves with it (`WebhookConfig`, `WebhookEvent`, `WebhookSignatureAlgorithm`). To have a webhook actually sent, declare it in the stack's top-level `webhooks:` collection, which is materialized into `sys_webhook` and delivered on record events — note that doing so STARTS deliveries this connector never made. Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand. | | **rateLimitConfig** | `never` | optional | [REMOVED] `connector.rateLimitConfig` was removed in @objectstack/spec 17.0.0 (ADR-0049 D2) — the entire shape is gone, not just this key: `ConnectorRateLimitConfig` and its `RateLimitStrategy` enum were removed with it, because no outbound rate-limiting engine ever existed. The platform's only token bucket (runtime `security/rate-limit.ts`) throttles INBOUND requests to us; nothing throttled the calls a connector makes out, so every knob here was inert while reading like a configured cap. Delete the key. Do NOT substitute `shared` `RateLimitConfig` — that is the inbound limiter and would cap the wrong direction; until an outbound throttle exists, rate-limit at the connector provider or upstream gateway. Run `os migrate meta --from 16` to list the mechanical edits for existing sources; apply them by hand. | diff --git a/packages/cli/test/validate-lint-mapping-connector-source.test.ts b/packages/cli/test/validate-lint-mapping-connector-source.test.ts new file mode 100644 index 00000000000..cf550f6d408 --- /dev/null +++ b/packages/cli/test/validate-lint-mapping-connector-source.test.ts @@ -0,0 +1,158 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#21127] `os validate` and `os lint` judge a stack whose mapping authors + * `connectorSource` — they do not crash on it. + * + * ## The defect, measured before the fix + * + * `mapping.connectorSource` was re-graded `live` in the liveness ledger (the + * connector sync executor reads every key) with its `authorWarn: true` kept. + * The author-side liveness lint picks the verdict it shows from a warned row's + * status and throws its ledger-integrity error on `live`, by design — so on + * this fixture both commands exited 1 and the whole answer was + * `lintLivenessProperties: ledger entry has unrecognised status "live" …`. The + * row carries no warning now, and `check:liveness` refuses a warned `live` row. + * + * ## The control + * + * A `planned` row with `authorWarn` is the legal shape, and it still warns at + * the same door: the second fixture also authors `object.externalSharingModel` + * (`planned` + `authorWarn` in tree), and `os lint` reports it under + * `liveness-planned-property` — still exit 0, because a liveness finding is + * advisory. + * + * The CLI runs through `bin/run-dev.js` (source, via tsx); `@objectstack/lint` + * resolves through `exports` to `dist/`, and both read the ledger JSON the spec + * package ships under `liveness/`. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import { execFile } from 'node:child_process'; +import { mkdtempSync, rmSync, writeFileSync, mkdirSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { childEnv } from './helpers/serve-process.js'; +import { defineStackSource, linkSpec } from './helpers/define-stack-fixture.js'; + +const HERE = resolve(fileURLToPath(import.meta.url), '..'); +const CLI = resolve(HERE, '../bin/run-dev.js'); +const TSX = resolve(HERE, '../../../node_modules/.bin/tsx'); +/** A cold `tsx` spawn of the CLI source entry runs well past vitest's 5 s default. */ +const SPAWN_TIMEOUT_MS = 120_000; + +/** The lint's ledger-integrity error — the whole answer both doors gave before the fix. */ +const SENTINEL = 'ledger entry has unrecognised status'; + +interface Run { + code: number; + stdout: string; + stderr: string; +} + +function runCli(args: string[], cwd: string): Promise { + return new Promise((resolvePromise) => { + execFile( + TSX, + [CLI, ...args], + { cwd, maxBuffer: 16 * 1024 * 1024, env: childEnv({ NO_COLOR: '1' }) }, + (err, stdout, stderr) => { + resolvePromise({ + code: err ? (typeof (err as { code?: unknown }).code === 'number' ? (err as unknown as { code: number }).code : 1) : 0, + stdout: String(stdout), + stderr: String(stderr), + }); + }, + ); + }); +} + +function payloadOf(run: Run, label: string): Record { + try { + return JSON.parse(run.stdout) as Record; + } catch { + throw new Error(`${label}: stdout was not one JSON document (exit ${run.code})\n${run.stdout}\n${run.stderr}`); + } +} + +/** The card's fixture: one object, one mapping that authors the pull binding. */ +function stack(objectExtra: Record = {}): Record { + return { + manifest: { id: 'com.example.fx-connector-source', name: 'fx', version: '0.1.0', type: 'app', namespace: 'fx' }, + objects: [ + { + name: 'fx_account', + label: 'Account', + sharingModel: 'private', + fields: { name: { type: 'text', label: 'Name' }, external_id: { type: 'text', label: 'External ID' } }, + ...objectExtra, + }, + ], + mappings: [ + { + name: 'fx_account_pull', + label: 'Account pull', + sourceFormat: 'json', + targetObject: 'fx_account', + mode: 'upsert', + fieldMapping: [ + { source: 'id', target: 'external_id', transform: 'none' }, + { source: 'name', target: 'name', transform: 'none' }, + ], + connectorSource: { connector: 'crm_api', action: 'request' }, + }, + ], + }; +} + +const dirs: Record = {}; +let root = ''; + +beforeAll(() => { + root = mkdtempSync(join(tmpdir(), 'os-validate-lint-connector-source-')); + const make = (label: string, s: Record) => { + const dir = join(root, label); + mkdirSync(dir, { recursive: true }); + writeFileSync(join(dir, 'objectstack.config.ts'), defineStackSource(s)); + linkSpec(dir); + dirs[label] = dir; + }; + make('card', stack()); + make('control', stack({ externalSharingModel: 'private' })); +}); + +afterAll(() => { + if (root) rmSync(root, { recursive: true, force: true }); +}); + +describe('#21127 — a stack authoring `mapping.connectorSource` validates and lints', () => { + it('`os validate --json` exits 0 and calls the stack valid', async () => { + const run = await runCli(['validate', '--json'], dirs.card); + const payload = payloadOf(run, 'validate'); + expect(`${run.stdout}${run.stderr}`).not.toContain(SENTINEL); + expect(run.code, `${run.stdout}\n${run.stderr}`).toBe(0); + expect(payload.valid).toBe(true); + }, SPAWN_TIMEOUT_MS); + + it('`os lint --json` exits 0, passes, and says nothing about the live binding', async () => { + const run = await runCli(['lint', '--json'], dirs.card); + const payload = payloadOf(run, 'lint'); + expect(`${run.stdout}${run.stderr}`).not.toContain(SENTINEL); + expect(run.code, `${run.stdout}\n${run.stderr}`).toBe(0); + expect(payload.passed).toBe(true); + const issues = (payload.issues ?? []) as Array<{ rule: string; message: string }>; + expect(issues.filter((i) => i.message.includes('connectorSource'))).toEqual([]); + }, SPAWN_TIMEOUT_MS); + + it('CONTROL: a `planned` row with `authorWarn` still warns at the same door', async () => { + const run = await runCli(['lint', '--json'], dirs.control); + const payload = payloadOf(run, 'lint (control)'); + expect(run.code, `${run.stdout}\n${run.stderr}`).toBe(0); + const issues = (payload.issues ?? []) as Array<{ rule: string; message: string; severity: string }>; + const planned = issues.filter((i) => i.rule === 'liveness-planned-property'); + expect(planned.map((i) => i.message).join(' | '), JSON.stringify(issues)).toContain('externalSharingModel'); + expect(planned.every((i) => i.severity === 'warning')).toBe(true); + expect(issues.filter((i) => i.message.includes('connectorSource'))).toEqual([]); + }, SPAWN_TIMEOUT_MS); +}); diff --git a/packages/lint/src/authoring-rules.ts b/packages/lint/src/authoring-rules.ts index 8ef62da7c88..902f75deaa2 100644 --- a/packages/lint/src/authoring-rules.ts +++ b/packages/lint/src/authoring-rules.ts @@ -1520,13 +1520,15 @@ export const AUTHORING_RULES: readonly AuthoringRule[] = [ // email_template.json` is 13 props / 0 warn keys and `mapping.json` was 7 / // 0 (lit control, same script, same dir: `tool.json` 6/1, `object.json` // 35/1), so these two writes dispatched this rule and it judged NOTHING. - // [#20919] `mapping.json` is 8 / 1 now: `connectorSource` (the connector - // sync binding) is `live` with `authorWarn`, because nothing schedules a - // pull until the `job` stage lands — so a `mapping` write that authors - // `connectorSource` is warned here, and one that does not is still judged - // silent. That is the ruled end state, not a half-landing: the ruling - // dispatched the wiring and ⛔ no ledger population («the empty warn maps - // stay empty until a real property needs a row — zero pull, the wiring is + // [#20919] `mapping.json` is 8 / 0 again: `connectorSource` (the connector + // sync binding) went `live` and, since #21127, carries no `authorWarn` — a + // warned `live` row made this rule throw instead of warn, so a `mapping` + // write authoring it got an `authoring-rule-threw` advisory and `os + // validate` / `os lint` exited 1. The scheduling caveat (nothing schedules + // a pull until the `job` stage lands) is on the key's description, and + // `check:liveness` refuses a warned `live` row. That is the ruled end + // state, not a half-landing: the ruling dispatched the wiring and ⛔ no + // ledger population («the empty warn maps stay empty until a real property needs a row — zero pull, the wiring is // the whole deliverable»). `runtime-gate.inert-type-writes.test.ts` pins // both halves — that the rule is dispatched, and that it is silent — so // the day a ledger row lands the door lights up with no second edit here. diff --git a/packages/lint/src/lint-liveness-properties.test.ts b/packages/lint/src/lint-liveness-properties.test.ts index 10f9d005e4b..5f49a400fe0 100644 --- a/packages/lint/src/lint-liveness-properties.test.ts +++ b/packages/lint/src/lint-liveness-properties.test.ts @@ -1826,3 +1826,53 @@ describe('the hint a verdict-triggered row shows an author (#16094)', () => { } }); }); + +// ── #21127: the shipped ledgers carry no warned `live` row ── +// +// `mapping.connectorSource` was re-graded `live` with its `authorWarn` kept, so +// `describe()` threw its ledger-integrity error (asserted LOUD by design in +// the COVERAGE block above) and `os validate` / `os lint` exited 1 on every +// stack that authored the binding. The row now carries no warning, and +// `check:liveness` refuses the combination at every depth; these pins hold the +// author's side of it against the REAL ledgers. +describe('#21127 — a stack authoring `mapping.connectorSource` lints clean, and a warned `planned` row still warns', () => { + // The card's fixture: one object, one mapping that authors the binding. + const FX_ACCOUNT = { + name: 'fx_account', + label: 'Account', + fields: { name: { type: 'text', label: 'Name' }, external_id: { type: 'text', label: 'External ID' } }, + }; + const FX_PULL = { + name: 'fx_account_pull', + label: 'Account pull', + sourceFormat: 'json', + targetObject: 'fx_account', + mode: 'upsert', + fieldMapping: [ + { source: 'id', target: 'external_id', transform: 'none' }, + { source: 'name', target: 'name', transform: 'none' }, + ], + connectorSource: { connector: 'crm_api', action: 'request' }, + }; + + it('REAL LEDGER: the card\'s fixture lints without throwing, and nothing is said about the live binding', () => { + expect(authorWarnedProperties('mapping').has('connectorSource')).toBe(false); + const findings = lintLivenessProperties({ objects: [FX_ACCOUNT], mappings: [FX_PULL] }); + expect(findings.filter((f) => f.message.includes('connectorSource'))).toEqual([]); + // Anti-vacuity: an unreadable ledger also yields no finding, and says so + // under its own rule id — which must not be what made this quiet. + expect(findings.filter((f) => f.rule === LIVENESS_LEDGER_UNREADABLE)).toEqual([]); + }); + + it('REAL LEDGER (the control): a `planned` row with `authorWarn` still warns, in the same call', () => { + // `object.externalSharingModel` is `planned` + `authorWarn` in tree — the + // legal shape: a consumer is being built, keep the key, it does nothing yet. + const findings = lintLivenessProperties({ + objects: [{ ...FX_ACCOUNT, externalSharingModel: 'private' }], + mappings: [FX_PULL], + }); + const planned = findings.filter((f) => f.rule === 'liveness-planned-property'); + expect(planned.map((f) => f.message).join(' | ')).toContain('externalSharingModel'); + expect(findings.filter((f) => f.message.includes('connectorSource'))).toEqual([]); + }); +}); diff --git a/packages/lint/src/runtime-gate.inert-type-writes.test.ts b/packages/lint/src/runtime-gate.inert-type-writes.test.ts index 8b7d87d5ff0..ec63404d57a 100644 --- a/packages/lint/src/runtime-gate.inert-type-writes.test.ts +++ b/packages/lint/src/runtime-gate.inert-type-writes.test.ts @@ -726,9 +726,10 @@ describe('#19542 — group C: wired, dispatched, and silent by ledger (email_tem // remembered. The ruling dispatched the wiring and ⛔ no ledger population, // so this silence is the ruled end state. The day a property earns an // `authorWarn` row the door lights up with no second edit — which is what - // the dispatch pin above is for. [#20919] `mapping` has one such row now, - // `connectorSource` (nothing schedules a pull until the `job` stage), and - // the item below does not author it. + // the dispatch pin above is for. [#20919] `mapping` had one such row, + // `connectorSource`, until #21127 dropped it: the key is `live`, and a + // warned `live` row made this rule throw (the case below pins the binding + // judged silent at this door). const item = type === 'email_template' ? { name: 'acme_welcome', label: 'Welcome', subject: 'Hi', bodyHtml: '

Hi

', category: 'workflow' } : { name: 'acme_feed', label: 'Feed', sourceFormat: 'csv', targetObject: 'acme_invoice', mode: 'upsert' }; @@ -738,6 +739,31 @@ describe('#19542 — group C: wired, dispatched, and silent by ledger (email_tem expect(result.advisories, dump(result)).toEqual([]); }); + it('[#21127] a `mapping` write that DOES author `connectorSource` is judged silent — not thrown on', () => { + // `connectorSource` is `live`: the connector sync executor reads every key, + // so authoring it warns nothing. While its ledger row still carried + // `authorWarn`, this write came back with an `authoring-rule-threw` + // advisory — the liveness rule's ledger-integrity error, not a verdict + // about the body. The binding is the card's; the target is this file's. + const result = runRuntimeAuthoringRules({ + type: 'mapping', + item: { + name: 'acme_invoice_pull', + label: 'Invoice pull', + sourceFormat: 'json', + targetObject: 'acme_invoice', + mode: 'upsert', + fieldMapping: [{ source: 'state', target: 'status', transform: 'none' }], + connectorSource: { connector: 'crm_api', action: 'request' }, + }, + context: CONTEXT, + }); + + expect(result.rulesRun).toContain('lintLivenessProperties'); + expect(result.errors, dump(result)).toEqual([]); + expect(result.advisories, dump(result)).toEqual([]); + }); + it('⭐ LIT CONTROL — the same instrument, in this same process, DOES fire on a ledger that warns', () => { // Without this the two zeros above would be unreadable: a silent rule and // an unresolvable ledger directory look identical from the outside. This diff --git a/packages/spec/docs/SYNC_ARCHITECTURE.md b/packages/spec/docs/SYNC_ARCHITECTURE.md index 832ab3a7ff7..5134794c036 100644 --- a/packages/spec/docs/SYNC_ARCHITECTURE.md +++ b/packages/spec/docs/SYNC_ARCHITECTURE.md @@ -230,7 +230,8 @@ Complete, production-grade integration with external systems. Includes authentic > the pages it never read. Point `connectorSource` at an endpoint that answers the > whole (incremental) set in one response. > - ⚠️ **Nothing schedules a pull yet.** A `job` drives it, and that stage has not -> landed, so authoring `connectorSource` still warns. +> landed, so the binding alone moves no rows — `connectorSource`'s own description +> says so. It is not a lint warning: every key of the binding is `live`. > Already authored the retired keys? `os migrate meta --from 17` lists the mechanical edits. ### Use Cases diff --git a/packages/spec/liveness/README.md b/packages/spec/liveness/README.md index 0611828490c..68d1a4164f3 100644 --- a/packages/spec/liveness/README.md +++ b/packages/spec/liveness/README.md @@ -569,6 +569,14 @@ Two rules keep it false-positive-free, **both of which the marker author must re `enable` block — so leave those unmarked (see `enable.searchable`'s `_authorWarnSkipped`). Object/string/array props warn when merely present, so this caveat is boolean-only. +And one the gate enforces for you: **never on a `live` row.** `live` says authoring the key +changes runtime behaviour, so there is nothing to warn about — and the lint has no verdict +for it: its `describe()` throws a ledger-integrity error on a warned `live` row, which makes +`os validate` and `os lint` exit 1 on every stack that authors the key. `check:liveness` +refuses the combination at every depth and prints the warned-row census on every run. A +caveat an author still needs about a live key (a cadence nothing schedules yet) goes in the +key's `.describe()`. + The lint is ledger-driven: coverage grows by marking more entries `authorWarn`, not by touching the lint code. It covers **every governed type**: objects (incl. `enable.*`) and their fields walk bespoke nesting; flows/actions/agents/tools/skills/datasets/ @@ -929,7 +937,7 @@ marker where the Notes cell goes, never a guess at what belongs there. | doc | seeded 2026-08-01 (#4488). Fully live: the kernel stores `content` unparsed, but the REST read layer localizes (resolveDocLocale), audience-gates, list-strips `content`, and the book resolver consumes name/label/description/order/group — plus the objectui console portal renders it all. The schema's own "docs are inert data" header describes the kernel, not the type. **`tags` DECLARED in 17.0.0 (#4509)** — the enforce half of enforce-or-remove: the book resolver's `include: { tag }` matcher, the REST transport and `ResolverDoc.tags` all already existed, but DocSchema is strict and had no `tags` key, so authoring one was a parse error and the variant could never match. Live on arrival | | email_template | this row read 8/–/13/– for one day (seeded 2026-08-01, #4488: "every authorable property is dead", the webhook shape on AUTH mail) and #4509 CLOSED it by ENFORCING — the second worked example, after `webhook`, that a dead verdict is a worklist entry rather than a tombstone. `bootstrapDeclaredEmailTemplates` materializes declared items into the `sys_email_template` rows `sendTemplate` reads, sharing `mapTemplateToRow` with the built-in seeder so the two doors cannot drift, and re-materializes on live metadata writes (`email_template` is `allowRuntimeCreate: true`, so boot-only would have left Studio saves inert). Three breaks had to close, not one: the engine never registered `emailTemplates:` into the registry, built-in seeds masqueraded as `managed_by: admin` and outranked declared templates, and nothing materialized. ADR-0054 proof bound on `subject` (`email-template-materialization`) | | job | seeded 2026-08-01 (#4488). The file-authored path is fully enforced: all three schedule shapes honored by the adapters, `retryPolicy`/`timeout` enforced since #3494 (this is the retryPolicy the datasource ledger warns about confusing with its dead namesake), `enabled: false` skips scheduling. Dead 3 = `id` (authorWarn — `name` is the identity everywhere) + label/description (docs-kept). The type-level gap CLOSED 2026-08-02 (#4509) by closing the door rather than bridging it: `handler` names a function in the compiled bundle's function table, which a runtime writer cannot name, so `allowRuntimeCreate` **and** `allowOrgOverride` are now false and `*.job.ts` / `defineStack({ jobs })` are the supported doors. The kind stays registered — its file loader is genuinely consumed (ADR-0088 admission test) **#4667**: `id` REMOVED (row deleted, strict removal) — nothing read it and its own describe() ("defaults to `name` when omitted") advertised an identity override that never existed; `name` is the scheduling key, the sys_job row key and the JobExecution.jobId stamp, so two jobs differing only in `id` were one job. **#7131** (PR #7425) takes the remaining two: `label` and `description` re-grade `dead` → `live` under the 2026-08-10 maintainer ruling that **designer previews count as consumers** — objectui's `JobPreview` had been reading `d.label` and `d.description` and rendering them as the preview card's title and subtitle the whole time, so the old "no runtime consumer" was a true statement about the *scheduler* and a false one about the system. **This row now has zero dead and the ADR-0033 exemption is still in force**, which is worth saying out loud because it is the first row in this table where those two facts hold together: the keys are still docs-shaped, still deliberately KEPT, still not `authorWarn`'d, and enforce-or-remove still has nothing to chase here. What changed is only that the exemption no longer has to carry the verdict — the measurement does. | -| mapping | seeded 2026-08-01 (#4488) at 8/11 live; **0 dead since #4509** retired the three that were not. The import half (#2611) is loudly enforced — unsupported transforms/formats are 400s, `mode`/`upsertKey` default the request, the wizard picker renders `label`. RETIRED 17.0.0: `extractQuery` (authorWarn — "for export only" promised an export path no exporter implements) + `errorPolicy`/`batchSize`, which were dead AND **unwarnable** (schema defaults materialize at parse, so presence ≠ authored — `_authorWarnSkipped`, the non-boolean instance of the default(true) rule). That unwarnability is why they went out in the 17.0.0 window rather than after a deprecation cycle: removal was the only channel that could ever reach the author. Rows DELETED, not tombstoned — MappingSchema is strict, so the keys left the walked shape. `connectorSource` (2026-09-30) is this type's first `planned` row: the pull binding the connector-attached sync family MOVED here when the connector's `syncConfig` / `fieldMappings` retired to tombstones, seeded contract-first with `authorWarn` (its seven drilled rows — five children and `watermark`'s two — `planned` with it) because the pull executor is the next stage | +| mapping | seeded 2026-08-01 (#4488) at 8/11 live; **0 dead since #4509** retired the three that were not. The import half (#2611) is loudly enforced — unsupported transforms/formats are 400s, `mode`/`upsertKey` default the request, the wizard picker renders `label`. RETIRED 17.0.0: `extractQuery` (authorWarn — "for export only" promised an export path no exporter implements) + `errorPolicy`/`batchSize`, which were dead AND **unwarnable** (schema defaults materialize at parse, so presence ≠ authored — `_authorWarnSkipped`, the non-boolean instance of the default(true) rule). That unwarnability is why they went out in the 17.0.0 window rather than after a deprecation cycle: removal was the only channel that could ever reach the author. Rows DELETED, not tombstoned — MappingSchema is strict, so the keys left the walked shape. `connectorSource` (2026-09-30) is this type's first `planned` row: the pull binding the connector-attached sync family MOVED here when the connector's `syncConfig` / `fieldMappings` retired to tombstones, seeded contract-first with `authorWarn` (its seven drilled rows — five children and `watermark`'s two — `planned` with it) because the pull executor is the next stage. **Live since 2026-10-01 (#20919)**: the connector sync executor reads every key. The container's `authorWarn` left with that re-grade (#21127) — kept at first, it made the author-side lint throw, so `os validate` / `os lint` exited 1 on every stack that authored the binding; the caveat it carried (nothing schedules a pull until the `job` stage lands) is on the key's description | | picklist | seeded 2026-09-30 (#19518) with every key `planned`, ahead of its runtime reader by ruling; **live since the runtime layer (#19519)**: the ObjectQL registry merges `picklistExtensions` into the list (additive only, a repeated value refused) and resolves it onto every field that names it, the write door judges against that set, and a field naming an undeclared list fails the boot. `field.picklist` left `authorWarn` with it. Display keys (`label`, `description`, `options.color`) are graded by their served form. | | seed | seeded 2026-08-01 (#4488). Fully live via SeedLoaderService on both doors (boot/per-org replay + runtime-draft publish). `records` is the z.record walk boundary: the keys an author writes are the target object's fields, governed by that object's own definitions — recorded in the entry, not silently skipped | | translation | seeded 2026-08-01 (#4488) — after fixing the walker: the registered schema is a z.preprocess pipe (#3778 retired-dialect guard) whose transform side the unwrap always took, so the type was literally unwalkable. 11 of 12 groups live across spec resolvers, REST localization, objectui client resolvers and plugin-audit (whose composed-key `t()` calls make `messages` easy to mis-verify as dead) — `flows` was the one that was not, and was `planned` at seeding. **#14253** added the twelfth, `datasets`, seeded LIVE and DRILLED (label / description / dimensions / measures) with its reader in the same change: `translateDataset` in the dispatch table, which is what `TRANSLATABLE_METADATA_TYPES` is derived from, so the REST boundary followed with nothing else to remember. The same change gave `objects.._views..bulkActions` and `objects.._validations..message` their first keys — both beneath the walk boundary, so neither adds a row here. Dead 1 = `validationMessages` (authorWarn) at seeding: nothing resolved it, and #3778's own legacy-key migration table steered `errors:` authors into it — a shipped false signpost, the capabilities.readOnly shape. **#4667**: `validationMessages` REMOVED (row deleted) — removed from the shared translationDataShape(), so it retired at BOTH doors at once, closing the item-only asymmetry #3778's original guard had. #3778's own `errors` guidance was rewritten in the same change: it had been steering authors INTO this dead group. ⚠️ **What that left behind is this table's own worked example of the defect it warns about** (#7377): the same commit that deleted the `validationMessages` row wrote a count column of `dead 2` beside a sentence that named exactly one dead key — and that one was the key it had just removed. The real two were `name` and `label`, which the cell never mentioned. Measured at that commit, not inferred: the ledger's dead set there is `{name, label}` and `validationMessages` is absent from `props`. The number was right and the prose was false, in the same cell, on the day it was written — which is why the counts are now generated and this cell holds prose only. **#7131** (PR #7425) resolves it: `name` and `label` re-grade `dead` → `live` under the designer-previews-count-as-consumers ruling (objectui `TranslationPreview.tsx:67` reads `label` first and falls back to `name`, both rendering at `:100`), so the dead set is empty and there is no dead-set sentence left to keep true. As on `job`, the ADR-0033 docs-shaped exemption is untouched — nothing about enforce-or-remove moved. **#19620** (ruling batch #210 item 2 letter B): `settings` row DELETED — the strict-delete route, because `TranslationItemSchema` no longer declares the key and refuses it by name (the item door now takes the per-app face, as the file door has since #15178). ⚠️ The deleted row read `live`, and that verdict was TRUE and stays true of the platform: its evidence read the SERVED tree, which the platform bundle feeds, so the deletion retires the key from the application-authored item and nothing else — the capability lives on `PlatformTranslationDataSchema`, outside this ledger. **#20296**: `flows` is now half-read. `flows.screens` re-graded `planned` → `live`: objectui's FlowRunner reads each screen's `title` and each field's `label` / `placeholder` at the `.objectui-sha` pin f8a9d0fb. `flows.label` stays `planned` because nothing reads it yet (#20318), and so does the container's `authorWarn`, whose `authorHint` now names the read half and the unread one. | diff --git a/packages/spec/liveness/mapping.json b/packages/spec/liveness/mapping.json index afc4d56ddf3..49cc72e394b 100644 --- a/packages/spec/liveness/mapping.json +++ b/packages/spec/liveness/mapping.json @@ -48,9 +48,7 @@ "status": "live", "verifiedAt": "2026-10-01", "evidence": "packages/services/service-automation/src/connector-pull.ts#pullConnectorSource (reads the binding through the protocol's `getMetaItem({ type: 'mapping', name })`, makes ONE action call on the declared connector, and writes through `@objectstack/core`'s `runImport` — the import door's runner)", - "authorWarn": true, - "authorHint": "`connectorSource` is pulled when a job drives it; nothing schedules it until stage ③ — schedule the pull with a `job` once that stage lands.", - "note": "Seeded 2026-09-30 `planned`, contract-first — the target-side half of the connector-attached sync family, ruled ENFORCE on the maintainer's criterion with the definition MOVED here from the retired `connector.syncConfig` / `connector.fieldMappings` (both `dead` tombstone rows in connector.json): every mainstream platform binds a sync to its TARGET, and this type already carries the executed target half (`targetObject`, `fieldMapping`, `mode`, `upsertKey`, all live). `connectorSource` adds only where the rows come from — a `rest` / `openapi` connector instance, the action that reads them and an optional timestamp watermark; version 1 is a one-way pull, full or incremental, credentials are the instance's own ADR-0097 static `auth`, and the cadence is a `job` (no schedule key, by the 2026-09-10 ruling). LIVE since 2026-10-01 (#20919, stage ②): `@objectstack/service-automation`'s connector sync executor reads every key below and writes through the import runner, which moved to `@objectstack/core` so the executor writes through the same runner, coercion and row verdicts as the import door. `authorWarn` STAYS, reworded: nothing schedules a pull until the `job` stage lands (stage ③), so an author who writes the binding today is told the pull runs only when a job drives it — the whole difference between this row and the silent `syncConfig` it replaces. Drop the warning when the job stage lands.", + "note": "Seeded 2026-09-30 `planned`, contract-first — the target-side half of the connector-attached sync family, ruled ENFORCE on the maintainer's criterion with the definition MOVED here from the retired `connector.syncConfig` / `connector.fieldMappings` (both `dead` tombstone rows in connector.json): every mainstream platform binds a sync to its TARGET, and this type already carries the executed target half (`targetObject`, `fieldMapping`, `mode`, `upsertKey`, all live). `connectorSource` adds only where the rows come from — a `rest` / `openapi` connector instance, the action that reads them and an optional timestamp watermark; version 1 is a one-way pull, full or incremental, credentials are the instance's own ADR-0097 static `auth`, and the cadence is a `job` (no schedule key, by the 2026-09-10 ruling). LIVE since 2026-10-01 (#20919, stage ②): `@objectstack/service-automation`'s connector sync executor reads every key below and writes through the import runner, which moved to `@objectstack/core` so the executor writes through the same runner, coercion and row verdicts as the import door. `authorWarn` was kept at that re-grade and dropped on 2026-10-01 (#21127): a `live` row carries no author warning — the author-side lint has no verdict for one and throws its ledger-integrity error, so `os validate` and `os lint` exited 1 on every stack that authored the binding (`check:liveness` refuses the combination now). The caveat the warning carried — nothing schedules a pull until the `job` stage lands (stage ③), so a pull runs only when a job drives it — is on the key's `.describe()`, where an author reads it; stage ③ removes it there.", "children": { "connector": { "status": "live", diff --git a/packages/spec/scripts/liveness/check-liveness.mts b/packages/spec/scripts/liveness/check-liveness.mts index 0ea9e57c68c..cae282b9fff 100644 --- a/packages/spec/scripts/liveness/check-liveness.mts +++ b/packages/spec/scripts/liveness/check-liveness.mts @@ -712,6 +712,8 @@ const report: any = { orphanEntries: [] as string[], // a ledger row whose property is gone from the schema (the reverse direction) tombstonedLive: [] as string[], // a `retiredKey()` tombstone whose row still claims a forbidden status (#19062) tombstones: [] as string[], // every `[REMOVED]` tombstone the walk reached — enumerated, not totalled, so the population is checkable (#18133's reason) + authorWarnRows: [] as string[], // every row, at every depth, that opts into `authorWarn` — `/ ()`, enumerated for the same reason + authorWarnOnLive: [] as string[], // ...of which these are graded `live` — the author-side lint throws on them, so they FAIL authorable: [] as string[], // the governance DENOMINATOR itself — printed and emitted so "never looked" cannot pass for "nothing to report" (#18133) authorableRegistered: 0, // how many of it are registered KINDS (listMetadataTypeSchemaTypes) authorableUnregisteredKinds: [] as string[], // …and which are unregistered-kind stack collections (#6245/#6931) @@ -1083,11 +1085,74 @@ function drillChildren( } } +// ── author warnings: a `live` row never opts in (#21127) ── +// +// `authorWarn: true` asks the author-side lint +// (packages/lint/src/lint-liveness-properties.ts) to warn whoever authors the +// key, and the lint picks the verdict it shows from the row's STATUS: +// `describe()` there answers experimental / planned / dead / live-elsewhere and +// THROWS on `live`, deliberately — a live key has its runtime effect, so there +// is nothing to warn about, and a warned `live` row is a shipped-ledger +// integrity bug. Thrown from inside a lint, that integrity error is the whole +// answer `os validate` and `os lint` give (exit 1) on EVERY stack that authors +// the key, and the runtime metadata door returns it as an +// `authoring-rule-threw` advisory. `mapping.connectorSource` shipped exactly +// that row: re-graded `live` with its `authorWarn` kept. +// +// Nothing here caught it, because the combination is legal for every other +// check: the status is in the vocabulary, the evidence resolves, and the walk +// above never even reads the row — a container that drills into `children` +// has only its CHILDREN graded, while the lint reads the container's own row +// (and its direct children) for `authorWarn`. So this pass walks the RAW rows, +// at every depth `children` nests, rather than the graded population. +// +// Census before switching it on, across all 41 governed ledgers on 665cab338f: +// 924 rows walked, 3 opt in with `authorWarn` — `planned` 2, `live` 1 (the +// `mapping.connectorSource` row this change re-grades). So the rule starts +// green on the population it measured, and only a NEW warned `live` row can +// red it — the zero-census argument the tombstone join was switched on under. +function scanAuthorWarnRows(type: string, rows: Record, prefix = ''): void { + for (const [key, row] of Object.entries(rows)) { + if (!row || typeof row !== 'object') continue; + const path = prefix ? `${prefix}.${key}` : key; + if (row.authorWarn === true) { + report.authorWarnRows.push(`${type}/${path} (${row.status})`); + if (row.status === 'live') report.authorWarnOnLive.push(`${type}/${path}`); + } + if (row.children && typeof row.children === 'object') scanAuthorWarnRows(type, row.children, path); + } +} + +/** + * The prescription printed under the warned-`live` list. The tempting wrong + * fix is the lint's sentinel, so it is ruled out in the same breath. + */ +const AUTHOR_WARN_ON_LIVE_GUIDANCE = [ + 'A `live` row says authoring the key changes runtime behaviour, so an author', + 'has nothing to be warned about — and the author-side lint has no verdict for', + 'it: `describe()` in packages/lint/src/lint-liveness-properties.ts throws its', + 'ledger-integrity error on a warned `live` row, so `os validate` and `os lint`', + 'exit 1 on every stack that authors the key instead of warning.', + '', + 'Fix the ROW: drop `authorWarn` (and its `authorHint`). A caveat an author still', + 'needs about a live key — a cadence nothing schedules yet, a limit of the', + 'executor — goes in the key\'s `.describe()`, where every author reads it.', + '', + 'If nothing reads the key yet, the STATUS is what is wrong: grade it `planned`', + '(a consumer is being built against it), `experimental` or `dead`, and keep', + 'the warning.', + '', + '⛔ Do NOT teach `describe()` a `live` branch: its throw is the loud answer to', + 'exactly this row, and a warning on a working key is noise an author learns', + 'to skim.', +]; + for (const type of GOVERNED) { const ledger = loadLedger(type); const props = ledger.props || {}; const cat = { classified: 0, unclassified: 0, byStatus: {} as Record }; const walked = topProps(type); + scanAuthorWarnRows(type, props); // ── reverse direction: a row whose property is gone (see orphans.mts) ── // Runs off the SAME walk the forward pass classifies against, so the two @@ -1423,6 +1488,11 @@ const failed = // only a NEW false claim can red the gate. A check that starts at (nearly) // zero can be red; that is why the census came first. report.tombstonedLive.length > 0 || + // A `live` row that opts into `authorWarn` (#21127). Red from day one on the + // census stated at scanAuthorWarnRows: the one such row is re-graded by the + // change that switched this on, so only a NEW one can red it — and each one + // is a crash of `os validate` / `os lint` waiting for the first author. + report.authorWarnOnLive.length > 0 || report.verification.errors.length > 0 || report.producers.errors.length > 0 || report.producerMissing.length > 0 || @@ -1718,6 +1788,14 @@ if (asJson) { console.log(''); TOMBSTONE_STATUS_GUIDANCE.forEach((line) => console.log(line ? ` ${line}` : '')); } + if (report.authorWarnOnLive.length) { + console.log( + `\n✗ ${report.authorWarnOnLive.length} \`live\` ledger row(s) opt into \`authorWarn\` — the author-side lint throws on them:`, + ); + report.authorWarnOnLive.forEach((s: string) => console.log(` ${s}`)); + console.log(''); + AUTHOR_WARN_ON_LIVE_GUIDANCE.forEach((line) => console.log(line ? ` ${line}` : '')); + } if (report.undrilledNew.length) { console.log(`\n✗ ${report.undrilledNew.length} UNDECLARED container inheritance — a blanket verdict covers keys nothing classified:`); report.undrilledNew.forEach((s: string) => console.log(` ${s}`)); @@ -1884,6 +1962,21 @@ if (asJson) { `${report.tombstones.length - report.tombstonedLive.length} graded with a status the tombstone allows` + (report.tombstonedLive.length ? `, ${report.tombstonedLive.length} FORBIDDEN` : '') + '.', ); + // ── author warnings: asked at every depth, and how many sit on a `live` row ── + // Two numbers again: "0 on a live row" reads identically whether no row is + // wrong or the raw-row walk reached nothing. + const warnedByStatus = new Map(); + for (const row of report.authorWarnRows as string[]) { + const status = /\(([^)]*)\)$/.exec(row)?.[1] ?? '?'; + warnedByStatus.set(status, (warnedByStatus.get(status) ?? 0) + 1); + } + const warnedParts = [...warnedByStatus].sort(([a], [b]) => a.localeCompare(b)).map(([s, n]) => `${s} ${n}`); + console.log( + `\nauthor warnings: ${report.authorWarnRows.length} ledger row(s) opt into \`authorWarn\`, at any depth` + + (warnedParts.length ? ` (${warnedParts.join(', ')})` : '') + + `; ${report.authorWarnOnLive.length} on a \`live\` row` + + (report.authorWarnOnLive.length ? ' — FORBIDDEN' : '') + '.', + ); // ── container coverage: how much rides on inheritance? ── // Printed every run, pass or fail. The gate used to say "all properties are @@ -1942,7 +2035,7 @@ if (asJson) { '\n✓ every governed-type property, at every depth the ledger drills, is classified, every ' + 'authorable type — registered kind or unregistered-kind stack collection — is governed or ' + 'explicitly pending, no ledger row outlives its property, no tombstoned key\'s row claims a ' + - 'status the tombstone forbids, ' + + 'status the tombstone forbids, no `live` row opts into an author warning, ' + `every container inheritance is declared, every ${EVIDENCE_SCANNED_LABEL} entry's repo-local evidence path ` + 'resolves, every `path:NNN` citation names a line that file actually has, every ' + '`path#symbol` anchor names a symbol its file contains, and every cited ' + diff --git a/packages/spec/scripts/liveness/check-liveness.test.ts b/packages/spec/scripts/liveness/check-liveness.test.ts index cfb65e438ec..c2a719d564f 100644 --- a/packages/spec/scripts/liveness/check-liveness.test.ts +++ b/packages/spec/scripts/liveness/check-liveness.test.ts @@ -1556,3 +1556,118 @@ describe('check:liveness — a tombstoned key may not be graded `live` (#19062)' expect(line).not.toContain('FORBIDDEN'); }); }); + +// ── #21127: a `live` row never opts into `authorWarn` ── +// +// The author-side lint (`packages/lint/src/lint-liveness-properties.ts`) picks +// the verdict it shows from a warned row's STATUS, and its `describe()` throws +// on `live` by design — so a warned `live` row turns `os validate` / `os lint` +// into exit 1 for every stack that authors the key. `mapping.connectorSource` +// shipped that row, a CONTAINER that drills into `children`, which is exactly +// the row the graded walk never reads. Every case runs the REAL gate via +// `--ledger-root`, for the #5623 reason the blocks above state. +// +// The carriers are derived from the shipped ledgers by SHAPE — a top-level +// `live` container row with `children`, and a `live` drilled child — never named, +// for the #19062 reason: a named row is a moving verdict. Adding `authorWarn: +// true` to a `live` row moves no status, so the count shards stay byte-identical +// and the red run has exactly one cause. + +/** What a derived carrier is: a coordinate in one shipped ledger. */ +interface WarnCarrier { + type: string; + /** `prop` or `prop.child` — the row the fixture marks. */ + path: string; +} + +/** Every `(type, path)` the shipped ledgers carry with a given shape, sorted. */ +function liveRows(shape: 'container' | 'child'): WarnCarrier[] { + const out: WarnCarrier[] = []; + for (const file of readdirSync(LEDGERS).filter((f) => f.endsWith('.json')).sort()) { + const type = file.slice(0, -'.json'.length); + const props: Record = JSON.parse(readFileSync(path.join(LEDGERS, file), 'utf8')).props ?? {}; + for (const [prop, row] of Object.entries(props)) { + if (shape === 'container' && row?.status === 'live' && row?.children && row?.authorWarn !== true) { + out.push({ type, path: prop }); + } + if (shape === 'child') { + for (const [child, crow] of Object.entries(row?.children ?? {})) { + if (crow?.status === 'live' && !crow?.children && crow?.authorWarn !== true) { + out.push({ type, path: `${prop}.${child}` }); + } + } + } + } + } + return out; +} + +describe('check:liveness — a `live` row may not opt into `authorWarn` (#21127)', () => { + let tmp: string; + + beforeAll(() => { + tmp = mkdtempSync(path.join(tmpdir(), 'os-liveness-authorwarn-')); + }); + afterAll(() => rmSync(tmp, { recursive: true, force: true })); + + /** Copy the shipped ledgers and mark one row `authorWarn: true`, read back from disk. */ + function sampleWarned(name: string, carrier: WarnCarrier): string { + const root = path.join(tmp, name); + cpSync(LEDGERS, root, { recursive: true }); + const file = path.join(root, `${carrier.type}.json`); + const ledger = JSON.parse(readFileSync(file, 'utf8')); + const [prop, child] = carrier.path.split('.'); + const row = child ? ledger.props[prop].children[child] : ledger.props[prop]; + row.authorWarn = true; + writeFileSync(file, `${JSON.stringify(ledger, null, 2)}\n`); + // Proof the edit reached disk — an editor's exit code is not evidence. + const back = JSON.parse(readFileSync(file, 'utf8')).props[prop]; + expect((child ? back.children[child] : back).authorWarn, `${carrier.type}/${carrier.path} not marked on disk`).toBe(true); + return root; + } + + // THE CONTROL, and the census. Green on a verbatim copy, and the line names + // the population it asked: a `planned` row with `authorWarn` is the legal + // shape (a consumer is being built — keep the key, it does nothing yet), so + // the shipped ledgers carry some and the gate leaves them alone. + it('is green on the shipped ledgers, which carry warned `planned` rows and no warned `live` one', () => { + const { status, output } = runGate(); + expect(status, output).toBe(0); + expect(output).toContain('no `live` row opts into an author warning'); + const line = output.split('\n').find((l) => l.startsWith('author warnings:')) ?? ''; + expect(line, 'the gate must publish the population it asked').not.toBe(''); + const reached = Number(/^author warnings: (\d+) /.exec(line)?.[1] ?? 0); + expect(reached, 'a walk reaching zero warned rows is a degraded walk, not a clean tree').toBeGreaterThan(0); + expect(line).toMatch(/\bplanned [1-9]\d*\b/); + expect(line).toContain('; 0 on a `live` row.'); + expect(line).not.toContain('FORBIDDEN'); + }); + + // The regression's own shape: a `live` CONTAINER row, whose status the graded + // walk never reads because only its children are classified. + it('FAILS when a `live` container row that drills into `children` opts in', () => { + const [carrier] = liveRows('container'); + expect(carrier, 'no top-level `live` row with `children` in the shipped ledgers to carry the sample').toBeDefined(); + + const { status, output } = runGate(sampleWarned('container', carrier)); + expect(status, output).toBe(1); + expect(output).toContain('✗ 1 `live` ledger row(s) opt into `authorWarn` — the author-side lint throws on them:'); + expect(output).toContain(` ${carrier.type}/${carrier.path}\n`); + // The prescription travels with the finding, and rules out the wrong fix. + expect(output).toContain('Do NOT teach `describe()` a `live` branch'); + // ONE cause: a second ✗ block would mean the sample dragged another check in. + expect(output.split('\n').filter((l) => l.startsWith('✗')), output).toHaveLength(1); + }); + + // The lint reads a container's direct children too, and the walk goes to any + // depth `children` nests — a drilled `live` row is the same crash. + it('FAILS when a drilled `live` child row opts in', () => { + const [carrier] = liveRows('child'); + expect(carrier, 'no drilled `live` child row in the shipped ledgers to carry the sample').toBeDefined(); + + const { status, output } = runGate(sampleWarned('child', carrier)); + expect(status, output).toBe(1); + expect(output).toContain(` ${carrier.type}/${carrier.path}\n`); + expect(output.split('\n').filter((l) => l.startsWith('✗')), output).toHaveLength(1); + }); +}); diff --git a/packages/spec/src/data/mapping-connector-source.test.ts b/packages/spec/src/data/mapping-connector-source.test.ts index 23b25b3e90e..85851561944 100644 --- a/packages/spec/src/data/mapping-connector-source.test.ts +++ b/packages/spec/src/data/mapping-connector-source.test.ts @@ -15,9 +15,11 @@ * What is pinned here is the CONTRACT — the shape, its closed door and the * keys it deliberately does not carry. The pull executor * (`@objectstack/service-automation`'s `pullConnectorSource`, #20919) reads - * the binding, so the ledger rows are `live`; nothing schedules a pull until - * the `job` stage lands, so the container keeps `authorWarn` - * (`liveness/mapping.json`), which the last block pins. + * the binding, so the ledger rows are `live` and carry no `authorWarn` + * (`liveness/mapping.json`) — a warned `live` row made the author-side lint + * throw, so `os validate` / `os lint` crashed on every stack authoring the + * binding (#21127). Nothing schedules a pull until the `job` stage lands, and + * the key's own description says so. The last block pins both. */ import fs from 'node:fs'; @@ -175,10 +177,10 @@ describe('mapping.connectorSource — the closed door and what it does not carry }); }); -describe('mapping.connectorSource — executed when a job drives it, scheduled by nothing yet: the ledger says so', () => { +describe('mapping.connectorSource — executed when a job drives it, scheduled by nothing yet: the ledger and the schema say so', () => { const LEDGER = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../liveness/mapping.json'); - it('every key of the binding is `live`, citing the executor, and the container still warns the author', () => { + it('every key of the binding is `live`, citing the executor, and no row of it opts into an author warning', () => { type Row = { status: string; evidence?: string; authorWarn?: boolean; authorHint?: string; children?: Record }; const ledger = JSON.parse(fs.readFileSync(LEDGER, 'utf8')) as { props: Record }; const EXECUTOR = 'packages/services/service-automation/src/connector-pull.ts#pullConnectorSource'; @@ -186,9 +188,20 @@ describe('mapping.connectorSource — executed when a job drives it, scheduled b expect(row, 'connectorSource must have a ledger row').toBeDefined(); expect(row!.status).toBe('live'); expect(row!.evidence).toContain(EXECUTOR); - // Nothing schedules a pull until the `job` stage lands, so the warning stays. - expect(row!.authorWarn).toBe(true); - expect(row!.authorHint).toContain('nothing schedules it'); + // A `live` row carries no `authorWarn`: the author-side lint has no verdict + // for one and throws, so authoring the binding crashed `os validate` and + // `os lint` (#21127). The scheduling caveat lives on the description. + expect(row!.authorWarn).toBeUndefined(); + expect(row!.authorHint).toBeUndefined(); + const warned: string[] = []; + const walk = (rows: Record, prefix: string) => { + for (const [key, r] of Object.entries(rows)) { + if (r.authorWarn !== undefined || r.authorHint !== undefined) warned.push(`${prefix}${key}`); + if (r.children) walk(r.children, `${prefix}${key}.`); + } + }; + walk({ connectorSource: row! }, ''); + expect(warned, 'a row of the binding carries an author warning').toEqual([]); const children = row!.children!; expect(Object.keys(children).sort()).toEqual(['action', 'connector', 'input', 'recordsPath', 'watermark']); for (const [key, child] of Object.entries(children)) { diff --git a/packages/spec/src/data/mapping.zod.ts b/packages/spec/src/data/mapping.zod.ts index 91c626d1de6..0a9183b80cf 100644 --- a/packages/spec/src/data/mapping.zod.ts +++ b/packages/spec/src/data/mapping.zod.ts @@ -365,10 +365,13 @@ export const MappingSchema = lazySchema(() => strictObject({ * * EXECUTED WHEN A JOB DRIVES IT: `@objectstack/service-automation`'s * connector sync executor (`pullConnectorSource`) reads every key here and - * writes through the import runner. Nothing schedules a pull yet — the - * `job` that drives it is the next stage — so the liveness ledger keeps - * `authorWarn` on the container until it lands. A pull makes ONE action - * call and reads ONE response (see `watermark`). `sourceFormat` keeps + * writes through the import runner, so the liveness ledger grades every key + * `live`. Nothing schedules a pull yet — the `job` that drives it is the + * next stage — and that is the one caveat an author must read before + * writing the binding, so the `.describe()` below carries it. It is not a + * ledger warning: a `live` row carries no `authorWarn` (the author-side lint + * has no verdict for one, and `check:liveness` refuses it). A pull makes ONE + * action call and reads ONE response (see `watermark`). `sourceFormat` keeps * governing the manual import door; a pulled row is the connector's JSON * record. */ @@ -424,7 +427,8 @@ export const MappingSchema = lazySchema(() => strictObject({ ), }).optional().describe( 'Pull binding: the rest/openapi connector this mapping pulls rows from (one-way, full or ' - + 'timestamp-incremental; a `job` sets the cadence). Pulled when a job drives it; nothing schedules it yet', + + 'timestamp-incremental; a `job` sets the cadence). Pulled when a job drives it; nothing schedules it yet, ' + + 'so the binding alone moves no rows — schedule the pull with a `job` once a job can drive one', ), // `extractQuery`, `errorPolicy` and `batchSize` were removed in 17.0.0 diff --git a/packages/spec/src/integration/connector-sync-retirement.test.ts b/packages/spec/src/integration/connector-sync-retirement.test.ts index 3473aeaa8fa..9dd7a825bd4 100644 --- a/packages/spec/src/integration/connector-sync-retirement.test.ts +++ b/packages/spec/src/integration/connector-sync-retirement.test.ts @@ -120,7 +120,7 @@ describe('connector sync retirement — the tombstones', () => { it('the `syncConfig` prescription is honest about the binding it points at: pulled when a job drives it, scheduled by nothing yet', () => { // The pull executor reads the target-side binding (its ledger rows are // `live`), but nothing schedules a pull until the `job` stage lands (the - // container keeps `authorWarn`), so a prescription that sent the author + // binding's own description says so), so a prescription that sent the author // there as if it ran on its own would be the defect this retirement // removes, moved one type over. const message = issueAt(ConnectorSchema.safeParse({ ...WELL_FORMED, syncConfig: {} }), 'syncConfig')!.message; diff --git a/packages/spec/src/integration/connector.zod.ts b/packages/spec/src/integration/connector.zod.ts index 7605b8ed44f..b9e89f48269 100644 --- a/packages/spec/src/integration/connector.zod.ts +++ b/packages/spec/src/integration/connector.zod.ts @@ -229,7 +229,7 @@ const SYNC_CONFIG_RETIRED = + 'a `mapping` (`targetObject`, `fieldMapping`, `mode`, `upsertKey`) whose `connectorSource` ' + 'names the `rest` or `openapi` connector it pulls from, the read action and an optional ' + 'timestamp `watermark`, with a `job` for the cadence. Its pull runs when a `job` drives it; ' - + 'nothing schedules one yet, and authoring the binding warns until a job can. ' + + 'nothing schedules one yet, so the binding alone moves no rows. ' + 'Run `os migrate meta --from 17` to list the mechanical edits for existing sources; apply them by hand.'; /** diff --git a/packages/spec/src/migrations/entries/semantic/18.connector-sync-keys-retired.ts b/packages/spec/src/migrations/entries/semantic/18.connector-sync-keys-retired.ts index ca4d6163857..0aca330a413 100644 --- a/packages/spec/src/migrations/entries/semantic/18.connector-sync-keys-retired.ts +++ b/packages/spec/src/migrations/entries/semantic/18.connector-sync-keys-retired.ts @@ -24,8 +24,8 @@ export const entry: SemanticMigration = { + 'instance it pulls from (`connector`), the action that reads the records (`action`, with a ' + 'fixed `input` and a `recordsPath`) and, for a timestamp-incremental pull, a `watermark` ' + '(`field` on the record, `param` on the request); a `job` sets the cadence. The pull executor ' - + 'reads the binding when a `job` drives it; nothing schedules a pull yet, and authoring it ' - + 'warns until a job can.', + + 'reads the binding when a `job` drives it; nothing schedules a pull yet, so the binding alone ' + + 'moves no rows.', reason: 'The D2 conversion `connector-sync-keys-removed` deletes `syncConfig` and ' + '`fieldMappings` from every connector, stack entry and stored connector row, one notice per ' + 'key, and the delete is lossless: no engine ever ran a connector-attached sync or moved a ' diff --git a/packages/spec/src/migrations/registry.ts b/packages/spec/src/migrations/registry.ts index a32c39272fd..da94c487ba6 100644 --- a/packages/spec/src/migrations/registry.ts +++ b/packages/spec/src/migrations/registry.ts @@ -8168,8 +8168,8 @@ const step18: MigrationStep = { + 'instance it pulls from (`connector`), the action that reads the records (`action`, with a ' + 'fixed `input` and a `recordsPath`) and, for a timestamp-incremental pull, a `watermark` ' + '(`field` on the record, `param` on the request); a `job` sets the cadence. The pull executor ' - + 'reads the binding when a `job` drives it; nothing schedules a pull yet, and authoring it ' - + 'warns until a job can.', + + 'reads the binding when a `job` drives it; nothing schedules a pull yet, so the binding alone ' + + 'moves no rows.', reason: 'The D2 conversion `connector-sync-keys-removed` deletes `syncConfig` and ' + '`fieldMappings` from every connector, stack entry and stored connector row, one notice per ' + 'key, and the delete is lossless: no engine ever ran a connector-attached sync or moved a '