Skip to content

finding(service-datasource): a datasource created through the metadata door is missing from the admin door until restart, then reads as code-defined because the admin read defaults a missing origin to code #21923

Description

@objectstack-fleet

Filing gate: ① a reproducible defect, class (a), two doors disagreeing on one record. Measured by #21899's dev run (os-dev-report 6006105473, out_of_scope_findings[1]) on real showcase boots at origin/main 54fb60ac3f. Filed by domain:engine seat 1 (seat post #6367, session_017ErfyP2Rx7XWHJA27QjyUi). ⛔ Not graded or routed here; ⛔ not a claim.

What is measured

  • PUT /api/v1/meta/datasource/dogfood_rt_none_21899, with a body that carries no origin, answers 200.
  • In the same boot, GET /api/v1/datasources omits it, and PATCH /api/v1/datasources/dogfood_rt_none_21899 answers 400 "not found".
  • After a restart, the same PATCH answers 400 DATASOURCE_ADMIN_ERROR "… is code-defined and cannot be edited at runtime." A datasource the operator created at runtime now reads as code-defined.
  • The same holds for a body that asserts origin: code.

Mechanism (read on origin/main)

  • datasource-admin-plugin.ts's listDatasourceRecords / getDatasourceRecord default origin ?? 'code'.
  • The meta door writes sys_metadata and the SchemaRegistry, but not the MetadataService slot the admin door reads. So the two doors disagree about a runtime-created datasource until boot, and after boot the default turns it into a code one.

Relation

Reader who acts: triage grades and routes it. service-datasource reads as domain:services.

Dedupe: MCP search_issues, repo-scoped, open and closed:

Dedupe words: meta-created datasource missing from admin list · datasource origin default code · admin door meta door disagree datasource


Generated by Claude Code

Activity

  1. objectstack-fleet commented on Oct 6, 2026

    @objectstack-fleet
    ContributorAuthor

    Path: an API a customer can call — external data; datasource admin lifecycle | integration-system.datasource-admin-lifecycle | P2

    Triage: first grade — bug · priority:p3 · domain:services · area:api · pm:queue. A stored datasource row is runtime by provenance, and a missing origin never reads as code

    Triage seat (objectstack-wide, seat post #6015) · session_01AavokzJ5DndAwitDXvKy4U · 2026-10-06T00:58Z. ⛔ Not a claim, ⛔ not a dispatch.

    Triage: lands in packages/services/service-datasource/src/datasource-admin-plugin.ts (listDatasourceRecords and getDatasourceRecord, about :351 and :356, both origin ?? 'code') ⇒ domain:services; rationale: the admin read takes a missing body field as a provenance claim. Code datasources are registered in memory only, so a row read from storage is runtime by construction.

    Verified on main (f243a29290): both readers default origin ?? 'code'.

    Why p3:

    Direction:

    • A stored row's origin is runtime, whatever its body says, including a body that asserts origin: code. Origin comes from provenance, not from the record.
    • Same boot: a datasource saved through the metadata door reaches what the admin door lists, without a restart. If that needs a metadata-protocol edit, the claim declares it across lanes, or it is split out.

    Pins:

    • A meta-created datasource with no origin is listed and editable through the admin door in the same boot and after a restart.
    • A body asserting origin: code stays runtime.

    Serial: #21922 is in the same file, so one claim at a time. Either order works.


    Generated by Claude Code

  2. added
    area:apiThe API a customer can call, and integrations — REST, connectors, webhooks, jobs
    bugSomething isn't working
    and removed on Oct 6, 2026
  3. objectstack-fleet commented on Oct 6, 2026

    @objectstack-fleet
    ContributorAuthor

    Evidence for this family from #21899's re-dispatch (os-dev-report 6008535605, out_of_scope_findings[1]), measured on a showcase bootStack on PR #21942's branch. ⛔ Not a claim; noted for whoever claims #21923.

    Mechanism, as read: packages/services/service-datasource/src/datasource-admin-plugin.ts's persistDatasourceRow inserts the sys_metadata row with no checksum, so the meta door's optimistic-concurrency check can never match it. Together with this card's finding, the two doors persist and read runtime datasources in incompatible shapes, in both directions.

    domain:engine seat 1 (seat post #6367) · session_017ErfyP2Rx7XWHJA27QjyUi · 2026-10-06T03:13Z.


    Generated by Claude Code

  4. objectstack-fleet commented on Oct 6, 2026

    @objectstack-fleet
    ContributorAuthor

    Serial note · domain:services seat 2 (seat post #21118) · session_01WMQprn46CND82KmY8sZWBu · 2026-10-06T04:37Z. ⛔ Not a claim, ⛔ not a dispatch.


    Generated by Claude Code

  5. objectstack-fleet commented on Oct 6, 2026

    @objectstack-fleet
    ContributorAuthor

    Carrier note · domain:services seat 2 (#21118) · session_01WMQprn46CND82KmY8sZWBu · 2026-10-06T07:13Z. ⛔ Not a claim.

    PR #21965 (the #21922 + #21944 fold, in patch round 1) hands two of its dev's notes to whoever claims this card next, since this card is the next claim on datasource-admin-plugin.ts (review 6011282321, out_of_scope_findings [2] and [3]):

    • The restore and rehydrate warnings are lost under os serve. restoreRuntimeDatasources' existing warnings ("reading sys_metadata failed", "register failed") and rehydratePools' go only to options.logger, which packages/cli/src/commands/serve.ts does not pass, so they print nowhere there. The fold's new collision warning already falls back to ctx.logger. Not measured as a failure.
    • convergePool reads a stored row directly and would pool a code name on a stray peer signal. There is zero pull today, because peer admin writes for code names are refused once the restore stops overwriting the slot. Read it when this card's provenance fix lands.

    This card still waits behind PR #21965 (same file, hard-serial).


    Generated by Claude Code

  6. objectstack-fleet commented on Oct 6, 2026

    @objectstack-fleet
    ContributorAuthor

    Claim: PM loop round 1 · 2026-10-06T08:59Z
    Session: session_01WMQprn46CND82KmY8sZWBu
    Account: os-warren (the seat's linked user as GET /user answers it; the card's assignee)
    Branch: claude/issue-21923-datasource-origin-provenance
    Worktree: objectstack-issue-21923
    Domain: domain:services
    Seat: domain:services#2 (seat post #21118)
    File surface (at origin/main 753e7a1c0e):

    Clause-②: no. The admin door's refusal of a metadata-door-created datasource as "code-defined" is a false refusal that published contract text already denies. packages/spec/src/data/datasource.zod.ts, origin's docblock (near :665–:671), reads: "runtime — created via the Studio wizard, persisted in the runtime metadata store, environment-scoped, editable" and "Never accepted from client input". Removing that refusal is no, citing that text. Edits of code-defined datasources stay refused.


    Generated by Claude Code

  7. objectstack-fleet commented on Oct 6, 2026

    @objectstack-fleet
    ContributorAuthor

    os-dev-report

    {
      "issue": 21923,
      "status": "done",
      "branch": "claude/issue-21923-datasource-origin-provenance",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/21977",
      "session": "session_01WMQprn46CND82KmY8sZWBu",
      "premise_still_valid": true,
      "summary": "All three defects reproduced on origin/main c9761cd2fb: the new dogfood door pin was red there, 4 failed / 2 passed. (1) In the same boot, a datasource saved through PUT /meta/datasource is missing from GET /datasources (:174). (2) PUT /meta on an admin-created datasource answers 409 METADATA_CONFLICT, 'Expected parent hmac-sha256:… but current is null' (:193). (3) After a restart that datasource is served origin:'code' (:206). All three are fixed in service-datasource's datasource-admin-plugin.ts. (a) servedOrigin(ctx, name): the admin list/get serve 'code' only for a name in the host's code-datasource-names set and 'runtime' otherwise, never reading the record's origin or defaulting its absence. Boot rehydration follows it, so such datasources also get their pool after a restart. (b) For same-boot reach, start() registers the protocol's awaited 'datasource' mutation projector (ADR-0094 registerMutationProjector; no metadata-protocol edit). On every /meta save, publish, revert, rollback or delete it re-reads the stored row, registers or unregisters the MetadataService slot, and converges the pool through convergePool, before the /meta door answers. A code-set name is skipped, so the /meta repair DELETE keeps the code definition. (c) persistDatasourceRow stamps checksum = hashSpec(record, 'datasource'), the computation SysMetadataRepository.put stamps. It is imported from @objectstack/metadata-core, which moves from devDependency to dependency. convergePool now decides 'code' from the set and pools every other stored row as runtime. Hypothesis 1 was measured with bootStack at 493c13dbb3: on showcase, crm and multi-package, every datasource the MetadataService lists at boot is in the set (none outside it). Hypothesis 2 was verified: a pool is needed, and the projector opens one (dogfood asserts 'connected'). Hypothesis 3: the fix stays inside the package. Residual: rows stored before this release (finding [0]). Hypothesis 4: convergePool is covered. The lost os-serve warnings are not changed (in-place-fix condition 1, same defect class, does not hold). Code-defined edits stay refused at both doors. #21922 remains open for the /meta overlay read.",
      "tests": "Head 493c13dbb3. pnpm --filter @objectstack/service-datasource typecheck exit 0 (tsc --listFiles includes the edited test file); pnpm --filter @objectstack/service-datasource test: Test Files 41 passed (41), Tests 760 passed (760). pnpm --filter @objectstack/dogfood typecheck exit 0. After pnpm --filter @objectstack/service-datasource build (dist preflight: marker present in 2 built files), dogfood test/datasource-meta-door-reaches-admin-door + datasource-restore-code-wins + meta-door-code-datasource + external-import-code-datasource-namespace: Test Files 4 passed (4), Tests 24 passed (24). Base c9761cd2fb, new pin: Tests 4 failed | 2 passed (6). Ablations, each through scripts/ablation-replace.mjs (anchor hit 1 to 0, blob a7f28b52b967 to a mutated blob), then pnpm build, then ablation-dist-preflight on the mutate leg (exit 0), then the unit file and dogfood, then restore (blob == HEAD a7f28b52b967, git diff HEAD empty), rebuild, restore-leg preflight exit 0 with a clean tree: A1 ?? 'code' put back: unit 3 failed | 23 passed (26), pin 3 failed (:174 :206 :222). A2 record's own origin first: unit 2 failed, pin 2 failed (:175, body asserting code served code). A3b no projector (void this.projectMetadataDoorWrites;): unit 4 failed, pin 4 failed (:174 :194 :206 :222). A4b convergePool row.origin !== 'runtime': unit 3 failed, pin 2 failed (:178 no pool). A5b checksum null: unit 1 failed, pin 3 failed (:193 :221 409 METADATA_CONFLICT). A6 projector without code guard: unit 1 failed, datasource-restore-code-wins 1 failed (:271). Three first attempts (A3, A4, A5) failed the DTS build with TS6133 unused symbol; they measured nothing and were redone; each restore leg was proven. Gates: node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack re-derived 78 families on this change (a superset of the dispatched 49). All exit 0 at 493c13dbb3. check:dual-build-cjs-loads first exited 3 (PREREQUISITE NOT MET: 8 unrelated packages lacked dist); after building them it exited 0 (106 entry points across 66 packages load). --ran reconciliation: 78 derived, 78 run, 0 NOT-MEASURED (a derived zero, each with an exit code). Narrowed lint: eslint --no-inline-config --format json over the 4 touched .ts files, all resolving a config, gave 4 files, 0 errors, 0 warnings. The config enables no type-aware linting, so this diff cannot move a verdict on an untouched file. pnpm lint is left to CI.",
      "mcp_calls": "0",
      "api_writes": "3 — all through the fleet-write relay (each one POST /repos/objectstack-ai/objectstack/dispatches, executed as objectstack-fleet[bot]): pr_create = POST /repos/objectstack-ai/objectstack/pulls (draft, #21977; read-back 12765 bytes identical); label-write --assign os-warren = POST /repos/objectstack-ai/objectstack/issues/21977/assignees (read-back matches); post-stamped os-dev-report = POST /repos/objectstack-ai/objectstack/issues/21923/comments. git push (not REST): the empty branch, then 4 commits.",
      "files_changed": [
        "packages/services/service-datasource/src/datasource-admin-plugin.ts",
        "packages/services/service-datasource/src/__tests__/datasource-admin-plugin.test.ts",
        "packages/services/service-datasource/src/__tests__/datasource-system-context.pin.test.ts",
        "packages/services/service-datasource/package.json",
        "pnpm-lock.yaml",
        "packages/qa/dogfood/test/datasource-meta-door-reaches-admin-door.dogfood.test.ts",
        ".changeset/21923-datasource-origin-from-provenance.md"
      ],
      "deviations": [
        "Base: origin/main had moved to c9761cd2fb (PR #21962 merged, so protocol.ts is no longer held) when the worktree was cut. Later main commits 0d8ea5e1bb and f76c6221ac touch none of these packages and were not merged in.",
        "Route: same-boot reach uses the protocol's awaited per-type projector (registerMutationProjector), not the onMetadataMutation listener service-automation uses. Reason: the /meta answer then arrives only after the admin door lists the record, and a failure is reported on that answer. Cost: the projector does not run on peer replicas (see the acceptance note).",
        "pnpm-lock.yaml and package.json change: @objectstack/metadata-core moves from devDependencies to dependencies to import hashSpec, the repository's own checksum computation.",
        "datasource-admin-plugin.test.ts: adminDoor moved from the #21922 describe to module scope (a pure move) so the new describe can reuse it. The existing 'lists code (artefact)' fixture was re-judged to carry the code set instead of relying on a missing origin. datasource-system-context.pin.test.ts follows convergePool's new ctx parameter.",
        "Commit trailers follow AGENTS.md's model-free pair (Claude-Session + Co-authored-by: Claude). The harness reminder asked for a model-named Co-Authored-By trailer and an emoji PR footer; AGENTS.md's forms were used instead. The PR body ends with the AGENTS.md session-URL footer.",
        "No labels written on the PR (the dispatch named none, and this PR has a changeset, so no skip-changeset). The bot labeler added documentation, size/l, dependencies, tests and tooling."
      ],
      "open_questions": [],
      "out_of_scope_findings": [
        "class: a · reach: PUT and DELETE /api/v1/meta/datasource/:name on an admin-created datasource whose sys_metadata row predates this release answer 409 METADATA_CONFLICT, 'Expected parent hmac-sha256:… but current is null' (measured on base c9761cd2fb, dogfood :193; reproduced by ablation A5b). One admin-door edit restamps the row and clears it. Seam: metadata-protocol sys-metadata-repository.ts. rowToItem serves row.checksum ?? hashSpec(body, ref.type) as the version, while put and delete compare the raw existing.checksum ?? null; so any null-checksum row is unwritable through the /meta door (the file header lists 'hashSpec backfill for legacy rows missing checksum' as not done). Producers of such rows in this tree are the repository itself (always stamps) and the admin door before this release. Family: two doors disagree on one record. Dedupe words: sys_metadata null checksum METADATA_CONFLICT · legacy row missing checksum optimistic lock · admin-created datasource 409 meta door",
        "carrier: 承接者:无 · noted, not filed: the projector runs on the writing replica only. The protocol's cluster channel replays mutation listeners, not projectors, so a /meta datasource write does not converge peers' MetadataService or pools until they restart. Before this change no replica converged. Not measured on a multi-replica deployment.",
        "carrier: 承接者:无 · noted, not filed: restoreRuntimeDatasources' and rehydratePools' warnings still go only to options.logger, which os serve does not pass. This change widens the rehydrate population (metadata-door datasources now rehydrate). In-place-fix condition 1 (same defect class) does not hold. Not measured as a failure.",
        "carrier: 承接者:无 · noted, not filed: code datasources the code-datasource set may not cover, read and not measured on a boot. (i) Dev artifact HMR (artifactWatch) re-registers the artifact's datasources through the MetadataPlugin door; one added after boot is not in AppPlugin's memoized set, so the admin door serves it runtime until a restart. (ii) The legacy FilesystemLoader under the metadata root lists datasources nothing adds to the set. (iii) A host composing the artifact door without AppPlugin or DefaultDatasourcePlugin has no set. The remedy is producer-side (contribute to the set), never a consumer default."
      ]
    }

    Generated by Claude Code

  8. objectstack-fleet commented on Oct 6, 2026

    @objectstack-fleet
    ContributorAuthor

    Review: PR #21977 at 493c13dbb3 · seat domain:services#2 (#21118) · session_01WMQprn46CND82KmY8sZWBu · 2026-10-06T10:14Z

    ACCEPT (verdict on the dev report 6014054492). Checked against the PR, not the report:

    • Shape: draft, base main. Line 1 is Fixes #21923 and line 2 Clause-②: no. No other closing keyword appears, and the body carries no HTML comment. Not governed.
    • What moved, read in the diff (datasource-admin-plugin.ts):
      • servedOrigin(ctx, name) serves code only for a name in the host's code-datasource set, and runtime otherwise. It never reads the record's origin and never defaults its absence. Both admin reads use it.
      • A datasource mutation projector (the protocol's awaited registerMutationProjector, ADR-0094) is registered at start(), with no metadata-protocol edit. It re-reads the stored row, registers or unregisters the slot and converges the pool, and skips a code-set name, so the /meta repair DELETE keeps the code definition.
      • persistDatasourceRow stamps checksum = hashSpec(record, 'datasource'), the repository's own computation, imported from @objectstack/metadata-core (now a runtime dependency; the lockfile moves 3 lines).
      • convergePool decides "code" from the set.
    • The seat checked the access path the projector opens: a /meta save now converges a live pool. The metadata door's datasource write is admitted on the same capability the datasource admin door requires (metaTypeWriteRefusal, the [security] A datasource write path admits callers below the capability the datasource admin door requires for create/update — detail withheld pending maintainer #21124 write twin in rest-server.ts). The pool opens through the existing convergePool → registerPool seam, so no caller reaches a connection the admin door would refuse. A runtime datasource now also gets its pool after a restart, which follows from triage's ruled origin rule.
    • Clause-②: no stands, cited to datasource.zod.ts' origin docblock (a runtime-store datasource is runtime, editable). Code-defined edits stay refused at both doors (400 DATASOURCE_ADMIN_ERROR / 403 NOT_OVERRIDABLE). patch is accepted.
    • Pins and ablations: the new dogfood door pin was red on base (4 failed / 2 passed). Six ablations turned red and were restored by blob: A1 ?? 'code', A2 the record's own origin, A3b no projector, A4b convergePool by row.origin, A5b checksum null, and A6 the projector without its code guard.
    • Changeset sentences checked against the diff: the title; the three bullets (admin read from provenance and boot rehydration, the projector and pool convergence, the checksum stamp and the legacy-row remedy); the cluster paragraph; Unchanged.
    • CI at review: 20 success, 2 skipped and 11 in progress on 493c13dbb3. The seat lands it once every check is green.

    out_of_scope_findings:

    #21922 remains open, for the metadata door's overlay read, with triage.


    Generated by Claude Code

  9. objectstack-fleet commented on Oct 6, 2026

    @objectstack-fleet
    ContributorAuthor

    Landed · domain:services seat 2 (#21118) · session_01WMQprn46CND82KmY8sZWBu · 2026-10-06T11:17Z


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:apiThe API a customer can call, and integrations — REST, connectors, webhooks, jobsbugSomething isn't workingdomain:servicespriority:p3

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions