fix(spec,objectql): declare the inert-JSON artifact and registry-record package body stages, and stop the record under-reporting functions

.changeset/17518-inert-json-package-body-stages.md

@@ -0,0 +1,14 @@
+---
+"@objectstack/spec": minor
+---
+
+A package body now has a declaration at every stage it really passes through: `ArtifactStagePackageBodySchema` and `RecordStagePackageBodySchema` join `AssembledPackageBodySchema`, and the installed-package read rows are declared against the record stage instead of two `z.unknown()` holes (#17518).
+
+ADR-0130 D4 says an artifact is inert JSON — "a plugin written inside `packages[i].manifest` could never be constructed by a loader, so a reader that resolved it there would register garbage where it used to skip in silence". Of `AssembledPackageBodySchema`'s 55 members exactly two declare that they accept a callable: `functions`, whose entry union opens with `z.function()`, and `hooks`, whose `handler` carries a `z.custom()` branch. One unrepresentable member costs every embedder its whole JSON Schema, which is why `api/ListInstalledPackagesResponse` and `api/GetInstalledPackageResponse` could only carry the body with both keys written `z.unknown().optional()` — accepted without being checked, as that file's own docblock said.
+
+- **⛔ The assembled body is untouched, and that is the point.** Those callables are LIVE on the stage it declares itself for: `composeStacks(stacks, { manifest: 'preserve' })` builds exactly such a body and the load path registers it, and `stack.zod.ts` states the invariant that binds the two. Narrowing in place would refuse a published composition function's own output. The two JSON stages are declared BESIDE it instead.
+- **Artifact stage** — what `objectstack build` writes: `functions` entries are the lowered spellings (a bare handler ref, or `FlowFunctionLoweredDeclarationSchema`), `hooks[].handler` is a string. **Record stage** — what `SchemaRegistry.installPackage` stores: the artifact stage with `functions[].handler` OPTIONAL, in both the map-record and the array form. That single difference is the whole distance between the two: `build` mints a ref for every callable, while `toRecordManifest`'s structural projection DROPS the callable and mints nothing in its place, so a record states what each function is named and what it declared with `handler` absent where the callable was. Measured: both bodies convert under `z.toJSONSchema` over the whole body, where the assembled body still does not.
+- **`FlowFunctionLoweredDeclarationSchema` is exported** from `@objectstack/spec/automation`, with its `FlowFunctionLoweredDeclaration` / `…Parsed` aliases. It was a module-local `const`, and `export * from './flow-function.zod'` only re-exports what is already exported — so `unemitted-schemas.baseline.json`'s reason for `Automation.FlowFunctionDeclarationSchema`, which says the lowered record "is the serialisable half … and it publishes normally", pointed at a schema no consumer could reach. It publishes now: `automation/FlowFunctionLoweredDeclaration` is in the schema manifest.
+- **`effect` is READ, not minted.** `FlowFunctionDeclarationSchema.effect` is `FlowFunctionEffectSchema.default('pure')` — a default, not a requirement — and the array member's is `.optional()` with no default. Both JSON stages inherit each form's optionality by deriving from it rather than restating it.
+- **⚠️ What narrows, stated plainly**: on the two installed-package responses, `functions` and `hooks` move from `unknown` (accepts anything) to their declared JSON shapes. No row the doors really serve is withdrawn — measured through the real `SchemaRegistry.installPackage` on the shape `examples/app-showcase` ships, on the array form, and on the already-lowered body an artifact boot installs. Every other key, `objects` included, is checked exactly as before, and both stages still refuse an authoring glob and an unknown key.
+- One correction in the same edit: `package-api.zod.ts` said those two members were also why `ArtifactPackageSchema` and `ObjectStackDefinitionSchema` publish no JSON Schema. They are not — `src/stack.zod.ts` is not one of the subpath namespaces `build-schemas.ts` walks, so neither is ever reached by the emit loop.

.changeset/17518-registry-record-reports-every-function.md

@@ -0,0 +1,12 @@
+---
+"@objectstack/objectql": patch
+---
+
+`GET /packages` reports every function a package declares. A bare callable `functions` entry is normalised to the declared form at the assembly boundary, so the registry record no longer drops it (#17518).
+
+`SchemaRegistry.installPackage` stores `toRecordManifest(manifest)`, a structural JSON projection whose rule is "a live object reached the record" and deliberately ⛔ not a key denylist. That rule treated the two authored `functions` spellings unequally through no fault of its own: a DECLARED entry (`{ handler, effect: 'writes' }`) is a plain object, so it survived with its callable dropped, while a BARE callable entry IS the callable, so the whole key vanished. `examples/app-showcase` ships one of each, so a package declaring two functions was reported as declaring one — a machine-readable read door under-reporting by construction.
+
+- **The repair is at the assembly boundary, ⛔ not in the projection.** `installPackage` makes the two spellings structurally equal before projecting, so the structural rule is untouched and no key name is special-cased. The projection then leaves `{ effect }` for both.
+- **⛔ No ref is minted.** `objectstack build` mints refs with `uniqueName(base, taken)` and dedupes by function identity, so a ref minted in the registry is not guaranteed to be the one `build` mints — a record could assert a handler that resolves in no sibling module. An absent `handler` is the honest statement "declared here, not serialisable", which is exactly what `@objectstack/spec`'s new `RecordStagePackageBodySchema` declares.
+- **⛔ No entry is dropped**, either: under-reporting by design was the other arm, and it also throws away the `effect` declaration, the one half that survived.
+- The caller's manifest is never mutated — `ObjectQL.registerApp` and the hook binder read the live callables off that object — and a copy is made only when an entry really needed rewriting. The ARRAY form is untouched: its entries are objects carrying their own `name`, so the projection already kept them.

content/docs/references/api/package-api.mdx

@@ -47,7 +47,7 @@ Installed package row whose manifest is the assembled package body
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **manifest** | `{ id: string; namespace?: string; defaultDatasource?: string; version: string; … }` | ✅ | The ASSEMBLED package body this row carries |
+| **manifest** | `{ id: string; namespace?: string; defaultDatasource?: string; version: string; … }` | ✅ | The ASSEMBLED package body this row carries, at the stage the registry records it |
 | **status** | `Enum<'installed' \| 'disabled' \| 'installing' \| 'upgrading' \| 'uninstalling' \| 'error'>` | optional (default: `"installed"`) | Package state: installed, disabled, installing, upgrading, uninstalling, or error |
 | **enabled** | `boolean` | optional (default: `true`) | Whether the package is currently enabled |
 | **installedAt** | `string` | optional | Installation timestamp |
@@ -89,7 +89,7 @@ Installed package row whose manifest is the assembled package body
 | **packaging** | `Enum<'bundled' \| 'manifest-deps'>` | optional | Dependency packaging strategy (ADR-0025 §3.3) |
 | **main** | `string` | optional | Entry module of a code-bearing plugin, relative to the plugin root; `os plugin build` bundles it and writes `dist/index.mjs` here in the compiled manifest (ADR-0025 §3.4) |
 | **integrity** | `Record<string, string>` | optional | Per-file content digests of the plugin artifact (ADR-0025 §3.2) |
-| **functions** | `any` | optional | Named handler functions, as they survived the record JSON projection |
+| **functions** | `Record<string, string \| { handler?: string; effect?: Enum<'pure' \| 'writes'> }> \| { name: string; handler?: string; packageId?: string; effect?: Enum<'pure' \| 'writes'> }[]` | optional | Named handler functions, lowered to the refs a JSON document carries |
 | **datasourceMapping** | `{ namespace?: string; package?: string; objectPattern?: string; default?: boolean; … }[]` | optional | Centralized datasource routing rules for packages/namespaces/objects |
 | **translations** | `Record<string, { objects?: Record<string, object>; apps?: Record<string, object>; messages?: Record<string, string>; globalActions?: Record<string, object>; … }>[]` | optional | I18n Translation Bundles |
 | **objectExtensions** | `{ extend: string; fields?: Record<string, object>; label?: string; pluralLabel?: string; … }[]` | optional | Extensions to objects owned by other packages |
@@ -113,7 +113,7 @@ Installed package row whose manifest is the assembled package body
 | **agents** | `{ name: string; label: string; avatar?: string; role: string; … }[]` | optional | AI Agents — platform-internal (ADR-0063 §2): the kernel ships exactly two (ask/build); third parties extend via skills, not agents |
 | **tools** | `{ name: string; label: string; description: string; parameters: Record<string, any>; … }[]` | optional | AI Tool metadata records — optional refinement layer, never required: the default path is skills referencing platform tools or materialised action_`<name>` tools (ADR-0109) |
 | **skills** | `{ name: string; label: string; description?: string; surface?: Enum<'ask' \| 'build' \| 'both'>; … }[]` | optional | AI Skills (reusable capability bundles — the third-party AI extension primitive, ADR-0063) |
-| **hooks** | `any` | optional | Object lifecycle hooks, as they survived the record JSON projection |
+| **hooks** | `{ name: string; label?: string; object: string \| string[]; events: Enum<'beforeFind' \| 'afterFind' \| 'beforeInsert' \| 'afterInsert' \| 'beforeUpdate' \| …>[]; … }[]` | optional | Object Lifecycle Hooks, as a JSON document carries them |
 | **mappings** | `{ name: string; label?: string; sourceFormat?: Enum<'csv' \| 'json' \| 'xml' \| 'sql'>; targetObject: string; … }[]` | optional | Data Import/Export Mappings |
 | **analyticsCubes** | `{ name: string; title?: string; description?: string; sql: string; … }[]` | optional | Analytics Semantic Layer Cubes |
 | **connectors** | `{ name: string; label: string; type: Enum<'saas' \| 'database' \| 'file_storage' \| 'message_queue' \| 'api' \| 'custom'>; description?: string; … }[]` | optional | External System Connectors. A provider-bound entry (has `provider`: openapi/mcp/rest) is materialized into a live, dispatchable connector at boot and referenced by flows via `connector_action`; credentials are `auth.credentialRef` references, never inline secrets. An entry with no `provider` is a catalog descriptor only (NOT dispatchable) — set `enabled: false` on deliberate descriptors. Unknown provider / unresolvable credentialRef / name conflict ⇒ hard boot error (ADR-0097). |
@@ -208,7 +208,7 @@ Installed package row whose manifest is the assembled package body
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **manifest** | `{ id: string; namespace?: string; defaultDatasource?: string; version: string; … }` | ✅ | The ASSEMBLED package body this row carries |
+| **manifest** | `{ id: string; namespace?: string; defaultDatasource?: string; version: string; … }` | ✅ | The ASSEMBLED package body this row carries, at the stage the registry records it |
 | **status** | `Enum<'installed' \| 'disabled' \| 'installing' \| 'upgrading' \| 'uninstalling' \| 'error'>` | optional (default: `"installed"`) | Package state: installed, disabled, installing, upgrading, uninstalling, or error |
 | **enabled** | `boolean` | optional (default: `true`) | Whether the package is currently enabled |
 | **installedAt** | `string` | optional | Installation timestamp |
@@ -303,7 +303,7 @@ Installed package row whose manifest is the assembled package body
 
 | Property | Type | Required | Description |
 | :--- | :--- | :--- | :--- |
-| **manifest** | `{ id: string; namespace?: string; defaultDatasource?: string; version: string; … }` | ✅ | The ASSEMBLED package body this row carries |
+| **manifest** | `{ id: string; namespace?: string; defaultDatasource?: string; version: string; … }` | ✅ | The ASSEMBLED package body this row carries, at the stage the registry records it |
 | **status** | `Enum<'installed' \| 'disabled' \| 'installing' \| 'upgrading' \| 'uninstalling' \| 'error'>` | optional (default: `"installed"`) | Package state: installed, disabled, installing, upgrading, uninstalling, or error |
 | **enabled** | `boolean` | optional (default: `true`) | Whether the package is currently enabled |
 | **installedAt** | `string` | optional | Installation timestamp |
@@ -345,7 +345,7 @@ Installed package row whose manifest is the assembled package body
 | **packaging** | `Enum<'bundled' \| 'manifest-deps'>` | optional | Dependency packaging strategy (ADR-0025 §3.3) |
 | **main** | `string` | optional | Entry module of a code-bearing plugin, relative to the plugin root; `os plugin build` bundles it and writes `dist/index.mjs` here in the compiled manifest (ADR-0025 §3.4) |
 | **integrity** | `Record<string, string>` | optional | Per-file content digests of the plugin artifact (ADR-0025 §3.2) |
-| **functions** | `any` | optional | Named handler functions, as they survived the record JSON projection |
+| **functions** | `Record<string, string \| { handler?: string; effect?: Enum<'pure' \| 'writes'> }> \| { name: string; handler?: string; packageId?: string; effect?: Enum<'pure' \| 'writes'> }[]` | optional | Named handler functions, lowered to the refs a JSON document carries |
 | **datasourceMapping** | `{ namespace?: string; package?: string; objectPattern?: string; default?: boolean; … }[]` | optional | Centralized datasource routing rules for packages/namespaces/objects |
 | **translations** | `Record<string, { objects?: Record<string, object>; apps?: Record<string, object>; messages?: Record<string, string>; globalActions?: Record<string, object>; … }>[]` | optional | I18n Translation Bundles |
 | **objectExtensions** | `{ extend: string; fields?: Record<string, object>; label?: string; pluralLabel?: string; … }[]` | optional | Extensions to objects owned by other packages |
@@ -369,7 +369,7 @@ Installed package row whose manifest is the assembled package body
 | **agents** | `{ name: string; label: string; avatar?: string; role: string; … }[]` | optional | AI Agents — platform-internal (ADR-0063 §2): the kernel ships exactly two (ask/build); third parties extend via skills, not agents |
 | **tools** | `{ name: string; label: string; description: string; parameters: Record<string, any>; … }[]` | optional | AI Tool metadata records — optional refinement layer, never required: the default path is skills referencing platform tools or materialised action_`<name>` tools (ADR-0109) |
 | **skills** | `{ name: string; label: string; description?: string; surface?: Enum<'ask' \| 'build' \| 'both'>; … }[]` | optional | AI Skills (reusable capability bundles — the third-party AI extension primitive, ADR-0063) |
-| **hooks** | `any` | optional | Object lifecycle hooks, as they survived the record JSON projection |
+| **hooks** | `{ name: string; label?: string; object: string \| string[]; events: Enum<'beforeFind' \| 'afterFind' \| 'beforeInsert' \| 'afterInsert' \| 'beforeUpdate' \| …>[]; … }[]` | optional | Object Lifecycle Hooks, as a JSON document carries them |
 | **mappings** | `{ name: string; label?: string; sourceFormat?: Enum<'csv' \| 'json' \| 'xml' \| 'sql'>; targetObject: string; … }[]` | optional | Data Import/Export Mappings |
 | **analyticsCubes** | `{ name: string; title?: string; description?: string; sql: string; … }[]` | optional | Analytics Semantic Layer Cubes |
 | **connectors** | `{ name: string; label: string; type: Enum<'saas' \| 'database' \| 'file_storage' \| 'message_queue' \| 'api' \| 'custom'>; description?: string; … }[]` | optional | External System Connectors. A provider-bound entry (has `provider`: openapi/mcp/rest) is materialized into a live, dispatchable connector at boot and referenced by flows via `connector_action`; credentials are `auth.credentialRef` references, never inline secrets. An entry with no `provider` is a catalog descriptor only (NOT dispatchable) — set `enabled: false` on deliberate descriptors. Unknown provider / unresolvable credentialRef / name conflict ⇒ hard boot error (ADR-0097). |

content/docs/references/automation/flow-function.mdx

@@ -65,8 +65,8 @@ the platform's own counters stop being wrong for it.
 ## TypeScript Usage
 
 ```typescript
-import { FlowFunctionEffectSchema } from '@objectstack/spec/automation';
-import type { FlowFunctionEffect } from '@objectstack/spec/automation';
+import { FlowFunctionEffectSchema, FlowFunctionLoweredDeclarationSchema } from '@objectstack/spec/automation';
+import type { FlowFunctionEffect, FlowFunctionLoweredDeclaration } from '@objectstack/spec/automation';
 
 // Validate data
 const result = FlowFunctionEffectSchema.parse(data);
@@ -86,3 +86,17 @@ What a script-node function does to data: 'pure' (computes and returns — the c
 
 ---
 
+## FlowFunctionLoweredDeclaration
+
+A lowered `functions` declaration: what the function declared about itself, with its callable replaced by a handler ref
+
+### Properties
+
+| Property | Type | Required | Description |
+| :--- | :--- | :--- | :--- |
+| **handler** | `string` | ✅ | The lowered handler ref (built artifacts) — the callable rides in the sibling ESM module |
+| **effect** | `Enum<'pure' \| 'writes'>` | optional (default: `"pure"`) | What the function does to data — omit for the pure default |
+
+
+---
+