fix(cli): os migrate meta converts objects built with ObjectSchema.create and the other strict authoring factories

.changeset/20696-migrate-meta-strict-factories.md

@@ -0,0 +1,54 @@
+---
+'@objectstack/cli': patch
+---
+
+fix(cli): `os migrate meta` converts an object built with `ObjectSchema.create(…)` instead of stopping at load when the object carries a retired key
+
+Clause-②: no
+
+`os migrate meta` reads a config the current schema refuses, so it can rewrite the
+retired keys in it. It did that for artifacts built with a `define*` helper and for
+plain object literals. It did not do it for artifacts built with a factory such as
+`ObjectSchema.create(…)`, which validates when it is called. An object like this:
+
+```ts
+ObjectSchema.create({
+  name: 'ticket',
+  fields: { title: { type: 'text' } },
+  tenancy: { enabled: true, organizationField: 'organization_id' },
+})
+```
+
+stopped `os migrate meta --from 17` at load with exit 1 and a raw JSON array of
+validation issues. The message in that array told the author to run
+`os migrate meta --from 17`.
+
+The command now loads it, applies the conversion (here
+`object-tenancy-organization-field-removed`), and reports `schemaValid` for the
+migrated stack, exactly as it does for the same object written as a plain literal.
+This covers the five factories in `@objectstack/spec` that validate when called:
+`ObjectSchema.create` (`@objectstack/spec/data`) and `App.create`,
+`Dashboard.create`, `Report.create` and `Action.create` (`@objectstack/spec/ui`).
+The other `create` factories spec exports return their argument unchanged and
+never refused anything, so nothing changes for them.
+
+A schema problem the migration cannot fix is still reported: it is listed among
+the refusals under the verdict, and `schemaValid` is `false`. A check that only
+the factory makes when it is called, such as `ObjectSchema.create` refusing a
+`managedBy: 'system-data'` object that grants no create, edit or delete, is not
+part of the stack schema. It is reported on the stderr line described below and
+does not change `schemaValid`, the same as `defineStack`'s own call-time checks.
+`os validate` still refuses it.
+
+While the config loads, `os migrate meta` prints one stderr line for each
+artifact the current schema refused. A raw validation error on that line is now
+printed as a block, for example `ObjectSchema.create validation failed (1 issue):`
+followed by one `✗ path: message` line per issue, instead of a raw JSON array.
+This also applies to `define*` helpers that throw a raw validation error, such
+as `defineAgent`.
+
+Nothing else changes. `os validate`, `os build` and every other command still
+refuse the retired key at load, with the same message. `ObjectSchema.create` and
+the other factories stay strict everywhere outside `os migrate meta`. The keys
+of the `--json` payload are unchanged, and a run whose migrated stack does not
+parse still exits 0.

packages/cli/src/utils/config.ts

@@ -257,7 +257,8 @@ export function resolveConfigPath(source?: string): string {
 
 /**
  * Every `@objectstack/spec` entrypoint an authored config can reach the
- * `define*` helpers through — the root and every subpath export. Real projects
+ * `define*` helpers and the {@link STRICT_AUTHORING_FACTORIES} through — the
+ * root and every subpath export. Real projects
  * use both: the example apps import `defineView`/`defineApp` from
  * `@objectstack/spec/ui` and `defineHook`/`defineDatasource` from
  * `@objectstack/spec/data`, so a shim that knew only the root package would
@@ -268,29 +269,161 @@ const SPEC_MODULE_RE = /^@objectstack\/spec(?:\/[\w./-]+)?$/;
 /** esbuild namespace the authored-source shim modules live in. */
 const AUTHORED_SOURCE_NAMESPACE = 'objectstack-authored-source';
 
+/** The package root, which carries `formatZodError` for the shim's refusal text. */
+const SPEC_ROOT_MODULE = '@objectstack/spec';
+
 /** `defineStack`, `defineView`, … — the authoring helpers, by naming convention. */
 const DEFINE_HELPER_RE = /^define[A-Z]/;
 
 /**
- * The `define*` helpers a given `@objectstack/spec` entrypoint exports, read
- * from the copy **the config itself would import** (resolved from the config's
- * own directory, not the CLI's).
- *
- * Returns `[]` — i.e. "shim nothing" — when the entrypoint cannot be resolved
- * or imported. That is the safe direction: an unshimmed load is exactly
- * today's behaviour, so a project the enumeration cannot read is no worse off
- * than before.
+ * One strict authoring factory that is not a `define*` helper: the function
+ * `member` of the exported value `owner`, which validates its argument AT THE
+ * CALL and throws on a shape the current schema refuses.
+ */
+export interface StrictAuthoringFactory {
+  /** The export that carries the factory: `ObjectSchema`, `App`, … */
+  readonly owner: string;
+  /** The factory on it: `create`. */
+  readonly member: string;
+  /**
+   * The `@objectstack/spec` entrypoint the entry was measured on. The shim does
+   * not read it — it wraps the factory on EVERY entrypoint whose `owner` export
+   * carries the member — and the pin reads it to prove the entry is still live.
+   */
+  readonly home: string;
+}
+
+/**
+ * Every strict authoring factory `@objectstack/spec` exports besides the
+ * `define*` helpers: the ONE list the authored-source shim wraps them from.
+ *
+ * `ObjectSchema.create(…)` is the authoring spelling of every example app's
+ * objects, and it is as strict as a `define*` helper — it parses at the call.
+ * Unwrapped, a retired key inside it aborted `os migrate meta` at load with a
+ * raw `ZodError` array, while the tombstone it printed told the author to run
+ * `os migrate meta`.
+ *
+ * ## Why a written list and not a pattern
+ *
+ * Measured over all 19 JS entrypoints of `@objectstack/spec`: 18 distinct
+ * exported values carry a `create` member (31 export names — each identity
+ * factory is exported a second time as its `*Schema`). Five validate — the
+ * five below — and thirteen (`ApiEndpoint`, `Task`, `RestServerConfig`, …) are
+ * identity factories, `(config) => config`, that refuse nothing and so have
+ * nothing to tolerate.
+ * The other function members of exported namespaces (`Field.*`, `SCIM.*`,
+ * `RLS.*`, `OData.*`) build or read values and validate nothing. So:
+ *
+ *  - a NAME pattern (`*.create`) would wrap thirteen no-ops and still say
+ *    nothing about which factories are strict;
+ *  - a SHAPE enumeration at load cannot even see the one this list exists for:
+ *    `ObjectSchema` is a lazy-schema Proxy whose `ownKeys` trap throws, so
+ *    `Object.keys(ObjectSchema)` never names `create`.
+ *
+ * The pin beside this file's tests holds the list to the spec surface in both
+ * directions: every entry resolves at its `home` and throws on a refused input,
+ * and every `create` member spec exports that is NOT listed returns its
+ * argument untouched. A new strict factory in spec therefore reddens the pin
+ * with its name instead of reopening this defect.
+ *
+ * ⛔ Unlisted factories are never wrapped. `ObjectSchema.create` itself stays
+ * strict everywhere else: this list is read by {@link authoredSourcePlugin}
+ * alone, which only `os migrate meta` installs.
+ */
+export const STRICT_AUTHORING_FACTORIES: readonly StrictAuthoringFactory[] = Object.freeze([
+  { owner: 'ObjectSchema', member: 'create', home: '@objectstack/spec/data' },
+  { owner: 'App', member: 'create', home: '@objectstack/spec/ui' },
+  { owner: 'Dashboard', member: 'create', home: '@objectstack/spec/ui' },
+  { owner: 'Report', member: 'create', home: '@objectstack/spec/ui' },
+  { owner: 'Action', member: 'create', home: '@objectstack/spec/ui' },
+]);
+
+/** What one `@objectstack/spec` entrypoint gives the shim to wrap. */
+interface AuthoredSourceHelpers {
+  /** Its `define*` helpers, by {@link DEFINE_HELPER_RE}. */
+  readonly defineHelpers: readonly string[];
+  /** Its {@link STRICT_AUTHORING_FACTORIES}, as owner export → factory members. */
+  readonly factories: ReadonlyMap<string, readonly string[]>;
+}
+
+/**
+ * The strict authoring surface a given `@objectstack/spec` entrypoint exports —
+ * its `define*` helpers and its {@link STRICT_AUTHORING_FACTORIES} — read from
+ * the copy **the config itself would import** (resolved from the config's own
+ * directory, not the CLI's).
+ *
+ * A listed factory is read by PROPERTY (`ns[owner][member]`), never by
+ * enumerating the owner: `ObjectSchema` is a lazy-schema Proxy, and its
+ * `ownKeys` trap throws.
+ *
+ * Returns nothing to wrap — i.e. "shim nothing" — when the entrypoint cannot
+ * be resolved or imported. That is the safe direction: an unshimmed load is
+ * exactly today's behaviour, so a project the enumeration cannot read is no
+ * worse off than before.
  */
-async function defineHelpersOf(specifier: string, requireFromConfig: NodeRequire): Promise<string[]> {
+async function authoredSourceHelpersOf(
+  specifier: string,
+  requireFromConfig: NodeRequire,
+): Promise<AuthoredSourceHelpers> {
   try {
     const resolved = requireFromConfig.resolve(specifier);
     const ns = (await import(pathToFileURL(resolved).href)) as Record<string, unknown>;
-    return Object.keys(ns).filter((k) => DEFINE_HELPER_RE.test(k) && typeof ns[k] === 'function');
+    const defineHelpers = Object.keys(ns).filter((k) => DEFINE_HELPER_RE.test(k) && typeof ns[k] === 'function');
+    const factories = new Map<string, string[]>();
+    for (const { owner, member } of STRICT_AUTHORING_FACTORIES) {
+      const value = ns[owner];
+      if (value === null || (typeof value !== 'object' && typeof value !== 'function')) continue;
+      if (typeof (value as Record<string, unknown>)[member] !== 'function') continue;
+      factories.set(owner, [...(factories.get(owner) ?? []), member]);
+    }
+    return { defineHelpers, factories };
   } catch {
-    return [];
+    return { defineHelpers: [], factories: new Map() };
   }
 }
 
+/**
+ * The helpers every generated shim module opens with.
+ *
+ * `__tolerant` is the try-real-then-authored wrap; `__tolerantMembers` applies
+ * it to a factory member and hands every other member of the owner through
+ * untouched — a Proxy rather than a copy, because the owner may itself be a
+ * lazy-schema Proxy that cannot be enumerated.
+ *
+ * `__refusal` renders a raw `ZodError` — whose `message` is its issues as a
+ * JSON array — through the project's own `formatZodError`, so the swallowed
+ * verdict reads like the loader's `defineStack validation failed` block rather
+ * than as a JSON dump. An error that already carries prose keeps its message.
+ */
+const AUTHORED_SOURCE_PRELUDE: readonly string[] = [
+  `const __refusal = (label, error) =>`,
+  `  error && error.name === 'ZodError' && Array.isArray(error.issues)`,
+  `    && typeof __specRoot.formatZodError === 'function'`,
+  `    ? __specRoot.formatZodError(error, label + ' validation failed')`,
+  `    : (error && error.message) || String(error);`,
+  `const __tolerant = (label, call) => (...authored) => {`,
+  `  try {`,
+  `    return call(...authored);`,
+  `  } catch (error) {`,
+  `    console.warn(`,
+  `      '[authored-source] ' + label + '(): the current schema refuses this '`,
+  `      + 'artifact, so it is handed to the migration chain exactly as authored. '`,
+  `      + __refusal(label, error),`,
+  `    );`,
+  `    return authored[0];`,
+  `  }`,
+  `};`,
+  `const __tolerantMembers = (owner, ownerName, members) => {`,
+  `  const wrapped = new Map(members.map((member) => [`,
+  `    member,`,
+  `    __tolerant(ownerName + '.' + member, (...authored) => owner[member](...authored)),`,
+  `  ]));`,
+  `  return new Proxy(owner, {`,
+  `    get: (target, prop) => (wrapped.has(prop) ? wrapped.get(prop) : Reflect.get(target, prop)),`,
+  `  });`,
+  `};`,
+];
+
 /**
  * Load an authored config **as authored**, for the one consumer whose input is
  * a source the CURRENT schema is expected to refuse: the `os migrate meta`
@@ -317,14 +450,21 @@ async function defineHelpersOf(specifier: string, requireFromConfig: NodeRequire
  *
  * Each `@objectstack/spec` entrypoint the config imports is replaced by a
  * generated module that re-exports the real one and wraps its `define*`
- * helpers as **try-real-then-authored**:
+ * helpers — and the {@link STRICT_AUTHORING_FACTORIES} it carries, such as
+ * `ObjectSchema.create` — as **try-real-then-authored**:
  *
  * ```js
  * export const defineView = (...authored) => {
  *   try { return realDefineView(...authored); } catch { return authored[0]; }
  * };
  * ```
  *
+ * A factory is wrapped in place on its owner: `ObjectSchema` stays the real
+ * schema for every other member (`parse`, `shape`, …), and only `create` is
+ * tolerant. Both kinds are strict at the call, so both must be wrapped for the
+ * chain to convert first — a `defineStack` wrap alone never ran, because the
+ * `ObjectSchema.create(…)` inside its argument threw before it was called.
+ *
  * The narrowness is the point, and it is what keeps this a restoration rather
  * than a widening of what the command accepts:
  *
@@ -370,26 +510,22 @@ function authoredSourcePlugin(configPath: string): Plugin {
       });
 
       build.onLoad({ filter: /.*/, namespace: AUTHORED_SOURCE_NAMESPACE }, async (args) => {
-        const helpers = await defineHelpersOf(args.path, requireFromConfig);
+        const { defineHelpers, factories } = await authoredSourceHelpersOf(args.path, requireFromConfig);
         const spec = JSON.stringify(args.path);
         const lines = [
           `import * as __real from ${spec};`,
+          `import * as __specRoot from ${JSON.stringify(SPEC_ROOT_MODULE)};`,
           `export * from ${spec};`,
+          ...AUTHORED_SOURCE_PRELUDE,
         ];
-        for (const name of helpers) {
+        for (const name of defineHelpers) {
+          lines.push(
+            `export const ${name} = __tolerant(${JSON.stringify(name)}, (...authored) => __real.${name}(...authored));`,
+          );
+        }
+        for (const [owner, members] of factories) {
           lines.push(
-            `export const ${name} = (...authored) => {`,
-            `  try {`,
-            `    return __real.${name}(...authored);`,
-            `  } catch (error) {`,
-            `    console.warn(`,
-            `      '[authored-source] ' + ${JSON.stringify(name)} + '(): the current schema refuses this '`,
-            `      + 'artifact, so it is handed to the migration chain exactly as authored. '`,
-            `      + ((error && error.message) || String(error)),`,
-            `    );`,
-            `    return authored[0];`,
-            `  }`,
-            `};`,
+            `export const ${owner} = __tolerantMembers(__real.${owner}, ${JSON.stringify(owner)}, ${JSON.stringify(members)});`,
           );
         }
         return { contents: lines.join('\n'), loader: 'js' };