Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .changeset/20677-ledger-disabled-stays-unbound.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
'@objectstack/service-automation': patch
---

fix(service-automation): a flow switched off in the activation ledger stays unbound after a restart, and a trigger-fired refusal no longer logs an ERROR claiming a run-history row (#20677)

Clause-②: no

**What was wrong.** A packaged flow switched off through the ADR-0126 activation
ledger (`POST /api/v1/automation/:name/toggle` with `enabled: false`) came back
`bound: true` after every cold restart. Its runs were still refused, so the switch
itself held, but its trigger was armed again. At boot the automation service pulls
the flows and applies the ledger, which leaves a switched-off flow unbound. The
trigger plugins register later, at `kernel:ready`, and registering a trigger armed
every matching flow without asking whether it may run. So `GET
/api/v1/automation/_status` reported the flow `enabled: false, bound: true`. Each
matching event also logged `ERROR Trigger-fired run of flow '…' failed`, saying the
failure "is recorded in the flow's run history", while no run row was written.

**What changed.**

- The engine checks whether a flow may run in one place: at the step that arms a
trigger. Every arming path goes through it: flow registration (boot pull,
publish, hot reload), trigger registration, and the enable toggle. A flow that
either disable dimension switches off (the activation ledger, or an `obsolete` /
`invalid` status) is never armed, whenever its trigger registers.
- Re-enabling a flow arms it on its trigger as before. Re-enabling the ledger bit
of a flow whose `status` is still `obsolete` or `invalid` no longer arms it, since
every run it fired would be refused.
- A trigger-fired run refused because the flow is disabled (for example, an event
already in flight when the flow was switched off) is logged at `info`, saying
nothing ran and no run-history row records it. It is no longer an `ERROR`.
- The `ERROR` line for any other trigger-fired failure says the failure is
recorded in the run history only for a run that dispatched and failed. A run
refused before it dispatched gets the same line without that claim.

**What is not affected.** The runtime refusal (`FLOW_DISABLED`) and its message are
unchanged. The enabled flows beside a disabled one arm exactly as before. No export,
option, route or response shape changes.
91 changes: 74 additions & 17 deletions packages/services/service-automation/src/engine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3371,6 +3371,10 @@ export class AutomationEngine implements IAutomationService {
// A trigger may be registered *after* its flows (e.g. AutomationServicePlugin
// pulls flows at start(); a trigger plugin wires up on kernel:ready, which
// fires later). Activate any already-registered flow that maps to this type.
// [ADR-0126 §7.2] A flow that may not run is skipped by
// `activateFlowTrigger`'s own enablement gate, so the flows the ledger
// hydration left unbound stay unbound here — ⛔ no second check in this
// loop: the gate is shared so no arming path can go without it.
for (const name of this.flows.keys()) {
if (this.boundFlowTriggers.has(name)) continue;
const resolved = this.resolveTriggerBinding(name);
Expand Down Expand Up @@ -3584,11 +3588,34 @@ export class AutomationEngine implements IAutomationService {

/**
* Bind a flow to its matching registered trigger (idempotent). No-op when
* the flow has no trigger binding or no trigger is registered for its type
* yet — {@link registerTrigger} re-attempts activation when one arrives.
* the flow may not run ({@link isFlowEnabled}), when it has no trigger
* binding, or when no trigger is registered for its type yet —
* {@link registerTrigger} re-attempts activation when one arrives.
*/
private activateFlowTrigger(flowName: string): void {
if (this.boundFlowTriggers.has(flowName)) return;
// [ADR-0126 §7.2] THE enablement gate for arming — here, at the one
// point every arming path crosses, and not in its callers. A flow
// either disable dimension switches off (the activation ledger, or an
// `obsolete` / `invalid` status) is never handed to a trigger, whichever
// path asks: {@link registerFlow} (boot pull, publish, hot reload),
// {@link registerTrigger}, the enable half of {@link toggleFlow}, and
// any path added later.
//
// Why it cannot live in the callers: `registerTrigger` runs when a
// trigger plugin registers at `kernel:ready`, AFTER `start()` pulled
// the flows and {@link hydrateFlowActivations} unbound the switched-off
// ones. While only `registerFlow` asked, that later registration
// re-armed every one of them on every cold boot — `/_status` reported a
// disabled flow `bound: true`, and each matching event fired a run
// `execute()` then refused.
//
// Silent on purpose: an unarmed disabled flow is the state the switch
// exists to produce, the host's hydration line already named it, and
// `getFlowRuntimeStates()` reports it `enabled: false`. Ahead of the
// scheduled-work policy gate below for the same reason — a flow that
// may not run has no policy refusal to record.
if (!this.isFlowEnabled(flowName)) return;
const resolved = this.resolveTriggerBinding(flowName);
if (!resolved) return;
// [#17396] The deployment gate, read HERE rather than only inside the
Expand Down Expand Up @@ -3661,18 +3688,44 @@ export class AutomationEngine implements IAutomationService {
// landed; the only other trace is the passive run-history row.
// That stderr also survives the CLI's boot-quiet stdout window is
// stream mechanics, not the verdict.
//
// [ADR-0126 §7.2] The line states only what happened. Two kinds of
// `execute()` answer are not a failed run:
// - `FLOW_DISABLED`: the flow was switched off after the trigger
// took the event — an event in flight at the switch-off, or a
// trigger whose `stop()` failed. The refusal IS the switch
// working, so it is said at `info`, never as an `error` that
// reads as a production failure. The enablement gate above
// keeps a disabled flow from being armed at all, so this is the
// residue, not the steady state.
// - every other never-dispatched exit (a `code` or a missing
// flow, and no `status`): no node ran, so the line claims no
// run-history row. Only a run that dispatched and failed
// carries `status: 'failed'` — the verdict that exit also wrote
// to the run history.
trigger.start(resolved.binding, (ctx: AutomationContext) =>
this.execute(flowName, ctx).then((result) => {
if (!result.success) {
this.logger.error(
`Trigger-fired run of flow '${flowName}' failed (trigger '${resolved.triggerType}') — ` +
`no caller holds this result and nothing retries the run; the terminal failure ` +
`is recorded in the flow's run history, and the run's failure envelope is in ` +
`this record's meta.`,
undefined,
{ error: result.error ?? 'unknown error' },
if (result.success) return;
if (result.code === 'FLOW_DISABLED') {
this.logger.info(
`Trigger '${resolved.triggerType}' fired flow '${flowName}', which is disabled — the ` +
`run was refused before it started, nothing ran, and no run-history row records ` +
`it. The refusal is in this record's meta.`,
{ error: result.error },
);
return;
}
this.logger.error(
`Trigger-fired run of flow '${flowName}' failed (trigger '${resolved.triggerType}') — ` +
`no caller holds this result and nothing retries the run; ` +
(result.status === 'failed'
? `the terminal failure is recorded in the flow's run history, and the run's ` +
`failure envelope is in this record's meta.`
: `it was refused before it dispatched, and the refusal envelope is in this ` +
`record's meta.`),
undefined,
{ error: result.error ?? 'unknown error' },
);
}),
);
this.boundFlowTriggers.set(flowName, resolved.triggerType);
Expand Down Expand Up @@ -4263,14 +4316,15 @@ export class AutomationEngine implements IAutomationService {
}

// Re-bind in case the definition changed its trigger, then (re)activate.
// [ADR-0126 §7.2] A ledger-disabled flow is NOT re-armed here, which is
// what makes the unbind survive a republish and a restart: the boot
// pull re-registers every flow, so a hydrated ledger row has to be
// able to keep a trigger unbound through exactly this path.
// [ADR-0126 §7.2] A disabled flow — ledger or status — is NOT re-armed
// here, which is what makes the unbind survive a republish and a
// restart: the boot pull re-registers every flow, so a hydrated ledger
// row has to be able to keep a trigger unbound through exactly this
// path. The refusal is `activateFlowTrigger`'s own enablement gate,
// the one every arming path shares — ⛔ not re-asked here, where a
// caller-side check once stood alone and `registerTrigger` had none.
this.deactivateFlowTrigger(name);
if (this.isFlowEnabled(name)) {
this.activateFlowTrigger(name);
}
this.activateFlowTrigger(name);

// #12206 (Option A) — hand the caller the canonicalized flow this
// registration stored: the same object `this.flows` now holds and
Expand Down Expand Up @@ -4657,6 +4711,9 @@ export class AutomationEngine implements IAutomationService {
// A disabled flow should stop receiving trigger events; a re-enabled one
// should resume. execute() also guards disabled flows, but unbinding
// avoids firing the trigger (and its event-source subscription) at all.
// Re-enabling moves only the LEDGER bit: a flow whose `status` still
// disables it stays unarmed, by `activateFlowTrigger`'s enablement gate
// — armed, it would only fire runs `execute()` refuses.
if (enabled) {
this.activateFlowTrigger(name);
} else {
Expand Down
Loading
Loading