Per-node osquery flags overrides

[javuto]

Per-node osquery flags overrides

Follow-up to per-node configuration overrides (#1091). Needs the companion osctrld change, which sends the host's identity and guards restarts. Either side can be deployed first.

Problem

During an investigation, operators need one host to collect more than its environment: file integrity monitoring, process events, other evented tables. Much of that is switched on by startup flags (--disable_events=false, --enable_file_events=true, --audit_allow_process_events=true, …). Remote configuration can't set them, so configuration overrides can't reach them, and changing the environment's flags changes every node.

What changes

Admins of a node's environment can now set a flags override for that node only. It is merged on top of the environment's flags and reaches the host through osctrld, which writes the flagfile and restarts osqueryd.

Node page (SPA)

• Switch: the Configuration tab gains a Configuration · Flags switch. Configuration is the default and is unchanged.
• Flags view: Applied (default) · Environment · Node override · Diff, read-only plain text.
• Status line: "Managed by osctrld — last fetched flags ‹time›", or a warning that no osctrld has been seen for the host. When osquery rejected the current override: "osquery on this host rejected this override on ‹time› and osctrld restored the environment's flags: ‹error›". Host-reported text is shown as plain text.
• Conflict warning: flags the override shares with the configuration's options, which can take precedence.
• Editing:
  • Edit flags override is behind a confirmation ("Override flags for ‹hostname›?" with the "I understand … I know what I'm doing" checkbox).
  • Optional expiry, inline server errors.
  • Remove flags override has its own confirmation.
• Indicators: the banner and tab dot cover flags overrides too.
• Visibility: shown, and fetched, only for admins of the node's environment.

osctrl-api

• GET / PUT / DELETE /api/v1/nodes/{env}/node/{uuid}/flags, reusing the configuration override's target resolution:
  • AdminLevel on the node's environment, with denials audited via denyEnv;
  • node lookup by UUID only, scoped to the environment;
  • 256 KiB body limit (413);
  • expiry must be in the future;
  • validation errors return 400 naming the flag or line;
  • every set and removal is audited with before and after.
• Response fields:
  • the environment flags, with secret and cert path placeholders;
  • the override and the applied (merged) flags;
  • expiry, expired, and who updated it and when;
  • conflicts;
  • osctrld: last fetched, the apply result and error as reported by the host, and rejected_current.
• rejected_current is true only when the override exists, hasn't expired, and the rejection is newer than the override's last update. Host-reported times are capped at the server's last-fetch time.
• Node deletion (API and CLI database mode) hard-deletes the node's flags override and its osctrld status.
• OpenAPI is regenerated.

osctrl-tls (osctrld endpoints)

• FlagsRequest gains optional uuid and apply fields.
• When an override is merged: only when uuid is a valid UUID of a node in that environment. The handler then records the fetch and merges that node's active override into the flags returned by osctrld-flags and osctrld-verify. An active override is unexpired and saved for the node's current environment.
• Untrusted apply input: recorded only with result = ok | rejected and a 64-hex SHA-256. A malformed block is dropped, and the fetch is still recorded.
• Re-validation: the stored override is validated again at serve time. Any error is logged and the environment flags are served.
• Unchanged otherwise: a request without uuid, an unknown node, or a node of another environment gets exactly today's response.

pkg/nodeconfig

• ValidateFlagsOverride:
  • one flag per line (--name, --name=value, or the single-dash forms), with # comments and blank lines allowed;
  • no duplicate names;
  • protected connection flags refused:
    • prefixes tls_, config_, logger_tls_, distributed_tls_, enroll_;
    • exact names: config_plugin, logger_plugin, distributed_plugin, disable_distributed, disable_enrollment, disable_reenrollment, host_identifier, specified_identifier, flagfile, database_path, db_path, the carver endpoints, proxy_hostname, read_max, force;
    • and the no<name> form of each.
• MergeFlags: byte-stable for the same inputs, so osctrld never restarts osquery for nothing. An override flag replaces every environment line with that name in place, keeping its dash prefix. New flags are appended in override order.
• FlagOptionConflicts: override flags that also appear in the environment configuration's or the node's configuration override's options.
• FlagsManager: table node_flags_overrides, cache osctrl:nodeflags:<uuid>, with its own expiry and removal, separate from configuration overrides.
• Shared cache protocol: one generic helper now backs both override types. osctrl-api writes the full state after every change; osctrl-tls only fills a missing entry with SET NX; if Redis fails, the override is read from SQL. The configuration override's API, cached JSON, keys and behaviour are unchanged, and its tests are untouched.
• osctrld status record: table node_osctrld_status (last fetched, apply result, SHA-256, error, apply time). Upserted per fetch; the error is capped at 1 KiB.

Security and operational impact

• Authorization and limits: checked server-side on all three methods, with CSRF protection on PUT and DELETE. Denials and changes are audited. A node from another environment returns 404, and a node can't be targeted by setting its hostname to another node's UUID.
• The connection to osctrl is protected. Flags controlling TLS, config, logger, distributed, enroll, identity, the flagfile and the database path are refused at save and again at serve.
  • The server only checks the format of other flags. An unknown flag or bad value is caught on the host: osctrld rolls back to the environment's flags if osqueryd won't start, and the page shows the rejection.
• Delivery and revert:
  • Overrides reach only hosts managed by osctrld in service mode, on its next sync (default 60 minutes), with an osquery restart.
  • Removing or expiring an override fully reverts the host on the following sync, because osquery restarts.
• Accepted trade-off: the host's UUID is claimed by osctrld and authorized only by the environment secret. Anyone holding that secret can read a given node's flags override, which holds paths and settings, not secrets.
• Schema: two new tables via AutoMigrate (node_flags_overrides, node_osctrld_status) with explicit, distinct index names. No changes to existing tables.
• Load: each osctrld sync that carries a UUID adds one node lookup and one upsert, about once an hour per host.

Validation

• go test ./... -count=1: 56 packages pass
• go vet ./...; gofmt -l pkg cmd (empty); golangci-lint run ./...: 0 issues
• make openapi-check: clean
• cd frontend && npm test: 66 files, 522 tests pass; npm run check: clean
• Redis cache test against a local Redis: pass
• New tests, each seen failing first:
  • pkg/nodeconfig:
    • every protected name pinned literally, including no forms and db_path;
    • format, duplicate and empty refusals;
    • merge: in place, every occurrence, append order, boolean forms, byte-stable;
    • conflicts;
    • FlagsManager cache protocol: cached absence, a refill never overwriting a newer write, SQL fallback, expiry, wrong environment;
    • both override tables migrating side by side;
    • status upsert and error truncation.
  • cmd/tls/handlers:
    • merged override on both endpoints;
    • unchanged response without, or with a foreign, unknown or malformed uuid;
    • fetch and apply recorded, and a malformed apply dropped;
    • expired or tampered stored overrides fall back;
    • wrong secret records nothing.
  • cmd/api/handlers and cmd/cli:
    • permissions, with every denial audited;
    • cross-environment 404 and hostname decoy;
    • validation errors, past expiry and oversized bodies refused;
    • audit before and after;
    • conflicts;
    • osctrld status, rejected_current including the future-time and expired cases;
    • node deletion through the API and both CLI paths.
  • Frontend:
    • Configuration · Flags switch;
    • read-only by default; unlock requires the checkbox;
    • save, cancel, inline errors, remove;
    • status line variants and plain-text host errors;
    • conflicts warning;
    • banner and dot for a flags-only override;
    • no request for non-admins.
• Not run: a live host rejecting a flag end to end, and Playwright.