fix(server): treat a WebAssembly trap as terminal instead of reopening forever - #8
Merged
Merged
Conversation
`localapp server run` asked the OS for an ephemeral port on every start, so the address changed on each restart. Anything that has to name the Server — a saved profile, a reverse proxy, a published container port, or a terminal that keeps one process alive — could not pin it, and the port a running Server reported had to be copied by hand into every client. Default to 50524 and keep `--port` as the override, including `--port 0` for callers that really want an ephemeral port. The container image already passes an explicit port, so its behaviour is unchanged. The foreground mode is also a first-class way to run a personal Server when a fixed address matters or when the machine cannot register a scheduled task: docs/local-runtime.md now says so next to the daemon path. Verified on the packaged build: no arguments listens on 127.0.0.1:50524 with /health answering 200, and `--port 55441` moves it there and releases 50524.
…g forever A production Server died from `RuntimeError: memory access out of bounds` at `openMetaDbFromDisk` -> `new SqlJs.Database(...)`. That stack is the clue: the open only happens when the cached database was already evicted, so the trap had been recognized before — the recovery was the bug. `initSqlJs()` returns the same Emscripten module instance on every call, so once a trap tears it, reopening the database in-process traps again on the very next construction. The previous code evicted the handle and reopened anyway, which turned one trap into every database request failing, and — because the 5s request-log flush swallows its own errors — did so invisibly for 46 minutes, until the 6h cleanup callback hit the same trap without a catch and killed the process. Record the runtime as terminal on the first trap, refuse later database access with `db_runtime_restart_required` (503) instead of a doomed reopen, and stop the process once so its supervisor starts a clean one — the only recovery that exists for a torn module instance. Timer callbacks that touch SQLite also had no error boundary at all: six of them plus two SSE heartbeats. Route them through `guardTimerCallback`, which sends a trap to the same terminal stop, keeps a genuine timer failure from vanishing, and captures asynchronous rejections. The previous app-db test asserted the opposite guarantee — it simulated the trap with a JS-thrown RuntimeError, which leaves the module intact, so in-process reopen appeared to work. It now asserts the real one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
修掉那次「越界会死」的根因。证据来自你机器上的致命栈,结论是:平台认出了 trap,但它的"恢复"本身就是 bug。
致命栈给出的关键信息
getDb()只在db === null时才重开 ⇒ evict 已经发生过一次,即平台认出了 WASM 错误。所以问题不是"没认出来",而是重开那一刻又 trap。根因:
initSqlJs()重复调用返回同一个模块实例平台原有的恢复是「弃用 handle + 从磁盘重开」(
app-db里还会把缓存的SqlJs置空)。我实测:模块实例是同一个、而且已经被 trap 撕裂,所以进程内不存在任何可用的恢复——重开必然在
new SQL.Database(...)上再 trap 一次。于是:try { … } catch { /* swallow */ },第一次 trap 就发生在它里面并被吞掉 ⇒ 进程带着坏模块又跑了 46 分钟没人知道;改了什么
① trap 一旦发生就是终局,不再假装恢复(
runtime-errors.ts、app-db.ts、meta-sqlite.ts)新增
markSqlJsRuntimeUnusable(err, scope):记录终局状态、打一行明确日志、让进程停止一次(退出码非 0),由监督者/看门狗拉起全新进程——对撕裂的模块实例,这是唯一存在的恢复。之后任何碰库的请求拿到db_runtime_restart_required(503),而不是再去重开撞一次。重复 trap 不会再次触发停止。② 碰库的定时器套上错误边界(新增
timer-guard.ts)一共 8 处:6 小时的 desktop/device action 清理、5 秒的请求日志 flush、30 秒的验证会话清理、60 秒的 app 库空闲关闭、两处 15 秒 SSE 心跳。
guardTimerCallback把 trap 送到上面同一条终局路径;普通定时器失败只记日志、不再静默消失;异步 reject 也会被捕获。③ 那条吞异常的 catch 不再吞致命错误(
request-logger.ts)——正是它让故障隐身 46 分钟。一条测试曾经保证了错的结论
tests/integration/app-db.test.ts里有个用例断言"evict 后重开可用",它用 JS 抛出的WebAssembly.RuntimeError模拟 trap——这种异常不会撕裂 emscripten 模块,所以测试里重开确实可用,真机上必然不可用。我把它改成断言真实语义(终局 + 拒绝访问 + 只请求停止一次),并在注释里写清为什么旧断言是错的。验证
packages/server-core:27 文件 / 205 用例全过packages/server:失败集合与本机改动前基线完全一致(8 个 Windows 环境类文件、14 条:可执行启动、rename EPERM、clone 夹具等),无新增失败runtime-errors.test.ts(3)、timer-guard.test.ts(3)、改写后的app-db.test.ts(5 全过)tsc通过未包含 / 仍需你那边确认
db为何会第一次 trap,本轮没有定论——数据文件在崩溃时是完好的(integrity_check=ok、816 KB),所以不是数据损坏或体积问题。code/details(平台是 Fastify 默认序列化),你们客户端也把json.code打出来——这次的 46 分钟静默里,客户端只保留了HTTP 500 Internal Server Error。