Timeout knob for aft_search

[tkozybski]

Description

Per 0.56.2 changelog:

Natural-language aft_search queries no longer stall when the local embedding model is slow. The query-time budget (3 s, then lexical results with a note) applied to remote embedding backends but not to the bundled ONNX one, so a slow inference held the whole search for as long as it took. The local embed now runs under the same budget on its own worker, a query that exceeds it falls back to lexical search, the late vector is kept for the next identical query, and ONNX worker threads are capped by the container's CPU quota instead of the visible core count. Identifier-shaped queries were unaffected, which is why single-word searches kept working.

aft_search hardcodes a 3-second budget. But some self-hosted embedding servers could simply be hosted on slower machines, the provider had a temporary spike, or any other short-term issue. aft_search is not used that often to require a 3-second timeout. It is only used for initial discovery, and accurate results are more desired than fast ones.

I propose two things to be changed:
a) Increase the default timeout to 30 seconds.
b) Add a configuration knob, probably to the embedding configuration, so the user can choose the query time budget they are comfortable with

Use case

Increase accuracy of aft_search. Utilizing the embedding feature is most of its strength, and a 3-second budget reduces the effectiveness of this tool substantially

[aft-alfonso]

Thanks — the knob already exists, and the 0.56.2 note should have named it: semantic.query_timeout_ms (user config, ~/.config/cortexkit/aft.jsonc), default 3000, currently clamped to 500–15000. It bounds only the interactive query embed; index builds use semantic.timeout_ms, which scales itself from measured per-item latency.

"semantic": { "query_timeout_ms": 15000 }

Two things worth knowing about what happens at the budget:

• A query that exceeds it is not lost. It falls back to lexical results with a note naming the knob, and the late vector is kept, so the same query re-run lands on the semantic path. A cold ONNX model on a slow box typically pays this once.
• The budget is per query embed, not per search; a 15 s value does not make every search take 15 s, only the ones whose embed would otherwise have stalled.

On the two proposals: (b) is done as above. For (a), the default stays at 3 s — the tool is called inside agent turns where a stalled embed blocks the whole turn, and the fallback is designed to keep those turns moving — but I agree the 15 s ceiling is arbitrary for a self-hosted provider on a spike. I'll raise the cap to 30 s so the knob can express your case; that ships in the next patch.

[aft-alfonso]

Fixed in v0.57.0, published just now to npm and crates.io.

Upgrade with npx @cortexkit/aft@latest doctor --fix, or let the plugin pick it up on its next start.

Thanks for the report — closing, but please reopen if you still see it on 0.57.0.