Skip to content

feat(openai,compose): harden Responses WebSocket delivery; steer verbatim text through input - #29

Merged
danielkov merged 4 commits into
fix/openai-websocket-proxy-policyfrom
feat/openai-responses-resilience
Sep 30, 2026
Merged

danielkov merged 4 commits into
fix/openai-websocket-proxy-policyfrom
feat/openai-responses-resilience

Conversation

@danielkov

Copy link
Copy Markdown
Owner

Summary

Makes the Responses WebSocket transport as resilient as HTTP/SSE and keeps prompt-cache hits across sessions that share a cache key. Also makes the Runlet compose tool steer text that would need escaping into input, and moves it to runlet 0.6.1.

Stacked on #28 and based on its branch; GitHub retargets this PR to main when #28 merges. Both provider changes ship together as agentkit-provider-openai 0.10.12. agentkit-tool-compose goes to 0.10.12.

Provider (agentkit-provider-openai)

  • Resend after visible output. A WebSocket that drops, stalls or times out after output has streamed used to fail the turn, while HTTP retried and superseded the attempt. With store: false and response-attempt supersession enabled, the adapter now emits ResponseAttemptSuperseded and resends the full request on a fresh socket. Without supersession, visible output is still never replayed.
  • Progress-based stall detection. with_progress_timeouts(first, gap) bounds an attempt by response progress, so keepalive and metadata traffic can no longer hold a stalled attempt open until the attempt timeout.
  • Cache routing. session-id follows the request's prompt cache key, and thread-id and x-codex-routing-hint are sent. Requests that share a key reach the backend holding their prefix, instead of starting cold.
  • Turn-state. Captured from codex.response.metadata, which can arrive before response.created. It is scoped per session and reset on a new user message or an authentication change.
  • No warmup request. It cost a round trip and added a failure mode without improving cache hits.

Compose (agentkit-tool-compose)

  • input comes before script in the schema and is described as verbatim. Text that would need escaping in a string literal, such as file contents or code with quotes or backslashes, goes in input and is referenced from the script.
  • The primer's JSON-parse example is replaced with a code edit that writes input unchanged, and it uses runlet 0.6.1 single-quoted strings.
  • Requires runlet 0.6.1, which adds raw newlines and single quotes in strings, verbatim unknown escapes, and separate binding and callable namespaces.

Impact

No API removals. New: OpenAIResponsesConfig::with_progress_timeouts. Consumers that opt into response-attempt supersession now receive superseded attempts on WebSocket as well as HTTP.

…uting

- Scope turn-state per session: capture it from `codex.response.metadata`
  (which can precede `response.created`), reset it on a new user message or
  authentication change, and forward it on WebSocket requests.
- Send `x-codex-routing-hint`, `thread-id`, and a `session-id` that follows
  the request's prompt cache key, so requests sharing a cache key reach the
  backend that holds their prefix.
- Detect stalls by response progress rather than raw traffic
  (`with_progress_timeouts`); keepalive and metadata events no longer reset
  the timers.
- Resend a lost, stalled, or timed-out WebSocket request in full on a fresh
  socket when `store: false`: before visible output always, and after it when
  the consumer opts into response-attempt supersession, as HTTP already does.
- Remove the warmup request.
…0.6.1

- Order `input` before `script` in the compose schema and describe it as
  verbatim: text that would need escaping as a string literal goes there.
- Replace the JSON-parse primer example with a code-edit example that writes
  `input` contents unchanged, and use runlet 0.6.1 single-quoted strings.
- Require runlet 0.6.1 and bump to 0.10.12.
@danielkov
danielkov added this pull request to stack #30 September 30, 2026 13:19

@kit-code-agent kit-code-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two correctness issues remain: transcript compaction can associate sticky routing state with the wrong user turn, and the first-progress timeout excludes the wait for HTTP response headers.

Comment thread crates/agentkit-provider-openai/src/responses.rs Outdated
Comment thread crates/agentkit-provider-openai/src/responses.rs
…first progress

- Scope x-codex-turn-state by the request's turn ID instead of the last user
  item's transcript position, which compaction can move or reuse.
- Start the first-progress deadline before sending, bound opening the attempt
  by it, and carry the elapsed time into the stream, so waiting for response
  headers counts against the same limit.

@kit-code-agent kit-code-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found. The changes look good to merge.

@danielkov
danielkov merged commit e0e5c77 into main Sep 30, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant