odai - pronounced like the trickster; it lives in your machine and does your chores.
odai is local-only - the primary backend is Chrome's built-in AI (the Prompt API, stable since Chrome 148) via installed Google Chrome on every platform; llama-server (loopback) is the local fallback; Apple FM and Phi Silica (Copilot+) are opportunistic per-OS extras. No cloud, no remote endpoints, no keys.
The Prompt API is model-agnostic by design, so odai names its backend for the interface rather than the model. The on-device model is Gemini Nano today; Gemma 4 is the base for the next Gemini Nano and is already testable in Chrome Canary behind the "Gemma 4 for Built-in AI" flag.
@socketsecurity/odai is a local, on-device AI library for browser and Node.
It wraps the browser's built-in AI Prompt API behind a type-safe,
backend-agnostic seam, hardens small-model JSON output, and ships bench - an
evaluation harness that scores any backend on real Socket workloads. It exists
so Socket code can feature-detect, prompt, and parse on-device model responses
without scattering DOM-specific checks across consumers.
pnpm install @socketsecurity/odai
ODAI_CHROME_MODEL=gemma4 pnpm exec odai setupodai setup discovers Google Chrome, prepares a dedicated persistent profile,
downloads the selected on-device model when needed, verifies the responding
model, closes Chrome, and prints a JSON receipt. Run it once after installation
and again when the receipt check reports that the model is unavailable.
Applications that manage their own setup can use the Node API. The cleanup callback receives the measured storage deficit and may remove only data the application owns. Odai measures free space again before it starts Chrome.
import { setupChromeBuiltin } from '@socketsecurity/odai/node'
await setupChromeBuiltin({
model: 'gemma4',
async reclaimStorage(pressure) {
await removeApplicationCaches(pressure.minimumBytes)
},
})import { createOdaiModel } from '@socketsecurity/odai'
const model = await createOdaiModel()
const { raw } = await model.promptStreaming(
'Summarize this page in one sentence.',
)
console.log(raw)createOdaiModel picks a backend by precedence: the explicit backend
option, then the ODAI_BACKEND env var, then the availability probe order -
chrome-builtin, llama-server, apple-fm, windows-phi-silica.
Select simulator explicitly for testing.
Node consumers can classify a request against their own action catalog:
import { classifyIntent, withOdaiModel } from '@socketsecurity/odai/node'
const result = await withOdaiModel(
model =>
classifyIntent(model, {
query: 'Inspect this project for dependency risks.',
candidates: [
{ id: 'inspect-project', description: 'Inspect project dependencies.' },
{
id: 'repair-project',
description: 'Apply dependency security repairs.',
},
],
}),
{ timeoutMs: 5000 },
)A successful result contains an allowed actionId or null for abstention.
The caller owns argument validation and execution.
Invalid output returns a failed task result.
Cancellation and backend errors reject the operation.
withOdaiModel requires an already running, cancellable loopback llama-server backend.
It performs no model installation or backend preparation.
Its shared deadline covers discovery, prompts, and session cleanup.
The callback's second argument supplies abortSignal for additional cancellable work.
probeBackendAvailability checks the same eligible backend without creating a model session.
The package ships a odai bin for single-shot, keyless AI steps in scripts
and CI. Input arrives on stdin or --input; the parsed result prints as one
JSON line on stdout, diagnostics go to stderr.
Command examples, the exit-code contract, and batch mode
git diff | odai commit-msg
printf 'Critical: 2\nHigh: 5\n' | odai triage
odai summarize --input README.md
odai patch --input src/greet.js --instruction "use a template literal"
odai classify-deps --input narrowed-dep-diff.json
odai lockstep --input prepared-lockstep.json
odai backends
printf '%s\n' '{"id":"a","task":"summarize","input":"release notes..."}' '{"id":"b","task":"commit-msg","input":"diff --git..."}' | odai batchEvery prompt runs under a hard budget - --timeout <ms> or the
ODAI_TIMEOUT_MS env var, default 120000 - so a wedged engine can never hang
a job. Exit codes are the CI contract:
| code | meaning |
|---|---|
| 0 | success - parsed JSON on stdout |
| 1 | model or task failure - invalid reply, timeout, or engine error |
| 2 | usage error |
| 69 | no backend available - treat as a clean skip in CI |
When nothing is provisioned the CLI prints the exact provisioning steps for
each backend and exits 69; odai backends prints per-backend availability
JSON with the reason for every unavailable engine.
odai batch reads a JSONL manifest (stdin or --input), runs every task over a single backend launch, and prints one JSON line per entry in manifest order - {"id","ok":true,"value":…} or {"id","ok":false,"error":…}. It exits 0 when the batch ran even if every task failed (failures are in-band lines), 2 on a malformed manifest (checked in full before any task runs), and 69 when no backend is available. --timeout is the per-task budget; --raw is not accepted.
odai lockstep analyzes one lockstep row with full or sparse materialization.
It validates cited evidence and proposed patches before returning a result.
The fleet runner verifies changes in a temporary copy with trusted commands.
See lockstep assistance for preparation, verification, and model evaluation.
Use the conversation API when requests need shared context. It supports transcript restore, streaming, cancellation, and bounded history across native and replay providers.
odai serve turns any backend into a loopback HTTP server speaking both wire
formats llama-server does, on one port: the Anthropic Messages API
(POST /v1/messages, /v1/messages/count_tokens) and the OpenAI
chat-completions API (POST /v1/chat/completions,
/v1/chat/completions/input_tokens, GET /v1/models), plus /health. odai
already talks to llama-server as a client, so serving the same routes lets
anything pointed at a local OpenAI base URL treat odai as the server it would
otherwise run. Tool calling works on both sides and is emulated for plain-text
engines: the shim teaches the model a one-line JSON tool-call protocol in the
system prompt and reads the reply back into tool_use blocks or tool_calls,
with the same JSON repair hardening the tasks use.
odai serve # 127.0.0.1:8402, backend from the registry probe
odai serve --port 0 # let the OS pick a free port
odai serve --backend llama-serverPointing a client at it - the two base-URL forms, and the one request field the shim ignores
Point any Anthropic-speaking client at the printed URL with
ANTHROPIC_BASE_URL and any non-empty ANTHROPIC_API_KEY (loopback only, no
auth). For example, communique runs its
release-notes agent loop against local inference with no code changes:
communique --provider anthropic --model local \
--base-url http://127.0.0.1:8402An OpenAI-speaking client points at the same port's /v1 prefix, the way it
would at llama-server:
OPENAI_BASE_URL=http://127.0.0.1:8402/v1 OPENAI_API_KEY=anythingKnown limitation: the shim ignores max_tokens in the request - odai's
session interface has no output-token budget - and never emits a max_tokens
stop reason, so a reply cut short by the engine itself (for example
llama-server's own n_predict default) arrives as end_turn.
Run the bench evaluation harness against the built-in simulator, or score a
real backend with --backend:
pnpm run bench
pnpm run bench --backend=chrome-builtin
pnpm run bench --scenario=lockstep --backend=chrome-builtin --jsonThe default simulator checks the evaluation harness. Use an explicit backend to measure a real model. The lockstep cases check response contracts. Upstream conformance requires the separate fleet verifier.
See the performance practices for package measurements and retained-context experiments. The performance journal records measured results, rejected changes, and model limitations. The automation practices define deterministic setup, validation, and model fallback boundaries.
Contributor commands
pnpm install
pnpm run check
pnpm run testMIT
