Repository navigation
feat(config): let model aliases carry an opt-in profile block (system prompt, MCP, client tools) - #12661
feat(config): let model aliases carry an opt-in profile block (system prompt, MCP, client tools)#12661walcz-de wants to merge 3 commits into
Conversation
An alias can now act as a profile of its target through an explicit `profile:` block carrying system_prompt, mcp (remote/stdio servers, plus the new mcp.auto) and the new client_tools. ResolveAlias applies only that block on top of a copy of the target, while Name, Model, Backend and the persisted revision stay the target's. The loader key is therefore unchanged and every profile runs on the target's backend process; today a persona needs a separate model config, which loads the weights a second time. The block is opt-in so existing aliases never change: top-level fields on an alias stay ignored, and an alias without a profile block resolves to exactly the target. A profile block on a non-alias config is rejected. A profile with its own MCP servers caches its sessions under the alias name (MCPSessionKey), because the cache slot keyed on the target's name is invalidated whenever its config hash changes and two server sets would tear each other's sessions down. mcp.auto enables all configured servers on /v1/chat/completions when the request does not name any in metadata.mcp_servers, which plain OpenAI clients cannot send. client_tools: drop discards client tools and tool_choice before the request is merged (chat completions, responses, Anthropic messages), so only the model's own MCP tools are offered. Both keys work on any model and inside an alias's profile block. Refs mudler#12548 Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
localai-org-maint-bot
left a comment
There was a problem hiding this comment.
Changes requested on 256997dce84cc44d26c703280c200a3dca655407.
[P1] Gate automatic MCP activation before discovering or executing tools. core/http/endpoints/openai/chat.go:333-345 turns a plain chat request into a server-side MCP request whenever mcp.auto is set, without checking startupOptions.DisableMCP or the caller's mcp permission. /v1/chat/completions is gated by the chat feature; its middleware does not add the MCP gate. The dedicated MCP routes are separately gated and are not registered when MCP is disabled.
Concrete cases: configure a local model with mcp.auto: true and a server, then (1) set LOCALAI_DISABLE_MCP=true, or (2) authenticate a user with chat: true, mcp: false. A normal chat POST without MCP metadata still selects the configured servers and enters discovery/tool injection. Previously that plain request did not opt into model MCP tools. Enforce the instance and user controls before the newly automatic path, and cover both denied cases with endpoint tests that assert no server connection/tool execution occurs.
I traced this through the route registry, middleware, and handler; I did not run a live MCP backend. This needs an access-control fix and regression coverage before sign-off.
…tted mcp.auto turns a plain /v1/chat/completions request into a server-side MCP request. It now applies the same controls as the dedicated MCP routes before any server is selected: the instance switch (LOCALAI_DISABLE_MCP) and, with auth on, the caller's "mcp" feature, checked with auth.HasFeatureAccess against the auth database that the chat route now passes to ChatEndpoint. A denied request is answered as a plain chat, without contacting a server or adding tools, rather than rejected: the client did not ask for MCP, it only reached a model that offers it. Endpoint specs drive ChatEndpoint against a counting MCP fixture server: MCP disabled and a user with chat but without mcp see no tools and no connection; MCP enabled and a user with mcp get the fixture tool. The user specs need the auth build tag (SQLite auth database). Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
|
Thanks, good catch. In 4418e03 the |
localai-org-maint-bot
left a comment
There was a problem hiding this comment.
@mudler Code-review sign-off at 4418e0321b428bd376736dcbbd110fc828062987, subject to CI validation. The new commit resolves my previous blocking finding: the automatic MCP path checks the instance disable flag and the authenticated caller's MCP permission before selecting servers, and the chat route supplies the auth database. The endpoint specs cover denied requests making zero MCP HTTP requests and permitted requests receiving the fixture tool; the docs describe the plain-chat fallback. No remaining blocking source findings.
Reviewed against latest master c44c32f15f6ca4861888ffc740f5ed1e6d0efe6b. git diff --check and Go formatting checks pass. I attempted go test -tags auth ./core/http/endpoints/openai -run Test -ginkgo.focus="ChatEndpoint mcp.auto" -count=1 -timeout=45s, but setup failed because proxy.golang.org returned HTTP 403 for the localrecall dependency. The tests did not execute; only DCO is reported in the current check snapshot, so this approval does not establish build/test readiness.
Bring the opt-in model-alias profile block into the fork build (agntsio mudler#973, operator decision F1: only mudler#12661). PR commits kept unchanged so the upstream SHA stays identifiable. Assisted-by: Claude Code:claude-opus-5-5
GET /v1/mcp/servers/:model, /v1/mcp/prompts/:model and /v1/mcp/resources/:model looked the model up with GetModelConfig, which returns the alias stub. A profile alias carries its servers in its profile block, so the stub has none of its own and the endpoints reported an empty list for an alias whose chat requests do use tools. Resolve the alias the way the request middleware does before reading the MCP config, and open the sessions under MCPSessionKey so discovery shares the profile's session cache slot instead of tearing down the target's. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
What
A model alias can now act as a profile of its target: a different assistant
persona on the same loaded model. An alias config may carry an optional
profile:block:profile.system_prompt– replaces the target's system promptprofile.mcp(remote/stdio) – replaces the target's MCP serversprofile.mcp.auto– enable all MCP servers on/v1/chat/completionswhen therequest does not name any via
metadata.mcp_serversprofile.client_tools: drop– discard client-senttools/functions/tool_choice,so only the model's own MCP tools are offered
mcp.autoandclient_toolsare also new keys for regular model configs.Why
Today the only way to offer several assistants (different instructions, different
tool sets) is one model config per assistant, each pointing at the same weights.
Each of them loads the model again (for a 26B model ~15 GB plus KV cache per
persona). Aliases already share the target's backend process, but ignore every
field except
alias. Related: #12548 asks for the same "alias as logicalprofile on one loaded instance" idea for request parameters; the
profileblockis a natural home for that in a follow-up.
Opt-in, no change for existing aliases
Only fields inside
profile:are applied. Top-level fields on an alias(
system_prompt,mcp, sampling parameters, ...) stay ignored exactly asbefore, and an alias without a
profileblock resolves to exactly its target(both asserted with deep-equality specs). A
profileblock on a config that isnot an alias is rejected at validation.
How
ResolveAliasapplies the profile block to the copy of the target it alreadyreturns.
Name,Model,Backendand the persisted config revision stay thetarget's, so the model loader key (
ModelID()) and the controller revision areunchanged: every profile runs on the target's process.
config hash changes. A profile with its own servers would thrash the target's
slot, so the resolved config carries an unexported session key
(
MCPSessionKey(), the alias name) used by chat, responses and Anthropicmessages.
client_tools: dropis applied before the request is merged into the config, sono grammar or forced
tool_choiceis derived from the dropped tools.client_toolsvalue, MCP YAML).mcp.auto,client_toolsandprofile.*.Tests
Ginkgo specs in
core/config(profile applied, loader identity and revision equalto the target, stored target not mutated, alias without profile equals the
target, top-level fields on an alias still ignored, validation),
core/http/middleware(full chat middleware chain: profile drops tools and forcedtool_choice and serves the target with the profile prompt; alias with top-level
fields but no profile unchanged; responses merge) and
core/http/endpoints/openai(
mcp.autoserver selection). Rango testforcore/config,core/config/meta,core/http/middleware,core/http/endpoints/{openai,openresponses,anthropic,localai,mcp}and
core/services/modeladmin.Docs
docs/content/features/model-aliases.md(new "Profile aliases" section) anddocs/content/features/mcp.md(auto,client_tools).Not in this PR (possible follow-ups)
mcp_tool_resultevents to OpenAI clients.profile.parametersfor request defaults (Allow aliases to inject request parameters #12548).🤖 Generated with Claude Code