Skip to content

feat(config): let model aliases carry an opt-in profile block (system prompt, MCP, client tools) - #12661

Open
walcz-de wants to merge 3 commits into
mudler:masterfrom
walcz-de:feat/profile-alias
Open

walcz-de wants to merge 3 commits into
mudler:masterfrom
walcz-de:feat/profile-alias

Conversation

@walcz-de

Copy link
Copy Markdown
Contributor

What

A model alias can now act as a profile of its target: a different assistant
persona on the same loaded model. An alias config may carry an optional
profile: block:

  • profile.system_prompt – replaces the target's system prompt
  • profile.mcp (remote/stdio) – replaces the target's MCP servers
  • profile.mcp.auto – enable all MCP servers on /v1/chat/completions when the
    request does not name any via metadata.mcp_servers
  • profile.client_tools: drop – discard client-sent tools/functions/tool_choice,
    so only the model's own MCP tools are offered

mcp.auto and client_tools are also new keys for regular model configs.

name: researcher
alias: my-llama-3
profile:
  system_prompt: You are a careful research assistant. Cite your sources.
  client_tools: drop
  mcp:
    auto: true
    remote: |
      {"mcpServers": {"search": {"url": "http://search.internal/mcp"}}}

Why

Today the only way to offer several assistants (different instructions, different
tool sets) is one model config per assistant, each pointing at the same weights.
Each of them loads the model again (for a 26B model ~15 GB plus KV cache per
persona). Aliases already share the target's backend process, but ignore every
field except alias. Related: #12548 asks for the same "alias as logical
profile on one loaded instance" idea for request parameters; the profile block
is a natural home for that in a follow-up.

Opt-in, no change for existing aliases

Only fields inside profile: are applied. Top-level fields on an alias
(system_prompt, mcp, sampling parameters, ...) stay ignored exactly as
before, and an alias without a profile block resolves to exactly its target
(both asserted with deep-equality specs). A profile block on a config that is
not an alias is rejected at validation.

How

  • ResolveAlias applies the profile block to the copy of the target it already
    returns. Name, Model, Backend and the persisted config revision stay the
    target's, so the model loader key (ModelID()) and the controller revision are
    unchanged: every profile runs on the target's process.
  • MCP sessions are cached by model name, and the cache slot is invalidated when the
    config hash changes. A profile with its own servers would thrash the target's
    slot, so the resolved config carries an unexported session key
    (MCPSessionKey(), the alias name) used by chat, responses and Anthropic
    messages.
  • client_tools: drop is applied before the request is merged into the config, so
    no grammar or forced tool_choice is derived from the dropped tools.
  • Alias configs now validate their profile (client_tools value, MCP YAML).
  • UI field metadata for mcp.auto, client_tools and profile.*.

Tests

Ginkgo specs in core/config (profile applied, loader identity and revision equal
to the target, stored target not mutated, alias without profile equals the
target, top-level fields on an alias still ignored, validation),
core/http/middleware (full chat middleware chain: profile drops tools and forced
tool_choice and serves the target with the profile prompt; alias with top-level
fields but no profile unchanged; responses merge) and core/http/endpoints/openai
(mcp.auto server selection). Ran go test for core/config, core/config/meta,
core/http/middleware, core/http/endpoints/{openai,openresponses,anthropic,localai,mcp}
and core/services/modeladmin.

Docs

docs/content/features/model-aliases.md (new "Profile aliases" section) and
docs/content/features/mcp.md (auto, client_tools).

Not in this PR (possible follow-ups)

  • A streaming mode that does not forward server-side (MCP) tool-call deltas and
    mcp_tool_result events to OpenAI clients.
  • profile.parameters for request defaults (Allow aliases to inject request parameters #12548).
  • MCP listing endpoints resolving plain aliases to the target's servers.

🤖 Generated with Claude Code

An alias can now act as a profile of its target through an explicit
`profile:` block carrying system_prompt, mcp (remote/stdio servers, plus
the new mcp.auto) and the new client_tools. ResolveAlias applies only
that block on top of a copy of the target, while Name, Model, Backend
and the persisted revision stay the target's. The loader key is
therefore unchanged and every profile runs on the target's backend
process; today a persona needs a separate model config, which loads the
weights a second time.

The block is opt-in so existing aliases never change: top-level fields
on an alias stay ignored, and an alias without a profile block resolves
to exactly the target. A profile block on a non-alias config is
rejected.

A profile with its own MCP servers caches its sessions under the alias
name (MCPSessionKey), because the cache slot keyed on the target's name
is invalidated whenever its config hash changes and two server sets
would tear each other's sessions down.

mcp.auto enables all configured servers on /v1/chat/completions when the
request does not name any in metadata.mcp_servers, which plain OpenAI
clients cannot send. client_tools: drop discards client tools and
tool_choice before the request is merged (chat completions, responses,
Anthropic messages), so only the model's own MCP tools are offered.
Both keys work on any model and inside an alias's profile block.

Refs mudler#12548

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>

@localai-org-maint-bot localai-org-maint-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested on 256997dce84cc44d26c703280c200a3dca655407.

[P1] Gate automatic MCP activation before discovering or executing tools. core/http/endpoints/openai/chat.go:333-345 turns a plain chat request into a server-side MCP request whenever mcp.auto is set, without checking startupOptions.DisableMCP or the caller's mcp permission. /v1/chat/completions is gated by the chat feature; its middleware does not add the MCP gate. The dedicated MCP routes are separately gated and are not registered when MCP is disabled.

Concrete cases: configure a local model with mcp.auto: true and a server, then (1) set LOCALAI_DISABLE_MCP=true, or (2) authenticate a user with chat: true, mcp: false. A normal chat POST without MCP metadata still selects the configured servers and enters discovery/tool injection. Previously that plain request did not opt into model MCP tools. Enforce the instance and user controls before the newly automatic path, and cover both denied cases with endpoint tests that assert no server connection/tool execution occurs.

I traced this through the route registry, middleware, and handler; I did not run a live MCP backend. This needs an access-control fix and regression coverage before sign-off.

…tted

mcp.auto turns a plain /v1/chat/completions request into a server-side
MCP request. It now applies the same controls as the dedicated MCP
routes before any server is selected: the instance switch
(LOCALAI_DISABLE_MCP) and, with auth on, the caller's "mcp" feature,
checked with auth.HasFeatureAccess against the auth database that the
chat route now passes to ChatEndpoint.

A denied request is answered as a plain chat, without contacting a
server or adding tools, rather than rejected: the client did not ask
for MCP, it only reached a model that offers it.

Endpoint specs drive ChatEndpoint against a counting MCP fixture server:
MCP disabled and a user with chat but without mcp see no tools and no
connection; MCP enabled and a user with mcp get the fixture tool. The
user specs need the auth build tag (SQLite auth database).

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
@walcz-de

Copy link
Copy Markdown
Contributor Author

Thanks, good catch. In 4418e03 the mcp.auto path now checks DisableMCP and, with auth on, the caller's mcp feature (auth.HasFeatureAccess, the auth DB is passed to ChatEndpoint from the chat route) before any server is selected. A denied request is answered as a plain chat, with no discovery, connection or tool injection, since the client did not ask for MCP; the docs (features/mcp.md, features/model-aliases.md) say so. New endpoint specs drive ChatEndpoint against a counting MCP fixture server: LOCALAI_DISABLE_MCP and a user with chat but without mcp get no tools and zero requests to the server, while MCP enabled and a user with mcp get the fixture tool (the user specs are under the auth build tag).

@localai-org-maint-bot localai-org-maint-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mudler Code-review sign-off at 4418e0321b428bd376736dcbbd110fc828062987, subject to CI validation. The new commit resolves my previous blocking finding: the automatic MCP path checks the instance disable flag and the authenticated caller's MCP permission before selecting servers, and the chat route supplies the auth database. The endpoint specs cover denied requests making zero MCP HTTP requests and permitted requests receiving the fixture tool; the docs describe the plain-chat fallback. No remaining blocking source findings.

Reviewed against latest master c44c32f15f6ca4861888ffc740f5ed1e6d0efe6b. git diff --check and Go formatting checks pass. I attempted go test -tags auth ./core/http/endpoints/openai -run Test -ginkgo.focus="ChatEndpoint mcp.auto" -count=1 -timeout=45s, but setup failed because proxy.golang.org returned HTTP 403 for the localrecall dependency. The tests did not execute; only DCO is reported in the current check snapshot, so this approval does not establish build/test readiness.

walcz-de added a commit to walcz-de/LocalAI that referenced this pull request Oct 14, 2026
Bring the opt-in model-alias profile block into the fork build
(agntsio mudler#973, operator decision F1: only mudler#12661). PR commits kept
unchanged so the upstream SHA stays identifiable.

Assisted-by: Claude Code:claude-opus-5-5
GET /v1/mcp/servers/:model, /v1/mcp/prompts/:model and
/v1/mcp/resources/:model looked the model up with GetModelConfig, which
returns the alias stub. A profile alias carries its servers in its
profile block, so the stub has none of its own and the endpoints
reported an empty list for an alias whose chat requests do use tools.

Resolve the alias the way the request middleware does before reading the
MCP config, and open the sessions under MCPSessionKey so discovery shares
the profile's session cache slot instead of tearing down the target's.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants