Skip to content

docs(cli): document removal of legacy config.json custom models - #76

Closed
lukealvoeiro wants to merge 1 commit into
mainfrom
cli-args-audit/byok-legacy-cfg
Closed

lukealvoeiro wants to merge 1 commit into
mainfrom
cli-args-audit/byok-legacy-cfg

Conversation

@lukealvoeiro

Copy link
Copy Markdown
Contributor

Gap

EXEC-N300 / ROOT-N302 (F, P2). The BYOK docs say that custom models in ~/.factory/config.json (custom_models, snake_case fields) are still supported and are merged with settings.json. The video walkthrough tells users to paste BYOK configuration into ~/.factory/config.json. The CLI no longer reads that file for custom models, so users who follow the docs end up with models that do not exist.

Evidence

  • docs/cli/byok/overview.mdx:46: "Legacy support: Custom models in ~/.factory/config.json using snake_case field names (custom_models, base_url, etc.) are still supported for backwards compatibility. Both files are loaded and merged, with settings.json taking priority." The same claim appears at :241 ("or config.json if using legacy format") and in the JP mirror at docs/jp/cli/byok/overview.mdx:46 and :179.
  • docs/cli/getting-started/video-walkthrough.mdx:250,263-291: the directory tree lists "config.json # Model configuration and BYOK", the section is titled "config.json - Bring Your Own Key", and a step says "Paste the configuration into ~/.factory/config.json". The example also uses a nonexistent schema (models, name, provider openrouter). The JP mirror has the same content.
  • Code: factory-mono 0fe2ea9458d (Factory-AI/factory-mono#25453, "feat(cli): remove legacy BYOK config parsing", merged 2026-10-15) deleted packages/runtime/settings/src/legacyCustomModelMigration.ts. A later commit in the same PR also removed the transitional startup warning ("refactor(cli): remove transitional BYOK warnings"). Custom models are now parsed only from customModels in settings (packages/runtime/settings/src/SettingsParsing.ts).

The audit used a hermetic home whose user config dir is $AUDIT_HOME/.factory-dev. config.json contained:
{"custom_models":[{"model":"legacy-mock","model_display_name":"Legacy Mock","base_url":"http://127.0.0.1:9/v1","api_key":"sk-test-placeholder","provider":"generic-chat-completion-api"}]}

$ droid exec -m custom:Legacy-Mock-0 --list-tools
Invalid model: custom:Legacy-Mock-0

Available built-in models:
  auto, claude-fable-5.1, ... (built-in list)

No custom models configured. Add them to ~/.factory/settings.json
EXIT=1

$ droid doctor --checks config --json
{ ... "results": [ { "id": "configuration.valid", "status": "pass", "detail": "No configuration issues found" }, ... ] }
EXIT=0

Why it matters

Users who follow the BYOK overview or the video walkthrough put models where the CLI never looks. The models do not show up in /model, droid exec -m rejects them, and droid doctor reports no problem. Nothing tells the user their config is being ignored.

Fix

This is a docs-only change. #25453 removed legacy support on purpose, so this PR does not restore it.

  • BYOK overview (EN/JP): the "Legacy support" note is replaced with a migration note. It says config.json custom_models is no longer read and maps each old field to its settings.json name: model_display_name→displayName, base_url→baseUrl, api_key→apiKey, max_tokens→maxOutputTokens, max_context_limit→maxContextLimit, enable_thinking→enableThinking, thinking_max_tokens→thinkingMaxTokens, reasoning_effort→reasoningEffort, extra_args→extraArgs, extra_headers→extraHeaders, and supports_images: false→noImageSupport: true. model, provider and bedrock are unchanged. The mapping comes from the deleted migration code and matches the current SettingsParsing.ts.
  • BYOK overview troubleshooting (EN/JP): the "(or config.json if using legacy format)" wording is replaced with a pointer to the migration note.
  • Video walkthrough (EN/JP): config.json is removed from the directory tree, the section now describes customModels in settings.json with a valid camelCase OpenRouter example (same baseUrl and provider as the OpenRouter BYOK page), and the step now says "Add to settings.json".
  • Stale snake_case field names are renamed: model_display_name→displayName in custom-droids (EN/JP), api_key→apiKey in ollama (EN/JP), and base_url→baseUrl in the enterprise gateways page (EN/JP).
  • Changelog entries are unchanged. The changelog does not list the release that shipped #25453, so no version is given.

Validation

  • The customModels form documented here is accepted. With settings.json = {"customModels":[{"model":"legacy-mock","displayName":"Legacy Mock","baseUrl":"http://127.0.0.1:9/v1","apiKey":"sk-test-placeholder","provider":"generic-chat-completion-api","maxOutputTokens":4096}]}:

    $ droid exec -m custom:Legacy-Mock-0 --list-tools
    Available tools for Legacy Mock
    Autonomy: read-only
    ...
    EXIT=0
    

    -m custom:legacy-mock (custom: plus the model field, as custom-droids describes) also resolves to "Available tools for Legacy Mock".

  • After the change, grep -rn -E "custom_models|model_display_name|base_url|config.json" docs --include=*.mdx (changelog excluded) finds those terms only in the new migration notes and the "no longer read" sentences.

  • MDX sanity: code-fence counts are even in every changed file, the <Note> and <Steps>/<Step> tags are balanced, and the tables inside <Note> have a blank line before them.

  • No docs lint or Mintlify render was run (the repo has no root lint script).

Found by the scheduled CLI arguments audit (2026-10-17).

The CLI no longer reads custom_models from ~/.factory/config.json.
Replace the legacy-support note with a migration note and field mapping
to customModels in settings.json, point the video walkthrough at
settings.json with camelCase fields, and rename stale snake_case field
references (EN and JP).

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
@lukealvoeiro lukealvoeiro added the cli-args-audit CLI arguments documentation and behavior audits label Oct 17, 2026
@factory-droid

factory-droid Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Droid finished @lukealvoeiro's task —— View job


LGTM. The documentation updates align the BYOK guidance with settings.json, and no actionable issues were found.

0 inline comments posted

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli-args-audit CLI arguments documentation and behavior audits

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant