Repository navigation
Open WebUI as LLM provider #427
Description
Hi!
I have been trying to use Open WebUI as an LLM provider with no luck. I have set the following parameters in the config.yaml file and exported the API key of course:
llm:
api_base: "http://localhost:3000/api/"
model: "glm-ocr"
When attempting to run one of the examples (function_minimization) instead of getting the expected result I receive heaps of failed attempts at parsing the LLMs output:
2026-03-03 03:48:11,747 - ERROR - LLM generation failed: list index out of range
2026-03-03 03:48:11,747 - ERROR - LLM generation failed: list index out of range
2026-03-03 03:48:11,756 - WARNING - Iteration 39 error: LLM generation failed: list index out of range
2026-03-03 03:48:11,757 - WARNING - Iteration 40 error: LLM generation failed: list index out of range
The same endpoint, key and model results in the expected behaviour when making API calls via the openai python interface.
I was unable to identify the problem beyond what is seen in the logs.
Any idea what the problem is? Is Open WebUI simply not supported?
Activity
codelion commentedon Mar 3, 2026
Can you try with base url
http://localhost:3000/api/v1bsulyok commentedon Mar 3, 2026
I have tried that already, same results!
bsulyok commentedon Mar 3, 2026
I identified the issue after some debugging. It seemed to be an important distinction to specify the model as "primary_model" as such:
Without this the model is not recognized and an empty list is loaded as the model ensemble. I must have missed this detail in the documentation!
akushonkamen commentedon Oct 8, 2026
Working on this. Plan: (1) raise an explicit
ValueErrorinConfig.from_dictwhen the invalidllm.modelkey is present, pointing atllm.primary_model/ thellm.modelsarray instead of letting dacite silently drop it and blow up later withIndexErrorinLLMEnsemble._sample_model; (2) fix the three README provider examples (Gemini, Ollama/vLLM, OptiLLM) that currently teach the brokenmodel:spelling. PR with tests shortly.akushonkamen commentedon Oct 8, 2026
PR up: #503 — explicit
ValueErrorfor thellm.modelkey inConfig.from_dict, plus all three README provider examples corrected toprimary_model:. Full suite green.Reject the invalid 'llm.model' config key with an actionable error (#503