New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Confused About Docker (container talking to a service running on host's localhost)
raindog308
Administrator, Veteran
in Help
I have a Debian 12 VPS with Ollama running on 127.0.0.1:11434.
Ollama itself is working just fine:
# curl http://127.0.0.1:11434/api/tags {"models":[{"name":"tinyllama:latest","model":"tinyllama:latest","modified_at":"2025-08-04T06:07:47.745085927-07:00","size":637700138,"digest":"2644915ede352ea7bdfaff0bfac0be74c719d5d5202acb63a6fb095b52f394a4","details":{"parent_model":"","format":"gguf","family":"llama","families":["llama"],"parameter_size":"1B","quantization_level":"Q4_0"}}]}
I fired up open-webui in docker and created an Nginx reverse proxy and all that is working fine (i.e., the openui web page works), but not seeing any models available. This is because in the container's logs I see:
(line breaks for LET)
2025-08-04 20:13:05.636 | ERROR \
| open_webui.routers.ollama:send_get_request:106 - \
Connection error: Cannot connect to host host.docker.internal:11434 \
ssl:default [Connect call failed ('172.17.0.1', 11434)] - {}
And:
# docker exec -it open-webui curl http://host.docker.internal:11434/api/tags curl: (7) Failed to connect to host.docker.internal port 11434 after 0 ms: \ Couldn't connect to server
This makes me sad.
My docker-compose.yaml:
services:
open-webui:
image: ghcr.io/open-webui/open-webui:${WEBUI_DOCKER_TAG-main}
container_name: open-webui
volumes:
- open-webui:/app/backend/data
ports:
- ${OPEN_WEBUI_PORT-3000}:8080
environment:
- OLLAMA_BASE_URL=http://host.docker.internal:11434
- DEBUG=true
extra_hosts:
- host.docker.internal:host-gateway
restart: unless-stopped
volumes:
open-webui: {}
I thought host.docker.internal and the extra_hosts was magic that bound 127.0.0.1 to host.docker.internal?
Comments
Did you try turning it on and off?
Ok
I use --net="host"
And it usually works for me
My googling suggests that is what extra_hosts is doing - ?
try this
This works for my caddy so it talks to an app on localhost
Also make sure your firewall is allowing the connection to the localhost
As you mentioned, ollama is running on
127.0.0.1:11434. It works when you access it on localhost.With the command
docker exec -it open-webui curl http://host.docker.internal:11434/api/tagsyou are try to access ollama on172.17.0.1:11434(not localhost).Docker container does not have access to your host's localhost. Instead, it shares a
docker0interface with the host and you can access host's services listening on it.I would try making ollama listen on
172.17.0.1or0.0.0.0. Then docker container should be able to access it as you tried beforeThis fixed it. Thank you @oloke !
Would it have worked with network_mode: host? I usually do that works great
Doesn't matters anymore, if it works don't touch it
100% fail.
Turn it off then on.
Ollama is quite unprotected on host/loopback listening. I personally run Ollama with a static IP in a private /29 bridge network with
internal: trueto shut it off from other networks/public internet. Then run OpenWebUI in its own bridge that'sinternal: falsefor internet connectivity and then simply attach it to the private internal bridge network where Ollama resides and have it directly connect to its static bridge IP. This ensures only OpenWebUI gets to access it with its somewhat authentication in front of it.In case you're wondering how the hell I get my models loaded into Ollama from outside: I don't use the WebUI for that because it's rather slow. I simply aria/wget the .gguf files directly from Hugging Face onto a mounted directory that the Ollama container has access to and then simpy run
docker exec -it ollama /opt/ollama/ollama create --file /models/llama3/prepared_Modelfile Llama3-model-lowendyappingwhere it's most important to have theFROMpart right in the Modelfile:FROM /models/llama3/llama3-model.ggufThe file wgetting and importing is drastically faster this way.