How do I run Open WebUI on a cloud GPU?

Cover: Run Open WebUI on a cloud GPU, Ollama, uv and an SSH tunnel, step by step

Deploy a Raw GPU instance, install Ollama with its official script and start ollama serve, run Open WebUI with uvx --python 3.12 open-webui@latest serve --host 127.0.0.1, then open it on your own computer through an SSH tunnel at http://localhost:8080.

Updated 2026-10-30

This guide follows the official Open WebUI docs (Quick Start, Starting with Ollama, Environment configuration) and the official Ollama docs (Linux, GPU, FAQ), all read on 2026-10-30. On that day the latest releases were Open WebUI 0.12.5 (PyPI, uploaded 2026-10-29) and Ollama v0.40.13 (GitHub release, published 2026-10-29). It is for people who use Ollama and want a browser chat interface backed by a rented NVIDIA GPU such as the RTX 5090 or the other cards listed on /pricing.

What you need

Item Official value (read 2026-10-30) Source
Python for Open WebUI 3.11 or 3.12; 3.13 is not supported yet Open WebUI Quick Start
Open WebUI default address http://localhost:8080 (change with --host and --port) Open WebUI Quick Start
Ollama default address binds 127.0.0.1 port 11434 (change with OLLAMA_HOST) Ollama FAQ
Where Open WebUI looks for Ollama OLLAMA_BASE_URL, default http://localhost:11434 Open WebUI environment configuration
NVIDIA GPU for Ollama compute capability 5.0+ and driver 550 or newer (5.0 to 6.2 need 570 or newer); the RTX 5090 is listed under compute capability 12.0 Ollama GPU docs
Models none bundled: “Open WebUI has no models of its own” Open WebUI Quick Start

VRAM need depends on model, precision and workflow; see How much VRAM do I need to run an LLM? and How to choose a rental GPU for Ollama. The VRAM of each card is in the Performance table on /pricing (CloudGPU’s own listing).

Step 1: Deploy a Raw GPU and connect

On /deploy, pick an NVIDIA GPU and the Raw GPU option (JupyterLab + CUDA pre-installed, root SSH on a dedicated container). Connect over SSH with the host, port and user shown for your instance:

ssh -p <ssh-port> <ssh-user>@<ssh-host>

This guide uses uvx, not Docker.

Step 2: Check the GPU and driver

nvidia-smi        # GPU name, driver version, VRAM
nvcc --version    # CUDA toolkit version
python3 --version

Compare the driver version in the nvidia-smi header with the Ollama minimum in the table above (550 or newer for most cards). Note the Python version: Open WebUI needs 3.11 or 3.12, and Step 5 handles that with uv.

Step 3: Install and start Ollama

According to the official Ollama Linux docs, install it with:

curl -fsSL https://ollama.com/install.sh | sh

The docs show the manual start as ollama serve, then ollama -v in another terminal. Run the server in tmux so it keeps running when your SSH session closes:

tmux new -s ollama
ollama serve
# detach with Ctrl-b then d

In a new SSH session (or after detaching), check that it answers:

ollama -v

If ollama -v already answers before you run ollama serve, Ollama is already running as a service; skip ollama serve. By default Ollama binds 127.0.0.1:11434, so it is not reachable from outside the instance; keep it that way, Open WebUI talks to it locally.

Step 4: Pull a model

Pick a model tag from the Ollama library. For example, the deepseek-r1 page listed deepseek-r1:8b on 2026-10-30. Any other tag from the Ollama library works instead; use it in both commands below.

ollama pull deepseek-r1:8b
ollama run deepseek-r1:8b "Say hello in one sentence."

Whether a model fits on your card depends on the model, its quantization and the context length; check before you pull a large tag. You can also pull models later from inside Open WebUI (Step 7).

Step 5: Install uv and start Open WebUI

According to the official Open WebUI Quick Start, install uv and run Open WebUI with Python 3.12:

curl -LsSf https://astral.sh/uv/install.sh | sh
# open a new shell (or run the PATH command the uv installer prints) so uvx is found

tmux new -s webui
cd ~
DATA_DIR=~/.open-webui uvx --python 3.12 open-webui@latest serve --host 127.0.0.1 --port 8080
# detach with Ctrl-b then d
  • DATA_DIR is where Open WebUI keeps chats, users, uploads and settings (default ./data).
  • --host 127.0.0.1 keeps the UI off the public network; you reach it through the SSH tunnel in Step 6.
  • The docs say Open WebUI signs sessions with a key it writes to .webui_secret_key in the directory you start it from. Always start from the same directory (here ~), or set WEBUI_SECRET_KEY, otherwise everyone gets signed out.
  • To pin the version this guide was written against, use open-webui@0.12.5 instead of open-webui@latest.

If you prefer pip, the docs give pip install open-webui then open-webui serve inside a Python 3.11 or 3.12 environment; see How to set up a conda environment on a cloud GPU.

Step 6: Open the UI through an SSH tunnel

Run this on your own computer, in a new terminal:

ssh -N -L 8080:127.0.0.1:8080 -p <ssh-port> <ssh-user>@<ssh-host>

Leave that terminal open and go to http://localhost:8080 in your browser. If port 8080 is taken on your computer, use another local port, for example -L 3000:127.0.0.1:8080 and http://localhost:3000. More options are in How to use SSH port forwarding on a cloud GPU.

Step 7: Create the admin account and connect Ollama

According to the Open WebUI Quick Start, the first screen offers Create Admin Account, and that first account is the administrator; sign-up switches itself off once it exists. Note the password.

The docs say that with the Python install, Ollama on the same machine is picked up automatically at http://localhost:11434 and shows under avatar > Settings > Admin > Connections > Manage Ollama API Connections. If it does not, add that URL there, or start Open WebUI with OLLAMA_BASE_URL=http://127.0.0.1:11434 in front of the command. As the admin you can also download a model by typing its name into the model selector of a new chat and confirming the pull.

Select the model you pulled in Step 4 and send a message.

Optional: add the CloudGPU API as an OpenAI API connection

Besides Ollama, Open WebUI can talk to any OpenAI-compatible API. CloudGPU has one at https://cloudgpu.app/v1. We have not tested this connection with Open WebUI. The steps follow the Open WebUI docs (OpenAI-Compatible, read 2026-10-30):

  1. Go to avatar > Settings > Admin > Connections and find Manage OpenAI API Connections.
  2. Click Add Connection.
  3. Set URL to https://cloudgpu.app/v1 and API Key to your CloudGPU API key, the same value you keep in CLOUDGPU_API_KEY.
  4. Click Verify Connection, then Save. The docs say that if the check fails, you can still type model IDs into the Model IDs list by hand.

The model IDs you can use are listed on the CloudGPU API page. Pick one there and select it in a new chat.

Check your own setup

These are checks you can run yourself; this page does not assume the result.

  • Driver: the nvidia-smi header shows a driver version that meets the Ollama minimum.
  • Ollama is up: ollama -v prints a version number.
  • The model is on the GPU: while a chat is answering, ollama ps lists the model; the Ollama FAQ says the PROCESSOR column shows which memory it was loaded into. nvidia-smi in a second session also shows the ollama process.
  • Open WebUI sees Ollama: the model selector in a new chat lists your pulled model.

If something does not work

  • uvx: command not found: open a new shell, or run the PATH command the uv installer printed.
  • No models in the selector: check Step 3 (ollama -v) and the connection URL in Step 7.
  • The model runs on the CPU or answers slowly: check ollama ps. If the model does not fit in VRAM, pick a smaller tag or deploy a card with more VRAM from /pricing. See How to check GPU utilization on a cloud GPU.
  • The page does not load in your browser: the tunnel terminal must stay open, and Open WebUI must be running in its tmux session (tmux attach -t webui to see its log).

Before you destroy the instance

According to /terms, data on the instance is permanently deleted when you destroy it. That includes your chats and settings in ~/.open-webui and the models Ollama downloaded. Copy what you want to keep to your computer first:

rsync -avP -e "ssh -p <ssh-port>" <ssh-user>@<ssh-host>:.open-webui/ ./open-webui-data/

GPU time is billed per minute, rounded up, with a 1-minute minimum, and billing stops when you destroy the instance; see /terms.

FAQ

Can I run Open WebUI on a rented GPU without Docker?

Yes. According to the official Open WebUI docs, you can install it with Python 3.11 or 3.12 instead of Docker, for example DATA_DIR=~/.open-webui uvx --python 3.12 open-webui@latest serve. Python 3.13 is not supported yet. Install Ollama with its official script on the same instance and Open WebUI finds it at http://localhost:11434.

How do I open Open WebUI from my own browser?

Start Open WebUI on the instance with --host 127.0.0.1 --port 8080, then run ssh -N -L 8080:127.0.0.1:8080 -p @ on your own computer and open http://localhost:8080. The UI stays off the public network and the traffic goes through your SSH login.

Why does Open WebUI show no models?

Open WebUI has no models of its own. Check that ollama serve is running and that ollama -v answers, then pull a model with ollama pull or type its name into the model selector as the admin. The Ollama connection is under avatar, Settings, Admin, Connections, and the default URL is http://localhost:11434.

How do I check that Ollama is using the GPU?

Run ollama ps while a chat is answering. The Ollama FAQ says the PROCESSOR column shows which memory the model was loaded into. You can also run nvidia-smi in a second SSH session and look for the ollama process. If the model does not fit in VRAM, use a smaller model or a card with more VRAM.

What happens to my chats when I destroy the instance?

Data on the instance is permanently deleted when you destroy it, including the Open WebUI data folder and downloaded models, so copy ~/.open-webui to your computer first. GPU time is billed per minute, rounded up, with a 1-minute minimum, and billing stops when you destroy the instance. See https://cloudgpu.app/terms for the full terms.

Can I connect Open WebUI to the CloudGPU API?

Possibly, but we have not tested this connection. According to the Open WebUI docs, any OpenAI-compatible API can be added under avatar, Settings, Admin, Connections, Manage OpenAI API Connections. Use the URL https://cloudgpu.app/v1 and your CloudGPU API key, the value you keep in CLOUDGPU_API_KEY, then click Verify Connection and Save. The model IDs are listed on the CloudGPU API page.

Sources