How to run LobeChat agents with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install the app with the Docker Compose commands the official README prints, serve a model on that machine, and set OLLAMA_PROXY_URL to an address those docs print. Copy the lobehub directory, and the database dump the Docker Compose page prints, before you destroy the instance, because destroying the instance deletes its disk.
An agent can rent the GPU and run this application on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the LobeChat install. A chat app whose Docker page runs docker compose up -d and whose Ollama page sets baseURL to http://localhost:11434/v1/ is How to run LibreChat agents with a self-hosted model on a cloud GPU. An agent whose Ollama example sets base_url to http://localhost:11434/v1 is How to run OpenManus with a self-hosted model on a cloud GPU. A terminal agent whose install page runs curl -fsSL https://www.openinterpreter.com/install | sh, and whose models page sets the Ollama base URL to http://localhost:11434/v1, is How to run Open Interpreter with a self-hosted model on a cloud GPU. A workspace app whose Docker quickstart publishes port 3001, and whose Ollama page sets the URL to http://127.0.0.1:11434, is How to run AnythingLLM agents with a self-hosted model on a cloud GPU. A repository agent whose local-models section sets api_base to http://localhost:11434 is How to run SWE-agent with a self-hosted model on a cloud GPU. A workflow editor whose Docker install publishes port 5678, and whose Ollama credentials default the Base URL to http://localhost:11434, is How to run n8n AI agents with a self-hosted model on a cloud GPU. A CLI whose providers page sets OLLAMA_HOST to http://localhost:11434 is How to run the goose agent with a self-hosted model on a cloud GPU. A coding agent whose providers page sets package to the OpenAI-compatible provider is How to use OpenCode with a self-hosted model on a cloud GPU. A canvas whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU. A CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A browser chat whose serve command listens on port 8080 is How to run Open WebUI on a cloud GPU. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose OllamaChatGenerator sets url to http://localhost:11434 is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b is How to run Agno agents with a self-hosted model on a cloud GPU. A TypeScript agent whose local-models section sets url on the model object is How to run Mastra agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.
The README says this project is LobeHub Community License licensed.
The commands below are copied from the upstream files named in each step, at the commit in Sources. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. Requesting lobehub/lobe-chat on 2026-11-14 returned lobehub/lobehub. The README title is LobeHub. The H1 on this page is the name the issue uses. A ready-timing line in the Docker success log is not copied. A provider list and a product list on the README are not copied. The resource section prints storage rows this page does not copy.
What the docs print about memory
The Docker Compose page, at the commit in Sources, prints a minimum of 2 CPU cores and 4 GB of RAM, and a recommended 4+ CPU cores and 8+ GB of RAM. Those figures are system memory. They are not vram_gb. The same section prints storage rows. This page does not copy them. The Ollama provider page and the Ollama integration page do not print a GPU GB figure. The integration page’s environment table says OLLAMA_KEEP_ALIVE is the duration that the model stays loaded in GPU memory, and it prints the default "5m". That cell is a duration the model stays loaded. It is not a start time, and it is not vram_gb. The model information table on unsloth/Llama-3.1-8B-Instruct prints parameter counts 8B, 70B, and 405B, and a context length of 128k. This page does not treat those parameter counts as vram_gb. A later section of that card names training hardware. This page does not copy that section and does not use it as an inference memory figure. This page does not mark a catalog row as enough.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T12:02:28.530Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Rows are sorted by vram_gb, then by name. Which listing you can start is on Deploy.
The catalog template list had no LobeHub or LobeChat template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as this app.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| Ascend 910B | 64 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
Main of lobehub/lobehub on 2026-11-14 was commit a8424d204a7ed1eb36985ebdfa6eac8041615392, committed 2026-11-14T11:11:12Z, on branch canary. pushed_at was 2026-11-14T11:51:58Z. The GitHub license field was NOASSERTION. The README says this project is LobeHub Community License licensed. The LICENSE file says that from 1.0, LobeChat is licensed under the LobeHub Community License, based on Apache License 2.0, with additional conditions. A badge in the README link group says apache 2.0. That badge and the LICENSE file are not the same sentence.
The README and docs/self-hosting/platform/docker-compose.mdx print:
mkdir lobehub && cd lobehub
bash <(curl -fsSL https://lobe.li/setup.sh) -l en
docker compose up -d
The README says to open http://localhost:3210. The Docker Compose page says ports 3210, 9000, 9001, 8787, and 8788 need to be available, and that the directory lobehub stores the configuration files and the later database files. The local-mode report example prints a storage username and a password. This page does not copy the password. The success log on that page prints http://localhost:3210. A ready-timing line in that log is not copied.
The Ollama integration page prints a different command. It assumes Ollama is already listening on port 11434:
docker run -d -p 3210:3210 -e OLLAMA_PROXY_URL=http://host.docker.internal:11434 lobehub/lobehub
This page does not merge that docker run with docker compose up -d. The single-container database page prints another docker run and a success log. This page does not merge that command either. The ready-timing line in that log is not copied.
The files this page read do not print an SSH command. The UI port those files print is 3210. The tunnel for a port you choose to publish is How to use SSH port forwarding on a cloud GPU.
Ollama and an OpenAI-compatible base URL
docs/self-hosting/environment-variables/model-provider.mdx says OLLAMA_PROXY_URL is required for the Ollama service. The example it prints is http://127.0.0.1:11434. ENABLED_OLLAMA defaults to 1. OLLAMA_MODEL_LIST is optional, and the example it prints is llama2:7B. .env.example comments the same URL: OLLAMA_PROXY_URL=http://127.0.0.1:11434.
The integration page’s Docker command sets OLLAMA_PROXY_URL to http://host.docker.internal:11434. That value and http://127.0.0.1:11434 are not the same. This page does not merge them. The provider page says to open App Settings, then AI Providers, then Ollama, enter the server URL, enable the provider, and select Check. It says to enable Client Request Mode if Ollama runs on the current device. The Linux section of the integration page sets:
[Service]
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_ORIGINS=*"
The table on that page says the OLLAMA_HOST default is "127.0.0.1:11434". The provider page’s Linux install line is curl -fsSL https://ollama.com/install.sh | sh. How to run Ollama on a cloud GPU is the Ollama install on the rented machine.
The same environment page says OPENAI_PROXY_URL overrides the default base URL for that provider. The default host printed there is not copied. The page says some proxies add a /v1 suffix and some do not, and that an empty reply during a check is a reason to try adding /v1. OPENAI_API_KEY is the key field. The provider page says to open App Settings, then AI Providers, enable the provider, enter the API key, and if you use a proxy or compatible endpoint, enter its proxy URL. The product name and the signup steps on that page are not copied.
The LobeHub files this page read do not print hf download. The block below is added on this page. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. This page did not download the repository. The LobeHub files this page read do not print this repository id. llama2:7B is the OLLAMA_MODEL_LIST example. This page does not treat it as this repository id.
pip install -U "huggingface_hub"
hf download unsloth/Llama-3.1-8B-Instruct
On 2026-11-14, GET https://huggingface.co/api/models/unsloth/Llama-3.1-8B-Instruct reported gated false, cardData.license llama3.1, pipeline_tag text-generation, lastModified 2025-05-09T19:32:57.000Z, and sha 4699cc75b550f9c6f3173fb80f4703b62d946aa5. The card YAML also includes extra_gated_prompt. Those two fields are not the same fact. The card’s base_model field names a repository this page does not copy. This page did not run hf download.
How to serve an OpenAI-compatible API with vLLM on a cloud GPU is the general server install. Point OPENAI_PROXY_URL at that server’s base URL. This page does not add a /v1 suffix the environment page does not print on a specific line.
Agents
The agent page says a system role and a model are enough to start. It says to click Create Agent in the left sidebar, or use Create Agent below the chat input. It says you can set the system prompt and the model yourself on the Agent Profile page. The table says modalities and tool support depend on the model and the provider. A list of external agents on that page names products this page does not copy.
When it fails
CUDA out of memory. The LobeHub files this page read print no GPU GB figure. If the model server raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The LobeHub README and the docs pages this page read do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this program on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
Port already in use. The Docker Compose page says a common cause is a port already in use, and to change LOBE_PORT in .env.
Empty reply after a proxy URL. The environment page says that if a check returns an empty message, try adding the /v1 suffix. It also says 127.0.0.1 inside Docker is the container, on the Studio proxy field. The Ollama integration command uses host.docker.internal for that reason. This page does not choose which of the two Ollama URLs to put in .env.
What to copy off, then destroy
The Docker Compose page says the lobehub directory stores the configuration files and the later database files. It also prints:
docker compose exec postgresql pg_dump -U postgres lobechat > backup.sql
And:
docker compose exec rustfs tar czf /tmp/backup.tar.gz /data
docker cp lobe-rustfs:/tmp/backup.tar.gz ./rustfs-backup.tar.gz
Copy backup.sql, rustfs-backup.tar.gz, and the lobehub directory, including .env, before you destroy the instance.
Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The billing page states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Amounts are not copied here.
Frequently asked questions
What memory figure do the LobeHub docs print?
The Docker Compose resource section prints a minimum of 2 CPU cores and 4 GB of RAM, and a recommended 4+ CPU cores and 8+ GB of RAM. Those figures are system memory. They are not vram_gb. The same section prints storage rows this page does not copy. The Ollama pages this page read do not print a GPU GB figure. The unsloth/Llama-3.1-8B-Instruct model information table prints parameter counts 8B, 70B, and 405B, and context length 128k. This page does not treat those figures as vram_gb. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which install lines do the docs print?
The README and the Docker Compose page print mkdir lobehub && cd lobehub, then the setup script at https://lobe.li/setup.sh with -l en, then docker compose up -d, and open http://localhost:3210. The install section copies that script line as the README prints it. The Ollama integration page prints a different command, docker run -d -p 3210:3210 -e OLLAMA_PROXY_URL=http://host.docker.internal:11434 lobehub/lobehub. This page does not merge those installs. Main of lobehub/lobehub on 2026-11-14 was commit a8424d204a7ed1eb36985ebdfa6eac8041615392 on branch canary. Requesting lobehub/lobe-chat returns that repository. This page did not run the commands. A ready-timing line in the success log is not copied.
How do the docs point the app at a server on the same machine?
The environment page’s OLLAMA_PROXY_URL example is http://127.0.0.1:11434. The Ollama integration docker command sets OLLAMA_PROXY_URL to http://host.docker.internal:11434. This page does not merge those URLs. The provider page says to open App Settings, then AI Providers, then Ollama, enter the server URL, and enable the provider. It says to enable Client Request Mode if Ollama runs on the current device. The agent page says a system role and a model are enough to start. The proxy field for an OpenAI-compatible base URL is OPENAI_PROXY_URL. The default host in that field’s docs is not copied. The files do not print an SSH command. The UI port is 3210.
What license sentence does the README print?
The README says this project is LobeHub Community License licensed. The LICENSE file says LobeChat is licensed under the LobeHub Community License, based on Apache License 2.0, with additional conditions. The GitHub license field on 2026-11-14 was NOASSERTION. That license text is not a CloudGPU rate.
Sources (read 2026-11-14, Asia/Shanghai)
- lobehub/lobehub README and LICENSE at commit
a8424d204a7ed1eb36985ebdfa6eac8041615392, committed 2026-11-14T11:11:12Z, branchcanary. GitHub license fieldNOASSERTION.pushed_at2026-11-14T11:51:58Z. Requesting lobehub/lobe-chat returned this repository. Docker Compose block:mkdir lobehub && cd lobehub, the setup script at https://lobe.li/setup.sh with-l en,docker compose up -d,http://localhost:3210. The install section copies the script line as the README prints it. License sentence: LobeHub Community License. A badge in the link group says apache 2.0. Provider names and a product list are not copied - LICENSE at that commit. From 1.0, LobeChat is licensed under the LobeHub Community License, based on Apache License 2.0, with additional conditions
docs/self-hosting/platform/docker-compose.mdxat that commit. Ports3210,9000,9001,8787,8788. Resource section: 2 cores and 4 GB RAM minimum, 4+ cores and 8+ GB RAM recommended. Storage rows are not copied.pg_dumpand the RustFStarplusdocker cplines are copied.LOBE_PORT. A ready-timing line in the success log is not copied. A sample password in the report is not copieddocs/self-hosting/examples/ollama.mdxat that commit.docker run -d -p 3210:3210 -e OLLAMA_PROXY_URL=http://host.docker.internal:11434 lobehub/lobehub.OLLAMA_HOSTdefault"127.0.0.1:11434".OLLAMA_KEEP_ALIVEdefault"5m". LinuxOLLAMA_HOST=0.0.0.0andOLLAMA_ORIGINS=*docs/usage/providers/ollama.mdxat that commit. App Settings, AI Providers, Ollama. Client Request Mode. Linux installcurl -fsSL https://ollama.com/install.sh | sh. No GPU GB figuredocs/self-hosting/environment-variables/model-provider.mdxand.env.exampleat that commit.OLLAMA_PROXY_URLexamplehttp://127.0.0.1:11434.OLLAMA_MODEL_LISTexamplellama2:7B.ENABLED_OLLAMAdefault1.OPENAI_PROXY_URLoverrides the default base URL. That default host is not copied./v1suffix notedocs/usage/getting-started/agent.mdxat that commit. A system role and a model are enough to start. Create Agent. Tool support depends on the model and the provider. The external-agent list is not copied- Hugging Face model API
unsloth/Llama-3.1-8B-Instruct:gatedfalse,cardData.licensellama3.1,pipeline_tagtext-generation,lastModified2025-05-09T19:32:57.000Z,sha4699cc75b550f9c6f3173fb80f4703b62d946aa5. The card YAML includesextra_gated_prompt. The model information table prints 8B, 70B, 405B, and 128k.base_modelnames a repository this page does not copy.hf downloadis added on this page and was not run. The training-hardware section is not copied - Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (
generated_at2026-11-14T12:02:28.530Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no LobeHub or LobeChat template
Cards you can start are on the Deploy page. Amounts are not copied here.
Last updated: 2026-11-14 (Asia/Shanghai).