How to run LibreChat agents with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install LibreChat with the Docker commands the official local Docker page prints, serve a model on that machine, and set the Ollama custom endpoint baseURL that page prints. Copy the bind-mounted project files and the two named volumes before you destroy the instance, because destroying the instance deletes its disk.
An agent can rent the GPU and run this application on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the LibreChat install. A chat app whose README runs docker compose up -d and whose environment page sets OLLAMA_PROXY_URL to http://127.0.0.1:11434 is How to run LobeChat agents with a self-hosted model on a cloud GPU. An agent whose Ollama example sets base_url to http://localhost:11434/v1 is How to run OpenManus with a self-hosted model on a cloud GPU. A terminal agent whose install page runs curl -fsSL https://www.openinterpreter.com/install | sh, and whose models page sets the Ollama base URL to http://localhost:11434/v1, is How to run Open Interpreter with a self-hosted model on a cloud GPU. A workspace app whose Docker quickstart publishes port 3001, and whose Ollama page sets the URL to http://127.0.0.1:11434, is How to run AnythingLLM agents with a self-hosted model on a cloud GPU. A repository agent whose local-models section sets api_base to http://localhost:11434 is How to run SWE-agent with a self-hosted model on a cloud GPU. A workflow editor whose Docker install publishes port 5678, and whose Ollama credentials default the Base URL to http://localhost:11434, is How to run n8n AI agents with a self-hosted model on a cloud GPU. A CLI whose providers page sets OLLAMA_HOST to http://localhost:11434 is How to run the goose agent with a self-hosted model on a cloud GPU. A coding agent whose providers page sets package to the OpenAI-compatible provider is How to use OpenCode with a self-hosted model on a cloud GPU. A canvas whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU. A CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A browser chat whose serve command listens on port 8080 is How to run Open WebUI on a cloud GPU. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose OllamaChatGenerator sets url to http://localhost:11434 is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b is How to run Agno agents with a self-hosted model on a cloud GPU. A TypeScript agent whose local-models section sets url on the model object is How to run Mastra agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.
The GitHub license field is MIT.
The commands below are copied from the upstream files named in each step, at the commits in Sources. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The README lists provider names this page does not copy. A first-launch sentence on the Docker page is a timing line this page does not copy.
What the docs print about memory
The Docker page, docs commit 1f374e68be25743cecad3bc9bdd3b9e5d574e8bc committed 2026-11-13T17:31:52Z, says LibreChat does not run AI models itself, so it does not need a GPU. It says the containers need at least 1 GiB of RAM and 1 vCPU, and to make at least 2 GB of memory available to Docker when the default stack is running. Those figures are system memory. They are not vram_gb. The same page says that if you connect LibreChat to models on the same machine, size the hardware for the model, and that the model runtime needs far more memory, and often a GPU, than LibreChat does. It does not print a GPU GB figure. It also quotes Docker Desktop system RAM of at least 4 GB on macOS and Linux and 8 GB on Windows. Those are the system RAM figures that page quotes. They are not vram_gb. The model information table on unsloth/Llama-3.1-8B-Instruct prints parameter counts 8B, 70B, and 405B, and a context length of 128k. This page does not treat those parameter counts as vram_gb. A later section of that card names training hardware. This page does not copy that section and does not use it as an inference memory figure. This page does not mark a catalog row as enough.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T12:02:28.530Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Rows are sorted by vram_gb, then by name. Which listing you can start is on Deploy.
The catalog template list had no LibreChat template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as LibreChat.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| Ascend 910B | 64 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
Requesting danny-avila/LibreChat on 2026-11-14 returned LibreChat-AI/LibreChat. Main of that repository was commit fad2c51da41bb46c9cdd44a720cea4df0d47876b, committed 2026-11-13T18:02:53Z. pushed_at was 2026-11-14T02:14:24Z. The GitHub license field was MIT. The LICENSE file begins with MIT License. The README this page read does not print a license line. The Docker page clones https://github.com/LibreChat-AI/LibreChat.git. Docs main the same day was commit 1f374e68be25743cecad3bc9bdd3b9e5d574e8bc.
The Docker page prints:
git clone https://github.com/LibreChat-AI/LibreChat.git
cd LibreChat
cp .env.example .env
docker compose up -d
It says to open http://localhost:3080. .env.example at the app commit sets PORT=3080. docker-compose.yml publishes "${PORT}:${PORT}" and sets the API image to registry.librechat.ai/librechat-ai/librechat-dev:latest. The page says the first account you register becomes the admin account in an unscoped single-tenant deployment, and that there are no default credentials. The local setup page prints the same clone, cp, and docker compose up -d.
The files this page read do not print an SSH command. The UI port those files print is 3080. The tunnel for a port you choose to publish is How to use SSH port forwarding on a cloud GPU.
Custom endpoint
The Docker page says a custom librechat.yaml has to be mounted. It prints:
cp docker-compose.override.yml.example docker-compose.override.yml
And this mount:
services:
api:
volumes:
- type: bind
source: ./librechat.yaml
target: /app/librechat.yaml
Then:
docker compose down && docker compose up -d
The Ollama page says Ollama exposes an OpenAI-compatible API. It says Ollama ignores the API key but still expects the field. The block it prints is:
endpoints:
custom:
- name: "Ollama"
apiKey: "ollama"
# use 'host.docker.internal' instead of localhost if running LibreChat in a docker container
baseURL: "http://localhost:11434/v1/"
models:
default: [
"llama2",
"mistral",
"codellama",
"mistral-openorca"
]
fetch: true
titleConvo: true
titleModel: "current_model"
modelDisplayLabel: "Ollama"
The Ollama page’s models.default list includes one more name. This page does not copy that name. The comment on baseURL says to use host.docker.internal instead of localhost when LibreChat runs in a Docker container. The troubleshooting section on that page says http://localhost:11434/v1/ is correct only when LibreChat is not in Docker, and that Docker Desktop should use http://host.docker.internal:11434/v1/. The custom endpoints page prints baseURL: 'http://host.docker.internal:11434/v1/' in its Ollama entry. This page does not merge http://localhost:11434/v1/ and http://host.docker.internal:11434/v1/. The custom endpoints page’s models.default list is not copied. docker-compose.yml sets extra_hosts to host.docker.internal:host-gateway. This page does not treat that line as a replacement for either URL.
The custom endpoints page says a custom endpoint is kept only if name, baseURL, apiKey, and models are all present. An entry missing any of them is removed with no error and no log line. models.default must list at least one model even when fetch is true. The Ollama page says the endpoint name must start with ollama, case-insensitively, for the native tags route. Any other name falls through to the OpenAI-compatible /v1/models request. It says to set OLLAMA_HOST=0.0.0.0 if Ollama is still bound to 127.0.0.1, and it says that binding is unauthenticated.
The LibreChat files this page read do not print hf download. The block below is added on this page. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. This page did not download the repository. The LibreChat files this page read do not print this repository id. llama2 and the other names in the yaml are the strings the Ollama page prints. This page does not treat them as this repository id.
pip install -U "huggingface_hub"
hf download unsloth/Llama-3.1-8B-Instruct
On 2026-11-14, GET https://huggingface.co/api/models/unsloth/Llama-3.1-8B-Instruct reported gated false, cardData.license llama3.1, pipeline_tag text-generation, lastModified 2025-05-09T19:32:57.000Z, and sha 4699cc75b550f9c6f3173fb80f4703b62d946aa5. The card YAML also includes extra_gated_prompt. Those two fields are not the same fact. The card’s base_model field names a repository this page does not copy. This page did not run hf download.
How to run Ollama on a cloud GPU is the Ollama install. How to serve an OpenAI-compatible API with vLLM on a cloud GPU is the general server install.
Agents
The agents page says agents are on by default. It says to open Agent Builder from the side panel and click Create New Agent, then give it a name, an optional description, and pick a model and a category. It says the model and the category are required. Then write the instructions and click Create. To chat, select the agent in the model selector, or type @ and pick it. A sentence on that page names a vendor this page does not copy. The tools section names integrations this page does not copy. Advanced Settings has Max Agent Steps. The page says leave it empty to use the server default, which is 50 unless an admin changed it. That 50 is a step cap. It is not vram_gb.
When it fails
CUDA out of memory. The LibreChat files this page read print no GPU GB figure. If the model server raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The LibreChat README and the docs pages this page read do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this program on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
Port 3080 is already in use. The Docker page says the simplest fix is to set PORT=3081 in .env, then docker compose down && docker compose up -d, and open http://localhost:3081.
The endpoint never appears. The custom endpoints page says to check name, baseURL, apiKey, and models, and to read docker compose logs api. The Ollama page says localhost from inside the API container is the container, not the host.
What to copy off, then destroy
docker-compose.yml bind-mounts ./.env, ./images, ./uploads, ./logs, ./skill, and ./data-node. The override mount adds ./librechat.yaml. Meilisearch uses ./meili_data_v1.35.1. Copy those host paths, including librechat.yaml and .env. That copy does not include the named volumes.
The same file mounts librechat-data at /app/data on the service whose container_name is LibreChat, and pgdata2 at /var/lib/postgresql/data on the service whose container_name is vectordb. The remote Docker page says the librechat-data volume keeps /app/data/.env.temp across container recreation, and that losing that file or changing established credentials can invalidate sessions and make encrypted records unreadable. The LibreChat files this page read do not print a backup or restore command for either named volume.
Docker’s volumes page backs up a volume with --volumes-from and tar. Its example container is named dbstore and the path is /dbdata. Those are not LibreChat, vectordb, /app/data, or /var/lib/postgresql/data. The commands below are added on this page. They use that page’s --volumes-from and tar cvf shape, and the container names and mount paths from docker-compose.yml. This page did not run them. The vectordb copy is a copy of the data directory. The Docker page does not say to stop the database first.
docker run --rm --volumes-from LibreChat -v "$(pwd)":/backup ubuntu tar cvf /backup/librechat-data.tar /app/data
docker run --rm --volumes-from vectordb -v "$(pwd)":/backup ubuntu tar cvf /backup/pgdata2.tar /var/lib/postgresql/data
Docker’s restore example is cd /dbdata && tar xvf /backup/backup.tar --strip 1, because that example tars the one-component path /dbdata. The archives above are rooted at app/data and var/lib/postgresql/data. This page does not use --strip 1 on them. -C / is added so those archive paths land on the same mounts. This page did not run the restore.
docker run --rm --volumes-from LibreChat -v "$(pwd)":/backup ubuntu tar xvf /backup/librechat-data.tar -C /
docker run --rm --volumes-from vectordb -v "$(pwd)":/backup ubuntu tar xvf /backup/pgdata2.tar -C /
Copy librechat-data.tar and pgdata2.tar off the instance with the bind-mounted files. The tar files are written into the working directory on the instance disk.
Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The billing page states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Amounts are not copied here.
Frequently asked questions
What memory figure do the LibreChat docs print?
The Docker page says the containers need at least 1 GiB of RAM and 1 vCPU, and to make at least 2 GB of memory available to Docker. It also quotes Docker Desktop system RAM of at least 4 GB on macOS and Linux and 8 GB on Windows. Those figures are system memory. They are not vram_gb. The same page does not print a GPU GB figure. The unsloth/Llama-3.1-8B-Instruct model information table prints parameter counts 8B, 70B, and 405B, and context length 128k. This page does not treat those figures as vram_gb. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which install lines do the docs print?
The Docker page prints git clone https://github.com/LibreChat-AI/LibreChat.git, then cd LibreChat, cp .env.example .env, and docker compose up -d. It says to open http://localhost:3080. .env.example sets PORT=3080. Main of LibreChat-AI/LibreChat on 2026-11-14 was commit fad2c51da41bb46c9cdd44a720cea4df0d47876b. Requesting danny-avila/LibreChat returns that repository. This page did not run the commands. A first-launch timing sentence on that page is not copied.
How do the docs point LibreChat at a server on the same machine?
The Ollama page sets baseURL to http://localhost:11434/v1/ and says to use host.docker.internal instead of localhost when LibreChat runs in Docker. The custom endpoints page sets baseURL to http://host.docker.internal:11434/v1/. This page does not merge those URLs. A custom endpoint is kept only if name, baseURL, apiKey, and models are present. The agents page says agents are on by default. The files do not print an SSH command. The published port is 3080.
What license sentence does the project print?
The GitHub license field on 2026-11-14 was MIT. The LICENSE file begins with MIT License. The README this page read does not print a license line. That license text is not a CloudGPU rate.
Sources (read 2026-11-14, Asia/Shanghai)
- LibreChat-AI/LibreChat README and LICENSE at commit
fad2c51da41bb46c9cdd44a720cea4df0d47876b, committed 2026-11-13T18:02:53Z. GitHub license field MIT.pushed_at2026-11-14T02:14:24Z. Requesting danny-avila/LibreChat returned this repository. Provider names on the README are not copied. The README this page read does not print a license line docker-compose.ymland.env.exampleat that commit. Imageregistry.librechat.ai/librechat-ai/librechat-dev:latest.PORT=3080. Bind mounts./.env,./images,./uploads,./logs,./skill,./data-node,./meili_data_v1.35.1.librechat-datais mounted at/app/dataoncontainer_nameLibreChat.pgdata2is mounted at/var/lib/postgresql/dataoncontainer_namevectordb.extra_hostshost.docker.internal:host-gateway. The file does not print a backup command for those named volumes- LibreChat-AI/docs
content/docs/remote/docker_linux.mdxat docs commit1f374e68be25743cecad3bc9bdd3b9e5d574e8bc. Thelibrechat-datavolume keeps/app/data/.env.temp. Losing that file or changing established credentials can invalidate sessions and make encrypted records unreadable. Placeholder tokens on that page are not copied - Docker docs
content/manuals/engine/storage/volumes.mdat commit553482ce632f00c6e7faabab863422b4916e7c18, committed 2026-11-13T17:05:54Z. Backup:docker run --rm --volumes-from dbstore -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar /dbdata. Restore:cd /dbdata && tar xvf /backup/backup.tar --strip 1. The example container isdbstoreand the path is/dbdata. The LibreChat commands in the copy section are added on this page and were not run - LibreChat-AI/docs
content/docs/local/docker.mdxat commit1f374e68be25743cecad3bc9bdd3b9e5d574e8bc, committed 2026-11-13T17:31:52Z. Clone,cp .env.example .env,docker compose up -d,http://localhost:3080. System RAM 1 GiB and 2 GB. Docker Desktop system RAM 4 GB and 8 GB. No GPU GB figure. A first-launch timing sentence is not copied.PORT=3081when 3080 is in use content/docs/configuration/librechat_yaml/ai_endpoints/ollama.mdxat that docs commit.baseURLhttp://localhost:11434/v1/. Docker notehost.docker.internal.apiKeyollama.fetch: true.OLLAMA_HOST=0.0.0.0warning. Model names inmodels.defaultare the strings that page printscontent/docs/quick_start/custom_endpoints.mdxat that docs commit. Required fieldsname,baseURL,apiKey,models. Ollama examplebaseURLhttp://host.docker.internal:11434/v1/. That page’s model list is not copiedcontent/docs/features/agents.mdxat that docs commit. Agents are on by default. Agent Builder, Create New Agent, model and category required. Max Agent Steps server default 50. A vendor name and the tools-section product names are not copied- Hugging Face model API
unsloth/Llama-3.1-8B-Instruct:gatedfalse,cardData.licensellama3.1,pipeline_tagtext-generation,lastModified2025-05-09T19:32:57.000Z,sha4699cc75b550f9c6f3173fb80f4703b62d946aa5. The card YAML includesextra_gated_prompt. The model information table prints 8B, 70B, 405B, and 128k.base_modelnames a repository this page does not copy.hf downloadis added on this page and was not run. The training-hardware section is not copied - Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (
generated_at2026-11-14T12:02:28.530Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no LibreChat template
Cards you can start are on the Deploy page. Amounts are not copied here.
Last updated: 2026-11-14 (Asia/Shanghai).