How to run Agent Zero with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Agent Zero from the official repository with the Docker command the README prints, serve a model on that machine, and choose the documented Ollama or OpenAI-compatible setting that points at that endpoint. Copy the a0_usr volume, or the backup zip the UI downloads, before you destroy the instance, because destroying the instance deletes its disk.
An agent can rent the GPU and run this program on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Agent Zero install. A software-company framework whose example llm block sets api_type, base_url, and model, and whose comment names ollama, is How to run MetaGPT with a self-hosted model on a cloud GPU. A platform whose single-container guide sets CHAT_BASE_URL to http://host.docker.internal:11434/v1 for a local OpenAI-compatible server is How to run AutoGPT with a self-hosted model on a cloud GPU. An agent whose Ollama example sets base_url to http://localhost:11434/v1 is How to run OpenManus with a self-hosted model on a cloud GPU. A chat app whose Docker page runs docker compose up -d and whose Ollama page sets baseURL to http://localhost:11434/v1/ is How to run LibreChat agents with a self-hosted model on a cloud GPU. A chat app whose README runs docker compose up -d and whose environment page sets OLLAMA_PROXY_URL to http://127.0.0.1:11434 is How to run LobeChat agents with a self-hosted model on a cloud GPU. A terminal agent whose install page runs curl -fsSL https://www.openinterpreter.com/install | sh, and whose models page sets the Ollama base URL to http://localhost:11434/v1, is How to run Open Interpreter with a self-hosted model on a cloud GPU. A workspace app whose Docker quickstart publishes port 3001, and whose Ollama page sets the URL to http://127.0.0.1:11434, is How to run AnythingLLM agents with a self-hosted model on a cloud GPU. A repository agent whose local-models section sets api_base to http://localhost:11434 is How to run SWE-agent with a self-hosted model on a cloud GPU. A workflow editor whose Docker install publishes port 5678, and whose Ollama credentials default the Base URL to http://localhost:11434, is How to run n8n AI agents with a self-hosted model on a cloud GPU. A CLI whose providers page sets OLLAMA_HOST to http://localhost:11434 is How to run the goose agent with a self-hosted model on a cloud GPU. A coding agent whose providers page sets package to the OpenAI-compatible provider is How to use OpenCode with a self-hosted model on a cloud GPU. A canvas whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU. A CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A browser chat whose serve command listens on port 8080 is How to run Open WebUI on a cloud GPU. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose OllamaChatGenerator sets url to http://localhost:11434 is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b is How to run Agno agents with a self-hosted model on a cloud GPU. A TypeScript agent whose local-models section sets url on the model object is How to run Mastra agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.
The README says Agent Zero is available under the MIT License.
The commands below are copied from the upstream files named in each step, at the commit in Sources. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. Requesting frdel/agent-zero on 2026-11-14 returned agent0ai/agent-zero. The README and the Docker command use agent0ai/agent-zero. A naming-table example uses an Ollama tag this page does not copy. Provider names other than Ollama, vLLM, and the OpenAI-compatible setting are not copied. Preset names on the model-presets page are not copied.
What the docs print about memory
The README, docs/setup/installation.md, and conf/model_providers.yaml, read 2026-11-14, do not print a GPU GB figure for the Docker install. The oMLX section says oMLX requires Apple Silicon and macOS 15+, and that on 16 GB machines you should start with small quantized models. That 16 GB line is system memory for that Apple Silicon path. This page does not treat it as vram_gb and does not use oMLX as the GPU install. The model card for Qwen/Qwen2.5-1.5B-Instruct prints Number of Parameters 1.54B. It also prints two context lines: long-context support up to 128K tokens and generation up to 8K tokens, and, in the model section, full 32,768 tokens and generation 8192 tokens. This page does not choose between those lines and does not treat them as vram_gb. The card says GPU memory requirements are on another site. This page does not copy a number from that site, because the card itself does not print one. This page does not mark a catalog row as enough.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T13:05:18.022Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Rows are sorted by vram_gb, then by name. Which listing you can start is on Deploy.
The catalog template list had no Agent Zero template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as Agent Zero.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| Ascend 910B | 64 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
Main of agent0ai/agent-zero on 2026-11-14 was commit ba9b67df0f54ac78106ae85c552ef5d2458f7c9a, committed 2026-11-01T21:10:32Z. pushed_at was 2026-11-13T11:24:28Z. The GitHub license field was NOASSERTION. The LICENSE file begins with MIT License. The README says Agent Zero is available under the MIT License.
The README’s direct Docker command, for a machine that already has Docker, is:
docker run -d -p 127.0.0.1:5080:80 -v a0_usr:/a0/usr agent0ai/agent-zero
It says to open http://localhost:5080 on the Docker host. The same command is in docs/setup/installation.md. That page also prints the installer, which is a different command:
curl -fsSL https://bash.agent-zero.ai | bash
The scripted form on that page is also different:
curl -fsSL https://bash.agent-zero.ai | bash -s -- --quick-start --name agent-zero --port 5080
This page does not merge the Docker command, the installer, and the scripted installer. A bind-mount example on the same page uses --mount type=bind,source=/home/user/agent-zero-data,target=/a0/usr with the same image and the same port. That host path is an example. It is not the named volume a0_usr. This page does not merge them.
The files this page read do not print an SSH command. Port 5080 in the README command is bound to 127.0.0.1 and maps to container port 80. The tunnel is How to use SSH port forwarding on a cloud GPU.
Ollama and the OpenAI-compatible setting
The README says a local model server is one of the ways to start, and the first-run choices include Local model. docs/setup/installation.md says to choose a Main model and a Utility model. They can use the same provider.
The Ollama provider in conf/model_providers.yaml sets api_base to http://host.docker.internal:11434. The chat block also sets default_base to that same URL and format to ollama. There is no /v1 suffix on those two URLs. The install page says Agent Zero includes that Docker-friendly default and to override the API base only if Ollama runs somewhere else. The configure steps it prints are: choose Ollama, write the model code in the form llama3.2 or qwen2.5:7b, then click Save. The pull section’s example is ollama pull mistral-large. That tag is not llama3.2 and it is not qwen2.5:7b. This page does not merge them. The Linux install line it prints is:
curl -fsSL https://ollama.com/install.sh | sh
The naming table’s Ollama example is a tag this page does not copy. The same table’s vLLM example is Qwen/Qwen2.5-1.5B-Instruct.
When Agent Zero runs in Docker, the install page says localhost and 127.0.0.1 inside an API base URL mean the Agent Zero container. For a server on the host it says to use http://host.docker.internal: plus the port, or http://172.17.0.1: plus the port on the default Linux bridge. If both containers share a Docker network, the Ollama note says you can use http:// plus the container name plus :11434 instead of host.docker.internal. This page does not merge those three addresses. The angle-bracket tokens in that page are written here inside code spans so the renderer does not treat them as tags.
The vLLM section says vLLM is most common on Linux GPU hosts, then prints an Apple Silicon install, then prints this server command:
vllm serve Qwen/Qwen2.5-1.5B-Instruct --host 0.0.0.0 --port 8000
It says Agent Zero expects vLLM at http://host.docker.internal:8000/v1. That URL has a /v1 suffix. The Ollama api_base does not. This page does not merge them. The page does not print a separate Linux install command for the vllm package. How to serve an OpenAI-compatible API with vLLM on a cloud GPU is the general server install. This page does not paste that guide’s command into Agent Zero’s page.
The provider id other in conf/model_providers.yaml has name: Other OpenAI compatible and litellm_provider set to the OpenAI-compatible setting. The install page says to add the key under External Services -> Other OpenAI-compatible API keys, then select OpenAI Compatible as the provider. That UI label is the documented OpenAI-compatible setting. This page does not copy the other provider names in that file.
The Agent Zero files this page read do not print hf download. The block below is added on this page, for the model id the vLLM section prints. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. This page did not download the repository. vllm serve is not this command.
pip install -U "huggingface_hub"
hf download Qwen/Qwen2.5-1.5B-Instruct
On 2026-11-14, GET https://huggingface.co/api/models/Qwen/Qwen2.5-1.5B-Instruct reported gated false, cardData.license apache-2.0, pipeline_tag text-generation, lastModified 2024-09-25T12:32:50.000Z, and sha 989aa7980e4cf806f80c7fef2b1adb7bc71aa306. The card YAML this page read does not include extra_gated_prompt. This page did not run hf download.
How to run Ollama on a cloud GPU is the Ollama install. The documented Ollama base in Agent Zero is http://host.docker.internal:11434. This page does not add a /v1 suffix that the Ollama block does not print.
A run
The README says to send a message from the welcome screen. The example it prints is: create a one-page plan for a 15-minute team kickoff, save it as a Markdown file, and give a link. If no model is configured, choose Local model, select a Main model and a Utility model, click Finish setup, then Continue. The quickstart prints a longer prompt that saves the file at /a0/usr/workdir/docs-demo-20261030/kickoff-plan.md. That path is not in the README sentence. This page does not merge them. The path sits under /a0/usr, which is the mount for the a0_usr volume.
When it fails
CUDA out of memory. The Agent Zero files this page read print no GPU GB figure. The Qwen2.5-1.5B-Instruct card does not print one either. If the model server raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The Agent Zero README and the install page this page read do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this program on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
No first reply. The README says to check that setup finished and that both models are available through the chosen provider, and to read any connection error shown in the chat. The install page says localhost inside the container is not the host. The Ollama default it ships is http://host.docker.internal:11434.
Port in use. The README says to replace 5080 with another port and to use that port in the browser URL. The published container port in the command is 80.
What to copy off, then destroy
The README says the a0_usr volume keeps your data when the container is replaced, and to keep backups too. docs/setup/installation.md says that volume holds settings, chats, memory, projects, and custom plugins. It says to create a backup from Settings -> Check for Updates -> Backup & Restore -> Create Backup and keep the downloaded .zip file. Copy that zip off the machine before you destroy the instance. A bind-mount install uses the host directory you passed as source instead of the named volume a0_usr. Copy the directory you actually mounted.
Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The billing page states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Amounts are not copied here.
Frequently asked questions
What memory figure do the Agent Zero docs print?
The README, installation.md, and model_providers.yaml do not print a GPU GB figure for the Docker install. The oMLX section says 16 GB machines, and that line is system memory for an Apple Silicon path. This page does not treat it as vram_gb. The Qwen/Qwen2.5-1.5B-Instruct card prints 1.54B parameters, one context line of 128K tokens with generation up to 8K tokens, and another line of 32,768 tokens with generation 8192 tokens. This page does not treat those figures as vram_gb. The card does not print a GPU GB number. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which install lines do the docs print?
The README prints docker run -d -p 127.0.0.1:5080:80 -v a0_usr:/a0/usr agent0ai/agent-zero and says to open http://localhost:5080. installation.md also prints curl -fsSL https://bash.agent-zero.ai | bash, and a scripted form that passes --quick-start --name agent-zero --port 5080. Requesting frdel/agent-zero returns agent0ai/agent-zero. This page does not merge those installs. Main on 2026-11-14 was commit ba9b67df0f54ac78106ae85c552ef5d2458f7c9a. This page did not run the commands.
How do the docs point Agent Zero at a server on the same machine?
The Ollama provider sets api_base to http://host.docker.internal:11434, with no /v1 suffix. The install page says to choose Ollama and a model string such as llama3.2 or qwen2.5:7b. The vLLM section prints vllm serve Qwen/Qwen2.5-1.5B-Instruct --host 0.0.0.0 --port 8000 and expects http://host.docker.internal:8000/v1. The provider named Other OpenAI compatible is the documented OpenAI-compatible setting. localhost inside the container is not the host. The files do not print an SSH command. The published host port in the README command is 5080, bound to 127.0.0.1.
What license sentence does the README print?
The README says Agent Zero is available under the MIT License. The LICENSE file begins with MIT License. The GitHub license field on 2026-11-14 was NOASSERTION. That license text is not a CloudGPU rate.
Sources (read 2026-11-14, Asia/Shanghai)
- agent0ai/agent-zero README at commit
ba9b67df0f54ac78106ae85c552ef5d2458f7c9a, committed 2026-11-01T21:10:32Z. GitHub license fieldNOASSERTION.pushed_at2026-11-13T11:24:28Z. Requesting frdel/agent-zero returned this repository. Docker command publishes127.0.0.1:5080:80and volumea0_usrat/a0/usr. License sentence: available under the MIT License. The first-message example asks for a one-page plan saved as Markdown. A product name in a later section is not copied LICENSEat that commit. Begins withMIT Licensedocs/setup/installation.mdat that commit. Installercurl -fsSL https://bash.agent-zero.ai | bash. Scripted--quick-start --name agent-zero --port 5080. Ollamaapi_basedefaulthttp://host.docker.internal:11434. Model-code examplesllama3.2andqwen2.5:7b. Pull exampleollama pull mistral-large. vLLM commandvllm serve Qwen/Qwen2.5-1.5B-Instruct --host 0.0.0.0 --port 8000and basehttp://host.docker.internal:8000/v1. OpenAI-compatible UI label OpenAI Compatible. oMLX 16 GB line is system memory for Apple Silicon. The naming table’s Ollama example tag is not copied. Backup path: Settings, Check for Updates, Backup & Restore, Create Backup, downloaded.zipdocs/quickstart.mdat that commit. Longer prompt saves/a0/usr/workdir/docs-demo-20261030/kickoff-plan.md. That path is not the README sentenceconf/model_providers.yamlat that commit. Ollama chatapi_basehttp://host.docker.internal:11434. ProviderothernameOther OpenAI compatible. Other provider names in that file are not copied- Hugging Face model API
Qwen/Qwen2.5-1.5B-Instruct:gatedfalse,cardData.licenseapache-2.0,pipeline_tagtext-generation,lastModified2024-09-25T12:32:50.000Z,sha989aa7980e4cf806f80c7fef2b1adb7bc71aa306. Card text: 1.54B parameters, 128K with generation 8K, and 32,768 with generation 8192. No GPU GB figure on the card.hf downloadis added on this page and was not run - Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (
generated_at2026-11-14T13:05:18.022Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no Agent Zero template
Cards you can start are on the Deploy page. Amounts are not copied here.
Last updated: 2026-11-14 (Asia/Shanghai).