How to run AutoGPT with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install the AutoGPT Platform with the Docker command the official README prints, serve a model on that machine, and point the documented local OpenAI-compatible or Ollama setting at that endpoint. Copy the autogpt-data volume before you destroy the instance, because destroying the instance deletes its disk.
An agent can rent the GPU and run this platform on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the AutoGPT Platform install. A software-company framework whose example llm block sets api_type, base_url, and model, and whose comment names ollama, is How to run MetaGPT with a self-hosted model on a cloud GPU. An agent whose Docker command publishes port 5080, and whose Ollama provider sets api_base to http://host.docker.internal:11434, is How to run Agent Zero with a self-hosted model on a cloud GPU. An agent whose Ollama example sets base_url to http://localhost:11434/v1 is How to run OpenManus with a self-hosted model on a cloud GPU. A chat app whose Docker page runs docker compose up -d and whose Ollama page sets baseURL to http://localhost:11434/v1/ is How to run LibreChat agents with a self-hosted model on a cloud GPU. A chat app whose README runs docker compose up -d and whose environment page sets OLLAMA_PROXY_URL to http://127.0.0.1:11434 is How to run LobeChat agents with a self-hosted model on a cloud GPU. A terminal agent whose install page runs curl -fsSL https://www.openinterpreter.com/install | sh, and whose models page sets the Ollama base URL to http://localhost:11434/v1, is How to run Open Interpreter with a self-hosted model on a cloud GPU. A workspace app whose Docker quickstart publishes port 3001, and whose Ollama page sets the URL to http://127.0.0.1:11434, is How to run AnythingLLM agents with a self-hosted model on a cloud GPU. A repository agent whose local-models section sets api_base to http://localhost:11434 is How to run SWE-agent with a self-hosted model on a cloud GPU. A workflow editor whose Docker install publishes port 5678, and whose Ollama credentials default the Base URL to http://localhost:11434, is How to run n8n AI agents with a self-hosted model on a cloud GPU. A CLI whose providers page sets OLLAMA_HOST to http://localhost:11434 is How to run the goose agent with a self-hosted model on a cloud GPU. A coding agent whose providers page sets package to the OpenAI-compatible provider is How to use OpenCode with a self-hosted model on a cloud GPU. A canvas whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU. A CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A browser chat whose serve command listens on port 8080 is How to run Open WebUI on a cloud GPU. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose OllamaChatGenerator sets url to http://localhost:11434 is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b is How to run Agno agents with a self-hosted model on a cloud GPU. A TypeScript agent whose local-models section sets url on the model object is How to run Mastra agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.
The repository LICENSE says everything inside the autogpt_platform folder is under the Polyform Shield License and everything outside that folder is under the MIT License.
The commands below are copied from the upstream files named in each step, at the commit in Sources. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The README and autogpt_platform/single-container/README.md each print a first-boot timing sentence. This page does not copy those sentences. The Models and memory section also prints remote profiles whose provider names this page does not copy. A spend-cap example prints positive amounts this page does not copy. A plans page linked from the README is not copied.
What the docs print about memory
docs/platform/single-container.md says test installations used about 5–6 GiB of memory during startup and steady-state health checks, and that the image needs more than that measured appliance memory use, with additional headroom for agents, local models, and workload growth. The same page says --shm-size 2g raises /dev/shm above Docker’s 64 MB default. Those figures are appliance memory and shared memory. They are not vram_gb. The local-profile section says to set OLLAMA_CONTEXT_LENGTH=32768 and says Ollama’s smaller default context cannot hold a system prompt of roughly 8k tokens. Those are token counts. The same section calls the default chat model a 4B model chosen for the appliance’s shared memory budget. This page does not treat 4B as vram_gb.
The model card for unsloth/Qwen3.5-4B-GGUF prints Number of Parameters 4B, a native context length of 262,144 tokens, and says that length is extensible up to 1,010,000 tokens. It also says that if you encounter out-of-memory errors, consider reducing the context window. This page did not find a GPU GB figure on that card. A serving note on the card mentions tensor parallel on 8 GPUs for a 262,144-token context. This page does not treat that GPU count as vram_gb and does not copy that command. docs/platform/ollama.md lists model strings including llama3.1:405b. That string is a model name. This page does not treat it as vram_gb. This page does not mark a catalog row as enough.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T13:05:18.022Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Rows are sorted by vram_gb, then by name. Which listing you can start is on Deploy.
The catalog template list had no AutoGPT template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as AutoGPT.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| Ascend 910B | 64 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
Main of Significant-Gravitas/AutoGPT on 2026-11-14 was commit ac4e88888d3838fd5790dff2a85845a654d05024, committed 2026-11-11T15:04:35Z. pushed_at was 2026-11-14T12:55:22Z. The default branch is master. The GitHub license field was NOASSERTION. The repository LICENSE splits Polyform Shield and MIT as the sentence above states. The README table says the same split for autogpt_platform/ and for classic/ plus everything else.
The README’s self-host section prints this single-container command. A timing sentence between the command and the next step is not copied.
docker run -d \
--name autogpt \
--restart unless-stopped \
--shm-size 2g \
--ulimit nofile=65536:65536 \
--log-driver json-file \
--log-opt max-size=50m \
--log-opt max-file=5 \
-p 127.0.0.1:3000:3000 \
-e AUTOGPT_PUBLIC_URL=http://localhost:3000 \
-v autogpt-data:/data \
significantgravitas/autogpt:latest
The README then says to open http://localhost:3000, create an account, and run:
docker exec autogpt autogpt-admin promote you@example.com
docs/platform/single-container.md prints a different quick start. It writes autogpt.env with AUTOGPT_PUBLIC_URL=http://localhost:3000 and AUTH_SIGNUP_ALLOWLIST=owner@example.com, then runs docker run with --env-file autogpt.env, --publish 127.0.0.1:3000:3000, --volume autogpt-data:/data, and "${IMAGE}". IMAGE in that page is significantgravitas/autogpt:latest when you use the pull it prints. That command is not the README command. This page does not merge them. The same page prints:
docker inspect --format '{{.State.Health.Status}}' autogpt
docs/platform/ollama.md prints a third install, from the platform source tree: cd autogpt_platform and docker compose up -d --build, then http://localhost:3000 and http://localhost:3000/build. That compose command is not the single-container image. This page does not merge them.
The files this page read do not print an SSH command. Port 3000 in the README command is bound to 127.0.0.1. The tunnel is How to use SSH port forwarding on a cloud GPU.
The documented local model settings
Upstream documents two local routes. They do not use the same host string. This page does not merge them.
docs/platform/single-container.md, section “Ollama or another local OpenAI-compatible server”, prints:
ollama pull hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M
ollama pull nomic-embed-text
It says to set OLLAMA_CONTEXT_LENGTH=32768 on the host Ollama service and restart Ollama before using the product. That line is not an autogpt.env entry. Then it prints:
CHAT_USE_LOCAL=true
CHAT_BASE_URL=http://host.docker.internal:11434/v1
CHAT_API_KEY=ollama
CHAT_FAST_STANDARD_MODEL=hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M
GRAPHITI_LLM_MODEL=hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M
GRAPHITI_RERANKER_MODEL=hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M
CHAT_API_KEY must be non-empty even if the local server ignores it. The page says Graphiti’s own local default is hf.co/ornith-ai/Ornith-1.5-9B-GGUF:Q4_K_M, so the two GRAPHITI_* lines keep extraction and reranking on the 4B chat model, and embeddings use nomic-embed-text. On Docker Engine, the same section says to add --add-host host.docker.internal:host-gateway to the run command, and to set OLLAMA_HOST=0.0.0.0:11434 because Ollama binds only to 127.0.0.1 by default. It says Docker Desktop provides that hostname without either change. The same settings can point at a remote vLLM or another OpenAI-compatible HTTPS endpoint, provided it serves every configured chat and Graphiti model slug. The sentence also names other servers. This page does not need a second list. Do not expose an unauthenticated model server to the internet. The check it prints is:
docker exec autogpt \
curl --fail --show-error http://host.docker.internal:11434/api/tags
docs/platform/ollama.md is the other route. It says Ollama integration is only available when self-hosting the platform. For a GPU host it prints:
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
docker exec -it ollama ollama run llama3.2
That page also prints a CPU-only docker run and an AMD docker run. This page does not treat those as the GPU command. Method C prints export OLLAMA_HOST=0.0.0.0:11434, ollama serve, and ollama pull llama3.2. That method is not Method B. The block steps say to enter any value in the API key field, and the examples are dummy and not-needed. The model dropdown example is llama3.2. The Ollama Host example is 192.168.0.39:11434, with no scheme and no /v1. The page says to use the host machine’s IP because the platform runs in Docker and cannot reach localhost on the host. That host string is not http://host.docker.internal:11434/v1. This page does not merge them. The compatible model strings that page lists are llama3.2, llama3.3, llama3, llama3.1:405b, and dolphin-mistral:latest. Those are strings the dropdown documents. They are not repository ids.
The AutoGPT files this page read do not print hf download. The block below is added on this page. The single-container guide links the repository unsloth/Qwen3.5-4B-GGUF and tells you to keep the pull identifier identical to the environment setting. It does not print hf download. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. This page did not download the repository. The Ollama tag hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M is not this command.
pip install -U "huggingface_hub"
hf download unsloth/Qwen3.5-4B-GGUF
On 2026-11-14, GET https://huggingface.co/api/models/unsloth/Qwen3.5-4B-GGUF reported gated false, cardData.license apache-2.0, pipeline_tag image-text-to-text, lastModified 2026-03-02T14:08:17.000Z, and sha e87f176479d0855a907a41277aca2f8ee7a09523. The card YAML this page read does not include extra_gated_prompt. This page did not run hf download.
How to run Ollama on a cloud GPU is the Ollama install. How to serve an OpenAI-compatible API with vLLM on a cloud GPU is the general server install. The single-container CHAT_BASE_URL keeps the /v1 suffix. The ollama.md host example does not have that suffix. This page does not remove either one.
A run
docs/platform/ollama.md says to add an AI Text Generator block, set the API key field to any value, select llama3.2 in the model dropdown, set Ollama Host to the host IP and port 11434, add prompts, save the graph, and run it. That run uses the ollama.md host string, not CHAT_BASE_URL.
The README run, after the container is up, is to open http://localhost:3000, create the account, and promote it with docker exec autogpt autogpt-admin promote you@example.com. The single-container guide promotes owner@example.com instead of you@example.com, because that is the address in its env example. This page does not merge the two email strings.
The custom-model section of docs/platform/ollama.md says a new catalog entry sets provider to ollama and context_window to 8192 in the example, and that a run-cost integer is fine for local usage because cost tracking is disabled for self-hosted instances. This page does not copy that integer and does not treat it as a rental rate. It also says the prebuilt single-container image cannot pick up a new catalog entry.
When it fails
CUDA out of memory. The AutoGPT pages this page read do not print a GPU GB figure. The 5–6 GiB line is appliance memory. The Qwen3.5 card says to reduce the context window if an out-of-memory error occurs. If the model server raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The AutoGPT README and the pages this page read do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this program on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
Connection refused. docs/platform/ollama.md says the usual cause is localhost or 127.0.0.1 in the Ollama Host field, and the fix it prints is the host IP with port 11434. It says curl http://localhost:11434/api/tags should work from the host machine. The single-container check uses http://host.docker.internal:11434/api/tags from inside the container. This page does not merge those two URLs.
Missing credential. The single-container page says provider-backed requests return a missing-credential error until a profile is configured, and that CHAT_API_KEY must be non-empty for the local profile. The ollama.md page says any value works for the API key field.
What to copy off, then destroy
The README mounts named volume autogpt-data at /data. docs/platform/single-container.md says all durable state is under /data, including /data/workspaces, /data/postgres, /data/falkordb, and /data/config. The Cold backup section prints a bash block that stops the container named autogpt and archives that volume. This page does not reprint that block. Run the block the guide prints, then copy the archive off the machine before you destroy the instance. The ollama.md GPU command uses a different volume, ollama, mounted at /root/.ollama. That volume is the model store for that container. It is not autogpt-data.
Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The billing page states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Amounts are not copied here.
Frequently asked questions
What memory figure do the AutoGPT docs print?
single-container.md says test installations used about 5–6 GiB of memory and that --shm-size 2g raises /dev/shm above Docker’s 64 MB default. Those figures are appliance memory and shared memory, not vram_gb. The local profile sets OLLAMA_CONTEXT_LENGTH to 32768 and describes a system prompt of roughly 8k tokens. The unsloth/Qwen3.5-4B-GGUF card prints 4B parameters, a native context of 262,144 tokens, and an out-of-memory sentence that says to reduce the context window. This page did not find a GPU GB figure on that card. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which install lines do the docs print?
The README prints docker run of significantgravitas/autogpt:latest, publishing 127.0.0.1:3000 to container port 3000, with AUTOGPT_PUBLIC_URL=http://localhost:3000 and volume autogpt-data at /data. It then prints docker exec autogpt autogpt-admin promote you@example.com and says to open http://localhost:3000. A first-boot timing sentence is not copied. single-container.md prints a different docker run that uses --env-file autogpt.env. ollama.md prints cd autogpt_platform and docker compose up -d --build. This page does not merge the three installs. Main on 2026-11-14 was commit ac4e88888d3838fd5790dff2a85845a654d05024. This page did not run the commands.
How do the docs point AutoGPT at a server on the same machine?
single-container.md sets CHAT_USE_LOCAL to true, CHAT_BASE_URL to http://host.docker.internal:11434/v1, and CHAT_API_KEY to ollama. The chat model string it prints is hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_M. On Docker Engine it also prints --add-host host.docker.internal:host-gateway and OLLAMA_HOST=0.0.0.0:11434. ollama.md instead sets the Ollama Host field to a host IP with port 11434, for example 192.168.0.39:11434, and selects llama3.2. That page says the API key field can be any value. This page does not merge the two host strings. The files do not print an SSH command. The published port in the README command is 3000, bound to 127.0.0.1.
What license sentence does the repository print?
The repository LICENSE says everything inside the autogpt_platform folder is under the Polyform Shield License and everything outside that folder is under the MIT License. The GitHub license field on 2026-11-14 was NOASSERTION. That license text is not a CloudGPU rate.
Sources (read 2026-11-14, Asia/Shanghai)
- Significant-Gravitas/AutoGPT README at commit
ac4e88888d3838fd5790dff2a85845a654d05024, committed 2026-11-11T15:04:35Z. Default branchmaster. GitHub license fieldNOASSERTION.pushed_at2026-11-14T12:55:22Z. Self-hostdocker runofsignificantgravitas/autogpt:latest.docker exec autogpt autogpt-admin promote you@example.com. A first-boot timing sentence, a plans link, and the cost row of the comparison table are not copied - Repository
LICENSEat that commit.autogpt_platformis Polyform Shield. Everything outside that folder is MIT docs/platform/single-container.mdat that commit. 5–6 GiB appliance memory.--shm-size 2g. Separate quick start with--env-file. Local profileCHAT_BASE_URL=http://host.docker.internal:11434/v1.ollama pull hf.co/unsloth/Qwen3.5-4B-GGUF:Q4_K_Mandollama pull nomic-embed-text.OLLAMA_CONTEXT_LENGTH=32768. Cold backup block is not reprinted. Remote profile names and the positive spend-cap amounts are not copied. Both cap variables default to-1. That sentinel is not a rental rateautogpt_platform/single-container/README.mdat that commit. Same image command shape as the root README, plus a first-boot timing sentence this page does not copydocs/platform/ollama.mdat that commit. GPUdocker run --gpus=allandollama run llama3.2. Ollama Host example192.168.0.39:11434. API key examplesdummyandnot-needed. Compatible stringsllama3.2,llama3.3,llama3,llama3.1:405b,dolphin-mistral:latest. Source composedocker compose up -d --build. The catalog example’s run-cost integer is not copied- Hugging Face model API
unsloth/Qwen3.5-4B-GGUF:gatedfalse,cardData.licenseapache-2.0,pipeline_tagimage-text-to-text,lastModified2026-03-02T14:08:17.000Z,shae87f176479d0855a907a41277aca2f8ee7a09523. Card text: 4B parameters, context 262,144 native and extensible to 1,010,000 tokens, and an out-of-memory sentence. No GPU GB figure found on the card.hf downloadis added on this page and was not run - Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (
generated_at2026-11-14T13:05:18.022Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no AutoGPT template
Cards you can start are on the Deploy page. Amounts are not copied here.
Last updated: 2026-11-14 (Asia/Shanghai).