How to run OpenManus with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install OpenManus with the commands the official README prints, serve a model on that machine, and set base_url, model, and api_key from the Ollama example config. Copy config/config.toml and the workspace directory before you destroy the instance, because destroying the instance deletes its disk.
An agent can rent the GPU and run this program on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the OpenManus install. A chat app whose Docker page runs docker compose up -d and whose Ollama page sets baseURL to http://localhost:11434/v1/ is How to run LibreChat agents with a self-hosted model on a cloud GPU. A chat app whose README runs docker compose up -d and whose environment page sets OLLAMA_PROXY_URL to http://127.0.0.1:11434 is How to run LobeChat agents with a self-hosted model on a cloud GPU. A terminal agent whose install page runs curl -fsSL https://www.openinterpreter.com/install | sh, and whose models page sets the Ollama base URL to http://localhost:11434/v1, is How to run Open Interpreter with a self-hosted model on a cloud GPU. A workspace app whose Docker quickstart publishes port 3001, and whose Ollama page sets the URL to http://127.0.0.1:11434, is How to run AnythingLLM agents with a self-hosted model on a cloud GPU. A repository agent whose local-models section sets api_base to http://localhost:11434 is How to run SWE-agent with a self-hosted model on a cloud GPU. A workflow editor whose Docker install publishes port 5678, and whose Ollama credentials default the Base URL to http://localhost:11434, is How to run n8n AI agents with a self-hosted model on a cloud GPU. A CLI whose providers page sets OLLAMA_HOST to http://localhost:11434 is How to run the goose agent with a self-hosted model on a cloud GPU. A coding agent whose providers page sets package to the OpenAI-compatible provider is How to use OpenCode with a self-hosted model on a cloud GPU. A canvas whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU. A CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A browser chat whose serve command listens on port 8080 is How to run Open WebUI on a cloud GPU. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose OllamaChatGenerator sets url to http://localhost:11434 is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b is How to run Agno agents with a self-hosted model on a cloud GPU. A TypeScript agent whose local-models section sets url on the model object is How to run Mastra agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.
The README license badge is MIT.
The commands below are copied from the upstream files named in each step, at the commit in Sources. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The README’s [llm] example and the active [llm] block in config/config.example.toml set model and base_url to values this page does not copy. A prototype timing sentence on the README is not copied. The sponsors section uses a comparison this page does not copy. The acknowledgement names repositories this page does not copy.
What the docs print about memory
The README, config/config.example.toml, and config/config.example-model-ollama.toml, read 2026-11-14, do not print a GPU GB figure. A commented sandbox block in config.example.toml sets memory_limit to "1g" and the same line’s comment says 512m. This page does not choose between those two strings and does not treat either as vram_gb. LLMSettings.max_tokens in app/config.py defaults to 4096. The field description says maximum number of tokens per request. The Ollama example comment says maximum number of tokens in the response. The active example in config.example.toml sets max_tokens to 8192. The Ollama example sets max_tokens to 4096. This page does not choose among those lines and does not treat them as vram_gb. max_input_tokens defaults to None. When it is unset, check_token_limit in app/llm.py returns True. The field description’s wording for that default is not copied. The model information table on unsloth/Llama-3.1-8B-Instruct prints parameter counts 8B, 70B, and 405B, and a context length of 128k. This page does not treat those parameter counts as vram_gb. A later section of that card names training hardware. This page does not copy that section and does not use it as an inference memory figure. This page does not mark a catalog row as enough.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T12:02:28.530Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Rows are sorted by vram_gb, then by name. Which listing you can start is on Deploy.
The catalog template list had no OpenManus template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as OpenManus.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| Ascend 910B | 64 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
Main of FoundationAgents/OpenManus on 2026-11-14 was commit 3309bf4e416fb1c74b008f3e86494439a31bad53, committed 2026-08-16T10:29:11Z. pushed_at was 2026-11-02T08:38:21Z. The GitHub license field was MIT. The README license badge is MIT. setup.py at that commit sets python_requires to >=3.12.
The README says method 2 is recommended. It prints:
curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/FoundationAgents/OpenManus.git
cd OpenManus
uv venv --python 3.12
source .venv/bin/activate
uv pip install -r requirements.txt
Method 1 prints conda create -n open_manus python=3.12, conda activate open_manus, the same clone, and pip install -r requirements.txt. This page does not merge the two installs.
The files this page read do not print an SSH command. The Ollama base_url below is http://localhost:11434/v1. The tunnel for a port you choose to publish is How to use SSH port forwarding on a cloud GPU.
base_url, model, and api_key
The README says to create config/config.toml by copying the example:
cp config/config.example.toml config/config.toml
The [llm] block the README then prints, and the active [llm] block in config.example.toml, set model and base_url to values this page does not copy. They are not the same block. Neither of those two blocks sets api_type. LLMSettings.api_type in app/config.py is a required field. The field description names providers this page does not copy.
The commented Ollama block in config.example.toml is:
# [llm] #OLLAMA:
# api_type = 'ollama'
# model = "llama3.2"
# base_url = "http://localhost:11434/v1"
# api_key = "ollama"
# max_tokens = 4096
# temperature = 0.0
config/config.example-model-ollama.toml prints the same keys without the comment marks:
[llm] #OLLAMA:
api_type = 'ollama'
model = "llama3.2"
base_url = "http://localhost:11434/v1"
api_key = "ollama"
max_tokens = 4096
temperature = 0.0
The README does not print a command that copies config.example-model-ollama.toml onto config/config.toml. This page does not add that command. llama3.2 is the model string that file prints. It is not a repository id.
app/llm.py ask_tool is documented as asking the model using functions and tools. It sends tools and tool_choice. It raises TokenLimitExceeded when max_input_tokens is set and the counted input is over that value. The error string it builds is Request may exceed input token limit. The example files this page read do not set max_input_tokens. When the field is unset, check_token_limit returns True. The README does not print a separate tool-calling requirement.
The OpenManus files this page read do not print hf download. The block below is added on this page. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. This page did not download the repository. The OpenManus files this page read do not print this repository id.
pip install -U "huggingface_hub"
hf download unsloth/Llama-3.1-8B-Instruct
On 2026-11-14, GET https://huggingface.co/api/models/unsloth/Llama-3.1-8B-Instruct reported gated false, cardData.license llama3.1, pipeline_tag text-generation, lastModified 2025-05-09T19:32:57.000Z, and sha 4699cc75b550f9c6f3173fb80f4703b62d946aa5. The card YAML also includes extra_gated_prompt. Those two fields are not the same fact. The card’s base_model field names a repository this page does not copy. This page did not run hf download.
How to run Ollama on a cloud GPU is the Ollama install. How to serve an OpenAI-compatible API with vLLM on a cloud GPU is the general server install. The Ollama example’s base_url is http://localhost:11434/v1. This page does not remove the /v1 suffix.
A run
The README prints:
python main.py
It says to input an idea via the terminal. main.py reads that line with input("Enter your prompt: "). The same file also accepts --prompt. The README does not print --prompt. This page does not add it to the README command.
app/config.py sets WORKSPACE_ROOT to workspace under the project root. workspace_root returns that path. The clone includes workspace/example.txt. The README does not name an output directory.
When it fails
CUDA out of memory. The OpenManus files this page read print no GPU GB figure. If the model server raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The OpenManus README and the config files this page read do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this program on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
Token limit. ask_tool raises TokenLimitExceeded only when max_input_tokens is set and the count is over it. The example configs this page read do not set that field. max_tokens is a different field. The Ollama example sets it to 4096. The field description and the toml comment do not use the same sentence. This page does not choose.
Authentication. ask_tool logs Authentication failed. Check API key. on an authentication error. The Ollama example sets api_key to ollama.
What to copy off, then destroy
Copy config/config.toml and the workspace directory, including any file the run wrote there, before you destroy the instance. The README does not name that directory. app/config.py does.
Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The billing page states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Amounts are not copied here.
Frequently asked questions
What memory figure do the OpenManus docs print?
The README, config.example.toml, and config.example-model-ollama.toml do not print a GPU GB figure. A commented sandbox block sets memory_limit to 1g and the same line’s comment says 512m. This page does not choose between those two strings and does not treat either as vram_gb. LLMSettings.max_tokens defaults to 4096 and the field description says maximum number of tokens per request. The Ollama example comment says maximum number of tokens in the response. This page does not treat either sentence as vram_gb. The unsloth/Llama-3.1-8B-Instruct model information table prints parameter counts 8B, 70B, and 405B, and context length 128k. This page does not treat those figures as vram_gb. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which install lines do the docs print?
The README prints two installs. Method 2, which it recommends, is curl -LsSf https://astral.sh/uv/install.sh | sh, then git clone https://github.com/FoundationAgents/OpenManus.git, cd OpenManus, uv venv --python 3.12, source .venv/bin/activate, and uv pip install -r requirements.txt. Method 1 uses conda create -n open_manus python=3.12 and pip install -r requirements.txt. setup.py requires Python >=3.12. Main on 2026-11-14 was commit 3309bf4e416fb1c74b008f3e86494439a31bad53. This page did not run the commands.
How do the docs point OpenManus at a server on the same machine?
The README says to copy config/config.example.toml to config/config.toml. That file’s active llm block and the README’s llm example set model and base_url to values this page does not copy. The Ollama example file sets model to llama3.2, base_url to http://localhost:11434/v1, and api_key to ollama, with max_tokens 4096. The commented Ollama block in config.example.toml sets the same three keys. The README run command is python main.py. ask_tool in app/llm.py sends tools and tool_choice. The example files this page read do not set max_input_tokens. When that field is unset, check_token_limit returns True. The files do not print an SSH command.
What license sentence does the README print?
The README license badge is MIT. The GitHub license field on 2026-11-14 was MIT. That license text is not a CloudGPU rate.
Sources (read 2026-11-14, Asia/Shanghai)
- FoundationAgents/OpenManus README at commit
3309bf4e416fb1c74b008f3e86494439a31bad53, committed 2026-08-16T10:29:11Z. GitHub license field MIT.pushed_at2026-11-02T08:38:21Z. License badge MIT. Method 2 install.cp config/config.example.toml config/config.toml.python main.py. The README[llm]example’s model and base URL are not copied. A prototype timing sentence, a sponsor comparison, and acknowledgement repository names are not copied setup.pyat that commit.python_requires>=3.12config/config.example.tomlat that commit. Active[llm]max_tokens8192. Model and base URL on that block are not copied. Commented Ollama block:api_typeollama,modelllama3.2,base_urlhttp://localhost:11434/v1,api_keyollama,max_tokens4096. Commentedmemory_limit"1g"and comment512mconfig/config.example-model-ollama.tomlat that commit. Same Ollama keys, without comment marks. The README does not print a command that copies this file ontoconfig/config.tomlapp/config.pyat that commit.LLMSettings.max_tokensdefault4096, description maximum number of tokens per request.max_input_tokensdefaultNone. The description’s wording for that default is not copied.api_typeis required.WORKSPACE_ROOTisworkspaceunder the project rootapp/llm.pyat that commit.ask_toolsendstoolsandtool_choice.TokenLimitExceededwhenmax_input_tokensis set and exceeded.check_token_limitreturnsTruewhen the field is unset. Authentication logAuthentication failed. Check API key.main.pyat that commit.input("Enter your prompt: "). Optional--prompt. The README does not print--prompt- Hugging Face model API
unsloth/Llama-3.1-8B-Instruct:gatedfalse,cardData.licensellama3.1,pipeline_tagtext-generation,lastModified2025-05-09T19:32:57.000Z,sha4699cc75b550f9c6f3173fb80f4703b62d946aa5. The card YAML includesextra_gated_prompt. The model information table prints 8B, 70B, 405B, and 128k.base_modelnames a repository this page does not copy.hf downloadis added on this page and was not run. The training-hardware section is not copied - Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (
generated_at2026-11-14T12:02:28.530Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no OpenManus template
Cards you can start are on the Deploy page. Amounts are not copied here.
Last updated: 2026-11-14 (Asia/Shanghai).