How to run OpenHands with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install OpenHands with the Docker command on the official local-LLM page, serve the open-weight model that page names with the vLLM or SGLang command it prints, and point OpenHands at that OpenAI-compatible endpoint. Copy anything you saved on the instance before you destroy it, because destroying the instance deletes its disk and stops the server.
An agent can rent the GPU and run this itself. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the OpenHands command. The model the local-LLM page names is also served by a different card command: the Qwen3.6-35B-A3B card. A coding-agent server whose card prints --tool-call-parser qwen3_coder is How to serve Qwen3-Coder for coding agents on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. The tunnel is How to use SSH port forwarding on a cloud GPU. A terminal client whose OpenAI-compatible page sets OPENAI_API_BASE and whose pages print no GPU GB figure is How to use Aider with a self-hosted model on a cloud GPU.
The OpenHands LICENSE file begins with “The MIT License (MIT)”. The copyright line is “Copyright © 2025 OpenHands contributors”. The GitHub license field on 2026-11-13 was MIT. The Qwen3.6-35B-A3B card front matter says license: apache-2.0, and the resolved LICENSE file begins with “Apache License, Version 2.0, January 2004”. That license text is not a CloudGPU price. Read both files before you redistribute the program or the weights.
The commands below are copied from the upstream files named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked.
What the local-LLM page recommends
The local-LLM page (fetched 2026-11-13) says to connect OpenHands to local LLM servers such as LM Studio, Ollama, vLLM, and SGLang. Its news line dated 2026/05/21 says it recommends Qwen3.6-35B-A3B as the first local model to try, and that it is an open-weight MoE model built for agentic coding. The hardware section says running Qwen3.6-35B-A3B requires a recent GPU with at least 24GB of VRAM for quantized variants, or multiple GPUs for full precision and larger context windows. The same section also names a Mac with Apple Silicon and 64GB of unified memory for quantized variants. This page does not copy that Mac line into a GPU row.
The quickstart on that page is LM Studio, a desktop app. Its load steps say to set Context Length to at least 22000 or 32768. This page does not repeat the desktop click path. The terminal commands for vLLM and SGLang are in the page’s alternative section, copied below.
A community-reported list on the same page names qwen2.5-coder-14b-instruct and qwopus3.5-27b-v3 Q8_0. The heading is “Community-reported working models”. This page does not treat that list as the news line’s recommendation.
The model card for Qwen/Qwen3.6-35B-A3B (read 2026-11-13) says type Causal Language Model with Vision Encoder, number of parameters 35B in total and 3B activated, and context length 262,144 natively and extensible up to 1,010,000 tokens. text_config.max_position_embeddings in config.json that day was 262144. num_experts was 256 and num_experts_per_tok was 8. The card’s overview says 8 routed plus 1 shared. The card front matter says pipeline_tag: image-text-to-text. The card this page read prints no GPU GB figure. Its OOM note says to consider reducing the context window, and it advises maintaining a context length of at least 128K tokens. The model API that day reported gated false, cardData.license apache-2.0, lastModified 2026-11-07T04:41:52.000Z, and sha 345d467b117d1fd5eda9b879a5988de9ae5f47a7.
The card’s standard vLLM and SGLang paragraphs say tensor parallel on 8 GPUs. The local-LLM page’s vLLM and SGLang examples say “with at least 2 GPUs” and pass --tensor-parallel-size 2 and --tp 2. This page does not pick one of those numbers.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-13T23:11:30.761Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-13T23:11:29.742Z) listed the same names and the same vram_gb values, plus Ascend 910B at 64, which is not in gpu-prices.json and is not listed here. The catalog response also includes an availability field. This page does not copy it. Every prices row has gpu_count 1. Every catalog row has min_gpu_count 1. Catalog rows whose vram_gb is 24 or greater are the values that match the number in “at least 24GB of VRAM for quantized variants”. That sentence is about quantized variants. The card’s serve commands and the local-LLM page’s vLLM command name Qwen/Qwen3.6-35B-A3B, and neither source prints a GPU GB figure for that id. The serve prose says 8 GPUs or at least 2 GPUs, and every catalog row’s min_gpu_count is 1. This page does not claim a listing with 2 GPUs or with 8 GPUs, and it does not mark a row as enough. Which listing you can start is on Deploy.
The catalog template list had no OpenHands template and no Qwen3.6 template.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install OpenHands
On Deploy, choose Raw GPU. The local-LLM page’s Docker command, copied. The page says to make sure Docker is installed and running, then run it.
docker run -it --rm --pull=always \
-e AGENT_SERVER_IMAGE_REPOSITORY=ghcr.io/openhands/agent-server \
-e AGENT_SERVER_IMAGE_TAG=1.26.0-python \
-e LOG_ALL_EVENTS=true \
-v /var/run/docker.sock:/var/run/docker.sock \
-v ~/.openhands:/.openhands \
-p 3000:3000 \
--add-host host.docker.internal:host-gateway \
--name openhands-app \
docker.openhands.dev/openhands/openhands:1.8
The log the page prints says Uvicorn running on http://0.0.0.0:3000 and tells you to visit http://localhost:3000. The command does not pass --gpus. It mounts /var/run/docker.sock and sets AGENT_SERVER_IMAGE_REPOSITORY and AGENT_SERVER_IMAGE_TAG. Those two variables and the socket mount are the runtime the command documents. This page did not start the container.
The same page’s development-mode heading says to use the instructions in Development.md and start OpenHands with make run. This page did not follow that file.
The quickstart lists Agent Canvas and a hosted product. This page follows the local-LLM page’s Docker command, not that hosted product.
Download the weights
The local-LLM page’s vLLM and SGLang section prints this download. Copied:
huggingface-cli download Qwen/Qwen3.6-35B-A3B --local-dir Qwen/Qwen3.6-35B-A3B
The block below is added on this page. It keeps the repository id and the local directory, and replaces huggingface-cli with hf. The Hub v1.0 migration doc says the deprecated huggingface-cli command has been removed and that hf (introduced in v0.34) replaces it. This page did not download the repository. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download.
# Added on this page. The local-LLM page calls huggingface-cli.
hf download Qwen/Qwen3.6-35B-A3B --local-dir Qwen/Qwen3.6-35B-A3B
Serve commands the two sources print
The local-LLM page says to install vLLM and SGLang by following their official documentation. It does not print a version pin. The model card says vllm>=0.19.0 is recommended and prints uv pip install vllm --torch-backend=auto. It says sglang>=0.5.10 is recommended and prints uv pip install sglang[all]. This page does not merge those install lines into the local-LLM page’s serve commands.
The local-LLM page’s vLLM example, copied. The heading says “with at least 2 GPUs”.
vllm serve Qwen/Qwen3.6-35B-A3B \
--host 0.0.0.0 --port 8000 \
--api-key mykey \
--tensor-parallel-size 2 \
--served-model-name Qwen3.6-35B-A3B \
--enable-prefix-caching
The local-LLM page’s SGLang example, copied. The heading says “with at least 2 GPUs”.
SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 python3 -m sglang.launch_server \
--model Qwen/Qwen3.6-35B-A3B \
--served-model-name Qwen3.6-35B-A3B \
--port 8000 \
--tp 2 --dp 1 \
--host 0.0.0.0 \
--api-key mykey --context-length 131072
The model card’s standard vLLM command, copied. The paragraph says maximum context length 262,144 tokens using tensor parallel on 8 GPUs, and an endpoint at http://localhost:8000/v1.
vllm serve Qwen/Qwen3.6-35B-A3B --port 8000 --tensor-parallel-size 8 --max-model-len 262144 --reasoning-parser qwen3
The card’s tool-use vLLM command adds --enable-auto-tool-choice --tool-call-parser qwen3_coder. The local-LLM page’s vLLM example does not pass those flags. This page does not add them to that example. The card also prints an MTP command and a command that passes --language-model-only. This page does not paste those two. The card’s standard SGLang command is:
python -m sglang.launch_server --model-path Qwen/Qwen3.6-35B-A3B --port 8000 --tp-size 8 --mem-fraction-static 0.8 --context-length 262144 --reasoning-parser qwen3
The card’s tool-use SGLang command adds --tool-call-parser qwen3_coder. Its long-context section prints commands with --max-model-len 1010000 and --context-length 1010000. This page does not paste the rope-override JSON those commands include. The local-LLM page also prints an Ollama command that sets OLLAMA_CONTEXT_LENGTH=32768 and says to set that variable to at least 22000, because the default 4096 is too small for the system prompt. It also prints an ArcticInference install and a second vLLM command with --speculative-config '{"method": "suffix"}'. This page does not copy that page’s speed sentence.
Point OpenHands at the endpoint
The local-LLM page’s alternative-backends settings, copied from that page’s list. It says to open Settings, the LLM tab, “see advanced settings”, and enable Advanced.
- Custom Model:
openai/Qwen3.6-35B-A3Bfor SGLang or vLLM. The LM Studio quickstart on the same page prints a different string,openai/qwen/qwen3.6-35b-a3b. This page does not choose which string the server returns. - Base URL:
http://host.docker.internal:<port>/v1. The page says to use port 8000 for SGLang and vLLM. - API Key: the same key passed when starting the server. The examples use
mykey.
The vLLM example above passes --served-model-name Qwen3.6-35B-A3B. The card’s vLLM command does not. The vLLM serve CLI says that if --served-model-name is not specified, the model name will be the same as the --model argument (read 2026-11-13). The card’s --model value is Qwen/Qwen3.6-35B-A3B. The local-LLM page’s Custom Model string for that case is openai/Qwen3.6-35B-A3B, which is the served name plus the prefix that page prints. This page did not save the settings.
The Docker command publishes port 3000. How to use SSH port forwarding on a cloud GPU is how a program on your computer reaches a port on the instance. This page does not add an ssh command. The local-LLM page does not document a Gradio launch. This page does not add one. The hosted gateway at https://cloudgpu.app/v1 is the other guide, not this server.
When it fails
CUDA out of memory. The local-LLM page prints 24GB for quantized variants and does not print a GB figure for Qwen/Qwen3.6-35B-A3B. The card prints no GPU GB figure. Its OOM note says to consider reducing the context window, and it advises at least 128K tokens. The card’s serve paragraphs say 8 GPUs. The local-LLM examples say at least 2 GPUs. If the process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The card recommends vllm>=0.19.0 and sglang>=0.5.10. It does not pin a PyTorch build and does not name compute capability 12.0. The Docker command does not pass --gpus. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this server on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
What to copy off, then destroy
The local-LLM page does not name an output directory. The Docker command mounts ~/.openhands. Responses are what the UI and the client print. Copy any file you wrote on the instance before you destroy it. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed. The server stops with the instance.
The pricing page FAQ states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Prices are not copied here.
Frequently asked questions
How much VRAM does the local-LLM page print?
The local-LLM page says running Qwen3.6-35B-A3B requires a recent GPU with at least 24GB of VRAM for quantized variants, or multiple GPUs for full precision. The model card this page read prints no GPU GB figure. Its standard vLLM and SGLang paragraphs say tensor parallel on 8 GPUs. The local-LLM page’s vLLM and SGLang examples say at least 2 GPUs and pass --tensor-parallel-size 2 and --tp 2. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which commands do the local-LLM page and the model card print?
The local-LLM page’s Docker command publishes port 3000 and sets AGENT_SERVER_IMAGE_TAG to 1.26.0-python. Its vLLM example is vllm serve Qwen/Qwen3.6-35B-A3B with --host 0.0.0.0 --port 8000 --api-key mykey --tensor-parallel-size 2 --served-model-name Qwen3.6-35B-A3B --enable-prefix-caching. Its SGLang example sets --port 8000 --tp 2 --dp 1 --context-length 131072. The model card recommends vllm>=0.19.0 and sglang>=0.5.10. Its standard vLLM command is vllm serve Qwen/Qwen3.6-35B-A3B --port 8000 --tensor-parallel-size 8 --max-model-len 262144 --reasoning-parser qwen3. Its tool-use command adds --enable-auto-tool-choice --tool-call-parser qwen3_coder. This page did not run them.
How does the UI reach the model server?
The local-LLM page’s log line says Uvicorn running on http://0.0.0.0:3000, and it says to visit http://localhost:3000. For SGLang or vLLM it sets Custom Model to openai/Qwen3.6-35B-A3B, Base URL to http://host.docker.internal:8000/v1, and API Key to the server key, which the examples set to mykey. The vLLM example passes --served-model-name Qwen3.6-35B-A3B. The model card’s vLLM command does not pass --served-model-name. The vLLM serve CLI says that if --served-model-name is not specified, the model name is the same as the --model argument. The SSH port-forwarding guide is how a program on your computer reaches a port on the instance. This page did not open a tunnel.
What license do the files print?
The OpenHands LICENSE file begins with The MIT License (MIT), and the copyright line is Copyright © 2025 OpenHands contributors. The GitHub license field on 2026-11-13 was MIT. The Qwen3.6-35B-A3B card front matter says license: apache-2.0, and the resolved LICENSE file begins with Apache License Version 2.0. That license text is not a CloudGPU price.
Sources (read 2026-11-13, Asia/Shanghai)
- OpenHands local-LLM page (Qwen3.6-35B-A3B recommendation, 24GB for quantized variants, context 22000 and 32768, Docker command,
huggingface-clidownload, vLLM and SGLang examples, Custom Model and Base URL, Ollama context note): https://docs.openhands.dev/openhands/usage/llms/local-llms, fetched 2026-11-13 - OpenHands quickstart (Agent Canvas and a hosted product): https://docs.openhands.dev/overview/quickstart, fetched 2026-11-13
- OpenHands/OpenHands
LICENSE(MIT, copyright line): repository HEAD46f20abafecbd03c422213bff4bc4499607b9b4f, committed 2026-11-13T21:29:19Z. GitHub license fieldMIT - Hugging Face model card
Qwen/Qwen3.6-35B-A3B(35B total and 3B activated, context 262,144 and 1,010,000,vllm>=0.19.0,sglang>=0.5.10, standard and tool-use commands, 8 GPUs, 128K OOM note,license: apache-2.0): read 2026-11-13. Model API:gatedfalse,cardData.licenseapache-2.0,pipeline_tagimage-text-to-text,lastModified2026-11-07T04:41:52.000Z,sha345d467b117d1fd5eda9b879a5988de9ae5f47a7.config.jsontext_config.max_position_embeddings262144. The resolvedLICENSEfile begins with Apache License Version 2.0 - vLLM serve CLI (
--served-model-name: if not specified, the model name will be the same as the--modelargument): https://docs.vllm.ai/en/latest/cli/serve/, read 2026-11-13 - Hub v1.0 migration doc (
huggingface-cliremoved,hfreplaces it): https://huggingface.co/docs/huggingface_hub/en/concepts/migration, read 2026-11-13 - Public GPU memory: https://cloudgpu.app/gpu-prices.json (
updated_at2026-11-13T23:11:30.761Z) and https://cloudgpu.app/v1/public/gpu-catalog (generated_at2026-11-13T23:11:29.742Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no OpenHands template and no Qwen3.6 template
Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.
Last updated: 2026-11-13 (Asia/Shanghai).