How to serve Qwen3-Coder for coding agents on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install the vLLM or SGLang version the Qwen3-Coder-Next model card prints, run that card’s serve command with --tool-call-parser qwen3_coder, and point an OpenAI-compatible client at the localhost URL the card prints, through an SSH tunnel. Copy anything you saved on the instance before you destroy it, because destroying the instance deletes its disk and stops the server.
An agent can rent the GPU and run this serve command itself. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Qwen3-Coder command. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. The Cursor fields on How do I use CloudGPU as a custom base URL in Cursor? are for that hosted URL, not for the localhost URL below. How published weight sizes are read: How much VRAM do I need to run an LLM?. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. The tunnel is How to use SSH port forwarding on a cloud GPU. A local agent UI whose Docker command publishes port 3000, and whose local-LLM page points at a vLLM or SGLang server, is How to run OpenHands with a self-hosted model on a cloud GPU.
The Qwen3-Coder-Next, Qwen3-Coder-30B-A3B-Instruct, and Qwen3-Coder-480B-A35B-Instruct card front matter this page read says license: apache-2.0. The GitHub repository root on 2026-11-13 had no LICENSE file, and https://huggingface.co/Qwen/Qwen3-Coder-Next/resolve/main/LICENSE returned HTTP 404. That license field is not a CloudGPU price.
The commands below are copied from the upstream files named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked.
Which model names the README lists
The GitHub README says Qwen3-Coder is available in multiple sizes and names Qwen3-Coder-480B-A35B-Instruct, Qwen3-Coder-30B-A3B-Instruct, and Qwen3-Coder-Next. Its table, length column 256k on every row, lists these download names:
| model name | type |
|---|---|
| Qwen3-Coder-Next | instruct |
| Qwen3-Coder-Next-Base | base |
| Qwen3-Coder-480B-A35B-Instruct | instruct |
| Qwen3-Coder-30B-A3B-Instruct | instruct |
| Qwen3-Coder-Next-FP8 | instruct |
| Qwen3-Coder-Next-GGUF | instruct |
| Qwen3-Coder-480B-A35B-Instruct-FP8 | instruct |
| Qwen3-Coder-30B-A3B-Instruct-FP8 | instruct |
The README’s key-features line says native support for 256K tokens, extendable up to 1M tokens using Yarn. The basic-information line says the context length of 256K tokens. An important note says function calling relies on the new tool parser in both SGLang and vLLM, and it links the Qwen3-Coder-Next model card. It also says the special tokens and their token ids were updated, and to use the new tokenizer. The quick-start heading says Qwen3-Coder are instruct models for chatting. This page does not copy the README’s comparison sentence or its list of product names.
The Next card’s overview says number of parameters 80B in total and 3B activated, and context length 262,144 natively. The 30B card says 30.5B in total and 3.3B activated, and context length 262,144 natively, and it also says extendable up to 1M tokens using Yarn. The 480B card says 480B in total and 35B activated, and context length 262,144 natively, and the same 1M Yarn line. The Next card’s overview does not print the 1M line. The 480B card’s transformers example sets model_name to Qwen/Qwen3-480B-A35B-Instruct. The table link and the tool-call example on that card use Qwen3-Coder-480B-A35B-Instruct. This page does not choose which string loads.
What the cards say about memory
The Next card, the Next-FP8 card, the 30B card, and the 480B card this page read do not print a GPU GB figure. The Next card says that for more details, including hardware requirements, refer to the blog, the GitHub repository, and the documentation. This page does not copy a GB figure from that blog, because the card itself does not print one. Use How much VRAM do I need to run an LLM? for the general method. The Next card’s OOM note says to consider reducing the context length to a shorter value, such as 32,768. The serve note says the default context length is 256K, and to consider 32768 if the server fails to start. Those are context lengths, not a GB figure.
The Next and Next-FP8 serve paragraphs say “using tensor parallel on 4 GPUs”. The commands under those sentences pass --tp-size 2 and --tensor-parallel-size 2. This page does not pick one of those numbers.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-13T22:12:44.591Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-13T22:12:44.965Z) listed the same names and the same vram_gb values, plus Ascend 910B at 64, which is not in gpu-prices.json and is not listed here. The catalog response also includes an availability field. This page does not copy it. Every prices row has gpu_count 1. Every catalog row has min_gpu_count 1. There is no upstream GPU GB cutoff to match these values against. The serve prose says 4 GPUs, and every catalog row’s min_gpu_count is 1. This page does not claim a listing with 4 GPUs, and it does not mark a row as enough. Which listing you can start is on Deploy.
The catalog template list had no Qwen3-Coder template. It did list Qwen 2.5 Coder 7B and Qwen 2.5 Coder 32B. Those names are not in the table above.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Serve commands the Next cards print
On Deploy, choose Raw GPU. The GitHub README does not print a vllm or sglang command. The Next card’s deployment section does. Its install lines and commands, copied. The paragraph above each command says the endpoint is at the localhost URL printed with that command, with maximum context length 256K tokens, using tensor parallel on 4 GPUs.
pip install 'sglang[all]>=v0.5.8'
python -m sglang.launch_server --model Qwen/Qwen3-Coder-Next --port 30000 --tp-size 2 --tool-call-parser qwen3_coder
The card says that command creates an API endpoint at http://localhost:30000/v1.
pip install 'vllm>=0.15.0'
vllm serve Qwen/Qwen3-Coder-Next --port 8000 --tensor-parallel-size 2 --enable-auto-tool-choice --tool-call-parser qwen3_coder
The card says that command creates an API endpoint at http://localhost:8000/v1. Neither command passes --host or --api-key. The note under each command says the default context length is 256K, and to consider reducing it to 32768 if the server fails to start.
The Next-FP8 card prints the same install lines, the same ports, the same parser flag, and the same “4 GPUs” sentence, with the model id changed. Its SGLang fence in the card this page fetched places the closing fence on the same line as the command. The command text is:
python -m sglang.launch_server --model Qwen/Qwen3-Coder-Next-FP8 --port 30000 --tp-size 2 --tool-call-parser qwen3_coder
vllm serve Qwen/Qwen3-Coder-Next-FP8 --port 8000 --tensor-parallel-size 2 --enable-auto-tool-choice --tool-call-parser qwen3_coder
The 30B and 480B cards, and their FP8 cards, do not print vllm serve or sglang.launch_server. The 30B-FP8 card says “You can use the Qwen3-30B-A3B-Instruct-FP8 model with serveral inference frameworks, including transformers, sglang, and vllm, as the original bfloat16 model.” The 480B-FP8 card says the same sentence with Qwen3-480B-A35B-Instruct-FP8. Those names are what the sentences print. This page does not invent a serve line for those ids. The Next card’s sampling line is temperature=1.0, top_p=0.95, top_k=40. The 30B card’s sampling line is temperature=0.7, top_p=0.8, top_k=20, repetition_penalty=1.05, and it says an output length of 65,536 tokens for most queries. This page did not run either server.
The model API for Qwen/Qwen3-Coder-Next on 2026-11-13 reported gated false, cardData.license apache-2.0, lastModified 2026-02-03T16:16:38.000Z, and sha a7fbcb5c0e12d62a448eaa0e260346bf5dcc0feb.
Point a client at the endpoint
The Next card’s client, copied. The comment in the card says “Use a custom endpoint compatible with OpenAI API”.
from openai import OpenAI
# Define LLM
client = OpenAI(
# Use a custom endpoint compatible with OpenAI API
base_url='http://localhost:8000/v1', # api_base
api_key="EMPTY"
)
The completion call in the same section sets model="Qwen3-Coder-Next", max_tokens=65536, and passes tools. The vLLM command above sets --model Qwen/Qwen3-Coder-Next and does not pass --served-model-name. The vLLM serve CLI says that if --served-model-name is not specified, the model name will be the same as the --model argument (read 2026-11-13). Added on this page: send Qwen/Qwen3-Coder-Next as the client model, or add --served-model-name Qwen3-Coder-Next to that vLLM command. The card does not print that flag. The Next-FP8 vLLM command’s --model is Qwen/Qwen3-Coder-Next-FP8, and the same CLI sentence applies. The 30B card writes import OpenAI rather than from openai import OpenAI, and its completion sets model="Qwen3-Coder-30B-A3B-Instruct". The 480B-FP8 card’s completion sets model="Qwen3-Coder-480B-A35B-Instruct", which is not the FP8 id. Those two cards do not print a vllm serve line. This page does not add one.
The serve commands bind the ports the card names and do not pass --host. How to use SSH port forwarding on a cloud GPU is how a program on your computer reaches a port on the instance. This page does not add an ssh command. After the tunnel, the card’s base_url is the one the client uses. The hosted gateway at https://cloudgpu.app/v1 is the other guide, not this server. This page did not open a tunnel and did not call the client.
The cards do not document a Gradio launch. This page does not add one.
When it fails
CUDA out of memory. The cards this page read print no GPU GB figure. The Next card’s OOM note says to reduce the context length to a shorter value such as 32,768. The serve note says to consider 32768 if the server fails to start. If the process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The Next card pins sglang>=v0.5.8 and vllm>=0.15.0. It does not pin a PyTorch build and does not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this server on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
What to copy off, then destroy
The cards do not name an output directory. Responses are what the client prints. Copy any file you wrote on the instance before you destroy it. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed. The server stops with the instance.
The pricing page FAQ states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Prices are not copied here.
Frequently asked questions
How much VRAM do the Qwen3-Coder cards print?
The Qwen3-Coder-Next and Qwen3-Coder-Next-FP8 cards do not print a GPU GB figure. Their serve paragraphs say tensor parallel on 4 GPUs, and the commands pass --tp-size 2 and --tensor-parallel-size 2. The 30B and 480B cards this page read also print no GPU GB figure. Use https://cloudgpu.app/guides/how-much-vram-do-i-need-to-run-an-llm for the general method. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which serve commands and context lengths do the cards print?
The GitHub README table lists eight model names, each with length 256k, and its key-features line says native support for 256K tokens, extendable up to 1M tokens using Yarn. The Next card overview says context length 262,144 natively. Its serve paragraph says maximum context length 256K tokens, and the note says the default is 256K and to consider 32768 if the server fails to start. The same card’s OOM note says a shorter value such as 32,768. The Next card prints python -m sglang.launch_server --model Qwen/Qwen3-Coder-Next --port 30000 --tp-size 2 --tool-call-parser qwen3_coder and vllm serve Qwen/Qwen3-Coder-Next --port 8000 --tensor-parallel-size 2 --enable-auto-tool-choice --tool-call-parser qwen3_coder. The Next-FP8 card prints the same flags with Qwen/Qwen3-Coder-Next-FP8. The 30B and 480B cards this page read do not print those commands. This page did not run them.
How does a client reach the server?
The Next card’s client sets base_url to http://localhost:8000/v1 and api_key to EMPTY. Its completion sets model to Qwen3-Coder-Next. The vLLM command sets --model to Qwen/Qwen3-Coder-Next and does not pass --served-model-name. The vLLM serve CLI says that if --served-model-name is not specified, the model name is the same as the --model argument. Send Qwen/Qwen3-Coder-Next, or add --served-model-name Qwen3-Coder-Next. That flag is not on the card. The Next-FP8 vLLM command’s --model is Qwen/Qwen3-Coder-Next-FP8, and the same CLI sentence applies. The serve commands do not pass --host or --api-key. SGLang’s printed URL is http://localhost:30000/v1. The SSH port-forwarding guide is how a program on your computer reaches a port on the instance. This page did not open a tunnel.
What license do the cards print?
The Next, 30B, and 480B card front matter this page read says license: apache-2.0. The GitHub repository root on 2026-11-13 had no LICENSE file, and the Next card’s license_link returned HTTP 404. That license field is not a CloudGPU price.
Sources (read 2026-11-13, Asia/Shanghai)
- QwenLM/Qwen3-Coder
README.md(the three size names, the eight-row table at length 256k, native 256K and 1M using Yarn, the SGLang and vLLM tool-parser note linking the Next card): repository HEAD33bc6aabd7791ad7b32f7e92104f11f2359ba890, committed 2026-03-24T08:22:10Z. The repository root listing had noLICENSEfile. https://github.com/QwenLM/Qwen3-Coder/blob/main/README.md - Hugging Face model card
Qwen/Qwen3-Coder-Next(80B total and 3B activated, context 262,144, SGLang and vLLM commands, 256K and 4 GPUs prose,32768note,32,768OOM note, clientbase_url, completionmodel="Qwen3-Coder-Next",license: apache-2.0): read 2026-11-13. Model API:gatedfalse,cardData.licenseapache-2.0,lastModified2026-02-03T16:16:38.000Z,shaa7fbcb5c0e12d62a448eaa0e260346bf5dcc0feb. TheLICENSEresolve URL returned HTTP 404 - vLLM serve CLI (
--served-model-name: if not specified, the model name will be the same as the--modelargument): https://docs.vllm.ai/en/latest/cli/serve/, read 2026-11-13 - Hugging Face model card
Qwen/Qwen3-Coder-Next-FP8(the same serve flags with the FP8 model id, and the broken SGLang fence): read 2026-11-13. Model API:gatedfalse,cardData.licenseapache-2.0,lastModified2026-02-03T14:13:28.000Z - Hugging Face model cards
Qwen/Qwen3-Coder-30B-A3B-Instruct(30.5B total and 3.3B activated, 262,144, 1M Yarn, no serve command) andQwen/Qwen3-Coder-480B-A35B-Instruct(480B total and 35B activated, 262,144, 1M Yarn, transformersmodel_nameQwen/Qwen3-480B-A35B-Instruct, no serve command): read 2026-11-13. Both reportedcardData.licenseapache-2.0andgatedfalse - Hugging Face model cards
Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8andQwen/Qwen3-Coder-480B-A35B-Instruct-FP8(the “serveral” sentence, novllm serveline): read 2026-11-13 - Public GPU memory: https://cloudgpu.app/gpu-prices.json (
updated_at2026-11-13T22:12:44.591Z) and https://cloudgpu.app/v1/public/gpu-catalog (generated_at2026-11-13T22:12:44.965Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no Qwen3-Coder template
Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.
Last updated: 2026-11-13 (Asia/Shanghai).