How to serve Qwen2.5-VL with vLLM on a cloud GPU

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install the vLLM line the Qwen2.5-VL README at commit 6e98a0a prints, run that file’s vllm serve command for Qwen/Qwen2.5-VL-7B-Instruct with the limit flag rewritten as JSON, and send the image chat request that file prints. Copy anything you saved on the instance before you destroy it, because destroying the instance deletes its disk and stops the server.

A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. The Cursor fields on How do I use CloudGPU as a custom base URL in Cursor? are for that hosted URL, not for the localhost URL below. A PDF pipeline whose model cards set base_model to Qwen/Qwen2.5-VL-7B-Instruct is How to convert PDFs to text with olmOCR on a cloud GPU. A different PDF converter, whose README states 8 GB+ VRAM on two standard rows, is How to convert PDFs to Markdown with MinerU on a cloud GPU. How published weight sizes are read: How much VRAM do I need to run an LLM?. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. The tunnel is How to use SSH port forwarding on a cloud GPU. A prompt model whose cards print 0.23B and 0.77B, trust_remote_code=True, and no GPU GB figure is How to run Florence-2 on a cloud GPU.

https://github.com/QwenLM/Qwen2.5-VL returned HTTP 301 to https://github.com/QwenLM/Qwen3-VL on 2026-11-13. The README at commit 6e98a0a62bce167c5802ae6f5f95fcd97d2634cf (committed 2025-05-13T02:22:59Z) is titled Qwen2.5-VL. The README at current HEAD 96588727e44c78b25ba03ea03b8e12f7e64fd0da (committed 2026-01-30T04:47:30Z) is titled Qwen3-VL. This page cites each file for the sentence it prints.

The 7B and 32B card front matter says license: apache-2.0. The LICENSE resolve URLs for those two repos returned HTTP 404 on 2026-11-13. The 3B LICENSE file is the Qwen RESEARCH LICENSE AGREEMENT, release date September 19, 2024. Its grant says “FOR NON-COMMERCIAL PURPOSES ONLY”, and the next sentence says that if you are commercially using the Materials, you shall request a license from us. Its definition says “We” shall mean Alibaba Cloud. The 72B LICENSE file is the Qwen LICENSE AGREEMENT, release date September 19, 2024. Section 4 says: “If you are commercially using the Materials, and your product or service has more than 100 million monthly active users, you shall request a license from us.” That user-count threshold is the agreement’s threshold, not a CloudGPU price. Read the file that applies to the checkpoint you download.

The commands below are copied from the upstream files named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked.

Which model names the README lists

The README at commit 6e98a0a has a performance table whose Qwen columns are Qwen2.5-VL-3B, Qwen2.5-VL-7B, Qwen2.5-VL-32B, and Qwen2.5-VL-72B, with links to Qwen/Qwen2.5-VL-3B-Instruct, Qwen/Qwen2.5-VL-7B-Instruct, Qwen/Qwen2.5-VL-32B-Instruct, and Qwen/Qwen2.5-VL-72B-Instruct. That table has other columns. This page does not copy them. The 2025.03.25 news line says the Qwen2.5-VL-32B was released. The 2025.02.20 news line names three AWQ repos: Qwen/Qwen2.5-VL-3B-Instruct-AWQ, Qwen/Qwen2.5-VL-7B-Instruct-AWQ, and Qwen/Qwen2.5-VL-72B-Instruct-AWQ. It does not name a 32B AWQ id. This page does not add one.

The model API on 2026-11-13 reported gated false for each of those seven ids. pipeline_tag was image-text-to-text. cardData.license was apache-2.0 for the 7B, the 32B, and the 7B-AWQ cards. The 3B and 3B-AWQ cards had license_name qwen-research. The 72B and 72B-AWQ cards had license_name qwen and license other. lastModified was 2025-04-06T16:23:42.000Z, 2025-04-06T16:23:01.000Z, 2025-04-14T10:55:03.000Z, 2025-06-06T02:48:50.000Z, 2025-04-06T16:24:35.000Z, 2025-04-06T16:25:10.000Z, and 2025-03-07T10:21:14.000Z, in the id order above. sha for the 7B Instruct card was cc594898137f460bfe9f0759e9844b3ce807cfb5.

What the README says about memory

The README at commit 6e98a0a prints this table under “Minimum VRAM requirements”. The note says the table presents the theoretical minimum video memory requirements for inference with transformers, and that in practice the actual memory usage is typically at least 1.2 times higher. This page does not multiply that note into a new GB figure. The current Qwen3-VL README wraps the same table in an HTML comment. This page does not treat that commented copy as an active section. The 32B name is not a column in the table.

Precision	Qwen2.5-VL-3B	Qwen2.5-VL-7B	Qwen2.5-VL-72B
FP32	11.5 GB	26.34 GB	266.21 GB
BF16	5.75 GB	13.17 GB	133.11 GB
INT8	2.87 GB	6.59 GB	66.5 GB
INT4	1.44 GB	3.29 GB	33.28 GB

The vLLM command in that README passes --dtype bfloat16 and the model id Qwen/Qwen2.5-VL-7B-Instruct. It does not print a GPU GB figure of its own. The vLLM Qwen2.5-VL recipe (read 2026-11-13) says BF16 is the precision it uses for the 72B and 7B server commands. It does not print a GB figure for those commands. One tip says tensor parallel should be larger than 2 for A100-80GB devices to avoid OOM. Another tip says vLLM conservatively uses 90% of GPU memory, and that you can set --gpu-memory-utilization=0.95. The 72B tip also says the multimodal encoder is very small compared to the language decoder, “ViT 675M v.s. LM 72B in Qwen2.5-VL-72B”. This page did not remeasure any of those figures.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-13T23:11:30.761Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-13T23:11:29.742Z) listed the same names and the same vram_gb values, plus Ascend 910B at 64, which is not in gpu-prices.json and is not listed here. The catalog response also includes an availability field. This page does not copy it. Every prices row has gpu_count 1. Every catalog row has min_gpu_count 1. Catalog rows whose vram_gb is greater than or equal to 13.17 are the values that match the 7B BF16 cell. The only row whose vram_gb is greater than or equal to 133.11 is H20-3e 141G, which matches the 72B BF16 cell. The FP32 72B cell is 266.21 GB, and every vram_gb in the catalog table below is lower than that number. The recipe headings say 4 GPUs. Every catalog row’s min_gpu_count is 1. This page does not claim a listing with 4 GPUs, and it does not mark a row as enough. Which listing you can start is on Deploy.

The catalog template list had no Qwen2.5-VL template. It did list Qwen 2.5 7B, Qwen 2.5 14B, and Qwen 2.5 32B. Those names are not the ids above.

Catalog name	vram_gb
RTX 2080	8
RTX 3080	10
RTX 2080 Ti	11
RTX 3080 Ti	12
V100 16G	16
A30	24
RTX 3090	24
RTX 4090	24
RTX 4090D	24
RTX 4080 SUPER 32G	32
RTX 5090	32
RTX 5090D	32
V100 32G	32
A100 40G	40
A800 40G	40
A40	48
RTX 4090 48G	48
RTX 4090D 48G	48
RTX A6000	48
NVIDIA RTX PRO 5000 72G	72
RTX PRO 5000 72G	72
A100 80G	80
A800 80G	80
H800	80
RTX 6000D	83
H20 96G	96
RTX PRO 6000 96G	96
RTX PRO 6000 Max-Q	96
H20-3e 141G	141

Context and pixel limits the files print

The 7B card and the README at commit 6e98a0a both say the default range for the number of visual tokens per image is 4-16384. Both print min_pixels = 256*28*28 and max_pixels = 1280*28*28 as an example. The 7B card says resized_height and resized_width are rounded to the nearest multiple of 28. The README at that commit says, for video, to set total_pixels below 24576 * 28 * 28. The current Qwen3-VL README says image_patch_size is 14 for Qwen2.5-VL and 16 for Qwen3-VL, and that exact dimensions are rounded to a multiple of 28 for Qwen2.5-VL and 32 for Qwen3-VL.

The 7B card says the current config.json is set for context length up to 32,768 tokens. Its YaRN snippet sets original_max_position_embeddings to 32768 and factor to 4. It also says max_position_embeddings can be modified to a larger value, such as 64k. The config.json files fetched on 2026-11-13 for Qwen/Qwen2.5-VL-3B-Instruct, Qwen/Qwen2.5-VL-7B-Instruct, Qwen/Qwen2.5-VL-32B-Instruct, and Qwen/Qwen2.5-VL-72B-Instruct each had max_position_embeddings 128000. The vLLM recipe says that by default the model’s context length is 128K, and that --max-model-len=65536 is usually what it suggests. This page does not choose which number the server uses. The recipe also prints benchmark commands and expected outputs. This page does not copy those outputs.

Serve commands

On Deploy, choose Raw GPU. The README at commit 6e98a0a says you need vllm>0.7.2 to enable Qwen2.5-VL support. Its deployment install, copied:

pip install git+https://github.com/huggingface/transformers@f3f6c86582611976e72be054675e2bf0abb5f775
pip install accelerate
pip install qwen-vl-utils
pip install 'vllm>0.7.2'

The same file’s chat quickstart, a different install, prints pip install transformers==4.51.3 accelerate. The vLLM recipe’s NVIDIA install, read 2026-11-13, is uv pip install -U vllm --torch-backend auto after uv venv. This page does not merge those three install blocks.

The README’s serve command, copied. Its install line is pip install 'vllm>0.7.2', which does not pin an upper version.

vllm serve Qwen/Qwen2.5-VL-7B-Instruct --port 8000 --host 0.0.0.0 --dtype bfloat16 --limit-mm-per-prompt image=5,video=5

The vLLM serve CLI says --limit-mm-per-prompt should be a valid JSON string or JSON keys passed individually (read 2026-11-13). vLLM issue 39687 reports argument --limit-mm-per-prompt: Value image=4,audio=1 cannot be converted. A comment on that issue says the parser accepts a JSON string and dotted keys and rejects the comma form. This page did not re-run that command. The block below is added on this page. It keeps the README’s model, port, host, dtype, and the counts 5 and 5, and writes the limit as a JSON string.

# Added on this page. The README prints --limit-mm-per-prompt image=5,video=5.
vllm serve Qwen/Qwen2.5-VL-7B-Instruct --port 8000 --host 0.0.0.0 --dtype bfloat16 --limit-mm-per-prompt '{"image":5,"video":5}'

The recipe’s 7B command is under a heading that says 4xA100 and also names other accelerators this page does not copy. The command text is:

export CUDA_VISIBLE_DEVICES=0,1,2,3
vllm serve Qwen/Qwen2.5-VL-7B-Instruct  \
  --host 0.0.0.0 \
  --port 8000 \
  --data-parallel-size 4 \
  --limit-mm-per-prompt '{"image":2,"video":0}'

The recipe’s 72B command uses the same host, port, and --limit-mm-per-prompt value, with --tensor-parallel-size 4 and --mm-encoder-tp-mode data, and the model id Qwen/Qwen2.5-VL-72B-Instruct. This page did not run these servers.

Image request

The README at commit 6e98a0a posts to http://localhost:8000/v1/chat/completions. The message content includes a part whose type is image_url. Copied:

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "Qwen/Qwen2.5-VL-7B-Instruct",
    "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "https://modelscope.oss-cn-beijing.aliyuncs.com/resource/qwen.png"}},
        {"type": "text", "text": "What is the text in the illustrate?"}
    ]}
    ]
    }'

The Python client in the same section sets base_url to http://localhost:8000/v1 and api_key to EMPTY, and model to Qwen/Qwen2.5-VL-7B-Instruct. A second example puts a base64 image in image_url.url as data:image;base64,{encoded_image_text}. The serve command does not pass --served-model-name. The vLLM serve CLI says that if --served-model-name is not specified, the model name will be the same as the --model argument (read 2026-11-13). The command’s --model value is the model string the curl sends. The serve command passes --host 0.0.0.0 and --port 8000. How to use SSH port forwarding on a cloud GPU is how a program on your computer reaches a port on the instance. This page does not add an ssh command. This page did not send the request.

The README at commit 6e98a0a also prints a web demo. Its commands are pip install -r requirements_web_demo.txt, then python web_demo_mm.py --flash-attn2 or python web_demo_mm.py. The line it says you will see is Running on local: http://127.0.0.1:7860/. Those commands do not pass a checkpoint path. The current Qwen3-VL README’s web demo is a different command: python web_demo_mm.py -c /your/path/to/qwen3vl/weight, and it prints the same 7860 line. This page did not start either demo.

What the current README says about Qwen3-VL

The current README’s news lines name Qwen3-VL-2B, Qwen3-VL-4B, Qwen3-VL-8B, Qwen3-VL-32B, Qwen3-VL-30B-A3B, and Qwen3-VL-235B-A22B, each with Instruct and Thinking links, plus FP8 weights in the collection it links. Its key-features line says native 256K context, expandable to 1M. It says the current config.json is set for context length up to 256K tokens. It says to install vllm>=0.11.0 to enable Qwen3-VL support, and it prints uv pip install -U vllm. The shell comment above its FP8 serve command names a GPU. This page does not copy that comment. The command under it is:

vllm serve Qwen/Qwen3-VL-235B-A22B-Instruct-FP8 \
  --tensor-parallel-size 8 \
  --mm-encoder-tp-mode data \
  --enable-expert-parallel \
  --async-scheduling \
  --media-io-kwargs '{"video": {"num_frames": -1}}' \
  --host 0.0.0.0 \
  --port 22002

The image client in that section sets base_url to http://127.0.0.1:22002/v1 and api_key to EMPTY, and a content part whose type is image_url. The YaRN serve line is vllm serve Qwen/Qwen3-VL-235B-A22B-Instruct with --max-model-len 1000000 and a --rope-scaling JSON. The note says to set factor to 2 or 3, not 4, for 1M context from a 256K length. The command passes --tensor-parallel-size 8. Every catalog row’s min_gpu_count is 1. This page does not claim a listing with 8 GPUs. The 2B and 235B-FP8 cards this page checked had cardData.license apache-2.0 and gated false. The repository LICENSE file is the Apache License, Version 2.0. This page did not run the Qwen3-VL server.

When it fails

CUDA out of memory. The transformers table prints the GB cells above, and its note says actual usage is typically at least 1.2 times higher. The vLLM commands print no GB figure. The recipe’s A100-80GB tip says tensor parallel should be larger than 2 to avoid OOM. If the process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The README at commit 6e98a0a pins vllm>0.7.2 and a transformers git commit. The recipe’s NVIDIA install does not pin a PyTorch build. Neither source names compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this server on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

What to copy off, then destroy

The serve commands do not name an output directory. Responses are what the client prints. Copy any file you wrote on the instance before you destroy it. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed. The server stops with the instance.

The pricing page FAQ states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Prices are not copied here.

Frequently asked questions

How much VRAM does the Qwen2.5-VL README print?

The README at commit 6e98a0a prints a theoretical minimum table for transformers. The BF16 cells are 5.75 GB, 13.17 GB, and 133.11 GB for Qwen2.5-VL-3B, Qwen2.5-VL-7B, and Qwen2.5-VL-72B. The note says actual memory usage is typically at least 1.2 times higher. The table has no 32B column. The vLLM command in that README prints no GPU GB figure. The current Qwen3-VL README wraps that table in an HTML comment, so this page does not treat that copy as an active section. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Which serve command and image request does the README print?

The README at that commit says to install vllm>0.7.2 and prints vllm serve Qwen/Qwen2.5-VL-7B-Instruct --port 8000 --host 0.0.0.0 --dtype bfloat16 --limit-mm-per-prompt image=5,video=5. The current vLLM CLI says that option should be a valid JSON string or JSON keys passed individually. vLLM issue 39687 reports that image=4,audio=1 cannot be converted, and a comment on that issue says the parser accepts a JSON string and dotted keys and rejects the comma form. The command added on this page keeps the counts 5 and 5 and uses --limit-mm-per-prompt ‘{“image”:5,“video”:5}’. The curl example posts to http://localhost:8000/v1/chat/completions with model Qwen/Qwen2.5-VL-7B-Instruct and a content part whose type is image_url. The vLLM recipe read the same day prints a 7B command with --data-parallel-size 4 and --limit-mm-per-prompt set to a JSON object with image 2 and video 0, and a 72B command with --tensor-parallel-size 4. This page did not run them.

What context and pixel limits do the files print?

The 7B card says the default range for visual tokens per image is 4-16384, and that config.json is set for context length up to 32,768 tokens. Its YaRN snippet sets original_max_position_embeddings to 32768. The config.json files for the 3B, 7B, 32B, and 72B Instruct repos fetched on 2026-11-13 each had max_position_embeddings 128000. The vLLM recipe says the default context length is 128K and names --max-model-len=65536. This page does not choose which number the server uses.

What license do the cards print?

The 7B and 32B card front matter says license: apache-2.0, and those two LICENSE resolve URLs returned HTTP 404 on 2026-11-13. The 3B LICENSE file is the Qwen RESEARCH LICENSE AGREEMENT, and its grant says FOR NON-COMMERCIAL PURPOSES ONLY. The 72B LICENSE file is the Qwen LICENSE AGREEMENT, and section 4 says that if you are commercially using the Materials, and your product or service has more than 100 million monthly active users, you shall request a license from us. That user-count threshold is the agreement’s threshold, not a CloudGPU price.

Sources (read 2026-11-13, Asia/Shanghai)

• Qwen2.5-VL README.md at commit 6e98a0a62bce167c5802ae6f5f95fcd97d2634cf, committed 2025-05-13T02:22:59Z (model columns, VRAM table, vllm>0.7.2, serve command, image_url curl, web demo on port 7860, visual-token range 4-16384). The GitHub path QwenLM/Qwen2.5-VL returned HTTP 301 to QwenLM/Qwen3-VL on 2026-11-13
• Qwen3-VL README.md at HEAD 96588727e44c78b25ba03ea03b8e12f7e64fd0da, committed 2026-01-30T04:47:30Z (the VRAM table is inside an HTML comment; Qwen3-VL size names; vllm>=0.11.0; FP8 serve command; image_url client on port 22002; YaRN --max-model-len 1000000). LICENSE is the Apache License, Version 2.0. https://github.com/QwenLM/Qwen3-VL/blob/main/README.md
• Hugging Face model card Qwen/Qwen2.5-VL-7B-Instruct (visual tokens 4-16384, context sentence 32,768, YaRN 32768, license: apache-2.0): read 2026-11-13. config.json max_position_embeddings 128000 for the 3B, 7B, 32B, and 72B Instruct repos. The 7B and 32B LICENSE resolve URLs returned HTTP 404
• LICENSE files for Qwen/Qwen2.5-VL-3B-Instruct (Qwen RESEARCH LICENSE AGREEMENT, non-commercial grant) and Qwen/Qwen2.5-VL-72B-Instruct (Qwen LICENSE AGREEMENT, section 4, 100 million monthly active users): read 2026-11-13
• vLLM Qwen2.5-VL recipe (7B and 72B serve commands, 128K sentence, --max-model-len=65536, A100-80GB tensor-parallel tip, ViT 675M and LM 72B): https://docs.vllm.ai/projects/recipes/en/latest/Qwen/Qwen2.5-VL.html, read 2026-11-13
• vLLM serve CLI (--served-model-name; --limit-mm-per-prompt should be a valid JSON string or JSON keys passed individually): https://docs.vllm.ai/en/latest/cli/serve/, read 2026-11-13
• vLLM issue 39687 (image=4,audio=1 cannot be converted; a comment says the JSON string and dotted keys are accepted and the comma form is rejected): https://github.com/vllm-project/vllm/issues/39687, read 2026-11-13
• Public GPU memory: https://cloudgpu.app/gpu-prices.json (updated_at 2026-11-13T23:11:30.761Z) and https://cloudgpu.app/v1/public/gpu-catalog (generated_at 2026-11-13T23:11:29.742Z). GET /v1/public/compute-capabilities returned feature flags and no GPU list. The catalog had no Qwen2.5-VL template

---

Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.

Last updated: 2026-11-13 (Asia/Shanghai).