What can you run on an RTX 6000D (83GB)?

Short answer: With 83 GB on one card (CloudGPU’s listing in /gpu-prices.json, read 2026-11-04) you can run a 70B model at 4-bit with a long context (about 40 GB of weights, roughly 40 GB left for the KV cache), gpt-oss-120b (about 65 GB of MXFP4 weights), a 32B model unquantised at BF16 (about 65.5 GB) or a 70B model at FP8 (72.7 GB) with a short context only. On the image side, the full 16-bit FLUX.1-dev pipeline (about 31.4 GB) and the FP8 Wan 2.2 14B files (about 35 GB) fit in memory with room left. The RTX 6000D is an NVIDIA card released for a specific market; NVIDIA has not published a public product page or CUDA GPUs entry for it that this page can cite (checked 2026-11-04), so this page quotes only its memory, as CloudGPU lists it, and how to check the rest on your instance.

Check the card first

nvidia-smi --query-gpu=name,memory.total,compute_cap,power.limit --format=csv
  • memory.total is the number to plan with. The arithmetic below uses 83 GB; plan against what nvidia-smi reports.
  • compute_cap tells you which software builds you need. If it reports 12.0 (Blackwell), use a PyTorch build for CUDA 12.8 or newer, the same rule as the RTX 5090 and RTX PRO 6000; older builds fail with “no kernel image” (how to fix it, and how to install a specific PyTorch and CUDA version).
python -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.get_device_capability(0), torch.cuda.get_arch_list())"

The architecture list printed by the last command must include the card’s compute capability (for example sm_120).

What fits in 83 GB

Weights are Hugging Face or Ollama file sizes (read 2026-11-04). KV-cache figures use the formula in How context length adds to VRAM use. “Left over” subtracts the weights and an assumed 1.5 GB of runtime overhead, and is an estimate; runtimes reserve extra buffers, so plan below it.

Workload Weights or files Left over on 83 GB What that means
Llama 3.3 70B, 4-bit W4A16 (RedHatAI) 39.5 GB about 42 GB 320 KiB per token at FP16 KV: 32k context with room for parallel requests
Llama 3.3 70B, Q4_K_M (Ollama llama3.3:70b) about 43 GB about 38 GB as above
Qwen2.5 72B, AWQ (Qwen) 41.6 GB about 40 GB as above
Llama 3.3 70B, FP8 (RedHatAI) 72.7 GB about 9 GB short context only, about 8k-16k; tight
gpt-oss-120b, native MXFP4 (openai; Ollama gpt-oss:120b) about 65 GB about 16 GB fits; its sliding-window layers keep the KV cache smaller than the formula suggests
Qwen3 32B, BF16 (unquantised) about 65.5 GB about 16 GB 256 KiB per token: the native 32k context (about 8 GB) fits
Qwen3 32B, FP8 (Qwen) 34.3 GB about 47 GB long contexts or many parallel requests
FLUX.1-dev, full 16-bit pipeline about 31.4 GB loaded in BF16 about 50 GB room for LoRAs, ControlNets and larger batches
Wan 2.2 14B, FP8 in ComfyUI two 14.29 GB experts plus a 6.74 GB text encoder about 46 GB less offloading than on 24-48 GB cards; measure at your settings

Sources for image rows: the Hugging Face Diffusers blog (FLUX.1-dev BF16 about 31.4 GB) and the Comfy-Org/Wan-2.2 file listing, read 2026-11-04. Model sizes for the Ollama rows are from the Ollama library (read 2026-11-04). VRAM need depends on model, precision and workflow.

Where 83 GB sits: it holds more than an 80 GB A100 or A800 and less than a 96 GB card. The FP8 70B row is the one that is tight here and comfortable on 96 GB; see RTX PRO 6000 Max-Q vs RTX PRO 6000. If your model fits in 48 GB, see which 48GB GPU to rent.

Commands

Install vLLM as in how to serve an OpenAI-compatible API with vLLM. Llama models are gated: accept the licence on Hugging Face and export HF_TOKEN=... first.

# 70B at 4-bit with a 32k context
vllm serve RedHatAI/Llama-3.3-70B-Instruct-quantized.w4a16 --max-model-len 32768 --gpu-memory-utilization 0.92

# 70B at FP8: cap the context and use an 8-bit KV cache
vllm serve RedHatAI/Llama-3.3-70B-Instruct-FP8-dynamic --max-model-len 8192 --kv-cache-dtype fp8 --gpu-memory-utilization 0.95

# Qwen3 32B unquantised, native 32k context
vllm serve Qwen/Qwen3-32B --max-model-len 32768

# gpt-oss-120b or a 70B with Ollama
export OLLAMA_FLASH_ATTENTION=1
export OLLAMA_CONTEXT_LENGTH=32768
ollama serve &
ollama run gpt-oss:120b

When it loads the model, vLLM logs how many tokens of KV cache it could reserve; if the context you asked for does not fit, lower --max-model-len. Watch peak use while you test with nvidia-smi --query-gpu=memory.used,memory.total --format=csv -l 1.

Rates and downloads

At 2026-11-04 19:06 (UTC+8) the RTX 6000D was $1.29/hour. Billing is per minute, rounded up, with a 1-minute minimum: a 45-minute session costs $0.97, and $20 of balance covers about 16 hours. The instance is billed while weights download, and the large models here are 40-73 GB, so download once with acceleration on (how to speed up model downloads) and check that the files fit in your instance’s storage first. Destroying the instance stops billing and deletes everything on it. Current rates are on /pricing; /deploy shows which cards can be started now and lists the vCPU and memory under each card.

Frequently asked questions

How much VRAM does the RTX 6000D have?

CloudGPU lists it at 83 GB. Plan with the memory.total figure that nvidia-smi reports on your instance.

Can an RTX 6000D run a 70B model?

Yes. At 4-bit (39.5-43 GB of weights) it leaves about 40 GB for the KV cache, enough for a 32k context with parallel requests. At FP8 (72.7 GB) it fits with about 9 GB left, so only a short context of about 8k-16k.

Can it run gpt-oss-120b?

Yes. The MXFP4 weights are about 65 GB on Hugging Face, and Ollama’s gpt-oss:120b is 65 GB, which leaves about 16 GB on an 83 GB card.

Do I need a special PyTorch build?

Check compute_cap with nvidia-smi. If it reports 12.0 (Blackwell), use a PyTorch build for CUDA 12.8 or newer, as on the RTX 5090; older builds fail with “no kernel image”.

Is it faster than an A100 80G?

This page does not compare speed: NVIDIA has not published RTX 6000D figures that it can cite. It holds a little more than an 80 GB card. Time a short run of your own workload on each.

Sources (read 2026-11-04, Asia/Shanghai)

The Deploy page lists the cards you can start. Rates on this page are read from CloudGPU’s live price list when the page is rendered; /pricing has the current rate for every card.

Last updated: 2026-11-04 (Asia/Shanghai).