Which 48GB GPU should I rent for 70B models and long context: RTX 4090 48G or RTX 4090D 48G?

Short answer: Both cards have 48 GB, so the same models fit: a 70B model at 4-bit (about 40-43 GB of weights) with a short context of about 8k tokens, or a 32B model at 4-bit or FP8 with a long context. The difference is the chip: the RTX 4090 48G uses the RTX 4090 GPU (16,384 CUDA cores) and the RTX 4090D 48G uses the RTX 4090 D GPU (14,592 CUDA cores), both Ada Lovelace with compute capability 8.9 (NVIDIA RTX 4090 page, Overclock3D on the RTX 4090 D, read 2026-11-04). If you need 70B at FP8 or a 70B model with 32k context, 48 GB is not enough; see the 83 GB and 96 GB options below.

The two cards

RTX 4090 48G RTX 4090D 48G
GPU RTX 4090 (Ada Lovelace) RTX 4090 D (Ada Lovelace)
VRAM 48 GB (CloudGPU listing) 48 GB (CloudGPU listing)
CUDA cores (NVIDIA chip) 16,384 14,592
Memory bandwidth 1,008 GB/s (CloudGPU /pricing) not published by NVIDIA for the 48 GB version
FP16 TFLOPS 165 (CloudGPU /pricing) not listed on /pricing
Compute capability 8.9 8.9 (same Ada generation; check with nvidia-smi)

NVIDIA sells the RTX 4090 and RTX 4090 D with 24 GB; neither 48 GB version is on NVIDIA’s product pages, so the 48 GB figure is CloudGPU’s listing in /gpu-prices.json (read 2026-11-04). The RTX 4090 D was released for a specific market under export rules, with fewer CUDA cores than the RTX 4090. Check what your instance reports:

nvidia-smi --query-gpu=name,memory.total,compute_cap --format=csv

What fits in 48 GB

Weights are Hugging Face or Ollama file sizes (read 2026-11-04). KV cache per token is from each model’s config.json; see How context length adds to VRAM use for the formula. “Left for KV” subtracts the weights and an assumed 1.5 GB of runtime overhead, and is an estimate.

Model and precision Weights KV cache per token (FP16) Left for KV on 48 GB Practical context
Llama 3.3 70B, Q4_K_M (Ollama llama3.3:70b) about 43 GB 320 KiB about 3.5-5 GB about 8k; tight
Llama 3.3 70B, 4-bit W4A16 (RedHatAI) 39.5 GB 320 KiB about 7 GB about 8k-16k
Qwen2.5 72B, AWQ (Qwen) 41.6 GB 320 KiB about 5 GB about 8k; tight
Llama 3.3 70B, FP8 (RedHatAI) 72.7 GB does not fit
Qwen3 32B, 4-bit AWQ (Qwen) 19.3 GB 256 KiB about 27 GB full native 32k, or several parallel requests
Qwen3 32B, FP8 (Qwen) 34.3 GB 256 KiB about 12 GB 16k-32k
Qwen3 30B-A3B Instruct 2507, FP8 (Qwen) 31.2 GB 96 KiB (48 layers, 4 KV heads, head_dim 128) about 15 GB long contexts; 64k and up by the formula

For 70B models, 48 GB is enough for a 4-bit model with a short context, which suits one user chatting. For long-context inference (long documents, RAG with large chunks, many parallel requests), a 32B model or a mixture-of-experts model with a small KV cache uses 48 GB better than a 70B. An 8-bit KV cache (q8_0 in Ollama, fp8 in vLLM) holds about twice the context in the same memory.

Commands

Ollama, 70B at 4-bit with a capped context:

export OLLAMA_FLASH_ATTENTION=1
export OLLAMA_KV_CACHE_TYPE=q8_0
export OLLAMA_CONTEXT_LENGTH=8192
export OLLAMA_NUM_PARALLEL=1           # each parallel slot needs its own KV cache
ollama serve &
ollama run llama3.3:70b

vLLM, 70B at 4-bit (install as in how to serve an OpenAI-compatible API with vLLM). Llama models are gated: accept the licence on Hugging Face and export HF_TOKEN=... first.

vllm serve RedHatAI/Llama-3.3-70B-Instruct-quantized.w4a16 \
  --max-model-len 8192 --kv-cache-dtype fp8 --gpu-memory-utilization 0.95

vLLM, long context with a 32B model at FP8 (Ada GPUs run FP8 W8A8 in vLLM, which needs compute capability 8.9 or newer per the vLLM FP8 docs):

vllm serve Qwen/Qwen3-32B-FP8 --max-model-len 32768 --kv-cache-dtype fp8

When it loads the model, vLLM logs how much memory it reserved for the KV cache and how many tokens that holds; if it reports that the context does not fit, lower --max-model-len. Watch peak use with nvidia-smi --query-gpu=memory.used,memory.total --format=csv -l 1.

RTX 4090 48G or RTX 4090D 48G?

  1. Same memory, so the same models fit. Pick by speed per dollar on your workload, not by what fits.
  2. Prompt processing and batched serving lean on compute, where the RTX 4090’s larger CUDA core count matters. Single-user token generation leans on memory bandwidth. NVIDIA does not publish the 48 GB versions’ memory specs, so measure both rather than assume.
  3. Test both with the same command. For llama.cpp, llama-bench -m <model>.gguf -p 2048 -n 256 -ngl 99 gives prompt (pp) and generation (tg) tokens per second (how to build llama.cpp with CUDA).

At 2026-11-04 19:06 (UTC+8): RTX 4090 48G $0.69/hour, RTX 4090D 48G $0.59/hour. A 30-minute test costs $0.345 on the RTX 4090 48G and $0.295 on the RTX 4090D 48G. Billing is per minute, rounded up, with a 1-minute minimum, and the instance is billed while a model downloads; a 43 GB download is part of the cost of a 70B test. Current rates are on /pricing.

When 48 GB is not enough

On /deploy, each card shows its vCPU and memory, and /deploy is where to check which cards can be started now.

Frequently asked questions

Can a 48GB GPU run a 70B model?

Yes at 4-bit: Llama 3.3 70B at Q4_K_M is about 43 GB and the W4A16 build is 39.5 GB. That leaves about 3.5-7 GB for the KV cache, which is about 8k tokens of context in practice. FP8 70B (72.7 GB) does not fit.

Is the RTX 4090D 48G slower than the RTX 4090 48G?

Its GPU has 14,592 CUDA cores against 16,384 on the RTX 4090, so compute-heavy work such as prompt processing and batched serving can be slower. Both have 48 GB. Time the same benchmark on both and compare work done per dollar.

Which 48GB card is better for long context?

Both have the same 48 GB, so they hold the same context for a given model. For long context, use a model with a small KV cache (Qwen3 32B at 4-bit leaves about 27 GB for the cache; Qwen3 30B-A3B uses 96 KiB per token) and an 8-bit KV cache.

Are the 48GB cards official NVIDIA products?

NVIDIA’s product pages list the RTX 4090 and RTX 4090 D with 24 GB. The 48 GB versions use the same GPUs with more memory; their memory figures are the ones CloudGPU publishes. Check nvidia-smi on your instance.

Do I need special software for these cards?

No. Both are Ada Lovelace (compute capability 8.9), supported by current PyTorch, vLLM, llama.cpp and Ollama builds.

Sources (read 2026-11-04, Asia/Shanghai)

The Deploy page lists the cards you can start. Rates on this page are read from CloudGPU’s live price list when the page is rendered; /pricing has the current rate for every card.

Last updated: 2026-11-04 (Asia/Shanghai).