RTX 4090 vs RTX 5090 vs A100: which GPU to rent for your workload

Short answer: rent by VRAM first, bandwidth second, price third. A 24 GB RTX 4090 covers 7B–14B inference, SDXL/FLUX at FP8 and QLoRA up to about 14B. A 32 GB RTX 5090 is the step up for 32B models at 4-bit with room for context and for FLUX at FP16. A 48 GB RTX 4090 48G covers 70B at 4-bit with short context and 32B QLoRA. An 80 GB A100 or A800 is for 70B with long context, gpt-oss-120b and 70B QLoRA. All are billed per minute on CloudGPU and start in about 2 minutes.

Spec and price table

Architecture, VRAM, FP16 TFLOPS and memory bandwidth are the figures published on our /pricing page. Prices are list prices from /gpu-prices.json on 2026-10-23 23:58 Beijing time. Stock changes through the day; the Deploy page shows what you can start now.

Card Arch VRAM FP16 TFLOPS Memory bandwidth List $/h $ per GB-hour
RTX 3090 24G Ampere 24 GB 71 936 GB/s 0.21 0.0088
RTX 4090 24G Ada 24 GB 165 1,008 GB/s 0.33 0.0138
RTX 5090 32G Blackwell 32 GB 209 1,792 GB/s 0.59 0.0184
RTX 4090 48G Ada 48 GB 165 1,008 GB/s 0.65 0.0135
A800 80G Ampere 80 GB 312 2,039 GB/s 1.09 0.0136
A100 80G Ampere 80 GB 312 1,935 GB/s 1.49 0.0186
H20 96G Hopper 96 GB 148 4,000 GB/s 1.29 0.0134

“$ per GB-hour” is simply the hourly price divided by VRAM. It tells you what memory costs, not how fast the card is.

What each card is for

RTX 4090 (24 GB)

  • Fits: 7B–14B models at Q4 with long context, 8B at FP16 (tight), SDXL, FLUX.1 at FP8, QLoRA of 7B–14B.
  • Limits: 32B at Q4 runs but leaves only about 8k tokens of context; no room for FLUX FP16 plus extras.
  • Note: the RTX 4090D is the China-market variant with the same 24 GB and about 10% fewer CUDA cores.

RTX 5090 (32 GB)

  • Fits: 32B models (Qwen3 32B, DeepSeek-R1-Distill-Qwen-32B, QwQ) at Q4 with 16k–32k context, FLUX.1 dev at FP16, faster 7B–14B serving.
  • Why it matters: the jump from 24 GB to 32 GB turns “32B that barely fits” into a comfortable setup, and its bandwidth (1,792 GB/s) is close to an A100’s.
  • Limits: 70B does not fit on one card. Blackwell needs recent CUDA/PyTorch builds; check nvidia-smi before installing frameworks.

RTX 4090 48G (48 GB)

  • Fits: Llama 3.3 70B / Qwen2.5 72B at Q4 with short context (about 8k), 32B at Q8, QLoRA of 32B, 16-bit LoRA of 8B.
  • Trade-off: same compute and bandwidth as a 24 GB 4090, so it buys capacity, not speed.

A100 80G / A800 80G (80 GB)

  • Fits: 70B at Q4 with 32k context, gpt-oss-120b at native MXFP4 (about 63 GB), QLoRA of 70B, larger batch serving.
  • Note: the A800 is the China-spec A100 with the same memory; check both on Deploy, as either may be out of stock at a given hour.

Choose by workload

You want to… Smallest comfortable class Cards to check on Deploy
Chat / RAG with 7B–14B at Q4 24 GB RTX 3090, RTX 4090, RTX 4090D
Run a 32B model at Q4 with 16k+ context 32 GB RTX 5090
FLUX.1 dev at FP16, or FLUX + LoRAs + ControlNet 32–48 GB RTX 5090, RTX 4090 48G
QLoRA a 32B, or 70B at Q4 with short context 48 GB RTX 4090 48G, RTX A6000
70B with long context, gpt-oss-120b, QLoRA 70B 80 GB A800 80G, A100 80G
More KV headroom or batch on 70B–120B 96 GB H20 96G, RTX PRO 6000 96G

Why bandwidth matters for LLM inference

For single-stream generation, each new token reads (roughly) all the weights once, so memory bandwidth sets a theoretical ceiling:

max_tokens_per_second ≈ memory_bandwidth_GB_s / weights_GB

This is a theoretical upper bound, not a benchmark; real throughput is lower and depends on the serving engine, KV cache, batch size and kernels. Worked example for Qwen3 32B at Q4_K_M (≈ 19.5 GB of weights, from our /run page):

Card Bandwidth Theoretical ceiling (single stream)
RTX 4090 / 4090 48G 1,008 GB/s ≈ 52 tokens/s
RTX 5090 1,792 GB/s ≈ 92 tokens/s
A100 80G 1,935 GB/s ≈ 99 tokens/s

Use it to compare cards relative to each other, then measure on the card you rent. Training speed tracks compute (TFLOPS) more than bandwidth.

Three common mistakes

  1. Sizing for weights and forgetting context. The KV cache grows linearly with context. For Qwen3 32B, the FP16 KV cache is about 0.25 GB per 1k tokens, so 32k of context needs about 8 GB on top of the weights. After 19–20 GB of Q4 weights, a 24 GB card has room for roughly 8k. See KV cache and context length.
  2. Assuming more VRAM means more speed. The 4090 48G has the same bandwidth as the 24 GB 4090. If the model already fits, the 48G only raises the bill.
  3. Picking a training card by bandwidth. Inference is bandwidth-bound; training leans on compute and capacity. Run a few dozen steps, measure, then decide — see LoRA fine-tuning GPU.

Pre-flight checklist

Check How
Weight size The model’s page under /run, or the size of the quantised file on Hugging Face
Context you need Compute the KV cache in GB and add it to the weights
Headroom Leave 1–2 GB for framework and CUDA overhead (rule-of-thumb assumption)
Framework Ollama, vLLM, ComfyUI and LLaMA-Factory need NVIDIA CUDA cards; the Ascend 910B supports CANN / MindSpore / PyTorch-NPU only
Budget minutes × hourly price / 60; keep at least one hour of pre-authorisation in your balance
Stock Live stock on /deploy; if your first choice is out, pick another card in the same VRAM class or the next one up

Cost of a working day

At the list prices above, eight hours of continuous use costs: RTX 4090 $2.64, RTX 5090 $4.72, RTX 4090 48G $5.20, A800 80G $8.72, A100 80G $11.92. Per-minute billing means you pay from the moment the GPU is ready until you destroy the instance (the roughly 2-minute startup is not billed); destroy it between sessions. See per-minute billing for the formula.

FAQ

Is an RTX 5090 better than an RTX 4090 for LLM inference? For models that fit both, the 5090’s higher bandwidth (1,792 vs 1,008 GB/s) gives a higher theoretical ceiling, and its 32 GB fits 32B models at Q4 with 16k–32k context, which a 24 GB 4090 cannot do comfortably.

When should I rent an A100 instead of an RTX 5090? When the model or context does not fit 32 GB: 70B models with long context, gpt-oss-120b, QLoRA of 70B, or large batches.

Is the RTX 4090 48G faster than the regular 4090? No. It has the same chip and bandwidth; it doubles the memory, so larger models and contexts fit.

Which GPUs are in stock? Stock changes during the day. Check the Deploy page for live availability; list prices are on /pricing and /gpu-prices.json.

How are these GPUs billed? Per minute, rounded up, with a 1-minute minimum. One hour is pre-authorised while the instance runs and unused minutes are refunded on destroy.

CloudGPU (cloudgpu.app) is an independent service. It has no partnership with, authorization from, or endorsement by the model vendor. GPU and model names are trademarks of their owners. Specs are as published on /pricing; throughput figures are theoretical ceilings, not measurements. Prices are list prices read on 2026-10-23 23:58 Beijing time (UTC+8); live prices and stock on /deploy take precedence.

Open Deploy and pick a cardSize your model first