Rent a GPU for LoRA and QLoRA fine-tuning: VRAM, card choice and cost per run
To fine-tune an open model with LoRA or QLoRA you need a GPU whose VRAM covers the frozen base weights plus activations; on CloudGPU you rent that card by the minute, train, save the adapter and destroy the instance. A QLoRA run on an 8B model fits a 24 GB card; a 32B QLoRA fits a 48 GB card; a 70B QLoRA needs an 80 GB card. Instances start in about 2 minutes; the startup is not billed, and from the moment the GPU is ready you pay per minute until you destroy the instance.
This page helps you size the card and price the run. For the click-by-click setup, see the LLaMA-Factory step-by-step guide.
How much VRAM: the estimate
Training memory has four parts: base weights, trainable parameters with their gradients and optimiser state, activations, and framework overhead. The first part is easy to compute; the rest depends on sequence length, batch size and gradient checkpointing.
base_weights_GB ≈ parameters (billions) × bytes_per_parameter
full fine-tune, mixed precision + AdamW: ~16 bytes/param (weights 2 + grads 2 + FP32 master 4 + Adam states 8)
LoRA on a 16-bit base: ~2 bytes/param (+ small adapter, + activations)
QLoRA on a 4-bit base: ~0.55 bytes/param (assumption: 4-bit NF4 plus quantisation constants)
Applied to public parameter counts (Hugging Face model metadata, read 2026-10-23). These are estimates of the base-weight footprint only, not measurements:
| Model (params) | Full FT, ~16 B/param | LoRA base, ~2 B/param | QLoRA base, ~0.55 B/param |
|---|---|---|---|
| Qwen3 8B (8.19B) | ~131 GB | ~16.4 GB | ~4.5 GB |
| Qwen3 14B (14.77B) | ~236 GB | ~29.5 GB | ~8.1 GB |
| Qwen3 32B (32.76B) | ~524 GB | ~65.5 GB | ~18.0 GB |
| Llama 3.3 70B (70.55B) | ~1,129 GB | ~141 GB | ~38.8 GB |
Add activations and overhead on top. Our LLaMA-Factory guide gives these rough totals for typical settings (sequence length and batch size move them):
| Job | Rough total VRAM | VRAM class to rent |
|---|---|---|
| QLoRA, 7B–8B | 6–10 GB | 24 GB |
| LoRA (16-bit), 7B–8B | 18–22 GB | 48 GB (24 GB is tight) |
| QLoRA, 14B | 12–16 GB | 24 GB |
| QLoRA, 32B | 24–30 GB | 48 GB |
| QLoRA, 70B | 45–55 GB | 80 GB |
Full fine-tuning of even an 8B model needs more than one 80 GB card; most individual developers should start with QLoRA or LoRA.
Card and cost per run
List prices from /gpu-prices.json on 2026-10-23 23:58 Beijing time. Stock changes during the day; the Deploy page shows what is available right now.
| Job | Example card (list $/h) | 1 h | 3 h | 8 h |
|---|---|---|---|---|
| QLoRA 8B / 14B | RTX 4090 24G ($0.33) | $0.33 | $0.99 | $2.64 |
| QLoRA 8B / 14B, budget | RTX 3090 24G ($0.21) | $0.21 | $0.63 | $1.68 |
| LoRA 8B, QLoRA 32B | RTX 4090 48G ($0.65) | $0.65 | $1.95 | $5.20 |
| LoRA 8B, QLoRA 32B | RTX A6000 48G ($0.44) | $0.44 | $1.32 | $3.52 |
| QLoRA 70B | A800 80G ($1.09) | $1.09 | $3.27 | $8.72 |
| QLoRA 70B | A100 80G ($1.49) | $1.49 | $4.47 | $11.92 |
Billing is per minute (rounded up, 1-minute minimum). One hour is pre-authorised while the instance runs, and unused minutes are refunded when you destroy it.
Estimate run time before you commit
You cannot read training speed off a spec sheet, so measure it on the card you rent:
total_steps = ceil(samples × epochs / (per_device_batch × grad_accum × num_gpus))
seconds_per_step = (time for the first ~50 steps after warm-up) / 50
train_minutes = total_steps × seconds_per_step / 60
job_minutes = downloads + train_minutes + saving/upload
cost_usd = ceil(job_minutes) × hourly_price / 60
Run 50 steps, read seconds_per_step from the log, and decide whether to continue on this card or switch. That probe costs a few minutes of GPU time.
A minimal QLoRA config (LLaMA-Factory)
# qwen3_8b_qlora.yaml
model_name_or_path: Qwen/Qwen3-8B
stage: sft
do_train: true
finetuning_type: lora
lora_target: all
quantization_bit: 4 # QLoRA; remove for 16-bit LoRA
dataset: your_dataset # registered in data/dataset_info.json
template: qwen3
cutoff_len: 2048
per_device_train_batch_size: 2
gradient_accumulation_steps: 8
learning_rate: 1.0e-4
num_train_epochs: 2
bf16: true
logging_steps: 10
save_steps: 200 # save checkpoints often; see below
output_dir: /workspace/out/qwen3-8b-qlora
llamafactory-cli train qwen3_8b_qlora.yaml
Lower cutoff_len or per_device_train_batch_size if you hit out-of-memory errors; raise gradient_accumulation_steps to keep the effective batch size.
Three rules that protect your run
- Save checkpoints off the machine. Destroying an instance deletes its disk, and there is no persistent storage across instances yet. Push adapters to Hugging Face or copy them with
scp/rsyncas you go — see save checkpoints off the machine. - Keep enough balance. An instance is destroyed when the balance runs out. For an 8-hour run, hold at least 8 hours of the card’s price plus a buffer.
- Plan for one card. The estimates on this page assume a single GPU; check Deploy for what you can start right now. For 70B, download a pre-quantised 4-bit base (about 38.8 GB) rather than the 16-bit weights (about 141 GB in the table above).
Paying
Top up by Alipay, Google Pay, Apple Pay, Visa/Mastercard (from $5) or USDT on TRC-20 (minimum $5). No Chinese phone number or bank card is needed. The same balance also pays for the model API; Claude models there are billed at about 50% of the official public pricing.
FAQ
How much VRAM do I need to QLoRA a 70B model? Roughly 45–55 GB in typical settings: about 39 GB for the 4-bit base weights plus activations and overhead. Rent an 80 GB card.
Can I LoRA fine-tune an 8B model on a 24 GB GPU? QLoRA, yes, comfortably (about 6–10 GB). 16-bit LoRA needs about 18–22 GB, which is tight on 24 GB; a 48 GB card gives headroom.
How do I estimate what a fine-tuning run will cost? Run about 50 steps, measure seconds per step, multiply by total steps, add download and saving minutes, then multiply minutes by the hourly price divided by 60.
What happens to my checkpoints when I destroy the instance? They are deleted with the disk. Copy adapters off the machine or push them to a model hub before you destroy it.
Is billing per minute or per hour? Per minute, rounded up, with a 1-minute minimum. One hour is pre-authorised and unused minutes are refunded when you destroy the instance.
CloudGPU (cloudgpu.app) is an independent service. It has no partnership with, authorization from, or endorsement by the model vendor. Model and software names are trademarks of their owners. VRAM figures are estimates with the stated assumptions. Prices are list prices read on 2026-10-23 23:58 Beijing time (UTC+8); live prices and stock on /deploy take precedence.