How do I fine-tune an LLM with Unsloth on a cloud GPU?

Start a Raw GPU on CloudGPU (JupyterLab + CUDA pre-installed), check the driver with nvidia-smi, install Unsloth with its official uv commands, run Unsloth’s QLoRA SFT recipe, and copy the LoRA adapter off the machine before you destroy the instance.

Updated 2026-10-27

This guide follows Unsloth’s own documentation, read on 2026-10-27, and uses its official Qwen3.8-27B recipe as the worked example. Swap the model name for any model Unsloth supports.

What you need

  • A CloudGPU balance and an instance from /deploy using the Raw GPU option. Our deploy page describes it as “a raw GPU with JupyterLab + CUDA pre-installed. SSH access, root on a dedicated container”. It is billed per minute, rounded up, with a 1-minute minimum, and billing stops when you destroy it (see /terms); current rates are on /pricing.
  • A training file, train.jsonl (format below).
  • The base model’s license checked for your use. The example model, Qwen3.8-27B, is released under Apache-2.0 on its Hugging Face model card.

Facts from Unsloth’s docs (read 2026-10-27):

Item What Unsloth’s docs say Source
Python 3.11 up to, but not including, 3.14 Unsloth Requirements
Linux Ubuntu 20.04+ or similar (64-bit), NVIDIA GPU with drivers installed same page
CUDA toolkit 12.4+ recommended, 12.8+ for Blackwell same page
GPUs (Unsloth Core) NVIDIA GPUs from 2018 on, CUDA capability 7.0 or higher, including Blackwell RTX 50 same page
License Core package Apache 2.0; optional components such as the Studio UI AGPL-3.0 Unsloth README

Which GPU should I pick?

VRAM need depends on model, precision and workflow; see How much VRAM do I need to run an LLM? for how to size it. Cards listed on /pricing on 2026-10-27 include the RTX 5090 32G, RTX 4090 48G, RTX 4090 24G, RTX 3090 24G, A100 80G and RTX PRO 6000 96G.

We have not run this recipe on CloudGPU, so this guide gives no peak VRAM or training-time figures for any card. Use Unsloth’s published figures below as a starting point and watch nvidia-smi on your own run.

What Unsloth’s docs say about VRAM

Unsloth publishes a VRAM table by model size (QLoRA = 4-bit, LoRA = 16-bit). It says some models need more than these figures, so treat them as a floor, not a plan. Official figures, read 2026-10-27 from the Unsloth Requirements page:

Model parameters QLoRA (4-bit) VRAM LoRA (16-bit) VRAM
8B 6 GB 22 GB
14B 8.5 GB 33 GB
27B 22 GB 64 GB
32B 26 GB 76 GB
70B 41 GB 164 GB

Unsloth’s Qwen3.8 fine-tuning guide says, for Qwen3.8-27B, “QLoRA works with 24GB and LoRA needs >36GB” (read 2026-10-27). Unsloth’s requirements page also says a batch size set too high is a common cause of out-of-memory errors and suggests 1, 2 or 3.

Step 1: Start a Raw GPU

  1. Open /deploy and choose Raw GPU.
  2. Pick a card from the list and deploy.
  3. When the instance is running, open JupyterLab and start a terminal, or connect over SSH, using the JupyterLab link or SSH details for that instance in the dashboard.

Step 2: Check the environment

Run these before installing anything:

nvidia-smi          # GPU name, driver version, VRAM
nvcc --version      # CUDA toolkit version (Unsloth recommends 12.4+, 12.8+ on Blackwell)
python3 --version   # Unsloth needs 3.11 up to, but not including, 3.14
df -h ~             # free space for model downloads

If nvidia-smi shows no GPU, stop here before you spend time on setup.

Step 3: Install Unsloth

These are the official commands from Unsloth’s pip/uv install page (read 2026-10-27). The uv venv line asks uv for a Python 3.13 environment; we have not checked which Python the Raw GPU image ships with. If the shell cannot find uv after the first line, opening a new terminal may help.

curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv unsloth_env --python 3.13
source unsloth_env/bin/activate
uv pip install unsloth --torch-backend=auto

Check that PyTorch sees the GPU:

python -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.is_available())"

Unsloth also has a web UI, Unsloth Studio (curl -fsSL https://unsloth.ai/install.sh | sh, then unsloth studio). This guide uses the code-based Unsloth Core so every step can be copied into a terminal.

Step 4: Prepare the dataset

Unsloth’s Qwen3.8 recipe expects “a text column already rendered with the Qwen chat template”. If your data is chat messages, one JSON object per line like {"messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]} in chats.jsonl, this helper (ours, not from Unsloth’s docs) renders it into train.jsonl:

# render.py: turn chat messages into the "text" column the recipe expects
import json
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("unsloth/Qwen3.8-27B-unsloth-bnb-4bit")
with open("chats.jsonl") as src, open("train.jsonl", "w") as dst:
    for line in src:
        messages = json.loads(line)["messages"]
        text = tok.apply_chat_template(messages, tokenize=False)
        dst.write(json.dumps({"text": text}) + "\n")

python render.py && head -c 500 train.jsonl

Unsloth’s fine-tuning guide recommends starting with an Instruct model and a curated, well-structured dataset, ideally as question-answer pairs.

Step 5: Run the official QLoRA recipe

This is Unsloth’s minimal SFT recipe for Qwen3.8-27B, copied from the Qwen3.8 fine-tuning guide (read 2026-10-27). Put it in a file named train.py:

from unsloth import FastModel
from datasets import load_dataset
from trl import SFTTrainer, SFTConfig

max_seq_length = 2048

dataset = load_dataset(
    "json",
    data_files = "train.jsonl",
    split = "train",
)

model, tokenizer = FastModel.from_pretrained(
    model_name = "unsloth/Qwen3.8-27B-unsloth-bnb-4bit",
    max_seq_length = max_seq_length,
    load_in_4bit = True,
    full_finetuning = False,
    offload_embedding = True,
)

model = FastModel.get_peft_model(
    model,
    finetune_vision_layers     = False,
    finetune_language_layers   = True,
    finetune_attention_modules = True,
    finetune_mlp_modules       = True,
    r = 16,
    lora_alpha = 16,
    lora_dropout = 0,
    bias = "none",
    use_gradient_checkpointing = "unsloth",
    random_state = 3407,
    use_rslora = False,
    loftq_config = None,
)

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = dataset,
    args = SFTConfig(
        dataset_text_field = "text",
        max_seq_length = max_seq_length,
        per_device_train_batch_size = 1,
        gradient_accumulation_steps = 4,
        warmup_steps = 10,
        max_steps = 100,
        learning_rate = 2e-4,
        logging_steps = 1,
        optim = "adamw_8bit",
        output_dir = "outputs_qwen38",
        seed = 3407,
        dataset_num_proc = 1,
        report_to = "none",
    ),
)
trainer.train()

# Keep only the LoRA adapter (from the same Unsloth page)
model.save_pretrained("qwen38_lora")
tokenizer.save_pretrained("qwen38_lora")

Run it and watch memory in a second terminal:

python train.py 2>&1 | tee train.log
# second terminal:
nvidia-smi --query-gpu=timestamp,memory.used,memory.total --format=csv -l 5

Notes from Unsloth’s docs:

  • max_steps = 100 is a short test run. For a full run the general guide suggests replacing max_steps with num_train_epochs = 1 (1 to 3 epochs).
  • On out-of-memory, the Qwen3.8 guide says to reduce max_seq_length and keep batch size 1 with "unsloth" gradient checkpointing.
  • The Qwen3.8 guide says the first run can take longer while the Gated DeltaNet kernels compile, and asks for the latest Transformers v5.
  • Watch the training loss. The general guide says a loss around 0.5 to 1.0 is often a good sign, but it depends on your dataset; a loss that drops to 0 can mean overfitting.

Step 6: Export the model

Pick one, all from Unsloth’s Qwen3.8 guide and GGUF export page:

Goal Code
LoRA adapter only model.save_pretrained("qwen38_lora") and tokenizer.save_pretrained("qwen38_lora")
GGUF for llama.cpp / Ollama model.save_pretrained_gguf("qwen38_gguf", tokenizer, quantization_method = "q4_k_m")
Merged 16-bit for vLLM model.save_pretrained_merged("qwen38_finetuned", tokenizer, save_method = "merged_16bit")

Unsloth’s docs say to use the same chat template and EOS token during deployment that you used during training. If the export runs out of memory, the GGUF page suggests lowering maximum_memory_usage from the default 0.75 to 0.5.

Step 7: Copy the result off, then destroy the instance

Our docs say a destroyed instance releases its GPU and you create a new instance to resume. Copy your work first:

tar czf qwen38_lora.tgz qwen38_lora train.log

Then download qwen38_lora.tgz from the JupyterLab file browser, copy it with scp over the instance’s SSH connection, or push it to your own Hugging Face repo (Unsloth’s docs show push_to_hub_gguf for GGUF).

Our terms say data on the instance is permanently deleted when it is destroyed. GPU time is billed per minute, rounded up, with a 1-minute minimum, and billing stops when you destroy the instance (see /terms).

FAQ

Can I fine-tune with Unsloth on a rented GPU without installing CUDA myself?

Yes. Choose the Raw GPU option on CloudGPU’s /deploy page, which comes with JupyterLab and CUDA pre-installed and root SSH access, then install Unsloth with the official uv commands.

How much VRAM do I need to fine-tune with Unsloth?

VRAM need depends on model, precision and workflow. Unsloth’s requirements page lists 6 GB for QLoRA and 22 GB for LoRA on an 8B model (read 2026-10-27) and says some models need more. We have not run this recipe on CloudGPU and publish no measurements for it.

Should I use QLoRA or LoRA with Unsloth?

Unsloth’s fine-tuning guide recommends starting with QLoRA, which loads the base model in 4-bit. LoRA keeps the base model in 16-bit and needs more VRAM.

What happens to my fine-tuned model when I destroy the instance?

A destroyed instance releases its GPU and you start a new instance to resume, so download the adapter or push it to Hugging Face first. Data on the instance is permanently deleted when it is destroyed.

How is a fine-tuning run billed on CloudGPU?

GPU time is billed per minute, rounded up, with a 1-minute minimum, and billing stops when you destroy the instance; see /terms. Current rates are on the pricing page.

Sources (read 2026-10-27)