How do I install Nunchaku on a cloud GPU?

Check your Python, PyTorch and CUDA versions, then pip install the Nunchaku 1.2.1 wheel from GitHub Releases whose file name matches all three, for example nunchaku-1.2.1+cu12.8torch2.9-cp312-cp312-linux_x86_64.whl, into the same Python environment that runs ComfyUI; then add the ComfyUI-nunchaku node and load FP4 models on an RTX 5090 or INT4 models on older cards.

Updated 2026-11-01

This guide follows the official Nunchaku and ComfyUI-nunchaku installation docs, the Nunchaku FAQ, the v1.2.1 release files and source, the PyTorch wheel index and ComfyUI-nunchaku issue #819, all read on 2026-11-01. It is for people who want 4-bit Nunchaku models in ComfyUI on a rented GPU and get stuck on the backend wheel.

What Nunchaku 1.2.1 needs

Nunchaku is an inference engine for 4-bit diffusion models (SVDQuant). ComfyUI-nunchaku is the ComfyUI node pack that uses it. You install two things: the ComfyUI-nunchaku plugin and the nunchaku backend wheel.

According to the official Nunchaku installation docs and the GitHub releases page (read 2026-11-01):

  • The latest stable release is v1.2.1, published on 2026-01-25. Its notes say “support torch2.10 and more cuda versions”. Development builds go up to 1.3.0dev20260306.
  • The docs list PyTorch 2.5 or newer as a prerequisite. For Blackwell GPUs (RTX 50-series) they ask for PyTorch 2.7 or newer with CUDA 12.8 or newer.
  • Wheels are prebuilt per Python version, PyTorch version and CUDA version. The v1.2.1 Linux wheels on GitHub Releases are:
Wheel tag PyTorch Python
cu12.8 2.8, 2.9, 2.10, 2.11 3.10, 3.11, 3.12, 3.13 (cp310 to cp313)
cu13.0 2.9, 2.10, 2.11 3.10, 3.11, 3.12, 3.13 (cp310 to cp313)
  • ComfyUI-nunchaku 1.2.1 declares Python >=3.10,<3.14 and logs a warning if the backend is older than nunchaku 1.0.0.
  • The Nunchaku 1.2.1 source accepts compute capability 7.5, 8.0, 8.6 and 8.9 (INT4 models) and 12.0 or 12.1 (FP4 models).

Step 1: Deploy a Raw GPU and connect

  1. Open /deploy and choose Raw GPU. Our deploy page describes it as “a raw GPU with JupyterLab + CUDA pre-installed. SSH access, root on a dedicated container, install any Python/CUDA libs you want.”
  2. Pick a card. Cards listed on /pricing on 2026-11-01 include the RTX 5090 32G, RTX PRO 6000 96G, RTX 6000D, RTX A6000, RTX 4090 48G, RTX 4090 24G, RTX 4090D, RTX 3090 24G, RTX 2080, A100 80G, A100 40G, A800 80G, A800 40G, H20 96G and V100 32G. See current rates on /pricing. Compute capability from the NVIDIA CUDA GPUs list (read 2026-11-01): RTX 5090 and RTX PRO 6000 Blackwell 12.0, RTX 4090 8.9, RTX 3090 and RTX A6000 8.6, A100 8.0, RTX 2080 7.5. For any other card, read compute_cap in Step 2 and compare it with the list above before you install. VRAM need depends on model, precision and workflow; see How much VRAM do I need to run an LLM?.
  3. Open a terminal in JupyterLab, or connect over SSH with your instance’s connection details:
ssh -p <port> <user>@<host>

For ComfyUI on our machines in general, see Rent a GPU for ComfyUI.

Step 2: Check the environment first

nvidia-smi                                              # driver and "CUDA Version" in the header
nvidia-smi --query-gpu=name,compute_cap --format=csv    # card name and compute capability
nvcc --version                                          # CUDA toolkit, only needed for a source build
python3 --version                                       # ComfyUI-nunchaku 1.2.1 needs 3.10 to 3.13
python3 -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.is_available())"

Write down four values: compute_cap, the “CUDA Version” in the nvidia-smi header, your Python version, and (if PyTorch is present) torch.__version__ and torch.version.cuda. The wheel you pick in Step 4 depends on the last three.

Our deploy page lists “JupyterLab + CUDA pre-installed” for Raw GPU but does not list the driver version, whether nvcc is on PATH, or whether PyTorch is pre-installed, so read them from the commands above on your own instance.

Step 3: Prepare the Python environment that runs ComfyUI

Nunchaku must go into the same Python environment as ComfyUI. If you already run ComfyUI, activate its environment and skip to Step 4.

For a fresh setup, create a virtual environment and install a PyTorch version that has a Nunchaku wheel (2.8 to 2.11) before ComfyUI, so pip install -r requirements.txt keeps it (ComfyUI’s requirements.txt lists torch without a version on 2026-11-01):

python3 -m venv ~/venvs/comfy
source ~/venvs/comfy/bin/activate
pip install -U pip

# PyTorch 2.11 built for CUDA 13.0 (use cu128 instead if nvidia-smi shows a CUDA Version below 13.0)
pip install "torch==2.11.*" torchvision --index-url https://download.pytorch.org/whl/cu130

git clone https://github.com/Comfy-Org/ComfyUI.git
cd ComfyUI
pip install -r requirements.txt
python -c "import torch; print(torch.__version__, torch.version.cuda)"   # should still print 2.11.x

On 2026-11-01 the PyTorch cu128 and cu130 indexes both have PyTorch 2.11.0 Linux wheels. The current PyTorch stable release is 2.15.0, which has no Nunchaku 1.2.1 wheel, so do not upgrade PyTorch in this environment. For more on pinning PyTorch, see How do I install a specific PyTorch CUDA version on a cloud GPU?.

Step 4: Install the matching Nunchaku wheel

The wheel file name has three tags that must match your environment:

nunchaku-1.2.1+cu<CUDA>torch<PyTorch major.minor>-cp<Python>-cp<Python>-linux_x86_64.whl

Build the name from your environment and install it from GitHub Releases:

python - <<'PY' > ~/nunchaku-wheel.txt
import sys, torch
t = ".".join(torch.__version__.split("+")[0].split(".")[:2])
cu = torch.version.cuda
py = f"cp{sys.version_info.major}{sys.version_info.minor}"
print(f"https://github.com/nunchux-ai/nunchaku/releases/download/v1.2.1/nunchaku-1.2.1+cu{cu}torch{t}-{py}-{py}-linux_x86_64.whl")
PY
cat ~/nunchaku-wheel.txt                                 # check the wheel name

python -c "import torch; print(torch.__version__)"      # note the version
pip install -r ~/nunchaku-wheel.txt
python -c "import torch; print(torch.__version__)"      # should print the same version

The printed name must contain cu12.8 or cu13.0 and a torch version from the table above. If pip reports a 404 for the URL, that combination is not in the release: check the table, then change PyTorch (Step 3) rather than renaming the file.

According to the official docs, the same wheels are also published on Hugging Face and ModelScope.

Step 5: Verify the backend on the GPU

python - <<'PY'
import importlib.metadata as m
import torch
import nunchaku
from nunchaku.utils import get_precision

print("nunchaku", m.version("nunchaku"))
print("torch", torch.__version__, "cuda", torch.version.cuda)
print("gpu", torch.cuda.get_device_name(0), torch.cuda.get_device_capability(0))
print("precision", get_precision())
PY

In the Nunchaku 1.2.1 source, get_precision() returns fp4 on compute capability 12.0 or 12.1 and int4 on other GPUs. That value tells you which model files to download.

Step 6: Install the ComfyUI-nunchaku plugin

The ComfyUI-nunchaku installation docs (read 2026-11-01) give three methods. Run them in the same environment as Step 3.

Method A: comfy-cli

pip install comfy-cli
comfy node registry-install ComfyUI-nunchaku

Method B: ComfyUI-Manager. With ComfyUI-Manager installed in custom_nodes, open the Manager, search for ComfyUI-nunchaku in the Custom Nodes Manager and install it.

Method C: git clone

cd ~/ComfyUI/custom_nodes        # adjust to your ComfyUI path
git clone https://github.com/nunchux-ai/ComfyUI-nunchaku nunchaku_nodes

Then restart ComfyUI and read its log. ComfyUI-nunchaku 1.2.1 prints Nunchaku version: and ComfyUI-nunchaku version: in its log when it loads the plugin, and a warning if the backend is older than 1.0.0. If you started ComfyUI over SSH, keep it running with tmux or screen (see How do I keep a job running after SSH disconnects?).

The plugin also ships an install_wheel.json workflow with a NunchakuWheelInstaller node that downloads the backend for you (ComfyUI-nunchaku v0.3.2 and later). If it fails, use Step 4 instead; see the next section.

Fix: “Failed to install from all available sources”

This message comes from the NunchakuWheelInstaller node. In ComfyUI-nunchaku issue #819, a user traced it to the online version list the node reads (nunchaku_versions.json). We read that file on 2026-11-01:

  • Its file name template is nunchaku-{version}+{torch_version}-{python_version}-{python_version}-{platform}.whl, with no CUDA part. The real v1.2.1 file names contain cu12.8 or cu13.0. As of 2026-11-01 (UTC+8) we observed that a HEAD request for nunchaku-1.2.1+torch2.9-cp312-cp312-linux_x86_64.whl on GitHub Releases returned 404, while the same name with cu12.8torch2.9 returned 200. This is our own dated check, not an official rule; the file list can change, so re-check the release page before you rely on it.
  • Its torch list stops at 2.9, so PyTorch 2.10 and 2.11 are not offered at all.

Fix: install the wheel with pip as in Step 4, using the full name with the CUDA tag, then restart ComfyUI.

Fix: FP4 vs INT4 errors

According to the official Nunchaku FAQ (read 2026-11-01): “For Blackwell GPUs (such as the RTX 50-series), please use our FP4 models for hardware compatibility. For all other GPUs, please use our INT4 models.” Nunchaku model repositories on Hugging Face carry both variants with the precision in the file name; for example, the Nunchaku FLUX.1-dev model page lists svdq-fp4_r32-flux.1-dev.safetensors and svdq-int4_r32-flux.1-dev.safetensors (read 2026-11-01).

The Nunchaku 1.2.1 source raises these errors when the model and card do not match:

  • Please use "fp4" quantization for Blackwell GPUs.: you loaded an INT4 model on a compute capability 12.0 or 12.1 card such as the RTX 5090 or RTX PRO 6000. Download the FP4 file.
  • Please use "int4" quantization for Turing, Ampere and Ada GPUs.: you loaded an FP4 model on an RTX 4090, RTX 3090, A100, RTX 2080 or similar. Download the INT4 file.
  • Unsupported GPU architecture ... due to the lack of 4-bit tensorcores.: your card’s compute capability is not in Nunchaku’s list (7.5, 8.0, 8.6, 8.9, 12.0, 12.1). Pick another card.

Each 4-bit model keeps its base model’s terms. Read them on the model’s page before you use it: the Nunchaku FLUX.1-dev model page and its base model page, FLUX.1-dev.

Fix: no wheel for my PyTorch (2.12 or newer, or CUDA 13.2+)

The v1.2.1 Linux wheels stop at PyTorch 2.11 and CUDA 13.0. Two ways forward:

  1. Use PyTorch 2.11 in a separate environment. Follow Step 3 in a new virtual environment so other projects keep their PyTorch.
  2. Build from source. According to the official docs, a Linux source build needs CUDA 12.2 or newer (12.8 or newer for Blackwell) and gcc/g++ 11 or newer; nvcc --version should match torch.version.cuda. On a Raw GPU you have root, so you can install build tools yourself. Run long builds inside tmux or screen.

If PyTorch itself fails on an RTX 5090 with “no kernel image is available”, fix that first: see How do I fix CUDA “no kernel image is available” on an RTX 5090?.

Speed and memory

The Nunchaku team reports, in the Nunchaku README (read 2026-11-01), a 3.6× memory reduction for 12B FLUX.1-dev compared with the BF16 model, and for NVFP4 “~3× speedup on the RTX 5090 over BF16”. These are the Nunchaku team’s figures, not ours. To see what you get on your own instance, run the same workflow once with the BF16 model and once with the svdq-fp4_r32-flux.1-dev.safetensors (RTX 5090) or svdq-int4_r32-flux.1-dev.safetensors (RTX 4090 and older) file, compare the per-image time ComfyUI prints in its log, and watch VRAM use with:

nvidia-smi --query-gpu=name,memory.used,memory.total --format=csv -l 2

If you enable xFormers in the same ComfyUI setup, see How do I install xFormers on a cloud GPU?. For another attention backend and a ready-made image UI, see How do I install FlashAttention on a cloud GPU? and How do I run Stable Diffusion WebUI Forge on a cloud GPU?.

Data on local instance storage is permanently deleted when you destroy the instance (see /terms), so keep your install commands in a script you can re-run on the next instance. Billed per minute, rounded up, with a 1-minute minimum; billing stops when you destroy the instance (see /terms).

Troubleshooting

Symptom Likely cause Fix
Failed to install from all available sources installer’s version list has no CUDA tag and stops at torch 2.9 install the wheel with pip using the full cu12.8/cu13.0 name (Step 4)
pip 404 on the wheel URL no wheel for that CUDA / torch / Python combination check the wheel table; switch PyTorch to 2.8–2.11 on cu128 or cu130
ModuleNotFoundError: No module named 'nunchaku' in ComfyUI wheel installed into a different Python activate ComfyUI’s environment and repeat Step 4
Please use "fp4" quantization for Blackwell GPUs. INT4 model on RTX 5090 / RTX PRO 6000 use the FP4 model file
Please use "int4" quantization for Turing, Ampere and Ada GPUs. FP4 model on a pre-Blackwell card use the INT4 model file
Unsupported GPU architecture compute capability not 7.5, 8.0, 8.6, 8.9, 12.0 or 12.1 pick a supported card
PyTorch changed after an install a package pulled a different torch reinstall torch==2.11.* from the same index, then the matching wheel

FAQ

How do I install Nunchaku on Linux?

Check your Python, PyTorch and CUDA versions with python -c “import torch, sys; print(torch.version, torch.version.cuda, sys.version)”, then pip install the Nunchaku 1.2.1 wheel from GitHub Releases whose name matches all three, for example nunchaku-1.2.1+cu12.8torch2.9-cp312-cp312-linux_x86_64.whl. Install it into the same Python environment that runs ComfyUI or your script.

Which Nunchaku wheel matches my PyTorch?

Read the cu, torch and cp parts of the file name. Nunchaku 1.2.1 Linux wheels exist for CUDA 12.8 with PyTorch 2.8 to 2.11 and for CUDA 13.0 with PyTorch 2.9 to 2.11, each for Python 3.10 to 3.13. The torch part must match the first two numbers of torch.version and the cu part must match torch.version.cuda.

Why does the Nunchaku wheel installer say “Failed to install from all available sources”?

In ComfyUI-nunchaku issue #819 a user traced it to the online version list: its file name template has no CUDA part, so the installer asks for a file name that does not exist on GitHub Releases, and its torch list stops at 2.9. Install the wheel by hand with pip, using the full file name that includes cu12.8 or cu13.0.

Should I use INT4 or FP4 Nunchaku models on an RTX 5090?

FP4. According to the official Nunchaku FAQ, Blackwell GPUs such as the RTX 50-series should use the FP4 models and all other GPUs the INT4 models. In the Nunchaku source, nunchaku.utils.get_precision() returns fp4 on compute capability 12.0 or 12.1 and int4 on other GPUs.

Which GPUs does Nunchaku support?

The Nunchaku 1.2.1 source accepts compute capability 7.5, 8.0, 8.6 and 8.9 for INT4 models and 12.0 or 12.1 for FP4 models. Other architectures raise an “Unsupported GPU architecture” error. Run nvidia-smi --query-gpu=name,compute_cap --format=csv to read the value for your card.

Is there a Nunchaku wheel for PyTorch 2.12 or newer?

Not in the v1.2.1 release on 2026-11-01: its Linux wheels stop at PyTorch 2.11. Install PyTorch 2.11 from the cu128 or cu130 PyTorch index in a separate virtual environment, or build Nunchaku from source, which the official docs say needs CUDA 12.2 or newer and gcc 11 or newer on Linux.

How do I check that Nunchaku is installed correctly?

Run python -c “import importlib.metadata as m; import nunchaku; from nunchaku.utils import get_precision; print(m.version(‘nunchaku’), get_precision())” in the same environment as ComfyUI. It should print the Nunchaku version and fp4 or int4. When ComfyUI loads the plugin, its log prints the Nunchaku and ComfyUI-nunchaku versions.

Sources