How to run Fish Speech on a cloud GPU

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Fish Speech from the official repository, download the weights named in the install doc the README links, run the API server, and copy the audio off the instance before you destroy it. Destroying the instance deletes its disk.

The commands below are copied from the upstream files named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked.

Only clone a voice you have the right to use. The repository LICENSE is the Fish Audio Community License Agreement, Version 1.0. Section IV(b) says you will not use the materials to generate speech that imitates the voice of an identifiable natural person without that person’s prior consent (or, for a minor, the consent of a parent or legal guardian, and for a deceased person, the consent of the holder of the relevant rights) or another legal right to do so. The same section also restricts impersonation in order to deceive, defraud, or harass, and sexual content involving a minor or a voice presented as a minor’s.

The README license notice says the codebase and its associated model weights are released under the FISH AUDIO COMMUNITY LICENSE, and that the S2-Pro weights remain under the Fish Audio Research License. The S2-Pro model card says that model is licensed under the Fish Audio Research License, that research and non-commercial use is permitted free of charge, and that commercial use requires a separate license from Fish Audio. The Drama 3 model card License section says the Fish Audio Community License. The same card’s YAML license_name, and the model API cardData.license_name, are fish-audio-research-license. This page does not resolve that disagreement. Section III(a) of the Community License limits the royalty-free commercial grant to annual revenue under the threshold written in that section. Read LICENSE and both model cards before commercial use.

Other TTS installs on the same kind of instance: how to run CosyVoice, how to run IndexTTS2, how to run GPT-SoVITS, how to run F5-TTS, how to run Kokoro TTS, and how to run Chatterbox TTS.

What the install doc says about memory

The README does not print a GB figure. It links docs/en/install.md as the installation page. That file’s requirements table (read 2026-11-12) says:

Model GPU memory GPU
Drama 3 at least 48 GB (FP8) NVIDIA GPU only: no CPU, Apple MPS or AMD ROCm, and no --compile
S2-Pro at least 24 GB NVIDIA GPU or CPU

The Docker section of the same file says “GPU memory: at least 48 GB for Drama 3 (FP8), 24 GB recommended for S2-Pro”. This page quotes both wordings and does not merge them. The README’s size column is “35B-A3B (3B active per token)” for Drama 3 and “4B parameters” for S2-Pro. This page does not treat those cells as VRAM. The model APIs reported gated: false and pipeline_tag: text-to-speech for both fishaudio/drama-3 and fishaudio/s2-pro. Benchmark tables in the README and the model cards stay in those files.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-12T12:59:08.958Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-12T12:59:08.060Z) listed the same names and the same vram_gb values, plus Ascend 910B, which is not in gpu-prices.json and is not listed here. Which listing you can start is on Deploy. This page does not say which listings are available, and it does not re-measure the 48 GB or 24 GB lines.

Rows whose vram_gb is 48 or higher, the Drama 3 “at least 48 GB” line as printed: A40, RTX 4090 48G, RTX 4090D 48G, RTX A6000, NVIDIA RTX PRO 5000 72G, RTX PRO 5000 72G, A100 80G, A800 80G, H800, RTX 6000D, H20 96G, RTX PRO 6000 96G, RTX PRO 6000 Max-Q.

Rows whose vram_gb is under 24, below the S2-Pro “at least 24 GB” line as printed: RTX 2080 (8), RTX 3080 (10), RTX 2080 Ti (11), RTX 3080 Ti (12), V100 16G (16). Every other row in the table is 24 or higher.

Catalog name vram_gb
RTX 2080 8
RTX 3080 10
RTX 2080 Ti 11
RTX 3080 Ti 12
V100 16G 16
A30 24
RTX 3090 24
RTX 4090 24
RTX 4090D 24
RTX 4080 SUPER 32G 32
RTX 5090 32
RTX 5090D 32
V100 32G 32
A100 40G 40
A800 40G 40
A40 48
RTX 4090 48G 48
RTX 4090D 48G 48
RTX A6000 48
NVIDIA RTX PRO 5000 72G 72
RTX PRO 5000 72G 72
A100 80G 80
A800 80G 80
H800 80
RTX 6000D 83
H20 96G 96
RTX PRO 6000 96G 96
RTX PRO 6000 Max-Q 96

The install doc says V100 and other GPUs older than SM80 have no bfloat16, so Drama 3 switches to float16 there, and that this path has not been tested on real hardware. Catalog V100 16G is 16 and V100 32G is 32. 16 is under the S2-Pro line. 32 is under the Drama 3 line and at or above the S2-Pro line. Those are the printed figures, not a test on either V100.

Install

On Deploy, choose Raw GPU. The conda block uses Python 3.12. If conda is not on PATH, use how to set up a conda environment on a cloud GPU. This page did not check whether the Raw GPU image already has conda. pyproject.toml (read 2026-11-12) says requires-python = ">=3.10" and pins torch==2.8.0 on the cu126, cu128, and cu129 extras.

The clone line is in the Docker Setup section of docs/en/install.md, not in the conda block. This page copies it here:

git clone https://github.com/fishaudio/fish-speech.git
cd fish-speech

System packages, copied:

apt install portaudio19-dev libsox-dev ffmpeg

The conda GPU block, copied. The comment in the file says to replace both cu129 values when using cu126 or cu128.

conda create -n fish-speech python=3.12
conda activate fish-speech

# GPU installation (choose your CUDA version: cu126, cu128, cu129)
# Replace both "cu129" values when using cu126 or cu128.
pip install --extra-index-url https://download.pytorch.org/whl/cu129 -e ".[cu129]"

The same file also prints a CPU extra, a pip install -e . line, and a uv sync --python 3.12 --extra cu129 block. This page follows the conda GPU block. How to install a specific PyTorch CUDA version is the wheel index.

Weights

Copied from the download section:

pip install -U "huggingface_hub[cli]"

hf download fishaudio/drama-3 --local-dir checkpoints/drama-3
hf download fishaudio/s2-pro --local-dir checkpoints/s2-pro

The install doc says that if the Drama 3 directory has no codec.pth, the server uses checkpoints/s2-pro/codec.pth, so download S2-Pro as well. How to download a Hugging Face model is the manual hf download reference. cloudgpu-net is not in the Fish Speech install doc. The download speed guide says to leave it off during pip, then turn it on for a weight download.

API server and WebUI

Copied from docs/en/inference.md. Drama 3 does not support --compile.

python tools/api_server.py \
    --listen 0.0.0.0:8080 \
    --llama-checkpoint-path checkpoints/s2-pro \
    --decoder-checkpoint-path checkpoints/s2-pro/codec.pth \
    --decoder-config-name modded_dac_vq

python tools/api_server.py \
    --listen 0.0.0.0:8080 \
    --llama-checkpoint-path checkpoints/drama-3 \
    --device cuda:0

The inference doc says the Docker images already include the WebUI at http://localhost:8080/ui. From source, build it first. Copied:

cd awesome_webui
npm install
npm run build

Then return to the project root. The WebUI section’s server command, copied. The note above it says --compile is S2-Pro only and to drop it when loading Drama 3. The command as printed does not set a checkpoint path. The two blocks above do, on port 8080. This page does not merge them into a new command.

python tools/api_server.py --listen 0.0.0.0:8888 --compile

The same section says to open http://localhost:8888/ui.

On your own computer, for the port you actually started. These lines are added by this page:

ssh -N -L 8080:127.0.0.1:8080 -p <ssh-port> <ssh-user>@<ssh-host>

ssh -N -L 8888:127.0.0.1:8888 -p <ssh-port> <ssh-user>@<ssh-host>

Then open http://localhost:8080/ui or http://localhost:8888/ui to match that forward. How to use SSH port forwarding is the forwarding reference. This page did not start the server.

The Drama 3 command-line example writes --output "output.wav". The S2-Pro command-line steps write fake.npy, fake.wav, and a codes_N file. The API client example passes --output demo. Only clone a voice you have the right to use.

The Fish Speech README and docs/en/inference.md do not document a ComfyUI route. This page does not add one. How to choose a rental GPU for ComfyUI and how to run FLUX.1-dev in ComfyUI are separate installs. The README points at an SGLang-Omni README and a vLLM-Omni recipe. This page does not copy those commands.

When it fails

CUDA out of memory. The install doc says at least 48 GB for Drama 3 (FP8) and at least 24 GB for S2-Pro. The Docker wording for S2-Pro is “24 GB recommended”. Catalog rows under those figures are listed above. Then use how to fix CUDA out of memory on a cloud GPU.

“CUDA error: no kernel image is available” on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32 GB. 32 is under the Drama 3 line and at or above the S2-Pro “at least 24 GB” line. That is a comparison of the printed figures with the catalog, not a test. The conda example uses the cu129 extra. The comment also lists cu126 and cu128. The RTX 5090 guide says a cu126 wheel has no sm_120 code and that the card needs a PyTorch build for CUDA 12.8 or newer. This page did not run the install and did not re-test it on an RTX 5090.

Drama 3 on a V100. The install doc says the float16 path for GPUs older than SM80 has not been tested, and to open an issue if you get silence, noise, or a “non-finite logits” error. This page did not run that path.

Save the audio, then destroy

Copy output.wav, fake.wav, fake.npy, codes_N, and the file named by --output before you destroy the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed. The billing rule is on the pricing page. Prices are not copied here.

Frequently asked questions

How much GPU memory does Fish Speech publish?

docs/en/install.md, which the README links as the installation page, says Drama 3 needs at least 48 GB (FP8) on an NVIDIA GPU only, and S2-Pro needs at least 24 GB on an NVIDIA GPU or CPU (read 2026-11-12). The Docker section of that file says at least 48 GB for Drama 3 (FP8) and 24 GB recommended for S2-Pro. The README’s 35B-A3B and 4B lines are parameter counts. This page does not treat them as VRAM. This page does not re-measure those figures and does not say which listings are available. Which listing you can start is on https://cloudgpu.app/deploy.

What license covers Fish Speech, and whose voice can I clone?

The README says the codebase and its associated model weights are released under the FISH AUDIO COMMUNITY LICENSE, and that the S2-Pro weights remain under the Fish Audio Research License. The LICENSE file is the Fish Audio Community License Agreement, Version 1.0. Section IV(b) says you will not generate speech that imitates the voice of an identifiable natural person without that person’s prior consent, or the consent that section requires for a minor or a deceased person, or another legal right to do so. Only clone a voice you have the right to use. The S2-Pro model card says research and non-commercial use is permitted free of charge and commercial use requires a separate license from Fish Audio. The Drama 3 card’s License section says the Fish Audio Community License, while its YAML license_name and the model API cardData.license_name are fish-audio-research-license. Read the files before commercial use.

How do I open the Fish Speech WebUI?

The S2-Pro server command is python tools/api_server.py --listen 0.0.0.0:8080 with checkpoints/s2-pro and checkpoints/s2-pro/codec.pth. The Drama 3 command listens on the same address with checkpoints/drama-3 and --device cuda:0. Docker images include the WebUI at http://localhost:8080/ui. From source, build awesome_webui, then the WebUI section runs python tools/api_server.py --listen 0.0.0.0:8888 --compile and opens http://localhost:8888/ui. That --compile flag is S2-Pro only. These commands were not run for this page. The README does not document ComfyUI.

Will the Fish Speech audio stay after the instance is destroyed?

No. The Drama 3 command-line example writes output.wav. The S2-Pro command-line steps write fake.npy, fake.wav, and codes_N. The API client example passes --output demo. Copy those files before you destroy the instance. The terms say data on local instance storage is permanently deleted when the instance is destroyed. The billing rule is on https://cloudgpu.app/pricing.

Sources (read 2026-11-12, Asia/Shanghai)

Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.

Last updated: 2026-11-12 (Asia/Shanghai).