How to run Higgs Audio V2 on a cloud GPU

Q: What GPU memory figure does the Higgs Audio V2 README print?

README_V2.md says to run the generation examples on a machine equipped with a GPU with at least 24GB memory. The vLLM README names an A100 with 40GB memory and an RTX 4090 with 24GB memory in its test paragraph. This page does not copy the throughput clauses in that paragraph. The model card says not loading the text language modeling head saves about 1.5GiB. That is a reduction, not a generation requirement. Catalog A100 40G is 40 and catalog RTX 4090 is 24. Those are catalog vram_gb values. This page does not mark a row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Q: Where does the audio go?

The README_V2.md snippet calls torchaudio.save on output.wav. examples/generation.py defaults --out_path to generation.wav, and the documented commands pass --out_path generation.wav. The vLLM README curl example pipes the response to ffmpeg, which writes speech.wav. examples/vllm/run_chat_completion.py writes output_voice_clone.wav for --task voice_clone, output_smart_voice.wav for --task smart_voice, and output_multispeaker.wav for --task multispeaker, in the current directory. The documented commands do not pass --stream. The cd this page adds puts those files at examples/vllm/output_voice_clone.wav, examples/vllm/output_smart_voice.wav, and examples/vllm/output_multispeaker.wav. This page did not run the commands.

Q: Which port does the vLLM server use, and does the README document ComfyUI?

examples/vllm/README.md passes --port 8000 and --network=host, and its curl example uses http://localhost:8000/v1/audio/speech. README_V2.md points at that directory for the API server. The ComfyUI examples top-level directory list on 2026-11-13 had no higgs directory. This page did not start the server.

Q: Which license covers the code and the weights?

The repository LICENSE file is the Apache License, Version 2.0, and the GitHub license field on 2026-11-13 was Apache-2.0. The weight LICENSE file, on the generation model and on bosonai/higgs-audio-v2-tokenizer, is the Boson Higgs Audio 2 Community License Agreement, release date June 20, 2025. A request for bosonai/higgs-audio-v2-generation-3B-base returned id bosonai/higgs-tts-2-3b-base. Section 2 says that if annual active users of the products or services made available by or for the licensee, or affiliates of the licensee, is greater than 100,000 in the preceding calendar year, you must request an expanded license. That count is the upstream license threshold, not a CloudGPU price and not a storage allowance.

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Higgs Audio V2 from the official boson-ai/higgs-audio file README_V2.md, which the repository README points at for v2, then run examples/generation.py or the vLLM server that file links. Copy generation.wav, output.wav, speech.wav, examples/vllm/output_voice_clone.wav, examples/vllm/output_smart_voice.wav, or examples/vllm/output_multispeaker.wav off the instance before you destroy it, because destroying the instance deletes its disk.

The repository README at HEAD says Higgs Audio v3 does not depend on this repository, and that the v2 and v2.5 documentation is README_V2.md. This page follows README_V2.md. It does not copy the v3 API example on the main README.

Other speech pages: How to run Spark-TTS on a cloud GPU, How to run Chatterbox TTS on a cloud GPU, How to run IndexTTS2 on a cloud GPU, and How to run F5-TTS on a cloud GPU. A general vLLM server install is How to serve an API with vLLM on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. The vLLM example listens on port 8000. The tunnel page is How to use SSH port forwarding on a cloud GPU.

The repository LICENSE file is the Apache License, Version 2.0, January 2004. The GitHub license field on 2026-11-13 was Apache-2.0. The weight LICENSE on the generation model and on the tokenizer is a different file: the Boson Higgs Audio 2 Community License Agreement, release date June 20, 2025. Section 2 of that file says that if annual active users of the products or services made available by or for the licensee, or the licensee’s affiliates, is greater than 100,000 in the preceding calendar year, you must request an expanded license. That count is the upstream license threshold. The model card’s front matter says license: other. Read the weight file before you redistribute the weights. The code LICENSE does not replace it.

The commands below are copied from the upstream files named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked.

What the README says about memory

README_V2.md (same commit as repository HEAD 05a145bb490501b534563bf51bf2f7aa2326b271, 2026-06-05) says: for optimal performance, run the generation examples on a machine equipped with a GPU with at least 24GB memory. The model card, read the same day, says that by default the model does not load the text language modeling head, and that this saves about 1.5GiB. That sentence is a reduction, not a generation requirement. The card also says the total parameter count is 3.6B for the language model plus 2.2B for the audio adapter. Those are parameter counts, not VRAM. This page does not copy the evaluation tables. They name products this page does not print.

examples/vllm/README.md (commit 8f9931d571655c76e379e6897cf293e03e76d829, 2025-07-23) says it tested on an A100 GPU with 40GB memory and on an RTX 4090 GPU with 24GB memory. The same paragraph states throughput. This page does not copy those throughput clauses. Catalog A100 40G is 40 and catalog RTX 4090 is 24. Those are catalog numbers. This page does not mark a row as enough.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-13T16:03:04.945Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-13T16:03:03.877Z) listed the same names and the same vram_gb values, plus Ascend 910B at 64, which is not in gpu-prices.json and is not listed here. The catalog response also includes an availability field. This page does not copy it. Every prices row has gpu_count 1. Every catalog row has min_gpu_count 1. Which listing you can start is on Deploy.

The catalog template list had no Higgs Audio template.

Catalog name	vram_gb
RTX 2080	8
RTX 3080	10
RTX 2080 Ti	11
RTX 3080 Ti	12
V100 16G	16
A30	24
RTX 3090	24
RTX 4090	24
RTX 4090D	24
RTX 4080 SUPER 32G	32
RTX 5090	32
RTX 5090D	32
V100 32G	32
A100 40G	40
A800 40G	40
A40	48
RTX 4090 48G	48
RTX 4090D 48G	48
RTX A6000	48
NVIDIA RTX PRO 5000 72G	72
RTX PRO 5000 72G	72
A100 80G	80
A800 80G	80
H800	80
RTX 6000D	83
H20 96G	96
RTX PRO 6000 96G	96
RTX PRO 6000 Max-Q	96
H20-3e 141G	141

Install

On Deploy, choose Raw GPU. README_V2.md says it recommends an NVIDIA Deep Learning Container and names two images, nvcr.io/nvidia/pytorch:25.02-py3 and nvcr.io/nvidia/pytorch:25.01-py3. The example it prints, copied:

docker run --gpus all --ipc=host --net=host --ulimit memlock=-1 --ulimit stack=67108864 -it --rm nvcr.io/nvidia/pytorch:25.02-py3 bash

Option 1 in the same file is a direct install. It prints the clone. Copied:

git clone https://github.com/boson-ai/higgs-audio.git
cd higgs-audio

pip install -r requirements.txt
pip install -e .

The same file also prints venv, conda, and uv blocks. Each one clones the same URL and ends in those two pip lines. This page does not repeat those three blocks. requirements.txt lists torch with no version and transformers>=4.45.1,<4.47.0. It does not pass a CUDA wheel index. requirements.txt was commit 21fd88286d771a8f0ab83a6a649e2eef48fce02f, committed 2025-07-21T23:31:43Z. This page did not run pip.

The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. requirements.txt does not pin that build. This page did not re-test Higgs Audio V2 on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

Generate audio

The get-started snippet in README_V2.md, copied. It sets MODEL_PATH to bosonai/higgs-audio-v2-generation-3B-base and AUDIO_TOKENIZER_PATH to bosonai/higgs-audio-v2-tokenizer, and it calls torchaudio.save on output.wav.

from boson_multimodal.serve.serve_engine import HiggsAudioServeEngine, HiggsAudioResponse
from boson_multimodal.data_types import ChatMLSample, Message, AudioContent

import torch
import torchaudio
import time
import click

MODEL_PATH = "bosonai/higgs-audio-v2-generation-3B-base"
AUDIO_TOKENIZER_PATH = "bosonai/higgs-audio-v2-tokenizer"

system_prompt = (
    "Generate audio following instruction.\n\n<|scene_desc_start|>\nAudio is recorded from a quiet room.\n<|scene_desc_end|>"
)

messages = [
    Message(
        role="system",
        content=system_prompt,
    ),
    Message(
        role="user",
        content="The sun rises in the east and sets in the west. This simple fact has been observed by humans for thousands of years.",
    ),
]
device = "cuda" if torch.cuda.is_available() else "cpu"

serve_engine = HiggsAudioServeEngine(MODEL_PATH, AUDIO_TOKENIZER_PATH, device=device)

output: HiggsAudioResponse = serve_engine.generate(
    chat_ml_sample=ChatMLSample(messages=messages),
    max_new_tokens=1024,
    temperature=0.3,
    top_p=0.95,
    top_k=50,
    stop_strings=["<|end_of_text|>", "<|eot_id|>"],
)
torchaudio.save(f"output.wav", torch.from_numpy(output.audio)[None, :], output.sampling_rate)

A request for bosonai/higgs-audio-v2-generation-3B-base on 2026-11-13 returned id bosonai/higgs-tts-2-3b-base. The resolve URL for LICENSE on the first id responded HTTP 307 to the second id. The card text says Higgs Audio V2 and Higgs Audio V2 Generation have been renamed to Higgs TTS 2. gated was false, cardData.license was other, pipeline_tag was text-to-speech, lastModified was 2026-06-25T16:58:29.000Z, and sha was d80c511612b3040ff2877ce3d408747df1739f11. The tokenizer card bosonai/higgs-audio-v2-tokenizer had gated false, cardData.license other, pipeline_tag feature-extraction, lastModified 2026-10-02T19:58:38.000Z, and sha 815a03f1b57dcb5ae918db415f22cd0c8a6f0f54. This page did not download the weights.

examples/generation.py (commit f04f5df76a6a7b14674e0d6d715b436c422883c6, 2025-08-04) defaults --model_path to bosonai/higgs-audio-v2-generation-3B-base, --audio_tokenizer to bosonai/higgs-audio-v2-tokenizer, and --out_path to generation.wav. The README commands below pass --out_path and do not pass --model_path. Voice cloning, copied. The reference name belinda is a file under examples/voice_prompts. Use a clip you have the right to clone.

python3 examples/generation.py \
--transcript "The sun rises in the east and sets in the west. This simple fact has been observed by humans for thousands of years." \
--ref_audio belinda \
--temperature 0.3 \
--out_path generation.wav

The same section’s device command, copied.

python3 examples/generation.py \
--transcript "The sun rises in the east and sets in the west. This simple fact has been observed by humans for thousands of years." \
--ref_audio belinda \
--temperature 0.3 \
--device_id 0 \
--out_path generation.wav

The file also prints the same command with --ref_audio broom_salesman. Another voice directory it names is examples/voice_prompts. The single-speaker command with no reference audio, copied:

python3 examples/generation.py \
--transcript "The sun rises in the east and sets in the west. This simple fact has been observed by humans for thousands of years." \
--temperature 0.3 \
--out_path generation.wav

Multi-speaker with the voices the README names, copied:

python3 examples/generation.py \
--transcript examples/transcript/multi_speaker/en_argument.txt \
--ref_audio belinda,broom_salesman \
--ref_audio_in_system_message \
--chunk_method speaker \
--seed 12345 \
--out_path generation.wav

The file also prints that transcript with --seed 12345 and no --ref_audio. This page does not repeat that block. generation.py writes the wav with sf.write to the --out_path it was given. This page did not run it.

vLLM server

README_V2.md says option 5 is an API server backed by vLLM, and points at examples/vllm. The server command in that README, copied:

docker run --gpus all --ipc=host --shm-size=20gb --network=host \
bosonai/higgs-audio-vllm:latest \
--served-model-name "higgs-audio-v2-generation-3B-base" \
--model "bosonai/higgs-audio-v2-generation-3B-base"  \
--audio-tokenizer-type "bosonai/higgs-audio-v2-tokenizer" \
--limit-mm-per-prompt audio=50 \
--max-model-len 8192 \
--port 8000 \
--gpu-memory-utilization 0.8 \
--disable-mm-preprocessor-cache

--network=host and --port 8000 are in that command. The curl example uses http://localhost:8000/v1/audio/speech and writes speech.wav through ffmpeg. On a rented GPU that port is on the instance. Forward port 8000 with how to use SSH port forwarding on a cloud GPU.

curl -X POST "http://localhost:8000/v1/audio/speech" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "higgs-audio-v2-generation-3B-base",
    "voice": "en_woman",
    "input": "Today is a wonderful day to build something people love!",
    "response_format": "pcm"
  }' \
  --output - | ffmpeg -f s16le -ar 24000 -ac 1 -i - speech.wav

The same file prints three client commands: python run_chat_completion.py --api-base http://localhost:8000/v1 --task voice_clone, the same command with --task smart_voice, and the same command with --task multispeaker. It does not print cd. run_chat_completion.py is examples/vllm/run_chat_completion.py. That file writes output_voice_clone.wav for voice_clone, output_smart_voice.wav for smart_voice, and output_multispeaker.wav for multispeaker, in the current directory. The documented commands do not pass --stream, so they take the open() branch. The --stream branch writes the same filename for voice clone and for multispeaker. run_chat_completion.py was commit ceb2133d9cd937628612805e2674c7d89d6c9cf9, committed 2025-07-29T00:05:57Z. The cd below is this page’s, so those files land in examples/vllm/. It was not run.

# The vLLM README does not print this cd. This line is this page's. It was not run.
cd examples/vllm
python run_chat_completion.py --api-base http://localhost:8000/v1 --task voice_clone

The file also says you can pass --voice-presets-dir and that the directory needs a config.json. It prints an example with keys belinda and broom_salesman. This page does not copy the transcript strings in that example. This page did not start the container.

ComfyUI

README_V2.md does not document ComfyUI. The ComfyUI examples top-level directory list on 2026-11-13 had no directory named higgs. Examples HEAD was f9431bb000ce792094ff345446e22cac1ea6cef3, committed 2025-11-26T20:33:58Z. This page does not copy a community ComfyUI install. Installing ComfyUI itself is How to install ComfyUI and ComfyUI-Manager on a cloud GPU.

When it fails

CUDA out of memory. README_V2.md says at least 24GB memory for the generation examples. The vLLM README names an A100 with 40GB and an RTX 4090 with 24GB in the test paragraph this page does not copy past the memory clause. If the process raises that error, use how to fix CUDA out of memory on a cloud GPU.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. See the install section. This page did not re-test the unpinned torch line on that card.

The model id in the snippet and the API id differ. The snippet and generation.py still pass bosonai/higgs-audio-v2-generation-3B-base. The model API on 2026-11-13 returned bosonai/higgs-tts-2-3b-base, and the card says the name changed to Higgs TTS 2. This page did not compare the weight bytes.

Save the wav, then destroy

Copy output.wav from the snippet, generation.wav from the documented generation.py commands, speech.wav from the vLLM curl example, and examples/vllm/output_voice_clone.wav, examples/vllm/output_smart_voice.wav, and examples/vllm/output_multispeaker.wav from the client commands before you destroy the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The pricing page FAQ states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Frequently asked questions

What GPU memory figure does the Higgs Audio V2 README print?

README_V2.md says to run the generation examples on a machine equipped with a GPU with at least 24GB memory. The vLLM README names an A100 with 40GB memory and an RTX 4090 with 24GB memory in its test paragraph. This page does not copy the throughput clauses in that paragraph. The model card says not loading the text language modeling head saves about 1.5GiB. That is a reduction, not a generation requirement. Catalog A100 40G is 40 and catalog RTX 4090 is 24. Those are catalog vram_gb values. This page does not mark a row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Where does the audio go?

The README_V2.md snippet calls torchaudio.save on output.wav. examples/generation.py defaults --out_path to generation.wav, and the documented commands pass --out_path generation.wav. The vLLM README curl example pipes the response to ffmpeg, which writes speech.wav. examples/vllm/run_chat_completion.py writes output_voice_clone.wav for --task voice_clone, output_smart_voice.wav for --task smart_voice, and output_multispeaker.wav for --task multispeaker, in the current directory. The documented commands do not pass --stream. The cd this page adds puts those files at examples/vllm/output_voice_clone.wav, examples/vllm/output_smart_voice.wav, and examples/vllm/output_multispeaker.wav. This page did not run the commands.

Which port does the vLLM server use, and does the README document ComfyUI?

examples/vllm/README.md passes --port 8000 and --network=host, and its curl example uses http://localhost:8000/v1/audio/speech. README_V2.md points at that directory for the API server. The ComfyUI examples top-level directory list on 2026-11-13 had no higgs directory. This page did not start the server.

Which license covers the code and the weights?

The repository LICENSE file is the Apache License, Version 2.0, and the GitHub license field on 2026-11-13 was Apache-2.0. The weight LICENSE file, on the generation model and on bosonai/higgs-audio-v2-tokenizer, is the Boson Higgs Audio 2 Community License Agreement, release date June 20, 2025. A request for bosonai/higgs-audio-v2-generation-3B-base returned id bosonai/higgs-tts-2-3b-base. Section 2 says that if annual active users of the products or services made available by or for the licensee, or affiliates of the licensee, is greater than 100,000 in the preceding calendar year, you must request an expanded license. That count is the upstream license threshold, not a CloudGPU price and not a storage allowance.

Sources (read 2026-11-13, Asia/Shanghai)

Repository README (v3 pointer, v2 docs in README_V2.md): https://github.com/boson-ai/higgs-audio/blob/main/README.md commit 05a145bb490501b534563bf51bf2f7aa2326b271, committed 2026-06-05T04:01:06Z. Repository HEAD is that commit
README_V2.md (container command, direct install, at least 24GB, model ids, output.wav, generation.py commands): same commit
requirements.txt (torch unpinned, transformers>=4.45.1,<4.47.0): commit 21fd88286d771a8f0ab83a6a649e2eef48fce02f
examples/generation.py (--model_path default, --audio_tokenizer default, --out_path default generation.wav): commit f04f5df76a6a7b14674e0d6d715b436c422883c6
examples/vllm/README.md (docker server, --port 8000, curl to /v1/audio/speech, A100 40GB and RTX 4090 24GB test sentence): commit 8f9931d571655c76e379e6897cf293e03e76d829
examples/vllm/run_chat_completion.py (output_voice_clone.wav, output_smart_voice.wav, output_multispeaker.wav): commit ceb2133d9cd937628612805e2674c7d89d6c9cf9
Repository LICENSE (Apache 2.0): commit 909eeb70eccbded68fae2fa3384f1665366d6466. GitHub license field Apache-2.0
Model card for the id README_V2.md prints. API id returned bosonai/higgs-tts-2-3b-base, gated false, cardData.license other, sha d80c511612b3040ff2877ce3d408747df1739f11. Weight LICENSE is the Boson Higgs Audio 2 Community License Agreement, June 20, 2025, section 2 threshold 100,000 annual active users. Tokenizer bosonai/higgs-audio-v2-tokenizer sha 815a03f1b57dcb5ae918db415f22cd0c8a6f0f54, same license text
ComfyUI examples top-level directories (no higgs directory): https://github.com/comfyanonymous/ComfyUI_examples commit f9431bb000ce792094ff345446e22cac1ea6cef3
Public GPU memory: https://cloudgpu.app/gpu-prices.json (updated_at 2026-11-13T16:03:04.945Z) and https://cloudgpu.app/v1/public/gpu-catalog (generated_at 2026-11-13T16:03:03.877Z). GET /v1/public/compute-capabilities returned feature flags and no GPU list. The catalog had no Higgs Audio template

Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.

Last updated: 2026-11-13 (Asia/Shanghai).