How to run MMAudio video-to-audio on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install MMAudio from the hkchengrex/MMAudio README, let the demo script download the weights that file names, and run the documented video-to-audio or text-to-audio command. Copy the output folder off the instance before you destroy it. Destroying the instance deletes its disk.
Video pages: How to run Wan 2.2 video generation on a cloud GPU, How to run HunyuanVideo on a cloud GPU, and How to run LTX-Video on a cloud GPU. A different audio model: How to run Stable Audio Open on a cloud GPU. Video-to-audio from a different repository, whose README and model card publish no GPU GB figure, is How to run ThinkSound video-to-audio on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. The tunnel for the Gradio app is How to use SSH port forwarding on a cloud GPU.
Generate audio only for video you have the right to use. The repository LICENSE is the MIT License, copyright 2024 Sony Research Inc. The README says the code is released under that MIT license, and that the checkpoints are released on Hugging Face under CC-BY-NC 4.0. The training-datasets section says the project does not guarantee that the pre-trained models are suitable for commercial use. The model API for hkchengrex/MMAudio on 2026-11-13 returned gated false and a top-level license field of null. Read the license files before you redistribute the program or the weights.
The commands below are copied from the upstream files named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked.
What the README says about memory
The README demo section says: “In our experiments, inference only takes around 6GB of GPU memory (in 16-bit mode) which should fit in most modern GPUs.” That sentence is the README’s. This page did not remeasure it. docs/MODELS.md prints file sizes, including 3.9G for mmaudio_large_44k_v2.pth, 3.9G for mmaudio_large_44k.pth, 2.4G for mmaudio_medium_44k.pth, 601M for each small flow network, 1.2G for v1-44.pth, 655M for v1-16.pth, 429M for best_netG.pt, and 907M for synchformer_state_dict.pth. Those are file sizes in that table, not a second GPU-memory figure.
The same README says the CLIP encoder resizes input frames to 384×384 pixels, that Synchformer resizes the shorter edge to 224 pixels and applies a center crop, that CLIP operates at 8 FPS, and that Synchformer works at 25 FPS. Those sentences are the README’s. This page did not remeasure them.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-13T05:11:23.012Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-13T05:11:24.062Z) listed the same names and the same vram_gb values, plus Ascend 910B at 64, which is not in gpu-prices.json and is not listed here. The catalog response also includes an availability field. This page does not copy it. Every prices row has gpu_count 1. Every vram_gb in that file is 8 or higher. The README’s inference figure is around 6GB in 16-bit mode. This page did not run MMAudio and does not mark a row as enough. Which listing you can start is on Deploy.
The catalog template list had no MMAudio template.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
On Deploy, choose Raw GPU. The deploy page says “SSH access and root within a dedicated container”. The README says the install was tested only on Ubuntu, and asks for Python 3.9+ and PyTorch 2.5.1+ with the matching torchvision and torchaudio. Its first command, which it says is one CUDA index and that any other CUDA version the GPU and driver support can be used instead, is:
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 --upgrade
The clone and install lines are:
git clone https://github.com/hkchengrex/MMAudio.git
cd MMAudio
pip install -e .
The README says to install PyTorch before that pip install -e . line. The commits API for the default branch main listed 974010a026c731054592d8f777218bd9d85a6c24, committed 2026-02-23T06:09:17Z. This page did not install the environment.
Download the weights
The README says the models are downloaded automatically when you run the demo script, that MD5 checksums are in mmaudio/utils/download_utils.py, and that the models are also at https://huggingface.co/hkchengrex/MMAudio/tree/main. docs/MODELS.md marks mmaudio_large_44k_v2.pth as recommended. It says a run needs a flow prediction network, Synchformer, CLIP, a VAE, and a vocoder, and that CLIP and the 44.1kHz vocoder are downloaded automatically. The minimal tree it prints for the recommended model is ext_weights/synchformer_state_dict.pth, ext_weights/v1-44.pth, and weights/mmaudio_large_44k_v2.pth.
download_utils.py sets the large v2 URL to https://huggingface.co/hkchengrex/MMAudio/resolve/main/weights/mmaudio_large_44k_v2.pth. demo.py defaults --variant to large_44k_v2. mmaudio/model/utils/features_utils.py passes a hub id to create_model_from_pretrained and calls open_clip.get_tokenizer with ViT-H-14-378-quickgelu. This page does not print that hub id. A model API request for it on 2026-11-13 returned HTTP 307. Copy the call from that file if you need the id. The model API for hkchengrex/MMAudio the same day returned gated false and a top-level license field of null. This page did not download the weights.
Video-to-audio and text-to-audio
The README says these scripts use the large_44k_v2 model by default. Its demo command is:
python demo.py --duration=8 --video=<path to video> --prompt "your prompt"
It says the output, audio in .flac and video in .mp4, is saved in ./output, and that omitting --video is the text-to-audio command. It says the default output and training duration is 8 seconds. demo.py defaults --variant to large_44k_v2, --duration to 8.0, --cfg_strength to 4.5, --num_steps to 25, --output to ./output, and --seed to 42. When --video is set it writes ./output/<video stem>.flac and, unless --skip_video_composite is set, ./output/<video stem>.mp4. When --video is omitted it builds a filename from the prompt and writes ./output/<that name>.flac. This page did not run the command.
Gradio
The README Gradio section says the app supports video-to-audio and text-to-audio, and an experimental image-to-audio path that duplicates the image into a video. It says that path is not something MMAudio has been trained for. It says to use port forwarding if necessary, gives ssh -L 7860:localhost:7860 server as the example, and says the default port is 7860, which you can specify with --port. The command is:
python gradio_demo.py
gradio_demo.py defaults --port to 7860. The launch call passes server_port=args.port and allowed_paths for output_dir. It does not pass server_name. output_dir is Path('./output/gradio'). The video and image functions write ./output/gradio/<timestamp>.mp4. The text function writes ./output/gradio/<timestamp>.flac. The timestamp format is %Y%m%d_%H%M%S. This page did not start the UI. How to use SSH port forwarding on a cloud GPU is the tunnel. The commits API for gradio_demo.py listed f7cc8a5f6d8e9d1b75d2ccbfe604d14d01e6a504, committed 2025-01-20T03:43:04Z.
The README relevant-repositories section links hkchengrex/av-benchmark and does not link a ComfyUI install. The ComfyUI examples top-level directory list on 2026-11-13 had no directory named for this project. audio/README.md in that repository documents other audio models and does not name MMAudio. Examples HEAD was f9431bb000ce792094ff345446e22cac1ea6cef3. This page does not add a ComfyUI install.
When it fails
CUDA out of memory. The README’s memory sentence is around 6GB in 16-bit mode. It does not print a second low-memory flag for inference. demo.py has --full_precision. This page did not compare that flag with the 16-bit sentence. If the process raises that error, use how to fix CUDA out of memory on a cloud GPU.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The README install example uses the cu118 index and asks for PyTorch 2.5.1+. It does not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this package on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
Save the audio, then destroy
Copy ./output, or the --output directory you passed, before you destroy the instance. The Gradio files are under ./output/gradio. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The pricing page FAQ states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Prices are not copied here.
Frequently asked questions
How much GPU memory does the MMAudio README print?
The README says inference only takes around 6GB of GPU memory in 16-bit mode. This page did not remeasure that sentence. MODELS.md prints file sizes, including 3.9G for mmaudio_large_44k_v2.pth. Those are file sizes, not a second memory figure. Every vram_gb in gpu-prices.json on 2026-11-13 is 8 or higher. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which commands are the documented video-to-audio and text-to-audio lines?
The README demo command is python demo.py --duration=8 --video= followed by a path placeholder and --prompt followed by a prompt placeholder. It says to omit --video for text-to-audio. demo.py defaults --variant to large_44k_v2, --duration to 8.0, and --output to ./output. This page did not run the command.
Where are the files saved, and which port does the Gradio app use?
demo.py writes ./output/ and the video stem with a .flac suffix, and unless --skip_video_composite is set, the same stem with a .mp4 suffix. Text-to-audio writes a .flac file under ./output/ from the prompt. gradio_demo.py writes ./output/gradio/ and a timestamp with .mp4 for the video and image tabs, and .flac for the text tab. The timestamp format is %Y%m%d_%H%M%S. The README says the default port is 7860. The script defaults --port to 7860 and passes server_port. It does not pass server_name. This page did not start the UI.
What licenses do the code and the checkpoints print?
The repository LICENSE is the MIT License, copyright 2024 Sony Research Inc. The grant includes the words free of charge. Those words are license text, not a CloudGPU price and not a storage allowance. The README says the checkpoints are released under CC-BY-NC 4.0. It also says the project does not guarantee that the pre-trained models are suitable for commercial use. The model API license field on 2026-11-13 was empty and gated was false. The file prints no dollar amount.
Sources (read 2026-11-13, Asia/Shanghai)
- hkchengrex/MMAudio README (Ubuntu, Python 3.9+, PyTorch 2.5.1+, cu118 example, clone,
pip install -e ., around 6GB,demo.py,./output, Gradio port 7860, the SSH example, CC-BY-NC, commercial-use sentence): https://github.com/hkchengrex/MMAudio/blob/main/README.md commit8e060ccf1f4171ca58ffed549975ea92d7b80165. Repository HEAD974010a026c731054592d8f777218bd9d85a6c24, committed 2026-02-23T06:09:17Z demo.py(defaults, flac and mp4 paths): commitd294fb839d845a596a0d5c63877463799b5570e5gradio_demo.py(--port7860,./output/gradio, noserver_name): commitf7cc8a5f6d8e9d1b75d2ccbfe604d14d01e6a504docs/MODELS.md(file sizes, recommendedmmaudio_large_44k_v2.pth, minimal tree): commit10c0cfbf599fd90421f59f5cd6c828136e14f847mmaudio/utils/download_utils.py(large v2 URL): commit5db62d61d3d3c674c2d3ef75dc3db889a0976366mmaudio/model/utils/features_utils.py(hub id not printed, tokenizerViT-H-14-378-quickgelu): commita758040e8dba70f3725c1a0245651827d52d4a88. A model API request for that hub id returned HTTP 307LICENSE(MIT, copyright 2024 Sony Research Inc.): commitecc4ca7b1a11eff8a98b679ed9f5e1cf258fc55e- Model API
hkchengrex/MMAudio(gatedfalse, top-levellicensenull, shaeb13a1a98fdbec91753775c57b074ccdfc60587c): https://huggingface.co/api/models/hkchengrex/MMAudio read 2026-11-13 - ComfyUI examples (no MMAudio directory): https://github.com/comfyanonymous/ComfyUI_examples commit
f9431bb000ce792094ff345446e22cac1ea6cef3 - Public GPU memory: https://cloudgpu.app/gpu-prices.json (
updated_at2026-11-13T05:11:23.012Z) and https://cloudgpu.app/v1/public/gpu-catalog (generated_at2026-11-13T05:11:24.062Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no MMAudio template
Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.
Last updated: 2026-11-13 (Asia/Shanghai).