How to run Moshi speech-to-speech on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install the PyTorch package from the official kyutai-labs/moshi repository the way the README prints, start python -m moshi.server, and open the web UI on port 8998 through an SSH tunnel. Copy anything you save off the instance before you destroy it, because destroying the instance deletes its disk.
Transcription with a published VRAM table is How to transcribe audio with faster-whisper on a cloud GPU. WhisperX, whose README says large-v2 with beam_size=5 requires under 8GB of GPU memory, is How to transcribe and diarize audio with WhisperX on a cloud GPU. F5-TTS is How to run F5-TTS on a cloud GPU. Higgs Audio V2’s README_V2.md says at least 24GB memory for the generation examples: How to run Higgs Audio V2 on a cloud GPU. Orpheus TTS is How to run Orpheus TTS on a cloud GPU. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU.
The README says the Python parts and the web client are under the MIT license, the Rust backend is under the Apache license, and the weights are under the CC-BY 4.0 license. moshi/pyproject.toml sets the license text to MIT. On 2026-11-13 the model API for kyutai/moshika-pytorch-bf16, kyutai/moshika-pytorch-q8, kyutai/moshiko-pytorch-bf16, and kyutai/moshiko-pytorch-q8 each reported license cc-by-4.0 and gated false.
The commands below are copied from the upstream files named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked.
What the README and the FAQ say about memory
The README (commit 43993f626659855941c332921dd28a12001c1dea, 2025-08-25; repository HEAD is e6a55d2722a65870ef52a6c9f6ecfc0e90f38362, committed 2026-05-16T13:14:41Z) says that, at the moment, quantization is not supported for the PyTorch version, so you will need a GPU with a significant amount of memory (24GB). The same file’s model list also names experimental int8 PyTorch repositories. This page quotes both and does not merge them.
FAQ.md (commit 061cc4c630d9e11722e08b7d02b1836ba58f30e8, 2025-04-01) says an 8GB GPU is not possible at the moment, and that a 12GB GPU is possible following instructions in issue #54. This page does not copy that issue. The same file says quantized Moshi with PyTorch is not available at the moment, and that the Rust backend should run in int8 with CUDA. The architecture section states a theoretical latency of 160ms (80ms for the frame size of Mimi plus 80ms of acoustic delay) and a practical overall latency as low as 200ms on an L4. This page did not measure those figures. They are not a startup time and they are not a second GPU GB figure. L4 is not a row in gpu-prices.json.
The model API safetensors total for kyutai/moshiko-pytorch-bf16 was 7687729152, sha 2bfc9ae6e890, lastModified 2024-09-18T12:46:23.000Z. The total for kyutai/moshika-pytorch-bf16 was 7687729152, sha a49141e28b3d, lastModified 2024-09-18T12:49:26.000Z. The two q8 cards returned no safetensors total (kyutai/moshika-pytorch-q8 sha 79c2f71d93e9, lastModified 2025-02-05T11:02:38.000Z; kyutai/moshiko-pytorch-q8 sha b07c7a4e1ac8, lastModified 2025-02-05T12:03:16.000Z). This page does not convert the BF16 totals into GB.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-13T21:06:02.664Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-13T21:06:03.185Z) listed the same names and the same vram_gb values, plus Ascend 910B at 64, which is not in gpu-prices.json and is not listed here. The catalog response also includes an availability field. This page does not copy it. Every prices row has gpu_count 1. Every catalog row has min_gpu_count 1. Catalog RTX 2080 is 8. Catalog RTX 3080 Ti is 12. Those are catalog numbers next to the FAQ’s 8GB and 12GB sentences. This page did not run Moshi and does not mark a row as enough. Which listing you can start is on Deploy.
The catalog template list had no Moshi template.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Models the README lists
The README says it releases Moshi fine-tuned on a male synthetic voice (Moshiko), Moshi fine-tuned on a female synthetic voice (Moshika), and Mimi, the speech codec. It says Mimi is bundled in each repository. The PyTorch entries it prints are:
kyutai/moshika-pytorch-bf16andkyutai/moshika-pytorch-q8(experimental)kyutai/moshiko-pytorch-bf16andkyutai/moshiko-pytorch-q8(experimental)
It also lists MLX repositories in int4, int8, and bf16, and Rust/Candle repositories in int8 and bf16. On the two Rust/Candle bf16 entries, the visible link text says kyutai/moshika-mlx-bf16 and kyutai/moshiko-mlx-bf16, and the hrefs say kyutai/moshika-candle-bf16 and kyutai/moshiko-candle-bf16. This page does not pick one of those ids. The line that says all models are released under the CC-BY 4.0 license is in that same section.
moshi/moshi/models/loaders.py (commit e6a55d2722a65870ef52a6c9f6ecfc0e90f38362, 2026-05-16) sets DEFAULT_REPO to kyutai/moshiko-pytorch-bf16. server.py uses that as the default of --hf-repo. Its help text says the default is Moshiko.
Install the PyTorch package
On Deploy, choose Raw GPU. The README says at least Python 3.10, with 3.12 recommended. moshi/pyproject.toml (commit 7f8fb1c97a84343fe0f0ab785dfc2ae7475f5381, 2026-01-07) sets requires-python to >= 3.10,<3.15 and depends on torch >= 2.2.0, < 2.10 and huggingface-hub >= 0.24, < 1.0.0. The README’s PyTorch install line is:
pip install -U moshi
The same block also prints pip install -U moshi_mlx and two editable git+https installs. This page’s path is the PyTorch package. The server downloads weights through huggingface_hub inside the process. The README does not print an hf download line for the server. This page does not add one. The pin is below 1.0.0, so this page does not upgrade huggingface_hub.
Server and web UI
The README’s server line, copied. The square brackets are the README’s notation for optional flags. They are not characters to type.
python -m moshi.server [--gradio-tunnel] [--hf-repo kyutai/moshika-pytorch-bf16]
The line below drops those brackets. It is this page’s. It was not run. Omitting --hf-repo uses DEFAULT_REPO, which is the Moshiko bf16 repository, not the Moshika id in the example.
# The README prints the optional flags inside square brackets. This line is this page's. It was not run.
python -m moshi.server --hf-repo kyutai/moshika-pytorch-bf16
The README says to access the web UI on localhost port 8998. server.py defaults --host to localhost and --port to 8998. It logs Access the Web UI directly at that host and port. The README says a remote GPU will not work in the browser over HTTP, because websites using HTTP are not allowed to use the microphone. It says to forward the remote 8998 port to localhost with the ssh -L flag, then connect to localhost:8998. It does not print the full ssh line. The form is how to use SSH port forwarding on a cloud GPU, with local and remote port 8998.
The same section says --gradio-tunnel sets up a tunnel with a URL accessible from anywhere, and that this tunnel goes through the US and can add significant latency (up to 500ms from Europe). That sentence is about the Gradio tunnel in the README. It is not a statement about where the GPU is. This page did not measure the latency and did not pass the flag. --gradio-tunnel-token sets a fixed secret token.
--half in server.py selects float16 instead of the default bfloat16. The help says it is better for old GPUs. That flag is a dtype switch. The README’s quantization list is the int8 and int4 repository list above, not this flag.
The README also prints python -m moshi.client and python -m moshi.client_gradio --url <moshi-server-url> after gradio-webrtc>=0.0.18. The client is described as barebones. The web UI is the one the README recommends, because it provides echo cancellation. The server command does not name an output file. This page did not start it.
The README’s MLX lines, python -m moshi_mlx.local -q 4 and -q 8, are a different backend. This page does not install that package.
ComfyUI
The README does not document ComfyUI. The ComfyUI examples top-level directory list on 2026-11-13 had no directory named moshi. Examples HEAD was f9431bb000ce792094ff345446e22cac1ea6cef3, committed 2025-11-26T20:33:58Z. This page did not install a node.
When it fails
CUDA out of memory. The README’s PyTorch sentence is a GPU with a significant amount of memory (24GB), in the paragraph that says PyTorch quantization is not supported. The FAQ says 8GB is not possible at the moment and points at issue #54 for 12GB. This page does not copy that issue. If the process raises that error, use how to fix CUDA out of memory on a cloud GPU.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. pyproject.toml requires torch >= 2.2.0, < 2.10 and does not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test Moshi on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
The browser cannot use the microphone. The README and the FAQ both say an HTTP page that is not localhost may block the microphone. The documented workaround is the ssh -L forward of port 8998.
“Moshi stopped talking after 5 min.” The upstream FAQ notes that the 5-minute stop applies to the MLX and Rust implementations.
Save your audio, then destroy
The PyTorch server command does not name a file to copy. If you record a file yourself, copy it off before you destroy the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The pricing page FAQ states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Frequently asked questions
What GPU memory figures do the Moshi README and FAQ print?
The README says quantization is not supported for the PyTorch version, so you will need a GPU with a significant amount of memory (24GB). The FAQ says an 8GB GPU is not possible at the moment, and that a 12GB GPU is possible following instructions in issue 54. This page does not copy that issue. The architecture section states a theoretical latency of 160ms and a practical overall latency as low as 200ms on an L4. This page did not measure those figures. They are not a startup time. L4 is not a row in gpu-prices.json. The model API safetensors total for kyutai/moshiko-pytorch-bf16 and kyutai/moshika-pytorch-bf16 was 7687729152. The q8 cards returned no total. This page does not convert the count into GB. Catalog RTX 2080 is 8 and catalog RTX 3080 Ti is 12. This page did not run Moshi and does not mark a row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which licenses cover the code and the weights?
The README says the Python code and the web client are under the MIT license, the Rust backend is under the Apache license, and the weights are under the CC-BY 4.0 license. pyproject.toml sets the license text to MIT. On 2026-11-13 the four PyTorch model cards reported license cc-by-4.0 and gated false. That statement is the upstream license, not a CloudGPU price.
How do I open the web UI, and does the server write a file?
The README says to start python -m moshi.server and open localhost port 8998. server.py defaults --host to localhost and --port to 8998. If the GPU is remote, the README says to forward port 8998 with ssh -L, because an HTTP page is not allowed to use the microphone. The README does not print the full ssh line. The server command does not name an output file. This page did not start the server.
Which quantization options does the README document for PyTorch?
The README lists PyTorch repositories in bf16 and int8, and it marks the int8 repositories experimental. The same file says quantization is not supported for the PyTorch version. The FAQ says quantized Moshi with PyTorch is not available at the moment, and that the Rust backend should run in int8 with CUDA. The MLX commands use -q 4 and -q 8. This page’s install is the PyTorch package. This page does not merge those sentences.
Sources (read 2026-11-13, Asia/Shanghai)
- Moshi README (PyTorch install, 24GB sentence, model list, server command, port 8998, ssh
-L, Gradio-tunnel sentence, MIT / Apache / CC-BY 4.0): https://github.com/kyutai-labs/moshi commit43993f626659855941c332921dd28a12001c1dea. Repository HEADe6a55d2722a65870ef52a6c9f6ecfc0e90f38362 FAQ.md(8GB not possible, 12GB via issue #54, PyTorch quantization answer, 5 min heading): commit061cc4c630d9e11722e08b7d02b1836ba58f30e8moshi/pyproject.tomlandmoshi/moshi/server.py(Python range, torch range,huggingface-hubbelow 1.0.0, host and port defaults,--half): commit7f8fb1c97a84343fe0f0ab785dfc2ae7475f5381moshi/moshi/models/loaders.py(DEFAULT_REPOkyutai/moshiko-pytorch-bf16): commite6a55d2722a65870ef52a6c9f6ecfc0e90f38362LICENSE-MITandLICENSE-APACHE: commit5fc0ef0ba79b4bcbf0dc1ec0acb961e7cd5a8bf0- Model API, 2026-11-13: the four PyTorch cards above,
licensecc-by-4.0,gatedfalse. BF16 safetensors total 7687729152. q8 cards returned no total - ComfyUI examples top-level directories (no
moshidirectory): https://github.com/comfyanonymous/ComfyUI_examples commitf9431bb000ce792094ff345446e22cac1ea6cef3 - Public GPU memory: https://cloudgpu.app/gpu-prices.json (
updated_at2026-11-13T21:06:02.664Z) and https://cloudgpu.app/v1/public/gpu-catalog (generated_at2026-11-13T21:06:03.185Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no Moshi template
Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.
Last updated: 2026-11-13 (Asia/Shanghai).