How to run Pipecat voice agents with a self-hosted model on a cloud GPU

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Pipecat from the official docs, serve an open-weight model on that machine, and pass that server’s URL as base_url on OpenAILLMService. Copy the text the pipeline prints before you destroy the instance, because destroying the instance deletes its disk and stops the server.

An agent can rent the GPU, serve the model, and run this voice framework against it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Pipecat install. An agent session whose OpenAI-compatible page sets base_url and baseURL on openai.LLM is How to run LiveKit Agents with a self-hosted model on a cloud GPU. A Docker server whose 2.3 quickstart publishes port 80 and whose chat example posts to /v1/chat/completions is How to run GPUStack on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. A CUDA container whose quickstart curls http://localhost:8080/v1/chat/completions is How to run LocalAI on a cloud GPU. A gateway whose README prints uv tool install 'litellm[proxy]' is How to run LiteLLM Proxy in front of a self-hosted model on a cloud GPU. A library whose vLLM chat page sets ChatOpenAI base_url to http://localhost:8000/v1 is How to build LangChain agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the server raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.

The upstream LICENSE is the BSD 2-Clause License.

The steps below are copied from the docs pages named in each step, fetched 2026-11-14. They were not run on a CloudGPU instance for this page. The LLM page names other providers and a speech-to-speech section. This page does not copy those names. A model string in a settings example is not copied. The pages this guide read do not print hf download or huggingface-cli. Weight download for the model server is a different guide: How to download a Hugging Face model.

What the docs print about memory

The LLM page, the OpenAI LLM service page, the local runner page, the Small WebRTC page, the Whisper STT page, the Pocket TTS page, and the Silero VAD page print no GPU GB figure. The Pocket TTS page says the service runs on CPU and does not use a GPU. The Silero VAD page says the analyzer runs on CPU by default. This page does not mark a catalog row as enough. The model server’s memory is a different page: How much VRAM do I need to run an LLM?.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T20:06:07.058Z). The catalog response also includes an availability field and a price field. This page does not copy them. Every catalog row below has min_gpu_count 1. Rows are sorted by vram_gb, then by name. Which listing you can start is on Deploy.

The catalog template list had no Pipecat template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as Pipecat.

Catalog name vram_gb
RTX 2080 8
RTX 3080 10
RTX 2080 Ti 11
RTX 3080 Ti 12
V100 16G 16
A30 24
RTX 3090 24
RTX 4090 24
RTX 4090D 24
RTX 4080 SUPER 32G 32
RTX 5090 32
RTX 5090D 32
V100 32G 32
A100 40G 40
A800 40G 40
A40 48
RTX 4090 48G 48
RTX 4090D 48G 48
RTX A6000 48
Ascend 910B 64
NVIDIA RTX PRO 5000 72G 72
RTX PRO 5000 72G 72
A100 80G 80
A800 80G 80
H800 80
RTX 6000D 83
H20 96G 96
RTX PRO 6000 96G 96
RTX PRO 6000 Max-Q 96
H20-3e 141G 141

Install

Repository HEAD was 1bc2e37b15566cd34bb6941a335ce09c3bbbab8c, committed 2026-11-14T16:54:12Z. That commit is 109 commits ahead of tag v1.16.0. pyproject.toml at that commit sets license to BSD-2-Clause and requires-python to 3.11 or newer. src/pipecat/__init__.py sets __version__ from the installed distribution. It does not print a literal version. PyPI on 2026-11-14 reported pipecat-ai 1.16.0. The docs pages below were the live pages that day. This page does not claim those pages were generated from that commit.

The OpenAI LLM service page’s install line, copied:

uv add "pipecat-ai[openai]"

The local runner page’s install line, copied:

uv add "pipecat-ai[runner]"

Point OpenAILLMService at the server

The LLM page says any OpenAI-spec compatible service can be used via the base_url parameter. The custom-endpoint example, copied:

from pipecat.services.openai.llm import OpenAILLMService

# OpenAI-compatible service via base_url
llm = OpenAILLMService(
    api_key=os.getenv("OTHER_API_KEY"),
    base_url="https://api.other-provider.com/v1"  # Custom endpoint
)

That snippet does not set model. Other examples on the LLM page and the OpenAI LLM service page set model to strings this page does not copy. The service page says base_url overrides the API for a proxied or self-hosted deployment, and that api_key falls back to the OPENAI_API_KEY environment variable when it is None. The settings table on that page lists temperature, max_tokens, top_p, top_k, frequency_penalty, presence_penalty, seed, and max_completion_tokens. It says NOT_GIVEN values are omitted from the request. It says top_k is not sent by this service. The default model cell is a string this page does not copy.

OLLamaLLMService is a different class. Its page says the default base_url is http://localhost:11434/v1 and the default api_key is "ollama". Its pull example is ollama pull llama2. This page does not use that class for the GPU endpoint. Ollama’s install is How to run Ollama on a cloud GPU.

The documented base_url value above is a placeholder. This page does not replace it with an unstated string, and it did not run a combined script. A vLLM server’s listen address is a different document: How to serve an OpenAI-compatible API with vLLM on a cloud GPU. That guide’s server starts at http://localhost:8000. Clients on that guide use http://localhost:8000/v1. Passing that URL as base_url is this page’s use of the documented parameter. It was not run here.

The same LLM page says function calls use FunctionCallsStartedFrame, FunctionCallInProgressFrame, and FunctionCallResultFrame. run_in_parallel defaults to True. The OpenAI LLM service page says to register function handlers with llm.register_function(). The function-calling page’s separate-handler line, copied:

llm.register_function("get_current_weather", fetch_weather_from_api)

The preferred tool on that page is a direct function whose first parameter is FunctionCallParams. The sample return dict prints a number. This page does not copy that dict. The context line, copied:

context = LLMContext(tools=[get_current_weather, get_restaurant_recommendation])

The second name is not defined in that snippet. This page does not merge the tool example with the base_url example.

Local runner

The running-bots-locally page says the development runner is a local development tool. The entry point, copied:

from pipecat.runner.types import RunnerArguments

async def bot(runner_args: RunnerArguments):
    """Entry point. The runner calls this once per session."""
    # ... configure the transport from runner_args, build a pipeline, run it.
    ...

if __name__ == "__main__":
    from pipecat.runner.run import main
    main()

The run lines, copied:

uv run bot.py

That page says the runner starts a local server on localhost:7860, serves a prebuilt client UI at /client, and accepts POST /start. It says -t / --transport restricts the local runner to one transport. The same page’s match statement also names transports this page does not copy. The Small WebRTC case is the local one. The Small WebRTC page says the transport makes a peer-to-peer connection and does not depend on another provider. Its audio install, copied:

uv add "pipecat-ai[webrtc]"

The runner flag that page prints for this transport, without the ICE-server arguments on the following lines, is:

python bot.py -t webrtc

Those following lines pass ICE server URLs this page does not copy. The table on that page says peers on the same local network need neither STUN nor TURN. This page does not add an ICE server.

The LLM page’s pipeline placement, copied:

from pipecat.pipeline.pipeline import Pipeline

pipeline = Pipeline([
    transport.input(),
    stt,                           # Creates TranscriptionFrames
    context_aggregator.user(),     # Processes user context → creates LLMContextFrame
    llm,                           # Processes context → streams LLMTextFrames
    tts,                           # Processes LLMTextFrames → creates TTSAudioRawFrames
    transport.output(),
    context_aggregator.assistant(),
])

stt, tts, and transport in that snippet are names, not classes. This page does not merge that list with the classes below, and it did not run it.

Local speech classes

The Whisper STT page documents WhisperSTTService running locally. Its standard install and basic example, copied:

uv add "pipecat-ai[whisper]"

from pipecat.services.whisper.stt import WhisperSTTService

stt = WhisperSTTService(
    settings=WhisperSTTService.Settings(
        model="base",
    ),
)

The CUDA example on that page sets device="cuda", compute_type="float16", and model=Model.LARGE. The settings table default for model is Model.DISTIL_MEDIUM_EN. The page says the first run downloads the selected model from the Hugging Face model hub. It does not print a repository path. This page does not invent one. It prints no GPU GB figure.

The Pocket TTS page documents PocketTTSService as local and CPU-only. Its install and basic example, copied:

uv add "pipecat-ai[pocket-tts]"

from pipecat.services.pocket_tts.tts import PocketTTSService

tts = PocketTTSService(
    settings=PocketTTSService.Settings(
        voice="alba",
    ),
)

That page says the extra installs pocket-tts>=2.1.0. It says weights are downloaded from Hugging Face on first use. It prints no GPU GB figure.

The Silero VAD page says SileroVADAnalyzer is included with the core install uv add "pipecat-ai". The sample rate must be 8000 or 16000. One example, copied in part:

from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.audio.vad.vad_analyzer import VADParams

vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=0.2))

That page says the analyzer runs on CPU by default and includes a built-in model file. It prints no GPU GB figure.

None of these pages print an ssh command. How to use SSH port forwarding on a cloud GPU is how a browser on your computer reaches a port on the instance. The runner page says the server listens on localhost:7860. This page does not add an ssh command. After a forward that lands on port 7860 of your computer, the path the runner page already prints is /client.

When it fails

CUDA out of memory. The pages this guide read print no GPU GB figure. If the model server raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a model id for you.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The pages this guide read do not pin a PyTorch build and do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120). This page did not re-test a model server on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

What to copy off, then destroy

The pipeline streams text frames. The pages this guide read do not name an output directory for that text. Copy it before you destroy the instance. The model server stops with the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The billing page states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Amounts are not copied here.

Frequently asked questions

What memory figure do the Pipecat pages print?

The LLM page, the OpenAI LLM service page, the local runner page, the Small WebRTC page, the Whisper STT page, the Pocket TTS page, and the Silero VAD page print no GPU GB figure. Pocket TTS is described as CPU-only. Silero VAD runs on CPU by default. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Which class takes a custom base URL?

OpenAILLMService takes base_url. The LLM page sets it to https://api.other-provider.com/v1 and reads the key from OTHER_API_KEY. The settings examples set a model string this page does not copy. Those pages do not print an SSH command. This page did not run them.

What local speech and transport classes do the docs print?

The local runner listens on localhost:7860 and can be restricted with -t webrtc. SmallWebRTCTransport takes a SmallWebRTCConnection. WhisperSTTService is a local STT class, and one example sets model to base. PocketTTSService is a local CPU TTS class, and the basic example sets voice to alba. SileroVADAnalyzer is included with pipecat-ai and one example sets stop_secs to 0.2. This page does not merge those files into one script.

What license sentence does the file print?

The upstream LICENSE is the BSD 2-Clause License. pyproject.toml sets license to BSD-2-Clause and requires-python to 3.11 or newer. Repository HEAD was 1bc2e37b15566cd34bb6941a335ce09c3bbbab8c, committed 2026-11-14T16:54:12Z, 109 commits ahead of tag v1.16.0. PyPI on 2026-11-14 reported pipecat-ai 1.16.0. That license text is not a CloudGPU rate.

Sources (read 2026-11-14, Asia/Shanghai)

  • pipecat commit 1bc2e37b15566cd34bb6941a335ce09c3bbbab8c, committed 2026-11-14T16:54:12Z. The upstream LICENSE is the BSD 2-Clause License. pyproject.toml license BSD-2-Clause. requires-python >=3.11. Tag v1.16.0 is 109 commits behind that HEAD. PyPI pipecat-ai 1.16.0 the same day. __version__ is read from the installed distribution
  • LLM page https://docs.pipecat.ai/pipecat/learn/llm , fetched 2026-11-14. OpenAILLMService with base_url="https://api.other-provider.com/v1" and api_key=os.getenv("OTHER_API_KEY"). The page names other providers. This page does not copy those names. Settings examples set a model string this page does not copy
  • OpenAI LLM service page https://docs.pipecat.ai/api-reference/server/services/llm/openai , fetched 2026-11-14. Install uv add "pipecat-ai[openai]". base_url is the override for a proxied or self-hosted deployment. llm.register_function() is the handler registration that page names
  • Local runner https://docs.pipecat.ai/pipecat/deployment/running-bots-locally , fetched 2026-11-14. uv add "pipecat-ai[runner]". uv run bot.py. localhost:7860. /client. -t / --transport
  • Small WebRTC https://docs.pipecat.ai/api-reference/server/services/transport/small-webrtc , fetched 2026-11-14. uv add "pipecat-ai[webrtc]". python bot.py -t webrtc. The ICE server URLs on that page are not copied
  • Whisper STT https://docs.pipecat.ai/api-reference/server/services/stt/whisper , fetched 2026-11-14. uv add "pipecat-ai[whisper]". model="base". Model.LARGE with device="cuda" and compute_type="float16". Default Model.DISTIL_MEDIUM_EN. No repository path and no GPU GB figure
  • Pocket TTS https://docs.pipecat.ai/api-reference/server/services/tts/pocket-tts , fetched 2026-11-14. uv add "pipecat-ai[pocket-tts]". pocket-tts>=2.1.0. voice="alba". CPU-only. No GPU GB figure
  • Silero VAD https://docs.pipecat.ai/api-reference/server/services/vad/silero-vad-analyzer , fetched 2026-11-14. Included with uv add "pipecat-ai". Sample rate 8000 or 16000. VADParams(stop_secs=0.2). CPU by default
  • Function calling https://docs.pipecat.ai/pipecat/learn/function-calling , fetched 2026-11-14. llm.register_function("get_current_weather", fetch_weather_from_api). LLMContext(tools=[get_current_weather, get_restaurant_recommendation]). The sample return dict’s number is not copied
  • Ollama LLM service page, fetched the same day, is a different class. Default base_url http://localhost:11434/v1. Pull example ollama pull llama2. This page does not use that class as the GPU endpoint
  • The pages above do not print hf download or an ssh command
  • Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (generated_at 2026-11-14T20:06:07.058Z). GET /v1/public/compute-capabilities returned feature flags and no GPU list. The catalog had no Pipecat template. It listed Ollama template ids jupyterlab-ollama, ollama-qwen3, and ollama

Cards you can start are on the Deploy page. Amounts are not copied here.

Last updated: 2026-11-14 (Asia/Shanghai).