How to run Pydantic AI agents with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Pydantic AI the way the official README prints, serve a model on that machine, and point OllamaModel or VLLMProvider at that server. Copy result.output before you destroy the instance, because destroying the instance deletes its disk.
An agent can rent the GPU and run this library on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Pydantic AI install. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. A TurboMind server whose lmdeploy serve api_server listens on port 23333 is How to serve LLMs with LMDeploy on a cloud GPU. A coding-agent server whose card prints --tool-call-parser qwen3_coder is How to serve Qwen3-Coder for coding agents on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose Ollama page sets url to http://localhost:11434, and whose vLLM page defaults api_base_url to http://localhost:8000/v1, is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b, and whose GPU-server example sets api_base to http://localhost:7501/v1, is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b, and whose VLLM class defaults base_url to http://localhost:8000/v1/, is How to run Agno agents with a self-hosted model on a cloud GPU.
The LICENSE file begins with “The MIT License (MIT)”, its copyright notice names Pydantic Services Inc. 2024 to present, and the GitHub license field on 2026-11-14 was MIT.
The commands below are copied from the upstream files named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The README and the compatible-APIs page also name other providers. This page does not list those names.
What the docs print about memory
The README, the installation page, the Ollama page, and the vLLM section of the compatible-APIs page print no GPU GB figure. The Qwen3.8-27B card prints “Number of Parameters: 27B” and no GPU GB figure. This page does not copy that card’s benchmark table and does not treat 27B as vram_gb. This page does not mark a catalog row as enough.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-14T06:12:25.272Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-14T06:12:26.381Z) listed the same names and the same vram_gb values, plus Ascend 910B at 64, which is not in gpu-prices.json and is not listed here. The catalog response also includes an availability field. This page does not copy it. Every prices row has gpu_count 1. Every catalog row has min_gpu_count 1. Which listing you can start is on Deploy.
The catalog template list had no Pydantic AI template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as Pydantic AI.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
The installation page says the package requires Python 3.11 or newer. pyproject.toml says requires-python = ">=3.11" and license = "MIT". The version field is dynamic. The README data-extraction install, copied:
uv add pydantic-ai
docs/install.md at the same commit prints the source token pip/uv-add pydantic-ai and the line “(Requires Python 3.11+)”. The rendered page at https://pydantic.dev/docs/ai/install/, fetched 2026-11-14, prints two commands instead of that token:
pip install pydantic-ai
uv add pydantic-ai
PyPI info.version for pydantic-ai on 2026-11-14 was 2.68.0, requires_python >=3.11, info.license empty, uploaded 2026-11-12T02:30:01.931003Z. pydantic-ai-slim the same day was 2.68.0, uploaded 2026-11-12T02:30:07.590173Z. Tag v2.68.0 is commit d8bcca9ce06da5cdb998c8e15275cb1ae0865280, committed 2026-11-12T01:39:38Z. Main is commit fb64b4100c0882b4d5e1be8b9a38d7ae6b70c795, committed 2026-11-13T22:51:51Z. Those are not the same commit. The README, LICENSE, docs/install.md, and docs/models/ollama.md are the same blob on both. docs/models/compatible-apis.md is not. This page did not install the package.
The Ollama page says to install pydantic-ai, or pydantic-ai-slim with the openai extra. The source token and the rendered pair, fetched the same day:
pip install "pydantic-ai-slim[openai]"
uv add "pydantic-ai-slim[openai]"
Agent with structured output
The README data-extraction example defines Sentiment and passes output_type=Sentiment. The Agent constructor on that example sets the model string to a closed-model id. This page does not copy it. The class, copied:
from typing import Literal
from pydantic import BaseModel, Field
from pydantic_ai import Agent
class Sentiment(BaseModel):
label: Literal['positive', 'negative', 'neutral']
score: float = Field(ge=-1, le=1)
The Ollama page’s structured-output example uses an open-weight tag and NativeOutput. Copied:
from pydantic import BaseModel
from pydantic_ai import Agent
from pydantic_ai.models.ollama import OllamaModel
from pydantic_ai.output import NativeOutput
from pydantic_ai.providers.ollama import OllamaProvider
class CityLocation(BaseModel):
city: str
country: str
model = OllamaModel(
'qwen3',
provider=OllamaProvider(base_url='http://localhost:11434/v1'),
)
agent = Agent(model, output_type=NativeOutput(CityLocation))
That section says self-hosted Ollama v0.5.0 and later, released December 2024, enforces response_format with json_schema. This page did not run agent.run_sync.
Ollama and an OpenAI-compatible server
The Ollama page says self-hosted servers use http://localhost:11434/v1, and that OllamaModel uses the OpenAI-compatible Chat Completions API. The name form and the direct constructor, copied:
agent = Agent('ollama:qwen3')
model = OllamaModel('qwen3')
agent = Agent(model)
The environment lines, copied:
export OLLAMA_BASE_URL='http://localhost:11434/v1'
qwen3 is an Ollama tag. The files this page read do not print a Hugging Face repository id for it, and they do not print ollama pull or hf download. This page does not add one. The page also names a cloud base URL. This page does not copy that URL.
The vLLM section of the compatible-APIs page says vLLM is an OpenAI-compatible API and shows VLLMProvider. Copied:
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.vllm import VLLMProvider
model = OpenAIChatModel(
'Qwen/Qwen3.8-27B',
provider=VLLMProvider(base_url='http://localhost:8000/v1'),
)
agent = Agent(model)
result = agent.run_sync('What is the capital of France?')
print(result.output)
The same section also prints agent = Agent('vllm:Qwen/Qwen3.8-27B') when VLLM_BASE_URL is set, and VLLM_API_KEY for an authenticated server. A note says tool choice needs the server started with --enable-auto-tool-choice and a model-specific --tool-call-parser. The note does not print a parser value for this model id. Tag v2.68.0’s copy of this file is a different blob. The vLLM heading, the model id Qwen/Qwen3.8-27B, and http://localhost:8000/v1 are on that blob too. This page did not diff the rest of the file.
The same page’s other-endpoints section uses OpenAICompatibleProvider and a placeholder base_url. That placeholder does not print a numeric port. This page does not copy it. A later sentence on that page names a closed-model id. This page does not copy that id.
On 2026-11-14, GET https://huggingface.co/api/models/Qwen/Qwen3.8-27B reported gated false, cardData.license apache-2.0, pipeline_tag image-text-to-text, lastModified 2026-11-07T04:42:17.000Z, and sha 37561816e66b5b9153b1b64c10968c18c1e6f4a0. The card says “Number of Parameters: 27B” and prints no GPU GB figure. The block below is added on this page. The vLLM example does not print hf download. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. This page did not download the repository.
# Added on this page. The vLLM example passes this id to OpenAIChatModel.
pip install -U "huggingface_hub"
hf download Qwen/Qwen3.8-27B
The Pydantic AI files this page read do not print a vLLM serve command. How to serve an OpenAI-compatible API with vLLM on a cloud GPU is the server install. How to run Ollama on a cloud GPU is the Ollama install. The Ollama blocks print port 11434. The vLLM example prints port 8000.
For a server you start on the instance, the SSH tunnel is How to use SSH port forwarding on a cloud GPU.
When it fails
CUDA out of memory. The Pydantic AI files this page read print no GPU GB figure for the Ollama and vLLM examples. If the process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The installation page does not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this library on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
What to copy off, then destroy
agent.run_sync returns result.output in the process that called it. The vLLM example prints that value. Copy the printed output before you destroy the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The pricing page FAQ states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Prices are not copied here.
Frequently asked questions
What memory figure do the Pydantic AI docs print?
The README, the installation page, the Ollama page, and the vLLM section print no GPU GB figure. The Qwen3.8-27B card prints Number of Parameters: 27B and no GPU GB figure. This page does not copy the benchmark table on that card and does not treat 27B as vram_gb. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which install lines and classes do the docs print?
The README prints uv add pydantic-ai. The install page source prints the token pip/uv-add pydantic-ai and says Python 3.11 or newer. The rendered install page fetched 2026-11-14 prints pip install pydantic-ai and uv add pydantic-ai. PyPI info.version on 2026-11-14 was 2.68.0, requires_python >=3.11. Tag v2.68.0 and main are not the same commit. The README blob is the same on both. The Ollama structured example defines CityLocation and passes NativeOutput to Agent. The README Sentiment example sets the model string to a closed-model id. This page does not copy that id. This page did not install the package.
How do the docs point the agent at a local or OpenAI-compatible server?
The Ollama page sets model to qwen3 and base_url to http://localhost:11434/v1, and prints OLLAMA_BASE_URL with that URL. It says OllamaModel uses the OpenAI-compatible Chat Completions API. The vLLM section sets model to Qwen/Qwen3.8-27B and VLLMProvider base_url to http://localhost:8000/v1, and also prints the name vllm:Qwen/Qwen3.8-27B. A note names --enable-auto-tool-choice and --tool-call-parser and does not print a parser value. OpenAICompatibleProvider is the class for an endpoint the page has no provider for. Its example base_url is a placeholder and is not copied here. The files do not print a vLLM serve command.
What license does the file print?
The LICENSE file begins with The MIT License (MIT). Its copyright notice names Pydantic Services Inc. 2024 to present. The GitHub license field on 2026-11-14 was MIT. PyPI info.license for pydantic-ai was empty on 2026-11-14. That license text is not a CloudGPU price.
Sources (read 2026-11-14, Asia/Shanghai)
- pydantic/pydantic-ai README (
uv add pydantic-ai,Sentiment,output_type): commitfb64b4100c0882b4d5e1be8b9a38d7ae6b70c795, committed 2026-11-13T22:51:51Z. GitHub license fieldMIT.pushed_at2026-11-14T01:52:22Z.LICENSEcopyright notice names Pydantic Services Inc. 2024 to present. The README model string on theSentimentexample is not copied. The README also names other providers. This page does not list them - Tag
v2.68.0, commitd8bcca9ce06da5cdb998c8e15275cb1ae0865280, committed 2026-11-12T01:39:38Z. README,LICENSE,docs/install.md, anddocs/models/ollama.mdmatch main.docs/models/compatible-apis.mddoes not. The vLLM model id andhttp://localhost:8000/v1are on the tag file. This page did not diff the rest of that file. PyPIpydantic-aiinfo.version2.68.0, uploaded 2026-11-12T02:30:01.931003Z,requires_python>=3.11,info.licenseempty. PyPIpydantic-ai-slim2.68.0, uploaded 2026-11-12T02:30:07.590173Z docs/install.md(Python 3.11+, source tokenpip/uv-add pydantic-ai, slim extraopenai): same blob as the tag. Rendered page https://pydantic.dev/docs/ai/install/ fetched 2026-11-14 printspip install pydantic-aianduv add pydantic-ai, and the matching pair forpydantic-ai-slim[openai]docs/models/ollama.md(ollama:qwen3,OllamaModel,http://localhost:11434/v1,OLLAMA_BASE_URL,CityLocation,NativeOutput, Ollama v0.5.0): same blob as the tag. The cloud base URL on that page is not copieddocs/models/compatible-apis.md(VLLMProvider,Qwen/Qwen3.8-27B,http://localhost:8000/v1,vllm:Qwen/Qwen3.8-27B,--enable-auto-tool-choice,--tool-call-parser,OpenAICompatibleProvider): main commit above. The placeholder base URL and a closed-model id on that page are not copied- Hugging Face model API
Qwen/Qwen3.8-27B:gatedfalse,cardData.licenseapache-2.0,pipeline_tagimage-text-to-text,lastModified2026-11-07T04:42:17.000Z,sha37561816e66b5b9153b1b64c10968c18c1e6f4a0. The card prints Number of Parameters: 27B and no GPU GB figure. This page does not copy the benchmark table.hf downloadis added on this page and was not run - Public GPU memory: https://cloudgpu.app/gpu-prices.json (
updated_at2026-11-14T06:12:25.272Z) and https://cloudgpu.app/v1/public/gpu-catalog (generated_at2026-11-14T06:12:26.381Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no Pydantic AI template
Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.
Last updated: 2026-11-14 (Asia/Shanghai).