How to run Mastra agents with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Mastra the way the official README prints, serve a model on that machine, and set the Agent model url to the OpenAI-compatible base URL the local-models section prints. Copy response.text before you destroy the instance, because destroying the instance deletes its disk.
An agent can rent the GPU and run this library on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Mastra install. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose OllamaChatGenerator sets url to http://localhost:11434 is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b is How to run Agno agents with a self-hosted model on a cloud GPU. A stateful CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. An embedding server is How to serve embeddings and rerankers with Text Embeddings Inference on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090. A Docker Compose app whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU.
LICENSE.md says content outside directories named ee/ is available under the Apache License 2.0, the copyright line is “Copyright © 2025 Kepler Software, Inc.”, and packages/core/package.json sets license to Apache-2.0. The GitHub license field on 2026-11-14 was NOASSERTION.
The commands below are copied from the upstream files named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The README and the models pages name other providers. This page does not list those names.
What the docs print about memory
The README, the manual-install page, the agents overview, the local-models section, and the Ollama provider page print no GPU GB figure. The local-models section does not print a Hugging Face repository id. The string lmstudio/qwen/qwen3-30b-a3b-2507 is a router id on that page, not a line this page treats as an hf download argument. This page does not add hf download. This page does not mark a catalog row as enough. A context-window number on the LM Studio provider page is not vram_gb. This page does not copy that table.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-14T08:11:19.550Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-14T08:11:20.588Z) listed the same names and the same vram_gb values, plus Ascend 910B at 64, which is not in gpu-prices.json and is not listed here. The catalog response also includes an availability field. This page does not copy it. Every prices row has gpu_count 1. Every catalog row has min_gpu_count 1. Which listing you can start is on Deploy.
The catalog template list had no Mastra template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as Mastra.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
The README recommended command, copied:
npm create mastra@latest
An alternative prompt later in that README names provider options this page does not copy. That prompt also prints Studio at http://localhost:4111. The recommended section above the prompt does not print a port. npm create-mastra latest on 2026-11-14 was 1.39.0, license Apache-2.0, published 2026-11-12T21:20:22.234Z. npm mastra latest the same day was 1.39.0, license Apache-2.0, published 2026-11-12T21:27:00.621Z. Both set engines.node to >=22.13.0.
The manual-install page prints a second path. The install lines, copied:
mkdir my-first-agent && cd my-first-agent
npm init
npm install -D typescript @types/node mastra@latest
npm install @mastra/core@latest zod@^4
That page then sets package.json scripts dev to mastra dev and build to mastra build, and prints npm run dev. It does not print port 4111. Its agent example sets model to a placeholder token this page does not copy. The same page names other providers in a note. This page does not list those names.
packages/core/package.json on tag @mastra/core@1.83.0 (commit d60ba71be015e2acbdc303208a14563902fb1f39, committed 2026-11-12T21:12:29Z) sets version to 1.83.0, license to Apache-2.0, and engines.node to >=22.13.0. npm @mastra/core latest on 2026-11-14 was 1.83.0, license Apache-2.0, published 2026-11-12T21:27:47.471Z. Main is commit 7277766d6c967c475675d58ff2d0e5c6d7be7c84, committed 2026-11-14T04:01:42Z. Main packages/core/package.json sets version to 1.84.0-alpha.1 and the same node engine. Those version fields are not the same. The README, LICENSE.md, the agents overview, the manual-install page, docs/src/content/en/models/index.mdx, and the Ollama provider page are the same blob on the tag and on main. This page did not install the packages.
Agent
The local-models section of docs/src/content/en/models/index.mdx says a local model server needs to provide an OpenAI-compatible API, and that url must be the base URL of that endpoint, not a chat path. It says provider/model is the form when the remote expects a bare model name such as llama3.2. The first example, copied:
import { Agent } from "@mastra/core/agent";
const agent = new Agent({
id: "my-agent",
name: "My Agent",
instructions: "You are a helpful assistant",
model: {
id: "custom/my-qwen3-model",
url: "http://your-custom-openai-compatible-endpoint.com/v1"
}
})
That URL is the placeholder the file prints. It is not a numeric port. The examples above do not pass apiKey. The custom-gateways page says a single OpenAI-compatible endpoint does not need a gateway: set url, and set apiKey when the endpoint requires authentication. The LM Studio example on the local-models section, copied:
import { Agent } from "@mastra/core/agent";
const agent = new Agent({
id: "my-agent",
name: "My Agent",
instructions: "You are a helpful assistant",
model: {
id: "lmstudio/qwen/qwen3-30b-a3b-2507",
url: "http://localhost:1234/v1"
}
})
The section says that after the LM Studio server starts, the local server is at http://localhost:1234 and the url is http://localhost:1234/v1. A later example on the same page sets id to a closed-vendor string. This page does not copy that id. The opening sentence of the section also names another model id this page does not copy, plus Qwen3 and DeepSeek. 30b inside the router id is not vram_gb.
The agents overview prints new Agent with id, name, instructions, and model, then:
const agent = mastra.getAgentById('test-agent')
const response = await agent.generate('Help me organize my day')
console.log(response.text)
The overview model value is a placeholder token this page does not copy. This page did not run generate.
Ollama and an OpenAI-compatible server
The Ollama provider page says model strings shaped like ollama and then a model name do not resolve through the model router. It says to pass a model instance from ollama-ai-provider-v2 to the agent model option. The install line, copied:
npm install ollama-ai-provider-v2
That page does not print a base URL, a port, or a model id. It links the AI SDK Ollama provider page. That linked page, fetched 2026-11-14, is not a file in the Mastra repository. It prints default prefix http://localhost:11434/api, createOllama, and ollama('qwen3:4b'). qwen3:4b and llama3.2 are not the same string. http://localhost:11434/api and http://localhost:1234/v1 are not the same URL. npm ollama-ai-provider-v2 latest on 2026-11-14 was 4.1.0, license Apache-2.0, engines.node >=22, published 2026-10-22T14:40:51.722Z. This page did not install it. How to run Ollama on a cloud GPU is the Ollama install.
The local-models section does not name vLLM and does not print port 8000. How to serve an OpenAI-compatible API with vLLM on a cloud GPU is the general server install. Put that server base URL in url only as the local-models section describes: the base URL, not a chat path. The files this page read do not print an SSH command. The tunnel is How to use SSH port forwarding on a cloud GPU.
When it fails
CUDA out of memory. The Mastra pages this page read print no GPU GB figure. If the process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The README, LICENSE.md, and packages/core/package.json do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this library on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
What to copy off, then destroy
The overview prints response.text from agent.generate. Copy that text before you destroy the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The pricing page FAQ states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Prices are not copied here.
Frequently asked questions
What memory figure do the Mastra docs print?
The README, the manual-install page, the agents overview, the local-models section, and the Ollama provider page print no GPU GB figure. The local-models section does not print a Hugging Face repository id. This page does not add hf download. A context-window number on the LM Studio provider page is not vram_gb. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which install lines and class do the docs print?
The README prints npm create mastra@latest. npm create-mastra and npm mastra on 2026-11-14 were 1.39.0, engines.node >=22.13.0. The manual-install page prints npm install @mastra/core@latest and npm install -D typescript @types/node mastra@latest, then npm run dev. Tag @mastra/core@1.83.0 is commit d60ba71be015e2acbdc303208a14563902fb1f39. npm @mastra/core on 2026-11-14 was 1.83.0. Main sets version 1.84.0-alpha.1. Those version fields are not the same. The docs blobs named on this page match on both commits. The Agent class is imported from @mastra/core/agent. This page did not install the packages.
How do the docs point the agent at a local or OpenAI-compatible server?
The local-models section sets model.url to the OpenAI-compatible base URL, not a chat path. The first example id is custom/my-qwen3-model and the url is http://your-custom-openai-compatible-endpoint.com/v1. The LM Studio example id is lmstudio/qwen/qwen3-30b-a3b-2507 and the url is http://localhost:1234/v1. The Ollama provider page prints npm install ollama-ai-provider-v2 and does not print a port. The AI SDK page it links prints http://localhost:11434/api and ollama qwen3:4b. Those URLs are not the same. The local-models section does not name vLLM and does not print port 8000. The files do not print an SSH command.
What license does the file print?
LICENSE.md says content outside ee directories is Apache License 2.0. The copyright line names 2025 Kepler Software, Inc. packages/core/package.json sets license to Apache-2.0. The GitHub license field on 2026-11-14 was NOASSERTION. npm license for @mastra/core 1.83.0 was Apache-2.0. That license text is not a CloudGPU price.
Sources (read 2026-11-14, Asia/Shanghai)
- mastra-ai/mastra README (
npm create mastra@latest): commit7277766d6c967c475675d58ff2d0e5c6d7be7c84, committed 2026-11-14T04:01:42Z. The same README blob is on tag@mastra/core@1.83.0. The alternative prompt printshttp://localhost:4111and names other providers. This page does not list those names.LICENSE.mdsays content outsideee/is Apache License 2.0. The copyright line is Copyright © 2025 Kepler Software, Inc. GitHub license fieldNOASSERTION.pushed_at2026-11-14T07:57:47Z - Tag
@mastra/core@1.83.0, commitd60ba71be015e2acbdc303208a14563902fb1f39, committed 2026-11-12T21:12:29Z.packages/core/package.jsonversion1.83.0, licenseApache-2.0,engines.node>=22.13.0. Main version field is1.84.0-alpha.1with the same node engine. npm@mastra/core1.83.0, published 2026-11-12T21:27:47.471Z. npmmastra1.39.0, published 2026-11-12T21:27:00.621Z. npmcreate-mastra1.39.0, published 2026-11-12T21:20:22.234Z. All three npm licenses were Apache-2.0 docs/src/content/en/reference/manual-install.mdx(install lines,mastra dev,npm run dev): same blob on the tag and main. The model placeholder and the provider note are not copied. The page does not print port 4111docs/src/content/en/docs/agents/overview.mdx(Agentfrom@mastra/core/agent,getAgentById,generate,response.text): same blob on the tag and main. The overview model placeholder is not copieddocs/src/content/en/models/index.mdxlocal-models section (url,custom/my-qwen3-model,llama3.2,lmstudio/qwen/qwen3-30b-a3b-2507,http://localhost:1234/v1): same blob on the tag and main. The section does not name vLLM and does not print port 8000. A later id names a closed vendor. This page does not copy it.30bin the router id is notvram_gbdocs/src/content/en/models/providers/ollama.mdx(npm install ollama-ai-provider-v2): same blob on the tag and main. It does not print a base URL. The linked AI SDK page https://ai-sdk.dev/providers/community-providers/ollama, fetched 2026-11-14, printshttp://localhost:11434/apiandqwen3:4b. npmollama-ai-provider-v24.1.0, published 2026-10-22T14:40:51.722Z, license Apache-2.0,engines.node>=22docs/src/content/en/models/gateways/custom-gateways.mdxsays a single OpenAI-compatible endpoint does not need a gateway: seturl, and setapiKeywhen the endpoint requires authentication. Same blob on the tag and main. That page also names other hosts and a per-token comparison. This page does not copy those sentences- Public GPU memory: https://cloudgpu.app/gpu-prices.json (
updated_at2026-11-14T08:11:19.550Z) and https://cloudgpu.app/v1/public/gpu-catalog (generated_at2026-11-14T08:11:20.588Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no Mastra template
Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.
Last updated: 2026-11-14 (Asia/Shanghai).