How to use the Vercel AI SDK with a self-hosted model on a cloud GPU

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install the Vercel AI SDK and the OpenAI-compatible provider package the way the official repository prints, serve an open-weight model on that machine, and set baseURL on createOpenAICompatible to that server. Copy the text the call prints before you destroy the instance, because destroying the instance deletes its disk and stops the server.

An agent can rent the GPU, serve the model, and call it from this SDK. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Vercel AI SDK setting. A gateway whose README prints uv tool install 'litellm[proxy]' is How to run LiteLLM Proxy in front of a self-hosted model on a cloud GPU. An agent whose local sample sets base_url to http://localhost:11434/v1 is How to run Semantic Kernel agents with a self-hosted model on a cloud GPU. An answering engine whose Docker command publishes port 3000, and whose Ollama base URL placeholder is http://localhost:11434, is How to run Perplexica AI search with a self-hosted model on a cloud GPU. An assistant whose setup page opens http://localhost:42110, and whose Ollama page sets OPENAI_BASE_URL to http://localhost:11434/v1/, is How to run Khoj AI assistant with a self-hosted model on a cloud GPU. An editor whose Ollama config sets baseURL to http://localhost:11434/v1 is How to use Kilo Code with a self-hosted model on a cloud GPU. An editor whose local-model page sets the Ollama api_url to http://localhost:11434 is How to use Zed’s AI agent with a self-hosted model on a cloud GPU. A coding assistant whose Docker command publishes port 8080 and passes --model and --chat-model is How to run Tabby self-hosted coding assistant on a cloud GPU. A terminal client whose OpenAI-compatible page sets OPENAI_API_BASE is How to use Aider with a self-hosted model on a cloud GPU. An editor client whose install page is the VS Code extension is How to use Cline with a self-hosted model on a cloud GPU. An editor whose config.yaml sets an apiBase for chat, autocomplete, and embeddings is How to use Continue with a self-hosted model on a cloud GPU. An editor whose Ollama provider defaults to http://localhost:11434 is How to use Roo Code with a self-hosted model on a cloud GPU. A coding agent whose providers page sets package to the OpenAI-compatible provider is How to use OpenCode with a self-hosted model on a cloud GPU. A platform whose single-container guide sets CHAT_BASE_URL to http://host.docker.internal:11434/v1 for a local OpenAI-compatible server is How to run AutoGPT with a self-hosted model on a cloud GPU. An agent whose Docker command publishes port 5080, and whose Ollama provider sets api_base to http://host.docker.internal:11434, is How to run Agent Zero with a self-hosted model on a cloud GPU. An agent whose Ollama example sets base_url to http://localhost:11434/v1 is How to run OpenManus with a self-hosted model on a cloud GPU. A chat app whose Docker page runs docker compose up -d and whose Ollama page sets baseURL to http://localhost:11434/v1/ is How to run LibreChat agents with a self-hosted model on a cloud GPU. A chat app whose README runs docker compose up -d and whose environment page sets OLLAMA_PROXY_URL to http://127.0.0.1:11434 is How to run LobeChat agents with a self-hosted model on a cloud GPU. A terminal agent whose install page runs curl -fsSL https://www.openinterpreter.com/install | sh, and whose models page sets the Ollama base URL to http://localhost:11434/v1, is How to run Open Interpreter with a self-hosted model on a cloud GPU. A workspace app whose Docker quickstart publishes port 3001, and whose Ollama page sets the URL to http://127.0.0.1:11434, is How to run AnythingLLM agents with a self-hosted model on a cloud GPU. A repository agent whose local-models section sets api_base to http://localhost:11434 is How to run SWE-agent with a self-hosted model on a cloud GPU. A workflow editor whose Docker install publishes port 5678, and whose Ollama credentials default the Base URL to http://localhost:11434, is How to run n8n AI agents with a self-hosted model on a cloud GPU. A CLI whose providers page sets OLLAMA_HOST to http://localhost:11434 is How to run the goose agent with a self-hosted model on a cloud GPU. A canvas whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU. A CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A browser chat whose serve command listens on port 8080 is How to run Open WebUI on a cloud GPU. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose OllamaChatGenerator sets url to http://localhost:11434 is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b is How to run Agno agents with a self-hosted model on a cloud GPU. A TypeScript agent whose local-models section sets url on the model object is How to run Mastra agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. The README install is pip install --upgrade metagpt: How to run MetaGPT with a self-hosted model on a cloud GPU. A coding-agent server whose card prints --tool-call-parser qwen3_coder is How to serve Qwen3-Coder for coding agents on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the server raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.

The LICENSE file is the Apache License, Version 2.0, and the copyright line is Copyright 2023 Vercel, Inc.

The steps below are copied from the upstream files named in each step, at the commit in Sources. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The README names other provider packages and prints example model strings. This page does not copy those names or those strings. The compatible-providers index lists other providers. This page does not copy that list.

What the docs print about memory

packages/ai/README.md, content/providers/04-openai-compatible-providers/index.mdx, content/docs/03-ai-sdk-core/05-generating-text.mdx, content/docs/03-ai-sdk-core/15-tools-and-tool-calling.mdx, and content/docs/03-agents/02-building-agents.mdx print no GPU GB figure. This page does not mark a catalog row as enough. The SDK is a client. The server’s memory is a different page: How much VRAM do I need to run an LLM?.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T18:05:24.586Z). The catalog response also includes an availability field and a price field. This page does not copy them. Every catalog row below has min_gpu_count 1. Rows are sorted by vram_gb, then by name. Which listing you can start is on Deploy.

The catalog template list had no Vercel AI SDK template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as this SDK.

Catalog name vram_gb
RTX 2080 8
RTX 3080 10
RTX 2080 Ti 11
RTX 3080 Ti 12
V100 16G 16
A30 24
RTX 3090 24
RTX 4090 24
RTX 4090D 24
RTX 4080 SUPER 32G 32
RTX 5090 32
RTX 5090D 32
V100 32G 32
A100 40G 40
A800 40G 40
A40 48
RTX 4090 48G 48
RTX 4090D 48G 48
RTX A6000 48
Ascend 910B 64
NVIDIA RTX PRO 5000 72G 72
RTX PRO 5000 72G 72
A100 80G 80
A800 80G 80
H800 80
RTX 6000D 83
H20 96G 96
RTX PRO 6000 96G 96
RTX PRO 6000 Max-Q 96
H20-3e 141G 141

Install

Repository HEAD was 0e8e0543aeda7d4dd4ce6eade0d5b7d5a43644c1, committed 2026-11-13T22:28:29Z. packages/ai/package.json at that commit sets version to 7.0.173 and license to Apache-2.0. packages/openai-compatible/package.json sets name to @ai-sdk/openai-compatible, version to 3.0.92, and license to Apache-2.0. The npm registry on 2026-11-14 reported the same two versions.

packages/ai/README.md says you need Node.js 22 or later. Its install, copied:

npm install ai

content/providers/04-openai-compatible-providers/index.mdx says the provider is the @ai-sdk/openai-compatible module and renders InstallPackages with packages="@ai-sdk/openai-compatible". apps/docs/components/docs/install-packages.tsx at this commit turns that into four commands. Copied:

pnpm add @ai-sdk/openai-compatible
npm install @ai-sdk/openai-compatible
yarn add @ai-sdk/openai-compatible
bun add @ai-sdk/openai-compatible

The pages this guide read do not print hf download or huggingface-cli. Weight download for the model server is a different guide: How to download a Hugging Face model.

Point createOpenAICompatible at the server

The provider-instance example in index.mdx, copied:

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';

const provider = createOpenAICompatible({
  name: 'providerName',
  apiKey: process.env.PROVIDER_API_KEY,
  baseURL: 'https://api.provider.com/v1',
  includeUsage: true, // Include usage information in streaming responses
});

The same file says baseURL is the URL prefix for API calls. A note on the fetch setting names a local base URL, copied as the string it prints: http://localhost:11434/v1. That note is about a guarded fetch rejecting a private-network baseURL unless the origin is in trustedOrigins. It is not an install command for a model server.

The model factory on that page, copied:

const model = provider('model-id');

model-id is the string that page prints. This page did not replace it.

None of these files print an ssh command. How to use SSH port forwarding on a cloud GPU is how a program on your computer reaches a port on the instance. This page does not add an ssh command. After a forward that lands on port 11434 of your computer, the local base URL the fetch note already prints is http://localhost:11434/v1. This page does not invent a different port. A vLLM server’s listen address is a different document: How to serve an OpenAI-compatible API with vLLM on a cloud GPU. Ollama’s port 11434 is How to run Ollama on a cloud GPU.

generateText, streamText, and a tool loop

The generateText example on the compatible-providers page, copied:

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { generateText } from 'ai';

const provider = createOpenAICompatible({
  name: 'providerName',
  apiKey: process.env.PROVIDER_API_KEY,
  baseURL: 'https://api.provider.com/v1',
});

const { text } = await generateText({
  model: provider('model-id'),
  prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});

content/docs/03-ai-sdk-core/05-generating-text.mdx prints streamText with two placeholder tokens, __PROVIDER_IMPORT__ and __MODEL__. Those tokens are the strings in the file. This page does not fill them in. Copied:

import { streamText } from 'ai';
__PROVIDER_IMPORT__;

const result = streamText({
  model: __MODEL__,
  prompt: 'Invent a new holiday and describe its traditions.',
});

// example: use textStream as an async iterable
for await (const textPart of result.textStream) {
  console.log(textPart);
}

The same file says streamText immediately starts streaming and suppresses errors, and that you use the onError callback to log errors. A sentence on that page about how long a response can take is not copied.

The tool-calling page’s multi-step example also uses __PROVIDER_IMPORT__ and __MODEL__. Copied:

import { z } from 'zod';
import { generateText, tool, isStepCount } from 'ai';
__PROVIDER_IMPORT__;

const { text, steps } = await generateText({
  model: __MODEL__,
  tools: {
    weather: tool({
      description: 'Get the weather in a location',
      inputSchema: z.object({
        location: z.string().describe('The location to get the weather for'),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72 + Math.floor(Math.random() * 21) - 10,
      }),
    }),
  },
  stopWhen: isStepCount(5), // stop after a maximum of 5 steps if tools were called
  prompt: 'What is the weather in San Francisco?',
});

That page says the default stop condition for generateText and streamText is isStepCount(1), and that stopWhen is evaluated when the last step contains tool results. It says you can use streamText in a similar way. The temperature expression is the expression that file prints. This page did not run it.

content/docs/03-agents/02-building-agents.mdx constructs ToolLoopAgent with the same two placeholders. The tools block, copied:

import { ToolLoopAgent, tool } from 'ai';
__PROVIDER_IMPORT__;
import { z } from 'zod';

const codeAgent = new ToolLoopAgent({
  model: __MODEL__,
  tools: {
    runCode: tool({
      description: 'Execute Python code',
      inputSchema: z.object({
        code: z.string(),
      }),
      execute: async ({ code }) => {
        // Execute code and return result
        return { output: 'Code executed successfully' };
      },
    }),
  },
});

The compatible-providers page says chat models created with this provider support tool calling. Whether a tool call comes back depends on the server. The vLLM tool-calling page at commit bb23c0a81779ba733923a7dab9b840b4e533f6a5 starts the server with --enable-auto-tool-choice and --tool-call-parser. The model id on that command is not copied. This page did not re-test that flag. A latency sentence on that vLLM page is not copied.

When it fails

Placeholder tokens. 05-generating-text.mdx and 15-tools-and-tool-calling.mdx print __PROVIDER_IMPORT__ and __MODEL__. Those are not a model id. The filled-in call is the provider('model-id') example on the compatible-providers page.

Private-network base URL. The fetch note says a guarded fetch rejects a baseURL on a private or local network, and the example it prints is http://localhost:11434/v1, unless that origin is in trustedOrigins.

CUDA out of memory. The pages this guide read print no GPU GB figure. If the model server raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller id for you.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The pages this guide read do not pin a PyTorch build and do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120). This page did not re-test a model server on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

What to copy off, then destroy

generateText returns text. streamText yields result.textStream. The tool example also returns steps. Copy that output before you destroy the instance. The model server stops with the instance. These files do not print an output directory. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The billing page states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Amounts are not copied here.

Frequently asked questions

What memory figure do the Vercel AI SDK pages print?

The AI SDK README, the OpenAI-compatible providers page, the generating-text page, the tool-calling page, and the building-agents page print no GPU GB figure. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Which call sets the base URL?

createOpenAICompatible takes baseURL. The provider-instance example sets it to https://api.provider.com/v1 and name to providerName. A fetch note on the same page prints http://localhost:11434/v1 as a local base URL. The model call is provider(‘model-id’). Those files do not print an SSH command. This page did not run them.

Where is the documented tool-calling loop?

The tool-calling page calls generateText with a weather tool and stopWhen set to isStepCount(5). The building-agents page constructs ToolLoopAgent with a runCode tool. Both files print the placeholders PROVIDER_IMPORT and MODEL. The compatible-providers page says chat models from this provider support tool calling. The vLLM tool-calling page starts the server with --enable-auto-tool-choice and --tool-call-parser. The model id on that command is not copied. This page did not re-test it.

What license sentence does the file print?

The LICENSE file is the Apache License, Version 2.0, and the copyright line is Copyright 2023 Vercel, Inc. packages/ai/package.json at this commit sets version to 7.0.173 and license to Apache-2.0. packages/openai-compatible/package.json sets version to 3.0.92 and license to Apache-2.0. npm on 2026-11-14 reported those same versions. Repository HEAD was 0e8e0543aeda7d4dd4ce6eade0d5b7d5a43644c1, committed 2026-11-13T22:28:29Z. That license text is not a CloudGPU rate.

Sources (read 2026-11-14, Asia/Shanghai)

  • vercel/ai commit 0e8e0543aeda7d4dd4ce6eade0d5b7d5a43644c1, committed 2026-11-13T22:28:29Z. LICENSE is the Apache License, Version 2.0, Copyright 2023 Vercel, Inc.
  • packages/ai/package.json version 7.0.173, license Apache-2.0. packages/openai-compatible/package.json name @ai-sdk/openai-compatible, version 3.0.92, license Apache-2.0. npm latest on 2026-11-14 matched both versions
  • packages/ai/README.md at that commit. Node.js 22 or later. npm install ai. The provider list and the example model strings are not copied
  • content/providers/04-openai-compatible-providers/index.mdx at that commit. createOpenAICompatible, baseURL https://api.provider.com/v1, local example http://localhost:11434/v1, provider('model-id'), the lasagna generateText example. The provider list on that page is not copied. A refusal sentence names a hosted provider. This page does not copy that name
  • apps/docs/components/docs/install-packages.tsx at that commit. pnpm add, npm install, yarn add, bun add
  • content/docs/03-ai-sdk-core/05-generating-text.mdx at that commit. streamText with __PROVIDER_IMPORT__ and __MODEL__. Prompt Invent a new holiday and describe its traditions. A duration sentence is not copied
  • content/docs/03-ai-sdk-core/15-tools-and-tool-calling.mdx at that commit. weather tool, stopWhen: isStepCount(5), default isStepCount(1). The temperature expression is copied as printed
  • content/docs/03-agents/02-building-agents.mdx at that commit. ToolLoopAgent and the runCode tool. Placeholders are not filled in
  • vllm-project/vllm commit bb23c0a81779ba733923a7dab9b840b4e533f6a5, docs/features/tool_calling.md. --enable-auto-tool-choice and --tool-call-parser. The model id and a latency sentence are not copied
  • The AI SDK files above do not print hf download or an ssh command
  • Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (generated_at 2026-11-14T18:05:24.586Z). GET /v1/public/compute-capabilities returned feature flags and no GPU list. The catalog had no Vercel AI SDK template. It listed Ollama template ids jupyterlab-ollama, ollama-qwen3, and ollama

Cards you can start are on the Deploy page. Amounts are not copied here.

Last updated: 2026-11-14 (Asia/Shanghai).