How to build LangChain agents with a self-hosted model on a cloud GPU

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install LangChain with the uv add langchain line the README prints, serve an ungated open-weight model with an OpenAI-compatible server on that machine, and pass a ChatOpenAI instance whose base_url is that server to create_agent. Copy the printed result before you destroy the instance, because destroying the instance deletes its disk.

An agent can rent the GPU and run this library against a model served on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the LangChain install. A graph runtime whose overview example does not name a model, and whose vLLM chat page is the same ChatOpenAI block, is How to run LangGraph agents with a self-hosted model on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. A gateway whose README prints uv tool install 'litellm[proxy]' is How to run LiteLLM Proxy in front of a self-hosted model on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090. A voice framework whose LLM page sets base_url on OpenAILLMService is How to run Pipecat voice agents with a self-hosted model on a cloud GPU. An agent session whose OpenAI-compatible page sets base_url and baseURL on openai.LLM is How to run LiveKit Agents with a self-hosted model on a cloud GPU. A Docker server whose 2.3 quickstart publishes port 80 and whose chat example posts to /v1/chat/completions is How to run GPUStack on a cloud GPU. A framework whose 2.0.11 quickstart pins agentscope==2.0.11, and whose OpenAICredential field base_url is passed to the chat client, is How to run AgentScope agents with a self-hosted model on a cloud GPU. A framework whose README comment sets model_server to http://localhost:8000/v1 and api_key to EMPTY is How to run Qwen-Agent with a self-hosted model on a cloud GPU. A library whose local-LLM page sets OpenAIGPTConfig chat_model and api_base for a vLLM-style endpoint is How to run Langroid agents with a self-hosted model on a cloud GPU. An agent whose Ollama page sets llm to an ollama/ tag, and whose default base URL text is http://localhost:11434/v1, is How to run PraisonAI agents with a self-hosted model on a cloud GPU. An agent whose OllamaPromptDriver sets model to llama3.1, and whose OpenAiChatPromptDriver example sets base_url to http://127.0.0.1:1234/v1, is How to run Griptape agents with a self-hosted model on a cloud GPU. An agent whose local branch sets base_url to http://localhost:11434/v1 and passes that client into AgentConfig is How to run Atomic Agents with a self-hosted model on a cloud GPU.

The LICENSE file begins with “MIT License”, and its copyright line is Copyright © LangChain, Inc.

The commands below are copied from the upstream files named in each step, except the model id marked as added. They were not run on a CloudGPU instance for this page. The providers page, the agents page, and the models page also name other providers and a hosted tracing product. This page does not list those names.

What the docs print about memory

The LangChain README, the providers-and-models page, the models page, the agents page, and the vLLM chat page print no GPU GB figure. The Qwen/Qwen3-8B README fetched on 2026-11-14 did not print a GPU GB figure in the text this page searched. This page does not mark a catalog row as enough.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T19:13:01.996Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Which listing you can start is on Deploy.

The catalog template list had no LangChain template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as LangChain.

Catalog name vram_gb
RTX 2080 8
RTX 3080 10
RTX 2080 Ti 11
RTX 3080 Ti 12
V100 16G 16
A30 24
RTX 3090 24
RTX 4090 24
RTX 4090D 24
RTX 4080 SUPER 32G 32
RTX 5090 32
RTX 5090D 32
V100 32G 32
A100 40G 40
A800 40G 40
A40 48
RTX 4090 48G 48
RTX 4090D 48G 48
RTX A6000 48
Ascend 910B 64
NVIDIA RTX PRO 5000 72G 72
RTX PRO 5000 72G 72
A100 80G 80
A800 80G 80
H800 80
RTX 6000D 83
H20 96G 96
RTX PRO 6000 96G 96
RTX PRO 6000 Max-Q 96
H20-3e 141G 141

Install

The repository README quickstart, on master commit ea17e48f0a4336c920394fd029de99f1568b4c38 (committed 2026-11-14T00:43:30Z), prints this line. Copied:

uv add langchain

The same quickstart then calls init_chat_model with a provider string. This page does not copy that string. libs/langchain_v1/pyproject.toml on that commit says name = "langchain" and version = "1.5.0". Its requires-python line is 3.11.0 or newer and below 4.0.0. The exact constraint, copied:

>=3.11.0,<4.0.0

The license field in that file is MIT. GitHub release langchain==1.5.0 is commit c2f4036e4c6ca45a8da8a42d8e07a25d2ea48b8f, published 2026-11-11T15:47:22Z. Master HEAD is a later commit than that release. The version string in libs/langchain_v1/pyproject.toml on master was still 1.5.0. PyPI langchain info.version on 2026-11-14 was 1.5.0, requires_python the same 3.11.0 bound, license MIT, uploaded 2026-11-11T15:47:07.964326Z. This page did not install the package. The GitHub license field on 2026-11-14 was MIT. pushed_at was 2026-11-14T08:44:18Z.

Point ChatOpenAI at the GPU server

The providers and models page (docs commit 72cc2978aadb8374cfffc97a87eab51276e74339 on src/oss/concepts/providers-and-models.mdx, committed 2026-11-11T17:49:38Z) says many providers offer endpoints compatible with the OpenAI Chat Completions API, and that you connect with ChatOpenAI and a custom base_url. The block it prints uses placeholder strings. Copied:

from langchain_openai import ChatOpenAI

model = ChatOpenAI(
    base_url="https://your-provider.com/v1",
    api_key="your-api-key",
    model="provider-model-name",
)

That page also prints a warning that ChatOpenAI targets the official API specification and does not keep non-standard response fields. The rest of the page names provider packages. This page does not list them.

The vLLM chat page (docs commit fd3330b80938e23ceb8d496492407548b3dcc953 on src/oss/python/integrations/chat/vllm.mdx, committed 2026-10-08T12:55:12Z) says vLLM can be deployed as a server that mimics the OpenAI API protocol. Its install line, copied:

pip install -qU langchain-openai

Its instantiation sets base_url to http://localhost:8000/v1, max_tokens to 5, and temperature to 0. The model string on that page is a repository id. On 2026-11-14, GET https://huggingface.co/api/models/mosaicml/mpt-7b returned HTTP 401, so this page does not copy that id, a license, a sha, or a gated flag, and it does not add hf download for it. The block below keeps the arguments the page prints and substitutes an id this page did verify. The comment is added on this page.

from langchain_openai import ChatOpenAI

inference_server_url = "http://localhost:8000/v1"

llm = ChatOpenAI(
    model="Qwen/Qwen3-8B",  # Added on this page. The vLLM chat page's model string is not copied. The Qwen function-calling page prints this id.
    api_key="your api key goes here",
    base_url=inference_server_url,
    max_tokens=5,
    temperature=0,
)

PyPI langchain-openai info.version on 2026-11-14 was 1.8.0, requires_python 3.11.0 or newer and below 4.0.0, uploaded 2026-11-13T16:48:10.728001Z. The vLLM page does not print that version. This page did not install the package. The vLLM page also prints a tracing setup. This page does not copy it. A general vLLM install, including the serve command that page does not print, is How to serve an OpenAI-compatible API with vLLM on a cloud GPU.

The download below is added on this page. The LangChain files this page read do not print this repository id. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. This page did not download the repository.

pip install -U "huggingface_hub"
hf download Qwen/Qwen3-8B

On 2026-11-14, GET https://huggingface.co/api/models/Qwen/Qwen3-8B reported gated false, private false, cardData.license apache-2.0, pipeline_tag text-generation, lastModified 2025-07-26T03:49:13.000Z, and sha b968826d9c46dd6066d109eabc6255188de91218. The model argument you pass to ChatOpenAI has to be the id the server actually exposes. This page did not run hf download.

For http://localhost:8000/v1, the tunnel is How to use SSH port forwarding on a cloud GPU.

Invoke and stream

The models page Invocation section (docs commit 3a2f25953af149b7642576790aec79bc5bba4233 on src/oss/langchain/models.mdx, committed 2026-11-13T15:01:00Z) prints these calls. Copied:

response = model.invoke("Why do parrots have colorful feathers?")
print(response)

for chunk in model.stream("Why do parrots have colorful feathers?"):
    print(chunk.text, end="|", flush=True)

That page says invoke() returns one message after generation finishes, and stream() yields chunks. The install blocks on that page name provider packages. This page does not copy those package names. The vLLM page’s invocation builds a SystemMessage and a HumanMessage, then calls llm.invoke(messages). This page does not copy that prompt.

A tool-calling agent

The agents page (docs commit ed49c697d8c55604b6de035a9022aaf7218f4db4 on src/oss/langchain/agents.mdx, committed 2026-11-12T16:46:54Z) says create_agent is the harness, and at its simplest you call create_agent. The model strings on that page name providers. This page does not copy those strings. The reference says model can be a string identifier or a direct chat model instance, and it names ChatOpenAI as that kind of instance. The tool in the block below is copied from the agents page. model=llm is added on this page so the call carries the ChatOpenAI instance above.

from langchain.agents import create_agent
from langchain.tools import tool

@tool
def search(query: str) -> str:
    """Search for information."""
    return f"Results for: {query}"

agent = create_agent(model=llm, tools=[search])

The vLLM chat page says specific model features, such as tool calling, depend on the hosted model. The vLLM tool-calling page says --enable-auto-tool-choice is mandatory, and that vLLM loads the parser named by --tool-call-parser only when that flag is set. The Qwen function-calling page says that for Qwen3 the chat template already includes Hermes-style tool use, and it prints this server command. Copied:

vllm serve Qwen/Qwen3-8B --enable-auto-tool-choice --tool-call-parser hermes --reasoning-parser deepseek_r1

create_agent with tools=[search] sends tool-enabled requests. Without those two flags, the server does not load a tool parser. This page did not run that serve command. The Qwen page’s client example names a client library. This page does not copy that library name.

The agents page Invocation section calls agent.invoke with a messages list. The create_agent call in that section uses a provider string this page does not copy, and the section’s tip names a hosted tracing product this page does not copy. The invoke it prints, copied:

result = agent.invoke(
    {"messages": [{"role": "user", "content": "What's the weather in San Francisco?"}]},
    config=config,
)

That section builds config as {"configurable": {"thread_id": str(uuid7())}} after from langchain_core.utils.uuid import uuid7, and it passes checkpointer=InMemorySaver() into create_agent. Those two lines are on the page. This page did not run the agent.

When it fails

CUDA out of memory. The LangChain files and chat pages this page read print no GPU GB figure. If the model server raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The README, the agents page, and the vLLM chat page do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this library on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

What to copy off, then destroy

model.invoke, model.stream, and agent.invoke return their result in the process that called them. The agents page does not name an output file. Copy the printed result before you destroy the instance. If you passed --local-dir to the added hf download, copy that directory too. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The pricing page FAQ states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Prices are not copied here.

Frequently asked questions

What memory figure do the LangChain docs print?

The LangChain README, the providers-and-models page, the models page, the agents page, and the vLLM chat page print no GPU GB figure. The Qwen/Qwen3-8B README fetched on 2026-11-14 did not print a GPU GB figure in the text this page searched. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Which install line does the README print?

The README prints uv add langchain. libs/langchain_v1/pyproject.toml on master says version 1.5.0 and requires-python 3.11.0 or newer and below 4.0.0. PyPI langchain info.version on 2026-11-14 was 1.5.0, uploaded 2026-11-11T15:47:07.964326Z. GitHub release langchain==1.5.0 is commit c2f4036e4c6ca45a8da8a42d8e07a25d2ea48b8f, published 2026-11-11T15:47:22Z. The README quickstart also calls init_chat_model with a provider string. This page does not copy that string. This page did not install the package.

How does ChatOpenAI point at a server on the GPU?

The providers page prints ChatOpenAI with base_url, api_key, and model. The vLLM chat page prints pip install -qU langchain-openai and sets base_url to http://localhost:8000/v1, max_tokens to 5, and temperature to 0. Its model string is a repository id this page does not copy, because the model API returned HTTP 401 on 2026-11-14. This page passes model Qwen/Qwen3-8B, which the model API reported gated false and license apache-2.0. The Qwen function-calling page prints vllm serve Qwen/Qwen3-8B with --enable-auto-tool-choice, --tool-call-parser hermes, and --reasoning-parser deepseek_r1. The vLLM tool-calling page says --enable-auto-tool-choice is mandatory before the parser loads. The model argument has to be the id the server exposes. The models page prints model.invoke and model.stream. The agents page builds create_agent with a tool named search. This page passes the ChatOpenAI instance as model. Those pages also name other providers. This page does not list those names.

What license does the file print?

The LICENSE file begins with MIT License, and its copyright line names LangChain, Inc. The GitHub license field on 2026-11-14 was MIT. PyPI license for langchain 1.5.0 was MIT.

Sources (read 2026-11-14, Asia/Shanghai)

  • langchain-ai/langchain README (uv add langchain; the init_chat_model argument is not copied): master commit ea17e48f0a4336c920394fd029de99f1568b4c38, committed 2026-11-14T00:43:30Z. pushed_at 2026-11-14T08:44:18Z. GitHub license field MIT. LICENSE copyright Copyright © LangChain, Inc.
  • libs/langchain_v1/pyproject.toml on that commit: name = "langchain", version = "1.5.0", requires-python 3.11.0 or newer and below 4.0.0, license MIT. GitHub release langchain==1.5.0 commit c2f4036e4c6ca45a8da8a42d8e07a25d2ea48b8f, published 2026-11-11T15:47:22Z. PyPI langchain 1.5.0, uploaded 2026-11-11T15:47:07.964326Z
  • Providers and models (ChatOpenAI, base_url, api_key, model): https://docs.langchain.com/oss/python/concepts/providers-and-models, fetched 2026-11-14. Docs commit 72cc2978aadb8374cfffc97a87eab51276e74339 on src/oss/concepts/providers-and-models.mdx, committed 2026-11-11T17:49:38Z. Other provider names on that page are not copied
  • vLLM chat page (pip install -qU langchain-openai, base_url http://localhost:8000/v1, max_tokens=5, temperature=0; tool calling depends on the hosted model): https://docs.langchain.com/oss/python/integrations/chat/vllm, fetched 2026-11-14. Docs commit fd3330b80938e23ceb8d496492407548b3dcc953, committed 2026-10-08T12:55:12Z. The page’s model string is not copied. GET https://huggingface.co/api/models/mosaicml/mpt-7b returned HTTP 401. PyPI langchain-openai 1.8.0, uploaded 2026-11-13T16:48:10.728001Z. The tracing setup is not copied
  • Models page (model.invoke, model.stream): https://docs.langchain.com/oss/python/langchain/models, fetched 2026-11-14. Docs commit 3a2f25953af149b7642576790aec79bc5bba4233 on src/oss/langchain/models.mdx, committed 2026-11-13T15:01:00Z. Provider install lines are not copied
  • Agents page (create_agent, @tool search, agent.invoke with a messages list and config): https://docs.langchain.com/oss/python/langchain/agents, fetched 2026-11-14. Docs commit ed49c697d8c55604b6de035a9022aaf7218f4db4 on src/oss/langchain/agents.mdx, committed 2026-11-12T16:46:54Z. Provider strings and the tracing tip are not copied. model=llm is added on this page. The reference says model accepts a chat model instance and names ChatOpenAI: https://reference.langchain.com/python/langchain/agents/create_agent, fetched 2026-11-14
  • Qwen function calling, vLLM section (vllm serve Qwen/Qwen3-8B --enable-auto-tool-choice --tool-call-parser hermes --reasoning-parser deepseek_r1): https://qwen.readthedocs.io/en/latest/framework/function_call.html, fetched 2026-11-14. The client library in that section is not copied
  • vLLM tool calling (--enable-auto-tool-choice mandatory; --tool-call-parser loads only when that flag is set): https://docs.vllm.ai/en/latest/features/tool_calling/ and docs/features/tool_calling.md commit 57988c08c27c273ef67816323aea507d4e229603, committed 2026-11-10T18:12:56Z. The quickstart model id on that page is not copied
  • Hugging Face model API Qwen/Qwen3-8B: gated false, private false, cardData.license apache-2.0, pipeline_tag text-generation, lastModified 2025-07-26T03:49:13.000Z, sha b968826d9c46dd6066d109eabc6255188de91218. The README search found no GPU GB figure. hf download Qwen/Qwen3-8B is added on this page and was not run
  • Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (generated_at 2026-11-14T19:13:01.996Z). GET /v1/public/compute-capabilities returned feature flags and no GPU list. The catalog had no LangChain template

Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.

Last updated: 2026-11-14 (Asia/Shanghai).