How to run LangGraph agents with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install LangGraph the way the official README prints, build the minimal graph the overview page prints, and point a node at ChatOllama or at the OpenAI-compatible base_url the vLLM chat page prints. Copy the printed result before you destroy the instance, because destroying the instance deletes its disk.
An agent can rent the GPU and run this library on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the LangGraph install. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. A coding-agent server whose card prints --tool-call-parser qwen3_coder is How to serve Qwen3-Coder for coding agents on a cloud GPU. A stateful CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A TypeScript agent whose local-models section sets url on the model object is How to run Mastra agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. A TurboMind server whose lmdeploy serve api_server listens on port 23333 is How to serve LLMs with LMDeploy on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose Ollama page sets base_url to http://localhost:11434/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A pipeline whose Ollama page sets url to http://localhost:11434, and whose vLLM page defaults api_base_url to http://localhost:8000/v1, is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b, and whose GPU-server example sets api_base to http://localhost:7501/v1, is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b, and whose VLLM class defaults base_url to http://localhost:8000/v1/, is How to run Agno agents with a self-hosted model on a cloud GPU. A Docker Compose app whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU.
The LangGraph LICENSE file in libs/langgraph begins with “MIT License”, its copyright notice names 2024 LangChain, Inc., and the GitHub license field on 2026-11-14 was MIT.
The commands below are copied from the upstream files named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The chat-model index also names other providers. This page does not list those names.
What the docs print about memory
The LangGraph README, the overview page, the quickstart, the ChatOllama page, and the vLLM chat page print no GPU GB figure. This page does not mark a catalog row as enough.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-14T04:12:38.192Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-14T04:12:39.202Z) listed the same names and the same vram_gb values, plus Ascend 910B at 64, which is not in gpu-prices.json and is not listed here. The catalog response also includes an availability field. This page does not copy it. Every prices row has gpu_count 1. Every catalog row has min_gpu_count 1. Which listing you can start is on Deploy.
The catalog template list had no LangGraph template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as LangGraph.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
The repository README, copied:
pip install -U langgraph
The library README at libs/langgraph/README.md prints uv add langgraph. libs/langgraph/pyproject.toml at release 1.2.22 says version = "1.2.22". That release is commit 08a2016953789a4f579b4d36bd1967941ec62012, published 2026-11-14T01:43:55Z, and the commit message is release(langgraph): 1.2.22. PyPI info.version for langgraph on 2026-11-14 was 1.2.22, requires_python 3.10 or newer, uploaded 2026-11-14T01:43:42.476889Z. The overview install block does not print a Python version. This page did not install the package.
The quickstart install line also names langchain and a provider package. This page does not copy that provider package name. The overview install, fetched 2026-11-14, is the two lines above: pip install -U langgraph and uv add langgraph.
A minimal graph
The overview (docs commit ed49c697d8c55604b6de035a9022aaf7218f4db4 on src/oss/langgraph/overview.mdx, committed 2026-11-12T16:46:54Z) prints this graph. It does not name a model. Copied:
from langgraph.graph import StateGraph, MessagesState, START, END
def mock_llm(state: MessagesState):
return {"messages": [{"role": "ai", "content": "hello world"}]}
graph = StateGraph(MessagesState)
graph.add_node(mock_llm)
graph.add_edge(START, "mock_llm")
graph.add_edge("mock_llm", END)
graph = graph.compile()
graph.invoke({"messages": [{"role": "user", "content": "hi!"}]})
The prebuilt README says create_react_agent takes model and tools. The call this page keeps, copied. The README’s model line constructs a closed-vendor class and a closed-model id. This page does not copy that class or that id:
from langgraph.prebuilt import create_react_agent
app = create_react_agent(model, tools)
app.invoke(
{"messages": [{"role": "user", "content": "what is the weather in sf"}]},
)
The quickstart’s init_chat_model argument is a closed-model id. This page does not copy it. The prebuilt README says the library is bundled with langgraph and that most users should install langgraph instead of installing langgraph-prebuilt directly. PyPI langgraph-prebuilt info.version on 2026-11-14 was 1.1.2, uploaded 2026-11-14T01:33:41.696482Z. This page did not install that package separately.
Ollama and an OpenAI-compatible server
The ChatOllama page (docs commit e404560b613ee2ad9bf4fe9077677a98beeec123 on src/oss/python/integrations/chat/ollama.mdx, committed 2026-10-22T15:16:00Z) says to install Ollama, then pull a model tag and pass that tag to ChatOllama. The page’s first example pulls a model tag with ollama pull and passes it to ChatOllama; later examples use llama3.1, gemma3:4b and granite3.2:8b. This page uses llama3.1 in both commands. That tag is one the log-probabilities section prints:
ollama pull llama3.1
The install line, copied:
pip install -qU langchain-ollama
from langchain_ollama import ChatOllama
llm = ChatOllama(
model="llama3.1",
temperature=0,
)
The first example sets temperature=0. A later example prints ollama pull llama3.1 and ChatOllama(model="llama3.1", logprobs=True). Other later examples print ChatOllama(model="gemma3:4b", temperature=0) and ChatOllama(model="granite3.2:8b"). The page says to use the same model tag in ollama pull and ChatOllama(model=...). It does not print a host or a port, and it does not print a GPU GB figure. PyPI langchain-ollama info.version on 2026-11-14 was 1.1.0, requires_python 3.10 or newer and below 4.0, uploaded 2026-04-07T02:47:59.642151Z. The page does not print that version. This page did not install it. The page does not print hf download or a Hugging Face repository id for llama3.1. This page does not add one.
The chat models page says you can use ChatOpenAI with a custom base_url for an endpoint compatible with the Chat Completions API. The vLLM chat page (docs commit fd3330b80938e23ceb8d496492407548b3dcc953 on src/oss/python/integrations/chat/vllm.mdx, committed 2026-10-08T12:55:12Z) prints pip install -qU langchain-openai and:
inference_server_url = "http://localhost:8000/v1"
llm = ChatOpenAI(
model="mosaicml/mpt-7b",
api_key="your api key goes here",
base_url=inference_server_url,
max_tokens=5,
temperature=0,
)
That page says vLLM can be deployed as a server that mimics the OpenAI API protocol. It does not print a GPU GB figure, and it does not print hf download. The block below is added on this page. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. On 2026-11-14, GET https://huggingface.co/api/models/mosaicml/mpt-7b returned HTTP 401 from this environment, so this page does not copy a license, a sha, or a gated flag for that id. This page did not download the repository.
# Added on this page. The vLLM chat page passes this id to ChatOpenAI. It does not print hf download.
pip install -U "huggingface_hub"
hf download mosaicml/mpt-7b
PyPI langchain-openai info.version on 2026-11-14 was 1.8.0, uploaded 2026-11-13T16:48:10.728001Z. The vLLM page does not print that version. A general vLLM install, including the serve command that page does not print, is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. Ollama’s port 11434 is How to run Ollama on a cloud GPU. This page does not add a port the ChatOllama page does not print.
For the vLLM page’s http://localhost:8000/v1, the tunnel is How to use SSH port forwarding on a cloud GPU.
When it fails
CUDA out of memory. The LangGraph files and chat pages this page read print no GPU GB figure. If the process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The README, the overview, and the two chat pages do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this library on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
What to copy off, then destroy
graph.invoke and app.invoke return their result in the process that called them. The overview example does not name an output file. Copy the printed result before you destroy the instance. If you passed --local-dir to the added hf download, copy that directory too. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The pricing page FAQ states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Prices are not copied here.
Frequently asked questions
What memory figure do the LangGraph docs print?
The LangGraph README, the overview page, the quickstart, the ChatOllama page, and the vLLM chat page print no GPU GB figure. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which install lines and graph example do the docs print?
The README prints pip install -U langgraph. The library README prints uv add langgraph. PyPI info.version on 2026-11-14 was 1.2.22, and requires_python is 3.10 or newer. pyproject.toml at release 1.2.22 says version 1.2.22. The overview builds a StateGraph with a mock_llm node that returns hello world. The prebuilt README calls create_react_agent with model and tools. Its model line is a closed-vendor class. This page does not copy that class or its model id. The quickstart install line names a provider package. This page does not copy that package name. This page did not install the package.
How do the chat integrations point at Ollama or an OpenAI-compatible server?
The first example on the ChatOllama page pulls a model tag with ollama pull and passes it to ChatOllama; later examples use llama3.1, gemma3:4b and granite3.2:8b. It does not print a host or a port. This page uses llama3.1 in the pull command and in ChatOllama. The page also prints pip install -qU langchain-ollama. PyPI langchain-ollama info.version on 2026-11-14 was 1.1.0. The page does not print that version. The vLLM chat page prints pip install -qU langchain-openai and ChatOpenAI with model mosaicml/mpt-7b and base_url http://localhost:8000/v1. The chat models page says ChatOpenAI can take a custom base_url for an OpenAI-compatible Chat Completions endpoint. Those pages also name other providers. This page does not list those names.
What license does the file print?
The LICENSE file begins with MIT License, its copyright notice names 2024 LangChain, Inc., and the GitHub license field on 2026-11-14 was MIT.
Sources (read 2026-11-14, Asia/Shanghai)
- langchain-ai/langgraph README (
pip install -U langgraph) andlibs/langgraph/README.md(uv add langgraph): commit08a2016953789a4f579b4d36bd1967941ec62012, committed 2026-11-14T01:25:56Z, releaselanggraph==1.2.22published 2026-11-14T01:43:55Z.libs/langgraph/pyproject.tomlversion1.2.22. GitHub license fieldMIT.libs/langgraph/LICENSEcopyright notice names 2024 LangChain, Inc.pushed_at2026-11-14T01:43:55Z libs/prebuilt/README.md(create_react_agent(model, tools),app.invoke): same commit. The example model class and model id are not copied. The file sayslanggraph-prebuiltis bundled withlanggraph. PyPIlanggraph-prebuiltinfo.version1.1.2, uploaded 2026-11-14T01:33:41.696482Z- PyPI
langgraphinfo.version1.2.22, uploaded 2026-11-14T01:43:42.476889Z,requires_python3.10 or newer - LangGraph overview (
StateGraph,mock_llm,hello world): https://docs.langchain.com/oss/python/langgraph/overview, fetched 2026-11-14. Docs commited49c697d8c55604b6de035a9022aaf7218f4db4onsrc/oss/langgraph/overview.mdx, committed 2026-11-12T16:46:54Z - LangGraph quickstart (install line names a provider package;
init_chat_modelargument is a closed-model id): https://docs.langchain.com/oss/python/langgraph/quickstart, fetched 2026-11-14. This page does not copy that package name or that model id - ChatOllama page (the first example pulls a model tag with
ollama pulland passes it toChatOllama; later examples usellama3.1,gemma3:4bandgranite3.2:8b;pip install -qU langchain-ollama; no host, no port, no GPU GB figure). This page usesllama3.1in the pull command and inChatOllama: https://docs.langchain.com/oss/python/integrations/chat/ollama, fetched 2026-11-14. Docs commite404560b613ee2ad9bf4fe9077677a98beeec123onsrc/oss/python/integrations/chat/ollama.mdx, committed 2026-10-22T15:16:00Z. PyPIlangchain-ollama1.1.0, uploaded 2026-04-07T02:47:59.642151Z. The page does not print that version - Chat models page (custom
base_urlonChatOpenAIfor a Chat Completions-compatible endpoint): https://docs.langchain.com/oss/python/integrations/chat, fetched 2026-11-14. The page also names other providers. This page does not list them - vLLM chat page (
pip install -qU langchain-openai,model="mosaicml/mpt-7b",base_urlhttp://localhost:8000/v1,max_tokens=5): https://docs.langchain.com/oss/python/integrations/chat/vllm, fetched 2026-11-14. Docs commitfd3330b80938e23ceb8d496492407548b3dcc953onsrc/oss/python/integrations/chat/vllm.mdx, committed 2026-10-08T12:55:12Z. PyPIlangchain-openai1.8.0, uploaded 2026-11-13T16:48:10.728001Z. The page does not print that version.hf download mosaicml/mpt-7bis added on this page. The model API returned HTTP 401 on 2026-11-14 - Public GPU memory: https://cloudgpu.app/gpu-prices.json (
updated_at2026-11-14T04:12:38.192Z) and https://cloudgpu.app/v1/public/gpu-catalog (generated_at2026-11-14T04:12:39.202Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no LangGraph template
Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.
Last updated: 2026-11-14 (Asia/Shanghai).