How to run Griptape agents with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Griptape with the core uv add griptape line the framework page prints, serve an ungated open-weight model on that machine, and pass OllamaPromptDriver or OpenAiChatPromptDriver with base_url into Agent. Copy the printed run output before you destroy the instance, because destroying the instance deletes its disk.
An agent can rent the GPU and run this library against a model served on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Griptape install. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. An OpenAI-compatible engine whose CUDA 12 container tag is localai/localai:latest-gpu-nvidia-cuda-12 is How to run LocalAI on a cloud GPU. A gateway whose README prints uv tool install 'litellm[proxy]' is How to run LiteLLM Proxy in front of a self-hosted model on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. An agent whose Ollama page sets llm to an ollama/ tag, and whose default base URL text is http://localhost:11434/v1, is How to run PraisonAI agents with a self-hosted model on a cloud GPU. An agent whose local branch sets base_url to http://localhost:11434/v1 and passes that client into AgentConfig is How to run Atomic Agents with a self-hosted model on a cloud GPU. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.
The LICENSE file is the Apache License, Version 2.0, January 2004, and the GitHub license field on 2026-11-14 was Apache-2.0.
The commands below are copied from the framework page and the prompt-drivers page, and from commit 5f6fd2376a5ac082eb77732de09216ff14e748bf, except the model id marked as substituted and the extra install line marked as composed. They were not run on a CloudGPU instance for this page. The prompt-drivers page also prints hosted drivers. This page does not copy those sections. The page says OpenAi Chat is the default prompt driver. This page does not use the default constructor that only sets model.
What the docs print about memory
The prompt-drivers page prints no GPU GB figure. The framework page prints no GPU GB figure. The Ollama example’s model string is llama3.1. This page does not treat that string as vram_gb. This page does not mark a catalog row as enough.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T23:15:10.977Z). The catalog response also includes an availability field and a price field. This page does not copy them. Every catalog row below has min_gpu_count 1. Which listing you can start is on Deploy.
The catalog template list had no Griptape template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as Griptape.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| Ascend 910B | 64 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
The framework page, read 2026-11-14, prints this core install after uv init griptape-quickstart and cd griptape-quickstart:
uv add griptape
It also prints pip install "griptape[all]" -U. This page does not use the [all] extra. The same page’s optional-extra example names other drivers. This page does not copy that example. The prompt-drivers page says the Ollama driver requires the drivers-prompt-ollama extra. The framework page’s extra syntax is uv add "griptape[...]". The line below is composed from those two sentences. It is not a single line the framework page prints.
# Composed on this page from the core extra syntax and the drivers-prompt-ollama name.
uv add "griptape[drivers-prompt-ollama]"
pyproject.toml on commit 5f6fd2376a5ac082eb77732de09216ff14e748bf (committed 2026-11-13T22:50:30Z) says version = "1.13.0", requires-python = ">=3.10, <4", license = "Apache-2.0", and drivers-prompt-ollama = ["ollama>=0.4.1"]. openai is a core dependency on that file (openai>=1.1.1), so OpenAiChatPromptDriver is in the core install. PyPI griptape info.version on 2026-11-14 was 1.13.0, requires_python <4,>=3.10, uploaded 2026-08-26T19:57:41.032683Z. The PyPI license field was empty. The GitHub license field on 2026-11-14 was Apache-2.0. pushed_at was 2026-11-13T22:58:10Z. The README does not print a pip install line. It points at the docs. This page did not install the package.
OllamaPromptDriver
The prompt-drivers page’s Ollama section, copied. The import path is the one that page prints.
from griptape.drivers.prompt.ollama import OllamaPromptDriver
from griptape.structures import Agent
from griptape.tools import CalculatorTool
agent = Agent(
prompt_driver=OllamaPromptDriver(
model="llama3.1",
),
tools=[CalculatorTool()],
)
agent.run("What is (192 + 12) ^ 4")
llama3.1 is the tag that section prints. It is not llama3.1:8b. This page does not change it. On the same commit, OllamaPromptDriver in griptape/drivers/prompt/ollama_prompt_driver.py sets host: str | None = None. The example does not pass host. This page does not invent a host string. How to run Ollama on a cloud GPU is the Ollama install. That server stays on the rented GPU. The prompt-drivers page does not print ollama pull or an SSH command.
OpenAiChatPromptDriver and base_url
The same page’s OpenAI-compatible section says to set base_url to the service endpoint and model to the model name. Its local example, with the model id substituted as marked under the block:
from griptape.drivers.prompt.openai import OpenAiChatPromptDriver
from griptape.rules import Rule
from griptape.structures import Agent
agent = Agent(
prompt_driver=OpenAiChatPromptDriver(
base_url="http://127.0.0.1:1234/v1",
model="llama3.1",
stream=True,
),
rules=[Rule(value="You are a helpful coding assistant.")],
)
agent.run("How do I init and update a git submodule?")
The page prints model="lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF" in that example. On 2026-11-14, GET https://huggingface.co/api/models/lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF reported gated false, cardData.license llama3, pipeline_tag text-generation, lastModified 2024-05-03T13:53:50.000Z, and sha 0910a3e69201d274d4fd68e89448114cd78e4c82. This page does not use that id. It substitutes llama3.1, which the Ollama section of the same page already prints. base_url, stream, the rule text, and agent.run are the ones the example prints. The tip under the example says to include v1 at the end of base_url. That base_url is http://127.0.0.1:1234/v1. It is not http://localhost:11434/v1, and the Ollama driver example does not set base_url. This page does not copy the hosted base_url example on that page.
How to serve an OpenAI-compatible API with vLLM on a cloud GPU is a server that speaks this same /v1 shape. The Griptape page does not print a vLLM command or port 8000. For a server you start on the instance, the tunnel is How to use SSH port forwarding on a cloud GPU. The prompt-drivers page does not print an SSH command.
When it fails
CUDA out of memory. The prompt-drivers page prints no GPU GB figure. If the server process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The README and pyproject.toml do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this library on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
What to copy off, then destroy
agent.run is the call both examples print. Copy the printed output before you destroy the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The pricing page FAQ states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Prices are not copied here.
Frequently asked questions
What memory figure do the Griptape docs print?
The prompt-drivers page prints no GPU GB figure. The framework page prints no GPU GB figure. The Ollama example sets model to llama3.1. This page does not treat that string as vram_gb. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which install line does the framework page print?
The framework page prints uv add griptape for a core install, and it also prints pip install griptape[all] -U. This page does not use the [all] extra. The prompt-drivers page says the Ollama driver requires the drivers-prompt-ollama extra. The uv extra line on this page is composed from that extra name and the extra syntax on the framework page. pyproject.toml on commit 5f6fd2376a5ac082eb77732de09216ff14e748bf says version 1.13.0, requires-python 3.10 or newer and below 4, and license Apache-2.0. PyPI info.version on 2026-11-14 was 1.13.0, requires_python that same bound, uploaded 2026-08-26T19:57:41.032683Z. The PyPI license field was empty. This page did not install the package.
How does the driver point at a server on the GPU?
The Ollama section imports OllamaPromptDriver from griptape.drivers.prompt.ollama and sets model to llama3.1. It does not set host. On that commit the host field default is None. The OpenAI-compatible section imports OpenAiChatPromptDriver from griptape.drivers.prompt.openai and sets base_url to http://127.0.0.1:1234/v1. The model id that example prints is not used on this page. This page sets model to llama3.1, which the Ollama section already prints. The tip says to include v1 at the end of base_url. Those pages do not print an SSH command. This page does not copy the hosted driver sections, and it does not use the default constructor that only sets model.
What license does the file print?
The LICENSE file is the Apache License, Version 2.0, January 2004. pyproject.toml on that commit sets license to Apache-2.0. The GitHub license field on 2026-11-14 was Apache-2.0. The PyPI license field for griptape 1.13.0 was empty.
Sources (read 2026-11-14)
- Griptape prompt drivers, read 2026-11-14: https://docs.griptape.ai/stable/griptape-framework/drivers/prompt-drivers/ .
OllamaPromptDriverimportgriptape.drivers.prompt.ollama,model="llama3.1",Agent(...).run. Extra namedrivers-prompt-ollama.OpenAiChatPromptDriverimportgriptape.drivers.prompt.openai,base_url="http://127.0.0.1:1234/v1",stream=True,agent.run. The printed model id in that example is not used. The tip says to includev1. Hosted driver sections are not copied. The page says OpenAi Chat is the default prompt driver - Griptape framework page, read 2026-11-14: https://docs.griptape.ai/stable/griptape-framework/ .
uv init griptape-quickstart,cd griptape-quickstart,uv add griptape,pip install "griptape[all]" -U. This page does not use[all]. The optional-extra example on that page is not copied. Thedrivers-prompt-ollamauv line on this page is composed - griptape-ai/griptape commit
5f6fd2376a5ac082eb77732de09216ff14e748bf, committed 2026-11-13T22:50:30Z.pyproject.tomlversion1.13.0,requires-python>=3.10, <4,licenseApache-2.0,drivers-prompt-ollama = ["ollama>=0.4.1"], coreopenai>=1.1.1.OllamaPromptDriver.hostdefaultNone. README does not print a pip line. GitHub license field Apache-2.0.pushed_at2026-11-13T22:58:10Z. PyPIgriptape1.13.0, uploaded 2026-08-26T19:57:41.032683Z,requires_python<4,>=3.10, license field empty - Hugging Face model API
lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF:gatedfalse,cardData.licensellama3,pipeline_tagtext-generation,lastModified2024-05-03T13:53:50.000Z,sha0910a3e69201d274d4fd68e89448114cd78e4c82. This page does not use that id - Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (
generated_at2026-11-14T23:15:10.977Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no Griptape template
Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.
Last updated: 2026-11-14.