How to run PraisonAI agents with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install PraisonAI with the pip install praisonai line the README prints, serve an ungated open-weight model on that machine, and point Agent at that server with the Ollama llm string or with base_url. Copy the printed start result before you destroy the instance, because destroying the instance deletes its disk.
An agent can rent the GPU and run this library against a model served on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the PraisonAI install. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. An OpenAI-compatible engine whose CUDA 12 container tag is localai/localai:latest-gpu-nvidia-cuda-12 is How to run LocalAI on a cloud GPU. A gateway whose README prints uv tool install 'litellm[proxy]' is How to run LiteLLM Proxy in front of a self-hosted model on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. An agent whose OllamaPromptDriver sets model to llama3.1 is How to run Griptape agents with a self-hosted model on a cloud GPU. An agent whose local branch sets base_url to http://localhost:11434/v1 and passes that client into AgentConfig is How to run Atomic Agents with a self-hosted model on a cloud GPU. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.
The repository LICENSE file begins with “MIT License”. src/praisonai/pyproject.toml on that commit sets license = {text = "MIT"}. The GitHub license field on 2026-11-14 was MIT.
The commands below are copied from the Ollama page, the OpenAI-compatible page, the local-models page, and the README at commit 851c386430b5e32cec7081442ee30080aedb7e45, except the model ids and the download lines marked as substituted or added. They were not run on a CloudGPU instance for this page. Those docs also print hosted endpoints and a hosted prefix. This page does not copy those sections.
What the docs print about memory
The Ollama page prints “522 MB” for qwen3:0.6b. That figure is not a GPU GB number. The OpenAI-compatible page and the local-models page print no GPU GB figure. The Qwen/Qwen2.5-7B-Instruct README fetched on 2026-11-14 does not print a GPU GB number. It points at a benchmark page. This page does not copy a number from that other page. This page does not mark a catalog row as enough.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T23:15:10.977Z). The catalog response also includes an availability field and a price field. This page does not copy them. Every catalog row below has min_gpu_count 1. Which listing you can start is on Deploy.
The catalog template list had no PraisonAI template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as PraisonAI.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| Ascend 910B | 64 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
The README installation block, on commit 851c386430b5e32cec7081442ee30080aedb7e45 (committed 2026-11-14T20:44:30Z), prints:
pip install praisonai
The next sentence prints pip install praisonaiagents for the Python library. A later sentence prints pip install "praisonaiagents[llm]" when only that library is installed and the local-model path is the one you want. Agent.__init__ on that commit raises an install message with the same praisonaiagents[llm] extra when the local LLM module is missing. src/praisonai/praisonai/version.py says __version__ = "4.7.25". src/praisonai/pyproject.toml says requires-python = ">=3.10,<3.15" and license = {text = "MIT"}, and it marks version as dynamic. src/praisonai-agents/pyproject.toml says version = "1.7.23" and requires-python = ">=3.10". That file does not set a license field. PyPI praisonai info.version on 2026-11-14 was 4.7.25, requires_python <3.15,>=3.10, license MIT, uploaded 2026-11-14T11:34:37.641559Z. PyPI praisonaiagents info.version was 1.7.23, requires_python >=3.10, uploaded 2026-11-14T11:32:49.348974Z. The PyPI license field for praisonaiagents 1.7.23 was empty. The GitHub license field on 2026-11-14 was MIT. pushed_at was 2026-11-14T20:51:56Z. This page did not install either package.
Point Agent at Ollama on the GPU
The Ollama page’s setup block prints ollama serve and ollama pull llama3.2. The same page’s Models section names ollama/qwen3:0.6b and prints “522 MB” for that tag. This page substitutes qwen3:0.6b for llama3.2. It does not invent the tag. The pull line below is that substitution. The install script on the page is copied.
curl -fsSL https://ollama.com/install.sh | sh
ollama serve
ollama pull qwen3:0.6b
The page says setting MODEL_NAME to an ollama/ model is enough, and the default base URL it prints is http://localhost:11434/v1. The tag in that sentence is llama3.2. The Python block it prints uses llm="ollama/llama3.2" and agent.start. The block below keeps that shape and uses the substituted tag:
from praisonaiagents import Agent
agent = Agent(
instructions="You are a helpful assistant",
llm="ollama/qwen3:0.6b",
)
agent.start("Explain deep learning")
The same page’s tool-calling section already prints llm="ollama_chat/qwen3:0.6b". That prefix is a different string from ollama/qwen3:0.6b. This page does not merge the two snippets. A non-default host example on that page sets base_url to http://192.168.1.10:11434 with llm="ollama/llama3.2". That address is the one in the page. It is not a command this page adds. The README offline section prints model="ollama/llama3.2" rather than llm=. On the same commit, the Agent docstring says model is the canonical name, llm is a deprecated alias, and passing both raises TypeError. The docs pages linked above still print llm=. This page does not pass both.
The Ollama page also prints a prefix table. One row is the vllm/ prefix. Another cell on that row names a hosted prefix. This page does not copy the hosted prefix or the rest of that table.
How to run Ollama on a cloud GPU is the Ollama install. That server stays on the rented GPU.
Point base_url at a /v1 server on the GPU
The OpenAI-compatible page’s vLLM block prints base_url="http://localhost:8000/v1", api_key="not-needed", and agent.start("Hello"). The llm string in that block is a repository id that, on 2026-11-14, GET https://huggingface.co/api/models/meta-llama/Meta-Llama-3-8B-Instruct reported as gated manual. This page does not use that id. The same page’s local-prefix example prints llm="vllm/meta-llama/Llama-3.1-8B-Instruct". On 2026-11-14, GET https://huggingface.co/api/models/meta-llama/Llama-3.1-8B-Instruct reported gated manual. This page does not use that id. The same page also prints a hosted_vllm/ example. This page does not copy it.
The vLLM guide is the server this block calls. Its Step 4 sets VLLM_API_KEY with openssl rand -hex 16 and passes --served-model-name qwen2.5-7b. Its Python client sets api_key=os.environ["VLLM_API_KEY"] and model="qwen2.5-7b". The flag table on that page quotes the engine arguments: --served-model-name is “The model name(s) used in the API”, and without it the name is the model path. Qwen/Qwen2.5-7B-Instruct is the repository that guide downloads. It is not the name the API uses after that flag. api_key="not-needed" is the literal the PraisonAI block prints. It is not the key that Step 4 generates. The block below keeps the PraisonAI base_url and agent.start("Hello"), and uses the served name and key from that guide.
import os
from praisonaiagents import Agent
agent = Agent(
name="assistant",
llm="qwen2.5-7b",
base_url="http://localhost:8000/v1",
api_key=os.environ["VLLM_API_KEY"],
)
agent.start("Hello")
The local-models page’s vLLM block prints:
export OPENAI_BASE_URL=http://localhost:8000/v1
praisonai run "hello"
That page says the probe adopts the first model the server returns. This page does not name that model. OPENAI_BASE_URL in that block is http://localhost:8000/v1. The Ollama page’s default base URL is http://localhost:11434/v1. Those two strings are not the same.
On 2026-11-14, GET https://huggingface.co/api/models/Qwen/Qwen2.5-7B-Instruct reported gated false, cardData.license apache-2.0, pipeline_tag text-generation, lastModified 2025-01-12T02:10:10.000Z, and sha a09a35458c702b33eeacc393d103063234e8bc28. The card README does not print a GPU GB number. It points at a benchmark page. This page does not treat 7B in that id as vram_gb.
The PraisonAI files this page read do not print hf download. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. The block below is added on this page. This page did not download the repository.
# Added on this page. The linked vLLM guide downloads this id and serves it as qwen2.5-7b.
pip install -U "huggingface_hub"
hf download Qwen/Qwen2.5-7B-Instruct
How to serve an OpenAI-compatible API with vLLM on a cloud GPU is the vLLM server install. The PraisonAI pages do not print an SSH command. For a server you start on the instance, the tunnel is How to use SSH port forwarding on a cloud GPU.
When it fails
CUDA out of memory. The Ollama page’s only size figure for qwen3:0.6b is “522 MB”, and that figure is not a GPU GB number. The Qwen/Qwen2.5-7B-Instruct card README does not print a GPU GB number. If the server process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The README and the two pyproject.toml files do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this library on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
What to copy off, then destroy
agent.start is the call the Ollama page and the vLLM block print. Copy the printed return value before you destroy the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The pricing page FAQ states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Prices are not copied here.
Frequently asked questions
What memory figure do the PraisonAI docs print?
The Ollama page prints 522 MB for qwen3:0.6b. That figure is not a GPU GB number. The OpenAI-compatible page and the local-models page print no GPU GB figure. The Qwen/Qwen2.5-7B-Instruct README fetched on 2026-11-14 does not print a GPU GB number. It points at a benchmark page. This page does not copy a number from that other page. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which install line does the README print?
The README prints pip install praisonai, and it also prints pip install praisonaiagents. A later sentence prints pip install praisonaiagents[llm] for a core-only install that uses the local-model path. The version file on commit 851c386430b5e32cec7081442ee30080aedb7e45 says 4.7.25. praisonaiagents pyproject.toml on that commit says version 1.7.23 and requires-python 3.10 or newer. PyPI praisonai on 2026-11-14 was 4.7.25, license MIT, requires_python 3.10 or newer and below 3.15, uploaded 2026-11-14T11:34:37.641559Z. PyPI praisonaiagents was 1.7.23, requires_python 3.10 or newer, uploaded 2026-11-14T11:32:49.348974Z. This page did not install the packages.
How does Agent point at a server on the GPU?
The Ollama page prints llm set to ollama/llama3.2 and says the default base URL is http://localhost:11434/v1. This page substitutes ollama/qwen3:0.6b, which that page already names. The OpenAI-compatible vLLM block sets base_url to http://localhost:8000/v1 and api_key to the literal that block prints. This page does not use that literal against the linked vLLM guide. That guide sets VLLM_API_KEY and served-model-name qwen2.5-7b. This page sets llm to qwen2.5-7b and api_key from that variable. The model id in the PraisonAI vLLM block reported gated manual on 2026-11-14. The vllm/ prefix example on that page uses a different repository id that also reported gated manual. This page does not copy the hosted_vllm prefix. The Agent docstring says model is the canonical name and llm is a deprecated alias, and passing both raises TypeError. Those pages do not print an SSH command.
What license does the file print?
The repository LICENSE file begins with MIT License. praisonai pyproject.toml sets license text to MIT. The GitHub license field on 2026-11-14 was MIT. PyPI license for praisonai 4.7.25 was MIT. praisonaiagents pyproject.toml does not set a license field, and the PyPI license field for praisonaiagents 1.7.23 was empty.
Sources (read 2026-11-14)
- PraisonAI Ollama page, read 2026-11-14: https://praison.ai/docs/models/ollama .
Agent(..., llm="ollama/llama3.2")andagent.start. Default base URL texthttp://localhost:11434/v1. Setup linesollama serveandollama pull llama3.2. This page substitutesqwen3:0.6b, which the same page names, and quotes that page’s “522 MB” figure.ollama_chat/qwen3:0.6bis a separate example on that page. The prefix table’s hosted prefix is not copied. Thevllm/prefix is named - PraisonAI OpenAI-compatible page, read 2026-11-14: https://praison.ai/docs/models/openai-compatible . vLLM block
base_url="http://localhost:8000/v1",api_key="not-needed",agent.start("Hello"). The model id in that block is not used, andnot-neededis not used against the linked vLLM guide. Local-prefix examplevllm/meta-llama/Llama-3.1-8B-Instructis not used. Thehosted_vllm/example and the hostedbase_urlexamples are not copied. The client on this page usesllm="qwen2.5-7b"andapi_key=os.environ["VLLM_API_KEY"]from the vLLM guide Step 4 and its Python client - PraisonAI local-models page, read 2026-11-14: https://praison.ai/docs/features/local-models . vLLM block
export OPENAI_BASE_URL=http://localhost:8000/v1andpraisonai run "hello". The page says the probe adopts the first model the server returns. This page does not name that model - MervinPraison/PraisonAI commit
851c386430b5e32cec7081442ee30080aedb7e45, committed 2026-11-14T20:44:30Z. READMEpip install praisonai,pip install praisonaiagents, andpip install "praisonaiagents[llm]". README offline block printsmodel="ollama/llama3.2".version.py__version__ = "4.7.25".praisonaiagentspyproject version1.7.23,requires-python>=3.10, no license field.Agentdocstring:modelcanonical,llmdeprecated alias, both together raisesTypeError. GitHub license field MIT.pushed_at2026-11-14T20:51:56Z. PyPIpraisonai4.7.25, uploaded 2026-11-14T11:34:37.641559Z,requires_python<3.15,>=3.10,licenseMIT. PyPIpraisonaiagents1.7.23, uploaded 2026-11-14T11:32:49.348974Z,requires_python>=3.10, license field empty - Hugging Face model API
Qwen/Qwen2.5-7B-Instruct:gatedfalse,cardData.licenseapache-2.0,pipeline_tagtext-generation,lastModified2025-01-12T02:10:10.000Z,shaa09a35458c702b33eeacc393d103063234e8bc28. The card README does not print a GPU GB number.meta-llama/Meta-Llama-3-8B-Instructandmeta-llama/Llama-3.1-8B-Instructboth reportedgatedmanual on 2026-11-14. This page does not use those ids.hf downloadis added on this page and was not run - Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (
generated_at2026-11-14T23:15:10.977Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no PraisonAI template
Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.
Last updated: 2026-11-14.