How to run AgentScope agents with a self-hosted model on a cloud GPU

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install AgentScope with the uv pip install "agentscope==2.0.11" line the 2.0.11 quickstart prints, serve an ungated open-weight model on that machine, and pass OpenAIChatModel an OpenAICredential whose base_url is that server. Copy reply_msg before you destroy the instance, because destroying the instance deletes its disk.

An agent can rent the GPU and run this library against a model served on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the AgentScope 2.0.11 install. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. An OpenAI-compatible engine whose CUDA 12 container tag is localai/localai:latest-gpu-nvidia-cuda-12 is How to run LocalAI on a cloud GPU. A gateway whose README prints uv tool install 'litellm[proxy]' is How to run LiteLLM Proxy in front of a self-hosted model on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. A library whose README prints uv add langchain, and whose vLLM chat page sets ChatOpenAI base_url to http://localhost:8000/v1, is How to build LangChain agents with a self-hosted model on a cloud GPU. An AgentChat library whose OpenAI-compatible example sets base_url to http://localhost:11434/v1 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A Python agent library whose OpenAIModel takes api_base is How to run smolagents with a self-hosted model on a cloud GPU. A framework whose README comment sets model_server to http://localhost:8000/v1 is How to run Qwen-Agent with a self-hosted model on a cloud GPU. A library whose local-LLM page sets OpenAIGPTConfig api_base for a vLLM-style endpoint is How to run Langroid agents with a self-hosted model on a cloud GPU. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.

The LICENSE file begins with “Apache License Version 2.0, January 2004”, pyproject.toml on tag v2.0.11 says license = "Apache-2.0", and the GitHub license field on 2026-11-14 was Apache-2.0.

The commands below are copied from the 2.0.11 docs and from tag v2.0.11, except the base_url string and the model id marked as added. They were not run on a CloudGPU instance for this page. The model overview and the LLM page also name other providers. This page does not list those names. The tutorial site at doc.agentscope.io still prints a 1.0.21 sample. This page does not copy that constructor.

What the docs print about memory

The 2.0.11 quickstart, the model overview, the LLM page, and the configure-agent page print no GPU GB figure. The Qwen/Qwen3-8B README fetched on 2026-11-14 prints no GPU GB figure. The Ollama example sets model to qwen3:8b. This page does not treat that tag as Qwen/Qwen3-8B. This page does not mark a catalog row as enough.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T21:12:05.060Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Which listing you can start is on Deploy.

The catalog template list had no AgentScope template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as AgentScope.

Catalog name vram_gb
RTX 2080 8
RTX 3080 10
RTX 2080 Ti 11
RTX 3080 Ti 12
V100 16G 16
A30 24
RTX 3090 24
RTX 4090 24
RTX 4090D 24
RTX 4080 SUPER 32G 32
RTX 5090 32
RTX 5090D 32
V100 32G 32
A100 40G 40
A800 40G 40
A40 48
RTX 4090 48G 48
RTX 4090D 48G 48
RTX A6000 48
Ascend 910B 64
NVIDIA RTX PRO 5000 72G 72
RTX PRO 5000 72G 72
A100 80G 80
A800 80G 80
H800 80
RTX 6000D 83
H20 96G 96
RTX PRO 6000 96G 96
RTX PRO 6000 Max-Q 96
H20-3e 141G 141

Install

The 2.0.11 quickstart says AgentScope requires Python 3.11 or higher, and recommends uv. The PyPI line, copied:

uv pip install "agentscope==2.0.11"

The same page prints the same quoted command for the full extra on Windows and on Mac and Linux:

uv pip install "agentscope[full]==2.0.11"

It says full includes extra dependencies for model APIs, tool functions and more. On tag v2.0.11 (8af9d25784e704a4b11d71bd673257c68505fcb8, committed 2026-10-27T02:10:36Z), pyproject.toml says requires-python = ">=3.11" and license = "Apache-2.0". The models extra includes agentscope[model-ollama], and model-ollama is ollama>=0.5.4. The full extra includes agentscope[models]. PyPI agentscope info.version on 2026-11-14 was 2.0.11, requires_python >=3.11, info.license empty, uploaded 2026-10-27T02:54:31.183313Z. Main’s src/agentscope/_version.py on 2026-11-14 said 2.0.12dev. Main is a later commit than tag v2.0.11. This page cites the 2.0.11 docs and that tag. This page did not install the package.

The docs index says to use the docs that match the installed version, and not to mix 1.x and 2.x APIs. The older page https://doc.agentscope.io/tutorial/quickstart_installation.html prints pip install agentscope and a sample agentscope.__version__ output of 1.0.21. This page does not use that install line or that constructor.

Point OpenAIChatModel at the GPU server

The 2.0.11 configure-agent page says switching the provider only changes the model argument, and every provider follows Model(credential=..., model=...). Its OpenAI block passes api_key="YOUR_API_KEY" and a hosted model id. This page does not copy that model id. On tag v2.0.11, OpenAICredential in src/agentscope/credential/_openai.py has a base_url field whose description is “The base URL for the OpenAI API. Can be used for OpenAI-compatible endpoints.” OpenAIChatModel in src/agentscope/model/_openai_chat/_model.py builds openai.AsyncClient with base_url=self.credential.base_url. The same constructor still takes client_kwargs. Its docstring says those arguments are forwarded to the client, and the examples it names are timeout, default_headers, and http_client. This page sets base_url on the credential.

The URL and the model id below are added on this page. The 2.0.11 pages this page read do not print http://localhost:8000/v1. That string is the /v1 prefix on the default server in How to serve an OpenAI-compatible API with vLLM on a cloud GPU. Qwen/Qwen3-8B is not the model string in the OpenAI block above.

from agentscope.model import OpenAIChatModel
from agentscope.credential import OpenAICredential

model = OpenAIChatModel(
    credential=OpenAICredential(
        api_key="YOUR_API_KEY",
        base_url="http://localhost:8000/v1",
    ),
    model="Qwen/Qwen3-8B",
)

The AgentScope files this page read do not print hf download. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. The block below is added on this page. This page did not download the repository.

# Added on this page. The OpenAI example does not name this repository.
pip install -U "huggingface_hub"
hf download Qwen/Qwen3-8B

On 2026-11-14, GET https://huggingface.co/api/models/Qwen/Qwen3-8B reported gated false, cardData.license apache-2.0, pipeline_tag text-generation, lastModified 2025-07-26T03:49:13.000Z, and sha b968826d9c46dd6066d109eabc6255188de91218. The card README this page read prints no GPU GB figure. This page does not treat 8B in that id as vram_gb.

The Ollama block on the same configure-agent page, copied:

from agentscope.model import OllamaChatModel
from agentscope.credential import OllamaCredential

model = OllamaChatModel(
    credential=OllamaCredential(host="http://localhost:11434"),
    model="qwen3:8b",
)

The tag’s OllamaCredential says host defaults to http://localhost:11434 if it is not specified. The example still passes that URL. qwen3:8b and Qwen/Qwen3-8B are not the same string. The base uv pip install "agentscope==2.0.11" line does not install the ollama package. The model-ollama extra does. How to run Ollama on a cloud GPU is the Ollama install. That server stays on the rented GPU.

For a server you start on the instance, the SSH tunnel is How to use SSH port forwarding on a cloud GPU. The 2.0.11 pages this page read do not print an SSH command.

Agent

The 2.0.11 agent overview says Agent is a stateless reasoning-acting loop. reply returns the final Msg. reply_stream yields events. The changelog says 2.0 refactors ReActAgent into Agent, and replaces the 1.0 __call__ method with reply and reply_stream. This page uses Agent.

The quickstart builds that agent with a different credential class and a hosted model id. Its tip says to swap that pair, and it names OpenAICredential and OpenAIChatModel as the example. The block below is the quickstart agent with that swap. base_url and model are the strings added in the previous section. api_key="YOUR_API_KEY" is the configure-agent OpenAI block. The quickstart’s own key line reads an environment variable. This page did not run it.

import asyncio

from agentscope.agent import Agent
from agentscope.credential import OpenAICredential
from agentscope.message import UserMsg
from agentscope.model import OpenAIChatModel
from agentscope.tool import Toolkit, Bash, Read, Write, Edit


async def main() -> None:
    agent = Agent(
        name="Friday",
        system_prompt="You are a helpful assistant named Friday.",
        model=OpenAIChatModel(
            credential=OpenAICredential(
                api_key="YOUR_API_KEY",
                base_url="http://localhost:8000/v1",
            ),
            model="Qwen/Qwen3-8B",
        ),
        toolkit=Toolkit(tools=[Bash(), Read(), Write(), Edit()]),
    )

    user_msg = UserMsg(name="user", content="Hello, who are you?")
    reply_msg = await agent.reply(user_msg)


asyncio.run(main())

The quickstart also shows reply_stream and EventType.TEXT_BLOCK_DELTA. This page does not repeat that loop. The configure-agent table says name, system_prompt, and model are required.

When it fails

CUDA out of memory. The AgentScope 2.0.11 pages this page read print no GPU GB figure. The Qwen/Qwen3-8B card README prints no GPU GB figure. If the process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The 2.0.11 quickstart and pyproject.toml do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this library on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

What to copy off, then destroy

agent.reply returns the final message. The quickstart assigns it to reply_msg. Copy that message before you destroy the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The pricing page FAQ states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Prices are not copied here.

Frequently asked questions

What memory figure do the AgentScope 2.0.11 docs print?

The 2.0.11 quickstart, the model overview, the LLM page, and the configure-agent page print no GPU GB figure. The Qwen/Qwen3-8B README fetched on 2026-11-14 prints no GPU GB figure. The Ollama example sets model to qwen3:8b. This page does not treat that tag as Qwen/Qwen3-8B. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Which install line does the 2.0.11 quickstart print?

The 2.0.11 quickstart prints uv pip install “agentscope==2.0.11” and uv pip install “agentscope[full]==2.0.11”. It says AgentScope requires Python 3.11 or higher. PyPI info.version on 2026-11-14 was 2.0.11, requires_python >=3.11, info.license empty, uploaded 2026-10-27T02:54:31.183313Z. Tag v2.0.11 is commit 8af9d25784e704a4b11d71bd673257c68505fcb8, committed 2026-10-27T02:10:36Z. Main’s version file says 2.0.12dev. The older tutorial site’s sample prints 1.0.21. This page does not use that sample. This page did not install the package.

How does OpenAIChatModel point at a server on the GPU?

The configure-agent page builds OpenAIChatModel with OpenAICredential. On tag v2.0.11 the credential field base_url is described as a base URL for OpenAI-compatible endpoints, and the chat model passes that field to the client as base_url. The client_kwargs docstring names timeout, default_headers, and http_client. This page sets base_url to http://localhost:8000/v1 and model to Qwen/Qwen3-8B. Those two strings are added on this page. The quickstart tip names OpenAICredential and OpenAIChatModel as the pair to swap in. The Ollama example sets host to http://localhost:11434 and model to qwen3:8b. Those pages do not print an SSH command.

What license does the file print?

The LICENSE file begins with Apache License Version 2.0, January 2004. pyproject.toml on tag v2.0.11 says license Apache-2.0. The GitHub license field on 2026-11-14 was Apache-2.0. PyPI info.license was empty.

Sources (read 2026-11-14)

  • AgentScope 2.0.11 quickstart (uv pip install "agentscope==2.0.11", Python 3.11+, Agent.reply, the tip that names OpenAICredential and OpenAIChatModel): https://docs.agentscope.io/en/versions/2.0.11/quickstart.md
  • AgentScope 2.0.11 LLM page (common constructor arguments, client_kwargs docstring examples timeout, default_headers, http_client): https://docs.agentscope.io/en/versions/2.0.11/building-blocks/model/llm.md . The provider table on that page is not copied
  • AgentScope 2.0.11 configure-agent page (OpenAIChatModel, OllamaChatModel, OllamaCredential(host="http://localhost:11434"), model="qwen3:8b"): https://docs.agentscope.io/en/versions/2.0.11/building-blocks/agent/configure-agent.md . The OpenAI block’s model id is not used. The other provider blocks on that page are not copied
  • AgentScope 2.0.11 agent overview (Agent, reply, reply_stream): https://docs.agentscope.io/en/versions/2.0.11/building-blocks/agent/overview.md
  • AgentScope 2.0.11 changelog (ReActAgent refactored into Agent): https://docs.agentscope.io/en/versions/2.0.11/others/change-log.md
  • Tag v2.0.11, commit 8af9d25784e704a4b11d71bd673257c68505fcb8, committed 2026-10-27T02:10:36Z. OpenAICredential.base_url and openai.AsyncClient(..., base_url=self.credential.base_url) are from that tag. model-ollama is ollama>=0.5.4. PyPI agentscope 2.0.11, uploaded 2026-10-27T02:54:31.183313Z, requires_python >=3.11, info.license empty. Main _version.py was 2.0.12dev. pushed_at 2026-11-13T09:12:23Z. GitHub license field Apache-2.0
  • Older tutorial https://doc.agentscope.io/tutorial/quickstart_installation.html sample output 1.0.21, read 2026-11-14. This page does not copy that constructor
  • Hugging Face model API Qwen/Qwen3-8B: gated false, cardData.license apache-2.0, pipeline_tag text-generation, lastModified 2025-07-26T03:49:13.000Z, sha b968826d9c46dd6066d109eabc6255188de91218. The card README prints no GPU GB figure. hf download is added on this page and was not run
  • Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (generated_at 2026-11-14T21:12:05.060Z). GET /v1/public/compute-capabilities returned feature flags and no GPU list. The catalog had no AgentScope template

Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.

Last updated: 2026-11-14.