How to run smolagents with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install smolagents the way the official installation page prints, download the weights the model class names, and run CodeAgent or ToolCallingAgent against TransformersModel, VLLMModel, or an OpenAI-compatible server on that machine. Copy the printed answer and any file the agent wrote before you destroy the instance, because destroying the instance deletes its disk.
An agent can rent the GPU and run this library on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the smolagents install. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. A TurboMind server whose lmdeploy serve api_server listens on port 23333 is How to serve LLMs with LMDeploy on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. A coding-agent server whose card prints --tool-call-parser qwen3_coder is How to serve Qwen3-Coder for coding agents on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.
The smolagents LICENSE file begins with “Apache License Version 2.0, January 2004”. The appendix copyright line is still the template “Copyright [yyyy] [name of copyright owner]”. The README header says “Copyright 2024 The HuggingFace Team.” The GitHub license field on 2026-11-14 was Apache-2.0. That license text is not a CloudGPU price.
The commands below are copied from the upstream files named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The README and the models page also name other providers. This page does not list those names.
What the docs print about memory
The README, the installation page, the models reference, and the TransformersModel and VLLMModel docstrings print no GPU GB figure. The secure-execution page’s longer Docker example sets a container memory limit. This page does not copy that limit and does not treat it as vram_gb. This page does not mark a catalog row as enough.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-14T03:12:17.736Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-14T03:12:18.737Z) listed the same names and the same vram_gb values, plus Ascend 910B at 64, which is not in gpu-prices.json and is not listed here. The catalog response also includes an availability field. This page does not copy it. Every prices row has gpu_count 1. Every catalog row has min_gpu_count 1. Which listing you can start is on Deploy.
The catalog template list had no smolagents template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as smolagents.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
The installation page says Python 3.10 or newer, then a virtualenv, then the core install. The pip lines, copied:
python -m venv .venv
source .venv/bin/activate
pip install smolagents
The README quickstart, copied:
pip install "smolagents[toolkit]"
The extras this page keeps, copied from the installation page. transformers is the extra for TransformersModel. vllm is the extra for VLLMModel. openai is the extra the OpenAIModel import error names. docker is the extra the Docker executor names:
pip install "smolagents[transformers]"
pip install "smolagents[vllm]"
pip install "smolagents[openai]"
pip install "smolagents[docker]"
pyproject.toml at the main commit below says version = "1.27.0.dev0", transformers>=4.0.0,!=5.13.0, and vllm>=0.10.2. PyPI info.version for smolagents on 2026-11-14 was 1.26.0, requires_python >=3.10, uploaded 2026-05-29T05:08:44Z. Tag v1.26.0 is commit 12c1bc820eca50ace6f80a21d90426d41d74f845, committed 2026-05-29T05:08:24Z, and that tag’s pyproject.toml says version = "1.26.0". Those are not the same release as 1.27.0.dev0. This page did not install either one. The models page says the API is experimental and can change.
CodeAgent and ToolCallingAgent
The README says CodeAgent writes its actions as Python code, and ToolCallingAgent writes actions as JSON or text. The agents reference says both take model and a list of tools. The README quickstart, limited to the lines this page keeps:
from smolagents import CodeAgent, WebSearchTool, InferenceClientModel
model = InferenceClientModel()
agent = CodeAgent(tools=[WebSearchTool()], model=model, stream_outputs=True)
That snippet constructs InferenceClientModel with no model_id. That class calls the Hub. It is not a weight file on the instance. The CLI line smolagent can take --model-type and --model-id. One example on the README sets --model-type "InferenceClientModel" and --model-id "Qwen/Qwen3-Next-80B-A3B-Thinking". This page did not run smolagent.
Local weights and an OpenAI-compatible server
TransformersModel loads a Hugging Face id on the machine that runs the agent. The class docstring’s example, copied in the parts this page keeps. The default max_new_tokens in that docstring is 4096. The example passes 5000 and device="cuda":
from smolagents import TransformersModel
engine = TransformersModel(
model_id="Qwen/Qwen3-Next-80B-A3B-Thinking",
device="cuda",
max_new_tokens=5000,
)
The README’s own block sets max_new_tokens=4096 and device_map="auto" on the same id. The models page’s shorter example uses HuggingFaceTB/SmolLM-135M-Instruct. Neither example prints a GPU GB figure. The tip under TransformersModel says to run pip install 'smolagents[transformers]' if transformers and torch are missing.
VLLMModel constructs vLLM’s LLM on model_id. The models page’s example id is HuggingFaceTB/SmolLM-135M-Instruct. The missing-package error is pip install 'smolagents[vllm]'. A comment in models.py says VLLMModel does not support vision models yet. This page did not load a vision model with that class.
OpenAIModel is the class for an OpenAI-compatible API server. The docstring says it connects to an OpenAI-compatible API server, and the parameters it names are model_id, api_base, and api_key. models.py at both the main commit and tag v1.26.0 assigns OpenAIServerModel = OpenAIModel. The models page says you can customise api_base to point at another server. The example api_base and the example model_id on that page are a hosted endpoint and a closed-model id. This page does not copy them. The page does not print a vLLM URL or an Ollama URL. For a server you start on the instance, api_base is the base URL that server’s own guide prints. How to serve an OpenAI-compatible API with vLLM on a cloud GPU and How to run Ollama on a cloud GPU are those guides. This page does not add a port those smolagents files do not print.
The block below is added on this page. The model classes call from_pretrained or vLLM’s loader. They do not print hf download. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. This page did not download the repository.
# Added on this page. The TransformersModel docstring loads this id itself.
pip install -U "huggingface_hub"
hf download Qwen/Qwen3-Next-80B-A3B-Thinking
On 2026-11-14, GET https://huggingface.co/api/models/Qwen/Qwen3-Next-80B-A3B-Thinking reported gated false, cardData.license apache-2.0, pipeline_tag text-generation, lastModified 2025-09-15T09:12:48.000Z, and sha e502dd4100cc68c0de57643fd4317ec93a128670. The SmolLM-135M-Instruct model API the same day reported gated false, cardData.license apache-2.0, pipeline_tag text-generation, lastModified 2026-10-29T14:03:10.000Z, and sha b5f4b394abe60fe0e4d666139e57bfce520302a9.
Sandboxed code execution
The README says code execution is a security concern and that you should run agent code in a sandbox. It lists Blaxel, E2B, Modal, and Docker. It says the built-in LocalPythonExecutor is not a security sandbox. The secure-execution page sets executor_type to "blaxel", "e2b", "modal", or "docker" on CodeAgent. The quick-start snippets on that page construct InferenceClientModel, not TransformersModel. examples/sandboxed_execution.py uses the same four strings. This page did not start those sandboxes. A sentence on that page states a launch duration. This page does not copy it.
When it fails
CUDA out of memory. The smolagents files this page read print no GPU GB figure. If the process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The installation page and pyproject.toml do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this library on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
Import errors. OpenAIModel.create_client raises Please install 'openai' extra to use OpenAIModel: pip install 'smolagents[openai]' when that extra is missing. VLLMModel raises the matching line for the vllm extra. This page did not reproduce those errors.
What to copy off, then destroy
agent.run returns the final answer in the process that called it. Any file that code writes is on the instance disk. Copy the printed answer, and the hf download directory if you passed --local-dir, before you destroy the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The pricing page FAQ states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Prices are not copied here.
Frequently asked questions
What memory figure do the smolagents docs print?
The README, the installation page, the models page, and the TransformersModel and VLLMModel docstrings print no GPU GB figure. The secure-execution page has a longer Docker example that sets a container memory limit. This page does not copy that limit and does not treat it as vram_gb. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which classes and install lines do the docs print?
The installation page prints pip install smolagents, and Python 3.10 or newer. The README quickstart prints pip install smolagents[toolkit] and constructs CodeAgent with InferenceClientModel. ToolCallingAgent writes actions as JSON. The TransformersModel docstring example sets model_id to Qwen/Qwen3-Next-80B-A3B-Thinking, device to cuda, and max_new_tokens to 5000. The README block on that id sets max_new_tokens to 4096 and device_map to auto. VLLMModel loads a model_id with the vLLM LLM class, and a comment in models.py says VLLMModel does not support vision models yet. OpenAIModel connects to an OpenAI-compatible API server. models.py assigns OpenAIServerModel = OpenAIModel. The docstring example model id is not copied on this page. PyPI info.version on 2026-11-14 was 1.26.0. pyproject.toml at the main commit says 1.27.0.dev0. Those are not the same release. This page did not install either.
Which sandbox options does the secure-execution page list?
The README lists Blaxel, E2B, Modal, and Docker, and says LocalPythonExecutor is not a security sandbox. The secure-execution page sets executor_type to blaxel, e2b, modal, or docker on CodeAgent. The quick-start snippets construct InferenceClientModel. This page did not run them. A self-hosted model uses TransformersModel, VLLMModel, or OpenAIModel in that constructor. The models page says you can customise api_base to point at another server. It does not print a vLLM or Ollama URL.
What license does the file print?
The smolagents LICENSE file begins with Apache License Version 2.0, January 2004. The appendix copyright line is the template Copyright [yyyy] [name of copyright owner]. The README header says Copyright 2024 The HuggingFace Team. The GitHub license field on 2026-11-14 was Apache-2.0. That license text is not a CloudGPU price.
Sources (read 2026-11-14, Asia/Shanghai)
- huggingface/smolagents README (
CodeAgent,ToolCallingAgent,pip install "smolagents[toolkit]",TransformersModelidQwen/Qwen3-Next-80B-A3B-Thinking, sandbox list,LocalPythonExecutoris not a security sandbox, Copyright 2024 The HuggingFace Team): commit96f33faaf028479119ec8d34507b47694cf14e34, committed 2026-10-06T09:38:53Z. GitHub license fieldApache-2.0.pushed_at2026-11-10T18:34:27Z.LICENSEappendix is still the template line. The README also names other providers. This page does not list them - Tag
v1.26.0, commit12c1bc820eca50ace6f80a21d90426d41d74f845, committed 2026-05-29T05:08:24Z.pyproject.tomlversion1.26.0.models.pyalso assignsOpenAIServerModel = OpenAIModel. PyPIsmolagentsinfo.version1.26.0, uploaded 2026-05-29T05:08:44Z,requires_python>=3.10 docs/source/en/installation.md(Python 3.10,pip install smolagents, extrastransformers,vllm,openai,docker): main commit above.pyproject.tomlat that commit says1.27.0.dev0,transformers>=4.0.0,!=5.13.0,vllm>=0.10.2docs/source/en/reference/models.mdandsrc/smolagents/models.py(TransformersModel,VLLMModel,OpenAIModel,api_base, vision comment onVLLMModel): same commits as named above. The models page’s examplemodel_idandapi_baseforOpenAIModelare not copieddocs/source/en/reference/agents.md(CodeAgentwrites Python,ToolCallingAgentwrites JSON, both takemodelandtools): main commitdocs/source/en/tutorials/secure_code_execution.mdandexamples/sandboxed_execution.py(executor_typeblaxel,e2b,modal,docker): main commit. This page does not copy that page’s launch-duration sentence or its container memory limit- Hugging Face model API
Qwen/Qwen3-Next-80B-A3B-Thinking:gatedfalse,cardData.licenseapache-2.0,pipeline_tagtext-generation,lastModified2025-09-15T09:12:48.000Z,shae502dd4100cc68c0de57643fd4317ec93a128670 - Hugging Face model API
HuggingFaceTB/SmolLM-135M-Instruct:gatedfalse,cardData.licenseapache-2.0,pipeline_tagtext-generation,lastModified2026-10-29T14:03:10.000Z,shab5f4b394abe60fe0e4d666139e57bfce520302a9 - Public GPU memory: https://cloudgpu.app/gpu-prices.json (
updated_at2026-11-14T03:12:17.736Z) and https://cloudgpu.app/v1/public/gpu-catalog (generated_at2026-11-14T03:12:18.737Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no smolagents template
Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.
Last updated: 2026-11-14 (Asia/Shanghai).