How to run the goose agent with a self-hosted model on a cloud GPU
Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install the goose CLI with the Linux command the official installation page prints, serve a model on that machine, and point goose configure at the Ollama host or the OpenAI-compatible host those docs print. Copy the project files the session writes, and the provider config under ~/.config/goose/custom_providers/ when you created one, before you destroy the instance, because destroying the instance deletes its disk.
An agent can rent the GPU and run this program on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the goose install. A workflow editor whose Docker install publishes port 5678, and whose Ollama credentials default the Base URL to http://localhost:11434, is How to run n8n AI agents with a self-hosted model on a cloud GPU. A coding agent whose providers page sets package to the OpenAI-compatible provider is How to use OpenCode with a self-hosted model on a cloud GPU. A canvas whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU. A CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A browser chat whose serve command listens on port 8080 is How to run Open WebUI on a cloud GPU. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose OllamaChatGenerator sets url to http://localhost:11434 is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b is How to run Agno agents with a self-hosted model on a cloud GPU. A TypeScript agent whose local-models section sets url on the model object is How to run Mastra agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.
The LICENSE file is the Apache License, Version 2.0, and the appendix copyright line is Copyright 2024 Block, Inc.
The commands below are copied from the upstream files named in each step, at the commit in Sources. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The README says goose works with 15+ providers and names Ollama. Other names in that sentence are not copied.
What the docs print about memory
The installation page, the providers page’s Ollama example, and the tool-shim page print no GPU GB figure. The providers page’s Other Models tip says the model’s default context length of 4096 tokens can be too low, and names OLLAMA_CONTEXT_LENGTH. That number is tokens. It is not vram_gb. The model information table on unsloth/Llama-3.1-8B-Instruct prints parameter counts 8B, 70B, and 405B, and a context length of 128k. This page does not treat those parameter counts as vram_gb. A later section of that card names training hardware. This page does not copy that section and does not use it as an inference memory figure. This page does not mark a catalog row as enough.
GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T10:04:57.194Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Rows are sorted by vram_gb, then by name. Which listing you can start is on Deploy.
The catalog template list had no goose template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as goose.
| Catalog name | vram_gb |
|---|---|
| RTX 2080 | 8 |
| RTX 3080 | 10 |
| RTX 2080 Ti | 11 |
| RTX 3080 Ti | 12 |
| V100 16G | 16 |
| A30 | 24 |
| RTX 3090 | 24 |
| RTX 4090 | 24 |
| RTX 4090D | 24 |
| RTX 4080 SUPER 32G | 32 |
| RTX 5090 | 32 |
| RTX 5090D | 32 |
| V100 32G | 32 |
| A100 40G | 40 |
| A800 40G | 40 |
| A40 | 48 |
| RTX 4090 48G | 48 |
| RTX 4090D 48G | 48 |
| RTX A6000 | 48 |
| Ascend 910B | 64 |
| NVIDIA RTX PRO 5000 72G | 72 |
| RTX PRO 5000 72G | 72 |
| A100 80G | 80 |
| A800 80G | 80 |
| H800 | 80 |
| RTX 6000D | 83 |
| H20 96G | 96 |
| RTX PRO 6000 96G | 96 |
| RTX PRO 6000 Max-Q | 96 |
| H20-3e 141G | 141 |
Install
https://github.com/block/goose returned HTTP 301 to https://github.com/aaif-goose/goose on 2026-11-14. Main of aaif-goose/goose that day was commit 2365bc5e898a38133c373eef2d1357416acd66e2, committed 2026-11-13T18:57:44Z. pushed_at the same day was 2026-11-14T03:56:38Z. The GitHub license field was Apache-2.0. The README and the docs files quoted here are from that commit.
The README and the Linux CLI tab of documentation/docs/getting-started/installation.md print the same install line:
curl -fsSL https://github.com/aaif-goose/goose/releases/download/stable/download_cli.sh | bash
The installation page says this script fetches the latest version. It also prints a line that skips interactive configuration:
curl -fsSL https://github.com/aaif-goose/goose/releases/download/stable/download_cli.sh | CONFIGURE=false bash
This page did not run either script. The same file also prints Desktop and macOS and Windows sections. This page copies the Linux CLI lines.
Configure and a session
documentation/docs/getting-started/providers.md at that commit says to run goose configure in the CLI. Under Local LLMs, the Other Models example prints:
ollama run qwen2.5
Then, in a separate terminal, goose configure, then Configure Providers, then Ollama. The prompt it prints is Provider Ollama requires OLLAMA_HOST, and the example value is http://localhost:11434. The same section says that if you do not provide a host, goose sets it to localhost:11434, and that it prepends http:// when the scheme is not http or https. The model line in that example is qwen2.5. The provider menu in that example names other providers. This page does not copy those names.
That page’s warning, copied: “goose extensively uses tool calling, so models without it can only do chat completion.” It then says that if you use a model without tool calling, all goose extensions must be disabled.
The quickstart at documentation/docs/quickstart.md starts a new session with:
goose session
The custom-provider example later on the providers page prints goose session start --provider custom_corp_api. Those two session commands are not the same. This page does not choose between them. The quickstart’s goose session is the line under its “start a new session” step. The other line belongs to the custom-provider JSON example.
The files this page read do not print an SSH command. When goose and Ollama are on the same machine, the host the example prints is http://localhost:11434. If the goose process is on your computer and Ollama is on the instance, the tunnel is How to use SSH port forwarding on a cloud GPU.
Ollama and an OpenAI-compatible server
The providers page section on custom endpoints says the built-in provider can connect to any OpenAI-compatible endpoint, and its list includes self-hosted models using vLLM. The parameter table says OPENAI_HOST is the custom endpoint URL. The default host printed in that table is not copied on this page. The vLLM example prints:
OPENAI_HOST=https://your-vllm-endpoint.internal
OPENAI_API_KEY=your-internal-api-key
That host is not http://127.0.0.1:8000. The CLI steps on that page say to run goose configure, select Configure Providers, choose that provider, and enter the host URL when using a custom endpoint. How to serve an OpenAI-compatible API with vLLM on a cloud GPU is the general server install.
The same page’s custom-provider section says custom providers must use an OpenAI-compatible format, an Ollama-compatible format, or a third format. This page copies the setting names OpenAI Compatible and Ollama Compatible. It does not copy the third name. The CLI path is goose configure, then Custom Providers, then Add A Custom Provider. The example JSON sets base_url to https://api.company.com/v1/chat/completions. The model names in that JSON are not copied. The directory it prints for the JSON file on Linux is ~/.config/goose/custom_providers/.
The experimental Ollama tool-shim page, documentation/docs/experimental/ollama.md, says the feature may change and points at the Tool Shim guide. The Tool Shim guide at documentation/docs/guides/tool-shim.md says the default interpreter model is mistral-nemo and prints:
ollama pull mistral-nemo
export GOOSE_TOOLSHIM=true
An optional line on that page is export GOOSE_TOOLSHIM_OLLAMA_MODEL=llama3.2. The experimental page’s session line is GOOSE_TOOLSHIM=true goose session. This page did not run those commands.
The goose files this page read do not print hf download. The block below is added on this page. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. This page did not download the repository. The goose files this page read do not print this repository id.
pip install -U "huggingface_hub"
hf download unsloth/Llama-3.1-8B-Instruct
On 2026-11-14, GET https://huggingface.co/api/models/unsloth/Llama-3.1-8B-Instruct reported gated false, cardData.license llama3.1, pipeline_tag text-generation, lastModified 2025-05-09T19:32:57.000Z, and sha 4699cc75b550f9c6f3173fb80f4703b62d946aa5. The card YAML also includes extra_gated_prompt. Those two fields are not the same fact. The card’s base_model field names a repository this page does not copy. This page did not run hf download.
When it fails
CUDA out of memory. The goose files this page read print no GPU GB figure. If the model server raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you.
“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The goose README and the docs files this page read do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this program on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.
The model does not call tools. The providers page says models without tool calling can only do chat completion, and that extensions must be disabled. The tool-shim guide says it is experimental and prints GOOSE_TOOLSHIM=true plus ollama pull mistral-nemo. This page does not claim the shim makes a model call tools.
What to copy off, then destroy
goose session runs in the working directory. The providers page says a custom provider JSON file lives in ~/.config/goose/custom_providers/ on Linux. Copy the project files the session wrote, and that directory if you created a provider file, before you destroy the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.
The billing page states the billing rule in these words:
Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.
Amounts are not copied here.
Frequently asked questions
What memory figure do the goose docs print?
The installation page, the providers page’s Ollama example, and the tool-shim page print no GPU GB figure. The providers page’s Other Models tip prints a default context length of 4096 tokens. The unsloth/Llama-3.1-8B-Instruct model information table prints parameter counts 8B, 70B, and 405B, and context length 128k. This page does not treat those figures as vram_gb. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.
Which install lines do the docs print?
The Linux CLI tab on the installation page prints curl -fsSL https://github.com/aaif-goose/goose/releases/download/stable/download_cli.sh | bash. The same page prints a second line with CONFIGURE=false. The README prints the same curl line. Main is commit 2365bc5e898a38133c373eef2d1357416acd66e2, committed 2026-11-13. https://github.com/block/goose returned 301 to https://github.com/aaif-goose/goose on 2026-11-14. This page did not run the install script.
How do the docs point goose at Ollama or an OpenAI-compatible server?
The providers page’s Other Models example runs ollama run qwen2.5, then goose configure, selects Ollama, and sets OLLAMA_HOST to http://localhost:11434. If you do not provide a host, that page says it sets localhost:11434. The same page says goose extensively uses tool calling, and models without it can only do chat completion. The custom-endpoint section says OPENAI_HOST is the custom endpoint URL. Its vLLM example sets OPENAI_HOST to https://your-vllm-endpoint.internal. The quickstart starts a session with goose session. The files do not print an SSH command.
What license does the file print?
The LICENSE file is the Apache License, Version 2.0, and the appendix copyright line is Copyright 2024 Block, Inc. The GitHub license field on 2026-11-14 was Apache-2.0. That license text is not a CloudGPU rate.
Sources (read 2026-11-14, Asia/Shanghai)
https://github.com/block/goosereturned HTTP 301 tohttps://github.com/aaif-goose/gooseon 2026-11-14- aaif-goose/goose README,
documentation/docs/getting-started/installation.md,documentation/docs/getting-started/providers.md,documentation/docs/quickstart.md,documentation/docs/experimental/ollama.md, anddocumentation/docs/guides/tool-shim.mdat commit2365bc5e898a38133c373eef2d1357416acd66e2, committed 2026-11-13T18:57:44Z. GitHub license field Apache-2.0.pushed_at2026-11-14T03:56:38Z - Linux CLI install:
curl -fsSL https://github.com/aaif-goose/goose/releases/download/stable/download_cli.sh | bash, and the same URL withCONFIGURE=false. This page did not run them - Providers page, Other Models:
ollama run qwen2.5,goose configure, Ollama,OLLAMA_HOSTexamplehttp://localhost:11434, modelqwen2.5. Empty host becomeslocalhost:11434. Tool-calling warning is quoted above. Other provider names in that menu are not copied. Context length 4096 tokens.OLLAMA_CONTEXT_LENGTH - Providers page, custom endpoint section:
OPENAI_HOSTis the custom endpoint URL. The vLLM example isOPENAI_HOST=https://your-vllm-endpoint.internal. The table’s default host is not copied. Custom provider API types copied here areOpenAI CompatibleandOllama Compatible. Examplebase_urlhttps://api.company.com/v1/chat/completions. Example model names are not copied. Linux directory~/.config/goose/custom_providers/ - Quickstart:
goose session. The custom-provider example printsgoose session start --provider custom_corp_api. This page does not choose between them - Tool shim: experimental. Default interpreter
mistral-nemo.ollama pull mistral-nemo.export GOOSE_TOOLSHIM=true. OptionalGOOSE_TOOLSHIM_OLLAMA_MODEL=llama3.2. Experimental page session lineGOOSE_TOOLSHIM=true goose session - LICENSE at that commit: Apache License, Version 2.0. Appendix copyright line Copyright 2024 Block, Inc.
- Hugging Face model API
unsloth/Llama-3.1-8B-Instruct:gatedfalse,cardData.licensellama3.1,pipeline_tagtext-generation,lastModified2025-05-09T19:32:57.000Z,sha4699cc75b550f9c6f3173fb80f4703b62d946aa5. The card YAML includesextra_gated_prompt. The model information table prints 8B, 70B, 405B, and 128k.base_modelnames a repository this page does not copy.hf downloadis added on this page and was not run. The training-hardware section is not copied - Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (
generated_at2026-11-14T10:04:57.194Z).GET /v1/public/compute-capabilitiesreturned feature flags and no GPU list. The catalog had no goose template
Cards you can start are on the Deploy page. Amounts are not copied here.
Last updated: 2026-11-14 (Asia/Shanghai).