How to run Tabby self-hosted coding assistant on a cloud GPU

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Tabby from the official repository the way the Docker or Linux page prints, with --model and --chat-model set to ids from Tabby’s model registry, and connect the VS Code extension to the endpoint that page prints. Copy $HOME/.tabby and the repo you had open in the editor before you destroy the instance, because destroying the instance deletes its disk and stops the server.

An agent can rent the GPU and run this server on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Tabby install. An editor whose Ollama config sets baseURL to http://localhost:11434/v1 is How to use Kilo Code with a self-hosted model on a cloud GPU. An editor whose local-model page sets the Ollama api_url to http://localhost:11434 is How to use Zed’s AI agent with a self-hosted model on a cloud GPU. A terminal client whose OpenAI-compatible page sets OPENAI_API_BASE is How to use Aider with a self-hosted model on a cloud GPU. An editor client whose install page is the VS Code extension is How to use Cline with a self-hosted model on a cloud GPU. An editor whose config.yaml sets an apiBase for chat, autocomplete, and embeddings is How to use Continue with a self-hosted model on a cloud GPU. An editor whose Ollama provider defaults to http://localhost:11434 is How to use Roo Code with a self-hosted model on a cloud GPU. A coding agent whose providers page sets package to the OpenAI-compatible provider is How to use OpenCode with a self-hosted model on a cloud GPU. A coding-agent server whose card prints --tool-call-parser qwen3_coder is How to serve Qwen3-Coder for coding agents on a cloud GPU. A platform whose single-container guide sets CHAT_BASE_URL to http://host.docker.internal:11434/v1 for a local OpenAI-compatible server is How to run AutoGPT with a self-hosted model on a cloud GPU. An agent whose Docker command publishes port 5080, and whose Ollama provider sets api_base to http://host.docker.internal:11434, is How to run Agent Zero with a self-hosted model on a cloud GPU. An agent whose Ollama example sets base_url to http://localhost:11434/v1 is How to run OpenManus with a self-hosted model on a cloud GPU. A chat app whose Docker page runs docker compose up -d and whose Ollama page sets baseURL to http://localhost:11434/v1/ is How to run LibreChat agents with a self-hosted model on a cloud GPU. A chat app whose README runs docker compose up -d and whose environment page sets OLLAMA_PROXY_URL to http://127.0.0.1:11434 is How to run LobeChat agents with a self-hosted model on a cloud GPU. A terminal agent whose install page runs curl -fsSL https://www.openinterpreter.com/install | sh, and whose models page sets the Ollama base URL to http://localhost:11434/v1, is How to run Open Interpreter with a self-hosted model on a cloud GPU. A workspace app whose Docker quickstart publishes port 3001, and whose Ollama page sets the URL to http://127.0.0.1:11434, is How to run AnythingLLM agents with a self-hosted model on a cloud GPU. A repository agent whose local-models section sets api_base to http://localhost:11434 is How to run SWE-agent with a self-hosted model on a cloud GPU. A workflow editor whose Docker install publishes port 5678, and whose Ollama credentials default the Base URL to http://localhost:11434, is How to run n8n AI agents with a self-hosted model on a cloud GPU. A CLI whose providers page sets OLLAMA_HOST to http://localhost:11434 is How to run the goose agent with a self-hosted model on a cloud GPU. A canvas whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU. A CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A browser chat whose serve command listens on port 8080 is How to run Open WebUI on a cloud GPU. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose OllamaChatGenerator sets url to http://localhost:11434 is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b is How to run Agno agents with a self-hosted model on a cloud GPU. A TypeScript agent whose local-models section sets url on the model object is How to run Mastra agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. The README install is pip install --upgrade metagpt: How to run MetaGPT with a self-hosted model on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. The Cursor fields on How do I use CloudGPU as a custom base URL in Cursor? are for that hosted URL, not for the localhost URL of a server you start. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the server raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.

The LICENSE file says content outside the ee directory and third-party components is available under the Apache 2.0 license, and the copyright line is Copyright © 2025 TabbyML, Inc.

The commands below are copied from the upstream pages named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The official repository is TabbyML/tabby. The model-administration page’s remote example uses model ids this page does not copy. The registry also prints ids this page does not copy.

What the registry prints about memory

The model registry, fetched 2026-11-14, prints no GPU GB figure. For completion models (--model) it says that for 1B to 3B models it is advisable to have at least an NVIDIA T4, a 10 Series GPU, or a 20 Series GPU, or Apple Silicon such as the M1, and that for 7B to 13B models it recommends an NVIDIA V100, an A100, a 30 Series GPU, or a 40 Series GPU. For chat models (--chat-model) it says to use a model with at least 1B parameters. It does not repeat the series list for chat. This page does not treat a series name as a vram_gb value, and it does not mark a catalog row as enough.

The Docker command’s ids are StarCoder-1B (registry license BigCode-OpenRAIL-M) and Qwen2-1.5B-Instruct (registry license Apache 2.0). StarCoder-1B is a 1B completion id, so the 1B-to-3B series sentence is the one that names hardware for that size. The registry’s Apache 2.0 chat ids also include Mistral-7B, Qwen2.5-Coder-0.5B-Instruct, Qwen2.5-Coder-1.5B-Instruct, Qwen2.5-Coder-7B-Instruct, Qwen2.5-Coder-14B-Instruct, Qwen2.5-Coder-32B-Instruct, Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B, Qwen3-8B, Qwen3-14B, Qwen3-32B, Qwen3-30B-A3B, Qwen3-235B-A22B, and Yi-Coder-9B-Chat. The Docker command does not use those ids. This page does not replace the command’s ids.

Catalog names that match series the registry names, with the catalog vram_gb only: 20 Series rows are RTX 2080 at 8 and RTX 2080 Ti at 11. V100 rows are V100 16G at 16 and V100 32G at 32. A100 rows are A100 40G at 40 and A100 80G at 80. 30 Series rows are RTX 3080 at 10, RTX 3080 Ti at 12, and RTX 3090 at 24. 40 Series rows are RTX 4090 at 24, RTX 4090D at 24, RTX 4080 SUPER 32G at 32, RTX 4090 48G at 48, and RTX 4090D 48G at 48. The catalog list this page read has no T4 row, no 10 Series row, and no M1 row.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T14:09:20.887Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Rows are sorted by vram_gb, then by name. Which listing you can start is on Deploy.

The catalog template list had no Tabby template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as Tabby.

Catalog name vram_gb
RTX 2080 8
RTX 3080 10
RTX 2080 Ti 11
RTX 3080 Ti 12
V100 16G 16
A30 24
RTX 3090 24
RTX 4090 24
RTX 4090D 24
RTX 4080 SUPER 32G 32
RTX 5090 32
RTX 5090D 32
V100 32G 32
A100 40G 40
A800 40G 40
A40 48
RTX 4090 48G 48
RTX 4090D 48G 48
RTX A6000 48
Ascend 910B 64
NVIDIA RTX PRO 5000 72G 72
RTX PRO 5000 72G 72
A100 80G 80
A800 80G 80
H800 80
RTX 6000D 83
H20 96G 96
RTX PRO 6000 96G 96
RTX PRO 6000 Max-Q 96
H20-3e 141G 141

Install Tabby with CUDA

The Docker page says to install the NVIDIA Container Toolkit, and then prints this command. It does not print the toolkit’s own install command. This page does not add one.

docker run -d \
  --name tabby \
  --gpus all \
  -p 8080:8080 \
  -v $HOME/.tabby:/data \
  registry.tabbyml.com/tabbyml/tabby \
    serve \
    --model StarCoder-1B \
    --chat-model Qwen2-1.5B-Instruct \
    --device cuda

The page says that after Tabby is running you can access it at http://localhost:8080, and that logs are docker logs -f tabby. For SELinux it adds :Z on the volume. The Docker Compose page uses the same image and the same model flags:

version: '3.5'
services:
  tabby:
    restart: always
    image: registry.tabbyml.com/tabbyml/tabby
    command: serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct --device cuda
    volumes:
      - "$HOME/.tabby:/data"
    ports:
      - 8080:8080
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

The README at commit 21b29048d7bcf6b94f9f482f2d0fd05efadfd19f prints a different image string and a different flag order. This page does not merge the two commands. The README section heading includes a duration. This page does not copy that heading. The command, copied:

docker run -it \
  --gpus all -p 8080:8080 -v $HOME/.tabby:/data \
  tabbyml/tabby \
  serve --model StarCoder-1B --device cuda --chat-model Qwen2-1.5B-Instruct

The Linux page says to download a zip from https://github.com/TabbyML/tabby/releases. For a GPU it says to download tabby_x86_64-manylinux2014-cuda117.zip, and that this example assumes CUDA 11.7. It says you need the nvidia-cuda-toolkit package, and on Ubuntu sudo apt install nvidia-cuda-toolkit. It says to ensure CUDA version 11 or higher, and to check with nvcc --version. It says to unzip, then chmod +x tabby llama-server. The GPU serve line, copied. The comment says DEVICE is cuda or vulkan:

./tabby serve --model StarCoder-1B --chat-model Qwen2-1.5B-Instruct --device $DEVICE

The page says you can choose different models, as shown in the model registry, and that you can then visit http://localhost:8080. It does not print a GPU GB figure. The cuda117 zip name and the “CUDA 11 or higher” sentence do not name compute capability 12.0. This page did not download the zip and did not run the container.

The same page’s Katana block is optional. It is a crawler for a docs context provider, not the model server. This page did not install it. The commands, copied:

curl -L https://github.com/projectdiscovery/katana/releases/download/v1.1.2/katana_1.1.2_linux_amd64.zip -o katana.zip
unzip katana.zip katana
sudo mv katana /usr/bin/
rm katana.zip

The pages this guide read do not print hf download or huggingface-cli. Tabby’s serve command takes a registry id, not a Hugging Face repository id. How to download a Hugging Face model is a different download. This page does not paste it into tabby serve.

The model administration page says local models are a llama.cpp subprocess, and that supported local models are in the registry. Its local example, copied:

[model.completion.local]
model_id = "StarCoder2-3B"

[model.chat.local]
model_id = "Mistral-7B"

[model.embedding.local]
model_id = "Nomic-Embed-Text"

StarCoder2-3B is a registry completion id (BigCode-OpenRAIL-M). Mistral-7B and Nomic-Embed-Text are registry ids with Apache 2.0. The page says the default embedding model is Nomic-Embed-Text. The remote example on that page sets kind to openai/chat. Its model ids and its endpoint are not copied. The page says the chat model is broadly compatible with the OpenAI-compatible API. This page does not copy the vendor wording in that sentence.

Connect the editor

The register-account page says to open the homepage URL the server printed, for example http://localhost:8080. The first registered account is the admin account and receives the owner role. The homepage shows the credentials for the IDE extension.

The VS Code page says to open Quick Open (Ctrl/Cmd+P), paste this, and press enter:

ext install TabbyML.vscode-tabby

It says to run Tabby: Connect to Server... and enter the server endpoint URL and account token. The configuration page prints this file, ~/.tabby-client/agent/config.toml:

[server]
endpoint = "http://localhost:8080"
token = "your-token-here"

The comment says that when token is set, the request header is Authorization = "Bearer $token". The setup page says to fill in the Endpoint from the homepage, then Set Credentials with the token from the homepage, and that a connected icon in the status bar means the connection succeeded.

The VS Code page says autocomplete suggests multi-line completions as you write. For chat it says to start a session in the chat view from the activity bar, to select code and run Tabby: Explain This, and to run Tabby: Start Inline Editing (Ctrl/Cmd+I). It does not print a sample completion string. This page does not invent one. This page did not install the extension.

The Tabby pages this guide read do not print an ssh command. How to use SSH port forwarding on a cloud GPU is how a program on your computer reaches a port on the instance. This page does not add an ssh command. The Docker command publishes 8080:8080. After a forward that lands on port 8080 of your computer, the endpoint the configuration page already prints is http://localhost:8080.

When it fails

CUDA out of memory. The registry, the Docker page, the Linux page, and the README print no GPU GB figure. If the process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller registry id for you.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The Linux page’s GPU zip example is tabby_x86_64-manylinux2014-cuda117.zip, and it says that example assumes CUDA 11.7 and that you need CUDA 11 or higher. It does not name compute capability 12.0. The Docker image strings are registry.tabbyml.com/tabbyml/tabby and tabbyml/tabby, with no digest. The RTX 5090 guide says that card is compute capability 12.0 (sm_120). This page did not re-test either image or the cuda117 zip on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

Two image strings. The docs Docker command and the README Docker command are not the same command. This page does not choose which image a later pull should use.

Account token. The VS Code page says Tabby: Connect to Server... takes an endpoint and an account token. The configuration comment says a token adds a Bearer header. The register-account page says the homepage shows those credentials. This page did not register an account.

What to copy off, then destroy

The Docker and Compose files mount $HOME/.tabby at /data. Copy that directory, and the repo you had open in the editor, before you destroy the instance. The server stops with the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The billing page states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Amounts are not copied here.

Frequently asked questions

What memory figure does the Tabby registry print?

The registry prints no GPU GB figure. For 1B to 3B completion models it names an NVIDIA T4, a 10 Series GPU, a 20 Series GPU, or an M1. For 7B to 13B completion models it names an NVIDIA V100, an A100, a 30 Series GPU, or a 40 Series GPU. For chat models it says at least 1B parameters and does not repeat the series list. This page does not mark a catalog row as enough. The catalog list this page read has no T4 row, no 10 Series row, and no M1 row. Which listing you can start is on https://cloudgpu.app/deploy.

Which command starts Tabby on a GPU?

The Docker page, after naming the NVIDIA Container Toolkit without printing its install command, runs registry.tabbyml.com/tabbyml/tabby with --model StarCoder-1B, --chat-model Qwen2-1.5B-Instruct, and --device cuda, and publishes 8080:8080. The README command uses image tabbyml/tabby and a different flag order. The Linux page’s GPU zip example is tabby_x86_64-manylinux2014-cuda117.zip, assumes CUDA 11.7, says CUDA 11 or higher, and serves with --device $DEVICE where DEVICE is cuda or vulkan. StarCoder-1B is registry license BigCode-OpenRAIL-M. Qwen2-1.5B-Instruct is registry license Apache 2.0. This page did not run the commands.

How does the VS Code page connect to the server?

The install command is ext install TabbyML.vscode-tabby. Tabby: Connect to Server… takes the endpoint URL and the account token. The configuration file sets endpoint to http://localhost:8080. The register-account page says the first account is the admin and the homepage shows the credentials. Chat is a session in the activity-bar chat view, Tabby: Explain This, or Tabby: Start Inline Editing. The pages do not print an SSH command and do not print a sample completion string. This page did not install the extension.

What license sentence does the file print?

The LICENSE file says content outside the ee directory and third-party components is available under the Apache 2.0 license. The copyright line is Copyright © 2025 TabbyML, Inc. The Apache header in that file is Version 2.0, January 2004. The GitHub license field on 2026-11-14 was Other, SPDX NOASSERTION. Repository HEAD was 21b29048d7bcf6b94f9f482f2d0fd05efadfd19f, committed 2026-06-30T16:53:14Z. That license text is not a CloudGPU rate.

Sources (read 2026-11-14, Asia/Shanghai)

Cards you can start are on the Deploy page. Amounts are not copied here.

Last updated: 2026-11-14 (Asia/Shanghai).