How to use Kilo Code with a self-hosted model on a cloud GPU

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Kilo Code from the official repository the way the install page prints, serve an open-weight model on that machine, and set the Ollama or OpenAI-compatible base URL to that server. Copy the project Kilo edits, and any session transcript you export, before you destroy the instance, because destroying the instance deletes its disk and stops the server.

An agent can rent the GPU and serve the model the editor calls. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Kilo Code setting. An editor whose local-model page sets the Ollama api_url to http://localhost:11434 is How to use Zed’s AI agent with a self-hosted model on a cloud GPU. A coding assistant whose Docker command publishes port 8080 and passes --model and --chat-model is How to run Tabby self-hosted coding assistant on a cloud GPU. A terminal client whose OpenAI-compatible page sets OPENAI_API_BASE is How to use Aider with a self-hosted model on a cloud GPU. An editor client whose install page is the VS Code extension is How to use Cline with a self-hosted model on a cloud GPU. An editor whose config.yaml sets an apiBase for chat, autocomplete, and embeddings is How to use Continue with a self-hosted model on a cloud GPU. An editor whose Ollama provider defaults to http://localhost:11434 is How to use Roo Code with a self-hosted model on a cloud GPU. A coding agent whose providers page sets package to the OpenAI-compatible provider is How to use OpenCode with a self-hosted model on a cloud GPU. A coding-agent server whose card prints --tool-call-parser qwen3_coder is How to serve Qwen3-Coder for coding agents on a cloud GPU. A platform whose single-container guide sets CHAT_BASE_URL to http://host.docker.internal:11434/v1 for a local OpenAI-compatible server is How to run AutoGPT with a self-hosted model on a cloud GPU. An agent whose Docker command publishes port 5080, and whose Ollama provider sets api_base to http://host.docker.internal:11434, is How to run Agent Zero with a self-hosted model on a cloud GPU. An agent whose Ollama example sets base_url to http://localhost:11434/v1 is How to run OpenManus with a self-hosted model on a cloud GPU. A chat app whose Docker page runs docker compose up -d and whose Ollama page sets baseURL to http://localhost:11434/v1/ is How to run LibreChat agents with a self-hosted model on a cloud GPU. A chat app whose README runs docker compose up -d and whose environment page sets OLLAMA_PROXY_URL to http://127.0.0.1:11434 is How to run LobeChat agents with a self-hosted model on a cloud GPU. A terminal agent whose install page runs curl -fsSL https://www.openinterpreter.com/install | sh, and whose models page sets the Ollama base URL to http://localhost:11434/v1, is How to run Open Interpreter with a self-hosted model on a cloud GPU. A workspace app whose Docker quickstart publishes port 3001, and whose Ollama page sets the URL to http://127.0.0.1:11434, is How to run AnythingLLM agents with a self-hosted model on a cloud GPU. A repository agent whose local-models section sets api_base to http://localhost:11434 is How to run SWE-agent with a self-hosted model on a cloud GPU. A workflow editor whose Docker install publishes port 5678, and whose Ollama credentials default the Base URL to http://localhost:11434, is How to run n8n AI agents with a self-hosted model on a cloud GPU. A CLI whose providers page sets OLLAMA_HOST to http://localhost:11434 is How to run the goose agent with a self-hosted model on a cloud GPU. A canvas whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU. A CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A browser chat whose serve command listens on port 8080 is How to run Open WebUI on a cloud GPU. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose OllamaChatGenerator sets url to http://localhost:11434 is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b is How to run Agno agents with a self-hosted model on a cloud GPU. A TypeScript agent whose local-models section sets url on the model object is How to run Mastra agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. The README install is pip install --upgrade metagpt: How to run MetaGPT with a self-hosted model on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. The Cursor fields on How do I use CloudGPU as a custom base URL in Cursor? are for that hosted URL, not for the localhost URL of a server you start. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the server raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.

The LICENSE file is the MIT License, and the GitHub license field on 2026-11-14 was MIT.

The steps below are copied from the upstream files named in each step, at the commit in Sources. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The official repository is Kilo-Org/kilocode. The provider pages name other providers. This page does not copy those names. It copies the Ollama provider, the LM Studio provider, and the OpenAI-compatible provider.

What the Ollama page prints about memory

The Ollama page (commit 85eb887a7560f452211bcd382eafe8cb2f8df497, read 2026-11-14) says you will need a GPU with a large amount of VRAM (24GB or more), or a MacBook with a large amount of unified RAM (32GB or more), to run the models that section discusses. The tags that section names are qwen3-coder:30b, qwen3-coder:480b, and devstral:24b. The page does not print a separate GPU GB figure for any one of those tags. This page does not split the 24GB sentence into a per-tag figure the page does not print, and it does not mark a catalog row as enough.

The same page says you need at least 32k of context to get decent results, and that increasing the context size increases memory usage. It says API requests time out after 10 minutes. Those lines are a context length and a request timeout. This page does not treat them as a startup time. The opening paragraph also includes a comparison this page does not copy, and it names hosted model families this page does not copy.

The OpenAI-compatible page and the LM Studio page print no GPU GB figure. The install page prints no GPU GB figure.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T14:09:20.887Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Rows are sorted by vram_gb, then by name. Which listing you can start is on Deploy.

The catalog template list had no Kilo Code template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as Kilo Code.

Catalog name vram_gb
RTX 2080 8
RTX 3080 10
RTX 2080 Ti 11
RTX 3080 Ti 12
V100 16G 16
A30 24
RTX 3090 24
RTX 4090 24
RTX 4090D 24
RTX 4080 SUPER 32G 32
RTX 5090 32
RTX 5090D 32
V100 32G 32
A100 40G 40
A800 40G 40
A40 48
RTX 4090 48G 48
RTX 4090D 48G 48
RTX A6000 48
Ascend 910B 64
NVIDIA RTX PRO 5000 72G 72
RTX PRO 5000 72G 72
A100 80G 80
A800 80G 80
H800 80
RTX 6000D 83
H20 96G 96
RTX PRO 6000 96G 96
RTX PRO 6000 Max-Q 96
H20-3e 141G 141

Install Kilo Code

The install page says the current extension is distributed as the pre-release version on the VS Code Marketplace. Its VS Code steps, copied:

  1. Open VS Code
  2. Go to Extensions (Ctrl+Shift+X / Cmd+Shift+X)
  3. Search for “Kilo Code”
  4. Click the dropdown arrow next to Install and select Install Pre-Release Version

The page says the pre-release label is a Marketplace distribution channel. It says to verify VS Code version 1.84.0 or later if installation fails. The same page’s Open VSX steps search for “Kilo Code” and click Install. Its VSIX steps download a .vsix from the GitHub releases page and use Extensions, “Install from VSIX…”. This page did not install the extension.

The install page’s CLI tab includes packages/kilo-docs/markdoc/partials/install-cli.md. That partial, copied:

npm install -g @kilocode/cli

kilo --version

The partial also says that on an older CPU without AVX the CLI may crash with “Illegal instruction”, and that the baseline archives on the releases page are kilo-linux-x64-baseline.tar.gz, kilo-darwin-x64-baseline.zip, and kilo-windows-x64-baseline.zip. The README at the same commit prints more installers than that partial. They are not the same list. README lines, copied, without the sentences on that page that name hosted models:

npm install -g @kilocode/cli
curl -fsSL https://kilo.ai/cli/install | bash
pnpm add -g @kilocode/cli
bun add -g @kilocode/cli
brew install Kilo-Org/tap/kilo
paru -S kilo-bin

The README says to run kilo in a project directory. This page did not run the installers.

The Kilo pages this guide read do not print hf download or huggingface-cli. The Ollama page prints ollama pull. Weight download for a vLLM or SGLang server is a different guide: How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. This page does not paste that command into Kilo’s config, because the Kilo pages do not print it.

Point Kilo at the endpoint

The Ollama page says to use the Ollama provider for a local daemon started with ollama serve. It says local models use an ollama/ prefix plus the model name. Its example is ollama/qwen3-coder:30b. It says the hosted path does not connect to http://localhost:11434. Its setup, copied:

ollama serve

ollama pull qwen3-coder:30b

The page also prints ollama pull with a placeholder model name, then the qwen3-coder:30b example. It says the current recommendation for the agent is qwen3-coder:30b, that this tag sometimes fails to call tools correctly, and that qwen3-coder:480b is less likely to have that problem. It names devstral:24b as an alternative. It says to set “Context Window Size (num_ctx)” in the API Provider settings. It does not print a numeric num_ctx in that sentence. The 32k figure above is the separate sentence about context length.

CLI config the page prints. The file path is ~/.config/kilo/kilo.json or ./kilo.json:

{
  "provider": {
    "ollama": {
      "baseURL": "http://localhost:11434/v1",
    },
  },
}

{
  "model": "ollama/qwen3-coder:30b",
}

The page says no API key is needed for local Ollama, and that you can configure the base URL if Ollama is running on a different host. It does not print an ssh command. How to use SSH port forwarding on a cloud GPU is how a program on your computer reaches a port on the instance. This page does not add an ssh command. After a forward that lands on port 11434 of your computer, the base URL the page already prints is http://localhost:11434/v1.

The LM Studio page says LM Studio’s local server emulates the OpenAI-compatible API. Its CLI config, copied. The file path on that tab is ~/.config/kilo/kilo.jsonc or ./kilo.jsonc. That path is not the .json path on the Ollama tab. This page does not merge them.

{
  "provider": {
    "lmstudio": {
      "options": {
        "baseURL": "http://localhost:1234/v1",
      },
    },
  },
}

{
  "model": "lmstudio/codellama-7b",
}

The page says to open the Local Server tab, select a downloaded model, and click Start Server. It says you can configure the base URL if LM Studio is running on a different host or port. It does not print an ssh command. The model id in that block is the one the page prints. The same page lists other example ids. This page does not add ids the page does not print.

The OpenAI-compatible page says to open Settings, the Providers tab, scroll to the bottom, and click Custom provider. The Provider API value this page uses is OpenAI Compatible. The page lists other Provider API values and names other providers. This page does not copy those names. The CLI example, copied. The provider key vllm is the example identifier that page prints. The model key qwen35 is the example id that page prints. It is not a Hugging Face repository id this page verified.

{
  "provider": {
    "vllm": {
      "npm": "@ai-sdk/openai-compatible",
      "models": {
        "qwen35": {
          "name": "Qwen 3.5",
          "limit": {
            "context": 262144,
            "output": 16384,
          },
        },
      },
      "options": {
        "apiKey": "none",
        "baseURL": "http://my.url:8000/v1",
      },
    },
  },
}

{
  "model": "vllm/qwen35",
}

The page says npm @ai-sdk/openai-compatible is the package for an OpenAI-compatible Chat Completions endpoint, and that it is the default when omitted. It says options.apiKey can be any non-empty string, for example none, when the server does not require authentication. It says a Base URL may be a root such as https://api.provider.com/v1 or a full chat-completions URL. The example base URL is http://my.url:8000/v1. This page does not replace that placeholder with a port the example does not print. A vLLM server’s listen address is a different document: How to serve an OpenAI-compatible API with vLLM on a cloud GPU. This page does not merge that port into the example.

The page says that after a valid Base URL and API key, Kilo can query /v1/models. If that fails, it says you can enter a model id manually. Its search example uses a model id this page does not copy.

A task

The chat page says to click the Kilo Code icon in VS Code’s Primary Side Bar to open the sidebar chat. In the CLI, it says to run kilo. It says to type a request and press Enter. One request it prints, copied:

create a new file named utils.py and add a function called add that takes two numbers as arguments and returns their sum

The page says /export in chat, or a local session’s History context menu, Export session transcript, writes a Markdown file. This page did not run a task.

When it fails

Tool calls. The Ollama page says qwen3-coder:30b sometimes fails to call tools correctly, and that changing the prompt is what it suggests. It does not name a different tag that reliably calls tools. This page does not pick one.

Invalid API key, model not found, connection errors. The OpenAI-compatible page’s troubleshooting lines are “Invalid API Key”, “Model Not Found”, and “Connection Errors”. It says to check the key, the model id, and the Base URL. This page did not reproduce those errors.

API request timeout. The Ollama page says API requests time out after 10 minutes, and that the setting is VS Code Extensions, Kilo Code gear menu, Settings, API Request Timeout. That line is a request timeout. This page does not treat it as a startup time.

CUDA out of memory. The only GPU GB figure the pages this guide read print is the Ollama page’s “24GB or more” for the models that section discusses. If the model server raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller tag for you.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The Kilo pages this guide read do not pin a PyTorch build and do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120). This page did not re-test a model server on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

What to copy off, then destroy

Kilo edits files in the project you started it in. The chat page says /export writes a Markdown transcript. Copy the project and that transcript before you destroy the instance. The model server stops with the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The billing page states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Amounts are not copied here.

Frequently asked questions

What memory figure does the Kilo Ollama page print?

The Ollama page says a GPU with 24GB or more of VRAM, or a MacBook with 32GB or more of unified RAM, for the models that section discusses. Those tags are qwen3-coder:30b, qwen3-coder:480b, and devstral:24b. The page does not print a separate GPU GB figure for any one tag. It says at least 32k of context, and that API requests time out after 10 minutes. Those lines are a context length and a request timeout. The OpenAI-compatible page, the LM Studio page, and the install page print no GPU GB figure. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Which settings point Kilo at a server on the GPU?

The Ollama page sets baseURL to http://localhost:11434/v1 and model to ollama/qwen3-coder:30b, after ollama serve and ollama pull qwen3-coder:30b. The LM Studio page sets baseURL to http://localhost:1234/v1 and model to lmstudio/codellama-7b. The OpenAI-compatible example sets npm to @ai-sdk/openai-compatible, baseURL to http://my.url:8000/v1, and model to vllm/qwen35. The Provider API value is OpenAI Compatible. Those pages do not print an SSH command. This page did not run them.

What task does the chat page print?

The chat page says to open the Kilo Code sidebar icon, or run kilo, then type a request and press Enter. One request it prints is: create a new file named utils.py and add a function called add that takes two numbers as arguments and returns their sum. /export writes a Markdown transcript. This page did not run the task.

What license sentence does the file print?

The LICENSE file is the MIT License. The copyright lines are Copyright © 2026 Kilo Code and Copyright © 2025 opencode. The GitHub license field on 2026-11-14 was MIT. Repository HEAD was 85eb887a7560f452211bcd382eafe8cb2f8df497, committed 2026-11-13T21:00:38Z. That license text is not a CloudGPU rate.

Sources (read 2026-11-14, Asia/Shanghai)

  • Kilo-Org/kilocode commit 85eb887a7560f452211bcd382eafe8cb2f8df497, committed 2026-11-13T21:00:38Z. GitHub license field MIT. LICENSE is the MIT License. Copyright lines: Copyright © 2026 Kilo Code and Copyright © 2025 opencode. The grant paragraph is not copied
  • packages/kilo-docs/pages/getting-started/installing.md at that commit. VS Code pre-release install. VS Code 1.84.0 or later. Open VSX and VSIX. CLI tab includes the partial below
  • packages/kilo-docs/markdoc/partials/install-cli.md at that commit. npm install -g @kilocode/cli. kilo --version. Baseline archives for a CPU without AVX. “Illegal instruction”
  • README at that commit. Extra installers: curl -fsSL https://kilo.ai/cli/install | bash, pnpm, bun, brew install Kilo-Org/tap/kilo, paru -S kilo-bin. Sentences on that page that name hosted models are not copied
  • packages/kilo-docs/pages/ai-providers/ollama.md at that commit. 24GB or more of VRAM. 32GB or more of unified RAM. Tags qwen3-coder:30b, qwen3-coder:480b, devstral:24b. At least 32k context. API request timeout 10 minutes. ollama serve. ollama pull qwen3-coder:30b. baseURL http://localhost:11434/v1. model ollama/qwen3-coder:30b. A comparison and hosted-model names in the opening paragraph are not copied
  • packages/kilo-docs/pages/ai-providers/lmstudio.md at that commit. baseURL http://localhost:1234/v1. model lmstudio/codellama-7b. No GPU GB figure. No SSH command
  • packages/kilo-docs/pages/ai-providers/openai-compatible.md at that commit. Provider API OpenAI Compatible. npm @ai-sdk/openai-compatible. Example baseURL http://my.url:8000/v1. Example model key qwen35. limit.context 262144. limit.output 16384. Other provider names and other Provider API values are not copied
  • packages/kilo-docs/pages/code-with-ai/agents/chat-interface.md at that commit. Sidebar icon or kilo. The utils.py request. /export
  • The pages above do not print hf download, an ssh command, or a per-tag GPU GB figure
  • Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (generated_at 2026-11-14T14:09:20.887Z). GET /v1/public/compute-capabilities returned feature flags and no GPU list. The catalog had no Kilo Code template. It listed Ollama template ids jupyterlab-ollama, ollama-qwen3, and ollama

Cards you can start are on the Deploy page. Amounts are not copied here.

Last updated: 2026-11-14 (Asia/Shanghai).