How to run Khoj AI assistant with a self-hosted model on a cloud GPU

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Khoj with the Docker or pip commands the official self-host page prints, serve a model on that machine, and set OPENAI_BASE_URL to that server. Copy the compose volumes before you destroy the instance, because destroying the instance deletes its disk and stops the server.

An agent can rent the GPU and serve the model this assistant calls. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Khoj install. A research agent whose Ollama block sets FAST_LLM to ollama:llama3 and EMBEDDING to ollama:nomic-embed-text is How to run GPT Researcher with a self-hosted model on a cloud GPU. An answering engine whose Docker command publishes port 3000, and whose Ollama base URL placeholder is http://localhost:11434, is How to run Perplexica AI search with a self-hosted model on a cloud GPU. An editor whose Ollama config sets baseURL to http://localhost:11434/v1 is How to use Kilo Code with a self-hosted model on a cloud GPU. An editor whose local-model page sets the Ollama api_url to http://localhost:11434 is How to use Zedā€™s AI agent with a self-hosted model on a cloud GPU. A coding assistant whose Docker command publishes port 8080 and passes --model and --chat-model is How to run Tabby self-hosted coding assistant on a cloud GPU. A terminal client whose OpenAI-compatible page sets OPENAI_API_BASE is How to use Aider with a self-hosted model on a cloud GPU. An editor client whose install page is the VS Code extension is How to use Cline with a self-hosted model on a cloud GPU. An editor whose config.yaml sets an apiBase for chat, autocomplete, and embeddings is How to use Continue with a self-hosted model on a cloud GPU. An editor whose Ollama provider defaults to http://localhost:11434 is How to use Roo Code with a self-hosted model on a cloud GPU. A coding agent whose providers page sets package to the OpenAI-compatible provider is How to use OpenCode with a self-hosted model on a cloud GPU. A platform whose single-container guide sets CHAT_BASE_URL to http://host.docker.internal:11434/v1 for a local OpenAI-compatible server is How to run AutoGPT with a self-hosted model on a cloud GPU. An agent whose Docker command publishes port 5080, and whose Ollama provider sets api_base to http://host.docker.internal:11434, is How to run Agent Zero with a self-hosted model on a cloud GPU. An agent whose Ollama example sets base_url to http://localhost:11434/v1 is How to run OpenManus with a self-hosted model on a cloud GPU. A chat app whose Docker page runs docker compose up -d and whose Ollama page sets baseURL to http://localhost:11434/v1/ is How to run LibreChat agents with a self-hosted model on a cloud GPU. A chat app whose README runs docker compose up -d and whose environment page sets OLLAMA_PROXY_URL to http://127.0.0.1:11434 is How to run LobeChat agents with a self-hosted model on a cloud GPU. A terminal agent whose install page runs curl -fsSL https://www.openinterpreter.com/install | sh, and whose models page sets the Ollama base URL to http://localhost:11434/v1, is How to run Open Interpreter with a self-hosted model on a cloud GPU. A workspace app whose Docker quickstart publishes port 3001, and whose Ollama page sets the URL to http://127.0.0.1:11434, is How to run AnythingLLM agents with a self-hosted model on a cloud GPU. A repository agent whose local-models section sets api_base to http://localhost:11434 is How to run SWE-agent with a self-hosted model on a cloud GPU. A workflow editor whose Docker install publishes port 5678, and whose Ollama credentials default the Base URL to http://localhost:11434, is How to run n8n AI agents with a self-hosted model on a cloud GPU. A CLI whose providers page sets OLLAMA_HOST to http://localhost:11434 is How to run the goose agent with a self-hosted model on a cloud GPU. A canvas whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU. A CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A browser chat whose serve command listens on port 8080 is How to run Open WebUI on a cloud GPU. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose OllamaChatGenerator sets url to http://localhost:11434 is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b is How to run Agno agents with a self-hosted model on a cloud GPU. A TypeScript agent whose local-models section sets url on the model object is How to run Mastra agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. The README install is pip install --upgrade metagpt: How to run MetaGPT with a self-hosted model on a cloud GPU. A coding-agent server whose card prints --tool-call-parser qwen3_coder is How to serve Qwen3-Coder for coding agents on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollamaā€™s install and port 11434 are How to run Ollama on a cloud GPU. CloudGPUā€™s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the server raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.

The LICENSE file is the GNU Affero General Public License, Version 3, 19 November 2007, and the GitHub license field on 2026-11-14 was AGPL-3.0.

The steps below are copied from the upstream files named in each step, at the commit in Sources. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The official repository is khoj-ai/khoj. The self-host page names other chat providers. This page does not copy those names. It copies the offline tab, the Ollama page, and the OpenAI-compatible proxy page.

What the self-host page prints about memory

The offline tab of the self-host page says minimum 8 GB RAM and recommends 16Gb VRAM. The same tip also prints a disk figure. This page does not copy it. The tip does not name a catalog card, and it does not print a separate GPU GB figure for llama3.1 or qwen3. This page does not mark a catalog row as enough. The Ollama page and the proxy page print no GPU GB figure.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T17:06:44.884Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Rows are sorted by vram_gb, then by name. Which listing you can start is on Deploy.

The catalog template list had no Khoj template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as Khoj.

Catalog name vram_gb
RTX 2080 8
RTX 3080 10
RTX 2080 Ti 11
RTX 3080 Ti 12
V100 16G 16
A30 24
RTX 3090 24
RTX 4090 24
RTX 4090D 24
RTX 4080 SUPER 32G 32
RTX 5090 32
RTX 5090D 32
V100 32G 32
A100 40G 40
A800 40G 40
A40 48
RTX 4090 48G 48
RTX 4090D 48G 48
RTX A6000 48
Ascend 910B 64
NVIDIA RTX PRO 5000 72G 72
RTX PRO 5000 72G 72
A100 80G 80
A800 80G 80
H800 80
RTX 6000D 83
H20 96G 96
RTX PRO 6000 96G 96
RTX PRO 6000 Max-Q 96
H20-3e 141G 141

Install Khoj

The self-host page says you can install with Docker or pip. Its Linux Docker steps, copied:

mkdir ~/.khoj && cd ~/.khoj
wget https://raw.githubusercontent.com/khoj-ai/khoj/master/docker-compose.yml

cd ~/.khoj
docker-compose up

The page says to set KHOJ_ADMIN_PASSWORD and KHOJ_DJANGO_SECRET_KEY, and optionally KHOJ_ADMIN_EMAIL. The next bullet names hosted API keys. This page does not copy those names. The following bullet says to uncomment OPENAI_BASE_URL to use Ollama running on the host, or set it to the URL of an OpenAI-compatible API such as vLLM or LM Studio. The page says setup is complete once the logs show šŸŒ– Khoj is ready to engage. It says to restart the server after the first run so settings apply. It says Khoj is only accessible on the machine it is running, and points at the remote-access page.

The Linux NVIDIA pip line, copied:

CMAKE_ARGS="-DGGML_CUDA=on" FORCE_CMAKE=1 python -m pip install 'khoj[local]'

The start command, copied:

USE_EMBEDDED_DB="true" khoj --anonymous-mode

The page says --anonymous-mode allows access without login, and that this is usually for local single-user setups. The same ready line is šŸŒ– Khoj is ready to engage. The CPU pip line on that page is python -m pip install 'khoj[local]' without CMAKE_ARGS. This page does not merge the two lines.

docker-compose.yml at this commit sets image: ghcr.io/khoj-ai/khoj:latest and publishes 42110:42110. The server command, copied:

--host="0.0.0.0" --port=42110 -vv --anonymous-mode --non-interactive

The search service is docker.io/searxng/searxng:latest. The server environment sets KHOJ_SEARXNG_URL=http://search:8080. That line does not set a search API key. The online-search page says that with Docker, online search works with SearxNG using that compose file. Other options on that page set a key. This page does not copy those names.

The pages this guide read do not print hf download or huggingface-cli. The Ollama page prints ollama pull. Weight download for a vLLM server is a different guide: How to download a Hugging Face model. This page does not paste that command into the admin form, because these pages do not print it.

Point Khoj at the endpoint

The Ollama page says Ollama exposes a local OpenAI-compatible server. Its pip first-run lines, copied:

ollama pull llama3.1

export OPENAI_BASE_URL="http://localhost:11434/v1/"
khoj --anonymous-mode

Its Docker first-run says to uncomment OPENAI_BASE_URL in the downloaded docker-compose.yml, then docker-compose up. The commented lines in docker-compose.yml at this commit, copied. The comment above them names other providers. This page does not copy that comment.

# - OPENAI_BASE_URL=http://host.docker.internal:11434/v1/
# - KHOJ_DEFAULT_CHAT_MODEL=qwen3

qwen3 is the string that comment prints. This page did not check it as a Hugging Face repository, and it does not copy a GPU GB figure for it. llama3.1 is the tag the Ollama pageā€™s ollama pull line prints. This page did not read a model card for that tag.

The Ollama pageā€™s update steps, for a server that is already running, say to create an AI Model API at http://localhost:42110/server/admin/database/aimodelapi/add with Name ollama, Api Key any string, and Api Base Url http://localhost:11434/v1/. Then a chat model at http://localhost:42110/server/admin/database/chatmodel/add with Name llama3.1, Model Type Openai, the AI Model API from the previous step, and Max prompt size 20000. Then select it at http://localhost:42110/settings.

The proxy page is not the same form. It says Name any name, Api Key any string, Api Base Url is the URL of the OpenAI-compatible API, chat model Name llama3, Model Type Openai, and Max prompt size 2000. Those prompt sizes are not the same number. This page does not merge them. The offline tab says to set the API URL to the local server, for example http://localhost:11434/v1/ for Ollama, restart, and if the models do not load, add the chat model by hand, then open http://localhost:42110.

The self-host page says the web app is http://localhost:42110. The admin panel is http://localhost:42110/server/admin. It says to use localhost, not 127.0.0.1, or the admin panel raises a CSRF error. The client note says the default host URL is http://127.0.0.1:42110 or http://localhost:42110, and that localhost may not work in all cases. Those two sentences are not the same. This page does not merge them.

The remote-access page says to set KHOJ_DOMAIN to a reachable IP or domain, and that the default port is 42110. It does not print an ssh command. How to use SSH port forwarding on a cloud GPU is how a browser on your computer reaches a port on the instance. This page does not add an ssh command. After a forward that lands on port 42110 of your computer, the URL the self-host page already prints is http://localhost:42110.

A chat

The self-host page says to open http://localhost:42110 and start interacting. The chat page says to enter your queries, and that /research goes deeper on a topic. It does not print a sample question. The online-search page prints example queries. Two of them, copied:

/online Deadline for filing taxes 2024.

Give me a summary of this article: https://en.wikipedia.org/wiki/Haitian_Revolution

That page prints other examples. This page does not copy them. This page did not send a query.

When it fails

CSRF on the admin panel. The self-host page says to use localhost, not 127.0.0.1. It also says a custom domain can raise DISALLOWED HOST or Bad Request (400), and to set KHOJ_DOMAIN and KHOJ_NO_HTTPS=True if required. This page did not reproduce those errors.

Settings not applied. The self-host page and the Ollama page both say to restart after the first run or after changing the settings. This page did not test that restart.

CUDA out of memory. The only GPU figure the pages this guide read print is the offline tabā€™s recommended 16Gb VRAM, next to a minimum of 8 GB RAM. If the model server raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller tag for you.

ā€œCUDA error: no kernel image is available for execution on the deviceā€ on an RTX 5090. The pip line sets CMAKE_ARGS="-DGGML_CUDA=on" and does not name compute capability 12.0. The Docker image string is ghcr.io/khoj-ai/khoj:latest and does not name a CUDA tag. The RTX 5090 guide says that card is compute capability 12.0 (sm_120). This page did not re-test the install on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

What to copy off, then destroy

docker-compose.yml declares volumes khoj_config, khoj_db, khoj_models, khoj_search, and khoj_computer. The self-host pageā€™s Docker uninstall is docker-compose down --volumes, and it says that removes containers, networks, images, and volumes. Copy the volumes before you destroy the instance, and before you run that command. The model server stops with the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The billing page states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Amounts are not copied here.

Frequently asked questions

What memory figure does the Khoj self-host page print?

The offline tab says minimum 8 GB RAM and recommends 16Gb VRAM. The same tip also prints a disk figure. This page does not copy it. The tip does not name a catalog card and does not print a per-tag GPU GB figure. The Ollama page and the proxy page print no GPU GB figure. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Which settings point Khoj at a server on the GPU?

The Linux CUDA install is CMAKE_ARGS=ā€œ-DGGML_CUDA=onā€ FORCE_CMAKE=1 python -m pip install ā€˜khoj[local]ā€™. The Ollama page runs ollama pull llama3.1 and sets OPENAI_BASE_URL to http://localhost:11434/v1/. The compose fileā€™s commented line is OPENAI_BASE_URL=http://host.docker.internal:11434/v1/ and KHOJ_DEFAULT_CHAT_MODEL=qwen3. The web app is http://localhost:42110. The ready log is šŸŒ– Khoj is ready to engage. Those pages do not print an SSH command. This page did not run them.

What chat use do the docs print?

The self-host page says to open http://localhost:42110. The chat page lists /research and does not print a sample question. The online-search pageā€™s copied examples are /online Deadline for filing taxes 2024. and a summary request for https://en.wikipedia.org/wiki/Haitian_Revolution. Docker online search uses SearxNG at KHOJ_SEARXNG_URL http://search:8080 and that line does not set a search API key. This page did not send a query.

What license sentence does the file print?

The LICENSE file is the GNU Affero General Public License, Version 3, 19 November 2007. The GitHub license field on 2026-11-14 was AGPL-3.0. Repository HEAD was ae229ca894c0b80ad84664afcfdde523b5e87057, committed 2026-08-02T01:55:40Z. That license text is not a CloudGPU rate.

Sources (read 2026-11-14, Asia/Shanghai)

  • khoj-ai/khoj commit ae229ca894c0b80ad84664afcfdde523b5e87057, committed 2026-08-02T01:55:40Z. GitHub license field AGPL-3.0. LICENSE is the GNU Affero General Public License, Version 3, 19 November 2007. The preamble is not copied
  • documentation/docs/get-started/setup.mdx at that commit. Docker wget and docker-compose up. Linux CUDA pip line. USE_EMBEDDED_DB="true" khoj --anonymous-mode. Ready log šŸŒ– Khoj is ready to engage. http://localhost:42110. Offline tab: minimum 8 GB RAM, recommends 16Gb VRAM. The disk figure in that tip is not copied. Hosted provider names are not copied. CSRF: localhost, not 127.0.0.1
  • docker-compose.yml at that commit. ghcr.io/khoj-ai/khoj:latest. Port 42110:42110. Command --host="0.0.0.0" --port=42110 -vv --anonymous-mode --non-interactive. KHOJ_SEARXNG_URL=http://search:8080. Commented OPENAI_BASE_URL=http://host.docker.internal:11434/v1/ and KHOJ_DEFAULT_CHAT_MODEL=qwen3. The comment that names other providers is not copied. Volumes khoj_config, khoj_db, khoj_models, khoj_search, khoj_computer
  • documentation/docs/advanced/ollama.mdx at that commit. ollama pull llama3.1. OPENAI_BASE_URL=http://localhost:11434/v1/. Admin fields Name ollama, Api Key any string, chat model llama3.1, Model Type Openai, Max prompt size 20000. No GPU GB figure. No ssh command
  • documentation/docs/advanced/use-openai-proxy.mdx at that commit. Chat model name llama3. Max prompt size 2000. Model Type Openai. No GPU GB figure
  • documentation/docs/advanced/remote.md at that commit. KHOJ_DOMAIN. Default port 42110. No ssh command. A certificate sentence on that page is not copied
  • documentation/docs/features/chat.md at that commit. /research. No sample question
  • documentation/docs/features/online_search.md at that commit. Docker SearxNG. Two example queries. Other examples and keyed search options are not copied
  • The pages above do not print hf download or an ssh command
  • Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (generated_at 2026-11-14T17:06:44.884Z). GET /v1/public/compute-capabilities returned feature flags and no GPU list. The catalog had no Khoj template. It listed Ollama template ids jupyterlab-ollama, ollama-qwen3, and ollama

Cards you can start are on the Deploy page. Amounts are not copied here.

Last updated: 2026-11-14 (Asia/Shanghai).