How to run Cognee with a self-hosted model on a cloud GPU

Q: What memory figure do the Cognee docs print?

The LLM providers page says an 8B model quantized to 4-bit needs roughly 6 GB of available VRAM, and it says llama3.1:8b is ~6 GB VRAM at 4-bit. Those are the page's figures. They are not the catalog column vram_gb. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Q: Which install line does the Ollama section print?

The installation page's Ollama section prints uv pip install "cognee[ollama]", then ollama pull llama3.1:8b and ollama pull nomic-embed-text:latest. The README on commit 4053f6cc5536757f26a3665eed95c50088e6dc0b also prints pip install cognee and says Python 3.10 through 3.14. pyproject.toml says version 1.6.8, license Apache-2.0, and requires-python 3.10 or newer and below 3.15. PyPI info.version on 2026-11-14 was 1.6.8, requires_python that same bound, uploaded 2026-11-12T22:37:53.918693Z. This page did not install the package.

Q: How does Cognee point at a server on the GPU?

The local Ollama guide sets LLM_PROVIDER to ollama, LLM_MODEL to llama3.1:8b, and LLM_ENDPOINT to http://localhost:11434. It sets EMBEDDING_PROVIDER to ollama, EMBEDDING_MODEL to nomic-embed-text, EMBEDDING_ENDPOINT to http://localhost:11434/api/embed, and EMBEDDING_DIMENSIONS to 768. The installation page's embedding tag is nomic-embed-text:latest. Those embedding tags are not the same string. remember and recall are the calls in the local guide. The custom section's vLLM accordion uses the hosted_vllm/ prefix and sets LLM_ENDPOINT to https://your-vllm-endpoint/v1. Those pages do not print an SSH command. This page does not copy the hosted-provider sections.

Q: What license does the file print?

The LICENSE file begins with Apache License Version 2.0, January 2004. The GitHub license field on 2026-11-14 was Apache-2.0. pyproject.toml on that commit says license Apache-2.0.

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Cognee with the uv pip install "cognee[ollama]" line the installation page prints for Ollama, pull the two tags that page prints, and set LLM_ENDPOINT and EMBEDDING_ENDPOINT to the local URLs the local Ollama guide prints. Copy the recall printout before you destroy the instance, because destroying the instance deletes its disk.

An agent can rent the GPU and run this library against a model served on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Cognee open-source install. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. An OpenAI-compatible engine whose CUDA 12 container tag is localai/localai:latest-gpu-nvidia-cuda-12 is How to run LocalAI on a cloud GPU. A gateway whose README prints uv tool install 'litellm[proxy]' is How to run LiteLLM Proxy in front of a self-hosted model on a cloud GPU. A memory layer whose Ollama page sets llm.provider to ollama and whose TypeScript example sets url to http://localhost:11434 is How to run Mem0 with a self-hosted model on a cloud GPU. A graph library whose Ollama section sets OpenAIGenericClient base_url to http://localhost:11434/v1 is How to run Graphiti with a self-hosted model on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.

The LICENSE file begins with “Apache License Version 2.0, January 2004”, and the GitHub license field on 2026-11-14 was Apache-2.0.

The commands below are copied from the docs pages and from commit 4053f6cc5536757f26a3665eed95c50088e6dc0b. They were not run on a CloudGPU instance for this page. The README’s first quickstart does not set LLM_PROVIDER. This page follows the local Ollama guide instead. The LLM providers page also names other providers. This page does not copy those sections.

What the docs print about memory

The LLM providers page, read 2026-11-14, says the local setup defaults to an 8B model, llama3.1:8b, and that an 8B model quantized to 4-bit needs roughly 6 GB of available VRAM. The same page’s recommended-models note says llama3.1:8b stays small enough to run on modest hardware, ~6 GB VRAM at 4-bit. Those sentences are the page’s figures. They are not a catalog measurement. The available VRAM figure in that sentence is not the catalog column vram_gb. This page does not mark a catalog row as enough.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T22:14:00.888Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Which listing you can start is on Deploy.

The catalog template list had no Cognee template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as Cognee.

Catalog name	vram_gb
RTX 2080	8
RTX 3080	10
RTX 2080 Ti	11
RTX 3080 Ti	12
V100 16G	16
A30	24
RTX 3090	24
RTX 4090	24
RTX 4090D	24
RTX 4080 SUPER 32G	32
RTX 5090	32
RTX 5090D	32
V100 32G	32
A100 40G	40
A800 40G	40
A40	48
RTX 4090 48G	48
RTX 4090D 48G	48
RTX A6000	48
Ascend 910B	64
NVIDIA RTX PRO 5000 72G	72
RTX PRO 5000 72G	72
A100 80G	80
A800 80G	80
H800	80
RTX 6000D	83
H20 96G	96
RTX PRO 6000 96G	96
RTX PRO 6000 Max-Q	96
H20-3e 141G	141

Install

The installation page says Python 3.10 through 3.14 is required. Its Ollama section prints this install and the two pulls:

uv pip install "cognee[ollama]"
ollama pull llama3.1:8b
ollama pull nomic-embed-text:latest

The README on commit 4053f6cc5536757f26a3665eed95c50088e6dc0b (committed 2026-11-13T00:23:56Z) says Python 3.10–3.14 and prints pip install cognee as the base package. It also prints uv pip install "cognee[gliner]" for the quickstart that does not set LLM_PROVIDER. That extra is not the Ollama extra. pyproject.toml on that commit says version = "1.6.8", license = "Apache-2.0", and requires-python = ">=3.10,<3.15". PyPI cognee info.version on 2026-11-14 was 1.6.8, requires_python >=3.10,<3.15, uploaded 2026-11-12T22:37:53.918693Z. The GitHub license field on 2026-11-14 was Apache-2.0. pushed_at was 2026-11-14T20:53:03Z. This page did not install the package.

The local Ollama guide’s pull lines are ollama pull llama3.1:8b and ollama pull nomic-embed-text. The installation page’s embedding pull is ollama pull nomic-embed-text:latest. Those embedding tags are not the same string. This page does not merge them.

Point Cognee at Ollama

The installation page’s Ollama environment block, copied:

LLM_PROVIDER="ollama"
LLM_MODEL="llama3.1:8b"
LLM_ENDPOINT="http://localhost:11434"
LLM_API_KEY="ollama"

EMBEDDING_PROVIDER="ollama"
EMBEDDING_MODEL="nomic-embed-text:latest"
EMBEDDING_ENDPOINT="http://localhost:11434/api/embed"
EMBEDDING_DIMENSIONS="768"
HUGGINGFACE_TOKENIZER="nomic-ai/nomic-embed-text-v1.5"

The page says LLM_API_KEY="ollama" is a placeholder the client library requires to be non-empty, and that LLM_ENDPOINT is the bare host. On the default structured-output backend, a trailing /v1 on LLM_ENDPOINT is the wrong path. EMBEDDING_ENDPOINT keeps /api/embed. The page says both the LLM and the embedding provider have to be set. This page sets both to ollama.

The local Ollama guide uses EMBEDDING_MODEL nomic-embed-text, without :latest, and it sets the same endpoint strings. It also sets LLM_TEMPERATURE to 0.0. The guide says the graph, vector, and relational stores in the script are embedded and ship with Cognee. CloudGPU does not provide those stores. The settings and the remember/recall calls, copied. The guide also defines a banner helper. This block keeps the configuration and the two calls:

import asyncio
import os
import tempfile
from pathlib import Path

_DATA_DIR = tempfile.mkdtemp(prefix="cognee_ollama_example_")
os.environ["ENABLE_BACKEND_ACCESS_CONTROL"] = "false"
os.environ["CACHING"] = "false"

os.environ["LLM_PROVIDER"] = "ollama"
os.environ["LLM_MODEL"] = "llama3.1:8b"
os.environ["LLM_ENDPOINT"] = "http://localhost:11434"
os.environ["LLM_API_KEY"] = "ollama"
os.environ["LLM_TEMPERATURE"] = "0.0"

os.environ["EMBEDDING_PROVIDER"] = "ollama"
os.environ["EMBEDDING_MODEL"] = "nomic-embed-text"
os.environ["EMBEDDING_ENDPOINT"] = "http://localhost:11434/api/embed"
os.environ["EMBEDDING_DIMENSIONS"] = "768"
os.environ["HUGGINGFACE_TOKENIZER"] = "nomic-ai/nomic-embed-text-v1.5"

import cognee
from cognee.infrastructure.llm.config import get_llm_config
from cognee.modules.search.types import SearchType

cognee.config.set_graph_database_provider("ladybug")
cognee.config.set_vector_db_provider("lancedb")
cognee.config.data_root_directory(str(Path(_DATA_DIR) / "data"))
cognee.config.system_root_directory(str(Path(_DATA_DIR) / "system"))

SAMPLE_TEXT = """\
Cognee is an open-source library that helps developers turn documents into AI memory.
It builds semantic graphs, indexes entities, and stores vectors to enable structured retrieval.
Cognee supports local execution via Ollama as well as hosted cloud providers.
"""

async def main() -> None:
    await cognee.prune.prune_data()
    await cognee.prune.prune_system(metadata=True)
    await cognee.remember(SAMPLE_TEXT, dataset_name="ollama_local_demo", self_improvement=False)
    results = await cognee.recall(
        query_text="What does Cognee help developers do?",
        query_type=SearchType.GRAPH_COMPLETION,
        datasets=["ollama_local_demo"],
    )
    print(results[0].text if results else "<no results>")

if __name__ == "__main__":
    asyncio.run(main())

The guide says llama3.1:8b extracts entities and nomic-embed-text embeds the chunks. It says three sentences is a small input because an 8B model on CPU is slower than a hosted one. That sentence is about CPU execution in the guide. It is not a statement about how long a CloudGPU instance takes to start. The guide says Ladybug is the embedded graph store and LanceDB is the embedded vector store, and that metadata is SQLite under the temporary directory. EMBEDDING_DIMENSIONS is 768 in that script. It is not a GPU GB figure.

On 2026-11-14, GET https://huggingface.co/api/models/nomic-ai/nomic-embed-text-v1.5 reported gated false, cardData.license apache-2.0, pipeline_tag sentence-similarity, lastModified 2026-04-07T14:17:02.000Z, and sha e9b6763023c676ca8431644204f50c2b100d9aab. The script sets HUGGINGFACE_TOKENIZER to that id. It does not print hf download. This page does not add a download line for it.

The LLM providers page says llama3.1, llama3.2, and llama3.3 are in the recommended set, and that a qwen2.5 tag of 14B or larger is in that set too. It says other tags can log a warning and that cognify() still runs. The local guide’s calls are remember and recall, not cognify. This page does not treat those names as one function. This page did not run the script.

How to run Ollama on a cloud GPU is the Ollama install. That server stays on the rented GPU. The local guide does not print an SSH command.

OpenAI-compatible endpoint

The LLM providers page’s custom section says LLM_PROVIDER custom talks to an endpoint that exposes /v1/chat/completions, and that LLM_ENDPOINT is the base URL, usually ending in /v1. Its vLLM accordion sets LLM_MODEL to the hosted_vllm/ prefix plus the id returned by the server’s /v1/models endpoint, LLM_ENDPOINT to https://your-vllm-endpoint/v1, and LLM_API_KEY to .. The filled example under that accordion uses a tag this page does not copy. The same page already names llama3.1:8b as the local model. This page does not put that Ollama tag after hosted_vllm/, because the page says the suffix must match the id the server returns. The openai/ prefix on that page names the API protocol for a compatible server. This page does not copy the hosted-provider examples.

The custom section says embeddings are configured separately. The local guide’s embedding block above is that separate setting. How to serve an OpenAI-compatible API with vLLM on a cloud GPU is the vLLM install. The Cognee vLLM accordion does not print port 8000.

When it fails

CUDA out of memory. The LLM providers page says an 8B model quantized to 4-bit needs roughly 6 GB of available VRAM, and the same page says “~6 GB VRAM at 4-bit” for llama3.1:8b. If the process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. The page says a model that does not fit spills to system RAM and CPU. This page does not pick a smaller tag for you.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The README and pyproject.toml do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this library on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

LLM_ENDPOINT returns 404. The LLM providers page says the default backend appends /api/generate to the bare host, so http://localhost:11434/v1 is the wrong LLM_ENDPOINT on that backend. The documented LLM value is http://localhost:11434. The embedding value stays http://localhost:11434/api/embed. The page also says Ollama binds to 127.0.0.1 by default, and it prints curl http://localhost:11434/api/tags as a check. This page did not run that curl.

Missing LLM variables. The same page says that when LLM_PROVIDER is ollama, setting any of LLM_MODEL, LLM_ENDPOINT, or LLM_API_KEY means all three must be set. The installation block above sets all three.

What to copy off, then destroy

The local guide prints results[0].text. The script writes under a temporary directory from tempfile.mkdtemp. Copy the printed text before you destroy the instance. If you change data_root_directory or system_root_directory to a path you choose, that path is on the instance disk. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The pricing page FAQ states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Prices are not copied here.

Frequently asked questions

What memory figure do the Cognee docs print?

The LLM providers page says an 8B model quantized to 4-bit needs roughly 6 GB of available VRAM, and it says llama3.1:8b is ~6 GB VRAM at 4-bit. Those are the page’s figures. They are not the catalog column vram_gb. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Which install line does the Ollama section print?

The installation page’s Ollama section prints uv pip install “cognee[ollama]”, then ollama pull llama3.1:8b and ollama pull nomic-embed-text:latest. The README on commit 4053f6cc5536757f26a3665eed95c50088e6dc0b also prints pip install cognee and says Python 3.10 through 3.14. pyproject.toml says version 1.6.8, license Apache-2.0, and requires-python 3.10 or newer and below 3.15. PyPI info.version on 2026-11-14 was 1.6.8, requires_python that same bound, uploaded 2026-11-12T22:37:53.918693Z. This page did not install the package.

How does Cognee point at a server on the GPU?

The local Ollama guide sets LLM_PROVIDER to ollama, LLM_MODEL to llama3.1:8b, and LLM_ENDPOINT to http://localhost:11434. It sets EMBEDDING_PROVIDER to ollama, EMBEDDING_MODEL to nomic-embed-text, EMBEDDING_ENDPOINT to http://localhost:11434/api/embed, and EMBEDDING_DIMENSIONS to 768. The installation page’s embedding tag is nomic-embed-text:latest. Those embedding tags are not the same string. remember and recall are the calls in the local guide. The custom section’s vLLM accordion uses the hosted_vllm/ prefix and sets LLM_ENDPOINT to https://your-vllm-endpoint/v1. Those pages do not print an SSH command. This page does not copy the hosted-provider sections.

What license does the file print?

The LICENSE file begins with Apache License Version 2.0, January 2004. The GitHub license field on 2026-11-14 was Apache-2.0. pyproject.toml on that commit says license Apache-2.0.

Sources (read 2026-11-14)

Cognee local Ollama guide, read 2026-11-14: https://docs.cognee.ai/guides/local-ollama . ollama pull llama3.1:8b, ollama pull nomic-embed-text. LLM_ENDPOINT http://localhost:11434. EMBEDDING_ENDPOINT http://localhost:11434/api/embed. EMBEDDING_DIMENSIONS 768. HUGGINGFACE_TOKENIZER nomic-ai/nomic-embed-text-v1.5. set_graph_database_provider("ladybug"), set_vector_db_provider("lancedb"). remember and recall with SearchType.GRAPH_COMPLETION. The banner helper is not copied into the block above
Cognee installation page, read 2026-11-14: https://docs.cognee.ai/getting-started/installation . Python 3.10 through 3.14. Ollama section: uv pip install "cognee[ollama]", ollama pull llama3.1:8b, ollama pull nomic-embed-text:latest, LLM_API_KEY ollama, EMBEDDING_MODEL nomic-embed-text:latest. The tab title’s marketing line is not copied
Cognee LLM providers page, read 2026-11-14: https://docs.cognee.ai/setup-configuration/llm-providers . The page says an 8B model quantized to 4-bit needs roughly 6 GB of available VRAM. “~6 GB VRAM at 4-bit” for llama3.1:8b. Recommended tags include llama3.1 and qwen2.5 at 14B or larger. Custom hosted_vllm/ prefix, LLM_ENDPOINT https://your-vllm-endpoint/v1, LLM_API_KEY .. The filled example’s tag is not copied. The hosted-provider sections are not copied
topoteretes/cognee commit 4053f6cc5536757f26a3665eed95c50088e6dc0b, committed 2026-11-13T00:23:56Z. README Python 3.10–3.14, pip install cognee, uv pip install "cognee[gliner]" for the quickstart that does not set LLM_PROVIDER. pyproject.toml version 1.6.8, license Apache-2.0, requires-python >=3.10,<3.15. GitHub license field Apache-2.0. pushed_at 2026-11-14T20:53:03Z. PyPI cognee 1.6.8, uploaded 2026-11-12T22:37:53.918693Z, requires_python >=3.10,<3.15
Hugging Face model API nomic-ai/nomic-embed-text-v1.5: gated false, cardData.license apache-2.0, pipeline_tag sentence-similarity, lastModified 2026-04-07T14:17:02.000Z, sha e9b6763023c676ca8431644204f50c2b100d9aab. The local guide sets this id as HUGGINGFACE_TOKENIZER and does not print hf download
Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (generated_at 2026-11-14T22:14:00.888Z). GET /v1/public/compute-capabilities returned feature flags and no GPU list. The catalog had no Cognee template

Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.

Last updated: 2026-11-14.