How to run Graphiti with a self-hosted model on a cloud GPU

Q: Which install line does the quick start print?

The quick start prints pip install graphiti-core and uv add graphiti-core, and it says Python 3.10 or higher. pyproject.toml on commit ed23196506eba8806fae6da831d73fcd2ad714d8 says version 0.31.2, license Apache-2.0, and requires-python 3.10 or newer and below 4. PyPI info.version on 2026-11-14 was 0.31.2, requires_python that same bound, uploaded 2026-10-20T17:25:54.496293Z. This page did not install the package.

Q: How does OpenAIGenericClient point at a server on the GPU?

The LLM configuration page's Ollama section uses OpenAIGenericClient, not OpenAIClient, and sets base_url to http://localhost:11434/v1. model and small_model are deepseek-r1:7b. The embedder sets embedding_model to nomic-embed-text, embedding_dim to 768, and base_url to the same URL. Graphiti is constructed with bolt://localhost:7687, user neo4j, and password password. build_indices_and_constraints, add_episode, and search run on that same instance. The quick start's Graphiti(neo4j_uri, neo4j_user, neo4j_password) omits the local clients. On commit ed23196506eba8806fae6da831d73fcd2ad714d8, the constructor then uses OpenAIClient, OpenAIEmbedder, and OpenAIRerankerClient. This page does not make that second assignment. The Neo4j page's docker run sets NEO4J_AUTH to neo4j/your_password and publishes ports 7474 and 7687. Those password strings are not the same. The page does not print an SSH command. This page does not copy the other provider sections.

Q: What license does the file print?

The LICENSE file begins with Apache License Version 2.0, January 2004. The quickstart file header on commit ed23196506eba8806fae6da831d73fcd2ad714d8 says Copyright 2025, Zep Software, Inc. The GitHub license field on 2026-11-14 was Apache-2.0. pyproject.toml says license Apache-2.0.

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Graphiti with the pip install graphiti-core line the quick start prints, start Neo4j with the Docker command the Neo4j page prints, serve a model on that machine, and set OpenAIGenericClient base_url to the local URL the LLM configuration page prints. Copy the printed facts before you destroy the instance, because destroying the instance deletes its disk.

An agent can rent the GPU and run this library against a model served on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Graphiti open-source install. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. An OpenAI-compatible engine whose CUDA 12 container tag is localai/localai:latest-gpu-nvidia-cuda-12 is How to run LocalAI on a cloud GPU. A gateway whose README prints uv tool install 'litellm[proxy]' is How to run LiteLLM Proxy in front of a self-hosted model on a cloud GPU. A memory layer whose Ollama page sets llm.provider to ollama and whose TypeScript example sets url to http://localhost:11434 is How to run Mem0 with a self-hosted model on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.

The commands below are copied from the help pages and from the README at commit ed23196506eba8806fae6da831d73fcd2ad714d8. They were not run on a CloudGPU instance for this page. The quick start constructs Graphiti with no llm_client. This page does not use that constructor. The LLM configuration page also documents other providers. This page does not copy those sections.

What the docs print about memory

The LLM configuration page, the Neo4j page, the quick start, and the README print no GPU GB figure for deepseek-r1:7b or nomic-embed-text. This page does not mark a catalog row as enough.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T22:14:00.888Z). The catalog response also includes an availability field. This page does not copy it. Every catalog row below has min_gpu_count 1. Which listing you can start is on Deploy.

The catalog template list had no Graphiti template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as Graphiti.

Catalog name	vram_gb
RTX 2080	8
RTX 3080	10
RTX 2080 Ti	11
RTX 3080 Ti	12
V100 16G	16
A30	24
RTX 3090	24
RTX 4090	24
RTX 4090D	24
RTX 4080 SUPER 32G	32
RTX 5090	32
RTX 5090D	32
V100 32G	32
A100 40G	40
A800 40G	40
A40	48
RTX 4090 48G	48
RTX 4090D 48G	48
RTX A6000	48
Ascend 910B	64
NVIDIA RTX PRO 5000 72G	72
RTX PRO 5000 72G	72
A100 80G	80
A800 80G	80
H800	80
RTX 6000D	83
H20 96G	96
RTX PRO 6000 96G	96
RTX PRO 6000 Max-Q	96
H20-3e 141G	141

Install

The quick start says Python 3.10 or higher. The install line, copied:

pip install graphiti-core

The same page also prints uv add graphiti-core. pyproject.toml on commit ed23196506eba8806fae6da831d73fcd2ad714d8 (committed 2026-11-14T09:12:37Z) says version = "0.31.2", license = "Apache-2.0", and requires-python = ">=3.10,<4". PyPI graphiti-core info.version on 2026-11-14 was 0.31.2, requires_python >=3.10,<4, uploaded 2026-10-20T17:25:54.496293Z. The GitHub license field on 2026-11-14 was Apache-2.0. pushed_at was 2026-11-14T09:12:40Z. This page did not install the package.

Start Neo4j on the same machine

The Neo4j page says version 5.26 or higher is required. Its Community Edition Docker command, copied:

docker run \
    --name neo4j-community \
    -p 7474:7474 -p 7687:7687 \
    -e NEO4J_AUTH=neo4j/your_password \
    -e NEO4J_PLUGINS='["apoc"]' \
    neo4j:5.26-community

The same page then prints:

export NEO4J_URI=bolt://localhost:7687
export NEO4J_USER=neo4j
export NEO4J_PASSWORD=your_password

your_password is the string in that command. It is not a password this page adds. The page also prints a Neo4j Desktop path. This page does not repeat those steps. The page’s AuraDB section is a hosted database. This page does not copy it. CloudGPU does not provide Neo4j.

The README at that commit also prints docker compose up and says it starts the Neo4j Docker service and related components. docker-compose.yml on that commit sets the neo4j service image to neo4j:5.26.2, publishes 127.0.0.1:7474:7474 and 127.0.0.1:${NEO4J_PORT:-7687}:${NEO4J_PORT:-7687}, and sets NEO4J_AUTH to ${NEO4J_USER:-neo4j}/${NEO4J_PASSWORD:-password}. The same file’s graph service sets OPENAI_API_KEY=${OPENAI_API_KEY}. That service is not the model client on this page. The image tag neo4j:5.26.2 is not neo4j:5.26-community. The default password string password is not your_password. This page did not run either command.

Point OpenAIGenericClient at the GPU server

The LLM configuration page’s Ollama section says to use OpenAIGenericClient, not OpenAIClient, because Ollama does not support the /v1/responses endpoint that OpenAIClient uses. OpenAIGenericClient uses /v1/chat/completions. The pull lines and the config, copied:

ollama pull deepseek-r1:7b     # LLM
ollama pull nomic-embed-text   # embeddings

from graphiti_core import Graphiti
from graphiti_core.llm_client.config import LLMConfig
from graphiti_core.llm_client.openai_generic_client import OpenAIGenericClient
from graphiti_core.embedder.openai import OpenAIEmbedder, OpenAIEmbedderConfig
from graphiti_core.cross_encoder.openai_reranker_client import OpenAIRerankerClient

llm_config = LLMConfig(
    api_key="ollama",  # Ollama doesn't require a real API key
    model="deepseek-r1:7b",
    small_model="deepseek-r1:7b",
    base_url="http://localhost:11434/v1",
)

llm_client = OpenAIGenericClient(config=llm_config)

graphiti = Graphiti(
    "bolt://localhost:7687",
    "neo4j",
    "password",
    llm_client=llm_client,
    embedder=OpenAIEmbedder(
        config=OpenAIEmbedderConfig(
            api_key="ollama",
            embedding_model="nomic-embed-text",
            embedding_dim=768,
            base_url="http://localhost:11434/v1",
        )
    ),
    cross_encoder=OpenAIRerankerClient(client=llm_client.client, config=llm_config),
)

The constructor’s third argument is the string password. The Neo4j page’s Docker auth value is neo4j/your_password. Those strings are not the same. The quick start’s default is os.environ.get('NEO4J_PASSWORD', 'password'). This page does not change any of those strings.

The same page says to run ollama serve and to pull the models you use. embedding_dim is 768 in that block. It is not a GPU GB figure. The Ollama section warns that smaller local models may not extract data or return the JSON structures Graphiti requires. How to run Ollama on a cloud GPU is the Ollama install. That server stays on the rented GPU.

The same page’s “OpenAI Compatible Services” section sets base_url on LLMConfig to a placeholder and says to replace it with the service URL. The only concrete local URL that page prints is http://localhost:11434/v1. This page does not copy that section’s example model ids. A vLLM server’s own /v1 URL is the same base_url field. The Graphiti page does not print port 8000. How to serve an OpenAI-compatible API with vLLM on a cloud GPU is the vLLM install.

The README at that commit passes client=llm_client to OpenAIRerankerClient. The help page passes client=llm_client.client. Those arguments are not the same. This page copies the help page. The README also sets structured_output_mode values json_schema (the default in that section) and json_object. The help Ollama section does not print structured_output_mode. This page did not run Graphiti.

The quick start says SEMAPHORE_LIMIT defaults to 20. The README at that commit says SEMAPHORE_LIMIT defaults to 10. Those numbers are not the same. This page does not pick one.

The Graphiti files this page read do not print hf download. The models in the Ollama section are Ollama tags, not Hugging Face repository ids.

Add an episode and search

The quick start says build_indices_and_constraints is required before other calls. It assigns graphiti = Graphiti(neo4j_uri, neo4j_user, neo4j_password) and then calls that method. On commit ed23196506eba8806fae6da831d73fcd2ad714d8, Graphiti.__init__ sets llm_client to OpenAIClient(), embedder to OpenAIEmbedder(), and cross_encoder to OpenAIRerankerClient() when those arguments are omitted. That assignment replaces the Ollama section’s clients. This page calls the method on the graphiti the Ollama section already constructed:

await graphiti.build_indices_and_constraints()

examples/quickstart/quickstart_neo4j.py on that commit does not contain build_indices_and_constraints. The help quick start does, on the constructor that omits the local clients. This page did not run the call.

The quick start’s add loop, copied from the first text episode. The page’s list has more episodes. This block keeps the first one and the same add_episode arguments:

episodes = [
    {
        'content': 'Kamala Harris is the Attorney General of California. She was previously '
        'the district attorney for San Francisco.',
        'type': EpisodeType.text,
        'description': 'podcast transcript',
    },
]

for i, episode in enumerate(episodes):
    await graphiti.add_episode(
        name=f'Freakonomics Radio {i}',
        episode_body=episode['content']
        if isinstance(episode['content'], str)
        else json.dumps(episode['content']),
        source=episode['type'],
        source_description=episode['description'],
        reference_time=datetime.now(timezone.utc),
    )

The search call, copied:

results = await graphiti.search('Who was the California Attorney General?')

The quick start also prints graphiti.search_. The file at that commit prints graphiti._search for the node-search call. Those names are not the same. This page copies search from the help page and did not run it.

For a server you start on the instance, the SSH tunnel is How to use SSH port forwarding on a cloud GPU. The Graphiti pages do not print an SSH command. The Ollama URL is port 11434. The Neo4j bolt URL is port 7687. The Docker command also publishes port 7474. Those ports are not the same.

When it fails

CUDA out of memory. The Graphiti pages print no GPU GB figure. If the process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a smaller model id for you. The Ollama section already warns that smaller local models may fail JSON extraction.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The README and pyproject.toml do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this library on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

Neo4j auth does not match. The Docker command sets NEO4J_AUTH=neo4j/your_password. The Ollama section passes "password" as the third argument to Graphiti. The quick start default for NEO4J_PASSWORD is password. This page did not test which string a running server accepts.

What to copy off, then destroy

The quick start prints result.fact for each search result, then calls await graphiti.close(). Copy the printed facts before you destroy the instance. Neo4j data stays in the container you started. The compose file names a volume neo4j_data. The docker run command above does not name a volume. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The pricing page FAQ states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Prices are not copied here.

Frequently asked questions

What memory figure do the Graphiti docs print?

The LLM configuration page, the Neo4j page, the quick start, and the README print no GPU GB figure for deepseek-r1:7b or nomic-embed-text. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Which install line does the quick start print?

The quick start prints pip install graphiti-core and uv add graphiti-core, and it says Python 3.10 or higher. pyproject.toml on commit ed23196506eba8806fae6da831d73fcd2ad714d8 says version 0.31.2, license Apache-2.0, and requires-python 3.10 or newer and below 4. PyPI info.version on 2026-11-14 was 0.31.2, requires_python that same bound, uploaded 2026-10-20T17:25:54.496293Z. This page did not install the package.

How does OpenAIGenericClient point at a server on the GPU?

The LLM configuration page’s Ollama section uses OpenAIGenericClient, not OpenAIClient, and sets base_url to http://localhost:11434/v1. model and small_model are deepseek-r1:7b. The embedder sets embedding_model to nomic-embed-text, embedding_dim to 768, and base_url to the same URL. Graphiti is constructed with bolt://localhost:7687, user neo4j, and password password. build_indices_and_constraints, add_episode, and search run on that same instance. The quick start’s Graphiti(neo4j_uri, neo4j_user, neo4j_password) omits the local clients. On commit ed23196506eba8806fae6da831d73fcd2ad714d8, the constructor then uses OpenAIClient, OpenAIEmbedder, and OpenAIRerankerClient. This page does not make that second assignment. The Neo4j page’s docker run sets NEO4J_AUTH to neo4j/your_password and publishes ports 7474 and 7687. Those password strings are not the same. The page does not print an SSH command. This page does not copy the other provider sections.

What license does the file print?

The LICENSE file begins with Apache License Version 2.0, January 2004. The quickstart file header on commit ed23196506eba8806fae6da831d73fcd2ad714d8 says Copyright 2025, Zep Software, Inc. The GitHub license field on 2026-11-14 was Apache-2.0. pyproject.toml says license Apache-2.0.

Sources (read 2026-11-14)

Graphiti LLM configuration, read 2026-11-14: https://help.getzep.com/v4/graphiti/configuration/llm-configuration . Ollama section: ollama pull deepseek-r1:7b, ollama pull nomic-embed-text, OpenAIGenericClient, LLMConfig base_url http://localhost:11434/v1, OpenAIEmbedderConfig embedding_model nomic-embed-text, embedding_dim 768, OpenAIRerankerClient(client=llm_client.client). Note says OpenAIClient uses /v1/responses and OpenAIGenericClient uses /v1/chat/completions. The other provider sections are not copied
Graphiti Neo4j configuration, read 2026-11-14: https://help.getzep.com/v4/graphiti/configuration/neo4j-configuration . Version 5.26 or higher. docker run image neo4j:5.26-community, ports 7474 and 7687, NEO4J_AUTH=neo4j/your_password, NEO4J_PLUGINS='["apoc"]'. Env NEO4J_URI=bolt://localhost:7687. The AuraDB section is not copied
Graphiti quick start, read 2026-11-14: https://help.getzep.com/v4/graphiti/getting-started/quick-start . pip install graphiti-core, uv add graphiti-core, Python 3.10 or higher. SEMAPHORE_LIMIT default 20. build_indices_and_constraints, add_episode, search. NEO4J_PASSWORD default password. The default Graphiti(neo4j_uri, neo4j_user, neo4j_password) constructor is not the Ollama constructor
getzep/graphiti commit ed23196506eba8806fae6da831d73fcd2ad714d8, committed 2026-11-14T09:12:37Z. README pip install graphiti-core, docker compose up, local section OpenAIRerankerClient(client=llm_client), structured_output_mode json_schema and json_object, SEMAPHORE_LIMIT default 10. docker-compose.yml image neo4j:5.26.2, auth default neo4j/password, ports 7474 and 7687 on 127.0.0.1. examples/quickstart/quickstart_neo4j.py does not contain build_indices_and_constraints and prints graphiti._search. Graphiti.__init__ uses OpenAIClient(), OpenAIEmbedder(), and OpenAIRerankerClient() when llm_client, embedder, and cross_encoder are omitted. This page calls build_indices_and_constraints on the Ollama section’s instance. pyproject.toml version 0.31.2, license Apache-2.0, requires-python >=3.10,<4. GitHub license field Apache-2.0. pushed_at 2026-11-14T09:12:40Z. PyPI graphiti-core 0.31.2, uploaded 2026-10-20T17:25:54.496293Z
Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (generated_at 2026-11-14T22:14:00.888Z). GET /v1/public/compute-capabilities returned feature flags and no GPU list. The catalog had no Graphiti template

Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.

Last updated: 2026-11-14.