How do I use Kimi K3 in LangChain?

Install langchain-openai, then create ChatOpenAI(model=“kimi-k3”, base_url=“https://cloudgpu.app/v1”, api_key=$CLOUDGPU_API_KEY). Kimi K3 always thinks, so set a large max_tokens.

Updated 2026-10-28

LangChain’s own docs say ChatOpenAI can connect to OpenAI-compatible endpoints “with a custom base_url” for basic chat (LangChain chat integrations, read 2026-10-28). The CloudGPU API is one such endpoint, https://cloudgpu.app/v1, and lists kimi-k3 in its model table (/api). This page shows the setup, the two Kimi K3 details that change your code, and what to check on your own machine.

Key facts

Item	Value	Source (read 2026-10-28)
Model ID	kimi-k3	cloudgpu.app/api model table
Base URL	https://cloudgpu.app/v1	cloudgpu.app/api/docs
Kimi K3 size	2.8T total parameters, 104B activated (Mixture-of-Experts)	Kimi-K3 model card
Kimi K3 context length	1,048,576 tokens (model card); the limit on our gateway is not tested yet	Kimi-K3 model card
Thinking	always on (“always has thinking enabled”)	Kimi-K3 model card, section 6
LangChain package	langchain-openai 1.7.3 (released 2026-10-27), Python 3.10 or newer	PyPI langchain-openai
reasoning_content in LangChain’s ChatOpenAI	ChatOpenAI behaviour: non-standard response fields “are not extracted or preserved”	LangChain ChatOpenAI docs, “API scope”
Gateway rate limits	100 requests per minute per API key; 10,000 requests per day per account	cloudgpu.app/api/docs
Our end-to-end LangChain run	not tested by us yet	TR-GEO-1028-01

Step 1: Check the key and the model outside LangChain

Create a key in the API console (/api); it starts with cgw-sk-. Test it with curl first, so a key or model problem is not mistaken for a LangChain problem. The request shape is the one on /api/docs.

export CLOUDGPU_API_KEY="cgw-sk-..."
export CLOUDGPU_BASE_URL="https://cloudgpu.app/v1"
curl "$CLOUDGPU_BASE_URL/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $CLOUDGPU_API_KEY" \
  -d '{"model":"kimi-k3","max_tokens":4096,"messages":[{"role":"user","content":"Say hi in one sentence."}]}'

Check: the reply has an answer in choices[0].message.content. A 401 means the key is wrong, and a 400 model_not_found means the model ID is misspelled (error codes from /api/docs).

Step 2: Install the LangChain package

python3 --version          # PyPI lists Python 3.10 or newer for langchain-openai 1.7.3
pip install -U langchain-openai
pip show langchain-openai  # note the version you got

Step 3: Create the model with ChatOpenAI

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="kimi-k3",
    base_url=os.environ.get("CLOUDGPU_BASE_URL", "https://cloudgpu.app/v1"),
    api_key=os.environ["CLOUDGPU_API_KEY"],
    max_tokens=8192,  # room for thinking plus the answer
)

print(llm.invoke("Explain what a vector database is in two sentences.").content)

Why each line matters:

• base_url: this is LangChain ChatOpenAI behaviour: its docs give a base-URL resolution order where an explicit base_url argument is the first match and wins over base-URL environment variables (LangChain ChatOpenAI docs). So the request goes to https://cloudgpu.app/v1 even if another tool set one.
• max_tokens: the Kimi K3 model card says the model “always has thinking enabled, and will return reasoning_content”. In our 2026-10-25 test of another Kimi thinking model on the gateway, a 200-token limit was used up by thinking, content came back empty and finish_reason was length. Give Kimi K3 a large limit. In LangChain’s ChatOpenAI, max_tokens “is automatically converted to max_completion_tokens internally” (LangChain ChatOpenAI docs, “Token parameter deprecation”); check the token counts in a reply to see the limit at work.

Step 4: Use it in a chain, a stream or a batch

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a concise technical writer."),
    ("human", "Summarize this in three bullet points:\n\n{text}"),
])
chain = prompt | llm | StrOutputParser()
print(chain.invoke({"text": "Paste a paragraph here."}))

for chunk in llm.stream("Name three uses of embeddings."):
    print(chunk.content, end="", flush=True)

answers = llm.batch(["Define RAG in one line.", "Define a token in one line."])
print([a.content for a in answers])

Each call above is a single request, so nothing from an earlier answer has to be sent back.

Step 5: Know the limit for multi-turn chats and agents

Two official statements meet here:

1. The Kimi K3 model card: for multi-turn conversations and tool calls, Kimi K3 “requires the complete assistant message returned by the API to be passed back to messages as-is — including reasoning_content and tool_calls, not just content”.
2. LangChain ChatOpenAI behaviour, per the LangChain ChatOpenAI docs (“API scope”): non-standard response fields from third-party providers, “e.g., reasoning_content”, “are not extracted or preserved”.

So a LangChain chat history or a tool-calling agent built on ChatOpenAI sends back earlier Kimi K3 turns without their reasoning_content. For those flows, either:

• call the gateway with the openai package and append the full assistant message to messages, as in the model card’s example (base URL https://cloudgpu.app/v1, key from $CLOUDGPU_API_KEY); or
• test a provider-specific LangChain package first, which is what the LangChain ChatOpenAI docs suggest when you need non-standard fields. We have not tested any such package with our base URL.

Check on your machine

• pip show langchain-openai prints a version, and python3 --version is 3.10 or newer.
• llm.invoke(...) returns text in .content. If it is empty, raise max_tokens.
• response.usage_metadata on the reply shows token counts, so you can see how much of the limit thinking used.
• A 429 rate_limit_exceeded error means the per-minute or per-day limit on /api/docs was hit.

Usage is billed per token from your prepaid balance; current rates are on /pricing and /api.

FAQ

Which model ID do I use for Kimi K3 in LangChain? Use kimi-k3, the ID listed in the model table on cloudgpu.app/api (read 2026-10-28). Pass it as model=“kimi-k3” to ChatOpenAI together with base_url=“https://cloudgpu.app/v1” and your API key.

Why is the answer empty or cut off? Kimi K3 always thinks before it answers, per its model card. If max_tokens is small, the thinking can use up the limit before any answer text appears. Set a large max_tokens.

Does LangChain keep Kimi K3’s reasoning_content? No. This is ChatOpenAI behaviour: the LangChain ChatOpenAI docs say non-standard response fields such as reasoning_content are not extracted or preserved. Kimi K3’s model card asks for the full assistant message, including reasoning_content, to be passed back in multi-turn chats and tool calls.

Can I build a multi-turn agent with Kimi K3 in LangChain? Single requests, prompt chains, streaming and batch calls work with ChatOpenAI. For multi-turn chats and tool-calling agents, the Kimi K3 model card needs reasoning_content passed back, which ChatOpenAI drops. Use the openai package directly for those turns, or test a provider-specific LangChain package first.

What rate limits apply to the API key? Per cloudgpu.app/api/docs (read 2026-10-28), the defaults are 100 requests per minute per API key and 10,000 requests per day per account. A 429 rate_limit_exceeded error means one of these limits was hit.

Sources (read 2026-10-28 07:32 Asia/Shanghai, UTC+8)

• Kimi-K3 model card, Moonshot AI: https://huggingface.co/moonshotai/Kimi-K3
• LangChain, ChatOpenAI integration docs (“API scope”, base-URL resolution order, “Token parameter deprecation”): https://docs.langchain.com/oss/python/integrations/chat/openai (re-checked 2026-10-28 07:41 UTC+8, HTTP 200)
• LangChain, chat integrations (Chat Completions API section): https://docs.langchain.com/oss/python/integrations/chat
• PyPI, langchain-openai 1.7.3 metadata: https://pypi.org/project/langchain-openai/
• cloudgpu.app/api and /api/docs (model ID, base URL, rate limits, error codes, env var name), re-checked 2026-10-28 07:40 UTC+8