Why is the reply empty when I call a thinking model through an OpenAI-compatible API?

An empty reply with finish_reason: "length" usually means the model spent the whole max_tokens budget on its reasoning before it started the visible answer, so raise max_tokens (or leave it unset) and check the reasoning token count in usage.

Updated 2026-11-02

Thinking models such as GLM write their reasoning first and the answer second. On the CloudGPU model API (base URL https://cloudgpu.app/v1) the reasoning comes back in a separate reasoning_content field, and its tokens are counted inside completion_tokens. In a recorded CloudGPU response for glm-5.2 (2026-10-30, prompt “Reply with exactly: ok”, max_tokens 800), usage.completion_tokens was 18 and usage.completion_tokens_details.reasoning_tokens was 15. A tool that reads only content and sets a small max_tokens can therefore show nothing at all, with no error.

This is not specific to one provider. The same symptom is reported in the goose issue tracker (goose #11142, read 2026-11-02), in vLLM for DeepSeek V4 (vLLM #52640, read 2026-11-02) and in LM Studio (LM Studio bug tracker #1990, read 2026-11-02).

Step 1: Confirm it is the reasoning budget

Set two environment variables, CLOUDGPU_API_KEY (your key from the API console) and CLOUDGPU_BASE_URL (https://cloudgpu.app/v1), then look at four fields in the raw response, not at what your tool displays.

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["CLOUDGPU_BASE_URL"],  # https://cloudgpu.app/v1
    api_key=os.environ["CLOUDGPU_API_KEY"],
)

resp = client.chat.completions.create(
    model="glm-5.2",  # use the exact ID from cloudgpu.app/api
    messages=[{"role": "user", "content": "Summarise what a KV cache is in two sentences."}],
    max_tokens=200,
)

choice = resp.choices[0]
msg = choice.message
details = resp.usage.completion_tokens_details
print("finish_reason:", choice.finish_reason)
print("content:", repr(msg.content))
print("reasoning_content present:", bool(getattr(msg, "reasoning_content", None)))
print("completion_tokens:", resp.usage.completion_tokens)
print("reasoning_tokens:", details.reasoning_tokens if details else None)

It is the reasoning budget when finish_reason is length, content is empty or almost empty, and reasoning_tokens is close to completion_tokens. If finish_reason is stop and content is still empty, it is a different problem, so check the request against the API docs.

Step 2: Give the answer room

Raise max_tokens, or leave it out and let the model stop on its own. Thinking models need room for reasoning plus the answer, and how much reasoning they write depends on the model and the prompt. Re-run Step 1 and check that finish_reason is now stop.

Step 3: Ask for less reasoning (if the model honours it)

Some models accept a lower reasoning effort:

resp = client.chat.completions.create(
    model="glm-5.2",
    messages=[{"role": "user", "content": "Summarise what a KV cache is in two sentences."}],
    max_tokens=200,
    reasoning_effort="low",
)

Reader check: our API docs do not list reasoning_effort, so check whether it takes effect for the model you use. Send the same prompt with and without the parameter and compare reasoning_tokens. If the number doesn’t drop, the parameter isn’t taking effect for that model, so use Step 2.

Step 4: Fix it in your coding agent

In Cline, Roo Code, Aider or goose, the setting that matters is the tool’s maximum output tokens for the custom OpenAI-compatible model. According to goose #11142 (read 2026-11-02), goose sends the configured max_tokens unchanged to OpenAI-compatible endpoints even when reasoning is enabled, so set that value high enough yourself. Model IDs and current rates are on the model API page, and request parameters are on the API docs.

Step 5: Or pick a model that answered without reasoning

For the same short prompt on 2026-10-30, the recorded kimi-k3 response showed reasoning_tokens 0. That is one prompt, not a rule.

Reader check: before you switch models, run Step 1 with your own prompts and the max_tokens value your tool sends, and confirm that finish_reason is stop and content is not empty.

FAQ

Why is content empty but reasoning_content full? The model used the whole max_tokens budget while reasoning, so the response ended with finish_reason: "length" before the visible answer began. Raise max_tokens or leave it unset.

Do reasoning tokens count toward max_tokens on CloudGPU? In recorded CloudGPU responses, reasoning tokens are counted inside usage.completion_tokens and listed separately under completion_tokens_details.reasoning_tokens, so they share the output budget with the answer. Check usage in your own responses to see the split.

How do I check whether reasoning_effort works for a model? Send the same prompt with and without reasoning_effort="low" and compare usage.completion_tokens_details.reasoning_tokens. If the number doesn’t drop, raise max_tokens instead.

Sources

All read on 2026-11-02.