Self-Host an LLM on a Rented GPU or Pay per Token? A Break-Even Calculator (2026)

Self-hosting on a rented GPU is cheaper than a per-token API only when you keep the GPU busy: the break-even is the hourly GPU price divided by the API’s price per token, expressed as tokens you must actually generate and process every hour. Below that throughput the API wins; above it the GPU wins. This post gives the formula, a worked example with an open 32B model on a single 32 GB card, and a script to run with your own measurements.

The formula

api_cost_per_hour  = (input_tokens_per_hour × input_price + output_tokens_per_hour × output_price) / 1,000,000
gpu_cost_per_hour  = hourly_gpu_price                     # you pay it whether the GPU is busy or idle

break_even_tokens_per_hour = hourly_gpu_price / blended_price_per_token
blended_price_per_token    = (r × input_price + output_price) / ((r + 1) × 1,000,000)    # r = input:output ratio

Then correct for utilisation: if your GPU is busy only 40% of the hours you rent it, the throughput you need while busy is break-even ÷ 0.4.

Worked example: Qwen3 32B (4-bit) on an RTX 5090 32G

Qwen3 32B is an open-weight model (Apache-2.0) that is not offered on the CloudGPU API, so self-hosting is the way to run it here. We use the 4-bit AWQ build because it fits a 32 GB card.

Model size (estimate). Hugging Face metadata for Qwen/Qwen3-32B lists 32,762,123,264 parameters. In Qwen/Qwen3-32B-AWQ, 31.21B of them are stored at 4 bits and 1.56B stay in BF16:

31.21B × 0.5 byte           ≈ 15.6 GB   (4-bit weights)
 1.56B × 2 bytes            ≈  3.1 GB   (BF16 embeddings, norms, output layer)
AWQ scales/zeros, group 128 ≈  0.6 GB
total                       ≈ 19.3 GB   (the repository's files add up to 19.33 GB)

The FP8 build (Qwen/Qwen3-32B-FP8, about 34.3 GB of files) does not fit in 32 GB of VRAM; it needs a 48 GB card such as the RTX 4090 48G. Figures read from the Hugging Face API on 2026-10-23 21:12 Beijing time; see also Qwen3 32B VRAM.

Context headroom (estimate). With an FP16 KV cache, Qwen3 32B stores 2 × 64 layers × 8 KV heads × 128 × 2 bytes = 262,144 bytes (about 0.26 MB) per token: about 2.1 GB at 8k tokens and 8.6 GB at 32k. After about 19.3 GB of weights, a 32 GB card has roughly 12.7 GB for the KV cache and runtime overhead, so at most about 48k cached tokens shared by all concurrent requests, fewer once the overhead is counted.

Inputs (2026-10-23 21:13 Beijing time):

Item	Value	Source
RTX 5090 32G list price	$0.59 per hour, billed per minute	/gpu-prices.json
Per-token reference prices on the same balance	glm-5.3-flash $0.16 in / $0.55 out; gpt-oss-120b $0.135 / $0.54; deepseek-v4-flash $0.27 / $1.08 (per 1M tokens)	/api

What a self-hosted token costs. At a fixed $0.59 per hour, the cost per million tokens depends only on the throughput you sustain. The throughputs below are inputs to the arithmetic, not measurements:

Sustained throughput (all requests, input + output)	Tokens per hour	Cost per 1M tokens
100 tokens/s	0.36M	≈ $1.64
250 tokens/s	0.90M	≈ $0.66
500 tokens/s	1.80M	≈ $0.33
1,000 tokens/s	3.60M	≈ $0.16

Break-even against per-token prices (3 input : 1 output):

Per-token reference	Blended price	Break-even per hour	Throughput needed at 100% / 50% utilisation
glm-5.3-flash	$0.2575 / 1M	≈ 2.29M tokens/h	≈ 636 / 1,273 tokens/s
gpt-oss-120b	$0.236 / 1M	≈ 2.50M tokens/h	≈ 694 / 1,387 tokens/s
deepseek-v4-flash	$0.4725 / 1M	≈ 1.25M tokens/h	≈ 347 / 694 tokens/s

These are different models from Qwen3 32B, so the table compares cost only: decide on quality first, then use the arithmetic. How to read it: on one card you would need to sustain several hundred tokens per second, aggregated across concurrent requests, for every hour you pay for. That needs batching and steady traffic, and you should verify it with a benchmark on the card. For spiky or low-volume traffic, a per-token API is usually the lower-cost choice.

We do not publish a measured tokens-per-second figure for this card here; measure your own with your prompt lengths and concurrency.

When renting the GPU wins anyway

Price per token is not the only reason to rent:

Situation	Why a GPU makes sense
The model is not offered as an API (e.g. Qwen3 32B on CloudGPU, a community fine-tune, your own LoRA)	There is no per-token option to compare against
Large batch jobs with steady load (offline eval, synthetic data, embeddings, transcription batches)	Utilisation is high and the batch finishes; destroy the instance afterwards
You need custom serving: LoRA adapters, constrained decoding, specific quantisation	You control the stack
You need image pipelines with custom nodes (ComfyUI)	APIs expose fixed models only

When the API wins

Situation	Why
Traffic is spiky or low	You would pay for idle GPU hours
The model is too large for one rented card (e.g. DeepSeek V4, Kimi K3)	Self-hosting would need far more VRAM than one card has
You need the model now, without ops work	No boot, no downloads, no monitoring
You need a closed frontier model	Only available as an API

For that last case, the same CloudGPU balance pays for the API: Claude models are billed at about 50% of the official public pricing, and other models are priced against each vendor’s public list price.

What an always-on GPU costs

If you keep a card running 24/7 to serve traffic, the bill is fixed regardless of tokens (list prices, 2026-10-23 21:13 Beijing time):

Card	Per day (24 h)	Per 30 days
RTX 4090 24G ($0.33/h)	$7.92	$237.60
RTX 5090 32G ($0.59/h)	$14.16	$424.80
RTX 4090 48G ($0.65/h)	$15.60	$468.00
A800 80G ($1.09/h)	$26.16	$784.80
A100 80G ($1.49/h)	$35.76	$1,072.80

“Fixed cost” does not mean unlimited output: the card’s throughput caps how many tokens you can serve. Also budget for restarts — destroying an instance deletes its disk, so a redeploy re-downloads weights. Stock changes during the day; check Deploy.

A script for your own numbers

def break_even(gpu_price_h, in_price_m, out_price_m, in_out_ratio=3.0, utilisation=1.0):
    blended = (in_out_ratio * in_price_m + out_price_m) / (in_out_ratio + 1) / 1e6   # $ per token
    tokens_per_hour = gpu_price_h / blended
    return tokens_per_hour, tokens_per_hour / 3600 / utilisation                       # tokens/h, tokens/s while busy

def monthly(gpu_price_h, hours_per_day=24, days=30):
    return gpu_price_h * hours_per_day * days

def cost_per_million(gpu_price_h, tokens_per_second):
    return gpu_price_h / (tokens_per_second * 3600 / 1e6)                              # $ per 1M tokens

# Qwen3 32B AWQ on an RTX 5090 32G at $0.59/h vs a $0.16 in / $0.55 out per-token reference (2026-10-23 21:13)
tph, tps = break_even(0.59, 0.16, 0.55, in_out_ratio=3, utilisation=0.5)
print(f"break-even: {tph/1e6:.2f}M tokens/h, {tps:.0f} tokens/s while busy")
print(f"self-hosted cost at 500 tokens/s: ${cost_per_million(0.59, 500):.2f} per 1M tokens")
print(f"always-on RTX 5090 32G for 30 days: ${monthly(0.59):.2f}")

Replace the prices with the live values from /pricing and /api on the day you decide, and replace utilisation and the input:output ratio with your logs.

FAQ

Is self-hosting an LLM cheaper than using an API? Only if you keep the GPU busy enough. Divide the GPU’s hourly price by the API’s blended price per token; if your measured sustained throughput is above that number, self-hosting is cheaper.

How do I calculate the break-even between GPU rental and API pricing? break-even tokens per hour = hourly GPU price ÷ blended API price per token, where the blended price weights input and output by your traffic ratio. Divide by utilisation to get the throughput you need while busy.

What utilisation do I need for self-hosting to pay off? It depends on the model and card; compute break-even from the formula, then compare with the throughput you measure under realistic concurrency.

Why rent a GPU if the API is cheaper per token? For models with no API, fine-tunes and LoRA adapters, custom serving stacks, ComfyUI pipelines, or steady batch jobs that keep the card busy.

Can I use the same balance for GPUs and the API? Yes. One prepaid balance covers per-minute GPU rental and per-token API calls.

CloudGPU (cloudgpu.app) is an independent service. It has no partnership with, authorization from, or endorsement by the model vendor. Model names are trademarks of their owners. Break-even figures are arithmetic on list prices, not performance measurements. Prices were read on 2026-10-23 21:13 Beijing time (UTC+8); model sizes are estimates from Hugging Face metadata read at 21:12; API prices are compared against official public pricing; live prices and stock on /deploy and /api take precedence.