How do I run Unlimited-OCR-2 on a cloud GPU?

Deploy a Raw GPU, check nvidia-smi, download the customized SGLang wheel that ships in the baidu/Unlimited-OCR-2 repository, install it in a Python 3.10 virtual environment with uv, and start python -m sglang.launch_server --model-path baidu/Unlimited-OCR-2 with the flags from the model card; for a quick single-page test, load the model with Transformers and trust_remote_code=True instead.

Updated 2026-10-27

This guide follows the Unlimited-OCR-2 model card and the baidu/Unlimited-OCR GitHub repository, the uv and Hugging Face CLI docs, and vLLM’s public pull requests, all read on 2026-10-27, plus CloudGPU’s /deploy, /pricing and /terms pages, read the same day. We have not run it on our instances yet, so it shows commands, not measured output.

What is Unlimited-OCR-2?

According to its model card, Unlimited-OCR-2 (Unlimited OCR Works 2) is a document-parsing vision-language model from Baidu, released on 2026-10-27. It takes page images and returns the parsed text, and the card describes a sparse attention scheme (SR-SWA) aimed at very long, multi-page documents. The card lists the license as MIT (see the model card).

The model card names two ways to run it: Transformers (“native, for quick tests”) and SGLang (“recommended for serving”). This guide covers both, SGLang first.

What you need

  • On CloudGPU: an instance from /deploy using the Raw GPU option, which our deploy page describes as “a raw GPU with JupyterLab + CUDA pre-installed. SSH access, root on a dedicated container”.
  • On your computer: an SSH client such as ssh in a terminal.
  • Your documents as page images (JPG/PNG) or PDFs.

VRAM need depends on model, precision and workflow, and we have not measured this model on our cards; check the current card list on /pricing and see How much VRAM do I need to run an LLM? for the general picture.

GPU time is billed per minute, rounded up, with a 1-minute minimum, and billing stops when you destroy the instance (see /terms).

Step 1: Check the environment

Open a terminal on the instance (SSH or the JupyterLab terminal) and run:

nvidia-smi                 # GPU name, driver version, "CUDA Version" the driver supports
nvcc --version || echo "nvcc not on PATH"
python3 --version
cat /etc/os-release
df -h /dev/shm             # shared memory size inside the container

What to look for:

  • nvidia-smi must list your GPU. If it fails, stop here and redeploy.
  • nvcc is optional for the steps below: the SGLang wheel and PyTorch pull their own CUDA runtime libraries from pip. A missing nvcc is not an error by itself.
  • The SGLang command in Step 4 passes large image tensors through /dev/shm (the model card’s SGLANG_UOCR_SHM_WRAP_MODEL_DATA=1). Note the size df shows; if the server later fails with shared-memory errors, see Troubleshooting.

Step 2: Install uv

The model card’s SGLang setup uses a uv-managed virtual environment. Install uv with the command from the uv installation docs. The Hugging Face CLI guide shows running the hf command through uvx without a separate install:

curl -LsSf https://astral.sh/uv/install.sh | sh
exec $SHELL -l                      # reload the shell so uv and uvx are on PATH
uvx hf version

Step 3: Download the SGLang wheel and create the environment

The model card installs a customized SGLang build, sglang-0.5.20.dev10+g8d62f5fb9c-py3-none-any.whl, stored in the model repository’s wheel/ folder. The regular SGLang package from PyPI is a different build. Download only that file, pinned to the commit this guide was written against:

mkdir -p ~/uocr2 && cd ~/uocr2
uvx hf download baidu/Unlimited-OCR-2 \
  wheel/sglang-0.5.20.dev10+g8d62f5fb9c-py3-none-any.whl \
  --revision 9cf077c4b206e0e2a31879df937b0b0bcba3661c \
  --local-dir .

Then follow the model card’s environment steps (Python 3.10):

uv venv --python 3.10
source .venv/bin/activate

uv pip install wheel/sglang-0.5.20.dev10+g8d62f5fb9c-py3-none-any.whl
uv pip install pymupdf==1.27.2.2
uv pip install fastokens==0.3.2

uv downloads Python 3.10 itself if the image doesn’t have it. pymupdf is used later to turn PDF pages into images.

Step 4: Start the SGLang server

This is the model card’s launch command with two changes from us: --host 127.0.0.1 instead of 0.0.0.0, so the server only listens inside the instance and you reach it through an SSH tunnel, and --revision to load the same commit as the wheel. The model card uses --host 0.0.0.0; we bind to localhost so the server isn’t exposed to the internet without auth. Everything else is copied as published, including the spelling of SGLANG_UOCR_EANABLE_DOC_LIMIT.

# Page-window scheduling for very long documents (the KV tail of the previous window is reused).
export SGLANG_UOCR_EANABLE_DOC_LIMIT=1          # 1 = enable, 0 = whole document in one window
export SGLANG_UOCR_DOC_GUNDAM_PAGE_WINDOW=10    # pages per window (gundam mode)
export SGLANG_UOCR_DOC_BASE_PAGE_WINDOW=10      # pages per window (base mode)
export SGLANG_UOCR_MAX_PAGE_TOKENS=20000        # per-page output cap; must be > 0
export SGLANG_OPT_UOCR_SAM_REL_POS_BIAS_MB=64
export SGLANG_UOCR_SHM_WRAP_MODEL_DATA=1
export SGLANG_UOCR_SHM_WRAP_MIN_MB=1
export SGLANG_VLM_CACHE_SIZE_MB=0
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

python -m sglang.launch_server \
    --model-path baidu/Unlimited-OCR-2 \
    --revision 9cf077c4b206e0e2a31879df937b0b0bcba3661c \
    --served-model-name Unlimited-OCR-2 \
    --chat-template unlimited-ocr \
    --tokenizer-backend fastokens \
    --tokenizer-worker-num 8 \
    --mem-fraction-static 0.7 \
    --max-running-requests 64 \
    --max-prefill-tokens 2048 \
    --enable-custom-logit-processor \
    --enable-streaming-session \
    --allow-auto-truncate \
    --disable-radix-cache \
    --mm-preprocess-cache-size-mb 0 \
    --cuda-graph-backend-decode full --cuda-graph-max-bs-decode 64 \
    --cuda-graph-backend-prefill full \
    --cuda-graph-max-bs-prefill 16384 \
    --cuda-graph-bs-prefill 1024 2048 \
    --cuda-graph-config '{"prefill": {"backend": "full", "full_prefill_max_req": 64}}' \
    --host 127.0.0.1 \
    --port 10000

Run it inside tmux (or nohup ... &) so it keeps running if your SSH session drops. If the customized wheel rejects --revision, remove that line; the server then loads the latest commit of the repository.

The model card’s notes on memory, in short:

  • --mem-fraction-static is the share reserved for weights and KV cache; the card keeps 0.7 as its setting.
  • --max-prefill-tokens sizes a temporary tensor, so lowering it lowers that peak. The largest --cuda-graph-bs-prefill bucket must equal --max-prefill-tokens.
  • A smaller page window (SGLANG_UOCR_DOC_*_PAGE_WINDOW) lowers peak memory but weakens cross-page merging.

Check that the server is up from a second terminal on the instance:

curl -s http://127.0.0.1:10000/v1/models

What to check: the command returns a JSON model list that includes the name Unlimited-OCR-2 (from --served-model-name). If the connection is refused, the server has not finished starting or has stopped; look at the terminal where you launched it.

Step 5: Parse a document

The SGLang server you started in Step 4 serves an OpenAI-compatible /v1/chat/completions endpoint. This is the server running on your own instance, not CloudGPU’s model API. The model card’s client script sends page images as base64, adds the model’s own prompt ("<PAGE><image>" * pages + "Multi page parsing.") and a no-repeat n-gram logit processor. Save the script from the model card’s SGLang section as parse.py in ~/uocr2 and run it on the instance, inside the same virtual environment, because it imports DeepseekOCRNoRepeatNGramLogitProcessor from the SGLang wheel:

cd ~/uocr2 && source .venv/bin/activate
python parse.py

The card’s script ends with three examples you can edit: a list of page images (generate(pages)), a single page, and a PDF converted at 300 DPI (generate(pdf_to_images("your_doc.pdf", dpi=300))). Its max_tokens is the output budget for the whole document and must be positive.

To upload documents and copy results back, use scp or rsync over SSH, or the JupyterLab file browser.

Step 6 (optional): Reach the server from your own computer

The server listens on 127.0.0.1:10000 inside the instance. To call it from your laptop, open an SSH tunnel with the host and port shown for your instance (see our SSH port-forwarding guide):

ssh -N -L 10000:127.0.0.1:10000 -p <port> root@<host>

Then point a client at http://127.0.0.1:10000 on your own machine. The card’s script imports a helper from the SGLang wheel, so running it locally needs that wheel installed locally too; running the script on the instance avoids this. Check that your instance’s SSH server allows port forwarding; if ssh -N -L reports “administratively prohibited”, run the client on the instance instead.

What to check: with the tunnel open, curl -s http://127.0.0.1:10000/v1/models on your own computer returns the same model list as in Step 4.

Fallback: Transformers for a quick test

If you only want to check a few pages, the model card’s Transformers path needs no server. It was tested by the authors with Python 3.10 and these pinned versions, which we keep as published:

cd ~/uocr2
uv venv --python 3.10 .venv-hf
source .venv-hf/bin/activate
uv pip install torch==2.13.0 torchvision pillow transformers==4.52.1 huggingface-hub==0.36.0 \
  tokenizers==0.21.4 addict==2.4.0 einops==0.8.2 easydict==1.13

import torch
from transformers import AutoModel, AutoTokenizer

model_name = "baidu/Unlimited-OCR-2"
revision = "9cf077c4b206e0e2a31879df937b0b0bcba3661c"   # the commit you reviewed
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True, revision=revision)
model = AutoModel.from_pretrained(model_name, trust_remote_code=True, use_safetensors=True,
                                  torch_dtype=torch.bfloat16, revision=revision).eval().cuda()

# Single page (dynamic tiling, "gundam mode", by default)
text = model.infer(tokenizer, image_file="your_doc/page_1.jpg",
                   output_path="output/single", save_results=True)

# Several pages of one document, in page order
pages = [f"your_doc/page_{i}.jpg" for i in range(1, 8)]
text, num_tokens = model.infer_multi(tokenizer, image_files=pages,
                                     output_path="output/multi", save_results=True)

The revision= argument is our addition; the model card’s own snippet loads the latest commit.

What trust_remote_code=True does

The repository’s config.json maps AutoConfig and AutoModel to classes in modeling_unlimitedocr.py, a Python file inside the model repository rather than inside the Transformers library. trust_remote_code=True tells Transformers to download that file (and the files it imports, such as deepencoder.py and modeling_deepseekv2.py) and run it on your instance. Without the flag, loading fails. Because this runs third-party code with your instance’s permissions:

  • Read the .py files on the model page before you run them.
  • Pin revision= to the commit you reviewed, so a later push to the repository doesn’t change the code you run.
  • Keep secrets such as API tokens out of the environment of that process when you don’t need them.

What about vLLM?

Status only: as of 2026-10-27, vLLM support is in an open PR. vllm-project/vllm #64016 (“Add Unlimited-OCR-2 DASH attention support”) was open and not merged when we checked. Until it lands in a vLLM release, use the SGLang or Transformers path above; this guide gives no vLLM commands for Unlimited-OCR-2.

What to check: open the PR and look at its state (open or merged) before you plan around vLLM.

Before you destroy the instance

Copy your parsed output off the instance first (scp/rsync, or download from JupyterLab), then destroy the instance. Billing stops when you destroy the instance (see /terms).

Troubleshooting

Symptom What to check
uv pip install wheel/... says the file is not found Run it from ~/uocr2, where uvx hf download --local-dir . placed the wheel/ folder.
Out-of-memory during prefill or CUDA graph capture Follow the model card’s memory notes: keep --mem-fraction-static 0.7, lower --max-prefill-tokens and make the largest --cuda-graph-bs-prefill bucket equal to it, lower --max-running-requests, or use a smaller page window.
Errors mentioning /dev/shm or shared memory Check df -h /dev/shm. You can set SGLANG_UOCR_SHM_WRAP_MODEL_DATA=0 so tensors are not passed through shared memory (the card enables it to avoid pickling large tensors).
ImportError for DeepseekOCRNoRepeatNGramLogitProcessor Run the client inside ~/uocr2/.venv, which has the customized SGLang wheel, not the regular PyPI build.
Empty or cut-off output max_tokens in the request is the budget for the whole document; raise tokens_per_page in the card’s script.
Transformers raises an error asking for trust_remote_code Pass trust_remote_code=True to both from_pretrained calls after reviewing the repository code.

FAQ

What is Unlimited-OCR-2?

Unlimited-OCR-2 (Unlimited OCR Works 2) is a document-parsing vision-language model from Baidu, released on 2026-10-27 according to its Hugging Face model card. It reads page images and returns the parsed text, and its model card says it is built for very long, multi-page documents.

Should I use SGLang or Transformers to run Unlimited-OCR-2?

The model card says Transformers is the native option for quick tests and SGLang is recommended for serving. Use SGLang with the customized wheel shipped in the model repository if you want an OpenAI-compatible endpoint, served by your own instance rather than by CloudGPU’s model API, for many pages; use Transformers for a quick single-page check.

Does vLLM support Unlimited-OCR-2?

Not in a released version as of 2026-10-27. A pull request adding Unlimited-OCR-2 support to vLLM (vllm-project/vllm #64016) was open and not merged on that date, and the official vLLM recipe covers the earlier Unlimited-OCR model. Until support lands in a vLLM release, use the SGLang or Transformers path from the model card.

Why does Unlimited-OCR-2 need trust_remote_code=True?

Its model repository ships its own Python model code, so Transformers has to download and run that code to load the model. Read the .py files in the repository first and pin a commit with revision= so the code you reviewed is the code that runs.

What license does Unlimited-OCR-2 use?

The model card lists the MIT license (see the model card). Check the card for the current terms before you use the model in a product.

Which GPU should I rent for Unlimited-OCR-2?

VRAM need depends on model, precision and workflow, and we have not measured Unlimited-OCR-2 on our cards. Check the current card list on /pricing.