How to run Florence-2 on a cloud GPU

Q: Which task prompts does the card print?

The large card prints , , , , , , , , and . The grounding task takes extra text. The large card's example text is A green car parked in front of a yellow building. The run_example function uses max_new_tokens 1024 and num_beams 3, and it does not pass do_sample. The card does not print a Gradio launch or a listen port.

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Load Florence-2 with the transformers snippet on the upstream model card, copy the repository id from that card instead of typing a vendor path from this page, run a task prompt the card prints, and copy the printed result off the instance before you destroy it. Destroying the instance deletes its disk.

Promptable detection from a different repository is How to run Grounding DINO on a cloud GPU. Promptable segmentation from a different repository is How to run SAM 2 (Segment Anything 2) on a cloud GPU. A document OCR pipeline whose docs print no GPU GB figure is How to run PaddleOCR on a cloud GPU. A PDF pipeline whose README says at least 12 GB of GPU RAM is How to convert PDFs to text with olmOCR on a cloud GPU. A vision-language server whose README prints a transformers minimum table is How to serve Qwen2.5-VL with vLLM on a cloud GPU. How published weight sizes are read: How much VRAM do I need to run an LLM?. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the process raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090. The card does not print a listen port. The tunnel, if a later process prints one, is How to use SSH port forwarding on a cloud GPU. An OCR repository whose README prints plain, format, fine-grained, and multi-page commands and no GPU GB figure is How to run GOT-OCR 2.0 on a cloud GPU.

The four cards this page read set front matter license: mit. The LICENSE file resolved from the large card’s license_link begins with “MIT License”. The copyright line names an organization. This page does not copy that name. That license text is not a CloudGPU price.

The commands below are copied from the upstream Florence-2 model cards read on 2026-11-14. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The cards also print evaluation tables. This page does not copy those tables.

What the model cards print

Each card’s size table prints the same four rows. The description cell says “Pretrained model with FLD-5B” for Florence-2-base and Florence-2-large, and “Finetuned model on a colletion of downstream tasks” for Florence-2-base-ft and Florence-2-large-ft. The spelling “colletion” is the cell this page read.

Model	Model size	Model description on the card
Florence-2-base	0.23B	Pretrained model with FLD-5B
Florence-2-large	0.77B	Pretrained model with FLD-5B
Florence-2-base-ft	0.23B	Finetuned model on a colletion of downstream tasks
Florence-2-large-ft	0.77B	Finetuned model on a colletion of downstream tasks

The Florence-2-large card’s summary says it is a continued pretrained version of Florence-2-large with 4k context length, that only 0.1B samples are used for continue pretraining, that the OCR task has been updated with a line separator, and it prints “COCO OD AP 39.8”. The next sentence names an organization. This page does not copy that name. The base, base-ft, and large-ft cards this page read do not print that 4k sentence.

Each card says all models are trained with float16. Each card’s summary says Florence-2 can interpret simple text prompts to perform tasks like captioning, object detection, and segmentation. The task section, copied below, prints caption, detection, and OCR prompts. This page did not find a separate segmentation prompt token in that section.

The cards print 0.23B and 0.77B in the size column. They print no GPU GB figure. This page does not turn those parameter counts into a GPU GB figure and does not mark a catalog row as enough.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-14T00:03:37.866Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-14T00:03:37.294Z) listed the same names and the same vram_gb values, plus Ascend 910B at 64, which is not in gpu-prices.json and is not listed here. The catalog response also includes an availability field. This page does not copy it. Every prices row has gpu_count 1. Every catalog row has min_gpu_count 1. Which listing you can start is on Deploy.

The catalog template list had no Florence-2 template.

The model API the same day reported gated false, cardData.license mit, and pipeline_tag image-text-to-text for all four cards.

Card title	sha	lastModified
Florence-2-base	5ca5edf5bd017b9919c05d08aebef5e4c7ac3bac	2025-08-04T17:36:57.000Z
Florence-2-large	21a599d414c4d928c9032694c424fb94458e3594	2025-08-04T17:09:36.000Z
Florence-2-base-ft	f6c1a25888ffc1d945ee8a1a77ac833c7303d46e	2025-08-04T17:12:06.000Z
Florence-2-large-ft	4a12a2b54b7016a48a22037fbd62da90cd566f2a	2025-08-04T17:11:22.000Z

Catalog name	vram_gb
RTX 2080	8
RTX 3080	10
RTX 2080 Ti	11
RTX 3080 Ti	12
V100 16G	16
A30	24
RTX 3090	24
RTX 4090	24
RTX 4090D	24
RTX 4080 SUPER 32G	32
RTX 5090	32
RTX 5090D	32
V100 32G	32
A100 40G	40
A800 40G	40
A40	48
RTX 4090 48G	48
RTX 4090D 48G	48
RTX A6000	48
NVIDIA RTX PRO 5000 72G	72
RTX PRO 5000 72G	72
A100 80G	80
A800 80G	80
H800	80
RTX 6000D	83
H20 96G	96
RTX PRO 6000 96G	96
RTX PRO 6000 Max-Q	96
H20-3e 141G	141

Download the weights

The card’s snippet does not print hf download or huggingface-cli. It loads with from_pretrained, and the string it passes is the repository id printed on that card. Copy that id from the card. This page does not print it.

The block below is added on this page. <repository-id-from-the-model-card> is a placeholder for the id you copy. How to download a Hugging Face model prints pip install -U "huggingface_hub" and hf download. That guide says that without --local-dir, libraries such as transformers read the Hugging Face cache, so a later from_pretrained of the same id finds the files. This page did not download the repository.

# Added on this page. The model card loads with from_pretrained and does not print hf download.
pip install -U "huggingface_hub"
hf download <repository-id-from-the-model-card>

Load and run a task

The card does not print a pip line for torch, Pillow, requests, or transformers. This page does not add a version pin.

The Florence-2-large card’s first snippet follows. The only change is the repository id. The card passes that id as a string literal. This page stores the id you copy in REPO_ID and says so. trust_remote_code=True is what the card passes to both from_pretrained calls.

import requests

import torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForCausalLM

device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32

# Added on this page. Copy the repository id from the model card.
# The card passes that id as a string literal. This page does not print it.
REPO_ID = "<repository-id-from-the-model-card>"

model = AutoModelForCausalLM.from_pretrained(REPO_ID, torch_dtype=torch_dtype, trust_remote_code=True).to(device)
processor = AutoProcessor.from_pretrained(REPO_ID, trust_remote_code=True)

prompt = "<OD>"

url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/car.jpg?download=true"
image = Image.open(requests.get(url, stream=True).raw)

inputs = processor(text=prompt, images=image, return_tensors="pt").to(device, torch_dtype)

generated_ids = model.generate(
    input_ids=inputs["input_ids"],
    pixel_values=inputs["pixel_values"],
    max_new_tokens=4096,
    num_beams=3,
    do_sample=False
)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=False)[0]

parsed_answer = processor.post_process_generation(generated_text, task="<OD>", image_size=(image.width, image.height))

print(parsed_answer)

The base card’s snippet passes the base card’s repository id. The base-ft card passes the base-ft id. The large-ft card passes the large-ft id. Copy the id from the card you opened. This page does not choose among the four.

The task section’s run_example uses max_new_tokens=1024 and num_beams=3, and it does not pass do_sample. The first snippet uses max_new_tokens=4096, num_beams=3, and do_sample=False. This page does not merge those two calls. When text_input is set, run_example sets prompt to task_prompt + text_input.

The task prompts the large card prints:

<CAPTION>
<DETAILED_CAPTION>
<MORE_DETAILED_CAPTION>
<CAPTION_TO_PHRASE_GROUNDING>, which the card says requires additional text input, for example A green car parked in front of a yellow building.
<OD>
<DENSE_REGION_CAPTION>
<REGION_PROPOSAL>
<OCR>
<OCR_WITH_REGION>

The large card’s grounding line is task_prompt = "<CAPTION_TO_PHRASE_GROUNDING>". The base-ft card’s line, as fetched, is task_prompt = '<CAPTION_TO_PHRASE_GROUNDING>", which opens with a single quote and closes with a double quote. This page copies the large card’s line.

The large card also prints run_example_with_score, which passes return_dict_in_generate=True and output_scores=True, then calls model.compute_transition_scores. The prompt in that example is <OD>. This page did not run it.

The large card links a sample inference notebook. This page does not copy that path. The card does not print a Gradio launch and does not print a listen port. This page does not add one.

When it fails

CUDA out of memory. The cards print 0.23B and 0.77B and print no GPU GB figure. If the process raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The cards do not pin a PyTorch build and do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and that PyTorch 2.7 and later, on a build for CUDA 12.8 or newer, is the line that announced support. This page did not install a torch wheel and did not re-test this snippet on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

Remote code. The card passes trust_remote_code=True to both from_pretrained calls. This page does not remove that argument. Read the files the card loads before you run them.

What to copy off, then destroy

The snippet prints parsed_answer. The card does not name an output directory. Copy any file you wrote, and the printout if you saved it, before you destroy the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The pricing page FAQ states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Prices are not copied here.

Frequently asked questions

How much VRAM do the Florence-2 cards print?

The size table prints 0.23B for Florence-2-base and Florence-2-base-ft, and 0.77B for Florence-2-large and Florence-2-large-ft. The cards print no GPU GB figure. The Florence-2-large summary also prints 4k context length, 0.1B samples for continue pretraining, and COCO OD AP 39.8. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Which loading code does the card print?

The card sets device to cuda:0 when CUDA is available, and torch_dtype to float16 on CUDA and float32 otherwise. It calls AutoModelForCausalLM.from_pretrained and AutoProcessor.from_pretrained with trust_remote_code=True. The first snippet uses prompt <OD>, max_new_tokens 4096, num_beams 3, and do_sample False, then post_process_generation with task <OD> and image_size from the image. Copy the repository id from the card. This page does not print that id and did not run the snippet.

Which task prompts does the card print?

The large card prints <CAPTION>, <DETAILED_CAPTION>, <MORE_DETAILED_CAPTION>, <CAPTION_TO_PHRASE_GROUNDING>, <OD>, <DENSE_REGION_CAPTION>, <REGION_PROPOSAL>, <OCR>, and <OCR_WITH_REGION>. The grounding task takes extra text. The large card’s example text is A green car parked in front of a yellow building. The run_example function uses max_new_tokens 1024 and num_beams 3, and it does not pass do_sample. The card does not print a Gradio launch or a listen port.

What license do the cards print?

The four cards set front matter license to mit. The LICENSE file resolved from the large card’s license_link begins with MIT License. The copyright line names an organization, and this page does not copy that name. Model API cardData.license was mit, gated was false, and pipeline_tag was image-text-to-text for all four cards on 2026-11-14.

Sources (read 2026-11-14, Asia/Shanghai)

Upstream Florence-2 model cards titled Florence-2-base, Florence-2-large, Florence-2-base-ft, and Florence-2-large-ft (size table 0.23B and 0.77B, float16, trust_remote_code=True, task prompts, large-card 4k summary and COCO OD AP 39.8, front matter license: mit). This page does not print the repository id. Copy it from the card. Model API: gated false, cardData.license mit, pipeline_tag image-text-to-text. Shas and lastModified are the table above
Technical report linked from the cards: https://arxiv.org/abs/2311.06242
LICENSE file resolved from the large card’s license_link (begins with MIT License; the copyright line names an organization this page does not copy)
Hub v1.0 migration is not cited here. The cards do not print huggingface-cli. The hf download block is the command shape in How to download a Hugging Face model
Public GPU memory: https://cloudgpu.app/gpu-prices.json (updated_at 2026-11-14T00:03:37.866Z) and https://cloudgpu.app/v1/public/gpu-catalog (generated_at 2026-11-14T00:03:37.294Z). GET /v1/public/compute-capabilities returned feature flags and no GPU list. The catalog had no Florence-2 template

Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.

Last updated: 2026-11-14 (Asia/Shanghai).