How to run Ollama on a cloud GPU

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Ollama with the official Linux install script, pull and run the open-weight tag the quickstart prints, and call the local API on port 11434 through an SSH tunnel. Copy the model files off the instance before you destroy it. Destroying the instance deletes its disk.

Which card to rent for a different model is How to choose a rental GPU for Ollama. This page does not repeat that guide’s weight table. A browser chat in front of this server is How to run Open WebUI on a cloud GPU. A different OpenAI-compatible server is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. Weight-and-context sizing for other models is How much VRAM do I need to run an LLM?.

The Ollama repository LICENSE file is the MIT License, copyright © Ollama (read 2026-11-12). The gemma4:e2b library page shows “Apache License Version 2.0” next to the model files (read the same day). Read both before you redistribute the program or the files.

The commands below are copied from the upstream files named in each step. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked.

What the quickstart says about memory

The Ollama quickstart local section for Gemma 4 E2B says: “The model download is about 7.2 GB. We recommend 8 GB of available VRAM, or unified memory on a Mac. Larger context windows need more memory. With less VRAM, Ollama can use system RAM, but responses may be slower.” (read 2026-11-12). The same library page’s model-information table lists Total Parameters for E2B as “2.3B effective (5.1B with embeddings)”. That is a parameter count, not a second VRAM figure. The library page also shows more than one file-size label. This page does not copy those labels, and it does not treat them as the quickstart’s “about 7.2 GB” sentence. The 7.2 GB sentence is a download size. The 8 GB sentence is the VRAM recommendation. This page does not add them together.

The Ollama README prints ollama run gemma4 with no size. This page does not treat that command as the quickstart’s gemma4:e2b tag.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from gpu-prices.json, updated_at 2026-11-12T19:13:48.059Z. GET /v1/public/gpu-catalog the same day (generated_at 2026-11-12T19:13:49.196Z) listed the same names and the same vram_gb values, plus Ascend 910B at 64, which is not in gpu-prices.json and is not listed here. The catalog response also includes an availability field. This page does not copy it. Every vram_gb below is 8 or higher, so every row meets the quickstart’s 8 GB recommendation for gemma4:e2b as a number. The quickstart says larger context windows need more memory. This page does not mark a row as enough for a longer context. Which listing you can start is on Deploy. This page does not say which listings are available.

Catalog name vram_gb
RTX 2080 8
RTX 3080 10
RTX 2080 Ti 11
RTX 3080 Ti 12
V100 16G 16
A30 24
RTX 3090 24
RTX 4090 24
RTX 4090D 24
RTX 4080 SUPER 32G 32
RTX 5090 32
RTX 5090D 32
V100 32G 32
A100 40G 40
A800 40G 40
A40 48
RTX 4090 48G 48
RTX 4090D 48G 48
RTX A6000 48
NVIDIA RTX PRO 5000 72G 72
RTX PRO 5000 72G 72
A100 80G 80
A800 80G 80
H800 80
RTX 6000D 83
H20 96G 96
RTX PRO 6000 96G 96
RTX PRO 6000 Max-Q 96
H20-3e 141G 141

The same catalog lists template ids ollama, ollama-qwen3, and jupyterlab-ollama. Each has min_vram_gb 24 and recommended_gpu RTX 4090 48G. Those are template fields, not the quickstart’s 8 GB sentence.

The hardware page says Ollama supports Nvidia GPUs with compute capability 5.0+ and driver version 550 and newer, and that compute capability 5.0 through 6.2 needs driver version 570 or newer. Its NVIDIA table names, among other strings, RTX 5090 at 12.0, RTX 4090 and RTX 4080 SUPER at 8.9, RTX 3090, RTX 3080 Ti, RTX 3080, A40, and RTX A6000 at 8.6, A100 and A30 at 8.0, RTX 2080 Ti and RTX 2080 at 7.5, and V100 at 7.0. Those rows are at 7.0 or higher, so the 570 sentence does not apply to them. The catalog names RTX 4080 SUPER 32G, A100 40G, A100 80G, V100 16G, and V100 32G add a memory suffix the hardware page does not print. This page does not add a compute capability for a catalog string the table does not print. Catalog names that do not appear in that table include RTX 4090D, RTX 4090 48G, RTX 4090D 48G, RTX 5090D, A800 40G, A800 80G, H800, RTX 6000D, H20 96G, H20-3e 141G, RTX PRO 6000 96G, RTX PRO 6000 Max-Q, RTX PRO 5000 72G, and NVIDIA RTX PRO 5000 72G. The table does name RTX PRO 5000 Blackwell and RTX PRO 6000 Blackwell at 12.0. Those are not the catalog strings. One compute-capability row on that page is not copied here.

Install on Linux

On Deploy, choose Raw GPU. The Linux page install section prints one command:

curl -fsSL https://ollama.com/install.sh | sh

The same command is the Linux block in the Ollama README. The Linux page’s manual section, which is a different path, prints:

ollama serve

and, in another terminal:

ollama -v

The manual section also prints sudo systemctl start ollama after you create the systemd unit on that page. This page did not read the body of install.sh and did not check whether that script already creates the unit. The Linux page’s example of pinning a version is OLLAMA_VERSION=0.5.7 on the same curl command. That is the page’s example. The GitHub releases API on 2026-11-12 listed the latest tag as v0.40.16, published 2026-11-09T18:21:19Z. The install command above does not pin that tag.

The Linux page’s driver check is nvidia-smi. How to install a specific PyTorch CUDA version is a PyTorch index check. Ollama’s install script is not that pip line.

To keep ollama serve after the SSH session closes, this page wraps it in tmux. The Linux page does not print tmux:

tmux new -s ollama
ollama serve

Pull a model and check the GPU

The quickstart’s local download and chat, copied:

ollama pull gemma4:e2b

ollama run gemma4:e2b

The quickstart says Ollama downloads the model and starts a chat, and that /bye leaves the chat. cloudgpu-net is not in the Ollama docs. The download speed guide is the switch for the shell that downloads weights.

The local FAQ says to use ollama ps to see what models are currently loaded into memory. It says the Processor column shows which memory the model was loaded into: 100% GPU means entirely into the GPU, 100% CPU means entirely in system memory, and 48%/52% CPU/GPU means partially onto both. The FAQ’s sample row is not copied here. The same FAQ says that by default models are kept in memory for 5 minutes before being unloaded. That sentence is about the loaded model, not about how long an instance takes to start. ollama stop gemma4 is the CLI reference’s stop command for that tag.

nvidia-smi on the Linux page is the driver check. It is not the FAQ’s Processor column. How to check GPU utilization on a cloud GPU is the utilization page.

Serve the API and open it through SSH

The local FAQ says Ollama binds 127.0.0.1 port 11434 by default, and that OLLAMA_HOST changes the bind address. This page leaves the bind on 127.0.0.1 and forwards the port. The tunnel command is not in the Ollama docs. It is the form in how to use SSH port forwarding on a cloud GPU:

ssh -N -L 11434:127.0.0.1:11434 -p <ssh-port> <ssh-user>@<ssh-host>

The quickstart’s local OpenAI-compatible chat example, copied. Run it on your computer while the tunnel is open, or on the instance against 127.0.0.1:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4:e2b",
    "messages": [
      {
        "role": "user",
        "content": "Say hello in one sentence."
      }
    ]
  }'

The quickstart says to read the answer from choices[0].message.content. The OpenAI compatibility page’s local section sets base_url to http://localhost:11434/v1/ and sets api_key to the string ollama, which that page says is required but ignored. That page’s local example uses a model tag this page does not paste. Its endpoint headings include /v1/chat/completions, /v1/completions, /v1/models, and /v1/models/{model}. The same page’s /v1/models note says owned_by defaults to "library".

The quickstart also prints a local Ollama API curl against http://localhost:11434/api/chat with the same gemma4:e2b tag. That is the Ollama API, not the /v1 path.

No ComfyUI install in the Ollama README

The Ollama README’s community list includes Open WebUI and does not document a ComfyUI install. The ComfyUI examples top-level directory list on 2026-11-12 had no Ollama directory. This page does not add a ComfyUI workflow. The chat UI path is the Open WebUI guide linked above.

When it fails

The Processor column is not 100% GPU. The local FAQ’s three Processor values are the check. The troubleshooting page says GPU discovery failures can show up in the server log as error codes such as “3” (not initialized), “46” (device unavailable), “100” (no device), “999” (unknown), or others, and it prints a log fragment failure during GPU discovery. This page did not run that log command.

CUDA out of memory. The Ollama pages this guide read do not print a CUDA out-of-memory flag or a VRAM cutoff other than the quickstart’s 8 GB recommendation for gemma4:e2b. If the process raises that error, use how to fix CUDA out of memory on a cloud GPU.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The hardware page lists RTX 5090 under compute capability 12.0. It does not print that error string. The RTX 5090 guide says that card is compute capability 12.0 (sm_120) and needs a PyTorch build for CUDA 12.8 or newer. Ollama’s install script is not that PyTorch wheel. This page did not re-test the script on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32, which is above the quickstart’s 8 GB recommendation. That comparison is the memory number only.

Save the models, then destroy

envconfig/config.go in Ollama v0.40.16 says the default directory is $HOME/.ollama/models unless OLLAMA_MODELS is set (read 2026-11-12). The manual ollama serve route on this page uses that default for the user who starts the process. The deploy page says the Raw GPU option includes “root within a dedicated container”. When that user is root and HOME is /root, the directory is /root/.ollama/models.

The local FAQ lists Linux as /usr/share/ollama/.ollama/models. The Linux page creates the service user with -d /usr/share/ollama, so that FAQ path is $HOME/.ollama/models for the ollama user. It applies when the systemd unit is the process that downloaded the models. It does not apply to a shell that runs ollama serve as root. Copy the directory that matches the process you actually started. OLLAMA_MODELS overrides both. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The pricing page states the billing rule in these words:

Per minute, rounded up, with a 1-minute minimum. One hour is pre-authorised while an instance runs, and unused minutes go back to your balance when you destroy the instance. There is no subscription and no minimum term.

Prices are not copied here.

Frequently asked questions

How much VRAM does the Ollama quickstart publish for gemma4:e2b?

The Ollama quickstart, read 2026-11-12, says the gemma4:e2b download is about 7.2 GB and recommends 8 GB of available VRAM, or unified memory on a Mac. It says larger context windows need more memory. Every vram_gb in https://cloudgpu.app/gpu-prices.json (updated_at 2026-11-12T19:13:48.059Z) is 8 or higher, so every row meets that 8 GB recommendation as a number. This page does not mark a row as enough for a longer context. Which listing you can start is on https://cloudgpu.app/deploy.

How do I tell whether Ollama loaded the model on the GPU?

The Ollama local FAQ says to run ollama ps. It says the Processor column shows 100% GPU when the model was loaded entirely into the GPU, 100% CPU when it was loaded entirely in system memory, and 48%/52% CPU/GPU when it was loaded partially onto both. The sample row in that FAQ is not copied on this page.

Which license covers the Ollama program and the gemma4:e2b files?

The Ollama repository LICENSE file is the MIT License. The copyright line names Ollama. The gemma4:e2b library page shows Apache License Version 2.0 next to the model files (read 2026-11-12). Read both before you redistribute the program or the files.

Do the downloaded models stay on the instance after I destroy it?

No. Copy the model directory off before you destroy the instance. envconfig/config.go in Ollama v0.40.16 says the default directory is $HOME/.ollama/models unless OLLAMA_MODELS is set. The manual ollama serve route on this page uses that default for the user who starts the process. The deploy page says the Raw GPU option includes root within a dedicated container. When that user is root and HOME is /root, the directory is /root/.ollama/models. The local FAQ lists /usr/share/ollama/.ollama/models for Linux. That path is the home of the ollama service user the Linux page creates with -d /usr/share/ollama. It applies to the systemd service, not to a shell that runs ollama serve as root. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

Sources (read 2026-11-12, Asia/Shanghai)

Cards you can start are on the Deploy page. Prices are on https://cloudgpu.app/pricing and are not repeated here.

Last updated: 2026-11-12 (Asia/Shanghai).