How to run Semantic Kernel agents with a self-hosted model on a cloud GPU

Rent a GPU on Deploy, then open a terminal in JupyterLab or over SSH. Install Semantic Kernel the way the Python README prints, serve an open-weight model on that machine, and point the documented OpenAI-compatible connector at that server. Copy response.content before you destroy the instance, because destroying the instance deletes its disk and stops the server.

An agent can rent the GPU, serve the model, and run this agent on it. The agent pages are GPU compute for AI agents and the agent integration guide. Those pages are how an agent discovers a GPU and runs work on it. This page is the Semantic Kernel setting. A gateway whose README prints uv tool install 'litellm[proxy]' is How to run LiteLLM Proxy in front of a self-hosted model on a cloud GPU. A TypeScript SDK whose compatible-providers page calls createOpenAICompatible is How to use the Vercel AI SDK with a self-hosted model on a cloud GPU. An answering engine whose Docker command publishes port 3000, and whose Ollama base URL placeholder is http://localhost:11434, is How to run Perplexica AI search with a self-hosted model on a cloud GPU. An assistant whose setup page opens http://localhost:42110, and whose Ollama page sets OPENAI_BASE_URL to http://localhost:11434/v1/, is How to run Khoj AI assistant with a self-hosted model on a cloud GPU. An editor whose Ollama config sets baseURL to http://localhost:11434/v1 is How to use Kilo Code with a self-hosted model on a cloud GPU. An editor whose local-model page sets the Ollama api_url to http://localhost:11434 is How to use Zed’s AI agent with a self-hosted model on a cloud GPU. A coding assistant whose Docker command publishes port 8080 and passes --model and --chat-model is How to run Tabby self-hosted coding assistant on a cloud GPU. A terminal client whose OpenAI-compatible page sets OPENAI_API_BASE is How to use Aider with a self-hosted model on a cloud GPU. An editor client whose install page is the VS Code extension is How to use Cline with a self-hosted model on a cloud GPU. An editor whose config.yaml sets an apiBase for chat, autocomplete, and embeddings is How to use Continue with a self-hosted model on a cloud GPU. An editor whose Ollama provider defaults to http://localhost:11434 is How to use Roo Code with a self-hosted model on a cloud GPU. A coding agent whose providers page sets package to the OpenAI-compatible provider is How to use OpenCode with a self-hosted model on a cloud GPU. A platform whose single-container guide sets CHAT_BASE_URL to http://host.docker.internal:11434/v1 for a local OpenAI-compatible server is How to run AutoGPT with a self-hosted model on a cloud GPU. An agent whose Docker command publishes port 5080, and whose Ollama provider sets api_base to http://host.docker.internal:11434, is How to run Agent Zero with a self-hosted model on a cloud GPU. An agent whose Ollama example sets base_url to http://localhost:11434/v1 is How to run OpenManus with a self-hosted model on a cloud GPU. A chat app whose Docker page runs docker compose up -d and whose Ollama page sets baseURL to http://localhost:11434/v1/ is How to run LibreChat agents with a self-hosted model on a cloud GPU. A chat app whose README runs docker compose up -d and whose environment page sets OLLAMA_PROXY_URL to http://127.0.0.1:11434 is How to run LobeChat agents with a self-hosted model on a cloud GPU. A terminal agent whose install page runs curl -fsSL https://www.openinterpreter.com/install | sh, and whose models page sets the Ollama base URL to http://localhost:11434/v1, is How to run Open Interpreter with a self-hosted model on a cloud GPU. A workspace app whose Docker quickstart publishes port 3001, and whose Ollama page sets the URL to http://127.0.0.1:11434, is How to run AnythingLLM agents with a self-hosted model on a cloud GPU. A repository agent whose local-models section sets api_base to http://localhost:11434 is How to run SWE-agent with a self-hosted model on a cloud GPU. A workflow editor whose Docker install publishes port 5678, and whose Ollama credentials default the Base URL to http://localhost:11434, is How to run n8n AI agents with a self-hosted model on a cloud GPU. A CLI whose providers page sets OLLAMA_HOST to http://localhost:11434 is How to run the goose agent with a self-hosted model on a cloud GPU. A canvas whose self-host page opens http://localhost/install is How to run Dify with a self-hosted model on a cloud GPU. A canvas whose ChatOllama page names a base URL is How to run Flowise with a self-hosted model on a cloud GPU. A flow editor whose Ollama component defaults the base URL to http://127.0.0.1:11434 is How to run Langflow with a self-hosted model on a cloud GPU. A CLI whose connect help sets --base-url to http://localhost:8000/v1 for openai-compatible is How to run Letta agents with a self-hosted model on a cloud GPU. A browser chat whose serve command listens on port 8080 is How to run Open WebUI on a cloud GPU. A Python agent library whose CodeAgent can take TransformersModel or an OpenAI-compatible server is How to run smolagents with a self-hosted model on a cloud GPU. A crew whose LLM class takes base_url for Ollama or an OpenAI-compatible server is How to run CrewAI with a self-hosted model on a cloud GPU. A graph runtime whose overview example does not name a model is How to run LangGraph agents with a self-hosted model on a cloud GPU. An AgentChat library whose Ollama example sets model to llama3.2 is How to run AutoGen agents with a self-hosted model on a cloud GPU. A typed agent whose vLLM example sets base_url to http://localhost:8000/v1 is How to run Pydantic AI agents with a self-hosted model on a cloud GPU. A framework whose README sets Ollama and HuggingFaceEmbedding is How to build LlamaIndex agents with a self-hosted model on a cloud GPU. A browser agent whose models page sets base_url to http://localhost:11434/v1 is How to run browser-use with a self-hosted model on a cloud GPU. A pipeline whose OllamaChatGenerator sets url to http://localhost:11434 is How to run Haystack agents with a self-hosted model on a cloud GPU. A program whose language-models page sets dspy.LM to ollama_chat/llama3.2:1b is How to run DSPy with a self-hosted model on a cloud GPU. An agent whose Ollama example sets id to llama3.1:8b is How to run Agno agents with a self-hosted model on a cloud GPU. A TypeScript agent whose local-models section sets url on the model object is How to run Mastra agents with a self-hosted model on a cloud GPU. A ChatAgent whose vLLM example sets ModelPlatformType.VLLM is How to run CAMEL-AI agents with a self-hosted model on a cloud GPU. A local agent UI whose Docker command publishes port 3000 is How to run OpenHands with a self-hosted model on a cloud GPU. The README install is pip install --upgrade metagpt: How to run MetaGPT with a self-hosted model on a cloud GPU. A coding-agent server whose card prints --tool-call-parser qwen3_coder is How to serve Qwen3-Coder for coding agents on a cloud GPU. A general vLLM install is How to serve an OpenAI-compatible API with vLLM on a cloud GPU. A general SGLang install is How to run SGLang on a cloud GPU. Ollama’s install and port 11434 are How to run Ollama on a cloud GPU. CloudGPU’s hosted gateway is a different base URL: How do I use an OpenAI-compatible API for open models?. How published weight sizes are read: How much VRAM do I need to run an LLM?. The tunnel is How to use SSH port forwarding on a cloud GPU. If the process runs out of memory: How to fix CUDA out of memory on a cloud GPU. If the server raises no kernel image on an RTX 5090: How to fix CUDA no kernel image on an RTX 5090.

The upstream LICENSE is the MIT License.

The steps below are copied from the upstream files named in each step, at the commit in Sources. They were not run on a CloudGPU instance for this page. Anything this page adds around them is marked. The README names other providers. This page does not list those names. A model id in the local samples is not copied.

What the docs print about memory

python/README.md, python/samples/concepts/local_models/ollama_chat_completion.py, python/samples/concepts/local_models/lm_studio_chat_completion.py, and python/samples/getting_started_with_agents/chat_completion/step04_chat_completion_agent_plugin_simple.py print no GPU GB figure. The Ollama sample says the default model is used because of its compact size. That sentence prints no GPU GB figure. This page does not mark a catalog row as enough. The library is a client. The server’s memory is a different page: How much VRAM do I need to run an LLM?.

GET /v1/public/compute-capabilities does not list GPUs. It returns feature flags. The memory column is vram_gb from GET /v1/public/gpu-catalog (generated_at 2026-11-14T18:05:24.586Z). The catalog response also includes an availability field and a price field. This page does not copy them. Every catalog row below has min_gpu_count 1. Rows are sorted by vram_gb, then by name. Which listing you can start is on Deploy.

The catalog template list had no Semantic Kernel template. It did have Ollama templates (jupyterlab-ollama, ollama-qwen3, ollama). Those templates are a different install. This page does not treat them as Semantic Kernel.

Catalog name	vram_gb
RTX 2080	8
RTX 3080	10
RTX 2080 Ti	11
RTX 3080 Ti	12
V100 16G	16
A30	24
RTX 3090	24
RTX 4090	24
RTX 4090D	24
RTX 4080 SUPER 32G	32
RTX 5090	32
RTX 5090D	32
V100 32G	32
A100 40G	40
A800 40G	40
A40	48
RTX 4090 48G	48
RTX 4090D 48G	48
RTX A6000	48
Ascend 910B	64
NVIDIA RTX PRO 5000 72G	72
RTX PRO 5000 72G	72
A100 80G	80
A800 80G	80
H800	80
RTX 6000D	83
H20 96G	96
RTX PRO 6000 96G	96
RTX PRO 6000 Max-Q	96
H20-3e 141G	141

Install

Repository HEAD was ff22ec32ed31096c98ad63376d4ecc9aefe0ff38, committed 2026-11-13T16:11:10Z. python/semantic_kernel/__init__.py at that commit sets __version__ to 1.47.0. PyPI on 2026-11-14 reported semantic-kernel 1.47.0. python/pyproject.toml says requires-python is >=3.11. The README says Python 3.11 through 3.14.

The README’s opening note points readers to a successor agent framework and links a migration guide. This page follows the Semantic Kernel install and samples at this commit. It does not switch the install to that other repository.

The install block, copied:

pip install --upgrade semantic-kernel

The same block also prints pip install --upgrade semantic-kernel[hugging_face] and pip install --upgrade semantic-kernel[all]. The local sample below does not use those extras.

The pages this guide read do not print hf download or huggingface-cli. Weight download for the model server is a different guide: How to download a Hugging Face model.

Point the connector at the server

python/samples/concepts/local_models/ollama_chat_completion.py says it uses the OpenAI connector with a local model running in Ollama. The client, copied:

service_id = "local-model"

openAIClient: AsyncOpenAI = AsyncOpenAI(
    api_key="fake-key",  # This cannot be an empty string, use a fake key
    base_url="http://localhost:11434/v1",
)

The next line passes that client to OpenAIChatCompletion and sets ai_model_id. The id string is not copied. The comment in that file says that, at the time the sample was written, Ollama’s compatibility with the chat/completions endpoint was experimental, and it links https://github.com/ollama/ollama/blob/main/docs/openai.md. This page did not rewrite that comment.

OllamaChatCompletion in python/semantic_kernel/connectors/ai/ollama/services/ollama_chat_completion.py is a different class. Its __init__ takes host. The docstring says host defaults to http://127.0.0.1:11434. The class sets SUPPORTS_FUNCTION_CALLING to True. It raises ServiceInitializationError with Ollama chat model ID is required. when chat_model_id is missing. The Ollama sample above does not call OllamaChatCompletion. This page does not merge the two files.

python/samples/concepts/local_models/lm_studio_chat_completion.py uses the same client shape with base_url="http://localhost:1234/v1". Its ai_model_id is the same string this page does not copy.

None of these files print an ssh command. How to use SSH port forwarding on a cloud GPU is how a program on your computer reaches a port on the instance. This page does not add an ssh command. After a forward that lands on port 11434 of your computer, the URL the Ollama sample already prints is http://localhost:11434/v1. Ollama’s install is How to run Ollama on a cloud GPU. A vLLM server’s listen address is a different document: How to serve an OpenAI-compatible API with vLLM on a cloud GPU.

ChatCompletionAgent with one plugin function

The README section “Build an Agent with Plugins and Tools” defines MenuPlugin. One function, copied:

class MenuPlugin:
    @kernel_function(description="Provides a list of specials from the menu.")
    def get_specials(self) -> Annotated[str, "Returns the specials from the menu."]:
        return """
        Special Soup: Clam Chowder
        Special Salad: Cobb Salad
        Special Drink: Chai Tea
        """

The same class has a second function. Its return string prints an amount. This page does not copy that return string. The agent constructor, with the service line omitted, is the rest of that example. The omitted line names a hosted connector. This page does not copy that name.

agent = ChatCompletionAgent(
    name="SK-Assistant",
    instructions="You are a helpful assistant.",
    plugins=[MenuPlugin()],
)

The call on that page, copied:

response = await agent.get_response("What is the price of the soup special?")
print(response.content)

The output comment under that call prints an amount. This page does not copy it. step04_chat_completion_agent_plugin_simple.py uses the same plugin and passes plugins=[MenuPlugin()] to ChatCompletionAgent with name="Host" and instructions="Answer questions about the menu." Its service line names a hosted connector and a credential class. This page does not copy that line. The sample output in that file also prints an amount. This page does not copy it.

The README agent and the Ollama sample are different files. One constructs ChatCompletionAgent. The other constructs OpenAIChatCompletion with base_url="http://localhost:11434/v1". This page does not merge them into one script, and it did not run a combined script. The constructor field those files do not show together is service. The README passes a service into that field. The local sample builds the service with the client above.

When it fails

Empty API key. The Ollama sample comment says the key cannot be an empty string, and it sets api_key to fake-key.

Missing Ollama model id. OllamaChatCompletion raises Ollama chat model ID is required. when chat_model_id is missing. The sample that sets base_url does not use that class.

CUDA out of memory. The files this guide read print no GPU GB figure. If the model server raises CUDA out of memory, use how to fix CUDA out of memory on a cloud GPU. This page does not pick a model id for you.

“CUDA error: no kernel image is available for execution on the device” on an RTX 5090. The files this guide read do not pin a PyTorch build and do not name compute capability 12.0. The RTX 5090 guide says that card is compute capability 12.0 (sm_120). This page did not re-test a model server on an RTX 5090. Catalog RTX 5090 and RTX 5090D are 32. That is the catalog number only.

What to copy off, then destroy

The README prints response.content. The step 4 sample prints the response in the terminal. Neither file writes an output directory. Copy that text before you destroy the instance. The model server stops with the instance. Save your work before you destroy a cloud GPU is the checklist. The terms say data on local instance storage is permanently deleted when the instance is destroyed.

The billing page states the billing rule in these words:

Per-minute options have a 1-minute minimum; unused prepaid minutes are refunded when you destroy the instance. Some configurations use whole-hour billing with a 1-hour minimum; unused time within a charged hour is not refunded on voluntary stop. The billing period, minimum charge and initial prepayment are shown before confirmation. No subscription.

Amounts are not copied here.

Frequently asked questions

What memory figure do the Semantic Kernel files print?

The Python README, the Ollama local sample, the LM Studio local sample, and the plugin agent sample print no GPU GB figure. The Ollama sample says the default model is used because of its compact size and prints no GPU GB figure. This page does not mark a catalog row as enough. Which listing you can start is on https://cloudgpu.app/deploy.

Which sample sets a custom base URL?

ollama_chat_completion.py sets base_url to http://localhost:11434/v1 and api_key to fake-key, then passes the client to OpenAIChatCompletion. The ai_model_id on that line is not copied. OllamaChatCompletion is a different class. Its host docstring defaults to http://127.0.0.1:11434. The LM Studio sample sets base_url to http://localhost:1234/v1. Those files do not print an SSH command. This page did not run them.

Where is the documented plugin agent?

The README defines MenuPlugin.get_specials and passes plugins to ChatCompletionAgent with name SK-Assistant. The call is agent.get_response with the question What is the price of the soup special? The service line names a hosted connector and is not copied. A second function’s return string prints an amount and is not copied. step04 uses the same plugin with name Host. This page does not merge that agent with the Ollama sample.

What license sentence does the file print?

The upstream LICENSE is the MIT License. python/pyproject.toml classifiers include License :: OSI Approved :: MIT License. version in python/semantic_kernel/init.py is 1.47.0. PyPI on 2026-11-14 reported semantic-kernel 1.47.0. Repository HEAD was ff22ec32ed31096c98ad63376d4ecc9aefe0ff38, committed 2026-11-13T16:11:10Z. That license text is not a CloudGPU rate.

Sources (read 2026-11-14, Asia/Shanghai)

• semantic-kernel commit ff22ec32ed31096c98ad63376d4ecc9aefe0ff38, committed 2026-11-13T16:11:10Z. The upstream LICENSE is the MIT License. python/pyproject.toml classifier License :: OSI Approved :: MIT License. __version__ 1.47.0. PyPI semantic-kernel 1.47.0 the same day
• python/README.md at that commit. Python 3.11 through 3.14. pip install --upgrade semantic-kernel. The README’s opening note points readers to a successor agent framework. The highlights list names other providers. This page does not copy those names. The agent section’s service line and the amount in the sample output are not copied. MenuPlugin.get_specials is copied
• python/samples/concepts/local_models/ollama_chat_completion.py at that commit. base_url http://localhost:11434/v1. api_key fake-key. service_id local-model. The ai_model_id is not copied. The experimental-compatibility comment is copied as written
• python/semantic_kernel/connectors/ai/ollama/services/ollama_chat_completion.py at that commit. host default http://127.0.0.1:11434. SUPPORTS_FUNCTION_CALLING = True. Error Ollama chat model ID is required.
• python/samples/concepts/local_models/lm_studio_chat_completion.py at that commit. base_url http://localhost:1234/v1. The ai_model_id is not copied
• python/samples/getting_started_with_agents/chat_completion/step04_chat_completion_agent_plugin_simple.py at that commit. name="Host". instructions="Answer questions about the menu." plugins=[MenuPlugin()]. The service line and the amount in the sample output are not copied
• The files above do not print hf download or an ssh command
• Public GPU memory: https://cloudgpu.app/v1/public/gpu-catalog (generated_at 2026-11-14T18:05:24.586Z). GET /v1/public/compute-capabilities returned feature flags and no GPU list. The catalog had no Semantic Kernel template. It listed Ollama template ids jupyterlab-ollama, ollama-qwen3, and ollama

---

Cards you can start are on the Deploy page. Amounts are not copied here.

Last updated: 2026-11-14 (Asia/Shanghai).