Skip to content

Commit 75e9bc4

Browse files
gloursaevesdocker
andauthored
add Compose how-to page for Docker Model Runner support with Compose (#22392)
<!--Delete sections as needed --> ## Description Add how-to page explaining how to use Docker Model Runner with Compose ## Related issues or tickets https://docker.atlassian.net/browse/APCLI-1068 ## Reviews <!-- Notes for reviewers here --> <!-- List applicable reviews (optionally @tag reviewers) --> - [x] Technical review - [x] Editorial review - [ ] Product review --------- Signed-off-by: Guillaume Lours <[email protected]> Co-authored-by: aevesdocker <[email protected]>
1 parent a7be08a commit 75e9bc4

File tree

2 files changed

+68
-0
lines changed

2 files changed

+68
-0
lines changed
Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
---
2+
title: Use Docker Model Runner
3+
description: Learn how to integrate Docker Model Runner with Docker Compose to build AI-powered applications
4+
keywords: compose, docker compose, model runner, ai, llm, artificial intelligence, machine learning
5+
weight: 111
6+
params:
7+
sidebar:
8+
badge:
9+
color: green
10+
text: New
11+
---
12+
13+
{{< summary-bar feature_name="Compose model runner" >}}
14+
15+
Docker Model Runner can be integrated with Docker Compose to run AI models as part of your multi-container applications.
16+
This lets you define and run AI-powered applications alongside your other services.
17+
18+
## Prerequisites
19+
20+
- Docker Compose v2.35 or later
21+
- Docker Desktop 4.41 or later
22+
- Docker Desktop for Mac with Apple Silicon or Docker Desktop for Windows with NVIDIA GPU
23+
- [Docker Model Runner enabled in Docker Desktop](/manuals/desktop/features/model-runner.md#enable-docker-model-runner)
24+
25+
## Provider services
26+
27+
Compose introduces a new service type called `provider` that allows you to declare platform capabilities required by your application. For AI models, you can use the `model` type to declare model dependencies.
28+
29+
Here's an example of how to define a model provider:
30+
31+
```yaml
32+
services:
33+
chat:
34+
image: my-chat-app
35+
depends_on:
36+
- ai-runner
37+
38+
ai-runner:
39+
provider:
40+
type: model
41+
options:
42+
model: ai/smollm2
43+
```
44+
45+
Notice the dedicated `provider` attribute in the `ai-runner` service.
46+
This attribute specifies that the service is a model provider and lets you define options such as the name of the model to be used.
47+
48+
There is also a `depends_on` attribute in the `chat` service.
49+
This attribute specifies that the `chat` service depends on the `ai-runner` service.
50+
This means that the `ai-runner` service will be started before the `chat` service to allow injection of model information to the `chat` service.
51+
52+
## How it works
53+
54+
During the `docker compose up` process, Docker Model Runner automatically pulls and runs the specified model.
55+
It also sends Compose the model tag name and the URL to access the model runner.
56+
57+
This information is then passed to services which declare a dependency on the model provider.
58+
In the example above, the `chat` service receives 2 environment variables prefixed by the service name:
59+
- `AI-RUNNER_URL` with the URL to access the model runner
60+
- `AI-RUNNER_MODEL` with the model name which could be passed with the URL to request the model.
61+
62+
This lets the `chat` service to interact with the model and use it for its own purposes.
63+
64+
## Reference
65+
66+
- [Docker Model Runner documentation](/manuals/desktop/features/model-runner.md)

data/summary.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,8 @@ Compose mac address:
105105
requires: Docker Compose [2.23.2](/manuals/compose/releases/release-notes.md#2232) and later
106106
Compose menu:
107107
requires: Docker Compose [2.26.0](/manuals/compose/releases/release-notes.md#2260) and later
108+
Compose model runner:
109+
requires: Docker Compose [2.35.0](/manuals/compose/releases/release-notes.md#2300) and later, and Docker Desktop 4.41 and later
108110
Compose OCI artifact:
109111
requires: Docker Compose [2.34.0](/manuals/compose/releases/release-notes.md#2340) and later
110112
Compose replace file:

0 commit comments

Comments
 (0)