Skip to content
This repository was archived by the owner on Apr 23, 2025. It is now read-only.

Usage -> User Guide (initial) #382

Closed
wants to merge 19 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions content/docs/ref/comment.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,16 +33,21 @@ Any [generic option](/doc/ref) in addition to:
(`pr`, `commit`, `issue`), optionally with a reference (`issue/12`, `pr/17`,
`commit/`[rev](https://git-scm.com/docs/gitrevisions) [default: `pr` falling
back to `commit/HEAD`].

- `--watch`: Watch for changes and automatically update the comment (doesn't
exit, consider
[appending `&` to run in the background](<https://en.wikipedia.org/wiki/Job_control_(Unix)#Implementation>)).

- `--publish=<true|false>`: Upload any local images found in the Markdown report
[default: `true`].

- `--publish-native`: Use `--driver`'s native capabilities to `--publish` assets
instead of `--publish-url` (not available on `--driver=github`).

- `--publish-url=<...>`: Self-hosted image server URL [default:
`https://asset.cml.dev`], see
[minroud-s3](https://github.com/iterative/minroud-s3).

- `--watermark-title=<...>`: Hidden comment marker (useful to
[specify which comment to update](#managing-multiple-comments) in subsequent
`cml comment update` calls); `"{workflow}"` and `"{run}"` are auto-replaced.
Expand Down
13 changes: 13 additions & 0 deletions content/docs/ref/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
# Command Reference

CML provides a number of commands to help package the outputs of ML workflows
into a CML report, which may include numeric data or model performance
visualizations. Let's look at the typical sequence (after
[configuration](/doc/user-guide)):
Comment on lines 1 to +6
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we move it back?

@casperdcl don't we need some content in the cmd ref index though? Currently it's empty except for a Generic Options section. Seems a little bare. Imagine a user lands in this page as their entry point to the docs site (per analytics this does happen sometimes) or as one of the first pages visited. It should give some general info on cml commands as a whole, link to GS or Usage, etc., I think.

Granted, this could be addressed separately and/or we can repeat the text above in you prefer to also keep this general guidance in the Usage (now UG index) page. Lmk

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please revert


∞ `cml runner` launches a runner hosted by a cloud compute provider or
[on-premise](/doc/self-hosted-runners).

∞ `cml pr` commits a set of files to a new branch and create a pull request.

∞ `cml comment` posts a Markdown report as a comment on a commit or pull/merge
request.

## Generic Options

All CML commands support the following options:
Expand Down
19 changes: 12 additions & 7 deletions content/docs/sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,19 @@
}
]
},
"usage",
{
"label": "CML with DVC",
"slug": "cml-with-dvc"
},
{
"label": "Self-hosted Runners",
"slug": "self-hosted-runners"
"slug": "user-guide",
"source": "user-guide/index.md",
"children": [
{
"label": "CML with DVC",
"slug": "cml-with-dvc"
},
{
"label": "Self-hosted Runners",
"slug": "self-hosted-runners"
Comment on lines +32 to +37
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved these 2 existing docs into the (new) UG section

}
]
},
{
"label": "Command Reference",
Expand Down
111 changes: 41 additions & 70 deletions content/docs/usage.md → content/docs/user-guide/index.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Using CML
# User Guide: Introduction

A GitLab, GitHub, or Bitbucket account is required. Familiarity with
Comment on lines -1 to 3
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Repurposed (simplified) much of this doc as the new UG index page

[GitHub Actions](https://help.github.com/en/actions),
Expand All @@ -9,7 +9,9 @@ also be beneficial.
<toggle>
<tab title="GitHub">

The key file in any CML project is `.github/workflows/cml.yaml`:
Integrate CML to your
[workflow configuration](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions)
using a `.github/workflows/cml.yaml` file, for example:

```yaml
name: CML
Expand Down Expand Up @@ -44,8 +46,10 @@ jobs:
cml comment create report.md
```

The example above generates visual reports in pull requests:
[![](/img/cml_first_report.png)](https://github.com/iterative/cml_base_case/pull/2)
This generates visual reports in pull requests (see `cml comment create`):

![First CML report in GitHub](/img/cml_first_report.png) _From sample
[increase forest depth](https://github.com/iterative/cml_base_case/pull/2) PR_

We helpfully provide CML and other useful libraries pre-installed on our
[custom Docker images](/doc/self-hosted-runners#docker-images). In the above
Expand All @@ -65,7 +69,9 @@ and CML set up on an Ubuntu LTS base for convenience.
</tab>
<tab title="GitLab">

The key file in any CML project is `.gitlab-ci.yml`:
Integrate CML to your
[pipeline configuration](https://docs.gitlab.com/ee/ci/pipelines/#configure-a-pipeline)
using the `.gitlab-ci.yml` file, for example:

```yaml
train-model:
Expand All @@ -84,12 +90,19 @@ create-CML-report:
- cml comment create report.md
```

⚠️ You _must_ provide a
<admon type="warn">

You _must_ provide a
[personal or project access token (PAT)](/doc/self-hosted-runners#personal-access-token)
via a `REPO_TOKEN` variable.

The example above generates visual reports in merge requests:
[![](/img/GitLab_CML_report.png '=400')](https://gitlab.com/iterative.ai/cml-base-case/-/merge_requests/1)
</admon>

This generates visual reports in pull requests (see `cml comment create`):

![First CML report in GitLab](/img/GitLab_CML_report.png '=400') _From sample
[Experiment](https://gitlab.com/iterative.ai/cml-base-case/-/merge_requests/1)
MR_

We helpfully provide CML and other useful libraries pre-installed on our
[custom Docker images](/doc/self-hosted-runners#docker-images). In the above
Expand All @@ -108,7 +121,9 @@ set up on an Ubuntu LTS base for convenience.
</tab>
<tab title="Bitbucket">

The key file in any CML project is `bitbucket-pipelines.yml`:
Integrate CML to your
[pipeline configuration](https://support.atlassian.com/bitbucket-cloud/docs/configure-bitbucket-pipelinesyml/)
using the `bitbucket-pipelines.yml` file, for example:

```yaml
image: iterativeai/cml:0-dvc2-base1
Expand All @@ -128,19 +143,31 @@ pipelines:
- cml comment create report.md
```

⚠️ You _must_ provide
<admon type="warn">

You _must_ provide
[access credentials](/doc/self-hosted-runners#personal-access-token) via a
`REPO_TOKEN` variable.

The example above generates visual reports in pull requests:
[![](/img/bitbucket_cloud_pr.png '=600')](https://bitbucket.org/iterative-ai/cml-base-case/pull-requests/1)
</admon>

This generates visual reports in pull requests (see `cml comment create`):

![First CML report in BitBucket](/img/bitbucket_cloud_pr.png '=600') _From
sample
[Experiment](https://bitbucket.org/iterative-ai/cml-base-case/pull-requests/1)
PR_

⚠️ CML works with Bitbucket Cloud, where you can use the
<admon type="warn">

CML works with Bitbucket Cloud, where you can use the
[Bitbucket Pipelines](https://bitbucket.org/product/features/pipelines) CI/CD
system to run workflows automatically on triggering events. Bitbucket Server is
not yet supported.

### Example projects
</admon>

### Example project

- [Basic CML project](https://bitbucket.org/iterative-ai/cml-base-case)
- [CML with DVC to pull data](https://bitbucket.org/iterative-ai/cml-dvc-case)
Expand All @@ -149,59 +176,3 @@ not yet supported.

</tab>
</toggle>

## CML Commands

CML provides a number of commands to help package the outputs of ML workflows
(including numeric data and visualizations about model performance) into a CML
report.

Below is a list of CML commands for starting cloud compute runners, writing and
publishing Markdown reports to your CI/CD system.

∞ **[`runner`](/doc/ref/runner)**\
Launch a runner hosted by a cloud compute provider or locally on-premise (see [self-hosted runners](/doc/self-hosted-runners))\
e.g. `cml runner launch --cloud={aws,azure,gcp,kubernetes} ...`

∞ **[`pr`](/doc/ref/pr)**\
Commit specified files to a new branch and create a pull request\
e.g. `cml pr create "**/*.json" "**/*.py" --md >> report.md`

∞ **[`comment`](/doc/ref/comment)**\
Post a Markdown report as a commit comment\
e.g. `cml comment create report.md`

∞ **[`check`](/doc/ref/check)**\
Post a Markdown report as a GitHub check\
e.g. `cml check create report.md`

∞ **[`tensorboard`](/doc/ref/tensorboard)**\
Return a link to a <https://tensorboard.dev> page\
e.g. `cml tensorboard connect --logdir=./logs --md >> report.md`

### CML Reports

The `cml comment create` command can be used to post reports. CML reports are
written in Markdown ([GitHub](https://github.github.com/gfm),
[GitLab](https://docs.gitlab.com/ee/user/markdown.html), or
[Bitbucket](https://confluence.atlassian.com/bitbucketserver/markdown-syntax-guide-776639995.html)
flavors). That means they can contain images, tables, formatted text, HTML
blocks, code snippets and more — really, what you put in a CML report is up to
you. Some examples:

📝 **Text** Write to your report using whatever method you prefer. For example,
copy the contents of a text file containing the results of ML model training:

```cli
$ cat results.txt >> report.md
```

🖼️ **Images** Display images using the markdown or HTML. Note that if an image
is an output of your ML workflow (i.e. it is produced by your workflow), you can
use markdown to embed it in a CML report. For example, if `plot.png` is output
by `python train.py`, run:

```cli
$ echo '![](./plot.png)' >> report.md
$ cml comment create report.md
```
12 changes: 8 additions & 4 deletions redirects-list.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,15 @@
"^/doc/start/start-github(/.*)?$ /doc/start/github",
"^/doc/start/start-gitlab(/.*)?$ /doc/start/gitlab",

"^/doc/ref/send-github-check(/.*)?$ /doc/ref/check$1 301",
"^/doc/ref/send-comment(/.*)?$ /doc/ref/comment$1 301",
"^/doc/ref/tensorboard-dev(/.*)?$ /doc/ref/tensorboard$1 301",
"^/doc/usage$ /doc/user-guide",
"^/doc/cml-with-dvc$ /doc/user-guide/cml-with-dvc 302",
"^/doc/self-hosted-runners$ /doc/user-guide/self-hosted-runners 302",

"^/doc/cml-with-npm(/.*)?$ /doc/install$1 301",
"^/doc/ref/send-github-check(/.*)?$ /doc/ref/check$1",
"^/doc/ref/send-comment(/.*)?$ /doc/ref/comment$1",
"^/doc/ref/tensorboard-dev(/.*)?$ /doc/ref/tensorboard$1",

"^/doc/cml-with-npm(/.*)?$ /doc/install$1",

"^/(.+)/$ /$1"
]