Skip to content
This repository was archived by the owner on Apr 23, 2025. It is now read-only.

Usage -> User Guide (initial) #382

Closed
wants to merge 19 commits into from
Closed

Usage -> User Guide (initial) #382

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
ae23976
guides: rename Usage->Config and
jorgeorpinel Nov 18, 2022
4b08551
redirect /usage -> /config
jorgeorpinel Nov 18, 2022
f70bf8e
Merge branch 'master' into config--ref and
jorgeorpinel Nov 24, 2022
d7dbc4e
guide: fix image links again (in captions)
jorgeorpinel Nov 24, 2022
42cd4ff
guide: put example repos back in main tabs (Config)
jorgeorpinel Nov 24, 2022
82b74d5
ref: bring comment-create details & examples
jorgeorpinel Nov 24, 2022
7c886ac
Update content/docs/ref/index.md
jorgeorpinel Nov 25, 2022
ff6bef4
guide: group lose guides into new section and
jorgeorpinel Nov 25, 2022
9662dd8
Merge branch 'config--ref' of github.com:iterative/cml.dev into confi…
jorgeorpinel Nov 25, 2022
9a0e162
Update content/docs/ref/index.md
jorgeorpinel Dec 7, 2022
4c222e9
Merge branch 'master' into config--ref
jorgeorpinel Dec 7, 2022
4b33274
Merge branch 'config--ref' of github.com:iterative/cml.dev into confi…
jorgeorpinel Dec 7, 2022
608f010
Merge branch 'master' into config--ref
jorgeorpinel Dec 15, 2022
3fc4554
ref: roll back cleanup to comment cmd
jorgeorpinel Dec 16, 2022
9a2d12b
guide: fix link
jorgeorpinel Dec 16, 2022
f862dfe
Merge branch 'master' into config--ref and
jorgeorpinel Dec 27, 2022
986736e
Merge branch 'master' into config--ref
jorgeorpinel Feb 24, 2023
2a4e24e
roll back unknown changes
jorgeorpinel Feb 24, 2023
84db09c
Roll back unrelated change
jorgeorpinel Mar 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions content/docs/ref/comment.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,27 @@ Any [generic option](/doc/ref) in addition to:

## Examples

### Post a text report

Write to your report using whatever method you prefer. For example, copy the
contents of a text file containing the results of ML model training:

```cli
$ cat results.txt >> report.md
```

### Post a graphic report

Display images using Markdown or HTML. Note that if an image is an output of
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved here from https://cml.dev/doc/usage#cml-reports

Copy link
Contributor

@casperdcl casperdcl Dec 19, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please revert

your ML workflow (i.e. it is produced by your workflow), you can use Markdown to
embed it in a CML report. For example, if `plot.png` is output by
`python train.py`, run:

```cli
$ echo '![](./plot.png)' >> report.md
$ cml comment create report.md
```

### Managing multiple comments

Repeatedly running `cml comment create` may produce too many comments. Meanwhile
Expand Down
13 changes: 13 additions & 0 deletions content/docs/ref/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
# Command Reference

CML provides a number of commands to help package the outputs of ML workflows
into a CML report, which may include numeric data or model performance
visualizations. Let's look at the typical sequence (after
[configuration](/doc/user-guide)):
Comment on lines 1 to +6
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved here from https://cml.dev/doc/usage#cml-commands

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we move it back?

@casperdcl don't we need some content in the cmd ref index though? Currently it's empty except for a Generic Options section. Seems a little bare. Imagine a user lands in this page as their entry point to the docs site (per analytics this does happen sometimes) or as one of the first pages visited. It should give some general info on cml commands as a whole, link to GS or Usage, etc., I think.

Granted, this could be addressed separately and/or we can repeat the text above in you prefer to also keep this general guidance in the Usage (now UG index) page. Lmk

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please revert


∞ `cml runner` launches a runner hosted by a cloud compute provider or
[on-premise](/doc/self-hosted-runners).

∞ `cml pr` commits a set of files to a new branch and create a pull request.

∞ `cml comment` posts a Markdown report as a comment on a commit or pull/merge
request.

## Generic Options

All CML commands support the following options:
Expand Down
19 changes: 12 additions & 7 deletions content/docs/sidebar.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,19 @@
}
]
},
"usage",
{
"label": "CML with DVC",
"slug": "cml-with-dvc"
},
{
"label": "Self-hosted Runners",
"slug": "self-hosted-runners"
"slug": "user-guide",
"source": "user-guide/index.md",
"children": [
{
"label": "CML with DVC",
"slug": "cml-with-dvc"
},
{
"label": "Self-hosted Runners",
"slug": "self-hosted-runners"
Comment on lines +32 to +37
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved these 2 existing docs into the (new) UG section

}
]
},
{
"label": "Command Reference",
Expand Down
0 content/docs/cml-with-dvc.md → content/docs/user-guide/cml-with-dvc.md
View file
Open in desktop
File renamed without changes.
111 changes: 41 additions & 70 deletions content/docs/usage.md → content/docs/user-guide/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Using CML
# User Guide: Introduction

A GitLab, GitHub, or Bitbucket account is required. Familiarity with
Comment on lines -1 to 3
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Repurposed (simplified) much of this doc as the new UG index page

[GitHub Actions](https://help.github.com/en/actions),
Expand All @@ -9,7 +9,9 @@ also be beneficial.
<toggle>
<tab title="GitHub">

The key file in any CML project is `.github/workflows/cml.yaml`:
Integrate CML to your
[workflow configuration](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions)
using a `.github/workflows/cml.yaml` file, for example:

```yaml
name: CML
Expand Down Expand Up @@ -46,8 +48,10 @@ jobs:
cml comment create report.md
```

The example above generates visual reports in pull requests:
[![](/img/cml_first_report.png)](https://github.com/iterative/cml_base_case/pull/2)
This generates visual reports in pull requests (see `cml comment create`):

![First CML report in GitHub](/img/cml_first_report.png) _From sample
[increase forest depth](https://github.com/iterative/cml_base_case/pull/2) PR_

We helpfully provide CML and other useful libraries pre-installed on our
[custom Docker images](/doc/self-hosted-runners#docker-images). In the above
Expand All @@ -67,7 +71,9 @@ and CML set up on an Ubuntu LTS base for convenience.
</tab>
<tab title="GitLab">

The key file in any CML project is `.gitlab-ci.yml`:
Integrate CML to your
[pipeline configuration](https://docs.gitlab.com/ee/ci/pipelines/#configure-a-pipeline)
using the `.gitlab-ci.yml` file, for example:

```yaml
train-model:
Expand All @@ -86,12 +92,19 @@ create-CML-report:
- cml comment create report.md
```

⚠️ You _must_ provide a
<admon type="warn">

You _must_ provide a
[personal or project access token (PAT)](/doc/self-hosted-runners#personal-access-token)
via a `REPO_TOKEN` variable.

The example above generates visual reports in merge requests:
[![](/img/GitLab_CML_report.png '=400')](https://gitlab.com/iterative.ai/cml-base-case/-/merge_requests/1)
</admon>

This generates visual reports in pull requests (see `cml comment create`):

![First CML report in GitLab](/img/GitLab_CML_report.png '=400') _From sample
[Experiment](https://gitlab.com/iterative.ai/cml-base-case/-/merge_requests/1)
MR_

We helpfully provide CML and other useful libraries pre-installed on our
[custom Docker images](/doc/self-hosted-runners#docker-images). In the above
Expand All @@ -110,7 +123,9 @@ set up on an Ubuntu LTS base for convenience.
</tab>
<tab title="Bitbucket">

The key file in any CML project is `bitbucket-pipelines.yml`:
Integrate CML to your
[pipeline configuration](https://support.atlassian.com/bitbucket-cloud/docs/configure-bitbucket-pipelinesyml/)
using the `bitbucket-pipelines.yml` file, for example:

```yaml
image: iterativeai/cml:0-dvc2-base1
Expand All @@ -130,19 +145,31 @@ pipelines:
- cml comment create report.md
```

⚠️ You _must_ provide
<admon type="warn">

You _must_ provide
[access credentials](/doc/self-hosted-runners#personal-access-token) via a
`REPO_TOKEN` variable.

The example above generates visual reports in pull requests:
[![](/img/bitbucket_cloud_pr.png '=600')](https://bitbucket.org/iterative-ai/cml-base-case/pull-requests/1)
</admon>

This generates visual reports in pull requests (see `cml comment create`):

![First CML report in BitBucket](/img/bitbucket_cloud_pr.png '=600') _From
sample
[Experiment](https://bitbucket.org/iterative-ai/cml-base-case/pull-requests/1)
PR_

⚠️ CML works with Bitbucket Cloud, where you can use the
<admon type="warn">

CML works with Bitbucket Cloud, where you can use the
[Bitbucket Pipelines](https://bitbucket.org/product/features/pipelines) CI/CD
system to run workflows automatically on triggering events. Bitbucket Server is
not yet supported.

### Example projects
</admon>

### Example project

- [Basic CML project](https://bitbucket.org/iterative-ai/cml-base-case)
- [CML with DVC to pull data](https://bitbucket.org/iterative-ai/cml-dvc-case)
Expand All @@ -151,59 +178,3 @@ not yet supported.

</tab>
</toggle>

## CML Commands

CML provides a number of commands to help package the outputs of ML workflows
(including numeric data and visualizations about model performance) into a CML
report.

Below is a list of CML commands for starting cloud compute runners, writing and
publishing Markdown reports to your CI/CD system.

∞ **[`runner`](/doc/ref/runner)**\
Launch a runner hosted by a cloud compute provider or locally on-premise (see [self-hosted runners](/doc/self-hosted-runners))\
e.g. `cml runner launch --cloud={aws,azure,gcp,kubernetes} ...`

∞ **[`pr`](/doc/ref/pr)**\
Commit specified files to a new branch and create a pull request\
e.g. `cml pr create "**/*.json" "**/*.py" --md >> report.md`

∞ **[`comment`](/doc/ref/comment)**\
Post a Markdown report as a commit comment\
e.g. `cml comment create report.md`

∞ **[`check`](/doc/ref/check)**\
Post a Markdown report as a GitHub check\
e.g. `cml check create report.md`

∞ **[`tensorboard`](/doc/ref/tensorboard)**\
Return a link to a <https://tensorboard.dev> page\
e.g. `cml tensorboard connect --logdir=./logs --md >> report.md`

### CML Reports

The `cml comment create` command can be used to post reports. CML reports are
written in Markdown ([GitHub](https://github.github.com/gfm),
[GitLab](https://docs.gitlab.com/ee/user/markdown.html), or
[Bitbucket](https://confluence.atlassian.com/bitbucketserver/markdown-syntax-guide-776639995.html)
flavors). That means they can contain images, tables, formatted text, HTML
blocks, code snippets and more — really, what you put in a CML report is up to
you. Some examples:

📝 **Text** Write to your report using whatever method you prefer. For example,
copy the contents of a text file containing the results of ML model training:

```cli
$ cat results.txt >> report.md
```

🖼️ **Images** Display images using the markdown or HTML. Note that if an image
is an output of your ML workflow (i.e. it is produced by your workflow), you can
use markdown to embed it in a CML report. For example, if `plot.png` is output
by `python train.py`, run:

```cli
$ echo '![](./plot.png)' >> report.md
$ cml comment create report.md
```
0 content/docs/self-hosted-runners.md → ...nt/docs/user-guide/self-hosted-runners.md
View file
Open in desktop
File renamed without changes.
12 changes: 8 additions & 4 deletions redirects-list.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,15 @@
"^/doc/start/start-github(/.*)?$ /doc/start/github",
"^/doc/start/start-gitlab(/.*)?$ /doc/start/gitlab",

"^/doc/ref/send-github-check(/.*)?$ /doc/ref/check$1 301",
"^/doc/ref/send-comment(/.*)?$ /doc/ref/comment$1 301",
"^/doc/ref/tensorboard-dev(/.*)?$ /doc/ref/tensorboard$1 301",
"^/doc/usage$ /doc/user-guide",
"^/doc/cml-with-dvc$ /doc/user-guide/cml-with-dvc 302",
"^/doc/self-hosted-runners$ /doc/user-guide/self-hosted-runners 302",

"^/doc/cml-with-npm(/.*)?$ /doc/install$1 301",
"^/doc/ref/send-github-check(/.*)?$ /doc/ref/check$1",
"^/doc/ref/send-comment(/.*)?$ /doc/ref/comment$1",
"^/doc/ref/tensorboard-dev(/.*)?$ /doc/ref/tensorboard$1",

"^/doc/cml-with-npm(/.*)?$ /doc/install$1",

"^/(.+)/$ /$1"
]