@@ -20,7 +20,7 @@ example report for a
20
20
21
21
CML principles:
22
22
23
- - ** [ GitFlow] ( https://nvie.com/posts/a-successful-git-branching-model/ ) for data
23
+ - ** [ GitFlow] ( https://nvie.com/posts/a-successful-git-branching-model ) for data
24
24
science.** Use GitLab or GitHub to manage ML experiments, track who trained ML
25
25
models or modified data and when. Codify data and models with
26
26
[ DVC] ( #using-cml-with-dvc ) instead of pushing to a Git repo.
@@ -30,7 +30,7 @@ CML principles:
30
30
- ** No additional services.** Build your own ML platform using GitLab,
31
31
Bitbucket, or GitHub. Optionally, use
32
32
[ cloud storage] ( #configuring-cloud-storage-providers ) as well as either
33
- self-hosted or cloud runners (such as AWS EC2, Azure, or GCP ). No databases,
33
+ self-hosted or cloud runners (such as AWS EC2 or Azure ). No databases,
34
34
services or complex setup needed.
35
35
36
36
:question : Need help? Just want to chat about continuous integration for ML?
@@ -140,10 +140,12 @@ those reports to your CI system.
140
140
# ### CML Reports
141
141
142
142
The `cml-send-comment` command can be used to post reports. CML reports are
143
- written in [GitHub Flavored Markdown](https://github.github.com/gfm/). That
144
- means they can contain images, tables, formatted text, HTML blocks, code
145
- snippets and more — really, what you put in a CML report is up to you. Some
146
- examples :
143
+ written in markdown ([GitHub](https://github.github.com/gfm),
144
+ [GitLab](https://docs.gitlab.com/ee/user/markdown.html), or
145
+ [Bitbucket](https://confluence.atlassian.com/bitbucketserver/markdown-syntax-guide-776639995.html)
146
+ flavors). That means they can contain images, tables, formatted text, HTML
147
+ blocks, code snippets and more — really, what you put in a CML report is up to
148
+ you. Some examples :
147
149
148
150
:spiral_notepad : **Text** Write to your report using whatever method you prefer.
149
151
For example, copy the contents of a text file containing the results of ML model
314
316
315
317
> :point_right: `AWS_SESSION_TOKEN` is optional.
316
318
319
+ > :point_right: `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` can also be used
320
+ > by `cml-runner` to launch EC2 instances. See [Environment Variables].
321
+
317
322
</details>
318
323
319
324
<details>
@@ -397,7 +402,8 @@ data.
397
402
398
403
When a workflow requires computational resources (such as GPUs), CML can
399
404
automatically allocate cloud instances using `cml-runner`. You can spin up
400
- instances on your AWS or Azure account (GCP support is forthcoming!).
405
+ instances on your AWS or Azure account (GCP & Kubernetes support is
406
+ forthcoming!).
401
407
402
408
For example, the following workflow deploys a `t2.micro` instance on AWS EC2 and
403
409
trains a model on the instance. After the job runs, the instance automatically
@@ -408,6 +414,11 @@ You might notice that this workflow is quite similar to the
408
414
environment variables for passing your cloud service credentials to the
409
415
workflow.
410
416
417
+ Note that `cml-runner` will also automatically restart your jobs (whether from a
418
+ [GitHub Actions 72-hour timeout](https://docs.github.com/en/actions/reference/usage-limits-billing-and-administration#usage-limits)
419
+ or a
420
+ [AWS EC2 spot instance interruption](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html)).
421
+
411
422
` ` ` yaml
412
423
name: Train-in-the-cloud
413
424
on: [push]
@@ -447,7 +458,8 @@ jobs:
447
458
448
459
In the workflow above, the `deploy-runner` step launches an EC2 `t2-micro`
449
460
instance in the `us-west` region. The `model-training` step then runs on the
450
- newly-launched instance.
461
+ newly-launched instance. See [Environment Variables] below for details on the
462
+ ` secrets` required.
451
463
452
464
> :tada: **Note that you can use any container with this workflow!** While you
453
465
> must [have CML and its dependencies set up](#local-package) to use functions
@@ -536,10 +548,11 @@ Options:
536
548
537
549
Note that you will also need to provide access credentials for your cloud
538
550
compute resources as secrets. In the above example, `AWS_ACCESS_KEY_ID` and
539
- ` AWS_SECRET_ACCESS_KEY` are required to deploy EC2 instances.
551
+ ` AWS_SECRET_ACCESS_KEY` (with privileges to create & destroy EC2 instances) are
552
+ required.
540
553
541
- Please see our docs about
542
- [configuring cloud storage providers ](#configuring-cloud-storage-providers).
554
+ For AWS, the same credentials can also be used for
555
+ [configuring cloud storage](#configuring-cloud-storage-providers).
543
556
544
557
# ### On-premise (Local) Runners
545
558
0 commit comments