feat(ci): add cache and job skip #60
Conversation
|
note: I wanted to use env var to make the thing less verbose, but it seems that adding |
| runs-on: ubuntu-20.04 | ||
| needs: setup | ||
| if: ${{ always() && needs.setup.outputs.RUN_SPECS_SEARCH == 'true' }} |
There was a problem hiding this comment.
So jobs that needs a job that has been skipped is not also skipped (not sure if it's clear)
There was a problem hiding this comment.
for example we can skip the spec but still run the cts ?
There was a problem hiding this comment.
I don't get the && then, for me always is always true right ?
There was a problem hiding this comment.
For example, if you change the JavaScript mustache template, it will skip the specs_* jobs since no spec changes but still run the clients_javascript* jobs, and as the clients_javascript* have changed, it will run the CTS
| echo "::set-output name=JAVA_CLIENT_CHANGED::$(git diff --shortstat origin/${{ github.base_ref }}..HEAD -- clients/algoliasearch-client-java-2 | wc -l)" | ||
| echo "::set-output name=JAVA_TEMPLATE_CHANGED::$(git diff --shortstat origin/${{ github.base_ref }}..HEAD -- templates/java | wc -l)" | ||
|
|
||
| outputs: |
|
|
||
| # js client variables | ||
| RUN_JS_CLIENT_SEARCH: | ||
| description: 'Determine if the `client_javascript_search` job should run' | ||
| value: ${{ steps.diff.outputs.GITHUB_ACTIONS_CHANGED > 0 || steps.diff.outputs.SEARCH_SPECS_CHANGED > 0 || steps.diff.outputs.SCRIPTS_CHANGED > 0 || steps.diff.outputs.JS_SEARCH_CLIENT_CHANGED > 0 || steps.diff.outputs.JS_TEMPLATE_CHANGED > 0 }} | ||
| value: ${{ steps.diff.outputs.GITHUB_ACTIONS_CHANGED > 0 || steps.diff.outputs.COMMON_SPECS_CHANGED > 0 || steps.diff.outputs.SEARCH_SPECS_CHANGED > 0 || steps.diff.outputs.SCRIPTS_CHANGED > 0 || steps.diff.outputs.JS_SEARCH_CLIENT_CHANGED > 0 || steps.diff.outputs.JS_TEMPLATE_CHANGED > 0 }} |
There was a problem hiding this comment.
Is it possible to use the RUN_SPECS_SEARCH value here ?
There was a problem hiding this comment.
I did not managed to do it, it would be ideal indeed
🧭 What and Why
🎟 JIRA Tickets:
Changes included:
This PR aims at improving our CI run time which is now around 8 minutes.
I'll change the
required checksonce this PR is ready to be mergedTask skipping
setup: runs everytimespecs_*: runs when GitHub actions, scripts or spec change (e.g. only runsspecs_searchwhenspecs/searchchanges)client_*: runs when GitHub actions, scripts, language template or related spec change (e.g. only runsclient_javascript_searchwhenspecs/searchchanges)cts: runs everytime except scriptsComparison
Previous workflow
Total duration: ~7m/8m
Current workflow
Total duration: uncached 5m 27s | cached 3m 44s
TODO
Clients are cached based on their dep file, we should find an other way
🧪 Test
CI :D