Skip to content

Azdo CI to run python tests #4034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
5 tasks done
DonJayamanne opened this issue Jan 17, 2019 · 0 comments
Closed
5 tasks done

Azdo CI to run python tests #4034

DonJayamanne opened this issue Jan 17, 2019 · 0 comments
Labels
area-testing debt Covers everything internal: CI, testing, refactoring of the codebase, etc.
Milestone
2019, week 3 - Ja...

Comments

@DonJayamanne
Copy link

DonJayamanne commented Jan 17, 2019 •
edited by d3r3kk
Loading

Add a job that will run pytest (I think this on), unittest, or whichever test framework we decide to use, on the Python scripts that we manage and that ship with our extension.

Test for completion

  • Run as part of the PR-validation
  • Run as part of the Nightly CI
  • Succeed across all OS, all Python version phases - no skipping.
  • Produce junit-xml style output that can be uploaded to AzDO (so that the results show up in the Test tab of each build).
  • Do not significantly add to the time it takes to run a build

Tips

We should be able to simply add a new phase to the build/ci/templates/test-phase-job.yaml file that does the python tests.

Using pytest might cause issues with running on Python 2 phases. We might have to consider using unittest here, or perhaps an older version of pytest.
@DonJayamanne DonJayamanne added area-testing needs PR debt Covers everything internal: CI, testing, refactoring of the codebase, etc. labels Jan 17, 2019
@ghost ghost added the triage-needed Needs assignment to the proper sub-team label Jan 17, 2019
@DonJayamanne DonJayamanne mentioned this issue Jan 17, 2019
Closed
@d3r3kk d3r3kk removed the triage-needed Needs assignment to the proper sub-team label Jan 17, 2019
@DonJayamanne DonJayamanne mentioned this issue Mar 5, 2019
Closed
8 tasks
@DonJayamanne DonJayamanne mentioned this issue Mar 13, 2019
Closed
This was referenced Mar 15, 2019
Merged
Closed
Closed
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 19, 2019
@d3r3kk
Enable venv tests, split out step-template, update CI
49d1b47 
For microsoft#4806, microsoft#4034, others...

- Venv tests are now in the combined step-template for all CI
- PR validation build definition uses the step template
- CI validation build definition uses the step template
  - Runs all tests against all OS, Py 3.7 & 3.6 only
- Some flaky tests skipped (tracked in GH)
- Problems with test runs not finishing in Windows remains

---
@d3r3kk d3r3kk mentioned this issue Mar 19, 2019
Merged
9 tasks
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 19, 2019
@d3r3kk
Enable venv tests, split out step-template, update CI
a1d2d4b 
For microsoft#4806, microsoft#4034, others...

- Venv tests are now in the combined step-template for all CI
- PR validation build definition uses the step template
- CI validation build definition uses the step template
  - Runs all tests against all OS, Py 3.7 & 3.6 only
- Some flaky tests skipped (tracked in GH)
- Problems with test runs not finishing in Windows remains

---
d3r3kk added a commit that referenced this issue Mar 19, 2019
@d3r3kk
Enable venv tests, split out step-template, update CI (#4830)
9716f08 
* Enable venv tests, split out step-template, update CI

For #4806, #4034, others...

- Venv tests are now in the combined step-template for all CI
- PR validation build definition uses the step template
- CI validation build definition uses the step template
  - Runs all tests against all OS, Py 3.7 & 3.6 only
- Some flaky tests skipped (tracked in GH)
- Problems with test runs not finishing in Windows remains
@d3r3kk d3r3kk closed this as completed Mar 20, 2019
@ghost ghost removed the needs PR label Mar 20, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Apr 17, 2019
@lock lock bot unassigned d3r3kk Apr 17, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area-testing debt Covers everything internal: CI, testing, refactoring of the codebase, etc.
Projects
None yet
Milestone
2019, week 3 - Jan Sprint 2
Development

No branches or pull requests
2 participants
@DonJayamanne @d3r3kk