Skip to content

Also record coverage for tests #3875

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -1,4 +1,23 @@
[run]
branch = True
source = _pytest,testing
omit =
# standlonetemplate is read dynamically and tested by test_genscript
*standalonetemplate.py
.tox/*
/usr/*
setup.py

[report]
show_missing = True
skip_covered = True
exclude_lines=
# Have to re-enable the standard pragma
\#\s*pragma: no cover

# Don't complain if tests don't hit defensive assertion code:
^\s*raise AssertionError\b
^\s*raise NotImplementedError\b
^\s*return NotImplemented\b
^\s*raise$

# Don't complain if non-runnable code isn't run:
^if __name__ == ['"]__main__['"]:$
2 changes: 1 addition & 1 deletion src/_pytest/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

try:
from ._version import version as __version__
except ImportError:
except ImportError: # pragma: no cover
# broken installation, we don't even try
# unknown only works because we do poor mans version compare
__version__ = "unknown"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better leave this code as uncovered.
We're not trying to hit 100%, but is is OK for this to be not covered - also for other things below.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're not trying to hit 100%

We're not? 😆

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmation that, in fact, we are:

#2800 (review)

Ultimately we should strive to obtain 100% coverage...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/cc @nicoddemus

I think the crucial part from there is:

Even if we "cheat", we gain the benefit of demanding that new code from now on has 100% coverage.

This should be achieved instead by requiring a diff to be covered 100% - something which codecov provides then.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, it is ok to add some # pragma: no cover here and there for code that we can't really cover on tests in a practical manner, here is a prime example of that: this is just a safe guard for broken pytest installations.

4 changes: 2 additions & 2 deletions src/_pytest/_argcomplete.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ def __call__(self, prefix, **kwargs):
return completion


if os.environ.get("_ARGCOMPLETE"):
if os.environ.get("_ARGCOMPLETE"): # pragma: no cover
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one e.g. would be good to get covered in the long run.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't you think this could be covered in the long run?

try:
import argcomplete.completers
except ImportError:
Expand All @@ -99,7 +99,7 @@ def try_argcomplete(parser):
argcomplete.autocomplete(parser, always_complete_options=False)


else:
else: # pragma: no cover
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same.


def try_argcomplete(parser):
pass
Expand Down
6 changes: 3 additions & 3 deletions src/_pytest/_code/code.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@

builtin_repr = repr

if _PY3:
if _PY3: # pragma: no cover
from traceback import format_exception_only
else:
else: # pragma: no cover
from ._py2traceback import format_exception_only


Expand Down Expand Up @@ -125,7 +125,7 @@ def eval(self, code, **vars):
def exec_(self, code, **vars):
""" exec 'code' in the frame

'vars' are optiona; additional local variables
'vars' are optional; additional local variables
"""
f_locals = self.f_locals.copy()
f_locals.update(vars)
Expand Down
15 changes: 0 additions & 15 deletions testing/test_skipping.py
Original file line number Diff line number Diff line change
Expand Up @@ -351,21 +351,6 @@ def test_this():
"""
)

def xtest_dynamic_xfail_set_during_setup(self, testdir):
p = testdir.makepyfile(
"""
import pytest
def setup_function(function):
pytest.mark.xfail(function)
def test_this():
assert 0
def test_that():
assert 1
"""
)
result = testdir.runpytest(p, "-rxX")
result.stdout.fnmatch_lines(["*XFAIL*test_this*", "*XPASS*test_that*"])

def test_dynamic_xfail_no_run(self, testdir):
p = testdir.makepyfile(
"""
Expand Down
2 changes: 1 addition & 1 deletion tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ deps =
coveralls
codecov
commands =
coverage run --source=_pytest -m pytest testing
coverage run -m pytest testing
coverage report -m
coveralls
codecov
Expand Down