condition can be evaluated in the real time #162

15klli · 2021-06-18T09:36:18Z

use the skipping.evaluate_condition from pytest in order to keep the same behavior with pytest.mark.skipif. It evaluates the condition in real time so that we can decide whether the case should be rerun in runtime

Change-Id: I2575d368c79480223c84498513a4ef605db0c576

icemac · 2021-06-22T06:26:54Z

@15klli Thank you for this PR. It currently fails when running the tests, could you please fix it.

15klli · 2021-06-22T08:48:16Z

@15klli Thank you for this PR. It currently fails when running the tests, could you please fix it.

ok, I will fix them.

15klli · 2021-06-23T09:05:54Z

I fix them by coping the evaluate_condition from skipping.py(python3.8.10), and all check passed in my PC. And I have to remind that this feature has compatible problem like the following:

@pytest.mark.flaky(condition="non_empty",reruns=2)
def test_demo():
     assert False

The above testcase will rerun 2 times(condition means True) with the front version but will fail and get error with the version having this feature, since it doesn't define non_empty.

But I believe almost no one will write "non_empty" string instead of writing True directly so that the case will always rerun if failed.

15klli · 2021-06-25T10:07:01Z

@icemac Hello, can you review it when you are free?

icemac · 2021-06-28T06:26:16Z

@15klli Currently the linter complains. Could you please fix these issues, too?

use the skipping.evaluate_condition from pytest in order to keep the same behavior with pytest.mark.skipif. It evaluates the condition in real time so that we can decide whether the case should be rerun in runtime Change-Id: I2575d368c79480223c84498513a4ef605db0c576

15klli · 2021-06-29T03:21:00Z

@15klli Currently the linter complains. Could you please fix these issues, too?

Done.

icemac

LGTM.

icemac · 2021-06-29T06:28:37Z

Thank you for this PR. 😃

sallner · 2021-06-29T08:07:05Z

Sorry for being late to the party. But there is still a changelog entry missing. And with this change in the condition parsing, it actually is a backwards incompatible change. @15klli what is the prospect of the copied code? As far as I understand the refactoring in pytest-dev/pytest#7388 it is include in pytest versions >= 6.0. That would be something to note at the code.

15klli · 2021-06-30T07:01:50Z

Sorry for being late to the party. But there is still a changelog entry missing. And with this change in the condition parsing, it actually is a backwards incompatible change. @15klli what is the prospect of the copied code? As far as I understand the refactoring in pytest-dev/pytest#7388 it is include in pytest versions >= 6.0. That would be something to note at the code.

The reasons why I copied the code instead of referring to the skipping.evaluate_condition are that:

The low version pytest doesn't have such function
The higher version pytest will throw a exception if the marker doesn't have the reason value which is required by skipif marker

That's why copied the code though I know it is not a graceful way.
Does That would be something to note at the code mean that I should write the note into the python code?

icemac · 2021-07-01T06:31:40Z

I added a change log entry in 138db6c.

15klli · 2021-07-02T08:28:03Z

@icemac when will you package it and upload to pypi.org ? I want to apply this feature on my project.

icemac · 2021-07-02T13:44:35Z

@15klli I just released https://pypi.org/project/pytest-rerunfailures/10.1/

icemac · 2021-07-02T13:45:29Z

@sallner Could you please fix the still open issues directly on master? Sorry for being too quick to merge the PR.

4926: Bump pytest-rerunfailures from 10.3 to 11.0 r=jenshnielsen a=dependabot[bot] Bumps [pytest-rerunfailures](https://github.com/pytest-dev/pytest-rerunfailures) from 10.3 to 11.0. <details> <summary>Changelog</summary> Sourced from <a href="https://github.com/pytest-dev/pytest-rerunfailures/blob/master/CHANGES.rst">pytest-rerunfailures's changelog</a>. <blockquote> <h2>11.0 (unreleased)</h2> Breaking changes ++++++++++++++++ <ul> <li> Drop support for Python 3.6. </li> <li> Drop support for pytest < 6. </li> </ul> Bug fixes +++++++++ <ul> <li> Fix crash when pytest-xdist is installed but disabled. (Thanks to <code>`@mgorny` <https://github.com/mgorny></code>_ for the PR.) </li> <li> Fix crash when xfail(strict=True) mark is used with --rerun-only flag. </li> </ul> Features ++++++++ <ul> <li> Added option <code>--rerun-except</code> to rerun failed tests those are other than the mentioned Error. </li> <li> Add support for Python 3.11. </li> <li> Add support for pytest 7.0, 7.1, 7.2. </li> </ul> <h2>10.2 (2021-09-17)</h2> Features ++++++++ <ul> <li>Allow recovery from crashed tests with pytest-xdist.</li> <li>Add support for Python 3.10 (as of Python 3.10.rc2). (Thanks to <code>`@hugovk` <https://github.com/hugovk></code>_ for the PR.)</li> </ul> <h2>10.1 (2021-07-02)</h2> Features ++++++++ <ul> <li>Allows using a <code>str</code> as condition for <code>`@pytest.mark.flaky(condition)</code>` which gets evaluated dynamically similarly to <code>`@pytest.mark.skipif(condition)</code>.` (<code>[#162](pytest-dev/pytest-rerunfailures#162) <https://github.com/pytest-dev/pytest-rerunfailures/pull/162></code>_</li> </ul>  </blockquote> ... (truncated) </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/4dc493befc4323310b568e461ad184881c0459d1"><code>4dc493b</code></a> Preparing release 11.0</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/3ffda7fad457fd6518915d0c2806b26566ba2b73"><code>3ffda7f</code></a> fix: don't bind all ips (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-rerunfailures/issues/201">#201</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/737183cb61c279292ed3a19c0b4655e64b0b4040"><code>737183c</code></a> Update dependencies: (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-rerunfailures/issues/202">#202</a>)</li> <li>See full diff in <a href="https://github.com/pytest-dev/pytest-rerunfailures/compare/10.3...11.0">compare view</a></li> </ul> </details> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pytest-rerunfailures&package-manager=pip&previous-version=10.3&new-version=11.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) You can trigger a rebase of this PR by commenting ``@dependabot` rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> You can trigger Dependabot actions by commenting on this PR: - ``@dependabot` rebase` will rebase this PR - ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it - ``@dependabot` merge` will merge this PR after your CI passes on it - ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it - ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging - ``@dependabot` reopen` will reopen this PR if it is closed - ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

4923: Move VoltageDivider into parameters and deprecate r=jenshnielsen a=jenshnielsen This is not an instrument but a Parameter so it makes more sense for it to live there. Since DelegateParamerer does the same more general and better deprecate this one 4926: Bump pytest-rerunfailures from 10.3 to 11.0 r=jenshnielsen a=dependabot[bot] Bumps [pytest-rerunfailures](https://github.com/pytest-dev/pytest-rerunfailures) from 10.3 to 11.0. <details> <summary>Changelog</summary> Sourced from <a href="https://github.com/pytest-dev/pytest-rerunfailures/blob/master/CHANGES.rst">pytest-rerunfailures's changelog</a>. <blockquote> <h2>11.0 (unreleased)</h2> Breaking changes ++++++++++++++++ <ul> <li> Drop support for Python 3.6. </li> <li> Drop support for pytest < 6. </li> </ul> Bug fixes +++++++++ <ul> <li> Fix crash when pytest-xdist is installed but disabled. (Thanks to <code>`@mgorny` <https://github.com/mgorny></code>_ for the PR.) </li> <li> Fix crash when xfail(strict=True) mark is used with --rerun-only flag. </li> </ul> Features ++++++++ <ul> <li> Added option <code>--rerun-except</code> to rerun failed tests those are other than the mentioned Error. </li> <li> Add support for Python 3.11. </li> <li> Add support for pytest 7.0, 7.1, 7.2. </li> </ul> <h2>10.2 (2021-09-17)</h2> Features ++++++++ <ul> <li>Allow recovery from crashed tests with pytest-xdist.</li> <li>Add support for Python 3.10 (as of Python 3.10.rc2). (Thanks to <code>`@hugovk` <https://github.com/hugovk></code>_ for the PR.)</li> </ul> <h2>10.1 (2021-07-02)</h2> Features ++++++++ <ul> <li>Allows using a <code>str</code> as condition for <code>`@pytest.mark.flaky(condition)</code>` which gets evaluated dynamically similarly to <code>`@pytest.mark.skipif(condition)</code>.` (<code>[#162](pytest-dev/pytest-rerunfailures#162) <https://github.com/pytest-dev/pytest-rerunfailures/pull/162></code>_</li> </ul>  </blockquote> ... (truncated) </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/4dc493befc4323310b568e461ad184881c0459d1"><code>4dc493b</code></a> Preparing release 11.0</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/3ffda7fad457fd6518915d0c2806b26566ba2b73"><code>3ffda7f</code></a> fix: don't bind all ips (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-rerunfailures/issues/201">#201</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/737183cb61c279292ed3a19c0b4655e64b0b4040"><code>737183c</code></a> Update dependencies: (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-rerunfailures/issues/202">#202</a>)</li> <li>See full diff in <a href="https://github.com/pytest-dev/pytest-rerunfailures/compare/10.3...11.0">compare view</a></li> </ul> </details> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pytest-rerunfailures&package-manager=pip&previous-version=10.3&new-version=11.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) You can trigger a rebase of this PR by commenting ``@dependabot` rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> You can trigger Dependabot actions by commenting on this PR: - ``@dependabot` rebase` will rebase this PR - ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it - ``@dependabot` merge` will merge this PR after your CI passes on it - ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it - ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging - ``@dependabot` reopen` will reopen this PR if it is closed - ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Co-authored-by: Jens H. Nielsen <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

4926: Bump pytest-rerunfailures from 10.3 to 11.0 r=jenshnielsen a=dependabot[bot] Bumps [pytest-rerunfailures](https://github.com/pytest-dev/pytest-rerunfailures) from 10.3 to 11.0. <details> <summary>Changelog</summary> Sourced from <a href="https://github.com/pytest-dev/pytest-rerunfailures/blob/master/CHANGES.rst">pytest-rerunfailures's changelog</a>. <blockquote> <h2>11.0 (unreleased)</h2> Breaking changes ++++++++++++++++ <ul> <li> Drop support for Python 3.6. </li> <li> Drop support for pytest < 6. </li> </ul> Bug fixes +++++++++ <ul> <li> Fix crash when pytest-xdist is installed but disabled. (Thanks to <code>`@mgorny` <https://github.com/mgorny></code>_ for the PR.) </li> <li> Fix crash when xfail(strict=True) mark is used with --rerun-only flag. </li> </ul> Features ++++++++ <ul> <li> Added option <code>--rerun-except</code> to rerun failed tests those are other than the mentioned Error. </li> <li> Add support for Python 3.11. </li> <li> Add support for pytest 7.0, 7.1, 7.2. </li> </ul> <h2>10.2 (2021-09-17)</h2> Features ++++++++ <ul> <li>Allow recovery from crashed tests with pytest-xdist.</li> <li>Add support for Python 3.10 (as of Python 3.10.rc2). (Thanks to <code>`@hugovk` <https://github.com/hugovk></code>_ for the PR.)</li> </ul> <h2>10.1 (2021-07-02)</h2> Features ++++++++ <ul> <li>Allows using a <code>str</code> as condition for <code>`@pytest.mark.flaky(condition)</code>` which gets evaluated dynamically similarly to <code>`@pytest.mark.skipif(condition)</code>.` (<code>[#162](pytest-dev/pytest-rerunfailures#162) <https://github.com/pytest-dev/pytest-rerunfailures/pull/162></code>_</li> </ul>  </blockquote> ... (truncated) </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/4dc493befc4323310b568e461ad184881c0459d1"><code>4dc493b</code></a> Preparing release 11.0</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/3ffda7fad457fd6518915d0c2806b26566ba2b73"><code>3ffda7f</code></a> fix: don't bind all ips (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-rerunfailures/issues/201">#201</a>)</li> <li><a href="https://github.com/pytest-dev/pytest-rerunfailures/commit/737183cb61c279292ed3a19c0b4655e64b0b4040"><code>737183c</code></a> Update dependencies: (<a href="https://github-redirect.dependabot.com/pytest-dev/pytest-rerunfailures/issues/202">#202</a>)</li> <li>See full diff in <a href="https://github.com/pytest-dev/pytest-rerunfailures/compare/10.3...11.0">compare view</a></li> </ul> </details> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pytest-rerunfailures&package-manager=pip&previous-version=10.3&new-version=11.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) You can trigger a rebase of this PR by commenting ``@dependabot` rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> You can trigger Dependabot actions by commenting on this PR: - ``@dependabot` rebase` will rebase this PR - ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it - ``@dependabot` merge` will merge this PR after your CI passes on it - ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it - ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging - ``@dependabot` reopen` will reopen this PR if it is closed - ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

15klli marked this pull request as draft June 18, 2021 09:38

15klli marked this pull request as ready for review June 18, 2021 09:38

15klli mentioned this pull request Jun 18, 2021

reruns condition evaluates not in real time #161

Open

15klli force-pushed the condition_canbe_str branch from 213f122 to 5e5a800 Compare June 23, 2021 08:45

15klli force-pushed the condition_canbe_str branch from 5e5a800 to 44dea8e Compare June 28, 2021 06:50

15klli force-pushed the condition_canbe_str branch from 44dea8e to bbc101d Compare June 29, 2021 03:10

icemac approved these changes Jun 29, 2021

View reviewed changes

icemac merged commit e0e5ba3 into pytest-dev:master Jun 29, 2021

icemac pushed a commit that referenced this pull request Jun 29, 2021

Add change log entry for #162

138db6c

sallner mentioned this pull request Jun 29, 2021

skipping: refactor mark evaluation pytest-dev/pytest#7388

Merged

15klli deleted the condition_canbe_str branch July 2, 2021 08:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

condition can be evaluated in the real time #162

condition can be evaluated in the real time #162

15klli commented Jun 18, 2021

icemac commented Jun 22, 2021

15klli commented Jun 22, 2021

15klli commented Jun 23, 2021 •

edited

Loading

15klli commented Jun 25, 2021

icemac commented Jun 28, 2021

15klli commented Jun 29, 2021

icemac left a comment

icemac commented Jun 29, 2021

sallner commented Jun 29, 2021

15klli commented Jun 30, 2021

icemac commented Jul 1, 2021

15klli commented Jul 2, 2021

icemac commented Jul 2, 2021

icemac commented Jul 2, 2021

condition can be evaluated in the real time #162

condition can be evaluated in the real time #162

Conversation

15klli commented Jun 18, 2021

icemac commented Jun 22, 2021

15klli commented Jun 22, 2021

15klli commented Jun 23, 2021 • edited Loading

15klli commented Jun 25, 2021

icemac commented Jun 28, 2021

15klli commented Jun 29, 2021

icemac left a comment

Choose a reason for hiding this comment

icemac commented Jun 29, 2021

sallner commented Jun 29, 2021

15klli commented Jun 30, 2021

icemac commented Jul 1, 2021

15klli commented Jul 2, 2021

icemac commented Jul 2, 2021

icemac commented Jul 2, 2021

15klli commented Jun 23, 2021 •

edited

Loading