3.7.0: Hookspec method signature regression #3753

jperras · 2018-07-31T23:09:37Z

When upgrading from pytest 3.6.x to 3.7, it seems as though the signature for hookspec has changed:

Traceback (most recent call last):
  File "<redacted>/python3.6/site-packages/_pytest/hookspec.py", line 27, in <module>
    @hookspec(historic=True, warn_on_impl=PYTEST_NAMESPACE)
TypeError: __call__() got an unexpected keyword argument 'warn_on_impl'
ERROR: InvocationError for command '<redacted>/bin/pytest -x --pdb' (exited with code 1)

Reverting back to 3.6.4 fixes the issue.

Plugins being used:

    pytest-mock
    pytest-asyncio
    pytest_profiling

The text was updated successfully, but these errors were encountered:

nicoddemus · 2018-07-31T23:57:24Z

Hi @jperras,

The hook signature did not change, but we are passing the warn_on_impl option to pluggy which is featured in pluggy 0.7. Which pluggy version do you have installed?

jperras · 2018-08-01T01:00:37Z

I believe the version that was running that caused the error was pluggy==0.6.0; I've performed a few permutations of the package versions when debugging the issue, so it's possible that I am also reporting the incorrect version.

The error was encountered during an automated test run that installed pytest and the various dependencies; perhaps there is an order-dependent installation issue that I had not noticed until now.

Our tox.ini segment that prepares the testing environment dependencies:

deps=
    freezegun
    pytest-mock
    pytest
    pyserial
    pytest-asyncio
    websockets
    pytest_profiling
    -rrequirements.txt

nicoddemus · 2018-08-01T02:03:53Z

It might be pip dependency resolver playing tricks then, because pytest 3.7 does require pluggy>=0.7:

pytest/setup.py

Line 72 in 2534193

install_requires.append("pluggy>=0.7")

I suggest to try recreating the environment with -r to see if the problem goes away.

pytest requires pluggy >= 0.7, while we always use pluggy 0.6, as specified by requirements.txt. as this version is good enough for tox. but in tox.ini, we do use pytest, and no version is specified, so we have good chance running into pytest-dev/pytest#3753 Signed-off-by: Kefu Chai <[email protected]>

pytest requires pluggy >= 0.7, while we always use pluggy 0.6, as specified by requirements.txt. as this version is good enough for tox. but in tox.ini, we do use pytest, and no version is specified, so we have good chance running into pytest-dev/pytest#3753 also, remove pytest from tox.ini, as this dependency has been added in requirements.txt Signed-off-by: Kefu Chai <[email protected]>

nedbat · 2018-08-07T11:20:55Z

This is happening to me on Travis, even after I clear the caches, but only on some of the jobs? https://travis-ci.com/nedbat/coveragepy/builds/81086869 The failures are on the jobs measuring coverage, which might reuse the environment (I'm unclear on the Travis details).

RonnyPfannschmidt · 2018-08-07T12:17:41Z

@nedbat looks like travis is broken

pytest-dev/pytest#3753 (comment)

nedbat · 2018-08-07T16:37:51Z

The problem seems to be that tox has this requirement: pluggy<1,>=0.3.0 and pip gets confused. I've solved it by adding "pluggy>=0.7" to my own requirements file.

RonnyPfannschmidt · 2018-08-07T16:52:00Z

@nedbat neat find, thanks for sharing, it might need a fix in tox

nicoddemus · 2018-08-09T00:27:24Z

Closing as this is not really related to pytest. Thanks to all participants!

kevindawson · 2018-09-03T13:56:12Z

Sorry to be the bearer of bad news, but

Installing collected packages: pluggy, pytest
  Found existing installation: pluggy 0.6.0
    Uninstalling pluggy-0.6.0:
      Successfully uninstalled pluggy-0.6.0
  Found existing installation: pytest 3.6.4
    Uninstalling pytest-3.6.4:
      Successfully uninstalled pytest-3.6.4
Successfully installed pluggy-0.7.1 pytest-3.7.4
kevin@dual:~/git/pylsusb$ pytest --help
Traceback (most recent call last):
  File "/usr/local/bin/pytest", line 7, in <module>
    from pytest import main
  File "/usr/local/lib/python3.6/dist-packages/pytest.py", line 9, in <module>
    from _pytest.config import main, UsageError, cmdline, hookspec, hookimpl
  File "/usr/local/lib/python3.6/dist-packages/_pytest/config/__init__.py", line 19, in <module>
    import _pytest.hookspec  # the extension point definitions
  File "/usr/local/lib/python3.6/dist-packages/_pytest/hookspec.py", line 27, in <module>
    @hookspec(historic=True, warn_on_impl=PYTEST_NAMESPACE)
TypeError: __call__() got an unexpected keyword argument 'warn_on_impl'

still not fixed, rolling back to pytest-3.6.4
Linux dual 4.15.0-33-generic #36-Ubuntu SMP Wed Aug 15 16:00:05 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

RonnyPfannschmidt · 2018-09-03T13:58:29Z

@kevindawson still a pip issue - pytest has correctly listed that it needs a newer pluggy version

RonnyPfannschmidt · 2018-09-03T13:59:44Z

@kevindawson on second thought - it seems to more of a system issue

kevindawson · 2018-09-03T14:29:57Z

@RonnyPfannschmidt thanks for the prompt baff ;)

if it's a local system issue on my side
please give me a kick :)
do I need to stop using pip and use git clone && setup instead?

RonnyPfannschmidt · 2018-09-03T14:42:23Z

@kevindawson as starting point, ask pip what version of pluggy it has installed, also check the pythonpath

its possible that a system level install is shadowing the newer version (it looks like you are on a debian or derivative and did a global install)

kevindawson · 2018-09-03T15:00:57Z

@RonnyPfannschmidt

thanks for the kick I will go and look n hack about

many thanks

kevindawson · 2018-09-03T15:12:09Z

@RonnyPfannschmidt

you were quite correct

system level install is shadowing the newer version

again many thanks

kevindawson · 2018-09-03T16:29:27Z

@RonnyPfannschmidt

this snippet might be of some use

$ pipdeptree
pytest-cov==2.5.1
  - coverage [required: >=3.7.1, installed: 4.5.1]
  - pytest [required: >=2.6.0, installed: 3.7.4]
    - atomicwrites [required: >=1.0, installed: 1.2.1]
    - attrs [required: >=17.4.0, installed: 18.2.0]
    - more-itertools [required: >=4.0.0, installed: 4.3.0]
      - six [required: >=1.0.0,<2.0.0, installed: 1.11.0]
    - pluggy [required: >=0.7, installed: 0.7.1]
    - py [required: >=1.5.0, installed: 1.6.0]
    - setuptools [required: Any, installed: 40.2.0]
    - six [required: >=1.10.0, installed: 1.11.0]

alive n kicking now ;)

RonnyPfannschmidt · 2018-09-03T19:19:37Z

@kevindawson thanks for the update on the working package-set, please include a reference to the hicup you faced for future reference

i'm happy its alright now, but me and future victims of the problem might want to know whats wrong

kevindawson · 2018-09-04T17:27:11Z

@RonnyPfannschmidt ok, I trust this is what you were looking for ;)

My Hiccup

It's possible that a system level install is shadowing the newer version ? YES

I am new to Python but not to nix, so decided I would use 3.6.

As all apps are easy to install, I only need ONE copy even thought I do have numerous user accounts.

sudo -H pip3 install --upgrade --force-reinstall pytest-cov
- pytest-cov==2.5.1
- pytest==3.7.4
- pluggy==0.7.1

Should have been all that I needed, But

pip[3] list |grep pluggy OR pip[3] list |grep pytest
- I assumed that pip is using the same search rules as python, thus we see some pluggy==0.6.0 lying around from disrto install or deb.
- showed remnants of the following
  - pytest==3.6.4
  - pluggy==0.6.0
Looked for pytest & pluggy in /usr/local/lib/python[x.x]/dist-packages/
- rm -rf to _ files dirs of pluggy & pytest

CLEAN

sudo -H pip3 install --upgrade --force-reinstall pytest-cov

All Good

RonnyPfannschmidt · 2018-09-04T18:47:29Z

@kevindawson thanks for following up with the steps and the problem that happened

the issue you faced is the main reason why we strongly recommend using virtualenvs instead of the system python (since it is entirely possible that you inadvertently broke something that's part of your system by using pip on the system python using sudo)

kevindawson · 2018-09-04T20:02:28Z

@RonnyPfannschmidt

Thanks for the info. I was trying to avoid virtualenvs per project

as for (since it is entirely possible that you inadvertently broke something that's part of your system by using pip on the system python using sudo)
that is why I was only using python3.6 as ubuntu18.04 uses python2.x
I thought it would provide separation by using pip3

Any pointers on Git & virtualenvs best practices, please

RonnyPfannschmidt · 2018-09-05T04:55:34Z

hmm, personally i just use virtualenvs and tox often
pip3 still installs into the system, personally i strongly recommend to just get used to virtualenv as part of the work-flow

kevindawson · 2018-09-05T12:29:32Z

@RonnyPfannschmidt

after some RTFM
watching Kenneth Reitz - Pipenv: The Future of Python Dependency Management - PyCon 2018
I have decided to hack around with pipenv
- no more pip plus virtualenvs all in one box;)

many thanks

RonnyPfannschmidt · 2018-09-05T13:16:18Z

@kevindawson fabulous, i'm happy you found a setup that works for you and shared it with us

The-Compiler · 2019-07-03T12:35:33Z

This still seems like an issue in some way or another? I get the same when I try doing tox -e docs with the current pytest master, with tox 3.12.1:

reading sources... [ 93%] reference

Warning, treated as error:
autodoc: failed to import function 'hookspec.pytest_assertrepr_compare' from module '_pytest'; the following exception was raised:
Traceback (most recent call last):
  File "/home/florian/proj/pytest/.tox/docs/lib/python3.7/site-packages/sphinx/ext/autodoc/importer.py", line 232, in import_module
    __import__(modname)
  File "/home/florian/proj/pytest/src/_pytest/hookspec.py", line 578, in <module>
    @hookspec(historic=True, warn_on_impl=PYTEST_LOGWARNING)
TypeError: __call__() got an unexpected keyword argument 'warn_on_impl'

Indeed pip installed pluggy==0.6.0, I wonder why though?!

edit: It also doesn't install more-itertools, but seems to do so if I upgrade the pip installed in the virtualenv (from 9.0.1 which virtualenv installs to 19.1.1)

nicoddemus · 2019-07-03T22:58:02Z

Weird, it works for me with tox 3.12.1 and current master... 🤔

Based off discussion [here](pytest-dev/pytest#3753)

nicoddemus added the status: needs information reporter needs to provide more information; can be closed after 2 or more weeks of inactivity label Jul 31, 2018

RonnyPfannschmidt added the type: question general question, might be closed after 2 weeks of inactivity label Aug 1, 2018

tchaikov mentioned this issue Aug 4, 2018

setup,py,requirements.txt: add pytest ceph/teuthology#1199

Merged

nedbat added a commit to nedbat/coveragepy that referenced this issue Aug 7, 2018

Trying to diagnose a failure

f4dd527

pytest-dev/pytest#3753 (comment)

jarun referenced this issue in jarun/pdd Aug 8, 2018

Attempt a release

9949d72

nicoddemus closed this as completed Aug 9, 2018

adamjstewart mentioned this issue Aug 24, 2018

Installation issue : faulty install py-pytest spack/spack#9037

Closed

jedie added a commit to jedie/django-tools that referenced this issue Sep 5, 2018

pluggy>0.7 for pytest-dev/pytest#3753

2ee607a

shuttle1987 mentioned this issue Oct 29, 2018

[MRG] Fix issues with feature extraction and empty wav files persephone-tools/persephone#211

Merged

mhchia mentioned this issue Dec 9, 2018

Remove the code checks in Protocol.__init__ multiformats/py-multiaddr#24

Closed

jhermann added a commit to jhermann/rituals that referenced this issue Jan 23, 2019

pluggy must be 0.7+ → pytest-dev/pytest#3753

b656d3e

AdamSpannbauer mentioned this issue May 1, 2019

importing the package gives error. AdamSpannbauer/pixel_art#1

Closed

Lawouach mentioned this issue May 7, 2019

importlib_metadata breaks TravisCI builds pytest-dev/pluggy#205

Closed

AndreMiras mentioned this issue Jul 26, 2019

unexpected keyword argument 'warn_on_impl' mgeier/pytest-auto-parametrize#3

Open

PeterGhimself added a commit to space-concordia-robotics/robotics-prototype that referenced this issue Dec 4, 2019

[#221] Update pluggy version to appease pytest

de7c156

Based off discussion [here](pytest-dev/pytest#3753)

tomato42 mentioned this issue Mar 23, 2020

Update test vector to released RFC version tlsfuzzer/tlslite-ng#397

Merged

Uh oh!

3.7.0: Hookspec method signature regression #3753

3.7.0: Hookspec method signature regression #3753

Comments

jperras commented Jul 31, 2018

nicoddemus commented Jul 31, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jperras commented Aug 1, 2018

Uh oh!

nicoddemus commented Aug 1, 2018

Uh oh!

nedbat commented Aug 7, 2018

Uh oh!

RonnyPfannschmidt commented Aug 7, 2018

Uh oh!

nedbat commented Aug 7, 2018

Uh oh!

RonnyPfannschmidt commented Aug 7, 2018

Uh oh!

nicoddemus commented Aug 9, 2018

Uh oh!

kevindawson commented Sep 3, 2018

Uh oh!

RonnyPfannschmidt commented Sep 3, 2018

Uh oh!

RonnyPfannschmidt commented Sep 3, 2018

Uh oh!

kevindawson commented Sep 3, 2018

Uh oh!

RonnyPfannschmidt commented Sep 3, 2018

Uh oh!

kevindawson commented Sep 3, 2018

Uh oh!

kevindawson commented Sep 3, 2018

Uh oh!

kevindawson commented Sep 3, 2018

Uh oh!

RonnyPfannschmidt commented Sep 3, 2018

Uh oh!

kevindawson commented Sep 4, 2018

My Hiccup

CLEAN

Uh oh!

RonnyPfannschmidt commented Sep 4, 2018

Uh oh!

kevindawson commented Sep 4, 2018

Uh oh!

RonnyPfannschmidt commented Sep 5, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kevindawson commented Sep 5, 2018

Uh oh!

RonnyPfannschmidt commented Sep 5, 2018

Uh oh!

The-Compiler commented Jul 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nicoddemus commented Jul 3, 2019

Uh oh!

nicoddemus commented Jul 31, 2018 •

edited

Loading

RonnyPfannschmidt commented Sep 5, 2018 •

edited

Loading

The-Compiler commented Jul 3, 2019 •

edited

Loading