Skip to content

The dependency resolver fails with ResolutionTooDeep error #10096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
1 task done
arewm opened this issue Jun 23, 2021 · 5 comments
Closed
1 task done

The dependency resolver fails with ResolutionTooDeep error #10096

arewm opened this issue Jun 23, 2021 · 5 comments

Comments

@arewm
Copy link

arewm commented Jun 23, 2021 •
edited
Loading

Description

When trying to install the azure-cli with pip 21.1.2, install takes a long time and eventually dies. Backing out to pip 21.1.1 seems to resolve this issue.

This might be a reoccurrence of #8314

Expected behavior

Packages should install as requested. In this situation, the install hangs on chardet which is already installed on the system. After backing out the pip version, the output looks something like

[...]
Collecting pynacl>=1.0.1
  Downloading PyNaCl-1.4.0.tar.gz (3.4 MB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Collecting bcrypt>=3.1.3
  Downloading bcrypt-3.2.0.tar.gz (42 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Collecting PyJWT[crypto]<3,>=1.0.0
  Downloading PyJWT-2.0.1-py3-none-any.whl (15 kB)
  Downloading PyJWT-2.0.0-py3-none-any.whl (15 kB)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/lib/python3.8/site-packages (from requests~=2.22->azure-cli-core==2.25.0->azure-cli) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /usr/lib/python3.8/site-packages (from requests~=2.22->azure-cli-core==2.25.0->azure-cli) (2.9)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/lib/python3.8/site-packages (from requests~=2.22->azure-cli-core==2.25.0->azure-cli) (1.25.9)
Collecting oauthlib>=3.0.0
  Downloading oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
INFO: pip is looking at multiple versions of urllib3 to determine which version is compatible with other requirements. This could take a while.
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
  Downloading urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
  Downloading urllib3-1.25.10-py2.py3-none-any.whl (127 kB)
  Downloading urllib3-1.25.8-py2.py3-none-any.whl (125 kB)
  Downloading urllib3-1.25.7-py2.py3-none-any.whl (125 kB)
  Downloading urllib3-1.25.6-py2.py3-none-any.whl (125 kB)
  Downloading urllib3-1.25.5-py2.py3-none-any.whl (125 kB)
  Downloading urllib3-1.25.4-py2.py3-none-any.whl (125 kB)
INFO: pip is looking at multiple versions of urllib3 to determine which version is compatible with other requirements. This could take a while.
  Downloading urllib3-1.25.3-py2.py3-none-any.whl (150 kB)
  Downloading urllib3-1.25.2-py2.py3-none-any.whl (150 kB)
  Downloading urllib3-1.24.3-py2.py3-none-any.whl (118 kB)
  Downloading urllib3-1.24.2-py2.py3-none-any.whl (131 kB)
  Downloading urllib3-1.24.1-py2.py3-none-any.whl (118 kB)
[...]

pip version

21.1.2

Python version

3.8.10

OS

alpine

How to Reproduce

I am hitting this bug when trying to install azure-cli in a Jenkins environment. The commands we are running are

apk add jq
apk del python2
apk update
apk add bash py-pip make gcc
apk add --virtual=build libffi-dev musl-dev openssl-dev python3-dev
apk add linux-headers musl-dev musl libc-dev libffi-dev
apk add openssl-dev openssl
apk add --no-cache ca-certificates
apk add --no-cache --virtual .build-deps curl
apk del .build-deps
apk add --no-cache python3
python3 -m ensurepip
rm -r /usr/lib/python*/ensurepip
pip3 install --upgrade pip setuptools
rm -r /root/.cache
pip3 install wheel
pip3 install --upgrade  azure-cli --no-cache-dir

https://github.com/open-cluster-management/bootstrap-ks/blob/main/aks/install.sh#L64-L82

Output

[...]
OK: 12766 distinct packages available
(1/32) Installing libbz2 (1.0.8-r1)
(2/32) Installing libffi (3.3-r2)
(3/32) Installing gdbm (1.13-r1)
(4/32) Installing xz-libs (5.2.5-r0)
(5/32) Installing sqlite-libs (3.32.1-r1)
(6/32) Installing python3 (3.8.10-r0)
(7/32) Installing py3-appdirs (1.4.4-r1)
(8/32) Installing py3-ordered-set (4.0.1-r0)
(9/32) Installing py3-parsing (2.4.7-r0)
(10/32) Installing py3-six (1.15.0-r0)
(11/32) Installing py3-packaging (20.4-r0)
(12/32) Installing py3-setuptools (47.0.0-r0)
(13/32) Installing py3-chardet (3.0.4-r4)
(14/32) Installing py3-idna (2.9-r0)
(15/32) Installing py3-certifi (2020.4.5.1-r0)
(16/32) Installing py3-urllib3 (1.25.9-r0)
[...]
Collecting oauthlib<3.0.0,>=2.1.0
  Downloading oauthlib-2.1.0-py2.py3-none-any.whl (121 kB)
Collecting requests-oauthlib>=0.5.0
  Downloading requests_oauthlib-1.0.0-py2.py3-none-any.whl (21 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
Collecting oauthlib>=0.6.2
  Downloading oauthlib-2.0.7-py2.py3-none-any.whl (124 kB)
  Downloading oauthlib-2.0.6.tar.gz (127 kB)
  Downloading oauthlib-2.0.5.tar.gz (129 kB)
  Downloading oauthlib-2.0.4.tar.gz (127 kB)
  Downloading oauthlib-2.0.3.tar.gz (127 kB)
  Downloading oauthlib-2.0.2.tar.gz (125 kB)
  Downloading oauthlib-2.0.1.tar.gz (122 kB)
  Downloading oauthlib-2.0.0.tar.gz (122 kB)
  Downloading oauthlib-1.1.2.tar.gz (111 kB)
  Downloading oauthlib-1.1.1.tar.gz (108 kB)
  Downloading oauthlib-1.1.0-1.tar.gz (106 kB)
WARNING: Discarding https://files.pythonhosted.org/packages/e5/54/4f96c51b171cf3a64a04b8c5167268803205bc5943b5cdf70bd770727b88/oauthlib-1.1.0-1.tar.gz#sha256=0f786c5573248a38efa86c48c59c0c93140ac836ab2a246aeefd8f9039e999ba (from https://pypi.org/simple/oauthlib/). Requested oauthlib>=0.6.2 from https://files.pythonhosted.org/packages/e5/54/4f96c51b171cf3a64a04b8c5167268803205bc5943b5cdf70bd770727b88/oauthlib-1.1.0-1.tar.gz#sha256=0f786c5573248a38efa86c48c59c0c93140ac836ab2a246aeefd8f9039e999ba (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-keyvault-administration==4.0.0b3->azure-cli) has inconsistent version: filename has '1.1.0.post1', but metadata has '1.1.0'
  Downloading oauthlib-1.0.3.tar.gz (109 kB)
  Downloading oauthlib-1.0.2.tar.gz (108 kB)
  Downloading oauthlib-1.0.1.tar.gz (108 kB)
  Downloading oauthlib-1.0.0.tar.gz (108 kB)
  Downloading oauthlib-0.7.2.tar.gz (106 kB)
  Downloading oauthlib-0.7.1.tar.gz (106 kB)
  Downloading oauthlib-0.7.0.tar.gz (105 kB)
  Downloading oauthlib-0.6.3.tar.gz (100 kB)
  Downloading oauthlib-0.6.2.tar.gz (100 kB)
Collecting requests-oauthlib>=0.5.0
  Downloading requests_oauthlib-0.8.0-py2.py3-none-any.whl (23 kB)
  Downloading requests_oauthlib-0.7.0-py2.py3-none-any.whl (22 kB)
  Downloading requests_oauthlib-0.6.2-py2.py3-none-any.whl (22 kB)
  Downloading requests_oauthlib-0.6.1-py2.py3-none-any.whl (21 kB)
INFO: pip is looking at multiple versions of requests-oauthlib to determine which version is compatible with other requirements. This could take a while.
  Downloading requests_oauthlib-0.6.0-py2.py3-none-any.whl (21 kB)
  Downloading requests_oauthlib-0.5.0-py2.py3-none-any.whl (19 kB)
INFO: pip is looking at multiple versions of idna to determine which version is compatible with other requirements. This could take a while.
Collecting idna<3,>=2.5
  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
  Downloading idna-2.9-py2.py3-none-any.whl (58 kB)
  Downloading idna-2.8-py2.py3-none-any.whl (58 kB)
  Downloading idna-2.7-py2.py3-none-any.whl (58 kB)
  Downloading idna-2.6-py2.py3-none-any.whl (56 kB)
  Downloading idna-2.5-py2.py3-none-any.whl (55 kB)
INFO: pip is looking at multiple versions of chardet to determine which version is compatible with other requirements. This could take a while.
Collecting chardet<4,>=3.0.2
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 180, in _main
    status = self.run(options, args)
  File "/usr/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 204, in wrapper
    return func(self, options, args)
  File "/usr/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 318, in run
    requirement_set = resolver.resolve(
  File "/usr/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 127, in resolve
    result = self._result = resolver.resolve(
  File "/usr/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 473, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 384, in resolve
    raise ResolutionTooDeep(max_rounds)
pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 2000000

Code of Conduct
@arewm arewm added S: needs triage Issues/PRs that need to be triaged type: bug A confirmed bug or unintended behavior labels Jun 23, 2021
@arewm arewm changed the title The dependency resolver fails with ResolutionTooDeep error for big project The dependency resolver fails with ResolutionTooDeep error Jun 23, 2021
@uranusjr
Copy link
Member

#10032 should be able to resolve this once and for all.

@uranusjr uranusjr added S: awaiting response Waiting for a response/more information and removed type: bug A confirmed bug or unintended behavior labels Jun 23, 2021
@uranusjr
Copy link
Member

It would be awesome if you could verify whether it works for you (it seems to for me).

@uranusjr
Copy link
Member

Side note: this is not likely related to #8314. ResolutioNTooDeep is a symptom for many issues, and I am quite sure the two issues have very different root causes.

@uranusjr uranusjr removed the S: needs triage Issues/PRs that need to be triaged label Jun 27, 2021
@no-response
Copy link

no-response bot commented Jul 8, 2021

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.

@no-response no-response bot closed this as completed Jul 8, 2021
@arewm
Copy link
Author

arewm commented Jul 9, 2021

Circling back to this, the issue might have actually been a problem with the alpine image we were using in Jenkins. We fixed it by adding

        apk add linux-headers alpine-sdk jq libc6-compat findutils
        apk add --no-cache python3 py3-pip

so this is working with the latest version of pip. Thanks for the eyes!

@no-response no-response bot removed the S: awaiting response Waiting for a response/more information label Jul 9, 2021
@no-response no-response bot reopened this Jul 9, 2021
@arewm arewm closed this as completed Jul 9, 2021
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 25, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@uranusjr @arewm