-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Remove easy_install #917
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
probably worth just deprecating easy_install and adding a header from the docs that points to pip |
It's not so simple. Easy-install is used by tests requires and setup requires, which has not replacement. Perhaps it makes sense to only replace those functions. |
If I'm not mistaken, running |
See pypa/pip#2745 and pypa/pip#2361. |
Tests requires should be deprecated in favour of tox |
I don't think these issues are relevant. In the case where pip is invoking setup.py to build a package from source and that package has setup_requires dependencies, those dependencies would still be "installed", but to a temporary location (perhaps as eggs to ./.eggs as they are now), meaning they would still be isolated from the target environment via subprocesses and temporary directories, so I would expect not to encounter those issues. |
FWIW longer term PEP 518 is the solution to |
maybe for now, just a docs header saying, "hey unless you know what you're doing: don't use easy_install, you probably want pip instead" |
pypa/pip#1820 (comment) gives a reference on why recursive pip calls are not a good solution. I think this should be handled by setuptools, because setup_requires is a setuptools feature, which pip is currently unaware of. PEP 518 is only a specification of the actual build time dependencies, but not an attempt to solve the installation problem:
So this is still unsolved, isn't it? People tend not to use setup_requires, because easy_install does not use wheel and people have to compile eg. hard to build projects like scipy. So one should definitely prefer a wheel supporting mechanism. |
How about remove |
@jaraco what do you think about autogenerating pyproject.toml for source distributions? |
Sorry for the late follow-up.
I think I'd rather users be incentivized to provide the declaration in the preferred location. I'd support a warning, though. |
This issue continues to be an issue if only because not every invocation of |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
running install /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html Signed-off-by: Rong Tao <[email protected]>
running install /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Fix: iovisor#4586 Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html Signed-off-by: Rong Tao <[email protected]>
running install /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Fix: iovisor#4586 Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html Signed-off-by: Rong Tao <[email protected]>
running install /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Fix: iovisor#4586 Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html Signed-off-by: Rong Tao <[email protected]>
When setuptools>58.2.0 setup.py installation is a deprecated method, a warning message will be prompted [1]: $ sudo make install .... running install /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Use the recommended solution [2] to replace setup.py with pypa/build and pip-install to completely solve the problem. It is worth noting that under Debian, you need to add the --break-system-packages parameter to pip-uninstall, as shown in the following prompt: Debian12:~$ sudo pip3 uninstall bcc error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.11/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. # Successfully uninstalled Debian12:~$ sudo pip3 uninstall bcc --break-system-packages Fix: iovisor#4586 [1] Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2] Cc: Jerome Marchand <[email protected]> Cc: Hengqi Chen <[email protected]> Signed-off-by: Rong Tao <[email protected]>
When setuptools>58.2.0 setup.py installation is a deprecated method, a warning message will be prompted [1]: $ sudo make install .... running install /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Use the recommended solution [2] to replace setup.py with pypa/build and pip-install to completely solve the problem. It is worth noting that under Debian, you need to add the --break-system-packages parameter to pip-uninstall, as shown in the following prompt: Debian12:~$ sudo pip3 uninstall bcc error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.11/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. # Successfully uninstalled Debian12:~$ sudo pip3 uninstall bcc --break-system-packages Fix: iovisor#4586 [1] Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2] Cc: Jerome Marchand <[email protected]> Cc: Hengqi Chen <[email protected]> Signed-off-by: Rong Tao <[email protected]>
When setuptools>58.2.0 setup.py installation is a deprecated method, a warning message will be prompted [1]: $ sudo make install .... running install /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Use the recommended solution [2] to replace setup.py with pypa/build and pip-install to completely solve the problem. It is worth noting that under Debian, you need to add the --break-system-packages parameter to pip-uninstall, as shown in the following prompt: Debian12:~$ sudo pip3 uninstall bcc error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.11/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. # Successfully uninstalled Debian12:~$ sudo pip3 uninstall bcc --break-system-packages Fix: iovisor#4586 [1] Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2] Cc: Jerome Marchand <[email protected]> Cc: Hengqi Chen <[email protected]> Signed-off-by: Rong Tao <[email protected]>
…sts` ### What changes were proposed in this pull request? This pr enable `--use-pep517` option in `dev/run-pip-tests` refer to: ``` 25l DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457 ``` ### Why are the changes needed? To restore Github Action test failed: - https://github.com/apache/spark/actions/runs/14778042174/job/41496865818 ``` Installing collected packages: py4j, pyspark 25l DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457 Running setup.py develop for pyspark ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] error: subprocess-exited-with-error × python setup.py develop did not run successfully. │ exit code: 1 ╰─> [74 lines of output] /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. !! ******************************************************************************** Please consider removing the following classifiers in favor of a SPDX license expression: License :: OSI Approved :: Apache Software License See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. ******************************************************************************** !! self._finalize_license_expression() running develop /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``develop``. Instead, use standards-based tools like pip or uv. By 2025-Oct-31, you need to update your project and remove deprecated calls or your builds will no longer be supported. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Obtaining file:///__w/spark/spark/python Installing build dependencies: started Installing build dependencies: finished with status 'done' Checking if build backend supports build_editable: started Checking if build backend supports build_editable: finished with status 'done' Getting requirements to build editable: started Getting requirements to build editable: finished with status 'error' error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> [1 lines of output] Temp path for symlink to parent already exists deps [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 35, in <module> File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module> setup( File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup return distutils.core.setup(**attrs) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup return run_commands(dist) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands dist.run_commands() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands self.run_command(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command super().run_command(command) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run subprocess.check_call(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × python setup.py develop did not run successfully. │ exit code: 1 ╰─> [74 lines of output] /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. !! ******************************************************************************** Please consider removing the following classifiers in favor of a SPDX license expression: License :: OSI Approved :: Apache Software License See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. ******************************************************************************** !! self._finalize_license_expression() running develop /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``develop``. Instead, use standards-based tools like pip or uv. By 2025-Oct-31, you need to update your project and remove deprecated calls or your builds will no longer be supported. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Obtaining file:///__w/spark/spark/python Installing build dependencies: started Installing build dependencies: finished with status 'done' Checking if build backend supports build_editable: started Checking if build backend supports build_editable: finished with status 'done' Getting requirements to build editable: started Getting requirements to build editable: finished with status 'error' error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> [1 lines of output] Temp path for symlink to parent already exists deps [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 35, in <module> File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module> setup( File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup return distutils.core.setup(**attrs) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup return run_commands(dist) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands dist.run_commands() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands self.run_command(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command super().run_command(command) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run subprocess.check_call(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] 25h Cleaning up temporary directory - /tmp/tmp.zGt6QETKsu ``` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass GitHub Actions ### Was this patch authored or co-authored using generative AI tooling? No Closes #50778 from LuciferYang/use-pep517. Authored-by: yangjie01 <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
…sts` ### What changes were proposed in this pull request? This pr enable `--use-pep517` option in `dev/run-pip-tests` refer to: ``` 25l DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457 ``` ### Why are the changes needed? To restore Github Action test failed: - https://github.com/apache/spark/actions/runs/14778042174/job/41496865818 ``` Installing collected packages: py4j, pyspark 25l DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457 Running setup.py develop for pyspark ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] error: subprocess-exited-with-error × python setup.py develop did not run successfully. │ exit code: 1 ╰─> [74 lines of output] /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. !! ******************************************************************************** Please consider removing the following classifiers in favor of a SPDX license expression: License :: OSI Approved :: Apache Software License See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. ******************************************************************************** !! self._finalize_license_expression() running develop /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``develop``. Instead, use standards-based tools like pip or uv. By 2025-Oct-31, you need to update your project and remove deprecated calls or your builds will no longer be supported. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Obtaining file:///__w/spark/spark/python Installing build dependencies: started Installing build dependencies: finished with status 'done' Checking if build backend supports build_editable: started Checking if build backend supports build_editable: finished with status 'done' Getting requirements to build editable: started Getting requirements to build editable: finished with status 'error' error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> [1 lines of output] Temp path for symlink to parent already exists deps [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 35, in <module> File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module> setup( File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup return distutils.core.setup(**attrs) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup return run_commands(dist) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands dist.run_commands() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands self.run_command(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command super().run_command(command) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run subprocess.check_call(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × python setup.py develop did not run successfully. │ exit code: 1 ╰─> [74 lines of output] /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. !! ******************************************************************************** Please consider removing the following classifiers in favor of a SPDX license expression: License :: OSI Approved :: Apache Software License See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. ******************************************************************************** !! self._finalize_license_expression() running develop /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``develop``. Instead, use standards-based tools like pip or uv. By 2025-Oct-31, you need to update your project and remove deprecated calls or your builds will no longer be supported. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Obtaining file:///__w/spark/spark/python Installing build dependencies: started Installing build dependencies: finished with status 'done' Checking if build backend supports build_editable: started Checking if build backend supports build_editable: finished with status 'done' Getting requirements to build editable: started Getting requirements to build editable: finished with status 'error' error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> [1 lines of output] Temp path for symlink to parent already exists deps [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 35, in <module> File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module> setup( File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup return distutils.core.setup(**attrs) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup return run_commands(dist) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands dist.run_commands() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands self.run_command(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command super().run_command(command) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run subprocess.check_call(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] 25h Cleaning up temporary directory - /tmp/tmp.zGt6QETKsu ``` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass GitHub Actions ### Was this patch authored or co-authored using generative AI tooling? No Closes #50778 from LuciferYang/use-pep517. Authored-by: yangjie01 <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 31f7278) Signed-off-by: Dongjoon Hyun <[email protected]>
…sts` ### What changes were proposed in this pull request? This pr enable `--use-pep517` option in `dev/run-pip-tests` refer to: ``` 25l DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457 ``` ### Why are the changes needed? To restore Github Action test failed: - https://github.com/apache/spark/actions/runs/14778042174/job/41496865818 ``` Installing collected packages: py4j, pyspark 25l DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457 Running setup.py develop for pyspark ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] error: subprocess-exited-with-error × python setup.py develop did not run successfully. │ exit code: 1 ╰─> [74 lines of output] /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. !! ******************************************************************************** Please consider removing the following classifiers in favor of a SPDX license expression: License :: OSI Approved :: Apache Software License See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. ******************************************************************************** !! self._finalize_license_expression() running develop /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``develop``. Instead, use standards-based tools like pip or uv. By 2025-Oct-31, you need to update your project and remove deprecated calls or your builds will no longer be supported. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Obtaining file:///__w/spark/spark/python Installing build dependencies: started Installing build dependencies: finished with status 'done' Checking if build backend supports build_editable: started Checking if build backend supports build_editable: finished with status 'done' Getting requirements to build editable: started Getting requirements to build editable: finished with status 'error' error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> [1 lines of output] Temp path for symlink to parent already exists deps [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 35, in <module> File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module> setup( File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup return distutils.core.setup(**attrs) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup return run_commands(dist) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands dist.run_commands() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands self.run_command(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command super().run_command(command) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run subprocess.check_call(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × python setup.py develop did not run successfully. │ exit code: 1 ╰─> [74 lines of output] /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. !! ******************************************************************************** Please consider removing the following classifiers in favor of a SPDX license expression: License :: OSI Approved :: Apache Software License See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. ******************************************************************************** !! self._finalize_license_expression() running develop /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``develop``. Instead, use standards-based tools like pip or uv. By 2025-Oct-31, you need to update your project and remove deprecated calls or your builds will no longer be supported. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Obtaining file:///__w/spark/spark/python Installing build dependencies: started Installing build dependencies: finished with status 'done' Checking if build backend supports build_editable: started Checking if build backend supports build_editable: finished with status 'done' Getting requirements to build editable: started Getting requirements to build editable: finished with status 'error' error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> [1 lines of output] Temp path for symlink to parent already exists deps [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 35, in <module> File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module> setup( File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup return distutils.core.setup(**attrs) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup return run_commands(dist) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands dist.run_commands() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands self.run_command(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command super().run_command(command) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run subprocess.check_call(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] 25h Cleaning up temporary directory - /tmp/tmp.zGt6QETKsu ``` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass GitHub Actions ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#50778 from LuciferYang/use-pep517. Authored-by: yangjie01 <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
When setuptools>58.2.0 setup.py installation is a deprecated method, a warning message will be prompted [1]: $ sudo make install .... running install /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Use the recommended solution [2] to replace setup.py with pypa/build and pip-install to completely solve the problem. It is worth noting that under Debian, you need to add the --break-system-packages parameter to pip-uninstall, as shown in the following prompt: Debian12:~$ sudo pip3 uninstall bcc error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.11/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. # Successfully uninstalled Debian12:~$ sudo pip3 uninstall bcc --break-system-packages Fix: iovisor#4586 [1] Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2] Cc: Jerome Marchand <[email protected]> Cc: Hengqi Chen <[email protected]> Signed-off-by: Rong Tao <[email protected]>
…sts` ### What changes were proposed in this pull request? This pr enable `--use-pep517` option in `dev/run-pip-tests` refer to: ``` 25l DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457 ``` ### Why are the changes needed? To restore Github Action test failed: - https://github.com/apache/spark/actions/runs/14778042174/job/41496865818 ``` Installing collected packages: py4j, pyspark 25l DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457 Running setup.py develop for pyspark ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] error: subprocess-exited-with-error × python setup.py develop did not run successfully. │ exit code: 1 ╰─> [74 lines of output] /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. !! ******************************************************************************** Please consider removing the following classifiers in favor of a SPDX license expression: License :: OSI Approved :: Apache Software License See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. ******************************************************************************** !! self._finalize_license_expression() running develop /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``develop``. Instead, use standards-based tools like pip or uv. By 2025-Oct-31, you need to update your project and remove deprecated calls or your builds will no longer be supported. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Obtaining file:///__w/spark/spark/python Installing build dependencies: started Installing build dependencies: finished with status 'done' Checking if build backend supports build_editable: started Checking if build backend supports build_editable: finished with status 'done' Getting requirements to build editable: started Getting requirements to build editable: finished with status 'error' error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> [1 lines of output] Temp path for symlink to parent already exists deps [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 35, in <module> File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module> setup( File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup return distutils.core.setup(**attrs) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup return run_commands(dist) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands dist.run_commands() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands self.run_command(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command super().run_command(command) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run subprocess.check_call(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × python setup.py develop did not run successfully. │ exit code: 1 ╰─> [74 lines of output] /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. !! ******************************************************************************** Please consider removing the following classifiers in favor of a SPDX license expression: License :: OSI Approved :: Apache Software License See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. ******************************************************************************** !! self._finalize_license_expression() running develop /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``develop``. Instead, use standards-based tools like pip or uv. By 2025-Oct-31, you need to update your project and remove deprecated calls or your builds will no longer be supported. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Obtaining file:///__w/spark/spark/python Installing build dependencies: started Installing build dependencies: finished with status 'done' Checking if build backend supports build_editable: started Checking if build backend supports build_editable: finished with status 'done' Getting requirements to build editable: started Getting requirements to build editable: finished with status 'error' error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> [1 lines of output] Temp path for symlink to parent already exists deps [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 35, in <module> File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module> setup( File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup return distutils.core.setup(**attrs) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup return run_commands(dist) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands dist.run_commands() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands self.run_command(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command super().run_command(command) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run subprocess.check_call(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] 25h Cleaning up temporary directory - /tmp/tmp.zGt6QETKsu ``` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass GitHub Actions ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#50778 from LuciferYang/use-pep517. Authored-by: yangjie01 <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
…ip-tests` ### What changes were proposed in this pull request? This pr enable `--use-pep517` option in `dev/run-pip-tests` refer to: ``` 25l DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457 ``` ### Why are the changes needed? To restore Github Action test failed: - https://github.com/apache/spark/actions/runs/14778042174/job/41496865818 ``` Installing collected packages: py4j, pyspark 25l DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457 Running setup.py develop for pyspark ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] error: subprocess-exited-with-error × python setup.py develop did not run successfully. │ exit code: 1 ╰─> [74 lines of output] /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. !! ******************************************************************************** Please consider removing the following classifiers in favor of a SPDX license expression: License :: OSI Approved :: Apache Software License See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. ******************************************************************************** !! self._finalize_license_expression() running develop /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``develop``. Instead, use standards-based tools like pip or uv. By 2025-Oct-31, you need to update your project and remove deprecated calls or your builds will no longer be supported. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Obtaining file:///__w/spark/spark/python Installing build dependencies: started Installing build dependencies: finished with status 'done' Checking if build backend supports build_editable: started Checking if build backend supports build_editable: finished with status 'done' Getting requirements to build editable: started Getting requirements to build editable: finished with status 'error' error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> [1 lines of output] Temp path for symlink to parent already exists deps [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 35, in <module> File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module> setup( File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup return distutils.core.setup(**attrs) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup return run_commands(dist) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands dist.run_commands() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands self.run_command(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command super().run_command(command) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run subprocess.check_call(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × python setup.py develop did not run successfully. │ exit code: 1 ╰─> [74 lines of output] /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. !! ******************************************************************************** Please consider removing the following classifiers in favor of a SPDX license expression: License :: OSI Approved :: Apache Software License See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. ******************************************************************************** !! self._finalize_license_expression() running develop /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``develop``. Instead, use standards-based tools like pip or uv. By 2025-Oct-31, you need to update your project and remove deprecated calls or your builds will no longer be supported. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Obtaining file:///__w/spark/spark/python Installing build dependencies: started Installing build dependencies: finished with status 'done' Checking if build backend supports build_editable: started Checking if build backend supports build_editable: finished with status 'done' Getting requirements to build editable: started Getting requirements to build editable: finished with status 'error' error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> [1 lines of output] Temp path for symlink to parent already exists deps [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build editable did not run successfully. │ exit code: 255 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 35, in <module> File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module> setup( File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup return distutils.core.setup(**attrs) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup return run_commands(dist) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands dist.run_commands() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands self.run_command(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command super().run_command(command) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command cmd_obj.run() File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run subprocess.check_call(cmd) File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark] 25h Cleaning up temporary directory - /tmp/tmp.zGt6QETKsu ``` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass GitHub Actions ### Was this patch authored or co-authored using generative AI tooling? No Closes #50861 from LuciferYang/SPARK-51980-3.5. Authored-by: yangjie01 <[email protected]> Signed-off-by: yangjie01 <[email protected]>
When setuptools>58.2.0 setup.py installation is a deprecated method, a warning message will be prompted [1]: $ sudo make install .... running install /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Use the recommended solution [2] to replace setup.py with pypa/build and pip-install to completely solve the problem. It is worth noting that under Debian, you need to add the --break-system-packages parameter to pip-uninstall, as shown in the following prompt: Debian12:~$ sudo pip3 uninstall bcc error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.11/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. # Successfully uninstalled Debian12:~$ sudo pip3 uninstall bcc --break-system-packages Fix: iovisor#4586 [1] Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2] Cc: Jerome Marchand <[email protected]> Cc: Hengqi Chen <[email protected]> Signed-off-by: Rong Tao <[email protected]>
Downgrade setuptools to 80.2, where `command.easy_install` is still not dummy (pypa/setuptools#917). The `easy_install.install_wrapper_scripts()` is used by windows build: https://github.com/OSGeo/gdal/blob/e0292a9/swig/python/setup.py.in#L163
Drop the monkey patch for setuptools.command.easy_install.easy_install. This class is dummy after setuptools 80.3, see pypa/setuptools#917.
When setuptools>58.2.0 setup.py installation is a deprecated method, a warning message will be prompted [1]: $ sudo make install .... running install /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See pypa/setuptools#917 for details. ******************************************************************************** !! self.initialize_options() Use the recommended solution [2] to replace setup.py with pypa/build and pip-install to completely solve the problem. It is worth noting that under Debian, you need to add the --break-system-packages parameter to pip-uninstall, as shown in the following prompt: Debian12:~$ sudo pip3 uninstall bcc error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.11/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. # Successfully uninstalled Debian12:~$ sudo pip3 uninstall bcc --break-system-packages Fix: iovisor#4586 [1] Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2] Cc: Jerome Marchand <[email protected]> Cc: Hengqi Chen <[email protected]> Signed-off-by: Rong Tao <[email protected]>
There are many open issues with Setuptools that stem from the differences between easy_install and pip install. Now that #250 is complete and there exists a mechanism by which pip install can largely satisfy the use cases currently filled by easy_install. Perhaps at some point I will link to or enumerate the issues that would be addressed by such a replacement but not right now.
In the replacement, I imagine the following:
Invocations of
easy_install
whether from a command-line entry point or an implicit invocation as part of asetup.py install
will instead shell out to pip to perform the equivalent operation. Some options may become unavailable while others will be translated to their pip equivalent.One big open question is whether installs should still be done as eggs (using the deprecated
pip install --egg
) or if it should rely on pip's use of wheel. I'm leaning toward the latter.There are probably other open issues that I haven't yet considered.
The text was updated successfully, but these errors were encountered: