Skip to content

Remove easy_install #917

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jaraco opened this issue Jan 9, 2017 · 36 comments · Fixed by #4963
Closed

Remove easy_install #917

jaraco opened this issue Jan 9, 2017 · 36 comments · Fixed by #4963

Comments

@jaraco
Copy link
Member

jaraco commented Jan 9, 2017

There are many open issues with Setuptools that stem from the differences between easy_install and pip install. Now that #250 is complete and there exists a mechanism by which pip install can largely satisfy the use cases currently filled by easy_install. Perhaps at some point I will link to or enumerate the issues that would be addressed by such a replacement but not right now.

In the replacement, I imagine the following:

Invocations of easy_install whether from a command-line entry point or an implicit invocation as part of a setup.py install will instead shell out to pip to perform the equivalent operation. Some options may become unavailable while others will be translated to their pip equivalent.

One big open question is whether installs should still be done as eggs (using the deprecated pip install --egg) or if it should rely on pip's use of wheel. I'm leaning toward the latter.

There are probably other open issues that I haven't yet considered.

@graingert
Copy link
Contributor

probably worth just deprecating easy_install and adding a header from the docs that points to pip

@jaraco
Copy link
Member Author

jaraco commented Jan 13, 2017

It's not so simple. Easy-install is used by tests requires and setup requires, which has not replacement. Perhaps it makes sense to only replace those functions.

@ghost
Copy link

ghost commented Jan 21, 2017

If I'm not mistaken, running pip install while pip is already running can cause problems. The only way that this is going to work is if pip understands setup_requires or setuptools does some sort of runtime hack/monkeypatch on pip.

@ghost
Copy link

ghost commented Jan 21, 2017

See pypa/pip#2745 and pypa/pip#2361.

@graingert
Copy link
Contributor

Tests requires should be deprecated in favour of tox

@jaraco
Copy link
Member Author

jaraco commented Jan 22, 2017

If I'm not mistaken, running pip install while pip is already running can cause problems. The only way that this is going to work is if pip understands setup_requires or setuptools does some sort of runtime hack/monkeypatch on pip.

I don't think these issues are relevant. In the case where pip is invoking setup.py to build a package from source and that package has setup_requires dependencies, those dependencies would still be "installed", but to a temporary location (perhaps as eggs to ./.eggs as they are now), meaning they would still be isolated from the target environment via subprocesses and temporary directories, so I would expect not to encounter those issues.

@dstufft
Copy link
Member

dstufft commented Jan 23, 2017

FWIW longer term PEP 518 is the solution to setup_requires.

@graingert
Copy link
Contributor

maybe for now, just a docs header saying, "hey unless you know what you're doing: don't use easy_install, you probably want pip instead"

@marscher
Copy link

marscher commented Mar 15, 2017

pypa/pip#1820 (comment) gives a reference on why recursive pip calls are not a good solution. I think this should be handled by setuptools, because setup_requires is a setuptools feature, which pip is currently unaware of.

PEP 518 is only a specification of the actual build time dependencies, but not an attempt to solve the installation problem:

This PEP covers step 2. It is fully expected that a future PEP will cover step 3, including how to have the build system dynamically specify more dependencies that the build system requires to perform its job. The purpose of this PEP though, is to specify the minimal set of requirements for the build system to simply begin execution.

So this is still unsolved, isn't it?

People tend not to use setup_requires, because easy_install does not use wheel and people have to compile eg. hard to build projects like scipy. So one should definitely prefer a wheel supporting mechanism.

@methane
Copy link
Contributor

methane commented Apr 23, 2017

How about remove easy_install command but keep python -m easy_install for setup_requires for a while?

@ghost
Copy link

ghost commented May 6, 2017

@jaraco what do you think about autogenerating pyproject.toml for source distributions?

@jaraco
Copy link
Member Author

jaraco commented Dec 13, 2018

Sorry for the late follow-up.

@jaraco what do you think about autogenerating pyproject.toml for source distributions?

I think I'd rather users be incentivized to provide the declaration in the preferred location. I'd support a warning, though.

@jaraco
Copy link
Member Author

jaraco commented Dec 13, 2018

This issue continues to be an issue if only because not every invocation of setup.py goes through a PEP 518 runner. For example, tox invokes setup.py --name to get the name of a package - that triggers setup_requires/easy_install. Package managers run setup.py --description or setup.py --version to get metadata. Other distutils commands still exist and are certainly in use in the wild. Perhaps we can eliminate the reliance on setup.py invocations entirely, though that prospect seems daunting.

@fungi

This comment has been minimized.

@fungi

This comment has been minimized.

@pganssle

This comment has been minimized.

Rtoax added a commit to Rtoax/bcc that referenced this issue Apr 15, 2025
    running install
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()

Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html
Signed-off-by: Rong Tao <[email protected]>
Rtoax added a commit to Rtoax/bcc that referenced this issue Apr 15, 2025
    running install
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()

Fix: iovisor#4586
Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html
Signed-off-by: Rong Tao <[email protected]>
Rtoax added a commit to Rtoax/bcc that referenced this issue Apr 15, 2025
    running install
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()

Fix: iovisor#4586
Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html
Signed-off-by: Rong Tao <[email protected]>
Rtoax added a commit to Rtoax/bcc that referenced this issue Apr 15, 2025
    running install
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()

Fix: iovisor#4586
Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html
Signed-off-by: Rong Tao <[email protected]>
Rtoax added a commit to Rtoax/bcc that referenced this issue Apr 15, 2025
When setuptools>58.2.0 setup.py installation is a deprecated method, a warning
message will be prompted [1]:

    $ sudo make install
    ....
    running install
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()

Use the recommended solution [2] to replace setup.py with pypa/build and
pip-install to completely solve the problem. It is worth noting that under
Debian, you need to add the --break-system-packages parameter to pip-uninstall,
as shown in the following prompt:

    Debian12:~$ sudo pip3 uninstall bcc
    error: externally-managed-environment

    × This environment is externally managed
    ╰─> To install Python packages system-wide, try apt install
        python3-xyz, where xyz is the package you are trying to
        install.

        If you wish to install a non-Debian-packaged Python package,
        create a virtual environment using python3 -m venv path/to/venv.
        Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
        sure you have python3-full installed.

        If you wish to install a non-Debian packaged Python application,
        it may be easiest to use pipx install xyz, which will manage a
        virtual environment for you. Make sure you have pipx installed.

        See /usr/share/doc/python3.11/README.venv for more information.

    note: If you believe this is a mistake, please contact your Python
    installation or OS distribution provider. You can override this, at the
    risk of breaking your Python installation or OS, by passing
    --break-system-packages.
    hint: See PEP 668 for the detailed specification.

    # Successfully uninstalled
    Debian12:~$ sudo pip3 uninstall bcc --break-system-packages

Fix: iovisor#4586 [1]
Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2]
Cc: Jerome Marchand <[email protected]>
Cc: Hengqi Chen <[email protected]>
Signed-off-by: Rong Tao <[email protected]>
@jaraco jaraco self-assigned this Apr 20, 2025
@jaraco jaraco changed the title Replace easy_install with pip install Remove easy_install Apr 20, 2025
@jaraco jaraco added technical debt standardization and removed Needs Discussion Issues where the implementation still needs to be discussed. labels Apr 20, 2025
Rtoax added a commit to Rtoax/bcc that referenced this issue Apr 27, 2025
When setuptools>58.2.0 setup.py installation is a deprecated method, a warning
message will be prompted [1]:

    $ sudo make install
    ....
    running install
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()

Use the recommended solution [2] to replace setup.py with pypa/build and
pip-install to completely solve the problem. It is worth noting that under
Debian, you need to add the --break-system-packages parameter to pip-uninstall,
as shown in the following prompt:

    Debian12:~$ sudo pip3 uninstall bcc
    error: externally-managed-environment

    × This environment is externally managed
    ╰─> To install Python packages system-wide, try apt install
        python3-xyz, where xyz is the package you are trying to
        install.

        If you wish to install a non-Debian-packaged Python package,
        create a virtual environment using python3 -m venv path/to/venv.
        Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
        sure you have python3-full installed.

        If you wish to install a non-Debian packaged Python application,
        it may be easiest to use pipx install xyz, which will manage a
        virtual environment for you. Make sure you have pipx installed.

        See /usr/share/doc/python3.11/README.venv for more information.

    note: If you believe this is a mistake, please contact your Python
    installation or OS distribution provider. You can override this, at the
    risk of breaking your Python installation or OS, by passing
    --break-system-packages.
    hint: See PEP 668 for the detailed specification.

    # Successfully uninstalled
    Debian12:~$ sudo pip3 uninstall bcc --break-system-packages

Fix: iovisor#4586 [1]
Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2]
Cc: Jerome Marchand <[email protected]>
Cc: Hengqi Chen <[email protected]>
Signed-off-by: Rong Tao <[email protected]>
Rtoax added a commit to Rtoax/bcc that referenced this issue May 1, 2025
When setuptools>58.2.0 setup.py installation is a deprecated method, a warning
message will be prompted [1]:

    $ sudo make install
    ....
    running install
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()

Use the recommended solution [2] to replace setup.py with pypa/build and
pip-install to completely solve the problem. It is worth noting that under
Debian, you need to add the --break-system-packages parameter to pip-uninstall,
as shown in the following prompt:

    Debian12:~$ sudo pip3 uninstall bcc
    error: externally-managed-environment

    × This environment is externally managed
    ╰─> To install Python packages system-wide, try apt install
        python3-xyz, where xyz is the package you are trying to
        install.

        If you wish to install a non-Debian-packaged Python package,
        create a virtual environment using python3 -m venv path/to/venv.
        Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
        sure you have python3-full installed.

        If you wish to install a non-Debian packaged Python application,
        it may be easiest to use pipx install xyz, which will manage a
        virtual environment for you. Make sure you have pipx installed.

        See /usr/share/doc/python3.11/README.venv for more information.

    note: If you believe this is a mistake, please contact your Python
    installation or OS distribution provider. You can override this, at the
    risk of breaking your Python installation or OS, by passing
    --break-system-packages.
    hint: See PEP 668 for the detailed specification.

    # Successfully uninstalled
    Debian12:~$ sudo pip3 uninstall bcc --break-system-packages

Fix: iovisor#4586 [1]
Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2]
Cc: Jerome Marchand <[email protected]>
Cc: Hengqi Chen <[email protected]>
Signed-off-by: Rong Tao <[email protected]>
dongjoon-hyun pushed a commit to apache/spark that referenced this issue May 2, 2025
…sts`

### What changes were proposed in this pull request?
This pr enable `--use-pep517` option in `dev/run-pip-tests` refer to:

```
25l  DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457
```

### Why are the changes needed?
To restore Github Action test failed:

- https://github.com/apache/spark/actions/runs/14778042174/job/41496865818

```
Installing collected packages: py4j, pyspark
25l  DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457

  Running setup.py develop for pyspark

   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]    error: subprocess-exited-with-error

    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [74 lines of output]
        /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
        !!

                ********************************************************************************
                Please consider removing the following classifiers in favor of a SPDX license expression:

                License :: OSI Approved :: Apache Software License

                See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
                ********************************************************************************

        !!
          self._finalize_license_expression()
        running develop
        /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated.
        !!

                ********************************************************************************
                Please avoid running ``setup.py`` and ``develop``.
                Instead, use standards-based tools like pip or uv.

                By 2025-Oct-31, you need to update your project and remove deprecated calls
                or your builds will no longer be supported.

                See pypa/setuptools#917 for details.
                ********************************************************************************

        !!
          self.initialize_options()
        Obtaining file:///__w/spark/spark/python
          Installing build dependencies: started
          Installing build dependencies: finished with status 'done'
          Checking if build backend supports build_editable: started
          Checking if build backend supports build_editable: finished with status 'done'
          Getting requirements to build editable: started
          Getting requirements to build editable: finished with status 'error'
          error: subprocess-exited-with-error

          × Getting requirements to build editable did not run successfully.
          │ exit code: 255
          ╰─> [1 lines of output]
              Temp path for symlink to parent already exists deps
              [end of output]

          note: This error originates from a subprocess, and is likely not a problem with pip.
        error: subprocess-exited-with-error

        × Getting requirements to build editable did not run successfully.
        │ exit code: 255
        ╰─> See above for output.

        note: This error originates from a subprocess, and is likely not a problem with pip.
        Traceback (most recent call last):
          File "<string>", line 2, in <module>
          File "<pip-setuptools-caller>", line 35, in <module>
          File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module>
            setup(
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
            return distutils.core.setup(**attrs)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup
            return run_commands(dist)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
            dist.run_commands()
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
            self.run_command(cmd)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command
            super().run_command(command)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
            cmd_obj.run()
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run
            subprocess.check_call(cmd)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call
            raise CalledProcessError(retcode, cmd)
        subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1.
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [74 lines of output]
    /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
    !!

            ********************************************************************************
            Please consider removing the following classifiers in favor of a SPDX license expression:

            License :: OSI Approved :: Apache Software License

            See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
            ********************************************************************************

    !!
      self._finalize_license_expression()
    running develop
    /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``develop``.
            Instead, use standards-based tools like pip or uv.

            By 2025-Oct-31, you need to update your project and remove deprecated calls
            or your builds will no longer be supported.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()
    Obtaining file:///__w/spark/spark/python
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Checking if build backend supports build_editable: started
      Checking if build backend supports build_editable: finished with status 'done'
      Getting requirements to build editable: started
      Getting requirements to build editable: finished with status 'error'
      error: subprocess-exited-with-error

      × Getting requirements to build editable did not run successfully.
      │ exit code: 255
      ╰─> [1 lines of output]
          Temp path for symlink to parent already exists deps
          [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: subprocess-exited-with-error

    × Getting requirements to build editable did not run successfully.
    │ exit code: 255
    ╰─> See above for output.

    note: This error originates from a subprocess, and is likely not a problem with pip.
    Traceback (most recent call last):
      File "<string>", line 2, in <module>
      File "<pip-setuptools-caller>", line 35, in <module>
      File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module>
        setup(
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
        return distutils.core.setup(**attrs)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup
        return run_commands(dist)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
        dist.run_commands()
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
        self.run_command(cmd)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command
        super().run_command(command)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
        cmd_obj.run()
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run
        subprocess.check_call(cmd)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1.
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
25h
Cleaning up temporary directory - /tmp/tmp.zGt6QETKsu
```

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #50778 from LuciferYang/use-pep517.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun pushed a commit to apache/spark that referenced this issue May 2, 2025
…sts`

### What changes were proposed in this pull request?
This pr enable `--use-pep517` option in `dev/run-pip-tests` refer to:

```
25l  DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457
```

### Why are the changes needed?
To restore Github Action test failed:

- https://github.com/apache/spark/actions/runs/14778042174/job/41496865818

```
Installing collected packages: py4j, pyspark
25l  DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457

  Running setup.py develop for pyspark

   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]    error: subprocess-exited-with-error

    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [74 lines of output]
        /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
        !!

                ********************************************************************************
                Please consider removing the following classifiers in favor of a SPDX license expression:

                License :: OSI Approved :: Apache Software License

                See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
                ********************************************************************************

        !!
          self._finalize_license_expression()
        running develop
        /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated.
        !!

                ********************************************************************************
                Please avoid running ``setup.py`` and ``develop``.
                Instead, use standards-based tools like pip or uv.

                By 2025-Oct-31, you need to update your project and remove deprecated calls
                or your builds will no longer be supported.

                See pypa/setuptools#917 for details.
                ********************************************************************************

        !!
          self.initialize_options()
        Obtaining file:///__w/spark/spark/python
          Installing build dependencies: started
          Installing build dependencies: finished with status 'done'
          Checking if build backend supports build_editable: started
          Checking if build backend supports build_editable: finished with status 'done'
          Getting requirements to build editable: started
          Getting requirements to build editable: finished with status 'error'
          error: subprocess-exited-with-error

          × Getting requirements to build editable did not run successfully.
          │ exit code: 255
          ╰─> [1 lines of output]
              Temp path for symlink to parent already exists deps
              [end of output]

          note: This error originates from a subprocess, and is likely not a problem with pip.
        error: subprocess-exited-with-error

        × Getting requirements to build editable did not run successfully.
        │ exit code: 255
        ╰─> See above for output.

        note: This error originates from a subprocess, and is likely not a problem with pip.
        Traceback (most recent call last):
          File "<string>", line 2, in <module>
          File "<pip-setuptools-caller>", line 35, in <module>
          File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module>
            setup(
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
            return distutils.core.setup(**attrs)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup
            return run_commands(dist)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
            dist.run_commands()
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
            self.run_command(cmd)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command
            super().run_command(command)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
            cmd_obj.run()
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run
            subprocess.check_call(cmd)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call
            raise CalledProcessError(retcode, cmd)
        subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1.
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [74 lines of output]
    /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
    !!

            ********************************************************************************
            Please consider removing the following classifiers in favor of a SPDX license expression:

            License :: OSI Approved :: Apache Software License

            See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
            ********************************************************************************

    !!
      self._finalize_license_expression()
    running develop
    /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``develop``.
            Instead, use standards-based tools like pip or uv.

            By 2025-Oct-31, you need to update your project and remove deprecated calls
            or your builds will no longer be supported.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()
    Obtaining file:///__w/spark/spark/python
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Checking if build backend supports build_editable: started
      Checking if build backend supports build_editable: finished with status 'done'
      Getting requirements to build editable: started
      Getting requirements to build editable: finished with status 'error'
      error: subprocess-exited-with-error

      × Getting requirements to build editable did not run successfully.
      │ exit code: 255
      ╰─> [1 lines of output]
          Temp path for symlink to parent already exists deps
          [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: subprocess-exited-with-error

    × Getting requirements to build editable did not run successfully.
    │ exit code: 255
    ╰─> See above for output.

    note: This error originates from a subprocess, and is likely not a problem with pip.
    Traceback (most recent call last):
      File "<string>", line 2, in <module>
      File "<pip-setuptools-caller>", line 35, in <module>
      File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module>
        setup(
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
        return distutils.core.setup(**attrs)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup
        return run_commands(dist)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
        dist.run_commands()
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
        self.run_command(cmd)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command
        super().run_command(command)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
        cmd_obj.run()
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run
        subprocess.check_call(cmd)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1.
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
25h
Cleaning up temporary directory - /tmp/tmp.zGt6QETKsu
```

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #50778 from LuciferYang/use-pep517.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 31f7278)
Signed-off-by: Dongjoon Hyun <[email protected]>
ericm-db pushed a commit to ericm-db/spark that referenced this issue May 5, 2025
…sts`

### What changes were proposed in this pull request?
This pr enable `--use-pep517` option in `dev/run-pip-tests` refer to:

```
25l  DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457
```

### Why are the changes needed?
To restore Github Action test failed:

- https://github.com/apache/spark/actions/runs/14778042174/job/41496865818

```
Installing collected packages: py4j, pyspark
25l  DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457

  Running setup.py develop for pyspark

   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]    error: subprocess-exited-with-error

    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [74 lines of output]
        /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
        !!

                ********************************************************************************
                Please consider removing the following classifiers in favor of a SPDX license expression:

                License :: OSI Approved :: Apache Software License

                See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
                ********************************************************************************

        !!
          self._finalize_license_expression()
        running develop
        /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated.
        !!

                ********************************************************************************
                Please avoid running ``setup.py`` and ``develop``.
                Instead, use standards-based tools like pip or uv.

                By 2025-Oct-31, you need to update your project and remove deprecated calls
                or your builds will no longer be supported.

                See pypa/setuptools#917 for details.
                ********************************************************************************

        !!
          self.initialize_options()
        Obtaining file:///__w/spark/spark/python
          Installing build dependencies: started
          Installing build dependencies: finished with status 'done'
          Checking if build backend supports build_editable: started
          Checking if build backend supports build_editable: finished with status 'done'
          Getting requirements to build editable: started
          Getting requirements to build editable: finished with status 'error'
          error: subprocess-exited-with-error

          × Getting requirements to build editable did not run successfully.
          │ exit code: 255
          ╰─> [1 lines of output]
              Temp path for symlink to parent already exists deps
              [end of output]

          note: This error originates from a subprocess, and is likely not a problem with pip.
        error: subprocess-exited-with-error

        × Getting requirements to build editable did not run successfully.
        │ exit code: 255
        ╰─> See above for output.

        note: This error originates from a subprocess, and is likely not a problem with pip.
        Traceback (most recent call last):
          File "<string>", line 2, in <module>
          File "<pip-setuptools-caller>", line 35, in <module>
          File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module>
            setup(
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
            return distutils.core.setup(**attrs)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup
            return run_commands(dist)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
            dist.run_commands()
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
            self.run_command(cmd)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command
            super().run_command(command)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
            cmd_obj.run()
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run
            subprocess.check_call(cmd)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call
            raise CalledProcessError(retcode, cmd)
        subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1.
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [74 lines of output]
    /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
    !!

            ********************************************************************************
            Please consider removing the following classifiers in favor of a SPDX license expression:

            License :: OSI Approved :: Apache Software License

            See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
            ********************************************************************************

    !!
      self._finalize_license_expression()
    running develop
    /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``develop``.
            Instead, use standards-based tools like pip or uv.

            By 2025-Oct-31, you need to update your project and remove deprecated calls
            or your builds will no longer be supported.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()
    Obtaining file:///__w/spark/spark/python
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Checking if build backend supports build_editable: started
      Checking if build backend supports build_editable: finished with status 'done'
      Getting requirements to build editable: started
      Getting requirements to build editable: finished with status 'error'
      error: subprocess-exited-with-error

      × Getting requirements to build editable did not run successfully.
      │ exit code: 255
      ╰─> [1 lines of output]
          Temp path for symlink to parent already exists deps
          [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: subprocess-exited-with-error

    × Getting requirements to build editable did not run successfully.
    │ exit code: 255
    ╰─> See above for output.

    note: This error originates from a subprocess, and is likely not a problem with pip.
    Traceback (most recent call last):
      File "<string>", line 2, in <module>
      File "<pip-setuptools-caller>", line 35, in <module>
      File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module>
        setup(
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
        return distutils.core.setup(**attrs)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup
        return run_commands(dist)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
        dist.run_commands()
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
        self.run_command(cmd)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command
        super().run_command(command)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
        cmd_obj.run()
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run
        subprocess.check_call(cmd)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1.
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
25h
Cleaning up temporary directory - /tmp/tmp.zGt6QETKsu
```

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#50778 from LuciferYang/use-pep517.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
Rtoax added a commit to Rtoax/bcc that referenced this issue May 12, 2025
When setuptools>58.2.0 setup.py installation is a deprecated method, a warning
message will be prompted [1]:

    $ sudo make install
    ....
    running install
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()

Use the recommended solution [2] to replace setup.py with pypa/build and
pip-install to completely solve the problem. It is worth noting that under
Debian, you need to add the --break-system-packages parameter to pip-uninstall,
as shown in the following prompt:

    Debian12:~$ sudo pip3 uninstall bcc
    error: externally-managed-environment

    × This environment is externally managed
    ╰─> To install Python packages system-wide, try apt install
        python3-xyz, where xyz is the package you are trying to
        install.

        If you wish to install a non-Debian-packaged Python package,
        create a virtual environment using python3 -m venv path/to/venv.
        Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
        sure you have python3-full installed.

        If you wish to install a non-Debian packaged Python application,
        it may be easiest to use pipx install xyz, which will manage a
        virtual environment for you. Make sure you have pipx installed.

        See /usr/share/doc/python3.11/README.venv for more information.

    note: If you believe this is a mistake, please contact your Python
    installation or OS distribution provider. You can override this, at the
    risk of breaking your Python installation or OS, by passing
    --break-system-packages.
    hint: See PEP 668 for the detailed specification.

    # Successfully uninstalled
    Debian12:~$ sudo pip3 uninstall bcc --break-system-packages

Fix: iovisor#4586 [1]
Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2]
Cc: Jerome Marchand <[email protected]>
Cc: Hengqi Chen <[email protected]>
Signed-off-by: Rong Tao <[email protected]>
LuciferYang added a commit to LuciferYang/spark that referenced this issue May 12, 2025
…sts`

### What changes were proposed in this pull request?
This pr enable `--use-pep517` option in `dev/run-pip-tests` refer to:

```
25l  DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457
```

### Why are the changes needed?
To restore Github Action test failed:

- https://github.com/apache/spark/actions/runs/14778042174/job/41496865818

```
Installing collected packages: py4j, pyspark
25l  DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457

  Running setup.py develop for pyspark

   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]    error: subprocess-exited-with-error

    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [74 lines of output]
        /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
        !!

                ********************************************************************************
                Please consider removing the following classifiers in favor of a SPDX license expression:

                License :: OSI Approved :: Apache Software License

                See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
                ********************************************************************************

        !!
          self._finalize_license_expression()
        running develop
        /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated.
        !!

                ********************************************************************************
                Please avoid running ``setup.py`` and ``develop``.
                Instead, use standards-based tools like pip or uv.

                By 2025-Oct-31, you need to update your project and remove deprecated calls
                or your builds will no longer be supported.

                See pypa/setuptools#917 for details.
                ********************************************************************************

        !!
          self.initialize_options()
        Obtaining file:///__w/spark/spark/python
          Installing build dependencies: started
          Installing build dependencies: finished with status 'done'
          Checking if build backend supports build_editable: started
          Checking if build backend supports build_editable: finished with status 'done'
          Getting requirements to build editable: started
          Getting requirements to build editable: finished with status 'error'
          error: subprocess-exited-with-error

          × Getting requirements to build editable did not run successfully.
          │ exit code: 255
          ╰─> [1 lines of output]
              Temp path for symlink to parent already exists deps
              [end of output]

          note: This error originates from a subprocess, and is likely not a problem with pip.
        error: subprocess-exited-with-error

        × Getting requirements to build editable did not run successfully.
        │ exit code: 255
        ╰─> See above for output.

        note: This error originates from a subprocess, and is likely not a problem with pip.
        Traceback (most recent call last):
          File "<string>", line 2, in <module>
          File "<pip-setuptools-caller>", line 35, in <module>
          File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module>
            setup(
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
            return distutils.core.setup(**attrs)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup
            return run_commands(dist)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
            dist.run_commands()
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
            self.run_command(cmd)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command
            super().run_command(command)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
            cmd_obj.run()
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run
            subprocess.check_call(cmd)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call
            raise CalledProcessError(retcode, cmd)
        subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1.
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [74 lines of output]
    /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
    !!

            ********************************************************************************
            Please consider removing the following classifiers in favor of a SPDX license expression:

            License :: OSI Approved :: Apache Software License

            See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
            ********************************************************************************

    !!
      self._finalize_license_expression()
    running develop
    /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``develop``.
            Instead, use standards-based tools like pip or uv.

            By 2025-Oct-31, you need to update your project and remove deprecated calls
            or your builds will no longer be supported.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()
    Obtaining file:///__w/spark/spark/python
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Checking if build backend supports build_editable: started
      Checking if build backend supports build_editable: finished with status 'done'
      Getting requirements to build editable: started
      Getting requirements to build editable: finished with status 'error'
      error: subprocess-exited-with-error

      × Getting requirements to build editable did not run successfully.
      │ exit code: 255
      ╰─> [1 lines of output]
          Temp path for symlink to parent already exists deps
          [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: subprocess-exited-with-error

    × Getting requirements to build editable did not run successfully.
    │ exit code: 255
    ╰─> See above for output.

    note: This error originates from a subprocess, and is likely not a problem with pip.
    Traceback (most recent call last):
      File "<string>", line 2, in <module>
      File "<pip-setuptools-caller>", line 35, in <module>
      File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module>
        setup(
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
        return distutils.core.setup(**attrs)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup
        return run_commands(dist)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
        dist.run_commands()
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
        self.run_command(cmd)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command
        super().run_command(command)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
        cmd_obj.run()
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run
        subprocess.check_call(cmd)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1.
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
25h
Cleaning up temporary directory - /tmp/tmp.zGt6QETKsu
```

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#50778 from LuciferYang/use-pep517.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
LuciferYang added a commit to apache/spark that referenced this issue May 12, 2025
…ip-tests`

### What changes were proposed in this pull request?
This pr enable `--use-pep517` option in `dev/run-pip-tests` refer to:

```
25l  DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457
```

### Why are the changes needed?
To restore Github Action test failed:

- https://github.com/apache/spark/actions/runs/14778042174/job/41496865818

```
Installing collected packages: py4j, pyspark
25l  DEPRECATION: Legacy editable install of pyspark==4.1.0.dev0 from file:///__w/spark/spark/python/packaging/classic (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at pypa/pip#11457

  Running setup.py develop for pyspark

   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]    error: subprocess-exited-with-error

    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [74 lines of output]
        /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
        !!

                ********************************************************************************
                Please consider removing the following classifiers in favor of a SPDX license expression:

                License :: OSI Approved :: Apache Software License

                See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
                ********************************************************************************

        !!
          self._finalize_license_expression()
        running develop
        /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated.
        !!

                ********************************************************************************
                Please avoid running ``setup.py`` and ``develop``.
                Instead, use standards-based tools like pip or uv.

                By 2025-Oct-31, you need to update your project and remove deprecated calls
                or your builds will no longer be supported.

                See pypa/setuptools#917 for details.
                ********************************************************************************

        !!
          self.initialize_options()
        Obtaining file:///__w/spark/spark/python
          Installing build dependencies: started
          Installing build dependencies: finished with status 'done'
          Checking if build backend supports build_editable: started
          Checking if build backend supports build_editable: finished with status 'done'
          Getting requirements to build editable: started
          Getting requirements to build editable: finished with status 'error'
          error: subprocess-exited-with-error

          × Getting requirements to build editable did not run successfully.
          │ exit code: 255
          ╰─> [1 lines of output]
              Temp path for symlink to parent already exists deps
              [end of output]

          note: This error originates from a subprocess, and is likely not a problem with pip.
        error: subprocess-exited-with-error

        × Getting requirements to build editable did not run successfully.
        │ exit code: 255
        ╰─> See above for output.

        note: This error originates from a subprocess, and is likely not a problem with pip.
        Traceback (most recent call last):
          File "<string>", line 2, in <module>
          File "<pip-setuptools-caller>", line 35, in <module>
          File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module>
            setup(
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
            return distutils.core.setup(**attrs)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup
            return run_commands(dist)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
            dist.run_commands()
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
            self.run_command(cmd)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command
            super().run_command(command)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
            cmd_obj.run()
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run
            subprocess.check_call(cmd)
          File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call
            raise CalledProcessError(retcode, cmd)
        subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1.
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [74 lines of output]
    /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
    !!

            ********************************************************************************
            Please consider removing the following classifiers in favor of a SPDX license expression:

            License :: OSI Approved :: Apache Software License

            See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
            ********************************************************************************

    !!
      self._finalize_license_expression()
    running develop
    /tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``develop``.
            Instead, use standards-based tools like pip or uv.

            By 2025-Oct-31, you need to update your project and remove deprecated calls
            or your builds will no longer be supported.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()
    Obtaining file:///__w/spark/spark/python
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Checking if build backend supports build_editable: started
      Checking if build backend supports build_editable: finished with status 'done'
      Getting requirements to build editable: started
      Getting requirements to build editable: finished with status 'error'
      error: subprocess-exited-with-error

      × Getting requirements to build editable did not run successfully.
      │ exit code: 255
      ╰─> [1 lines of output]
          Temp path for symlink to parent already exists deps
          [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: subprocess-exited-with-error

    × Getting requirements to build editable did not run successfully.
    │ exit code: 255
    ╰─> See above for output.

    note: This error originates from a subprocess, and is likely not a problem with pip.
    Traceback (most recent call last):
      File "<string>", line 2, in <module>
      File "<pip-setuptools-caller>", line 35, in <module>
      File "/__w/spark/spark/python/packaging/classic/setup.py", line 254, in <module>
        setup(
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 117, in setup
        return distutils.core.setup(**attrs)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 186, in setup
        return run_commands(dist)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
        dist.run_commands()
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
        self.run_command(cmd)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/dist.py", line 1104, in run_command
        super().run_command(command)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
        cmd_obj.run()
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/site-packages/setuptools/command/develop.py", line 39, in run
        subprocess.check_call(cmd)
      File "/tmp/tmp.zGt6QETKsu/3.9/lib/python3.9/subprocess.py", line 373, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/tmp/tmp.zGt6QETKsu/3.9/bin/python3.9', '-m', 'pip', 'install', '-e', '.', '--use-pep517', '--no-deps']' returned non-zero exit status 1.
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
   ━━━━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━ 1/2 [pyspark]
25h
Cleaning up temporary directory - /tmp/tmp.zGt6QETKsu
```

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
- Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #50861 from LuciferYang/SPARK-51980-3.5.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: yangjie01 <[email protected]>
Rtoax added a commit to Rtoax/bcc that referenced this issue May 15, 2025
When setuptools>58.2.0 setup.py installation is a deprecated method, a warning
message will be prompted [1]:

    $ sudo make install
    ....
    running install
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()

Use the recommended solution [2] to replace setup.py with pypa/build and
pip-install to completely solve the problem. It is worth noting that under
Debian, you need to add the --break-system-packages parameter to pip-uninstall,
as shown in the following prompt:

    Debian12:~$ sudo pip3 uninstall bcc
    error: externally-managed-environment

    × This environment is externally managed
    ╰─> To install Python packages system-wide, try apt install
        python3-xyz, where xyz is the package you are trying to
        install.

        If you wish to install a non-Debian-packaged Python package,
        create a virtual environment using python3 -m venv path/to/venv.
        Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
        sure you have python3-full installed.

        If you wish to install a non-Debian packaged Python application,
        it may be easiest to use pipx install xyz, which will manage a
        virtual environment for you. Make sure you have pipx installed.

        See /usr/share/doc/python3.11/README.venv for more information.

    note: If you believe this is a mistake, please contact your Python
    installation or OS distribution provider. You can override this, at the
    risk of breaking your Python installation or OS, by passing
    --break-system-packages.
    hint: See PEP 668 for the detailed specification.

    # Successfully uninstalled
    Debian12:~$ sudo pip3 uninstall bcc --break-system-packages

Fix: iovisor#4586 [1]
Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2]
Cc: Jerome Marchand <[email protected]>
Cc: Hengqi Chen <[email protected]>
Signed-off-by: Rong Tao <[email protected]>
trundev added a commit to trundev/setup-gdal that referenced this issue May 17, 2025
Downgrade setuptools to 80.2, where `command.easy_install` is still not
dummy (pypa/setuptools#917).
The `easy_install.install_wrapper_scripts()` is used by windows build:
https://github.com/OSGeo/gdal/blob/e0292a9/swig/python/setup.py.in#L163
trundev added a commit to trundev/gdal that referenced this issue May 18, 2025
Drop the monkey patch for setuptools.command.easy_install.easy_install.
This class is dummy after setuptools 80.3, see pypa/setuptools#917.
Rtoax added a commit to Rtoax/bcc that referenced this issue May 21, 2025
When setuptools>58.2.0 setup.py installation is a deprecated method, a warning
message will be prompted [1]:

    $ sudo make install
    ....
    running install
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    /usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
    EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer or other
            standards-based tools.

            See pypa/setuptools#917 for details.
            ********************************************************************************

    !!
      self.initialize_options()

Use the recommended solution [2] to replace setup.py with pypa/build and
pip-install to completely solve the problem. It is worth noting that under
Debian, you need to add the --break-system-packages parameter to pip-uninstall,
as shown in the following prompt:

    Debian12:~$ sudo pip3 uninstall bcc
    error: externally-managed-environment

    × This environment is externally managed
    ╰─> To install Python packages system-wide, try apt install
        python3-xyz, where xyz is the package you are trying to
        install.

        If you wish to install a non-Debian-packaged Python package,
        create a virtual environment using python3 -m venv path/to/venv.
        Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
        sure you have python3-full installed.

        If you wish to install a non-Debian packaged Python application,
        it may be easiest to use pipx install xyz, which will manage a
        virtual environment for you. Make sure you have pipx installed.

        See /usr/share/doc/python3.11/README.venv for more information.

    note: If you believe this is a mistake, please contact your Python
    installation or OS distribution provider. You can override this, at the
    risk of breaking your Python installation or OS, by passing
    --break-system-packages.
    hint: See PEP 668 for the detailed specification.

    # Successfully uninstalled
    Debian12:~$ sudo pip3 uninstall bcc --break-system-packages

Fix: iovisor#4586 [1]
Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2]
Cc: Jerome Marchand <[email protected]>
Cc: Hengqi Chen <[email protected]>
Signed-off-by: Rong Tao <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.