Issues with pysha3 in windows

[ramarivera]

There seems to be some kind of incompatibility or missing dependencies in order to compile pysha3 on Windows + Python 3.11.

I didn't dive much deeper than this: here.

As I understand, the thing is like this:

web3-input-decoder directly consumes:
  - eth-abi
  - eth-utils:
        but then eth-utils requires: 
            -  eth-hash, which in turn has optional backends:
            -     # optional backends:
                   'pycryptodome': [
                        "pycryptodome>=3.6.6,<4",
                   ],
                    'pysha3': [
                          "pysha3>=1.0.0,<2.0.0",
                   ],

So, you are installing as a mandatory dependency pysha3, to force that backend.

If how I understand things is correct, would you be open to a PR that kinda lifts the backend choice from eth-hash to the consumers of web3-input-decoder?

Full logs when trying to install your package latest version

$ poetry install -vv
Using virtualenv: C:\Users\ramar\AppData\Local\pypoetry\Cache\virtualenvs\some-project-mBcAJQ45-py3.11
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 16 installs, 0 updates, 0 removals, 107 skipped

  • Installing pysha3 (1.0.2)

  Stack trace:

  2  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\env.py:1472 in _run
       output = subprocess.check_output(

  1  ~\.pyenv\pyenv-win\versions\3.10.7\lib\subprocess.py:420 in check_output
       return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,

  CalledProcessError

  Command 'C:\Users\ramar\AppData\Local\pypoetry\Cache\virtualenvs\some-project-mBcAJQ45-py3.11\Scripts\python.exe -m pip install --use-pep517 --disable-pip-version-check --prefix C:\Users\ramar\AppData\Local\pypoetry\Cache\virtualenvs\some-project-mBcAJQ45-py3.11 --no-deps C:\Users\ramar\AppData\Local\pypoetry\Cache\artifacts\33\22\27\44ab89b078748d2875e75bf8d7bc377dd893064d20d8c3acb92d7b2d64\pysha3-1.0.2.tar.gz' returned non-zero exit status 1.

  at ~\.pyenv\pyenv-win\versions\3.10.7\lib\subprocess.py:524 in run
       520│             # We don't call process.wait() as .__exit__ does that for us.
       521│             raise
       522│         retcode = process.poll()
       523│         if check and retcode:
    →  524│             raise CalledProcessError(retcode, process.args,
       525│                                      output=stdout, stderr=stderr)
       526│     return CompletedProcess(process.args, retcode, stdout, stderr)
       527│
       528│

The following error occurred when trying to handle this error:


  Stack trace:

  3  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\pip.py:49 in pip_install
       return environment.run_pip(*args)

  2  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\env.py:1435 in run_pip
       return self._run(cmd, **kwargs)

  1  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\env.py:1712 in _run
       return super()._run(cmd, **kwargs)

  EnvCommandError

  Command C:\Users\ramar\AppData\Local\pypoetry\Cache\virtualenvs\some-project-mBcAJQ45-py3.11\Scripts\python.exe -m pip install --use-pep517 --disable-pip-version-check --prefix C:\Users\ramar\AppData\Local\pypoetry\Cache\virtualenvs\some-project-mBcAJQ45-py3.11 --no-deps C:\Users\ramar\AppData\Local\pypoetry\Cache\artifacts\33\22\27\44ab89b078748d2875e75bf8d7bc377dd893064d20d8c3acb92d7b2d64\pysha3-1.0.2.tar.gz errored with the following return code 1, and output:
  Processing c:\users\ramar\appdata\local\pypoetry\cache\artifacts\33\22\27\44ab89b078748d2875e75bf8d7bc377dd893064d20d8c3acb92d7b2d64\pysha3-1.0.2.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: pysha3
    Building wheel for pysha3 (pyproject.toml): started
    Building wheel for pysha3 (pyproject.toml): finished with status 'error'
    error: subprocess-exited-with-error

    Building wheel for pysha3 (pyproject.toml) did not run successfully.
    exit code: 1

    [16 lines of output]
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-cpython-311
    copying sha3.py -> build\lib.win-amd64-cpython-311
    running build_ext
    building '_pysha3' extension
    creating build\temp.win-amd64-cpython-311
    creating build\temp.win-amd64-cpython-311\Release
    creating build\temp.win-amd64-cpython-311\Release\Modules
    creating build\temp.win-amd64-cpython-311\Release\Modules\_sha3
    "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -DPY_WITH_KECCAK=1 -IC:\Users\ramar\AppData\Local\pypoetry\Cache\virtualenvs\some-project-mBcAJQ45-py3.11\include -IC:\Users\ramar\.pyenv\pyenv-win\versions\3.11.0rc2\include -IC:\Users\ramar\.pyenv\pyenv-win\versions\3.11.0rc2\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /TcModules/_sha3/sha3module.c /Fobuild\temp.win-amd64-cpython-311\Release\Modules/_sha3/sha3module.obj
    sha3module.c
    C:\Users\ramar\AppData\Local\Temp\pip-req-build-ldsc8xsx\Modules\_sha3\backport.inc(78): fatal error C1083: Cannot open include file: 'pystrhex.h': No such file or directory
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
    [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
    ERROR: Failed building wheel for pysha3
  Failed to build pysha3
  ERROR: Could not build wheels for pysha3, which is required to install pyproject.toml-based projects


  at ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\env.py:1476 in _run
      1472│                 output = subprocess.check_output(
      1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1474│                 )
      1475│         except CalledProcessError as e:
    → 1476│             raise EnvCommandError(e, input=input_)
      1477│
      1478│         return decode(output)
      1479│
      1480│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  Stack trace:

  5  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\installation\executor.py:261 in _execute_operation
       result = self._do_execute_operation(operation)

  4  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\installation\executor.py:334 in _do_execute_operation
       result: int = getattr(self, f"_execute_{method}")(operation)

  3  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\installation\executor.py:454 in _execute_install
       status_code = self._install(operation)

  2  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\installation\executor.py:496 in _install
       return self.pip_install(archive, upgrade=operation.job_type == "update")

  1  ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\installation\executor.py:123 in pip_install
       pip_install(req, self._env, upgrade=upgrade, editable=editable)

  PoetryException

  Failed to install C:/Users/ramar/AppData/Local/pypoetry/Cache/artifacts/33/22/27/44ab89b078748d2875e75bf8d7bc377dd893064d20d8c3acb92d7b2d64/pysha3-1.0.2.tar.gz

  at ~\AppData\Roaming\pypoetry\venv\lib\site-packages\poetry\utils\pip.py:51 in pip_install
       47│
       48│     try:
       49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
    →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       52│

[kigawas]

You can try the master branch on 3.11. But due to eth-utils, it'll probably fail building

[ramarivera]

Oh, you completely changed the backend.
I was thinking something more like this master...ramarivera:web3-input-decoder:master

In my case, I don't mind not using pysha, but I read from ppl complaining it was 10x slower T.T

[kigawas]

pysha is not available anymore on Python >= 3.11. I don't think we need to support it.

FYI:

python/cpython#89639 (comment)

[kigawas]

By the way, eth-abi will also fail on 3.11

https://github.com/kigawas/web3-input-decoder/actions/runs/3244138084/jobs/5319976228