Skip to content

flash-attn==2.5.6 not match #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
cosxsinxds opened this issue May 16, 2025 · 2 comments
Open

flash-attn==2.5.6 not match #4

cosxsinxds opened this issue May 16, 2025 · 2 comments

Comments

@cosxsinxds
Copy link

When I ran 'pip install flash-attn==2.5.6 --no-build-isolation', it failed because the wheel simply has not been published.

Image

(revisionllm) zj@R7000:~/tmp$ pip install flash-attn==2.5.6 --no-build-isolation
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting flash-attn==2.5.6
Using cached https://pypi.tuna.tsinghua.edu.cn/packages/85/52/9834eb88dd8a8be13b5399a32f23a79f0a307d9708f92e2b4778afaae0d4/flash_attn-2.5.6.tar.gz (2.5 MB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: torch in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from flash-attn==2.5.6) (2.7.0)
Requirement already satisfied: einops in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from flash-attn==2.5.6) (0.8.1)
Requirement already satisfied: packaging in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from flash-attn==2.5.6) (25.0)
Requirement already satisfied: ninja in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from flash-attn==2.5.6) (1.11.1.4)
Requirement already satisfied: filelock in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (3.18.0)
Requirement already satisfied: typing-extensions>=4.10.0 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (4.13.2)
Requirement already satisfied: sympy>=1.13.3 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (1.14.0)
Requirement already satisfied: networkx in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (3.4.2)
Requirement already satisfied: jinja2 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (3.1.6)
Requirement already satisfied: fsspec in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (2025.3.2)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.6.77 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (12.6.77)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.6.77 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (12.6.77)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.6.80 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (12.6.80)
Requirement already satisfied: nvidia-cudnn-cu12==9.5.1.17 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (9.5.1.17)
Requirement already satisfied: nvidia-cublas-cu12==12.6.4.1 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (12.6.4.1)
Requirement already satisfied: nvidia-cufft-cu12==11.3.0.4 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (11.3.0.4)
Requirement already satisfied: nvidia-curand-cu12==10.3.7.77 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (10.3.7.77)
Requirement already satisfied: nvidia-cusolver-cu12==11.7.1.2 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (11.7.1.2)
Requirement already satisfied: nvidia-cusparse-cu12==12.5.4.2 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (12.5.4.2)
Requirement already satisfied: nvidia-cusparselt-cu12==0.6.3 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (0.6.3)
Requirement already satisfied: nvidia-nccl-cu12==2.26.2 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (2.26.2)
Requirement already satisfied: nvidia-nvtx-cu12==12.6.77 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (12.6.77)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.6.85 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (12.6.85)
Requirement already satisfied: nvidia-cufile-cu12==1.11.1.6 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (1.11.1.6)
Requirement already satisfied: triton==3.3.0 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from torch->flash-attn==2.5.6) (3.3.0)
Requirement already satisfied: setuptools>=40.8.0 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from triton==3.3.0->torch->flash-attn==2.5.6) (80.1.0)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from sympy>=1.13.3->torch->flash-attn==2.5.6) (1.3.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages (from jinja2->torch->flash-attn==2.5.6) (3.0.2)
Building wheels for collected packages: flash-attn
DEPRECATION: Building 'flash-attn' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the --use-pep517 option, (possibly combined with --no-build-isolation), or adding a pyproject.toml file to the source tree of 'flash-attn'. Discussion can be found at pypa/pip#6334
Building wheel for flash-attn (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [158 lines of output]
fatal: 不是 git 仓库（或者任何父目录）：.git

  torch.__version__  = 2.7.0+cu126
  
  
  /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/__init__.py:94: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
  !!
  
          ********************************************************************************
          Requirements should be satisfied by a PEP 517 installer.
          If you are using pip, you can try `pip install --use-pep517`.
          ********************************************************************************
  
  !!
    dist.fetch_build_eggs(dist.setup_requires)
  /home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
  !!
  
          ********************************************************************************
          Please consider removing the following classifiers in favor of a SPDX license expression:
  
          License :: OSI Approved :: BSD License
  
          See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
          ********************************************************************************
  
  !!
    self._finalize_license_expression()
  running bdist_wheel
  Guessing wheel URL:  https://github.com/Dao-AILab/flash-attention/releases/download/v2.5.6/flash_attn-2.5.6+cu122torch2.7cxx11abiTRUE-cp310-cp310-linux_x86_64.whl
  Precompiled wheel not found. Building from source...
  running build
  running build_py
  creating build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/flash_blocksparse_attention.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/flash_attn_interface.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/flash_attn_triton.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/fused_softmax.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/flash_attn_triton_og.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/bert_padding.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  copying flash_attn/flash_blocksparse_attn_interface.py -> build/lib.linux-x86_64-cpython-310/flash_attn
  creating build/lib.linux-x86_64-cpython-310/flash_attn/losses
  copying flash_attn/losses/cross_entropy.py -> build/lib.linux-x86_64-cpython-310/flash_attn/losses
  copying flash_attn/losses/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/losses
  creating build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/btlm.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/baichuan.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/vit.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/gpt.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/falcon.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/opt.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/gpt_neox.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/gptj.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/llama.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/bert.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  copying flash_attn/models/bigcode.py -> build/lib.linux-x86_64-cpython-310/flash_attn/models
  creating build/lib.linux-x86_64-cpython-310/flash_attn/layers
  copying flash_attn/layers/patch_embed.py -> build/lib.linux-x86_64-cpython-310/flash_attn/layers
  copying flash_attn/layers/rotary.py -> build/lib.linux-x86_64-cpython-310/flash_attn/layers
  copying flash_attn/layers/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/layers
  creating build/lib.linux-x86_64-cpython-310/flash_attn/utils
  copying flash_attn/utils/benchmark.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
  copying flash_attn/utils/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
  copying flash_attn/utils/generation.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
  copying flash_attn/utils/pretrained.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
  copying flash_attn/utils/distributed.py -> build/lib.linux-x86_64-cpython-310/flash_attn/utils
  creating build/lib.linux-x86_64-cpython-310/flash_attn/modules
  copying flash_attn/modules/block.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
  copying flash_attn/modules/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
  copying flash_attn/modules/embedding.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
  copying flash_attn/modules/mha.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
  copying flash_attn/modules/mlp.py -> build/lib.linux-x86_64-cpython-310/flash_attn/modules
  creating build/lib.linux-x86_64-cpython-310/flash_attn/ops
  copying flash_attn/ops/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
  copying flash_attn/ops/layer_norm.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
  copying flash_attn/ops/activations.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
  copying flash_attn/ops/fused_dense.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
  copying flash_attn/ops/rms_norm.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops
  creating build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/k_activations.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/linear.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/rotary.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/cross_entropy.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/__init__.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/layer_norm.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  copying flash_attn/ops/triton/mlp.py -> build/lib.linux-x86_64-cpython-310/flash_attn/ops/triton
  running build_ext
  Traceback (most recent call last):
    File "/tmp/pip-install-y1sc2w5u/flash-attn_40c2a4de5c7b4e4b84ab80c24924f9d5/setup.py", line 265, in run
      urllib.request.urlretrieve(wheel_url, wheel_filename)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/urllib/request.py", line 241, in urlretrieve
      with contextlib.closing(urlopen(url, data)) as fp:
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/urllib/request.py", line 216, in urlopen
      return opener.open(url, data, timeout)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/urllib/request.py", line 525, in open
      response = meth(req, response)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/urllib/request.py", line 634, in http_response
      response = self.parent.error(
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/urllib/request.py", line 563, in error
      return self._call_chain(*args)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/urllib/request.py", line 496, in _call_chain
      result = func(*args)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/urllib/request.py", line 643, in http_error_default
      raise HTTPError(req.full_url, code, msg, hdrs, fp)
  urllib.error.HTTPError: HTTP Error 404: Not Found
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 35, in <module>
    File "/tmp/pip-install-y1sc2w5u/flash-attn_40c2a4de5c7b4e4b84ab80c24924f9d5/setup.py", line 305, in <module>
      setup(
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/__init__.py", line 117, in setup
      return distutils.core.setup(**attrs)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 186, in setup
      return run_commands(dist)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
      dist.run_commands()
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
      self.run_command(cmd)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
      super().run_command(command)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-y1sc2w5u/flash-attn_40c2a4de5c7b4e4b84ab80c24924f9d5/setup.py", line 282, in run
      super().run()
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/command/bdist_wheel.py", line 370, in run
      self.run_command("build")
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
      self.distribution.run_command(command)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
      super().run_command(command)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
      cmd_obj.run()
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
      self.distribution.run_command(command)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
      super().run_command(command)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
      cmd_obj.run()
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 99, in run
      _build_ext.run(self)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 368, in run
      self.build_extensions()
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 611, in build_extensions
      _check_cuda_version(compiler_name, compiler_version)
    File "/home/zj/anaconda3/envs/revisionllm/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 479, in _check_cuda_version
      raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
  RuntimeError:
  The detected CUDA version (11.6) mismatches the version that was used to compile
  PyTorch (12.6). Please make sure to use the same CUDA versions.
  
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for flash-attn
Running setup.py clean for flash-attn
Failed to build flash-attn
ERROR: Failed to build installable wheels for some pyproject.toml based projects (flash-attn)
@Tanveer81
Copy link
Owner

Tanveer81 commented May 22, 2025 •
edited
Loading

Hi, thank you for your interest in the project. This command generally works. However, you can follow the steps below:

First, download the required flash-attn with a suitable CUDA and Torch version from this link: https://github.com/Dao-AILab/flash-attention/releases/tag/v2.5.6

Then, do pip install ./flash_attn-2.5.6+cuXXXtorchXX-cpXXX-cpXXX-linux_x86_64.whl

@cosxsinxds
Copy link
Author

Hi, thank you for your interest in the project. This command generally works. However, you can follow the steps below:

First, download the required flash-attn with a suitable CUDA and Torch version from this link: https://github.com/Dao-AILab/flash-attention/releases/tag/v2.5.6

Then, do pip install ./flash_attn-2.5.6+cuXXXtorchXX-cpXXX-cpXXX-linux_x86_64.whl

Thanks for your help.
And I found a better solution.

# 直接安装flash-attn可能会因为环境中没有cuda报错：OSError: CUDA_HOME environment variable is not set. 在conda之外直接安装过cuda可能不报错
conda install -c nvidia cuda-toolkit=12.6 
pip install flash-attn --no-build-isolation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Tanveer81 @cosxsinxds