build wheels for musl aarch64

[codefromthecrypt]

My team uses a lot of python examples for genai, and tiktoken currently requires system packages when run on alpine (via docker on apple silicon). By supplying wheels for musl, we'll be able to have fast small builds which are helpful as these images are built on demand in system tests.

$ docker run -it --rm --entrypoint /bin/sh python:3.12-alpine -c "pip install --upgrade pip && pip install tiktoken"
Requirement already satisfied: pip in /usr/local/lib/python3.12/site-packages (24.3.1)
Collecting pip
  Downloading pip-25.0.1-py3-none-any.whl.metadata (3.7 kB)
Downloading pip-25.0.1-py3-none-any.whl (1.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 59.4 MB/s eta 0:00:00
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 24.3.1
    Uninstalling pip-24.3.1:
      Successfully uninstalled pip-24.3.1
Successfully installed pip-25.0.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
Collecting tiktoken
  Downloading tiktoken-0.9.0.tar.gz (35 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting regex>=2022.1.18 (from tiktoken)
  Downloading regex-2024.11.6-cp312-cp312-musllinux_1_2_aarch64.whl.metadata (40 kB)
Collecting requests>=2.26.0 (from tiktoken)
  Downloading requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting charset-normalizer<4,>=2 (from requests>=2.26.0->tiktoken)
  Downloading charset_normalizer-3.4.1-cp312-cp312-musllinux_1_2_aarch64.whl.metadata (35 kB)
Collecting idna<4,>=2.5 (from requests>=2.26.0->tiktoken)
  Downloading idna-3.10-py3-none-any.whl.metadata (10 kB)
Collecting urllib3<3,>=1.21.1 (from requests>=2.26.0->tiktoken)
  Downloading urllib3-2.3.0-py3-none-any.whl.metadata (6.5 kB)
Collecting certifi>=2017.4.17 (from requests>=2.26.0->tiktoken)
  Downloading certifi-2025.1.31-py3-none-any.whl.metadata (2.5 kB)
Downloading regex-2024.11.6-cp312-cp312-musllinux_1_2_aarch64.whl (781 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 781.7/781.7 kB 55.8 MB/s eta 0:00:00
Downloading requests-2.32.3-py3-none-any.whl (64 kB)
Downloading certifi-2025.1.31-py3-none-any.whl (166 kB)
Downloading charset_normalizer-3.4.1-cp312-cp312-musllinux_1_2_aarch64.whl (141 kB)
Downloading idna-3.10-py3-none-any.whl (70 kB)
Downloading urllib3-2.3.0-py3-none-any.whl (128 kB)
Building wheels for collected packages: tiktoken
  Building wheel for tiktoken (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for tiktoken (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [36 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build/lib.linux-aarch64-cpython-312/tiktoken
      copying tiktoken/model.py -> build/lib.linux-aarch64-cpython-312/tiktoken
      copying tiktoken/__init__.py -> build/lib.linux-aarch64-cpython-312/tiktoken
      copying tiktoken/core.py -> build/lib.linux-aarch64-cpython-312/tiktoken
      copying tiktoken/registry.py -> build/lib.linux-aarch64-cpython-312/tiktoken
      copying tiktoken/_educational.py -> build/lib.linux-aarch64-cpython-312/tiktoken
      copying tiktoken/load.py -> build/lib.linux-aarch64-cpython-312/tiktoken
      creating build/lib.linux-aarch64-cpython-312/tiktoken_ext
      copying tiktoken_ext/openai_public.py -> build/lib.linux-aarch64-cpython-312/tiktoken_ext
      running egg_info
      writing tiktoken.egg-info/PKG-INFO
      writing dependency_links to tiktoken.egg-info/dependency_links.txt
      writing requirements to tiktoken.egg-info/requires.txt
      writing top-level names to tiktoken.egg-info/top_level.txt
      reading manifest file 'tiktoken.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching 'Makefile'
      adding license file 'LICENSE'
      writing manifest file 'tiktoken.egg-info/SOURCES.txt'
      copying tiktoken/py.typed -> build/lib.linux-aarch64-cpython-312/tiktoken
      running build_ext
      running build_rust
      error: can't find Rust compiler
      
      If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
      
      To update pip, run:
      
          pip install --upgrade pip
      
      and then retry package installation.
      
      If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tiktoken
Failed to build tiktoken
ERROR: Failed to build installable wheels for some pyproject.toml based projects (tiktoken)

[xrmx]

I think this is matter of removing this line

tiktoken/pyproject.toml

Line 34 in e35ab09

"*-musllinux_aarch64",

but the history ed27aa3 suggests that there was something not working

Oh and this is a dup of #350

[codefromthecrypt]

thanks for looking. I'm a fan of not dup'ing and missed that, so closing in favor of #350