Fix macOS arm64 and universal2 wheels #33
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If we install ada_url on macOS M1 then we will get this error:
```
$ pip3 install ada_url==1.3.0
...
$ python3 -c "import ada_url"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File ".../python/3.11.3/lib/python3.11/site-packages/ada_url/__init__.py", line 1, in <module>
from ada_url.ada_adapter import (
File ".../installs/python/3.11.3/lib/python3.11/site-packages/ada_url/ada_adapter.py", line 4, in <module>
from ada_url._ada_wrapper import ffi, lib
ImportError: dlopen(.../installs/python/3.11.3/lib/python3.11/site-packages/ada_url/_ada_wrapper.abi3.so, 0x0002): symbol not found in flat namespace '_ada_can_parse'
```
GitHub Actions doesn’t support macOS arm64 (M1/Apple Silicon) managed
runners, so we need to do cross-compilation on macOS. `cibuildwheel` sets
`ARCHFLAGS` variable we can use with `CXX` compiler to set the correct
target.
I decided to move `before-all` to `[tool.cibuildwheel.linux]` and add
`before-build` to `[tool.cibuildwheel.macos]` for optimizing CI build time
execution purposes.
We can’t use `before-all` in `[tool.cibuildwheel]` section because it will
be running only once per platform (`linux`/`macos`), and we need to run it
before all builds for all architectures for macOS (`x86_64`, `arm64`,
`universal2`) because we will have different values for the `ARCHFLAGS`
environment variable.
If we add `before-build` to `[tool.cibuildwheel]` then it will be executed
many times for linux platform because builds are executed inside different
docker containers and because we use QEMU then the compilation process
is slow (GitHub Action Workflow execution time: 25 minutes vs 48 minutes).
|
Many thanks for this writeup and PR. I will merge it soon! |
bbayles
approved these changes
Sep 12, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If we install ada_url on macOS M1 then we will get this error:
GitHub Actions doesn’t support macOS arm64 (M1/Apple Silicon) managed runners, so we need to do cross-compilation on macOS.
cibuildwheelsetsARCHFLAGSvariable we can use withCXXcompiler to set the correct target.I decided to move
before-allto[tool.cibuildwheel.linux]and addbefore-buildto[tool.cibuildwheel.macos]for optimizing CI build time execution purposes.We can’t use
before-allin[tool.cibuildwheel]section because it will be running only once per platform (linux/macos), and we need to run it before all builds for all architectures for macOS (x86_64,arm64,universal2) because we will have different values for theARCHFLAGSenvironment variable.If we add
before-buildto[tool.cibuildwheel]then it will be executed many times for linux platform because builds are executed inside different docker containers and because we use QEMU then the compilation process is slow (GitHub Action Workflow execution time: 25 minutes vs 48 minutes).