Skip to content

create and validate build_variables.bzl #8326

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open

Conversation

swolchok
Copy link
Contributor

@swolchok swolchok commented Feb 8, 2025

First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation.

[ghstack-poisoned]
Copy link

pytorch-bot bot commented Feb 8, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/8326

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 7f5cfd3 with merge base f3fc096 (image):

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 8, 2025
swolchok added a commit that referenced this pull request Feb 8, 2025
First step of #8268.

ghstack-source-id: b29aa49
ghstack-comment-id: 2644414497
Pull Request resolved: #8326
@swolchok swolchok marked this pull request as draft February 8, 2025 01:37
Copy link

github-actions bot commented Feb 8, 2025

This PR needs a release notes: label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[ghstack-poisoned]
swolchok added a commit that referenced this pull request Feb 9, 2025
First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation.

ghstack-source-id: 07b627b
ghstack-comment-id: 2644414497
Pull Request resolved: #8326
@swolchok
Copy link
Contributor Author

Current error:

Buck-generated _custom_ops__srcs does not match hardcoded CUSTOM_OPS_SRCS
    in build_variables.bzl.  Left:
    extension/llm/custom_ops/op_fallback.cpp;extension/llm/custom_ops/op_fast_hadamard_transform.cpp;extension/llm/custom_ops/op_sdpa.cpp;extension/llm/custom_ops/op_update_cache.cpp;extension/llm/custom_ops/spinquant/fast_hadamard_transform.cpp;extension/llm/custom_ops/spinquant/third-party/FFHT/fht_avx.c;kernels/portable/cpu/util/reduce_util.cpp


     Right: extension/llm/custom_ops/op_fallback.cpp;extension/llm/custom_ops/op_fast_hadamard_transform.cpp;extension/llm/custom_ops/op_sdpa.cpp;extension/llm/custom_ops/op_update_cache.cpp;extension/llm/custom_ops/spinquant/fast_hadamard_transform.cpp;extension/llm/custom_ops/spinquant/third-party/FFHT/fht_neon.c;kernels/portable/cpu/util/reduce_util.cpp

So, we need a mechanism for choosing fht_avx.c on Intel and fht_neon.c on ARM. Best way is probably to add a CMake-native way to do this and filter these files out of the Buck-to-CMake pipeline.

[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
@swolchok swolchok changed the base branch from main to gh/swolchok/243/head February 10, 2025 19:00
swolchok added a commit that referenced this pull request Feb 10, 2025
First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation.

ghstack-source-id: 91476c9
ghstack-comment-id: 2644414497
Pull Request resolved: #8326
@swolchok swolchok added the release notes: build Changes related to build, including dependency upgrades, build flags, optimizations, etc. label Feb 10, 2025
[ghstack-poisoned]
swolchok added a commit that referenced this pull request Feb 10, 2025
First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation.

ghstack-source-id: b2ffb3b
ghstack-comment-id: 2644414497
Pull Request resolved: #8326
@swolchok swolchok changed the title WIP: create build_variables.bzl create and validate build_variables.bzl Feb 10, 2025
@swolchok swolchok marked this pull request as ready for review February 10, 2025 20:54
[ghstack-poisoned]
swolchok added a commit that referenced this pull request Feb 10, 2025
First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation.

ghstack-source-id: dc50258
ghstack-comment-id: 2644414497
Pull Request resolved: #8326
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
swolchok added a commit that referenced this pull request Feb 25, 2025
First step of #8268 -- we are moving from buck-extracted filelist to using one shared filelist, and the first step is to create the shared filelist and validate it against the buck generation.

ghstack-source-id: c4195ce
ghstack-comment-id: 2644414497
Pull Request resolved: #8326
Base automatically changed from gh/swolchok/243/head to main February 26, 2025 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. release notes: build Changes related to build, including dependency upgrades, build flags, optimizations, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants