Re-shardable Hash Zch #2538
Closed
Re-shardable Hash Zch #2538
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Fully reshardable ZCH: we can handle any value which is in common with default value (768) so WS 1,2,4,8,16,24,32,48,64,96,128, etc. and go up and down. Differential Revision: D62483238
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D62483238 |
|
This pull request has been reverted by a97cf28. |
dstaay-fb
added a commit
to dstaay-fb/torchrec
that referenced
this pull request
Nov 13, 2024
Summary: Fully reshardable ZCH: we can handle any value which is in common with default value (768) so WS 1,2,4,8,16,24,32,48,64,96,128, etc. and go up and down. Differential Revision: D65912888
facebook-github-bot
pushed a commit
that referenced
this pull request
Nov 22, 2024
Summary: Pull Request resolved: #2556 Pull Request resolved: #2538 Fully reshardable ZCH: we can handle any value which is in common with default value (768); this allows training or inference world size to be any factor of `total_num_buckets` (ie. 1,2,4,8,16,24,32,48,64,96,128, etc) and still have identical numerics. Reviewed By: emlin Differential Revision: D65912888 fbshipit-source-id: 55f68903ab0a7007e2dc7f2d22ce08613f5abb48
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: Fully reshardable ZCH: we can handle any value which is in common with default value (768) so WS 1,2,4,8,16,24,32,48,64,96,128, etc. and go up and down.
Differential Revision: D62483238