Support swin/bert with dynamic batch #1270
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Cheng Hang [email protected]
Description
To support dynamic-batch compilation of swin transformer/BERT, I have added dynamic batch support of the following relevant op converters:
aten::reshape
aten::select
aten::roll
aten::flatten
aten::slice
The older version can support dynamic shape, but will report a lot of trt ERRORs, meanwhile compiles successfully. This can be reproduced for TRT 8.4, and it did not appear for TRT 8.2. I suppose it is caused by some new checks performed in TRT8.4.To resolve this issue, I have slightly changed the order of codes for dynamic slice, and it did not change the logics. By such simple manipulation, the ERRORs disappear.
Fixes #1271
Type of change
Please delete options that are not relevant and/or add your own.
Checklist: