-
Notifications
You must be signed in to change notification settings - Fork 557
Eliminating dynamic shaped tensors for the static inputs #911
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
So the current plan is to add canonicalization for add.tensor and friends that can folds 0d operands, these operand can be coming either from a
after canonicalization
Example 2:
Will also have the exact same IR after canonicalization |
👍 |
Hi @nirvedhmeshram, I think this IR:
after canonicalization should look like this:
The argument Please correct me if I'm wrong. |
Yup, I misunderstood that. Thanks for the correction. |
@sjarus FYI |
SUPER helpful for our TorchToTosa work! |
@sjarus do you think we can try the TOSA Bert lowering now we have a WIP for this ? |
I'll try it out as a local patch @powderluv |
Update: I tried out #935 locally using the code that was present yesterday evening. I see that aten.ones does not appear to generate a static shaped form. This is consumed by a slice , which also propagates the unknown shape, so it looks like the aten.ones is a blocker here:
|
I suspect this is a phase ordering issue with when we apply this canonicalization transformation. See my comments here: #963 |
@powderluv did we ever find out where in the source Python code this pattern was coming from? |
@silvasean I tried briefly, but could not get the location information in the heavydep test from which we derive the model so could not find the source. |
Could you try "non-briefly"? :D This is pretty important, as the way that PyTorch is architected "should not" produce this, so it's really valuable information for us to know how it gets created. |
The aten.ones should take the %int1 and %int7 as inputs in list form and emit a tensor of shape (1, 7). However it computes the shape as (1, ?) . It's picking up the parameters explicitly because changing the shape to something else still picks up the first parameter right. Where is this shape resolved ? |
The shape transfer function for aten.ones is trivial. So if it isn't resolved it means that at the point where we are simplifying shapes (SimplifyShapeCalculations), the list elements are not fully resolved to %int1 and %int7. So the IR before that pass (ideally reduced) is necessary to debug this. |
Thanks @silvasean . Yes, I wondered what I'm missing about what seemed like a fairly straightforward op in terms of shape resolution. |
Is this solved byhttps://github.com//pull/935 ? |
Yes @silvasean , this one was solved by #935 |
Uh oh!
There was an error while loading. Please reload this page.
This issue is related to #935
For the torch IR of miniLM model, we observed dynamic tensors like below even if we provided static input as
torch.randint(2, (1, 128))
Dynamic shape of %194 makes many of the
torch.aten.add.Tensor
operations also dynamic in the next part of the IR. So solving this code snipped as static might also can solve other dynamic shaped tensor issues.I am working on to find a canonical way eliminate those dynamic shaped tensors.
The text was updated successfully, but these errors were encountered: