Skip to content

🐛 [Bug] Encountered bug when using torch.ops.aten.cat converter #1862

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gs-olive opened this issue Apr 27, 2023 · 0 comments · Fixed by #1863
Closed

🐛 [Bug] Encountered bug when using torch.ops.aten.cat converter #1862

gs-olive opened this issue Apr 27, 2023 · 0 comments · Fixed by #1863
Assignees
Labels
bug Something isn't working

Comments

@gs-olive
Copy link
Collaborator

Bug Description

When compiling the T5-Base Model model via the aten path, the following error is encountered:

IndexError: tuple index out of range
While executing %cat_default : [#users=1] = call_function[target=torch.ops.aten.cat.default](args = ([%arg0, %arg1, %arg2],), kwargs = {_itensor_to_tensor_meta: {<tensorrt.tensorrt.ITensor object at 0x7ff49c2fae30>: None, <tensorrt.tensorrt.ITensor object at 0x7ff49c2e6e30>: None, <tensorrt.tensorrt.ITensor object at 0x7ff49c2e1f30>: None}})

To Reproduce

Steps to reproduce the behavior:

  1. Initialize model: T5Model.from_pretrained("t5-base").eval().cuda()
  2. Initialize three input tensors, for example: torch.randint(0, 1, (1, 14), dtype=torch.int32).to("cuda") ("input_ids", "attention_mask", "decoder_input_ids")
  3. (Optional) Use the transformers tools to trace the model via: transformers.utils.fx.symbolic_trace(model, input_names=["input_ids", "attention_mask", "decoder_input_ids"])
  4. Compile the model using FX

Expected behavior

Model should compile via the aten path

Environment

  • Transformers: 4.26.1
  • Torch-TensorRT Version (e.g. 1.0.0): b3f433a
  • PyTorch Version (e.g. 1.0): 2.1.0.dev20230419+cu117
  • CPU Architecture: Intel Xeon CPU
  • OS: Ubuntu 20.04
  • How you installed PyTorch: pip
  • Build command you used: python setup.py develop
  • Are you using local sources or building from archives: local
  • Python version: 3.8.13
  • CUDA version: 11.7
@gs-olive gs-olive added the bug Something isn't working label Apr 27, 2023
@gs-olive gs-olive self-assigned this Apr 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant