Skip to content

🐛 [Bug] Encountered bug when converting deepset/roberta-base-squad2 Hugging Face Network #1410

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gs-olive opened this issue Oct 17, 2022 · 0 comments · Fixed by #1403
Closed
Labels
bug Something isn't working

Comments

@gs-olive
Copy link
Collaborator

Bug Description

When converting the deepset/roberta-base-squad2 Hugging Face network (https://huggingface.co/deepset/roberta-base-squad2) from TorchScript to Torch-TRT, the following error is encountered:

RuntimeError: [Error thrown at core/conversion/conversion.cpp:428] Expected eval_list->elements().size() == n->outputs().size() to be true but got false
Size of evaluated results: 3 and node outputs size: 2 must match.

To Reproduce

Steps to reproduce the behavior:

  1. Run torch_tensorrt.compile with deepset/roberta-base-squad2 model as input, using fp32 precision.
  2. Choose fixed input size of [1, 128] and enable truncate_long_and_double with 8 GB workspace.
  3. Pass in model keyword args to disable attention and hidden state outputs

Expected behavior

Model should successfully compile to Torch-TRT. Specifically, the size of evaluated results and node outputs should be matching

Environment

  • Torch-TensorRT Version: 1.3.0a0+b70c9132
  • PyTorch Version: 1.13.0.dev20220921+cu116
  • CPU Architecture: Intel Xeon CPU
  • OS: Ubuntu 20.04
  • How you installed PyTorch: pip
  • Build command you used: python setup.py develop
  • Are you using local sources or building from archives: local
  • Python version: 3.8.13
  • CUDA version: 11.6

Additional context

Bug was reproduced on PyTorch 22.08 and 22.09 containers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant