❓ [Question] How to run Torch-Tensorrt on JETSON AGX ORIN?

[krmayankb]

❓ Question

Not able to run Torch-Tensorrt on Jetson AGX ORIN
As per the release note, it is mentioned that current release doesn't have support for Jetpack 5.0DP but ORIN only supports Jetpack 5.0DP (I might be wrong but inferring from this Jetpack Archives.. Is there a way to run Torch-Tensort on ORIN? if not what's the possible timeline for new release with this support?

What you have already tried

I did tried building for python, as suggested in the repo, it enables import torch_tensorrt but doesn't supports any attributes.

Environment

• PyTorch Version (e.g., 1.0): 1.11
• CPU Architecture: arm64
• OS (e.g., Linux): Ubuntu 20.04
• How you installed PyTorch (conda, pip, libtorch, source): tried both, wheels provided here and building from source(instruction from here).
• Build command you used (if compiling from source): python3 setup.py --use_cxx11_abi (however, this refers to jetpack 4.6 by default)
• Python version: 3.8
• CUDA version: 11.4
• GPU models and configuration: Jetson ORIN
• Any other relevant information:

[peri044]

We are currently working on that and plan to release a version compatible with Jetpack 5.0 soon. Here is a working PR you can try #1158

[haichuanwang001]

@krmayankb Were you able to run torch_tensorrt on Orin successfully?

[krmayankb]

@krmayankb Were you able to run torch_tensorrt on Orin successfully?

@haichuanwang001 No, I couldn't. I stop trying based on feedback received.

Currently, I am working with the Python API of TensorRT. It's a bit of work to wrap your head around but it works. Let me know if you need more info about this.

[narendasan]

torch-tensorrt 1.1.1 was released with Orin support https://github.com/pytorch/TensorRT/releases/tag/v1.1.1

Follow the instructions here to install https://pytorch.org/TensorRT/tutorials/installation.html

[krmayankb]

@narendasan good to see this release.

Please refer above image. I don't see any change to setup.py it considers Jetpack 4.5 and 4.6. refer this
line. Is it fine to modify setup.py or file need an update?

---

UPDATE: library build and functionalities are working on ORIN with Jetpack 5.0DP, while the version of torch_tensorrt is: 1.1.0+c2e396a8 . Thanks for the support.