Skip to content

fix bugs in embedding converter #403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 22, 2021
Merged

fix bugs in embedding converter #403

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions core/conversion/converters/impl/select.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,9 +174,11 @@ auto select_registrations TRTORCH_UNUSED =
{"aten::embedding(Tensor weight, Tensor indices, int padding_idx=-1, bool scale_grad_by_freq=False, bool sparse=False) -> (Tensor)",
[](ConversionCtx* ctx, const torch::jit::Node* n, args& args) -> bool {
auto embeddingTensor = args[0].ITensorOrFreeze(ctx);
auto indicesTensor = args[1].ITensor();
auto indicesTensor = args[1].ITensorOrFreeze(ctx);
// Set datatype for indices tensor to INT32
indicesTensor->setType(nvinfer1::DataType::kINT32);
auto identity = ctx->net->addIdentity(*indicesTensor);
identity->setOutputType(0, nvinfer1::DataType::kINT32);
indicesTensor = identity->getOutput(0);

// IGatherLayer takes in input tensor, the indices, and the axis of input tensor to take indices from
auto gather_layer = ctx->net->addGather(*embeddingTensor, *indicesTensor, 0);
Comment on lines 174 to 184
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only modified this part in select.cpp.

Expand Down
2 changes: 1 addition & 1 deletion tests/core/conversion/converters/test_select.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ TEST(Converters, ATenEmbeddingConvertsCorrectly) {
auto jit_results = trtorch::tests::util::RunGraph(g, params, {jit_in});

// Run TensorRT
auto options_trt = torch::TensorOptions().device(torch::kCUDA, 0).dtype(torch::kI32);
auto options_trt = torch::TensorOptions().device(torch::kCUDA, 0).dtype(torch::kFloat);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do the indices only come in float? can they come in other types? We should maybe instead duplicate instead of modifying the test

Copy link
Contributor Author

@ruoqianguo ruoqianguo Mar 18, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When using setType function in embbedding converter, the input data type will be changed from kFloat to kInt32. But i have replaced setType function with addIdentity function, the input data type will not change. If i still use kInt32, it will get an error like that "Expected inputs[pyt_idx].dtype() == expected_type to be true but got false. Expected input tensors to have type Float, found type int".
In my opinion, a single operation at one layer of the network directly changes the input data type of the entire network, which maybe is not a cool thing, right? And the issue #388 maybe can be resolved through set all input data type as kFloat.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this is a good catch, I was just wondering about the test and why it wasnt duplicated. I think its not a massive deal, other than that this patch lgtm

auto trt_in = at::tensor({0, 1, 2}, options_trt);
auto trt_results = trtorch::tests::util::RunGraphEngine(g, params, {trt_in});
auto trt = trt_results[0].reshape(jit_results[0].sizes());
Expand Down