Skip to content

[LV][EVL] Support call instruction with EVL-vectorization #110412

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Nov 28, 2024
Merged

[LV][EVL] Support call instruction with EVL-vectorization #110412

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
18e479f
[LV][EVL][Test] Prepare test for adding intrinsics of call Recipe
LiqinWeng Sep 28, 2024
fa5eab3
[LV][EVL] Support call instruction with EVL-vectorization
LiqinWeng Oct 16, 2024
428aa65
fix the comments
LiqinWeng Oct 29, 2024
6212dad
fix the comments
LiqinWeng Oct 30, 2024
e9389b1
add the some comments
LiqinWeng Oct 31, 2024
a0bfd0e
fix the comments
LiqinWeng Nov 5, 2024
8081249
fix the comments
LiqinWeng Nov 6, 2024
7cc3734
fix the comments
LiqinWeng Nov 9, 2024
8bcac22
fix the comments
LiqinWeng Nov 18, 2024
d3258eb
Rebase && add vp_cttz test && correct the failed tests
LiqinWeng Nov 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions llvm/lib/Analysis/VectorUtils.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,9 +118,13 @@ bool llvm::isVectorIntrinsicWithScalarOpAtArg(Intrinsic::ID ID,
unsigned ScalarOpdIdx) {
switch (ID) {
case Intrinsic::abs:
case Intrinsic::vp_abs:
case Intrinsic::ctlz:
case Intrinsic::vp_ctlz:
case Intrinsic::cttz:
case Intrinsic::vp_cttz:
case Intrinsic::is_fpclass:
case Intrinsic::vp_is_fpclass:
Comment on lines +121 to +127
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you still need these changes?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The second parameter of this intrinsic is a scalar, which should remain in scalar form after vectorization.
llvm.vp.abs.nxv1i8(<vscale x 1 x i8> %va, i1 false, <vscale x 1 x i1> splat (i1 true), i32 %evl)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And where exactly it is used, why do you need this change?

Copy link
Contributor Author

@LiqinWeng LiqinWeng Nov 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uploading image.png…

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if not change the code,the second parament of vp.abs will splat: <vscale x 4 x i1> shufflevector (<vscale x 4 x i1> insertelement (<vscale x 4 x i1> poison, i1 true, i64 0), <vscale x 4 x i1> poison, <vscale x 4 x i32> zeroinitializer)

case Intrinsic::powi:
return (ScalarOpdIdx == 1);
case Intrinsic::smul_fix:
Expand All @@ -145,10 +149,13 @@ bool llvm::isVectorIntrinsicWithOverloadTypeAtArg(
case Intrinsic::fptoui_sat:
case Intrinsic::lrint:
case Intrinsic::llrint:
case Intrinsic::vp_lrint:
case Intrinsic::vp_llrint:
case Intrinsic::ucmp:
case Intrinsic::scmp:
return OpdIdx == -1 || OpdIdx == 0;
case Intrinsic::is_fpclass:
case Intrinsic::vp_is_fpclass:
return OpdIdx == 0;
case Intrinsic::powi:
return OpdIdx == -1 || OpdIdx == 1;
Expand Down
9 changes: 9 additions & 0 deletions llvm/lib/Transforms/Vectorize/VPlanRecipes.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -993,6 +993,15 @@ InstructionCost VPWidenIntrinsicRecipe::computeCost(ElementCount VF,
for (const auto &[Idx, Op] : enumerate(operands())) {
auto *V = Op->getUnderlyingValue();
if (!V) {
// Push all the VP Intrinsic's ops into the Argments even if is nullptr.
// Some VP Intrinsic's cost will assert the number of parameters.
// Mainly appears in the following two scenarios:
// 1. EVL Op is nullptr
// 2. The Argmunt of the VP Intrinsic is also the VP Intrinsic
if (VPIntrinsic::isVPIntrinsic(VectorIntrinsicID)) {
Arguments.push_back(V);
continue;
}
if (auto *UI = dyn_cast_or_null<CallBase>(getUnderlyingValue())) {
Arguments.push_back(UI->getArgOperand(Idx));
continue;
Expand Down
20 changes: 20 additions & 0 deletions llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1481,6 +1481,26 @@ static void transformRecipestoEVLRecipes(VPlan &Plan, VPValue &EVL) {
VPValue *NewMask = GetNewMask(Red->getCondOp());
return new VPReductionEVLRecipe(*Red, EVL, NewMask);
})
.Case<VPWidenIntrinsicRecipe>(
[&](VPWidenIntrinsicRecipe *CInst) -> VPRecipeBase * {
auto *CI = cast<CallInst>(CInst->getUnderlyingInstr());
Intrinsic::ID VPID = VPIntrinsic::getForIntrinsic(
CI->getCalledFunction()->getIntrinsicID());
if (VPID == Intrinsic::not_intrinsic)
return nullptr;

SmallVector<VPValue *> Ops(CInst->operands());
assert(VPIntrinsic::getMaskParamPos(VPID) &&
VPIntrinsic::getVectorLengthParamPos(VPID) &&
"Expected VP intrinsic");
VPValue *Mask = Plan.getOrAddLiveIn(ConstantInt::getTrue(
IntegerType::getInt1Ty(CI->getContext())));
Ops.push_back(Mask);
Ops.push_back(&EVL);
return new VPWidenIntrinsicRecipe(
*CI, VPID, Ops, TypeInfo.inferScalarType(CInst),
CInst->getDebugLoc());
})
.Case<VPWidenSelectRecipe>([&](VPWidenSelectRecipe *Sel) {
SmallVector<VPValue *> Ops(Sel->operands());
Ops.push_back(&EVL);
Expand Down
Loading