Skip to content

Commit bf2a6c7

Browse files
authored
perf: more options for kv tile size (#336)
For small query size setting, we might use large kv tile size.
1 parent ea89492 commit bf2a6c7

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

Diff for: include/flashinfer/utils.cuh

+7-1
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,13 @@
7272
}
7373

7474
#define DISPATCH_NUM_FRAGS_Z(max_frags_z, NUM_FRAGS_Z, ...) \
75-
if (max_frags_z >= 4) { \
75+
if (max_frags_z >= 8) { \
76+
constexpr size_t NUM_FRAGS_Z = 8; \
77+
__VA_ARGS__ \
78+
} else if (max_frags_z >= 6) { \
79+
constexpr size_t NUM_FRAGS_Z = 6; \
80+
__VA_ARGS__ \
81+
} else if (max_frags_z >= 4) { \
7682
constexpr size_t NUM_FRAGS_Z = 4; \
7783
__VA_ARGS__ \
7884
} else if (max_frags_z >= 2) { \

0 commit comments

Comments
 (0)