Skip to content

Commit 7c040b4

Browse files
committed
whisper : make beam candidate sort more stable
All else being otherwise equal, this encourages the beam candidate selection to re-use the same decoder, which slightly reduces the cache size. I wouldn't expect it to make much of a performance difference, but it helps when debug printing the cache and beam. Added as part of understanding ggml-org#1941.
1 parent 2f5a5a6 commit 7c040b4

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

whisper.cpp

+4-1
Original file line numberDiff line numberDiff line change
@@ -5357,7 +5357,10 @@ int whisper_full_with_state(
53575357
beam_candidates.begin(),
53585358
beam_candidates.end(),
53595359
[](const beam_candidate & a, const beam_candidate & b) {
5360-
return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
5360+
if (a.sequence.sum_logprobs_all != b.sequence.sum_logprobs_all) {
5361+
return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
5362+
}
5363+
return a.decoder_idx < b.decoder_idx;
53615364
});
53625365

53635366
uint32_t cur_c = 0;

0 commit comments

Comments
 (0)