Skip to content

Commit a585c3e

Browse files
josharianiThalay
authored andcommitted
whisper : make beam candidate sort more stable (ggml-org#1943)
All else being otherwise equal, this encourages the beam candidate selection to re-use the same decoder, which slightly reduces the cache size. I wouldn't expect it to make much of a performance difference, but it helps when debug printing the cache and beam. Added as part of understanding ggml-org#1941.
1 parent 56cb094 commit a585c3e

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

whisper.cpp

+4-1
Original file line numberDiff line numberDiff line change
@@ -5357,7 +5357,10 @@ int whisper_full_with_state(
53575357
beam_candidates.begin(),
53585358
beam_candidates.end(),
53595359
[](const beam_candidate & a, const beam_candidate & b) {
5360-
return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
5360+
if (a.sequence.sum_logprobs_all != b.sequence.sum_logprobs_all) {
5361+
return a.sequence.sum_logprobs_all > b.sequence.sum_logprobs_all;
5362+
}
5363+
return a.decoder_idx < b.decoder_idx;
53615364
});
53625365

53635366
uint32_t cur_c = 0;

0 commit comments

Comments
 (0)