Skip to content

Commit ce83296

Browse files
Update run.py for draft_target_model (#3615)
This change makes the draft target model works without mismatch in the vocab size Signed-off-by: mayani-nv <[email protected]> Co-authored-by: rakib-hasan <[email protected]>
1 parent ae56716 commit ce83296

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

examples/run.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -440,7 +440,7 @@ def main(args):
440440

441441
outputs = run_dtm_pld(batch_input_ids, args, runtime_rank, end_id,
442442
pad_id, stop_words_list, bad_words_list,
443-
tokenizer.vocab_size)
443+
len(tokenizer))
444444
if not args.streaming: # Unpack runner from the return value in No-Streaming mode
445445
outputs, runner = list(outputs)[0]
446446

0 commit comments

Comments
 (0)