File tree 1 file changed +1
-1
lines changed
1 file changed +1
-1
lines changed Original file line number Diff line number Diff line change @@ -194,7 +194,7 @@ A few important things to consider when using the EAGLE based draft models:
194
194
be able to be loaded and used directly by vLLM after [ PR 12304] ( https://github.com/vllm-project/vllm/pull/12304 ) .
195
195
If you are using vllm version before [ PR 12304] ( https://github.com/vllm-project/vllm/pull/12304 ) , please use the
196
196
[ script] ( https://gist.github.com/abhigoyal1997/1e7a4109ccb7704fbc67f625e86b2d6d ) to convert the speculative model,
197
- and specify ` speculative_model="path/to/modified/eagle/model", ` . If weight-loading problems still occur when using
197
+ and specify ` speculative_model="path/to/modified/eagle/model" ` . If weight-loading problems still occur when using
198
198
the latest version of vLLM, please leave a comment or raise an issue.
199
199
200
200
2 . The EAGLE based draft models need to be run without tensor parallelism
You can’t perform that action at this time.
0 commit comments