Skip to content

Commit 01b9ecc

Browse files
committed
move to new file, add in different inference runs
1 parent 86ff638 commit 01b9ecc

File tree

1 file changed

+0
-67
lines changed

1 file changed

+0
-67
lines changed

src/deepsparse/transformers/README.md

-67
Original file line numberDiff line numberDiff line change
@@ -355,73 +355,6 @@ To learn more about benchmarking, refer to the appropriate documentation.
355355
Also, check out our [Benchmarking tutorial](https://github.com/neuralmagic/deepsparse/tree/main/src/deepsparse/benchmark)!
356356

357357

358-
<h1><summary>Text Generation GenerationConfig Features Supported </h1></summary>
359-
360-
<details>
361-
<h2> Parameters controlling the output length: </h2>
362-
363-
| Feature | Description | Deepsparse Default | HuggingFace Default | Supported |
364-
| :--- | :----: | :----: | :----: | ---:|
365-
| max_length | Maximum length of generated tokens. Equal to input_prompt + max_new_tokens. Overridden by max_new_tokens | 1024 | 20 | Yes|
366-
| max_new_tokens | Maximum number of tokens to generate, ignoring prompt tokens. | None | None | Yes |
367-
| min_length | Minimum length of generated tokens. Equal to input_prompt + min_new_tokens. Overridden by min_new_tokens | - | 0 | No
368-
| min_new_tokens | Minomum number of tokens to generate, ignoring prompt tokens. | - | None | No |
369-
| max_time | - | - | - | No |
370-
371-
<br/>
372-
<h2> Parameters for manipulation of the model output logits </h2>
373-
374-
| Feature | Description | Deepsparse Default | HuggingFace Default | Supported |
375-
| :--- | :----: | :----: | :----: | ---:|
376-
| top_k | The number of highest probability vocabulary tokens to keep for top-k-filtering | 0 | 50 | Yes
377-
| top_p | Keep the generated tokens where its cumulative probability is >= top_p | 0.0 | 1.0 | Yes
378-
| repetition_penalty | Penalty applied for generating new token. Existing token frequencies summed to subtraction the logit of its corresponding logit value | 0.0 | 1.0 | Yes |
379-
| temperature | The temperature to use when sampling from the probability distribution computed from the logits. Higher values will result in more random samples. Should be greater than 0.0 | 1.0 | 1.0 | Yes |
380-
| typical_p | - | - | - | No |
381-
| epsilon_cutoff | - | - | - | No |
382-
| eta_cutoff | - | - | - | No |
383-
| diversity_penalty | - | - | - | No |
384-
| length_penalty | - | - | - | No |
385-
| bad_words_ids | - | - | - | No |
386-
| force_words_ids | - | - | - | No |
387-
| renormalize_logits | - | - | - | No |
388-
| constraints | - | - | - | No |
389-
| forced_bos_token_id | - | - | - | No |
390-
| forced_eos_token_id | - | - | - | No |
391-
| remove_invalid_values | - | - | - | No |
392-
| exponential_decay_length_penalty | - | - | - | No |
393-
| suppress_tokens | - | - | - | No |
394-
| begin_suppress_tokens | - | - | - | No |
395-
| forced_decoder_ids | - | - | - | No |
396-
397-
<br/>
398-
<h2> Parameters that control the generation strategy used </h2>
399-
400-
| Feature | Description | Deepsparse Default | HuggingFace Default | Supported |
401-
| :--- | :----: | :----: | :----: | ---:|
402-
| do_sample | If True, will apply sampling from the probability distribution computed from the logits | False | False | Yes |
403-
404-
<br/>
405-
<h2> Parameters for output variables: </h2>
406-
407-
| Feature | Description | Deepsparse Default | HuggingFace Default | Supported |
408-
| :--- | :----: | :----: | :----: | ---:|
409-
| num_return_sequences | The number of sequences generated for each prompt | 1 | 1 | Yes |
410-
| output_scores | Whether to return the generated logits | False | False | Yes |
411-
| return_dict_generate | - | - | - | No |
412-
413-
<br/>
414-
<h2> Special Tokens: </h2>
415-
416-
| Feature | Description | Deepsparse Default | HuggingFace Default | Supported |
417-
| :--- | :----: | :----: | :----: | ---:|
418-
| pad_token_id | - | - | - | No |
419-
| bos_token_id | - | - | - | No |
420-
| eos_token_id | - | - | - | No |
421-
422-
</details>
423-
<br/>
424-
425358
## Tutorials:
426359
For a deeper dive into using transformers within the Neural Magic ecosystem, refer to the detailed tutorials on our [website](https://neuralmagic.com/):
427360
- [Token Classification: Named Entity Recognition](https://neuralmagic.com/use-cases/sparse-named-entity-recognition/)

0 commit comments

Comments
 (0)