Skip to content

Commit 3f9b7ab

Browse files
authored
[Doc] Update examples to remove SparseAutoModelForCausalLM (#12062)
Signed-off-by: Kyle Sayers <[email protected]>
1 parent ad34c0d commit 3f9b7ab

File tree

2 files changed

+8
-10
lines changed

2 files changed

+8
-10
lines changed

docs/source/features/quantization/fp8.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -54,16 +54,15 @@ The quantization process involves three main steps:
5454

5555
### 1. Loading the Model
5656

57-
Use `SparseAutoModelForCausalLM`, which wraps `AutoModelForCausalLM`, for saving and loading quantized models:
57+
Load your model and tokenizer using the standard `transformers` AutoModel classes:
5858

5959
```python
60-
from llmcompressor.transformers import SparseAutoModelForCausalLM
61-
from transformers import AutoTokenizer
60+
from transformers import AutoTokenizer, AutoModelForCausalLM
6261

6362
MODEL_ID = "meta-llama/Meta-Llama-3-8B-Instruct"
64-
65-
model = SparseAutoModelForCausalLM.from_pretrained(
66-
MODEL_ID, device_map="auto", torch_dtype="auto")
63+
model = AutoModelForCausalLM.from_pretrained(
64+
MODEL_ID, device_map="auto", torch_dtype="auto",
65+
)
6766
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
6867
```
6968

docs/source/features/quantization/int8.md

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -30,14 +30,13 @@ The quantization process involves four main steps:
3030

3131
### 1. Loading the Model
3232

33-
Use `SparseAutoModelForCausalLM`, which wraps `AutoModelForCausalLM`, for saving and loading quantized models:
33+
Load your model and tokenizer using the standard `transformers` AutoModel classes:
3434

3535
```python
36-
from llmcompressor.transformers import SparseAutoModelForCausalLM
37-
from transformers import AutoTokenizer
36+
from transformers import AutoTokenizer, AutoModelForCausalLM
3837

3938
MODEL_ID = "meta-llama/Meta-Llama-3-8B-Instruct"
40-
model = SparseAutoModelForCausalLM.from_pretrained(
39+
model = AutoModelForCausalLM.from_pretrained(
4140
MODEL_ID, device_map="auto", torch_dtype="auto",
4241
)
4342
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)

0 commit comments

Comments
 (0)