Skip to content

Commit 64dfa45

Browse files
SamsonboadiSvetlana Karslioglu
and
Svetlana Karslioglu
committed
created original copy of the model by loading from disk (#2406)
* created original copy of the model by loading from disk * Update fx_graph_mode_ptq_dynamic.py --------- Co-authored-by: Svetlana Karslioglu <[email protected]>
1 parent 8966d94 commit 64dfa45

File tree

1 file changed

+21
-3
lines changed

1 file changed

+21
-3
lines changed

Diff for: prototype_source/fx_graph_mode_ptq_dynamic.py

+21-3
Original file line numberDiff line numberDiff line change
@@ -239,9 +239,27 @@ def evaluate(model_, data_source):
239239
.set_object_type(nn.LSTM, default_dynamic_qconfig)
240240
.set_object_type(nn.Linear, default_dynamic_qconfig)
241241
)
242-
# Deepcopying the original model because quantization api changes the model inplace and we want
242+
# Load model to create the original model because quantization api changes the model inplace and we want
243243
# to keep the original model for future comparison
244-
model_to_quantize = copy.deepcopy(model)
244+
245+
246+
model_to_quantize = LSTMModel(
247+
ntoken = ntokens,
248+
ninp = 512,
249+
nhid = 256,
250+
nlayers = 5,
251+
)
252+
253+
model_to_quantize.load_state_dict(
254+
torch.load(
255+
model_data_filepath + 'word_language_model_quantize.pth',
256+
map_location=torch.device('cpu')
257+
)
258+
)
259+
260+
model_to_quantize.eval()
261+
262+
245263
prepared_model = prepare_fx(model_to_quantize, qconfig_mapping, example_inputs)
246264
print("prepared model:", prepared_model)
247265
quantized_model = convert_fx(prepared_model)
@@ -289,4 +307,4 @@ def time_model_evaluation(model, test_data):
289307
# 3. Conclusion
290308
# -------------
291309
# This tutorial introduces the api for post training dynamic quantization in FX Graph Mode,
292-
# which dynamically quantizes the same modules as Eager Mode Quantization.
310+
# which dynamically quantizes the same modules as Eager Mode Quantization.

0 commit comments

Comments
 (0)