Skip to content

Commit 9860194

Browse files
authored
Small fix for micro benchmark code (#711)
Summary: There seems to be some problems running benchmark_aq.py: ``` torch._dynamo.exc.Unsupported: torch.* op returned non-Tensor int call_function <method 'size' of 'torch._C.TensorBase' objects> ``` when we run the benchmark with multiple shapes sometimes. But the problem will be gone if we reset the dynamo caches before each benchmark run Test Plan: python benchmarks/benchmark_aq.py Reviewers: Subscribers: Tasks: Tags:
1 parent b09307a commit 9860194

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

benchmarks/benchmark_aq.py

+4-1
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
_replace_with_custom_fn_if_matches_filter,
1818
)
1919
import copy
20+
from torchao.utils import unwrap_tensor_subclass
2021

2122
def _int8wo_api(mod, **kwargs):
2223
if TORCH_VERSION_AT_LEAST_2_4:
@@ -133,15 +134,17 @@ def _bench_quantized_tensor_subclass_perf(api, ref_api, M, N, K, kwargs=None):
133134
WARMUP = 20
134135
RUNS = 100
135136

137+
torch._dynamo.reset()
136138
m_ref = torch.compile(m_ref, mode='max-autotune', fullgraph=True)
137139
benchmark_model(m_ref, WARMUP, example_inputs)
138140
ref_elapsed_time = benchmark_model(m_ref, RUNS, example_inputs)
139141

142+
torch._dynamo.reset()
140143
m = torch.compile(m, mode='max-autotune', fullgraph=True)
141144
benchmark_model(m, WARMUP, example_inputs)
142145
elapsed_time = benchmark_model(m, RUNS, example_inputs)
143146

144-
147+
torch._dynamo.reset()
145148
m_bf16 = torch.compile(m_bf16, mode='max-autotune', fullgraph=True)
146149
benchmark_model(m_bf16, WARMUP, example_inputs)
147150
bf16_elapsed_time = benchmark_model(m_bf16, RUNS, example_inputs)

0 commit comments

Comments
 (0)