Skip to content

Commit 9e0a0f0

Browse files
mszhanyiYi Zhang
and
Yi Zhang
authored
Check whether required tests are executed. (microsoft#19884)
### Description Check the onnx node tests and model tests worked ### Motivation and Context onnx node test data and model data are mount in one dir. And onnxruntime_test_all search the dir and load the data. If the dir does exist or there's some change in onnxruntime_test_all, those tests may not be executed. For example, all onnx node test data is 32M. It's hardly for us aware of the regression. So I add the simple check to ensure those tests are executed. --------- Co-authored-by: Yi Zhang <[email protected]>
1 parent 7313aa4 commit 9e0a0f0

File tree

3 files changed

+31
-2
lines changed

3 files changed

+31
-2
lines changed

tools/ci_build/github/azure-pipelines/linux-ci-pipeline.yml

+7-2
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ stages:
8686
-e NIGHTLY_BUILD \
8787
-e BUILD_BUILDNUMBER \
8888
onnxruntimecpubuildcentos8x64 \
89-
/bin/bash -c "
89+
/bin/bash -c '
9090
set -ex; \
9191
python3.9 /onnxruntime_src/tools/ci_build/build.py \
9292
--build_dir /build --cmake_generator 'Ninja' \
@@ -105,7 +105,8 @@ stages:
105105
--parallel --use_binskim_compliant_compile_flags \
106106
--build_csharp \
107107
--enable_onnx_tests --enable_address_sanitizer \
108-
--test;"
108+
--test;
109+
'
109110
workingDirectory: $(Build.SourcesDirectory)
110111

111112
- task: PublishTestResults@2
@@ -285,6 +286,10 @@ stages:
285286
"
286287
displayName: 'Run Release tests and symbolic shape infer test'
287288
289+
- template: templates/check_test_result.yml
290+
parameters:
291+
FileName: '$(Build.BinariesDirectory)/Release/onnxruntime_test_all.Release.results.xml'
292+
288293
- task: PublishTestResults@2
289294
displayName: 'Publish unit test results'
290295
inputs:

tools/ci_build/github/azure-pipelines/linux-gpu-ci-pipeline.yml

+4
Original file line numberDiff line numberDiff line change
@@ -265,4 +265,8 @@ stages:
265265
'
266266
displayName: 'Run Tests'
267267

268+
- template: templates/check_test_result.yml
269+
parameters:
270+
FileName: '$(Build.BinariesDirectory)/Release/onnxruntime_test_all.Release.results.xml'
271+
268272
- template: templates/clean-agent-build-directory-step.yml
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
parameters:
2+
- name: FileName
3+
type: string
4+
5+
steps:
6+
- task: UsePythonVersion@0
7+
inputs:
8+
versionSpec: '3.x'
9+
addToPath: true
10+
architecture: 'x64'
11+
12+
- task: PythonScript@0
13+
displayName: 'Check test result yml'
14+
inputs:
15+
scriptSource: 'inline'
16+
script: |
17+
with open('${{parameters.FileName}}', 'r') as file:
18+
content = file.read()
19+
assert 'data_onnx_opset' in content, "operator test not found in test result file"
20+
assert 'models_zoo_opset' in content, "models_zoo model not found in test reuslt file"

0 commit comments

Comments
 (0)