Skip to content

Fix smoke test #8337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Feb 5, 2025
Merged

Fix smoke test #8337

merged 1 commit into from
Feb 5, 2025

Conversation

jpbempel
Copy link
Member

@jpbempel jpbempel commented Feb 4, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@pr-commenter
Copy link

pr-commenter bot commented Feb 4, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-flaky-smoke-test
git_commit_date 1738688431 1738693211
git_commit_sha 312af33 ab09a20
release_version 1.47.0-SNAPSHOT~312af335ab 1.47.0-SNAPSHOT~ab09a2059c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1738701191 1738701191
ci_job_id 793985235 793985235
ci_pipeline_id 54891021 54891021
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-dpmxkxx9-project-304-concurrent-2-ic1q6323 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-dpmxkxx9-project-304-concurrent-2-ic1q6323 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 11 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:tracing:Remote Config better
[-56.367µs; -15.490µs] or [-7.592%; -2.086%]		 706.529µs 742.458µs
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.47.0-SNAPSHOT~ab09a2059c, baseline=1.47.0-SNAPSHOT~312af335ab

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.042 s) : 0, 1042221
Total [baseline] (10.479 s) : 0, 10478778
Agent [candidate] (1.043 s) : 0, 1043391
Total [candidate] (10.552 s) : 0, 10551599
section appsec
Agent [baseline] (1.182 s) : 0, 1181529
Total [baseline] (10.704 s) : 0, 10704412
Agent [candidate] (1.181 s) : 0, 1180547
Total [candidate] (10.708 s) : 0, 10708201
section iast
Agent [baseline] (1.175 s) : 0, 1174861
Total [baseline] (10.925 s) : 0, 10925300
Agent [candidate] (1.172 s) : 0, 1172220
Total [candidate] (10.927 s) : 0, 10927309
section profiling
Agent [baseline] (1.263 s) : 0, 1263002
Total [baseline] (10.857 s) : 0, 10857412
Agent [candidate] (1.266 s) : 0, 1266365
Total [candidate] (10.934 s) : 0, 10933697
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.042 s -
Agent appsec 1.182 s 139.308 ms (13.4%)
Agent iast 1.175 s 132.64 ms (12.7%)
Agent profiling 1.263 s 220.781 ms (21.2%)
Total tracing 10.479 s -
Total appsec 10.704 s 225.633 ms (2.2%)
Total iast 10.925 s 446.521 ms (4.3%)
Total profiling 10.857 s 378.633 ms (3.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.043 s -
Agent appsec 1.181 s 137.157 ms (13.1%)
Agent iast 1.172 s 128.829 ms (12.3%)
Agent profiling 1.266 s 222.974 ms (21.4%)
Total tracing 10.552 s -
Total appsec 10.708 s 156.602 ms (1.5%)
Total iast 10.927 s 375.71 ms (3.6%)
Total profiling 10.934 s 382.098 ms (3.6%) 
gantt
    title petclinic - break down per module: candidate=1.47.0-SNAPSHOT~ab09a2059c, baseline=1.47.0-SNAPSHOT~312af335ab

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (715.481 ms) : 0, 715481
BytebuddyAgent [candidate] (718.731 ms) : 0, 718731
GlobalTracer [baseline] (243.45 ms) : 0, 243450
GlobalTracer [candidate] (242.826 ms) : 0, 242826
AppSec [baseline] (55.252 ms) : 0, 55252
AppSec [candidate] (55.108 ms) : 0, 55108
Remote Config [baseline] (742.458 µs) : 0, 742
Remote Config [candidate] (706.529 µs) : 0, 707
Telemetry [baseline] (12.137 ms) : 0, 12137
Telemetry [candidate] (10.718 ms) : 0, 10718
section appsec
BytebuddyAgent [baseline] (731.057 ms) : 0, 731057
BytebuddyAgent [candidate] (731.135 ms) : 0, 731135
GlobalTracer [baseline] (239.829 ms) : 0, 239829
GlobalTracer [candidate] (239.584 ms) : 0, 239584
AppSec [baseline] (175.405 ms) : 0, 175405
AppSec [candidate] (174.918 ms) : 0, 174918
Remote Config [baseline] (650.034 µs) : 0, 650
Remote Config [candidate] (661.095 µs) : 0, 661
Telemetry [baseline] (8.572 ms) : 0, 8572
Telemetry [candidate] (8.264 ms) : 0, 8264
IAST [baseline] (21.64 ms) : 0, 21640
IAST [candidate] (21.663 ms) : 0, 21663
section iast
BytebuddyAgent [baseline] (835.923 ms) : 0, 835923
BytebuddyAgent [candidate] (834.053 ms) : 0, 834053
GlobalTracer [baseline] (234.119 ms) : 0, 234119
GlobalTracer [candidate] (233.845 ms) : 0, 233845
AppSec [baseline] (55.865 ms) : 0, 55865
AppSec [candidate] (55.693 ms) : 0, 55693
Remote Config [baseline] (626.223 µs) : 0, 626
Remote Config [candidate] (630.092 µs) : 0, 630
Telemetry [baseline] (8.805 ms) : 0, 8805
Telemetry [candidate] (8.76 ms) : 0, 8760
IAST [baseline] (24.221 ms) : 0, 24221
IAST [candidate] (24.043 ms) : 0, 24043
section profiling
ProfilingAgent [baseline] (95.406 ms) : 0, 95406
ProfilingAgent [candidate] (97.047 ms) : 0, 97047
BytebuddyAgent [baseline] (706.439 ms) : 0, 706439
BytebuddyAgent [candidate] (706.648 ms) : 0, 706648
GlobalTracer [baseline] (353.989 ms) : 0, 353989
GlobalTracer [candidate] (355.694 ms) : 0, 355694
AppSec [baseline] (55.137 ms) : 0, 55137
AppSec [candidate] (54.819 ms) : 0, 54819
Remote Config [baseline] (709.511 µs) : 0, 710
Remote Config [candidate] (735.673 µs) : 0, 736
Telemetry [baseline] (9.001 ms) : 0, 9001
Telemetry [candidate] (9.061 ms) : 0, 9061
Profiling [baseline] (95.43 ms) : 0, 95430
Profiling [candidate] (97.071 ms) : 0, 97071
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.47.0-SNAPSHOT~ab09a2059c, baseline=1.47.0-SNAPSHOT~312af335ab

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.044 s) : 0, 1043970
Total [baseline] (8.668 s) : 0, 8668150
Agent [candidate] (1.046 s) : 0, 1046360
Total [candidate] (8.684 s) : 0, 8684476
section iast
Agent [baseline] (1.181 s) : 0, 1181186
Total [baseline] (9.245 s) : 0, 9244579
Agent [candidate] (1.188 s) : 0, 1187879
Total [candidate] (9.297 s) : 0, 9297122
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.173 s) : 0, 1172748
Total [baseline] (9.211 s) : 0, 9210965
Agent [candidate] (1.186 s) : 0, 1186173
Total [candidate] (9.204 s) : 0, 9203975
section iast_TELEMETRY_OFF
Agent [baseline] (1.168 s) : 0, 1167881
Total [baseline] (9.21 s) : 0, 9210291
Agent [candidate] (1.172 s) : 0, 1172324
Total [candidate] (9.231 s) : 0, 9231238
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.044 s -
Agent iast 1.181 s 137.216 ms (13.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.173 s 128.778 ms (12.3%)
Agent iast_TELEMETRY_OFF 1.168 s 123.911 ms (11.9%)
Total tracing 8.668 s -
Total iast 9.245 s 576.428 ms (6.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.211 s 542.814 ms (6.3%)
Total iast_TELEMETRY_OFF 9.21 s 542.141 ms (6.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.046 s -
Agent iast 1.188 s 141.519 ms (13.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.186 s 139.813 ms (13.4%)
Agent iast_TELEMETRY_OFF 1.172 s 125.964 ms (12.0%)
Total tracing 8.684 s -
Total iast 9.297 s 612.646 ms (7.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.204 s 519.499 ms (6.0%)
Total iast_TELEMETRY_OFF 9.231 s 546.762 ms (6.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.47.0-SNAPSHOT~ab09a2059c, baseline=1.47.0-SNAPSHOT~312af335ab

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (716.714 ms) : 0, 716714
BytebuddyAgent [candidate] (718.407 ms) : 0, 718407
GlobalTracer [baseline] (243.636 ms) : 0, 243636
GlobalTracer [candidate] (245.528 ms) : 0, 245528
AppSec [baseline] (55.314 ms) : 0, 55314
AppSec [candidate] (56.09 ms) : 0, 56090
Remote Config [baseline] (723.909 µs) : 0, 724
Remote Config [candidate] (732.528 µs) : 0, 733
Telemetry [baseline] (12.409 ms) : 0, 12409
Telemetry [candidate] (10.249 ms) : 0, 10249
section iast
BytebuddyAgent [baseline] (841.393 ms) : 0, 841393
BytebuddyAgent [candidate] (846.287 ms) : 0, 846287
GlobalTracer [baseline] (235.252 ms) : 0, 235252
GlobalTracer [candidate] (236.237 ms) : 0, 236237
AppSec [baseline] (56.196 ms) : 0, 56196
AppSec [candidate] (52.54 ms) : 0, 52540
Remote Config [baseline] (629.152 µs) : 0, 629
Remote Config [candidate] (636.662 µs) : 0, 637
Telemetry [baseline] (8.774 ms) : 0, 8774
Telemetry [candidate] (8.903 ms) : 0, 8903
IAST [baseline] (23.554 ms) : 0, 23554
IAST [candidate] (27.769 ms) : 0, 27769
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (834.36 ms) : 0, 834360
BytebuddyAgent [candidate] (844.979 ms) : 0, 844979
GlobalTracer [baseline] (233.821 ms) : 0, 233821
GlobalTracer [candidate] (235.805 ms) : 0, 235805
AppSec [baseline] (55.611 ms) : 0, 55611
AppSec [candidate] (55.309 ms) : 0, 55309
Remote Config [baseline] (636.215 µs) : 0, 636
Remote Config [candidate] (617.12 µs) : 0, 617
Telemetry [baseline] (8.731 ms) : 0, 8731
Telemetry [candidate] (8.821 ms) : 0, 8821
IAST [baseline] (24.348 ms) : 0, 24348
IAST [candidate] (25.235 ms) : 0, 25235
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (830.606 ms) : 0, 830606
BytebuddyAgent [candidate] (835.044 ms) : 0, 835044
GlobalTracer [baseline] (233.29 ms) : 0, 233290
GlobalTracer [candidate] (233.731 ms) : 0, 233731
AppSec [baseline] (55.794 ms) : 0, 55794
AppSec [candidate] (54.623 ms) : 0, 54623
Remote Config [baseline] (618.538 µs) : 0, 619
Remote Config [candidate] (621.469 µs) : 0, 621
Telemetry [baseline] (8.684 ms) : 0, 8684
Telemetry [candidate] (8.651 ms) : 0, 8651
IAST [baseline] (23.667 ms) : 0, 23667
IAST [candidate] (24.374 ms) : 0, 24374
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-02-04T20:03:10 2025-02-04T20:10:12
git_branch master jpbempel/fix-flaky-smoke-test
git_commit_date 1738688431 1738693211
git_commit_sha 312af33 ab09a20
release_version 1.47.0-SNAPSHOT~312af335ab 1.47.0-SNAPSHOT~ab09a2059c
start_time 2025-02-04T20:02:56 2025-02-04T20:09:59
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1738700169 1738700169
ci_job_id 793985237 793985237
ci_pipeline_id 54891021 54891021
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-fvygfmf-project-304-concurrent-1-9ycyvo60 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-fvygfmf-project-304-concurrent-1-9ycyvo60 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~ab09a2059c, baseline=1.47.0-SNAPSHOT~312af335ab
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.342 ms) : 1323, 1362
.   : milestone, 1342,
appsec (1.742 ms) : 1718, 1766
.   : milestone, 1742,
appsec_no_iast (1.751 ms) : 1726, 1775
.   : milestone, 1751,
iast (1.501 ms) : 1476, 1526
.   : milestone, 1501,
profiling (1.512 ms) : 1489, 1535
.   : milestone, 1512,
tracing (1.488 ms) : 1463, 1514
.   : milestone, 1488,
section candidate
no_agent (1.341 ms) : 1321, 1361
.   : milestone, 1341,
appsec (1.759 ms) : 1737, 1782
.   : milestone, 1759,
appsec_no_iast (1.746 ms) : 1721, 1771
.   : milestone, 1746,
iast (1.502 ms) : 1478, 1526
.   : milestone, 1502,
profiling (1.503 ms) : 1479, 1526
.   : milestone, 1503,
tracing (1.463 ms) : 1438, 1488
.   : milestone, 1463,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.342 ms [1.323 ms, 1.362 ms] -
appsec 1.742 ms [1.718 ms, 1.766 ms] 399.562 µs (29.8%)
appsec_no_iast 1.751 ms [1.726 ms, 1.775 ms] 408.247 µs (30.4%)
iast 1.501 ms [1.476 ms, 1.526 ms] 158.71 µs (11.8%)
profiling 1.512 ms [1.489 ms, 1.535 ms] 169.695 µs (12.6%)
tracing 1.488 ms [1.463 ms, 1.514 ms] 146.105 µs (10.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.341 ms [1.321 ms, 1.361 ms] -
appsec 1.759 ms [1.737 ms, 1.782 ms] 418.182 µs (31.2%)
appsec_no_iast 1.746 ms [1.721 ms, 1.771 ms] 405.124 µs (30.2%)
iast 1.502 ms [1.478 ms, 1.526 ms] 160.925 µs (12.0%)
profiling 1.503 ms [1.479 ms, 1.526 ms] 161.824 µs (12.1%)
tracing 1.463 ms [1.438 ms, 1.488 ms] 122.018 µs (9.1%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~ab09a2059c, baseline=1.47.0-SNAPSHOT~312af335ab
    dateFormat X
    axisFormat %s
section baseline
no_agent (376.76 µs) : 356, 398
.   : milestone, 377,
iast (496.25 µs) : 475, 518
.   : milestone, 496,
iast_FULL (740.5 µs) : 719, 762
.   : milestone, 740,
iast_GLOBAL (550.716 µs) : 528, 573
.   : milestone, 551,
iast_HARDCODED_SECRET_DISABLED (499.425 µs) : 478, 521
.   : milestone, 499,
iast_INACTIVE (458.113 µs) : 437, 479
.   : milestone, 458,
iast_TELEMETRY_OFF (495.736 µs) : 473, 519
.   : milestone, 496,
tracing (452.57 µs) : 431, 474
.   : milestone, 453,
section candidate
no_agent (382.861 µs) : 363, 403
.   : milestone, 383,
iast (507.108 µs) : 485, 529
.   : milestone, 507,
iast_FULL (737.191 µs) : 715, 759
.   : milestone, 737,
iast_GLOBAL (552.613 µs) : 531, 574
.   : milestone, 553,
iast_HARDCODED_SECRET_DISABLED (509.773 µs) : 487, 532
.   : milestone, 510,
iast_INACTIVE (456.671 µs) : 435, 479
.   : milestone, 457,
iast_TELEMETRY_OFF (492.877 µs) : 470, 516
.   : milestone, 493,
tracing (455.051 µs) : 434, 476
.   : milestone, 455,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 376.76 µs [355.831 µs, 397.689 µs] -
iast 496.25 µs [474.85 µs, 517.651 µs] 119.49 µs (31.7%)
iast_FULL 740.5 µs [718.727 µs, 762.272 µs] 363.74 µs (96.5%)
iast_GLOBAL 550.716 µs [528.418 µs, 573.014 µs] 173.956 µs (46.2%)
iast_HARDCODED_SECRET_DISABLED 499.425 µs [477.968 µs, 520.883 µs] 122.665 µs (32.6%)
iast_INACTIVE 458.113 µs [436.753 µs, 479.473 µs] 81.353 µs (21.6%)
iast_TELEMETRY_OFF 495.736 µs [472.571 µs, 518.9 µs] 118.976 µs (31.6%)
tracing 452.57 µs [431.341 µs, 473.799 µs] 75.81 µs (20.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 382.861 µs [363.002 µs, 402.721 µs] -
iast 507.108 µs [485.139 µs, 529.077 µs] 124.247 µs (32.5%)
iast_FULL 737.191 µs [715.355 µs, 759.028 µs] 354.33 µs (92.5%)
iast_GLOBAL 552.613 µs [530.873 µs, 574.352 µs] 169.751 µs (44.3%)
iast_HARDCODED_SECRET_DISABLED 509.773 µs [487.263 µs, 532.282 µs] 126.911 µs (33.1%)
iast_INACTIVE 456.671 µs [434.683 µs, 478.66 µs] 73.81 µs (19.3%)
iast_TELEMETRY_OFF 492.877 µs [470.231 µs, 515.523 µs] 110.015 µs (28.7%)
tracing 455.051 µs [433.672 µs, 476.43 µs] 72.19 µs (18.9%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-flaky-smoke-test
git_commit_date 1738688431 1738693211
git_commit_sha 312af33 ab09a20
release_version 1.47.0-SNAPSHOT~312af335ab 1.47.0-SNAPSHOT~ab09a2059c
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1738700711 1738700711
ci_job_id 793985239 793985239
ci_pipeline_id 54891021 54891021
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-dpmxkxx9-project-304-concurrent-3-ja5n21ix 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-dpmxkxx9-project-304-concurrent-3-ja5n21ix 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~ab09a2059c, baseline=1.47.0-SNAPSHOT~312af335ab
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.899 s) : 14899000, 14899000
.   : milestone, 14899000,
appsec (15.146 s) : 15146000, 15146000
.   : milestone, 15146000,
iast (18.573 s) : 18573000, 18573000
.   : milestone, 18573000,
iast_GLOBAL (17.996 s) : 17996000, 17996000
.   : milestone, 17996000,
profiling (15.274 s) : 15274000, 15274000
.   : milestone, 15274000,
tracing (15.062 s) : 15062000, 15062000
.   : milestone, 15062000,
section candidate
no_agent (14.91 s) : 14910000, 14910000
.   : milestone, 14910000,
appsec (15.015 s) : 15015000, 15015000
.   : milestone, 15015000,
iast (18.413 s) : 18413000, 18413000
.   : milestone, 18413000,
iast_GLOBAL (17.846 s) : 17846000, 17846000
.   : milestone, 17846000,
profiling (15.557 s) : 15557000, 15557000
.   : milestone, 15557000,
tracing (15.022 s) : 15022000, 15022000
.   : milestone, 15022000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.899 s [14.899 s, 14.899 s] -
appsec 15.146 s [15.146 s, 15.146 s] 247.0 ms (1.7%)
iast 18.573 s [18.573 s, 18.573 s] 3.674 s (24.7%)
iast_GLOBAL 17.996 s [17.996 s, 17.996 s] 3.097 s (20.8%)
profiling 15.274 s [15.274 s, 15.274 s] 375.0 ms (2.5%)
tracing 15.062 s [15.062 s, 15.062 s] 163.0 ms (1.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.91 s [14.91 s, 14.91 s] -
appsec 15.015 s [15.015 s, 15.015 s] 105.0 ms (0.7%)
iast 18.413 s [18.413 s, 18.413 s] 3.503 s (23.5%)
iast_GLOBAL 17.846 s [17.846 s, 17.846 s] 2.936 s (19.7%)
profiling 15.557 s [15.557 s, 15.557 s] 647.0 ms (4.3%)
tracing 15.022 s [15.022 s, 15.022 s] 112.0 ms (0.8%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~ab09a2059c, baseline=1.47.0-SNAPSHOT~312af335ab
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.477 ms) : 1465, 1488
.   : milestone, 1477,
appsec (2.367 ms) : 2323, 2410
.   : milestone, 2367,
iast (2.126 ms) : 2071, 2181
.   : milestone, 2126,
iast_GLOBAL (2.164 ms) : 2109, 2219
.   : milestone, 2164,
profiling (1.975 ms) : 1931, 2019
.   : milestone, 1975,
tracing (1.954 ms) : 1912, 1997
.   : milestone, 1954,
section candidate
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (2.369 ms) : 2326, 2413
.   : milestone, 2369,
iast (2.117 ms) : 2062, 2172
.   : milestone, 2117,
iast_GLOBAL (2.168 ms) : 2113, 2224
.   : milestone, 2168,
profiling (1.985 ms) : 1940, 2029
.   : milestone, 1985,
tracing (1.957 ms) : 1914, 1999
.   : milestone, 1957,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.477 ms [1.465 ms, 1.488 ms] -
appsec 2.367 ms [2.323 ms, 2.41 ms] 889.742 µs (60.2%)
iast 2.126 ms [2.071 ms, 2.181 ms] 648.994 µs (43.9%)
iast_GLOBAL 2.164 ms [2.109 ms, 2.219 ms] 687.069 µs (46.5%)
profiling 1.975 ms [1.931 ms, 2.019 ms] 498.172 µs (33.7%)
tracing 1.954 ms [1.912 ms, 1.997 ms] 477.295 µs (32.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.476 ms [1.464 ms, 1.487 ms] -
appsec 2.369 ms [2.326 ms, 2.413 ms] 893.537 µs (60.5%)
iast 2.117 ms [2.062 ms, 2.172 ms] 641.596 µs (43.5%)
iast_GLOBAL 2.168 ms [2.113 ms, 2.224 ms] 692.53 µs (46.9%)
profiling 1.985 ms [1.94 ms, 2.029 ms] 509.146 µs (34.5%)
tracing 1.957 ms [1.914 ms, 1.999 ms] 480.806 µs (32.6%)

@jpbempel jpbempel marked this pull request as ready for review February 5, 2025 15:57
@jpbempel jpbempel requested a review from a team as a code owner February 5, 2025 15:57
@jpbempel jpbempel requested review from shatzi and removed request for a team February 5, 2025 15:57
@jpbempel jpbempel added comp: debugger Dynamic Instrumentation type: bug tag: no release notes Changes to exclude from release notes labels Feb 5, 2025
@jpbempel jpbempel merged commit 88f59f8 into master Feb 5, 2025
200 of 202 checks passed
@jpbempel jpbempel deleted the jpbempel/fix-flaky-smoke-test branch February 5, 2025 17:03
@github-actions github-actions bot added this to the 1.47.0 milestone Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shatzi shatzi shatzi approved these changes

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation tag: no release notes Changes to exclude from release notes type: bug
Projects
None yet
Milestone
1.47.0
Development

Successfully merging this pull request may close these issues.
2 participants
@jpbempel @shatzi