Skip to content

Fix JMS flaky tests #7551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Sep 3, 2024
Merged

Fix JMS flaky tests #7551

merged 1 commit into from
Sep 3, 2024

Conversation

smola
Copy link
Member

@smola smola commented Sep 3, 2024 •
edited
Loading

What Does This Do

Some tests seem to be leaking messages to queues to the next test. Make sure each test uses unique topics/queues.

Motivation

Additional Notes

Contributor Checklist

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes tag: flaky test Flaky tests labels Sep 3, 2024
@pr-commenter
Copy link

pr-commenter bot commented Sep 3, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/jms-flaky
git_commit_date 1725274779 1725344132
git_commit_sha 352673a 0f230b3
release_version 1.39.0-SNAPSHOT~352673a4d2 1.40.0-SNAPSHOT~0f230b38c7
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1725346548 1725346548
ci_job_id 624980481 624980481
ci_pipeline_id 43383096 43383096
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 16 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~0f230b38c7, baseline=1.39.0-SNAPSHOT~352673a4d2

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057274
Total [baseline] (10.344 s) : 0, 10344421
Agent [candidate] (1.051 s) : 0, 1051268
Total [candidate] (10.376 s) : 0, 10376236
section appsec
Agent [baseline] (1.18 s) : 0, 1180276
Total [baseline] (10.556 s) : 0, 10556243
Agent [candidate] (1.178 s) : 0, 1178352
Total [candidate] (10.569 s) : 0, 10569474
section iast
Agent [baseline] (1.181 s) : 0, 1180719
Total [baseline] (10.775 s) : 0, 10775259
Agent [candidate] (1.2 s) : 0, 1200005
Total [candidate] (10.854 s) : 0, 10854473
section profiling
Agent [baseline] (1.246 s) : 0, 1245512
Total [baseline] (10.561 s) : 0, 10560901
Agent [candidate] (1.244 s) : 0, 1244052
Total [candidate] (10.586 s) : 0, 10586041
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.18 s 123.002 ms (11.6%)
Agent iast 1.181 s 123.445 ms (11.7%)
Agent profiling 1.246 s 188.238 ms (17.8%)
Total tracing 10.344 s -
Total appsec 10.556 s 211.822 ms (2.0%)
Total iast 10.775 s 430.838 ms (4.2%)
Total profiling 10.561 s 216.48 ms (2.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.051 s -
Agent appsec 1.178 s 127.083 ms (12.1%)
Agent iast 1.2 s 148.737 ms (14.1%)
Agent profiling 1.244 s 192.784 ms (18.3%)
Total tracing 10.376 s -
Total appsec 10.569 s 193.237 ms (1.9%)
Total iast 10.854 s 478.236 ms (4.6%)
Total profiling 10.586 s 209.805 ms (2.0%) 
gantt
    title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~0f230b38c7, baseline=1.39.0-SNAPSHOT~352673a4d2

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (674.693 ms) : 0, 674693
BytebuddyAgent [candidate] (670.31 ms) : 0, 670310
GlobalTracer [baseline] (308.898 ms) : 0, 308898
GlobalTracer [candidate] (307.606 ms) : 0, 307606
AppSec [baseline] (51.809 ms) : 0, 51809
AppSec [candidate] (51.486 ms) : 0, 51486
Remote Config [baseline] (671.299 µs) : 0, 671
Remote Config [candidate] (674.278 µs) : 0, 674
Telemetry [baseline] (7.501 ms) : 0, 7501
Telemetry [candidate] (7.56 ms) : 0, 7560
section appsec
BytebuddyAgent [baseline] (688.553 ms) : 0, 688553
BytebuddyAgent [candidate] (686.903 ms) : 0, 686903
GlobalTracer [baseline] (299.944 ms) : 0, 299944
GlobalTracer [candidate] (299.54 ms) : 0, 299540
AppSec [baseline] (158.465 ms) : 0, 158465
AppSec [candidate] (158.994 ms) : 0, 158994
Remote Config [baseline] (611.185 µs) : 0, 611
Remote Config [candidate] (615.014 µs) : 0, 615
Telemetry [baseline] (8.528 ms) : 0, 8528
Telemetry [candidate] (8.903 ms) : 0, 8903
IAST [baseline] (21.25 ms) : 0, 21250
IAST [candidate] (19.652 ms) : 0, 19652
section iast
BytebuddyAgent [baseline] (785.458 ms) : 0, 785458
BytebuddyAgent [candidate] (798.434 ms) : 0, 798434
GlobalTracer [baseline] (297.659 ms) : 0, 297659
GlobalTracer [candidate] (302.406 ms) : 0, 302406
AppSec [baseline] (52.243 ms) : 0, 52243
AppSec [candidate] (54.286 ms) : 0, 54286
Remote Config [baseline] (601.369 µs) : 0, 601
Remote Config [candidate] (609.108 µs) : 0, 609
Telemetry [baseline] (8.203 ms) : 0, 8203
Telemetry [candidate] (8.155 ms) : 0, 8155
IAST [baseline] (22.867 ms) : 0, 22867
IAST [candidate] (22.216 ms) : 0, 22216
section profiling
ProfilingAgent [baseline] (95.724 ms) : 0, 95724
ProfilingAgent [candidate] (95.534 ms) : 0, 95534
BytebuddyAgent [baseline] (663.416 ms) : 0, 663416
BytebuddyAgent [candidate] (662.609 ms) : 0, 662609
GlobalTracer [baseline] (388.651 ms) : 0, 388651
GlobalTracer [candidate] (388.132 ms) : 0, 388132
AppSec [baseline] (51.882 ms) : 0, 51882
AppSec [candidate] (52.056 ms) : 0, 52056
Remote Config [baseline] (683.561 µs) : 0, 684
Remote Config [candidate] (686.158 µs) : 0, 686
Telemetry [baseline] (7.362 ms) : 0, 7362
Telemetry [candidate] (7.312 ms) : 0, 7312
Profiling [baseline] (95.747 ms) : 0, 95747
Profiling [candidate] (95.559 ms) : 0, 95559
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~0f230b38c7, baseline=1.39.0-SNAPSHOT~352673a4d2

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1056631
Total [baseline] (8.496 s) : 0, 8496472
Agent [candidate] (1.049 s) : 0, 1049478
Total [candidate] (8.495 s) : 0, 8495152
section iast
Agent [baseline] (1.175 s) : 0, 1174516
Total [baseline] (8.987 s) : 0, 8987459
Agent [candidate] (1.173 s) : 0, 1172991
Total [candidate] (8.977 s) : 0, 8977471
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.173 s) : 0, 1173063
Total [baseline] (8.919 s) : 0, 8918975
Agent [candidate] (1.171 s) : 0, 1170994
Total [candidate] (8.911 s) : 0, 8910674
section iast_TELEMETRY_OFF
Agent [baseline] (1.178 s) : 0, 1177502
Total [baseline] (8.938 s) : 0, 8937671
Agent [candidate] (1.188 s) : 0, 1187790
Total [candidate] (8.963 s) : 0, 8963395
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent iast 1.175 s 117.885 ms (11.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.173 s 116.432 ms (11.0%)
Agent iast_TELEMETRY_OFF 1.178 s 120.871 ms (11.4%)
Total tracing 8.496 s -
Total iast 8.987 s 490.987 ms (5.8%)
Total iast_HARDCODED_SECRET_DISABLED 8.919 s 422.503 ms (5.0%)
Total iast_TELEMETRY_OFF 8.938 s 441.199 ms (5.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.049 s -
Agent iast 1.173 s 123.513 ms (11.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.171 s 121.516 ms (11.6%)
Agent iast_TELEMETRY_OFF 1.188 s 138.312 ms (13.2%)
Total tracing 8.495 s -
Total iast 8.977 s 482.32 ms (5.7%)
Total iast_HARDCODED_SECRET_DISABLED 8.911 s 415.522 ms (4.9%)
Total iast_TELEMETRY_OFF 8.963 s 468.243 ms (5.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~0f230b38c7, baseline=1.39.0-SNAPSHOT~352673a4d2

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (674.614 ms) : 0, 674614
BytebuddyAgent [candidate] (669.877 ms) : 0, 669877
GlobalTracer [baseline] (308.648 ms) : 0, 308648
GlobalTracer [candidate] (306.301 ms) : 0, 306301
AppSec [baseline] (51.563 ms) : 0, 51563
AppSec [candidate] (51.534 ms) : 0, 51534
Remote Config [baseline] (670.914 µs) : 0, 671
Remote Config [candidate] (665.908 µs) : 0, 666
Telemetry [baseline] (7.455 ms) : 0, 7455
Telemetry [candidate] (7.509 ms) : 0, 7509
section iast
BytebuddyAgent [baseline] (780.255 ms) : 0, 780255
BytebuddyAgent [candidate] (779.398 ms) : 0, 779398
GlobalTracer [baseline] (295.817 ms) : 0, 295817
GlobalTracer [candidate] (296.123 ms) : 0, 296123
AppSec [baseline] (51.491 ms) : 0, 51491
AppSec [candidate] (52.499 ms) : 0, 52499
IAST [baseline] (25.389 ms) : 0, 25389
IAST [candidate] (20.348 ms) : 0, 20348
Remote Config [baseline] (586.781 µs) : 0, 587
Remote Config [candidate] (592.962 µs) : 0, 593
Telemetry [baseline] (7.348 ms) : 0, 7348
Telemetry [candidate] (10.412 ms) : 0, 10412
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (779.424 ms) : 0, 779424
BytebuddyAgent [candidate] (778.076 ms) : 0, 778076
GlobalTracer [baseline] (296.193 ms) : 0, 296193
GlobalTracer [candidate] (295.364 ms) : 0, 295364
AppSec [baseline] (50.699 ms) : 0, 50699
AppSec [candidate] (49.643 ms) : 0, 49643
IAST [baseline] (24.456 ms) : 0, 24456
IAST [candidate] (24.103 ms) : 0, 24103
Remote Config [baseline] (590.509 µs) : 0, 591
Remote Config [candidate] (581.873 µs) : 0, 582
Telemetry [baseline] (8.078 ms) : 0, 8078
Telemetry [candidate] (9.618 ms) : 0, 9618
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (782.06 ms) : 0, 782060
BytebuddyAgent [candidate] (789.42 ms) : 0, 789420
GlobalTracer [baseline] (297.73 ms) : 0, 297730
GlobalTracer [candidate] (299.952 ms) : 0, 299952
AppSec [baseline] (53.166 ms) : 0, 53166
AppSec [candidate] (55.732 ms) : 0, 55732
IAST [baseline] (22.967 ms) : 0, 22967
IAST [candidate] (20.211 ms) : 0, 20211
Remote Config [baseline] (574.52 µs) : 0, 575
Remote Config [candidate] (589.003 µs) : 0, 589
Telemetry [baseline] (7.268 ms) : 0, 7268
Telemetry [candidate] (8.054 ms) : 0, 8054
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-03T06:26:34 2024-09-03T06:33:23
git_branch master smola/jms-flaky
git_commit_date 1725274779 1725344132
git_commit_sha 352673a 0f230b3
release_version 1.39.0-SNAPSHOT~352673a4d2 1.40.0-SNAPSHOT~0f230b38c7
start_time 2024-09-03T06:26:20 2024-09-03T06:33:09
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1725345550 1725345550
ci_job_id 624980482 624980482
ci_pipeline_id 43383096 43383096
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 11 metrics, 16 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling worse
[+32.215µs; +83.672µs] or [+2.173%; +5.644%]		 unstable
[-813.991op/s; +339.917op/s] or [-25.437%; +10.622%]		 1.540ms 2962.963op/s 1.482ms 3200.000op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~0f230b38c7, baseline=1.39.0-SNAPSHOT~352673a4d2
    dateFormat X
    axisFormat %s
section baseline
no_agent (373.254 µs) : 354, 393
.   : milestone, 373,
iast (481.53 µs) : 459, 504
.   : milestone, 482,
iast_FULL (560.216 µs) : 539, 581
.   : milestone, 560,
iast_GLOBAL (512.183 µs) : 491, 533
.   : milestone, 512,
iast_HARDCODED_SECRET_DISABLED (490.791 µs) : 468, 513
.   : milestone, 491,
iast_INACTIVE (448.468 µs) : 428, 469
.   : milestone, 448,
iast_TELEMETRY_OFF (476.149 µs) : 454, 499
.   : milestone, 476,
tracing (448.087 µs) : 427, 469
.   : milestone, 448,
section candidate
no_agent (372.099 µs) : 351, 393
.   : milestone, 372,
iast (493.402 µs) : 471, 515
.   : milestone, 493,
iast_FULL (555.263 µs) : 534, 576
.   : milestone, 555,
iast_GLOBAL (513.865 µs) : 493, 535
.   : milestone, 514,
iast_HARDCODED_SECRET_DISABLED (496.246 µs) : 474, 519
.   : milestone, 496,
iast_INACTIVE (449.935 µs) : 429, 471
.   : milestone, 450,
iast_TELEMETRY_OFF (474.439 µs) : 452, 497
.   : milestone, 474,
tracing (445.69 µs) : 425, 466
.   : milestone, 446,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 373.254 µs [353.556 µs, 392.952 µs] -
iast 481.53 µs [459.32 µs, 503.741 µs] 108.277 µs (29.0%)
iast_FULL 560.216 µs [538.975 µs, 581.457 µs] 186.962 µs (50.1%)
iast_GLOBAL 512.183 µs [490.889 µs, 533.477 µs] 138.93 µs (37.2%)
iast_HARDCODED_SECRET_DISABLED 490.791 µs [468.298 µs, 513.284 µs] 117.537 µs (31.5%)
iast_INACTIVE 448.468 µs [428.126 µs, 468.809 µs] 75.214 µs (20.2%)
iast_TELEMETRY_OFF 476.149 µs [453.534 µs, 498.764 µs] 102.896 µs (27.6%)
tracing 448.087 µs [427.164 µs, 469.009 µs] 74.833 µs (20.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 372.099 µs [351.151 µs, 393.047 µs] -
iast 493.402 µs [471.321 µs, 515.483 µs] 121.303 µs (32.6%)
iast_FULL 555.263 µs [534.217 µs, 576.309 µs] 183.164 µs (49.2%)
iast_GLOBAL 513.865 µs [492.594 µs, 535.136 µs] 141.766 µs (38.1%)
iast_HARDCODED_SECRET_DISABLED 496.246 µs [473.615 µs, 518.876 µs] 124.147 µs (33.4%)
iast_INACTIVE 449.935 µs [428.733 µs, 471.138 µs] 77.836 µs (20.9%)
iast_TELEMETRY_OFF 474.439 µs [451.82 µs, 497.058 µs] 102.34 µs (27.5%)
tracing 445.69 µs [425.007 µs, 466.373 µs] 73.591 µs (19.8%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~0f230b38c7, baseline=1.39.0-SNAPSHOT~352673a4d2
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.336 ms) : 1316, 1356
.   : milestone, 1336,
appsec (1.712 ms) : 1687, 1736
.   : milestone, 1712,
appsec_no_iast (1.741 ms) : 1718, 1765
.   : milestone, 1741,
iast (1.469 ms) : 1447, 1491
.   : milestone, 1469,
profiling (1.482 ms) : 1460, 1505
.   : milestone, 1482,
tracing (1.468 ms) : 1443, 1492
.   : milestone, 1468,
section candidate
no_agent (1.34 ms) : 1321, 1359
.   : milestone, 1340,
appsec (1.735 ms) : 1710, 1760
.   : milestone, 1735,
appsec_no_iast (1.731 ms) : 1707, 1755
.   : milestone, 1731,
iast (1.487 ms) : 1464, 1510
.   : milestone, 1487,
profiling (1.54 ms) : 1515, 1566
.   : milestone, 1540,
tracing (1.48 ms) : 1456, 1504
.   : milestone, 1480,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.336 ms [1.316 ms, 1.356 ms] -
appsec 1.712 ms [1.687 ms, 1.736 ms] 375.779 µs (28.1%)
appsec_no_iast 1.741 ms [1.718 ms, 1.765 ms] 405.477 µs (30.4%)
iast 1.469 ms [1.447 ms, 1.491 ms] 133.191 µs (10.0%)
profiling 1.482 ms [1.46 ms, 1.505 ms] 146.587 µs (11.0%)
tracing 1.468 ms [1.443 ms, 1.492 ms] 131.699 µs (9.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.34 ms [1.321 ms, 1.359 ms] -
appsec 1.735 ms [1.71 ms, 1.76 ms] 395.144 µs (29.5%)
appsec_no_iast 1.731 ms [1.707 ms, 1.755 ms] 391.587 µs (29.2%)
iast 1.487 ms [1.464 ms, 1.51 ms] 147.187 µs (11.0%)
profiling 1.54 ms [1.515 ms, 1.566 ms] 200.863 µs (15.0%)
tracing 1.48 ms [1.456 ms, 1.504 ms] 140.382 µs (10.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/jms-flaky
git_commit_date 1725274779 1725344132
git_commit_sha 352673a 0f230b3
release_version 1.39.0-SNAPSHOT~352673a4d2 1.40.0-SNAPSHOT~0f230b38c7
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1725346077 1725346077
ci_job_id 624980483 624980483
ci_pipeline_id 43383096 43383096
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~0f230b38c7, baseline=1.39.0-SNAPSHOT~352673a4d2
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.019 s) : 15019000, 15019000
.   : milestone, 15019000,
appsec (15.285 s) : 15285000, 15285000
.   : milestone, 15285000,
iast (18.902 s) : 18902000, 18902000
.   : milestone, 18902000,
iast_GLOBAL (17.777 s) : 17777000, 17777000
.   : milestone, 17777000,
profiling (15.245 s) : 15245000, 15245000
.   : milestone, 15245000,
tracing (15.025 s) : 15025000, 15025000
.   : milestone, 15025000,
section candidate
no_agent (15.017 s) : 15017000, 15017000
.   : milestone, 15017000,
appsec (15.005 s) : 15005000, 15005000
.   : milestone, 15005000,
iast (18.712 s) : 18712000, 18712000
.   : milestone, 18712000,
iast_GLOBAL (18.034 s) : 18034000, 18034000
.   : milestone, 18034000,
profiling (15.945 s) : 15945000, 15945000
.   : milestone, 15945000,
tracing (15.112 s) : 15112000, 15112000
.   : milestone, 15112000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.019 s [15.019 s, 15.019 s] -
appsec 15.285 s [15.285 s, 15.285 s] 266.0 ms (1.8%)
iast 18.902 s [18.902 s, 18.902 s] 3.883 s (25.9%)
iast_GLOBAL 17.777 s [17.777 s, 17.777 s] 2.758 s (18.4%)
profiling 15.245 s [15.245 s, 15.245 s] 226.0 ms (1.5%)
tracing 15.025 s [15.025 s, 15.025 s] 6.0 ms (0.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.017 s [15.017 s, 15.017 s] -
appsec 15.005 s [15.005 s, 15.005 s] -12.0 ms (-0.1%)
iast 18.712 s [18.712 s, 18.712 s] 3.695 s (24.6%)
iast_GLOBAL 18.034 s [18.034 s, 18.034 s] 3.017 s (20.1%)
profiling 15.945 s [15.945 s, 15.945 s] 928.0 ms (6.2%)
tracing 15.112 s [15.112 s, 15.112 s] 95.0 ms (0.6%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~0f230b38c7, baseline=1.39.0-SNAPSHOT~352673a4d2
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.45 ms) : 1438, 1461
.   : milestone, 1450,
appsec (2.215 ms) : 2180, 2250
.   : milestone, 2215,
iast (1.963 ms) : 1921, 2006
.   : milestone, 1963,
iast_GLOBAL (2.015 ms) : 1971, 2058
.   : milestone, 2015,
profiling (1.841 ms) : 1806, 1876
.   : milestone, 1841,
tracing (1.823 ms) : 1791, 1855
.   : milestone, 1823,
section candidate
no_agent (1.452 ms) : 1441, 1463
.   : milestone, 1452,
appsec (2.22 ms) : 2185, 2255
.   : milestone, 2220,
iast (1.963 ms) : 1921, 2005
.   : milestone, 1963,
iast_GLOBAL (2.012 ms) : 1968, 2055
.   : milestone, 2012,
profiling (1.838 ms) : 1803, 1872
.   : milestone, 1838,
tracing (1.84 ms) : 1806, 1873
.   : milestone, 1840,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.45 ms [1.438 ms, 1.461 ms] -
appsec 2.215 ms [2.18 ms, 2.25 ms] 765.313 µs (52.8%)
iast 1.963 ms [1.921 ms, 2.006 ms] 513.534 µs (35.4%)
iast_GLOBAL 2.015 ms [1.971 ms, 2.058 ms] 564.932 µs (39.0%)
profiling 1.841 ms [1.806 ms, 1.876 ms] 391.173 µs (27.0%)
tracing 1.823 ms [1.791 ms, 1.855 ms] 373.422 µs (25.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.452 ms [1.441 ms, 1.463 ms] -
appsec 2.22 ms [2.185 ms, 2.255 ms] 767.862 µs (52.9%)
iast 1.963 ms [1.921 ms, 2.005 ms] 511.15 µs (35.2%)
iast_GLOBAL 2.012 ms [1.968 ms, 2.055 ms] 559.695 µs (38.5%)
profiling 1.838 ms [1.803 ms, 1.872 ms] 385.801 µs (26.6%)
tracing 1.84 ms [1.806 ms, 1.873 ms] 387.647 µs (26.7%)

@smola smola marked this pull request as ready for review September 3, 2024 08:46
@smola smola requested a review from a team as a code owner September 3, 2024 08:46
amarziali
amarziali approved these changes Sep 3, 2024
View reviewed changes
Copy link
Collaborator

@amarziali amarziali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gracias!

@smola smola merged commit 7b95d60 into master Sep 3, 2024
100 checks passed
@smola smola deleted the smola/jms-flaky branch September 3, 2024 09:17
@github-actions github-actions bot added this to the 1.40.0 milestone Sep 3, 2024
@smola smola mentioned this pull request Sep 13, 2024
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@amarziali amarziali amarziali approved these changes

Assignees
No one assigned
Labels
comp: testing Testing tag: flaky test Flaky tests tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.40.0
Development

Successfully merging this pull request may close these issues.
2 participants
@smola @amarziali