Skip to content

Fix flaky test on TomcatServlet3TestAsync #7550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Sep 12, 2024
Merged

Fix flaky test on TomcatServlet3TestAsync #7550

merged 1 commit into from
Sep 12, 2024

Conversation

smola
Copy link
Member

@smola smola commented Sep 3, 2024 •
edited
Loading

What Does This Do

Under high load, the order of spans by duration might be altered. Make sure we check local root spans, and that we allow for different order in traces and access logs.

Motivation

Additional Notes

Contributor Checklist

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes tag: flaky test Flaky tests labels Sep 3, 2024
@pr-commenter
Copy link

pr-commenter bot commented Sep 3, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/tomcat-async-flaky
git_commit_date 1726062980 1726064446
git_commit_sha 4abf841 4f0abe1
release_version 1.40.0-SNAPSHOT~4abf841368 1.40.0-SNAPSHOT~4f0abe12a2
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1726066757 1726066757
ci_job_id 635568523 635568523
ci_pipeline_id 44081872 44081872
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 12 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.07 s) : 0, 1069702
Total [baseline] (10.333 s) : 0, 10333364
Agent [candidate] (1.071 s) : 0, 1070851
Total [candidate] (10.383 s) : 0, 10382745
section appsec
Agent [baseline] (1.194 s) : 0, 1194163
Total [baseline] (10.581 s) : 0, 10581449
Agent [candidate] (1.201 s) : 0, 1201266
Total [candidate] (10.596 s) : 0, 10595997
section iast
Agent [baseline] (1.189 s) : 0, 1189280
Total [baseline] (10.798 s) : 0, 10798165
Agent [candidate] (1.191 s) : 0, 1191377
Total [candidate] (10.838 s) : 0, 10838252
section profiling
Agent [baseline] (1.262 s) : 0, 1261836
Total [baseline] (10.642 s) : 0, 10642303
Agent [candidate] (1.27 s) : 0, 1270035
Total [candidate] (10.587 s) : 0, 10586594
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.07 s -
Agent appsec 1.194 s 124.461 ms (11.6%)
Agent iast 1.189 s 119.578 ms (11.2%)
Agent profiling 1.262 s 192.134 ms (18.0%)
Total tracing 10.333 s -
Total appsec 10.581 s 248.085 ms (2.4%)
Total iast 10.798 s 464.801 ms (4.5%)
Total profiling 10.642 s 308.94 ms (3.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.071 s -
Agent appsec 1.201 s 130.416 ms (12.2%)
Agent iast 1.191 s 120.526 ms (11.3%)
Agent profiling 1.27 s 199.185 ms (18.6%)
Total tracing 10.383 s -
Total appsec 10.596 s 213.252 ms (2.1%)
Total iast 10.838 s 455.507 ms (4.4%)
Total profiling 10.587 s 203.849 ms (2.0%) 
gantt
    title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (683.112 ms) : 0, 683112
BytebuddyAgent [candidate] (683.388 ms) : 0, 683388
GlobalTracer [baseline] (313.145 ms) : 0, 313145
GlobalTracer [candidate] (313.267 ms) : 0, 313267
AppSec [baseline] (51.565 ms) : 0, 51565
AppSec [candidate] (52.206 ms) : 0, 52206
Remote Config [baseline] (701.256 µs) : 0, 701
Remote Config [candidate] (714.873 µs) : 0, 715
Telemetry [baseline] (7.475 ms) : 0, 7475
Telemetry [candidate] (7.538 ms) : 0, 7538
section appsec
BytebuddyAgent [baseline] (697.998 ms) : 0, 697998
BytebuddyAgent [candidate] (701.593 ms) : 0, 701593
GlobalTracer [baseline] (303.654 ms) : 0, 303654
GlobalTracer [candidate] (305.051 ms) : 0, 305051
AppSec [baseline] (160.367 ms) : 0, 160367
AppSec [candidate] (161.15 ms) : 0, 161150
IAST [baseline] (19.807 ms) : 0, 19807
IAST [candidate] (20.167 ms) : 0, 20167
Remote Config [baseline] (641.269 µs) : 0, 641
Remote Config [candidate] (649.256 µs) : 0, 649
Telemetry [baseline] (7.998 ms) : 0, 7998
Telemetry [candidate] (9.044 ms) : 0, 9044
section iast
BytebuddyAgent [baseline] (791.569 ms) : 0, 791569
BytebuddyAgent [candidate] (791.289 ms) : 0, 791289
GlobalTracer [baseline] (299.499 ms) : 0, 299499
GlobalTracer [candidate] (301.19 ms) : 0, 301190
AppSec [baseline] (52.232 ms) : 0, 52232
AppSec [candidate] (51.923 ms) : 0, 51923
IAST [baseline] (23.563 ms) : 0, 23563
IAST [candidate] (25.358 ms) : 0, 25358
Remote Config [baseline] (1.387 ms) : 0, 1387
Remote Config [candidate] (614.379 µs) : 0, 614
Telemetry [baseline] (7.343 ms) : 0, 7343
Telemetry [candidate] (7.326 ms) : 0, 7326
section profiling
BytebuddyAgent [baseline] (673.132 ms) : 0, 673132
BytebuddyAgent [candidate] (678.268 ms) : 0, 678268
GlobalTracer [baseline] (394.044 ms) : 0, 394044
GlobalTracer [candidate] (396.984 ms) : 0, 396984
AppSec [baseline] (52.04 ms) : 0, 52040
AppSec [candidate] (52.534 ms) : 0, 52534
Remote Config [baseline] (697.467 µs) : 0, 697
Remote Config [candidate] (731.1 µs) : 0, 731
Telemetry [baseline] (7.404 ms) : 0, 7404
Telemetry [candidate] (7.45 ms) : 0, 7450
ProfilingAgent [baseline] (96.497 ms) : 0, 96497
ProfilingAgent [candidate] (95.736 ms) : 0, 95736
Profiling [baseline] (96.521 ms) : 0, 96521
Profiling [candidate] (95.761 ms) : 0, 95761
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063476
Total [baseline] (8.549 s) : 0, 8549165
Agent [candidate] (1.066 s) : 0, 1065991
Total [candidate] (8.557 s) : 0, 8556767
section iast
Agent [baseline] (1.187 s) : 0, 1187172
Total [baseline] (8.988 s) : 0, 8988144
Agent [candidate] (1.187 s) : 0, 1186866
Total [candidate] (8.973 s) : 0, 8972514
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.198 s) : 0, 1197530
Total [baseline] (8.998 s) : 0, 8998454
Agent [candidate] (1.206 s) : 0, 1205825
Total [candidate] (9.002 s) : 0, 9001669
section iast_TELEMETRY_OFF
Agent [baseline] (1.186 s) : 0, 1185599
Total [baseline] (8.982 s) : 0, 8981816
Agent [candidate] (1.194 s) : 0, 1193914
Total [candidate] (8.996 s) : 0, 8996441
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent iast 1.187 s 123.696 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.198 s 134.055 ms (12.6%)
Agent iast_TELEMETRY_OFF 1.186 s 122.123 ms (11.5%)
Total tracing 8.549 s -
Total iast 8.988 s 438.979 ms (5.1%)
Total iast_HARDCODED_SECRET_DISABLED 8.998 s 449.289 ms (5.3%)
Total iast_TELEMETRY_OFF 8.982 s 432.651 ms (5.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent iast 1.187 s 120.875 ms (11.3%)
Agent iast_HARDCODED_SECRET_DISABLED 1.206 s 139.834 ms (13.1%)
Agent iast_TELEMETRY_OFF 1.194 s 127.923 ms (12.0%)
Total tracing 8.557 s -
Total iast 8.973 s 415.748 ms (4.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.002 s 444.903 ms (5.2%)
Total iast_TELEMETRY_OFF 8.996 s 439.675 ms (5.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (678.88 ms) : 0, 678880
BytebuddyAgent [candidate] (679.544 ms) : 0, 679544
GlobalTracer [baseline] (311.207 ms) : 0, 311207
GlobalTracer [candidate] (312.409 ms) : 0, 312409
AppSec [baseline] (51.496 ms) : 0, 51496
AppSec [candidate] (52.135 ms) : 0, 52135
Remote Config [baseline] (714.087 µs) : 0, 714
Remote Config [candidate] (722.944 µs) : 0, 723
Telemetry [baseline] (7.566 ms) : 0, 7566
Telemetry [candidate] (7.562 ms) : 0, 7562
section iast
BytebuddyAgent [baseline] (789.528 ms) : 0, 789528
BytebuddyAgent [candidate] (789.541 ms) : 0, 789541
GlobalTracer [baseline] (299.523 ms) : 0, 299523
GlobalTracer [candidate] (299.769 ms) : 0, 299769
AppSec [baseline] (50.456 ms) : 0, 50456
AppSec [candidate] (53.37 ms) : 0, 53370
IAST [baseline] (26.116 ms) : 0, 26116
IAST [candidate] (22.679 ms) : 0, 22679
Remote Config [baseline] (644.366 µs) : 0, 644
Remote Config [candidate] (587.667 µs) : 0, 588
Telemetry [baseline] (7.275 ms) : 0, 7275
Telemetry [candidate] (7.301 ms) : 0, 7301
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (796.914 ms) : 0, 796914
BytebuddyAgent [candidate] (803.669 ms) : 0, 803669
GlobalTracer [baseline] (302.231 ms) : 0, 302231
GlobalTracer [candidate] (302.937 ms) : 0, 302937
AppSec [baseline] (55.565 ms) : 0, 55565
AppSec [candidate] (54.568 ms) : 0, 54568
IAST [baseline] (21.061 ms) : 0, 21061
IAST [candidate] (22.811 ms) : 0, 22811
Remote Config [baseline] (597.673 µs) : 0, 598
Remote Config [candidate] (598.947 µs) : 0, 599
Telemetry [baseline] (7.375 ms) : 0, 7375
Telemetry [candidate] (7.37 ms) : 0, 7370
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (787.837 ms) : 0, 787837
BytebuddyAgent [candidate] (793.691 ms) : 0, 793691
GlobalTracer [baseline] (300.197 ms) : 0, 300197
GlobalTracer [candidate] (301.855 ms) : 0, 301855
AppSec [baseline] (54.382 ms) : 0, 54382
AppSec [candidate] (55.789 ms) : 0, 55789
IAST [baseline] (21.616 ms) : 0, 21616
IAST [candidate] (20.88 ms) : 0, 20880
Remote Config [baseline] (611.028 µs) : 0, 611
Remote Config [candidate] (608.906 µs) : 0, 609
Telemetry [baseline] (7.26 ms) : 0, 7260
Telemetry [candidate] (7.334 ms) : 0, 7334
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-11T14:29:56 2024-09-11T14:36:46
git_branch master smola/tomcat-async-flaky
git_commit_date 1726062980 1726064446
git_commit_sha 4abf841 4f0abe1
release_version 1.40.0-SNAPSHOT~4abf841368 1.40.0-SNAPSHOT~4f0abe12a2
start_time 2024-09-11T14:29:43 2024-09-11T14:36:33
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1726065752 1726065752
ci_job_id 635568524 635568524
ci_pipeline_id 44081872 44081872
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368
    dateFormat X
    axisFormat %s
section baseline
no_agent (375.076 µs) : 355, 395
.   : milestone, 375,
iast (482.44 µs) : 461, 504
.   : milestone, 482,
iast_FULL (562.097 µs) : 541, 584
.   : milestone, 562,
iast_GLOBAL (508.695 µs) : 487, 531
.   : milestone, 509,
iast_HARDCODED_SECRET_DISABLED (483.104 µs) : 461, 505
.   : milestone, 483,
iast_INACTIVE (447.074 µs) : 426, 468
.   : milestone, 447,
iast_TELEMETRY_OFF (481.405 µs) : 459, 504
.   : milestone, 481,
tracing (441.139 µs) : 421, 461
.   : milestone, 441,
section candidate
no_agent (379.076 µs) : 359, 399
.   : milestone, 379,
iast (481.721 µs) : 460, 504
.   : milestone, 482,
iast_FULL (556.225 µs) : 535, 577
.   : milestone, 556,
iast_GLOBAL (508.266 µs) : 487, 529
.   : milestone, 508,
iast_HARDCODED_SECRET_DISABLED (494.021 µs) : 473, 515
.   : milestone, 494,
iast_INACTIVE (446.813 µs) : 426, 467
.   : milestone, 447,
iast_TELEMETRY_OFF (476.836 µs) : 454, 500
.   : milestone, 477,
tracing (444.096 µs) : 423, 465
.   : milestone, 444,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 375.076 µs [354.726 µs, 395.426 µs] -
iast 482.44 µs [461.247 µs, 503.632 µs] 107.364 µs (28.6%)
iast_FULL 562.097 µs [540.692 µs, 583.503 µs] 187.021 µs (49.9%)
iast_GLOBAL 508.695 µs [486.866 µs, 530.524 µs] 133.619 µs (35.6%)
iast_HARDCODED_SECRET_DISABLED 483.104 µs [461.147 µs, 505.062 µs] 108.029 µs (28.8%)
iast_INACTIVE 447.074 µs [426.285 µs, 467.863 µs] 71.998 µs (19.2%)
iast_TELEMETRY_OFF 481.405 µs [458.629 µs, 504.181 µs] 106.329 µs (28.3%)
tracing 441.139 µs [420.877 µs, 461.401 µs] 66.063 µs (17.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 379.076 µs [358.842 µs, 399.309 µs] -
iast 481.721 µs [459.932 µs, 503.51 µs] 102.645 µs (27.1%)
iast_FULL 556.225 µs [535.051 µs, 577.399 µs] 177.149 µs (46.7%)
iast_GLOBAL 508.266 µs [487.279 µs, 529.253 µs] 129.19 µs (34.1%)
iast_HARDCODED_SECRET_DISABLED 494.021 µs [472.95 µs, 515.092 µs] 114.945 µs (30.3%)
iast_INACTIVE 446.813 µs [426.23 µs, 467.396 µs] 67.737 µs (17.9%)
iast_TELEMETRY_OFF 476.836 µs [454.101 µs, 499.572 µs] 97.76 µs (25.8%)
tracing 444.096 µs [423.331 µs, 464.861 µs] 65.02 µs (17.2%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.339 ms) : 1320, 1358
.   : milestone, 1339,
appsec (1.721 ms) : 1696, 1746
.   : milestone, 1721,
appsec_no_iast (1.704 ms) : 1679, 1729
.   : milestone, 1704,
iast (1.473 ms) : 1450, 1496
.   : milestone, 1473,
profiling (1.47 ms) : 1446, 1495
.   : milestone, 1470,
tracing (1.44 ms) : 1414, 1465
.   : milestone, 1440,
section candidate
no_agent (1.34 ms) : 1322, 1359
.   : milestone, 1340,
appsec (1.716 ms) : 1691, 1741
.   : milestone, 1716,
appsec_no_iast (1.737 ms) : 1713, 1761
.   : milestone, 1737,
iast (1.47 ms) : 1447, 1493
.   : milestone, 1470,
profiling (1.491 ms) : 1467, 1515
.   : milestone, 1491,
tracing (1.45 ms) : 1425, 1475
.   : milestone, 1450,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.339 ms [1.32 ms, 1.358 ms] -
appsec 1.721 ms [1.696 ms, 1.746 ms] 382.181 µs (28.5%)
appsec_no_iast 1.704 ms [1.679 ms, 1.729 ms] 365.087 µs (27.3%)
iast 1.473 ms [1.45 ms, 1.496 ms] 134.457 µs (10.0%)
profiling 1.47 ms [1.446 ms, 1.495 ms] 131.365 µs (9.8%)
tracing 1.44 ms [1.414 ms, 1.465 ms] 100.816 µs (7.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.34 ms [1.322 ms, 1.359 ms] -
appsec 1.716 ms [1.691 ms, 1.741 ms] 375.823 µs (28.0%)
appsec_no_iast 1.737 ms [1.713 ms, 1.761 ms] 396.455 µs (29.6%)
iast 1.47 ms [1.447 ms, 1.493 ms] 129.557 µs (9.7%)
profiling 1.491 ms [1.467 ms, 1.515 ms] 150.661 µs (11.2%)
tracing 1.45 ms [1.425 ms, 1.475 ms] 109.616 µs (8.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/tomcat-async-flaky
git_commit_date 1726062980 1726064446
git_commit_sha 4abf841 4f0abe1
release_version 1.40.0-SNAPSHOT~4abf841368 1.40.0-SNAPSHOT~4f0abe12a2
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1726066397 1726066397
ci_job_id 635568525 635568525
ci_pipeline_id 44081872 44081872
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.463 ms) : 1452, 1475
.   : milestone, 1463,
appsec (2.281 ms) : 2240, 2321
.   : milestone, 2281,
iast (2.024 ms) : 1976, 2072
.   : milestone, 2024,
iast_GLOBAL (2.099 ms) : 2048, 2150
.   : milestone, 2099,
profiling (1.931 ms) : 1889, 1972
.   : milestone, 1931,
tracing (1.901 ms) : 1862, 1939
.   : milestone, 1901,
section candidate
no_agent (1.458 ms) : 1447, 1470
.   : milestone, 1458,
appsec (2.288 ms) : 2248, 2329
.   : milestone, 2288,
iast (2.034 ms) : 1986, 2083
.   : milestone, 2034,
iast_GLOBAL (2.074 ms) : 2025, 2124
.   : milestone, 2074,
profiling (1.912 ms) : 1872, 1952
.   : milestone, 1912,
tracing (1.882 ms) : 1844, 1919
.   : milestone, 1882,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.463 ms [1.452 ms, 1.475 ms] -
appsec 2.281 ms [2.24 ms, 2.321 ms] 817.563 µs (55.9%)
iast 2.024 ms [1.976 ms, 2.072 ms] 560.575 µs (38.3%)
iast_GLOBAL 2.099 ms [2.048 ms, 2.15 ms] 635.919 µs (43.5%)
profiling 1.931 ms [1.889 ms, 1.972 ms] 467.541 µs (32.0%)
tracing 1.901 ms [1.862 ms, 1.939 ms] 437.561 µs (29.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.458 ms [1.447 ms, 1.47 ms] -
appsec 2.288 ms [2.248 ms, 2.329 ms] 829.78 µs (56.9%)
iast 2.034 ms [1.986 ms, 2.083 ms] 576.086 µs (39.5%)
iast_GLOBAL 2.074 ms [2.025 ms, 2.124 ms] 615.811 µs (42.2%)
profiling 1.912 ms [1.872 ms, 1.952 ms] 453.75 µs (31.1%)
tracing 1.882 ms [1.844 ms, 1.919 ms] 423.163 µs (29.0%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~4f0abe12a2, baseline=1.40.0-SNAPSHOT~4abf841368
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.977 s) : 14977000, 14977000
.   : milestone, 14977000,
appsec (15.35 s) : 15350000, 15350000
.   : milestone, 15350000,
iast (18.919 s) : 18919000, 18919000
.   : milestone, 18919000,
iast_GLOBAL (18.232 s) : 18232000, 18232000
.   : milestone, 18232000,
profiling (16.034 s) : 16034000, 16034000
.   : milestone, 16034000,
tracing (15.103 s) : 15103000, 15103000
.   : milestone, 15103000,
section candidate
no_agent (15.429 s) : 15429000, 15429000
.   : milestone, 15429000,
appsec (15.069 s) : 15069000, 15069000
.   : milestone, 15069000,
iast (18.856 s) : 18856000, 18856000
.   : milestone, 18856000,
iast_GLOBAL (18.001 s) : 18001000, 18001000
.   : milestone, 18001000,
profiling (15.446 s) : 15446000, 15446000
.   : milestone, 15446000,
tracing (14.953 s) : 14953000, 14953000
.   : milestone, 14953000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.977 s [14.977 s, 14.977 s] -
appsec 15.35 s [15.35 s, 15.35 s] 373.0 ms (2.5%)
iast 18.919 s [18.919 s, 18.919 s] 3.942 s (26.3%)
iast_GLOBAL 18.232 s [18.232 s, 18.232 s] 3.255 s (21.7%)
profiling 16.034 s [16.034 s, 16.034 s] 1.057 s (7.1%)
tracing 15.103 s [15.103 s, 15.103 s] 126.0 ms (0.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.429 s [15.429 s, 15.429 s] -
appsec 15.069 s [15.069 s, 15.069 s] -360.0 ms (-2.3%)
iast 18.856 s [18.856 s, 18.856 s] 3.427 s (22.2%)
iast_GLOBAL 18.001 s [18.001 s, 18.001 s] 2.572 s (16.7%)
profiling 15.446 s [15.446 s, 15.446 s] 17.0 ms (0.1%)
tracing 14.953 s [14.953 s, 14.953 s] -476.0 ms (-3.1%)

@smola smola marked this pull request as ready for review September 3, 2024 09:22
@smola smola requested a review from a team as a code owner September 3, 2024 09:22
@smola smola added the tag: do not merge Do not merge changes label Sep 3, 2024
@PerfectSlayer
Copy link
Contributor

Is this PR still not to merge? (according the do not merge tag)

@smola smola force-pushed the smola/tomcat-async-flaky branch from a0f7296 to 4f0abe1 Compare September 11, 2024 14:21
@smola smola removed the tag: do not merge Do not merge changes label Sep 11, 2024
@smola
Copy link
Member Author

smola commented Sep 11, 2024

@PerfectSlayer I was waiting to confirm this actually fixed the issue. I now think it does, also fixed an additional test with the same issue. Good to merge now if it passes CI.

@smola smola enabled auto-merge (squash) September 12, 2024 06:53
@smola smola merged commit bbd40f0 into master Sep 12, 2024
97 of 98 checks passed
@smola smola deleted the smola/tomcat-async-flaky branch September 12, 2024 07:00
@github-actions github-actions bot added this to the 1.40.0 milestone Sep 12, 2024
@smola smola mentioned this pull request Sep 13, 2024
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@amarziali amarziali amarziali approved these changes

Assignees
No one assigned
Labels
comp: testing Testing tag: flaky test Flaky tests tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.40.0
Development

Successfully merging this pull request may close these issues.
3 participants
@smola @PerfectSlayer @amarziali