Skip to content

Make Crashtracking smoke test more resilient #8685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Apr 18, 2025
Merged

Make Crashtracking smoke test more resilient #8685

merged 9 commits into from
Apr 18, 2025

Conversation

randomanderson
Copy link
Contributor

What Does This Do

Improves the resiliency of the Crashtracking Smoke Test. Previously, the smoke test relied on parsing output logs which in inherently flaky. With this PR, the smoke test instead listens for the crash telemetry submitted to mock servers.

The changes are:

  • Creation of a test UDP server
  • Check the telemetry data submitted to the MockHttpServer rather than log output of the crashed process
  • Check the telemetry event submitted to the UDP server for OOM events rather than log output of the crashed process
  • Extract the output log processing from ProcessManager into its own class. This gets rid of a bunch of code duplication.

Motivation

When migrating to Gitlab, this smoke test was extra flaky.

Additional Notes

TestUDPServer could probably go in a more generic location in the codebase.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@randomanderson randomanderson added tag: no release notes Changes to exclude from release notes type: refactoring comp: crash tracking Crash tracking labels Apr 9, 2025
@randomanderson randomanderson requested a review from a team as a code owner April 9, 2025 16:33
@randomanderson randomanderson force-pushed the landerson/fix-crashtrackingsmoke branch from 1b8480c to 561f192 Compare April 9, 2025 16:35
@randomanderson randomanderson marked this pull request as draft April 9, 2025 17:05
@randomanderson
Copy link
Contributor Author

Back to draft. Found some issues

@pr-commenter
Copy link

pr-commenter bot commented Apr 9, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master landerson/fix-crashtrackingsmoke
git_commit_date 1744296074 1744390695
git_commit_sha b304f4b ac863f0
release_version 1.49.0-SNAPSHOT~b304f4bd62 1.49.0-SNAPSHOT~ac863f0f44
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1744395302 1744395302
ci_job_id 893396591 893396591
ci_pipeline_id 61928858 61928858
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-u8dbh5tw-project-304-concurrent-0-s2fufd85 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-u8dbh5tw-project-304-concurrent-0-s2fufd85 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 64 metrics, 7 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.062 s) : 0, 1062012
Total [baseline] (8.717 s) : 0, 8716625
Agent [candidate] (1.064 s) : 0, 1063824
Total [candidate] (8.713 s) : 0, 8713020
section iast
Agent [baseline] (1.201 s) : 0, 1200903
Total [baseline] (9.304 s) : 0, 9303619
Agent [candidate] (1.187 s) : 0, 1186675
Total [candidate] (9.224 s) : 0, 9223811
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.193 s) : 0, 1193147
Total [baseline] (9.242 s) : 0, 9241804
Agent [candidate] (1.183 s) : 0, 1182865
Total [candidate] (9.2 s) : 0, 9199886
section iast_TELEMETRY_OFF
Agent [baseline] (1.189 s) : 0, 1188675
Total [baseline] (9.288 s) : 0, 9288220
Agent [candidate] (1.182 s) : 0, 1181885
Total [candidate] (9.265 s) : 0, 9264841
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent iast 1.201 s 138.891 ms (13.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.193 s 131.135 ms (12.3%)
Agent iast_TELEMETRY_OFF 1.189 s 126.663 ms (11.9%)
Total tracing 8.717 s -
Total iast 9.304 s 586.995 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.242 s 525.179 ms (6.0%)
Total iast_TELEMETRY_OFF 9.288 s 571.595 ms (6.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent iast 1.187 s 122.851 ms (11.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.183 s 119.041 ms (11.2%)
Agent iast_TELEMETRY_OFF 1.182 s 118.061 ms (11.1%)
Total tracing 8.713 s -
Total iast 9.224 s 510.791 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.2 s 486.865 ms (5.6%)
Total iast_TELEMETRY_OFF 9.265 s 551.821 ms (6.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (725.336 ms) : 0, 725336
BytebuddyAgent [candidate] (725.271 ms) : 0, 725271
GlobalTracer [baseline] (239.945 ms) : 0, 239945
GlobalTracer [candidate] (240.559 ms) : 0, 240559
AppSec [baseline] (54.948 ms) : 0, 54948
AppSec [candidate] (54.782 ms) : 0, 54782
Debugger [baseline] (4.429 ms) : 0, 4429
Debugger [candidate] (4.404 ms) : 0, 4404
Remote Config [baseline] (4.349 ms) : 0, 4349
Remote Config [candidate] (2.976 ms) : 0, 2976
Telemetry [baseline] (9.465 ms) : 0, 9465
Telemetry [candidate] (12.295 ms) : 0, 12295
section iast
BytebuddyAgent [baseline] (853.157 ms) : 0, 853157
BytebuddyAgent [candidate] (840.463 ms) : 0, 840463
GlobalTracer [baseline] (231.004 ms) : 0, 231004
GlobalTracer [candidate] (230.054 ms) : 0, 230054
IAST [baseline] (23.105 ms) : 0, 23105
IAST [candidate] (24.735 ms) : 0, 24735
AppSec [baseline] (56.192 ms) : 0, 56192
AppSec [candidate] (54.497 ms) : 0, 54497
Debugger [baseline] (4.332 ms) : 0, 4332
Debugger [candidate] (4.263 ms) : 0, 4263
Remote Config [baseline] (638.81 µs) : 0, 639
Remote Config [candidate] (604.119 µs) : 0, 604
Telemetry [baseline] (8.792 ms) : 0, 8792
Telemetry [candidate] (8.733 ms) : 0, 8733
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (846.336 ms) : 0, 846336
BytebuddyAgent [candidate] (838.468 ms) : 0, 838468
GlobalTracer [baseline] (230.64 ms) : 0, 230640
GlobalTracer [candidate] (229.199 ms) : 0, 229199
IAST [baseline] (24.507 ms) : 0, 24507
IAST [candidate] (22.719 ms) : 0, 22719
AppSec [baseline] (54.458 ms) : 0, 54458
AppSec [candidate] (55.61 ms) : 0, 55610
Debugger [baseline] (4.259 ms) : 0, 4259
Debugger [candidate] (4.204 ms) : 0, 4204
Remote Config [baseline] (625.875 µs) : 0, 626
Remote Config [candidate] (605.928 µs) : 0, 606
Telemetry [baseline] (8.732 ms) : 0, 8732
Telemetry [candidate] (8.645 ms) : 0, 8645
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (842.895 ms) : 0, 842895
BytebuddyAgent [candidate] (837.177 ms) : 0, 837177
GlobalTracer [baseline] (230.756 ms) : 0, 230756
GlobalTracer [candidate] (229.495 ms) : 0, 229495
IAST [baseline] (22.287 ms) : 0, 22287
IAST [candidate] (22.465 ms) : 0, 22465
AppSec [baseline] (55.866 ms) : 0, 55866
AppSec [candidate] (55.894 ms) : 0, 55894
Debugger [baseline] (4.152 ms) : 0, 4152
Debugger [candidate] (4.125 ms) : 0, 4125
Remote Config [baseline] (619.212 µs) : 0, 619
Remote Config [candidate] (641.48 µs) : 0, 641
Telemetry [baseline] (8.651 ms) : 0, 8651
Telemetry [candidate] (8.691 ms) : 0, 8691
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.061 s) : 0, 1060656
Total [baseline] (10.511 s) : 0, 10510685
Agent [candidate] (1.061 s) : 0, 1060776
Total [candidate] (10.492 s) : 0, 10491807
section appsec
Agent [baseline] (1.195 s) : 0, 1194691
Total [baseline] (10.808 s) : 0, 10807874
Agent [candidate] (1.2 s) : 0, 1200224
Total [candidate] (10.772 s) : 0, 10771932
section iast
Agent [baseline] (1.186 s) : 0, 1185843
Total [baseline] (11.051 s) : 0, 11051335
Agent [candidate] (1.185 s) : 0, 1184896
Total [candidate] (11.073 s) : 0, 11073494
section profiling
Agent [baseline] (1.292 s) : 0, 1291855
Total [baseline] (10.929 s) : 0, 10928720
Agent [candidate] (1.299 s) : 0, 1299350
Total [candidate] (10.912 s) : 0, 10911969
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent appsec 1.195 s 134.035 ms (12.6%)
Agent iast 1.186 s 125.187 ms (11.8%)
Agent profiling 1.292 s 231.199 ms (21.8%)
Total tracing 10.511 s -
Total appsec 10.808 s 297.189 ms (2.8%)
Total iast 11.051 s 540.65 ms (5.1%)
Total profiling 10.929 s 418.036 ms (4.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent appsec 1.2 s 139.448 ms (13.1%)
Agent iast 1.185 s 124.12 ms (11.7%)
Agent profiling 1.299 s 238.575 ms (22.5%)
Total tracing 10.492 s -
Total appsec 10.772 s 280.125 ms (2.7%)
Total iast 11.073 s 581.686 ms (5.5%)
Total profiling 10.912 s 420.162 ms (4.0%) 
gantt
    title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (725.055 ms) : 0, 725055
BytebuddyAgent [candidate] (721.941 ms) : 0, 721941
GlobalTracer [baseline] (239.434 ms) : 0, 239434
GlobalTracer [candidate] (239.385 ms) : 0, 239385
AppSec [baseline] (54.759 ms) : 0, 54759
AppSec [candidate] (55.224 ms) : 0, 55224
Debugger [baseline] (4.374 ms) : 0, 4374
Debugger [candidate] (4.388 ms) : 0, 4388
Remote Config [baseline] (709.158 µs) : 0, 709
Remote Config [candidate] (692.622 µs) : 0, 693
Telemetry [baseline] (12.902 ms) : 0, 12902
Telemetry [candidate] (15.763 ms) : 0, 15763
section appsec
BytebuddyAgent [baseline] (736.461 ms) : 0, 736461
BytebuddyAgent [candidate] (740.09 ms) : 0, 740090
GlobalTracer [baseline] (235.163 ms) : 0, 235163
GlobalTracer [candidate] (236.201 ms) : 0, 236201
IAST [baseline] (21.723 ms) : 0, 21723
IAST [candidate] (21.688 ms) : 0, 21688
AppSec [baseline] (175.226 ms) : 0, 175226
AppSec [candidate] (176.256 ms) : 0, 176256
Debugger [baseline] (4.944 ms) : 0, 4944
Debugger [candidate] (4.598 ms) : 0, 4598
Remote Config [baseline] (637.549 µs) : 0, 638
Remote Config [candidate] (640.386 µs) : 0, 640
Telemetry [baseline] (8.126 ms) : 0, 8126
Telemetry [candidate] (8.238 ms) : 0, 8238
section iast
BytebuddyAgent [baseline] (839.862 ms) : 0, 839862
BytebuddyAgent [candidate] (838.844 ms) : 0, 838844
GlobalTracer [baseline] (230.223 ms) : 0, 230223
GlobalTracer [candidate] (230.013 ms) : 0, 230013
IAST [baseline] (23.804 ms) : 0, 23804
IAST [candidate] (22.899 ms) : 0, 22899
AppSec [baseline] (54.951 ms) : 0, 54951
AppSec [candidate] (56.222 ms) : 0, 56222
Debugger [baseline] (4.226 ms) : 0, 4226
Debugger [candidate] (4.269 ms) : 0, 4269
Remote Config [baseline] (606.134 µs) : 0, 606
Remote Config [candidate] (622.49 µs) : 0, 622
Telemetry [baseline] (8.736 ms) : 0, 8736
Telemetry [candidate] (8.666 ms) : 0, 8666
section profiling
BytebuddyAgent [baseline] (711.442 ms) : 0, 711442
BytebuddyAgent [candidate] (716.53 ms) : 0, 716530
GlobalTracer [baseline] (359.98 ms) : 0, 359980
GlobalTracer [candidate] (361.811 ms) : 0, 361811
AppSec [baseline] (53.965 ms) : 0, 53965
AppSec [candidate] (53.819 ms) : 0, 53819
Debugger [baseline] (4.398 ms) : 0, 4398
Debugger [candidate] (4.419 ms) : 0, 4419
Remote Config [baseline] (680.578 µs) : 0, 681
Remote Config [candidate] (661.614 µs) : 0, 662
Telemetry [baseline] (9.017 ms) : 0, 9017
Telemetry [candidate] (9.047 ms) : 0, 9047
ProfilingAgent [baseline] (102.166 ms) : 0, 102166
ProfilingAgent [candidate] (102.663 ms) : 0, 102663
Profiling [baseline] (102.193 ms) : 0, 102193
Profiling [candidate] (102.69 ms) : 0, 102690
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-04-11T17:45:12 2025-04-11T17:53:00
git_branch master landerson/fix-crashtrackingsmoke
git_commit_date 1744296074 1744390695
git_commit_sha b304f4b ac863f0
release_version 1.49.0-SNAPSHOT~b304f4bd62 1.49.0-SNAPSHOT~ac863f0f44
start_time 2025-04-11T17:44:58 2025-04-11T17:52:46
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1744394380 1744394380
ci_job_id 893396592 893396592
ci_pipeline_id 61928858 61928858
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-fbx1gztc-project-304-concurrent-0-9wwzf5w9 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-fbx1gztc-project-304-concurrent-0-9wwzf5w9 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62
    dateFormat X
    axisFormat %s
section baseline
no_agent (388.391 µs) : 368, 409
.   : milestone, 388,
iast (523.685 µs) : 501, 547
.   : milestone, 524,
iast_FULL (738.922 µs) : 717, 761
.   : milestone, 739,
iast_GLOBAL (562.875 µs) : 541, 585
.   : milestone, 563,
iast_HARDCODED_SECRET_DISABLED (516.263 µs) : 495, 538
.   : milestone, 516,
iast_INACTIVE (476.324 µs) : 455, 498
.   : milestone, 476,
iast_TELEMETRY_OFF (511.96 µs) : 490, 534
.   : milestone, 512,
tracing (469.309 µs) : 448, 491
.   : milestone, 469,
section candidate
no_agent (384.765 µs) : 364, 406
.   : milestone, 385,
iast (517.306 µs) : 496, 539
.   : milestone, 517,
iast_FULL (737.188 µs) : 715, 759
.   : milestone, 737,
iast_GLOBAL (569.584 µs) : 547, 593
.   : milestone, 570,
iast_HARDCODED_SECRET_DISABLED (521.371 µs) : 499, 543
.   : milestone, 521,
iast_INACTIVE (471.942 µs) : 450, 494
.   : milestone, 472,
iast_TELEMETRY_OFF (510.031 µs) : 488, 532
.   : milestone, 510,
tracing (472.035 µs) : 451, 493
.   : milestone, 472,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 388.391 µs [367.956 µs, 408.826 µs] -
iast 523.685 µs [500.785 µs, 546.584 µs] 135.294 µs (34.8%)
iast_FULL 738.922 µs [716.849 µs, 760.994 µs] 350.531 µs (90.3%)
iast_GLOBAL 562.875 µs [540.791 µs, 584.959 µs] 174.484 µs (44.9%)
iast_HARDCODED_SECRET_DISABLED 516.263 µs [494.678 µs, 537.848 µs] 127.872 µs (32.9%)
iast_INACTIVE 476.324 µs [454.532 µs, 498.116 µs] 87.933 µs (22.6%)
iast_TELEMETRY_OFF 511.96 µs [489.98 µs, 533.941 µs] 123.57 µs (31.8%)
tracing 469.309 µs [448.009 µs, 490.609 µs] 80.918 µs (20.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 384.765 µs [363.812 µs, 405.719 µs] -
iast 517.306 µs [495.585 µs, 539.027 µs] 132.541 µs (34.4%)
iast_FULL 737.188 µs [715.177 µs, 759.2 µs] 352.423 µs (91.6%)
iast_GLOBAL 569.584 µs [546.631 µs, 592.537 µs] 184.819 µs (48.0%)
iast_HARDCODED_SECRET_DISABLED 521.371 µs [499.405 µs, 543.337 µs] 136.606 µs (35.5%)
iast_INACTIVE 471.942 µs [450.264 µs, 493.621 µs] 87.177 µs (22.7%)
iast_TELEMETRY_OFF 510.031 µs [488.109 µs, 531.952 µs] 125.265 µs (32.6%)
tracing 472.035 µs [450.613 µs, 493.457 µs] 87.27 µs (22.7%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.364 ms) : 1344, 1384
.   : milestone, 1364,
appsec (1.754 ms) : 1730, 1777
.   : milestone, 1754,
appsec_no_iast (1.754 ms) : 1731, 1778
.   : milestone, 1754,
code_origins (1.701 ms) : 1674, 1728
.   : milestone, 1701,
iast (1.537 ms) : 1513, 1560
.   : milestone, 1537,
profiling (1.561 ms) : 1535, 1587
.   : milestone, 1561,
tracing (1.491 ms) : 1466, 1517
.   : milestone, 1491,
section candidate
no_agent (1.343 ms) : 1324, 1363
.   : milestone, 1343,
appsec (1.738 ms) : 1714, 1762
.   : milestone, 1738,
appsec_no_iast (1.745 ms) : 1721, 1769
.   : milestone, 1745,
code_origins (1.698 ms) : 1670, 1725
.   : milestone, 1698,
iast (1.51 ms) : 1486, 1534
.   : milestone, 1510,
profiling (1.583 ms) : 1559, 1607
.   : milestone, 1583,
tracing (1.511 ms) : 1486, 1536
.   : milestone, 1511,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.364 ms [1.344 ms, 1.384 ms] -
appsec 1.754 ms [1.73 ms, 1.777 ms] 389.294 µs (28.5%)
appsec_no_iast 1.754 ms [1.731 ms, 1.778 ms] 390.148 µs (28.6%)
code_origins 1.701 ms [1.674 ms, 1.728 ms] 336.585 µs (24.7%)
iast 1.537 ms [1.513 ms, 1.56 ms] 172.216 µs (12.6%)
profiling 1.561 ms [1.535 ms, 1.587 ms] 196.359 µs (14.4%)
tracing 1.491 ms [1.466 ms, 1.517 ms] 126.907 µs (9.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.343 ms [1.324 ms, 1.363 ms] -
appsec 1.738 ms [1.714 ms, 1.762 ms] 394.837 µs (29.4%)
appsec_no_iast 1.745 ms [1.721 ms, 1.769 ms] 401.539 µs (29.9%)
code_origins 1.698 ms [1.67 ms, 1.725 ms] 354.24 µs (26.4%)
iast 1.51 ms [1.486 ms, 1.534 ms] 166.183 µs (12.4%)
profiling 1.583 ms [1.559 ms, 1.607 ms] 239.529 µs (17.8%)
tracing 1.511 ms [1.486 ms, 1.536 ms] 167.539 µs (12.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master landerson/fix-crashtrackingsmoke
git_commit_date 1744296074 1744390695
git_commit_sha b304f4b ac863f0
release_version 1.49.0-SNAPSHOT~b304f4bd62 1.49.0-SNAPSHOT~ac863f0f44
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1744394802 1744394802
ci_job_id 893396593 893396593
ci_pipeline_id 61928858 61928858
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-ubwazs2a-project-304-concurrent-2-sifzleix 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-ubwazs2a-project-304-concurrent-2-sifzleix 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.542 s) : 15542000, 15542000
.   : milestone, 15542000,
appsec (15.146 s) : 15146000, 15146000
.   : milestone, 15146000,
iast (18.454 s) : 18454000, 18454000
.   : milestone, 18454000,
iast_GLOBAL (17.952 s) : 17952000, 17952000
.   : milestone, 17952000,
profiling (15.209 s) : 15209000, 15209000
.   : milestone, 15209000,
tracing (15.026 s) : 15026000, 15026000
.   : milestone, 15026000,
section candidate
no_agent (14.922 s) : 14922000, 14922000
.   : milestone, 14922000,
appsec (14.806 s) : 14806000, 14806000
.   : milestone, 14806000,
iast (18.127 s) : 18127000, 18127000
.   : milestone, 18127000,
iast_GLOBAL (18.167 s) : 18167000, 18167000
.   : milestone, 18167000,
profiling (15.2 s) : 15200000, 15200000
.   : milestone, 15200000,
tracing (15.074 s) : 15074000, 15074000
.   : milestone, 15074000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.542 s [15.542 s, 15.542 s] -
appsec 15.146 s [15.146 s, 15.146 s] -396.0 ms (-2.5%)
iast 18.454 s [18.454 s, 18.454 s] 2.912 s (18.7%)
iast_GLOBAL 17.952 s [17.952 s, 17.952 s] 2.41 s (15.5%)
profiling 15.209 s [15.209 s, 15.209 s] -333.0 ms (-2.1%)
tracing 15.026 s [15.026 s, 15.026 s] -516.0 ms (-3.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.922 s [14.922 s, 14.922 s] -
appsec 14.806 s [14.806 s, 14.806 s] -116.0 ms (-0.8%)
iast 18.127 s [18.127 s, 18.127 s] 3.205 s (21.5%)
iast_GLOBAL 18.167 s [18.167 s, 18.167 s] 3.245 s (21.7%)
profiling 15.2 s [15.2 s, 15.2 s] 278.0 ms (1.9%)
tracing 15.074 s [15.074 s, 15.074 s] 152.0 ms (1.0%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~ac863f0f44, baseline=1.49.0-SNAPSHOT~b304f4bd62
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (2.375 ms) : 2328, 2421
.   : milestone, 2375,
iast (2.162 ms) : 2103, 2220
.   : milestone, 2162,
iast_GLOBAL (2.202 ms) : 2143, 2261
.   : milestone, 2202,
profiling (2.484 ms) : 2303, 2665
.   : milestone, 2484,
tracing (1.981 ms) : 1936, 2026
.   : milestone, 1981,
section candidate
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (2.375 ms) : 2328, 2422
.   : milestone, 2375,
iast (2.152 ms) : 2094, 2211
.   : milestone, 2152,
iast_GLOBAL (2.201 ms) : 2142, 2260
.   : milestone, 2201,
profiling (1.999 ms) : 1952, 2045
.   : milestone, 1999,
tracing (1.98 ms) : 1936, 2025
.   : milestone, 1980,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.464 ms, 1.487 ms] -
appsec 2.375 ms [2.328 ms, 2.421 ms] 899.306 µs (61.0%)
iast 2.162 ms [2.103 ms, 2.22 ms] 686.334 µs (46.5%)
iast_GLOBAL 2.202 ms [2.143 ms, 2.261 ms] 726.397 µs (49.2%)
profiling 2.484 ms [2.303 ms, 2.665 ms] 1.009 ms (68.4%)
tracing 1.981 ms [1.936 ms, 2.026 ms] 505.676 µs (34.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.48 ms [1.469 ms, 1.492 ms] -
appsec 2.375 ms [2.328 ms, 2.422 ms] 894.645 µs (60.4%)
iast 2.152 ms [2.094 ms, 2.211 ms] 671.769 µs (45.4%)
iast_GLOBAL 2.201 ms [2.142 ms, 2.26 ms] 720.59 µs (48.7%)
profiling 1.999 ms [1.952 ms, 2.045 ms] 518.314 µs (35.0%)
tracing 1.98 ms [1.936 ms, 2.025 ms] 500.055 µs (33.8%)

@randomanderson randomanderson force-pushed the landerson/fix-crashtrackingsmoke branch from c2e0084 to b5a1003 Compare April 10, 2025 20:34
@randomanderson randomanderson mentioned this pull request Apr 11, 2025
Merged
@randomanderson randomanderson changed the base branch from master to landerson/dd-dogstatsd-port April 11, 2025 16:49
@randomanderson randomanderson force-pushed the landerson/fix-crashtrackingsmoke branch from a64de40 to ac863f0 Compare April 11, 2025 17:13
Base automatically changed from landerson/dd-dogstatsd-port to master April 14, 2025 06:31
@randomanderson randomanderson marked this pull request as ready for review April 14, 2025 14:33
@randomanderson randomanderson enabled auto-merge (squash) April 14, 2025 14:33
@randomanderson randomanderson merged commit 0382750 into master Apr 18, 2025
250 of 252 checks passed
@randomanderson randomanderson deleted the landerson/fix-crashtrackingsmoke branch April 18, 2025 07:08
@github-actions github-actions bot added this to the 1.49.0 milestone Apr 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jbachorik jbachorik jbachorik approved these changes

@amarziali amarziali Awaiting requested review from amarziali amarziali is a code owner automatically assigned from DataDog/apm-java

@dougqh dougqh Awaiting requested review from dougqh

Assignees
No one assigned
Labels
comp: crash tracking Crash tracking tag: no release notes Changes to exclude from release notes type: refactoring
Projects
None yet
Milestone
1.49.0
Development

Successfully merging this pull request may close these issues.
2 participants
@randomanderson @jbachorik