Skip to content

Minimize changes needed to test code origin on instrumentations #8577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Mar 20, 2025
Merged

Minimize changes needed to test code origin on instrumentations #8577

merged 2 commits into from
Mar 20, 2025

Conversation

evanchooly
Copy link
Contributor

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@evanchooly
Copy link
Contributor Author

just trying to see CI run

@pr-commenter
Copy link

pr-commenter bot commented Mar 17, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1742401900 1742402283
end_time 2025-03-19T16:32:58 2025-03-19T16:39:20
git_branch master evanchooly/autoCodeOriginTesting
git_commit_sha fc9989c b1802d2
start_time 2025-03-19T16:31:41 2025-03-19T16:38:04
See matching parameters
Baseline Candidate
ci_job_id 855325081 855325081
ci_pipeline_id 59342921 59342921
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1742400880 1742400880

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 6 unstable metrics.

See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-38.341µs; +13.358µs] or [-13.666%; +4.761%]		 unstable
[-54.754µs; +21.172µs] or [-17.022%; +6.582%]		 unstable
[-70.267µs; +32.070µs] or [-20.792%; +9.490%]		 unstable
[-97.943µs; +111.863µs] or [-16.129%; +18.421%]		 same
scenario:basic same same same unstable
[-61.219µs; +22.719µs] or [-11.227%; +4.167%]		 unstable
[-160.088op/s; +160.088op/s] or [-6.404%; +6.404%]
scenario:loop unsure
[+31.333µs; +80.391µs] or [+0.294%; +0.755%]		 unsure
[+31.337µs; +124.040µs] or [+0.290%; +1.147%]		 same same same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (321.665 µs) : 278, 365
.   : milestone, 322,
basic (309.965 µs) : 299, 321
.   : milestone, 310,
loop (10.815 ms) : 10789, 10842
.   : milestone, 10815,
section candidate
noprobe (304.874 µs) : 278, 331
.   : milestone, 305,
basic (311.334 µs) : 303, 319
.   : milestone, 311,
loop (10.893 ms) : 10838, 10948
.   : milestone, 10893,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 321.665 µs [277.948 µs, 365.382 µs]
basic 309.965 µs [298.696 µs, 321.233 µs]
loop 10.815 ms [10.789 ms, 10.842 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 304.874 µs [278.295 µs, 331.452 µs]
basic 311.334 µs [303.484 µs, 319.184 µs]
loop 10.893 ms [10.838 ms, 10.948 ms]

@pr-commenter
Copy link

pr-commenter bot commented Mar 17, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master evanchooly/autoCodeOriginTesting
git_commit_date 1742481362 1742484759
git_commit_sha 563b2c9 98a473e
release_version 1.48.0-SNAPSHOT~563b2c9769 1.48.0-SNAPSHOT~98a473e4a7
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1742487345 1742487345
ci_job_id 857263824 857263824
ci_pipeline_id 59478059 59478059
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-9n9spsyq-project-304-concurrent-0-jwqguasq 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-9n9spsyq-project-304-concurrent-0-jwqguasq 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 4 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~98a473e4a7, baseline=1.48.0-SNAPSHOT~563b2c9769

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.039 s) : 0, 1039036
Total [baseline] (10.454 s) : 0, 10453771
Agent [candidate] (1.047 s) : 0, 1046909
Total [candidate] (10.455 s) : 0, 10455156
section appsec
Agent [baseline] (1.188 s) : 0, 1188393
Total [baseline] (10.782 s) : 0, 10781664
Agent [candidate] (1.187 s) : 0, 1186574
Total [candidate] (10.792 s) : 0, 10791766
section iast
Agent [baseline] (1.175 s) : 0, 1174719
Total [baseline] (11.022 s) : 0, 11022138
Agent [candidate] (1.171 s) : 0, 1171032
Total [candidate] (10.97 s) : 0, 10970081
section profiling
Agent [baseline] (1.259 s) : 0, 1258622
Total [baseline] (10.896 s) : 0, 10895631
Agent [candidate] (1.266 s) : 0, 1266307
Total [candidate] (10.879 s) : 0, 10879469
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.039 s -
Agent appsec 1.188 s 149.357 ms (14.4%)
Agent iast 1.175 s 135.683 ms (13.1%)
Agent profiling 1.259 s 219.586 ms (21.1%)
Total tracing 10.454 s -
Total appsec 10.782 s 327.893 ms (3.1%)
Total iast 11.022 s 568.367 ms (5.4%)
Total profiling 10.896 s 441.86 ms (4.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.047 s -
Agent appsec 1.187 s 139.665 ms (13.3%)
Agent iast 1.171 s 124.123 ms (11.9%)
Agent profiling 1.266 s 219.398 ms (21.0%)
Total tracing 10.455 s -
Total appsec 10.792 s 336.61 ms (3.2%)
Total iast 10.97 s 514.924 ms (4.9%)
Total profiling 10.879 s 424.313 ms (4.1%) 
gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~98a473e4a7, baseline=1.48.0-SNAPSHOT~563b2c9769

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (717.003 ms) : 0, 717003
BytebuddyAgent [candidate] (721.725 ms) : 0, 721725
GlobalTracer [baseline] (238.402 ms) : 0, 238402
GlobalTracer [candidate] (240.518 ms) : 0, 240518
AppSec [baseline] (54.826 ms) : 0, 54826
AppSec [candidate] (55.022 ms) : 0, 55022
Remote Config [baseline] (702.59 µs) : 0, 703
Remote Config [candidate] (696.077 µs) : 0, 696
Telemetry [baseline] (12.149 ms) : 0, 12149
Telemetry [candidate] (12.917 ms) : 0, 12917
section appsec
BytebuddyAgent [baseline] (738.179 ms) : 0, 738179
BytebuddyAgent [candidate] (737.948 ms) : 0, 737948
GlobalTracer [baseline] (237.434 ms) : 0, 237434
GlobalTracer [candidate] (236.445 ms) : 0, 236445
IAST [baseline] (21.703 ms) : 0, 21703
IAST [candidate] (21.602 ms) : 0, 21602
AppSec [baseline] (176.922 ms) : 0, 176922
AppSec [candidate] (176.432 ms) : 0, 176432
Remote Config [baseline] (678.16 µs) : 0, 678
Remote Config [candidate] (659.615 µs) : 0, 660
Telemetry [baseline] (8.341 ms) : 0, 8341
Telemetry [candidate] (8.299 ms) : 0, 8299
section iast
BytebuddyAgent [baseline] (840.155 ms) : 0, 840155
BytebuddyAgent [candidate] (837.007 ms) : 0, 837007
GlobalTracer [baseline] (230.433 ms) : 0, 230433
GlobalTracer [candidate] (230.012 ms) : 0, 230012
IAST [baseline] (22.816 ms) : 0, 22816
IAST [candidate] (22.697 ms) : 0, 22697
AppSec [baseline] (56.026 ms) : 0, 56026
AppSec [candidate] (56.206 ms) : 0, 56206
Remote Config [baseline] (599.801 µs) : 0, 600
Remote Config [candidate] (596.782 µs) : 0, 597
Telemetry [baseline] (8.696 ms) : 0, 8696
Telemetry [candidate] (8.61 ms) : 0, 8610
section profiling
BytebuddyAgent [baseline] (708.325 ms) : 0, 708325
BytebuddyAgent [candidate] (713.443 ms) : 0, 713443
GlobalTracer [baseline] (349.48 ms) : 0, 349480
GlobalTracer [candidate] (351.155 ms) : 0, 351155
AppSec [baseline] (54.009 ms) : 0, 54009
AppSec [candidate] (53.82 ms) : 0, 53820
Remote Config [baseline] (706.011 µs) : 0, 706
Remote Config [candidate] (710.922 µs) : 0, 711
Telemetry [baseline] (8.947 ms) : 0, 8947
Telemetry [candidate] (8.978 ms) : 0, 8978
ProfilingAgent [baseline] (95.691 ms) : 0, 95691
ProfilingAgent [candidate] (96.51 ms) : 0, 96510
Profiling [baseline] (95.716 ms) : 0, 95716
Profiling [candidate] (96.535 ms) : 0, 96535
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~98a473e4a7, baseline=1.48.0-SNAPSHOT~563b2c9769

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.041 s) : 0, 1041374
Total [baseline] (8.669 s) : 0, 8669412
Agent [candidate] (1.039 s) : 0, 1038767
Total [candidate] (8.648 s) : 0, 8647759
section iast
Agent [baseline] (1.172 s) : 0, 1171729
Total [baseline] (9.237 s) : 0, 9236830
Agent [candidate] (1.171 s) : 0, 1171177
Total [candidate] (9.248 s) : 0, 9247745
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.173 s) : 0, 1172542
Total [baseline] (9.18 s) : 0, 9179908
Agent [candidate] (1.188 s) : 0, 1188095
Total [candidate] (9.222 s) : 0, 9221906
section iast_TELEMETRY_OFF
Agent [baseline] (1.169 s) : 0, 1168576
Total [baseline] (9.261 s) : 0, 9260624
Agent [candidate] (1.176 s) : 0, 1175728
Total [candidate] (9.278 s) : 0, 9277716
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.041 s -
Agent iast 1.172 s 130.355 ms (12.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.173 s 131.168 ms (12.6%)
Agent iast_TELEMETRY_OFF 1.169 s 127.202 ms (12.2%)
Total tracing 8.669 s -
Total iast 9.237 s 567.419 ms (6.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.18 s 510.496 ms (5.9%)
Total iast_TELEMETRY_OFF 9.261 s 591.213 ms (6.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.039 s -
Agent iast 1.171 s 132.41 ms (12.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.188 s 149.328 ms (14.4%)
Agent iast_TELEMETRY_OFF 1.176 s 136.961 ms (13.2%)
Total tracing 8.648 s -
Total iast 9.248 s 599.986 ms (6.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.222 s 574.147 ms (6.6%)
Total iast_TELEMETRY_OFF 9.278 s 629.957 ms (7.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~98a473e4a7, baseline=1.48.0-SNAPSHOT~563b2c9769

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (718.454 ms) : 0, 718454
BytebuddyAgent [candidate] (716.444 ms) : 0, 716444
GlobalTracer [baseline] (239.878 ms) : 0, 239878
GlobalTracer [candidate] (238.821 ms) : 0, 238821
AppSec [baseline] (54.794 ms) : 0, 54794
AppSec [candidate] (54.73 ms) : 0, 54730
Remote Config [baseline] (697.897 µs) : 0, 698
Remote Config [candidate] (693.34 µs) : 0, 693
Telemetry [baseline] (11.536 ms) : 0, 11536
Telemetry [candidate] (12.119 ms) : 0, 12119
section iast
BytebuddyAgent [baseline] (837.019 ms) : 0, 837019
BytebuddyAgent [candidate] (836.863 ms) : 0, 836863
GlobalTracer [baseline] (230.065 ms) : 0, 230065
GlobalTracer [candidate] (230.173 ms) : 0, 230173
IAST [baseline] (23.78 ms) : 0, 23780
IAST [candidate] (22.845 ms) : 0, 22845
AppSec [baseline] (55.592 ms) : 0, 55592
AppSec [candidate] (56.056 ms) : 0, 56056
Remote Config [baseline] (622.293 µs) : 0, 622
Remote Config [candidate] (612.603 µs) : 0, 613
Telemetry [baseline] (8.765 ms) : 0, 8765
Telemetry [candidate] (8.754 ms) : 0, 8754
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (837.994 ms) : 0, 837994
BytebuddyAgent [candidate] (850.095 ms) : 0, 850095
GlobalTracer [baseline] (230.267 ms) : 0, 230267
GlobalTracer [candidate] (232.589 ms) : 0, 232589
IAST [baseline] (23.759 ms) : 0, 23759
IAST [candidate] (23.253 ms) : 0, 23253
AppSec [baseline] (55.242 ms) : 0, 55242
AppSec [candidate] (56.46 ms) : 0, 56460
Remote Config [baseline] (610.856 µs) : 0, 611
Remote Config [candidate] (610.799 µs) : 0, 611
Telemetry [baseline] (8.723 ms) : 0, 8723
Telemetry [candidate] (8.883 ms) : 0, 8883
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (834.634 ms) : 0, 834634
BytebuddyAgent [candidate] (840.201 ms) : 0, 840201
GlobalTracer [baseline] (230.281 ms) : 0, 230281
GlobalTracer [candidate] (231.497 ms) : 0, 231497
IAST [baseline] (22.418 ms) : 0, 22418
IAST [candidate] (22.387 ms) : 0, 22387
AppSec [baseline] (56.146 ms) : 0, 56146
AppSec [candidate] (56.285 ms) : 0, 56285
Remote Config [baseline] (611.41 µs) : 0, 611
Remote Config [candidate] (610.463 µs) : 0, 610
Telemetry [baseline] (8.577 ms) : 0, 8577
Telemetry [candidate] (8.681 ms) : 0, 8681
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-03-20T15:47:20 2025-03-20T15:55:07
git_branch master evanchooly/autoCodeOriginTesting
git_commit_date 1742481362 1742484759
git_commit_sha 563b2c9 98a473e
release_version 1.48.0-SNAPSHOT~563b2c9769 1.48.0-SNAPSHOT~98a473e4a7
start_time 2025-03-20T15:47:05 2025-03-20T15:54:53
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1742486506 1742486506
ci_job_id 857263826 857263826
ci_pipeline_id 59478059 59478059
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-k-wdby3f-project-304-concurrent-0-amnl1rkd 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-k-wdby3f-project-304-concurrent-0-amnl1rkd 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~98a473e4a7, baseline=1.48.0-SNAPSHOT~563b2c9769
    dateFormat X
    axisFormat %s
section baseline
no_agent (381.253 µs) : 361, 401
.   : milestone, 381,
iast (521.266 µs) : 499, 544
.   : milestone, 521,
iast_FULL (736.19 µs) : 714, 758
.   : milestone, 736,
iast_GLOBAL (564.226 µs) : 543, 586
.   : milestone, 564,
iast_HARDCODED_SECRET_DISABLED (515.593 µs) : 494, 537
.   : milestone, 516,
iast_INACTIVE (473.893 µs) : 452, 496
.   : milestone, 474,
iast_TELEMETRY_OFF (503.11 µs) : 482, 525
.   : milestone, 503,
tracing (472.248 µs) : 451, 494
.   : milestone, 472,
section candidate
no_agent (383.322 µs) : 363, 404
.   : milestone, 383,
iast (519.172 µs) : 497, 541
.   : milestone, 519,
iast_FULL (739.607 µs) : 718, 762
.   : milestone, 740,
iast_GLOBAL (558.623 µs) : 537, 580
.   : milestone, 559,
iast_HARDCODED_SECRET_DISABLED (516.14 µs) : 494, 538
.   : milestone, 516,
iast_INACTIVE (476.317 µs) : 455, 498
.   : milestone, 476,
iast_TELEMETRY_OFF (504.527 µs) : 483, 526
.   : milestone, 505,
tracing (466.425 µs) : 445, 488
.   : milestone, 466,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 381.253 µs [361.266 µs, 401.241 µs] -
iast 521.266 µs [498.843 µs, 543.688 µs] 140.012 µs (36.7%)
iast_FULL 736.19 µs [714.128 µs, 758.252 µs] 354.937 µs (93.1%)
iast_GLOBAL 564.226 µs [542.632 µs, 585.819 µs] 182.972 µs (48.0%)
iast_HARDCODED_SECRET_DISABLED 515.593 µs [493.907 µs, 537.28 µs] 134.34 µs (35.2%)
iast_INACTIVE 473.893 µs [452.125 µs, 495.661 µs] 92.64 µs (24.3%)
iast_TELEMETRY_OFF 503.11 µs [481.51 µs, 524.711 µs] 121.857 µs (32.0%)
tracing 472.248 µs [450.925 µs, 493.572 µs] 90.995 µs (23.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 383.322 µs [363.134 µs, 403.509 µs] -
iast 519.172 µs [497.386 µs, 540.958 µs] 135.85 µs (35.4%)
iast_FULL 739.607 µs [717.617 µs, 761.597 µs] 356.285 µs (92.9%)
iast_GLOBAL 558.623 µs [536.926 µs, 580.32 µs] 175.301 µs (45.7%)
iast_HARDCODED_SECRET_DISABLED 516.14 µs [494.421 µs, 537.86 µs] 132.818 µs (34.6%)
iast_INACTIVE 476.317 µs [454.953 µs, 497.682 µs] 92.996 µs (24.3%)
iast_TELEMETRY_OFF 504.527 µs [483.197 µs, 525.858 µs] 121.206 µs (31.6%)
tracing 466.425 µs [445.335 µs, 487.515 µs] 83.104 µs (21.7%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~98a473e4a7, baseline=1.48.0-SNAPSHOT~563b2c9769
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.35 ms) : 1331, 1370
.   : milestone, 1350,
appsec (1.747 ms) : 1723, 1770
.   : milestone, 1747,
appsec_no_iast (1.775 ms) : 1750, 1799
.   : milestone, 1775,
code_origins (1.684 ms) : 1657, 1711
.   : milestone, 1684,
iast (1.526 ms) : 1502, 1550
.   : milestone, 1526,
profiling (1.514 ms) : 1490, 1537
.   : milestone, 1514,
tracing (1.509 ms) : 1484, 1534
.   : milestone, 1509,
section candidate
no_agent (1.365 ms) : 1345, 1384
.   : milestone, 1365,
appsec (1.748 ms) : 1725, 1772
.   : milestone, 1748,
appsec_no_iast (1.732 ms) : 1708, 1757
.   : milestone, 1732,
code_origins (1.691 ms) : 1665, 1718
.   : milestone, 1691,
iast (1.536 ms) : 1511, 1560
.   : milestone, 1536,
profiling (1.514 ms) : 1490, 1538
.   : milestone, 1514,
tracing (1.505 ms) : 1480, 1531
.   : milestone, 1505,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.35 ms [1.331 ms, 1.37 ms] -
appsec 1.747 ms [1.723 ms, 1.77 ms] 396.326 µs (29.4%)
appsec_no_iast 1.775 ms [1.75 ms, 1.799 ms] 424.399 µs (31.4%)
code_origins 1.684 ms [1.657 ms, 1.711 ms] 333.727 µs (24.7%)
iast 1.526 ms [1.502 ms, 1.55 ms] 175.655 µs (13.0%)
profiling 1.514 ms [1.49 ms, 1.537 ms] 163.311 µs (12.1%)
tracing 1.509 ms [1.484 ms, 1.534 ms] 158.82 µs (11.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.365 ms [1.345 ms, 1.384 ms] -
appsec 1.748 ms [1.725 ms, 1.772 ms] 383.75 µs (28.1%)
appsec_no_iast 1.732 ms [1.708 ms, 1.757 ms] 367.767 µs (26.9%)
code_origins 1.691 ms [1.665 ms, 1.718 ms] 326.629 µs (23.9%)
iast 1.536 ms [1.511 ms, 1.56 ms] 170.908 µs (12.5%)
profiling 1.514 ms [1.49 ms, 1.538 ms] 148.964 µs (10.9%)
tracing 1.505 ms [1.48 ms, 1.531 ms] 140.656 µs (10.3%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master evanchooly/autoCodeOriginTesting
git_commit_date 1742481362 1742484759
git_commit_sha 563b2c9 98a473e
release_version 1.48.0-SNAPSHOT~563b2c9769 1.48.0-SNAPSHOT~98a473e4a7
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1742486941 1742486941
ci_job_id 857263828 857263828
ci_pipeline_id 59478059 59478059
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-9n9spsyq-project-304-concurrent-1-mbq71v63 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-9n9spsyq-project-304-concurrent-1-mbq71v63 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~98a473e4a7, baseline=1.48.0-SNAPSHOT~563b2c9769
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (2.353 ms) : 2309, 2397
.   : milestone, 2353,
iast (2.137 ms) : 2081, 2193
.   : milestone, 2137,
iast_GLOBAL (2.185 ms) : 2128, 2242
.   : milestone, 2185,
profiling (2.01 ms) : 1964, 2056
.   : milestone, 2010,
tracing (1.96 ms) : 1917, 2002
.   : milestone, 1960,
section candidate
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (2.364 ms) : 2320, 2408
.   : milestone, 2364,
iast (2.129 ms) : 2073, 2185
.   : milestone, 2129,
iast_GLOBAL (2.18 ms) : 2124, 2236
.   : milestone, 2180,
profiling (1.992 ms) : 1947, 2037
.   : milestone, 1992,
tracing (1.974 ms) : 1931, 2017
.   : milestone, 1974,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.48 ms [1.469 ms, 1.492 ms] -
appsec 2.353 ms [2.309 ms, 2.397 ms] 872.806 µs (59.0%)
iast 2.137 ms [2.081 ms, 2.193 ms] 656.886 µs (44.4%)
iast_GLOBAL 2.185 ms [2.128 ms, 2.242 ms] 704.917 µs (47.6%)
profiling 2.01 ms [1.964 ms, 2.056 ms] 529.606 µs (35.8%)
tracing 1.96 ms [1.917 ms, 2.002 ms] 479.465 µs (32.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.48 ms [1.469 ms, 1.492 ms] -
appsec 2.364 ms [2.32 ms, 2.408 ms] 883.541 µs (59.7%)
iast 2.129 ms [2.073 ms, 2.185 ms] 648.262 µs (43.8%)
iast_GLOBAL 2.18 ms [2.124 ms, 2.236 ms] 699.255 µs (47.2%)
profiling 1.992 ms [1.947 ms, 2.037 ms] 511.308 µs (34.5%)
tracing 1.974 ms [1.931 ms, 2.017 ms] 493.699 µs (33.3%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~98a473e4a7, baseline=1.48.0-SNAPSHOT~563b2c9769
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.725 s) : 14725000, 14725000
.   : milestone, 14725000,
appsec (15.058 s) : 15058000, 15058000
.   : milestone, 15058000,
iast (18.323 s) : 18323000, 18323000
.   : milestone, 18323000,
iast_GLOBAL (18.327 s) : 18327000, 18327000
.   : milestone, 18327000,
profiling (15.33 s) : 15330000, 15330000
.   : milestone, 15330000,
tracing (14.665 s) : 14665000, 14665000
.   : milestone, 14665000,
section candidate
no_agent (14.965 s) : 14965000, 14965000
.   : milestone, 14965000,
appsec (15.217 s) : 15217000, 15217000
.   : milestone, 15217000,
iast (18.848 s) : 18848000, 18848000
.   : milestone, 18848000,
iast_GLOBAL (18.15 s) : 18150000, 18150000
.   : milestone, 18150000,
profiling (15.087 s) : 15087000, 15087000
.   : milestone, 15087000,
tracing (15.043 s) : 15043000, 15043000
.   : milestone, 15043000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.725 s [14.725 s, 14.725 s] -
appsec 15.058 s [15.058 s, 15.058 s] 333.0 ms (2.3%)
iast 18.323 s [18.323 s, 18.323 s] 3.598 s (24.4%)
iast_GLOBAL 18.327 s [18.327 s, 18.327 s] 3.602 s (24.5%)
profiling 15.33 s [15.33 s, 15.33 s] 605.0 ms (4.1%)
tracing 14.665 s [14.665 s, 14.665 s] -60.0 ms (-0.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.965 s [14.965 s, 14.965 s] -
appsec 15.217 s [15.217 s, 15.217 s] 252.0 ms (1.7%)
iast 18.848 s [18.848 s, 18.848 s] 3.883 s (25.9%)
iast_GLOBAL 18.15 s [18.15 s, 18.15 s] 3.185 s (21.3%)
profiling 15.087 s [15.087 s, 15.087 s] 122.0 ms (0.8%)
tracing 15.043 s [15.043 s, 15.043 s] 78.0 ms (0.5%)

@pr-commenter
Copy link

pr-commenter bot commented Mar 17, 2025 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master evanchooly/autoCodeOriginTesting
git_commit_date 1742477068 1742484759
git_commit_sha 5edd9fd 98a473e
See matching parameters
Baseline Candidate
ci_job_date 1742486017 1742486017
ci_job_id 857263832 857263832
ci_pipeline_id 59478059 59478059
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume unsure
[-17745.661op/s; -98.449op/s] or [-5.981%; -0.033%]
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@evanchooly evanchooly force-pushed the evanchooly/autoCodeOriginTesting branch from 3e050e0 to 77e8ab1 Compare March 17, 2025 21:40
@pr-commenter
Copy link

pr-commenter bot commented Mar 17, 2025 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master evanchooly/autoCodeOriginTesting
git_commit_date 1742477068 1742484759
git_commit_sha 5edd9fd 98a473e
See matching parameters
Baseline Candidate
ci_job_date 1742485972 1742485972
ci_job_id 857263831 857263831
ci_pipeline_id 59478059 59478059
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@evanchooly evanchooly marked this pull request as ready for review March 18, 2025 17:26
@evanchooly evanchooly requested review from a team as code owners March 18, 2025 17:26
@evanchooly evanchooly requested review from shatzi and ygree and removed request for a team March 18, 2025 17:26
@evanchooly evanchooly force-pushed the evanchooly/autoCodeOriginTesting branch from f406227 to 4fbf185 Compare March 18, 2025 17:29
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Mar 18, 2025 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@evanchooly evanchooly force-pushed the evanchooly/autoCodeOriginTesting branch 2 times, most recently from b1802d2 to 41444a3 Compare March 20, 2025 13:56
@evanchooly evanchooly force-pushed the evanchooly/autoCodeOriginTesting branch from 41444a3 to ede71ff Compare March 20, 2025 13:59
@evanchooly evanchooly added type: enhancement comp: debugger Dynamic Instrumentation inst: java Core Java language instrumentation labels Mar 20, 2025
@evanchooly evanchooly requested review from jpbempel and tylfin March 20, 2025 14:02
@evanchooly
Copy link
Contributor Author

@jpbempel @tylfin i pushed a smaller, cleaner version after rebasing with my other PR

@evanchooly evanchooly added the tag: no release notes Changes to exclude from release notes label Mar 20, 2025
@evanchooly evanchooly merged commit 9bd19d3 into master Mar 20, 2025
274 checks passed
@evanchooly evanchooly deleted the evanchooly/autoCodeOriginTesting branch March 20, 2025 17:43
@github-actions github-actions bot added this to the 1.48.0 milestone Mar 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tylfin tylfin tylfin approved these changes

@jpbempel jpbempel jpbempel approved these changes

@shatzi shatzi Awaiting requested review from shatzi shatzi was automatically assigned from DataDog/debugger-java

@ygree ygree Awaiting requested review from ygree ygree is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation inst: java Core Java language instrumentation tag: no release notes Changes to exclude from release notes type: enhancement
Projects
None yet
Milestone
1.48.0
Development

Successfully merging this pull request may close these issues.
3 participants
@evanchooly @tylfin @jpbempel