-
Notifications
You must be signed in to change notification settings - Fork 324
Implement HTTP Endpoint Inference for Trace Resource Renaming (RFC-1051) #10393
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 10 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.59.0-SNAPSHOT~b6fd7f4c86, baseline=1.59.0-SNAPSHOT~3d62379c90
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.093 s) : 0, 1092978
Total [baseline] (10.895 s) : 0, 10894828
Agent [candidate] (1.095 s) : 0, 1095253
Total [candidate] (10.857 s) : 0, 10856527
section appsec
Agent [baseline] (1.274 s) : 0, 1274377
Total [baseline] (11.03 s) : 0, 11030164
Agent [candidate] (1.276 s) : 0, 1275828
Total [candidate] (11.104 s) : 0, 11104236
section iast
Agent [baseline] (1.226 s) : 0, 1225901
Total [baseline] (11.227 s) : 0, 11226758
Agent [candidate] (1.24 s) : 0, 1239530
Total [candidate] (11.301 s) : 0, 11300622
section profiling
Agent [baseline] (1.207 s) : 0, 1206921
Total [baseline] (10.921 s) : 0, 10920669
Agent [candidate] (1.213 s) : 0, 1212993
Total [candidate] (10.863 s) : 0, 10863330
gantt
title petclinic - break down per module: candidate=1.59.0-SNAPSHOT~b6fd7f4c86, baseline=1.59.0-SNAPSHOT~3d62379c90
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.194 ms) : 0, 1194
crashtracking [candidate] (1.196 ms) : 0, 1196
BytebuddyAgent [baseline] (656.499 ms) : 0, 656499
BytebuddyAgent [candidate] (657.773 ms) : 0, 657773
GlobalTracer [baseline] (284.982 ms) : 0, 284982
GlobalTracer [candidate] (285.595 ms) : 0, 285595
AppSec [baseline] (32.937 ms) : 0, 32937
AppSec [candidate] (32.859 ms) : 0, 32859
Debugger [baseline] (67.655 ms) : 0, 67655
Debugger [candidate] (67.244 ms) : 0, 67244
Remote Config [baseline] (608.208 µs) : 0, 608
Remote Config [candidate] (617.474 µs) : 0, 617
Telemetry [baseline] (8.888 ms) : 0, 8888
Telemetry [candidate] (8.89 ms) : 0, 8890
Flare Poller [baseline] (4.612 ms) : 0, 4612
Flare Poller [candidate] (5.434 ms) : 0, 5434
section appsec
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.188 ms) : 0, 1188
BytebuddyAgent [baseline] (695.892 ms) : 0, 695892
BytebuddyAgent [candidate] (696.549 ms) : 0, 696549
GlobalTracer [baseline] (261.449 ms) : 0, 261449
GlobalTracer [candidate] (261.429 ms) : 0, 261429
IAST [baseline] (24.726 ms) : 0, 24726
IAST [candidate] (24.822 ms) : 0, 24822
AppSec [baseline] (174.642 ms) : 0, 174642
AppSec [candidate] (173.004 ms) : 0, 173004
Debugger [baseline] (67.01 ms) : 0, 67010
Debugger [candidate] (69.426 ms) : 0, 69426
Remote Config [baseline] (749.814 µs) : 0, 750
Remote Config [candidate] (756.428 µs) : 0, 756
Telemetry [baseline] (9.41 ms) : 0, 9410
Telemetry [candidate] (9.306 ms) : 0, 9306
Flare Poller [baseline] (3.738 ms) : 0, 3738
Flare Poller [candidate] (3.752 ms) : 0, 3752
section iast
crashtracking [baseline] (1.176 ms) : 0, 1176
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (791.981 ms) : 0, 791981
BytebuddyAgent [candidate] (802.731 ms) : 0, 802731
GlobalTracer [baseline] (257.032 ms) : 0, 257032
GlobalTracer [candidate] (259.394 ms) : 0, 259394
IAST [baseline] (27.034 ms) : 0, 27034
IAST [candidate] (27.227 ms) : 0, 27227
AppSec [baseline] (34.551 ms) : 0, 34551
AppSec [candidate] (34.521 ms) : 0, 34521
Debugger [baseline] (66.234 ms) : 0, 66234
Debugger [candidate] (66.288 ms) : 0, 66288
Remote Config [baseline] (583.823 µs) : 0, 584
Remote Config [candidate] (576.701 µs) : 0, 577
Telemetry [baseline] (8.471 ms) : 0, 8471
Telemetry [candidate] (8.452 ms) : 0, 8452
Flare Poller [baseline] (3.576 ms) : 0, 3576
Flare Poller [candidate] (3.621 ms) : 0, 3621
section profiling
crashtracking [baseline] (1.216 ms) : 0, 1216
crashtracking [candidate] (1.221 ms) : 0, 1221
BytebuddyAgent [baseline] (702.657 ms) : 0, 702657
BytebuddyAgent [candidate] (706.336 ms) : 0, 706336
GlobalTracer [baseline] (222.231 ms) : 0, 222231
GlobalTracer [candidate] (222.978 ms) : 0, 222978
AppSec [baseline] (32.311 ms) : 0, 32311
AppSec [candidate] (32.669 ms) : 0, 32669
Debugger [baseline] (67.751 ms) : 0, 67751
Debugger [candidate] (68.404 ms) : 0, 68404
Remote Config [baseline] (631.604 µs) : 0, 632
Remote Config [candidate] (675.167 µs) : 0, 675
Telemetry [baseline] (8.871 ms) : 0, 8871
Telemetry [candidate] (9.018 ms) : 0, 9018
Flare Poller [baseline] (3.72 ms) : 0, 3720
Flare Poller [candidate] (3.807 ms) : 0, 3807
ProfilingAgent [baseline] (97.594 ms) : 0, 97594
ProfilingAgent [candidate] (97.645 ms) : 0, 97645
Profiling [baseline] (98.184 ms) : 0, 98184
Profiling [candidate] (98.227 ms) : 0, 98227
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.59.0-SNAPSHOT~b6fd7f4c86, baseline=1.59.0-SNAPSHOT~3d62379c90
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.091 s) : 0, 1090919
Total [baseline] (8.819 s) : 0, 8818807
Agent [candidate] (1.087 s) : 0, 1087339
Total [candidate] (8.769 s) : 0, 8768798
section iast
Agent [baseline] (1.228 s) : 0, 1228409
Total [baseline] (9.36 s) : 0, 9359677
Agent [candidate] (1.226 s) : 0, 1226373
Total [candidate] (9.398 s) : 0, 9397688
gantt
title insecure-bank - break down per module: candidate=1.59.0-SNAPSHOT~b6fd7f4c86, baseline=1.59.0-SNAPSHOT~3d62379c90
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.188 ms) : 0, 1188
crashtracking [candidate] (1.178 ms) : 0, 1178
BytebuddyAgent [baseline] (657.097 ms) : 0, 657097
BytebuddyAgent [candidate] (653.418 ms) : 0, 653418
GlobalTracer [baseline] (284.116 ms) : 0, 284116
GlobalTracer [candidate] (283.766 ms) : 0, 283766
AppSec [baseline] (32.473 ms) : 0, 32473
AppSec [candidate] (32.589 ms) : 0, 32589
Debugger [baseline] (66.378 ms) : 0, 66378
Debugger [candidate] (65.157 ms) : 0, 65157
Remote Config [baseline] (620.104 µs) : 0, 620
Remote Config [candidate] (633.758 µs) : 0, 634
Telemetry [baseline] (8.831 ms) : 0, 8831
Telemetry [candidate] (8.901 ms) : 0, 8901
Flare Poller [baseline] (4.657 ms) : 0, 4657
Flare Poller [candidate] (6.102 ms) : 0, 6102
section iast
crashtracking [baseline] (1.186 ms) : 0, 1186
crashtracking [candidate] (1.182 ms) : 0, 1182
BytebuddyAgent [baseline] (795.919 ms) : 0, 795919
BytebuddyAgent [candidate] (793.479 ms) : 0, 793479
GlobalTracer [baseline] (257.468 ms) : 0, 257468
GlobalTracer [candidate] (257.443 ms) : 0, 257443
AppSec [baseline] (34.146 ms) : 0, 34146
AppSec [candidate] (33.511 ms) : 0, 33511
Debugger [baseline] (64.828 ms) : 0, 64828
Debugger [candidate] (65.741 ms) : 0, 65741
Remote Config [baseline] (601.931 µs) : 0, 602
Remote Config [candidate] (583.382 µs) : 0, 583
Telemetry [baseline] (8.491 ms) : 0, 8491
Telemetry [candidate] (8.436 ms) : 0, 8436
Flare Poller [baseline] (3.562 ms) : 0, 3562
Flare Poller [candidate] (3.617 ms) : 0, 3617
IAST [baseline] (26.854 ms) : 0, 26854
IAST [candidate] (27.074 ms) : 0, 27074
LoadParameters
See matching parameters
SummaryFound 3 performance improvements and 1 performance regressions! Performance is the same for 16 metrics, 16 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~b6fd7f4c86, baseline=1.59.0-SNAPSHOT~3d62379c90
dateFormat X
axisFormat %s
section baseline
no_agent (17.186 ms) : 17011, 17361
. : milestone, 17186,
appsec (18.772 ms) : 18581, 18962
. : milestone, 18772,
code_origins (17.622 ms) : 17448, 17797
. : milestone, 17622,
iast (17.595 ms) : 17421, 17769
. : milestone, 17595,
profiling (18.542 ms) : 18358, 18727
. : milestone, 18542,
tracing (17.595 ms) : 17421, 17770
. : milestone, 17595,
section candidate
no_agent (18.06 ms) : 17879, 18241
. : milestone, 18060,
appsec (18.621 ms) : 18433, 18809
. : milestone, 18621,
code_origins (17.692 ms) : 17516, 17867
. : milestone, 17692,
iast (17.633 ms) : 17455, 17810
. : milestone, 17633,
profiling (18.767 ms) : 18578, 18956
. : milestone, 18767,
tracing (18.121 ms) : 17944, 18299
. : milestone, 18121,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~b6fd7f4c86, baseline=1.59.0-SNAPSHOT~3d62379c90
dateFormat X
axisFormat %s
section baseline
no_agent (1.171 ms) : 1159, 1182
. : milestone, 1171,
iast (3.187 ms) : 3143, 3231
. : milestone, 3187,
iast_FULL (5.95 ms) : 5888, 6011
. : milestone, 5950,
iast_GLOBAL (3.863 ms) : 3797, 3928
. : milestone, 3863,
profiling (2.229 ms) : 2209, 2248
. : milestone, 2229,
tracing (1.799 ms) : 1784, 1814
. : milestone, 1799,
section candidate
no_agent (1.184 ms) : 1173, 1196
. : milestone, 1184,
iast (3.087 ms) : 3048, 3127
. : milestone, 3087,
iast_FULL (5.683 ms) : 5627, 5739
. : milestone, 5683,
iast_GLOBAL (3.584 ms) : 3525, 3643
. : milestone, 3584,
profiling (2.113 ms) : 2094, 2131
. : milestone, 2113,
tracing (1.809 ms) : 1794, 1825
. : milestone, 1809,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~b6fd7f4c86, baseline=1.59.0-SNAPSHOT~3d62379c90
dateFormat X
axisFormat %s
section baseline
no_agent (1.476 ms) : 1465, 1488
. : milestone, 1476,
appsec (2.459 ms) : 2407, 2511
. : milestone, 2459,
iast (2.216 ms) : 2151, 2281
. : milestone, 2216,
iast_GLOBAL (2.256 ms) : 2190, 2321
. : milestone, 2256,
profiling (2.097 ms) : 2043, 2151
. : milestone, 2097,
tracing (2.048 ms) : 1997, 2099
. : milestone, 2048,
section candidate
no_agent (1.479 ms) : 1467, 1490
. : milestone, 1479,
appsec (2.498 ms) : 2445, 2550
. : milestone, 2498,
iast (2.222 ms) : 2157, 2287
. : milestone, 2222,
iast_GLOBAL (2.255 ms) : 2190, 2320
. : milestone, 2255,
profiling (2.072 ms) : 2019, 2125
. : milestone, 2072,
tracing (2.05 ms) : 1999, 2101
. : milestone, 2050,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~b6fd7f4c86, baseline=1.59.0-SNAPSHOT~3d62379c90
dateFormat X
axisFormat %s
section baseline
no_agent (15.056 s) : 15056000, 15056000
. : milestone, 15056000,
appsec (14.691 s) : 14691000, 14691000
. : milestone, 14691000,
iast (18.349 s) : 18349000, 18349000
. : milestone, 18349000,
iast_GLOBAL (17.671 s) : 17671000, 17671000
. : milestone, 17671000,
profiling (14.756 s) : 14756000, 14756000
. : milestone, 14756000,
tracing (14.68 s) : 14680000, 14680000
. : milestone, 14680000,
section candidate
no_agent (15.717 s) : 15717000, 15717000
. : milestone, 15717000,
appsec (14.596 s) : 14596000, 14596000
. : milestone, 14596000,
iast (18.186 s) : 18186000, 18186000
. : milestone, 18186000,
iast_GLOBAL (17.745 s) : 17745000, 17745000
. : milestone, 17745000,
profiling (15.285 s) : 15285000, 15285000
. : milestone, 15285000,
tracing (15.031 s) : 15031000, 15031000
. : milestone, 15031000,
|
c8d180b to
daa5a49
Compare
| this.isTraceRoot = isTraceRoot; | ||
| this.spanKind = null == spanKind ? EMPTY : UTF8BytesString.create(spanKind); | ||
| this.peerTags = peerTags == null ? Collections.emptyList() : peerTags; | ||
| this.httpMethod = null == httpMethod ? EMPTY : UTF8BytesString.create(httpMethod); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if the feature is opt-in I think you should not add aggregation dimentions for httpMethod and httpEndpoint if disabled otherwise the metrics buckets will change
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! solved in b6fd7f4
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2c46dc2faf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
dd-trace-core/src/main/java/datadog/trace/core/endpoint/EndpointResolver.java
Show resolved
Hide resolved
c27ecbb to
b6fd7f4
Compare
What Does This Do
This PR implements automatic HTTP endpoint inference to improve trace resource naming and reduce cardinality in APM metrics. When http.route is unavailable or unreliable, the tracer now computes a simplified endpoint from the URL path by replacing dynamic segments (IDs, UUIDs, long strings) with standardized placeholders.
Changes
{param:int}, {param:int_id}, {param:hex}, {param:hex_id}, {param:str}{METHOD} {endpoint}DD_TRACE_RESOURCE_RENAMING_ENABLED: Main feature flagDD_APPSEC_ENABLED=true)DD_TRACE_RESOURCE_RENAMING_ALWAYS_SIMPLIFIED_ENDPOINT: Testing flagMotivation
RFC
Additional Notes
Confirmed, it passes the ST
Contributor Checklist
type:and (comp:orinst:) labels in addition to any useful labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: APPSEC-58611