enhancement(agent-data-plane): OTTL library lexer & API update#1246
Merged
enhancement(agent-data-plane): OTTL library lexer & API update#1246
Conversation
Binary Size Analysis (Agent Data Plane)Target: 850def7 (baseline) vs 99a37f6 (comparison) diff
|
| Module | File Size | Symbols |
|---|---|---|
core |
-9.81 KiB | 73 |
chumsky |
+1.14 KiB | 29 |
ottl::parser::grammar |
-917 B | 1 |
[Unmapped] |
-843 B | 1 |
ottl::lexer::Token |
+768 B | 33 |
agent_data_plane::components::ottl_transform_processor |
+527 B | 5 |
agent_data_plane::components::ottl_filter_processor |
+484 B | 7 |
alloc |
-386 B | 113 |
ottl::parser::arena |
+319 B | 8 |
ottl::parser::eval |
-191 B | 5 |
[sections] |
-154 B | 4 |
agent_data_plane::cli::dogstatsd |
-16 B | 1 |
agent_data_plane::cli::debug |
-8 B | 1 |
h2 |
+3 B | 24 |
saluki_io::net::listener |
+2 B | 4 |
rustc_demangle |
+2 B | 1 |
rustls |
+1 B | 12 |
http |
+1 B | 28 |
hashbrown |
+1 B | 6 |
axum_core |
+1 B | 2 |
Detailed Symbol Changes
FILE SIZE VM SIZE
-------------- --------------
[NEW] +14.1Ki [NEW] +13.9Ki _<chumsky::primitive::Choice<$LP$U_,V_,W_,X_,Y_,Z_$RP$> as chumsky::Parser<I,O,E>>::go::h92c9ba1c5cddff1c
[NEW] +10.7Ki [NEW] +10.6Ki <saluki_core::data_model::event::Event as core::clone::Clone>::clone.10057
[NEW] +7.42Ki [NEW] +7.24Ki _<chumsky::primitive::Choice<$LP$U_,V_,W_,X_,Y_,Z_$RP$> as chumsky::Parser<I,O,E>>::go_check::h64dc5d92370373af
[NEW] +6.84Ki [NEW] +6.76Ki chumsky::Parser::parse_with_state::hd726057c3531a002
[NEW] +6.24Ki [NEW] +6.08Ki _<chumsky::primitive::Choice<$LP$V_,W_,X_,Y_,Z_$RP$> as chumsky::Parser<I,O,E>>::go::h63107a7b19120838
[NEW] +5.52Ki [NEW] +5.38Ki <hickory_proto::rr::record_data::RData as core::clone::Clone>::clone.8513
[NEW] +5.44Ki [NEW] +5.28Ki _<chumsky::combinator::Foldl<F,A,B,OB,E> as chumsky::Parser<I,O,E>>::go_emit::h18ce1914111fc9f8
[NEW] +5.13Ki [NEW] +5.00Ki <hickory_proto::rr::record_data::RData as core::clone::Clone>::clone.11181
[NEW] +4.71Ki [NEW] +4.59Ki <rustls::error::Error as core::clone::Clone>::clone.10035
[NEW] +4.71Ki [NEW] +4.59Ki <rustls::error::Error as core::clone::Clone>::clone.11196
[DEL] -4.71Ki [DEL] -4.59Ki <rustls::error::Error as core::clone::Clone>::clone.11157
[DEL] -4.71Ki [DEL] -4.59Ki <rustls::error::Error as core::clone::Clone>::clone.9996
[DEL] -5.13Ki [DEL] -5.00Ki <hickory_proto::rr::record_data::RData as core::clone::Clone>::clone.11142
[DEL] -5.44Ki [DEL] -5.28Ki _<chumsky::combinator::Foldl<F,A,B,OB,E> as chumsky::Parser<I,O,E>>::go_emit::ha25c87fd0f9442b8
[DEL] -5.52Ki [DEL] -5.38Ki <hickory_proto::rr::record_data::RData as core::clone::Clone>::clone.8474
[DEL] -5.68Ki [DEL] -5.51Ki _<chumsky::primitive::Choice<$LP$V_,W_,X_,Y_,Z_$RP$> as chumsky::Parser<I,O,E>>::go::hb744c010081be0a4
[DEL] -6.84Ki [DEL] -6.76Ki chumsky::Parser::parse_with_state::h644c6b0ec12cf9c0
[DEL] -7.12Ki [DEL] -6.94Ki _<chumsky::primitive::Choice<$LP$U_,V_,W_,X_,Y_,Z_$RP$> as chumsky::Parser<I,O,E>>::go_check::h3c524ae30820c25e
-0.7% -10.5Ki -0.1% -646 [556 Others]
[DEL] -10.7Ki [DEL] -10.6Ki <saluki_core::data_model::event::Event as core::clone::Clone>::clone.10018
[DEL] -13.5Ki [DEL] -13.3Ki _<chumsky::primitive::Choice<$LP$U_,V_,W_,X_,Y_,Z_$RP$> as chumsky::Parser<I,O,E>>::go::h0b9e871e0d5c8f2e
-0.0% -9.06Ki +0.0% +844 TOTAL
Regression Detector (Agent Data Plane)Regression Detector ResultsRun ID: 57e6c0fe-e623-4009-aba1-38e0def3c7aa Baseline: 5eb804c Optimization Goals: ✅ No significant changes detected
|
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ❌ | otlp_ingest_logs_5mb_memory | memory utilization | +17.40 | [+16.77, +18.03] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_logs_5mb_throughput | ingress throughput | +0.01 | [-0.12, +0.14] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_logs_5mb_cpu | % cpu utilization | -0.87 | [-5.87, +4.14] | 1 | (metrics) (profiles) (logs) |
Fine details of change detection per experiment
| perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
|---|---|---|---|---|---|---|
| ❌ | otlp_ingest_logs_5mb_memory | memory utilization | +17.40 | [+16.77, +18.03] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_1mb_3k_contexts_cpu | % cpu utilization | +13.40 | [-43.39, +70.19] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_500mb_3k_contexts_throughput | ingress throughput | +2.93 | [+2.80, +3.06] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_metrics_5mb_cpu | % cpu utilization | +2.89 | [-5.53, +11.31] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_5mb_cpu | % cpu utilization | +2.17 | [-0.07, +4.40] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_100mb_3k_contexts_cpu | % cpu utilization | +1.25 | [-4.92, +7.41] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_5mb_memory | memory utilization | +0.74 | [+0.50, +0.99] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_500mb_3k_contexts_memory | memory utilization | +0.54 | [+0.38, +0.71] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_ottl_filtering_5mb_memory | memory utilization | +0.53 | [+0.19, +0.86] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_ottl_filtering_5mb_cpu | % cpu utilization | +0.25 | [-2.10, +2.60] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_512kb_3k_contexts_memory | memory utilization | +0.16 | [-0.01, +0.33] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_100mb_3k_contexts_memory | memory utilization | +0.15 | [-0.04, +0.34] | 1 | (metrics) (profiles) (logs) |
| ➖ | quality_gates_rss_dsd_medium | memory utilization | +0.14 | [-0.05, +0.33] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_10mb_3k_contexts_memory | memory utilization | +0.09 | [-0.09, +0.28] | 1 | (metrics) (profiles) (logs) |
| ➖ | quality_gates_rss_dsd_ultraheavy | memory utilization | +0.07 | [-0.06, +0.21] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_logs_5mb_throughput | ingress throughput | +0.01 | [-0.12, +0.14] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_5mb_throughput | ingress throughput | +0.00 | [-0.02, +0.02] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_10mb_3k_contexts_throughput | ingress throughput | +0.00 | [-0.14, +0.14] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_ottl_transform_5mb_throughput | ingress throughput | +0.00 | [-0.02, +0.02] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_ottl_filtering_5mb_throughput | ingress throughput | -0.00 | [-0.02, +0.02] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_512kb_3k_contexts_throughput | ingress throughput | -0.00 | [-0.05, +0.05] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_1mb_3k_contexts_throughput | ingress throughput | -0.00 | [-0.06, +0.06] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_metrics_5mb_throughput | ingress throughput | -0.00 | [-0.14, +0.13] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_100mb_3k_contexts_throughput | ingress throughput | -0.01 | [-0.05, +0.03] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_1mb_3k_contexts_memory | memory utilization | -0.09 | [-0.26, +0.09] | 1 | (metrics) (profiles) (logs) |
| ➖ | quality_gates_rss_idle | memory utilization | -0.15 | [-0.18, -0.12] | 1 | (metrics) (profiles) (logs) |
| ➖ | quality_gates_rss_dsd_low | memory utilization | -0.16 | [-0.35, +0.04] | 1 | (metrics) (profiles) (logs) |
| ➖ | quality_gates_rss_dsd_heavy | memory utilization | -0.25 | [-0.39, -0.11] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_ottl_transform_5mb_cpu | % cpu utilization | -0.51 | [-2.74, +1.72] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_traces_ottl_transform_5mb_memory | memory utilization | -0.55 | [-0.79, -0.31] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_logs_5mb_cpu | % cpu utilization | -0.87 | [-5.87, +4.14] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_500mb_3k_contexts_cpu | % cpu utilization | -1.09 | [-2.56, +0.39] | 1 | (metrics) (profiles) (logs) |
| ➖ | otlp_ingest_metrics_5mb_memory | memory utilization | -2.08 | [-2.33, -1.83] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_10mb_3k_contexts_cpu | % cpu utilization | -3.13 | [-32.79, +26.53] | 1 | (metrics) (profiles) (logs) |
| ➖ | dsd_uds_512kb_3k_contexts_cpu | % cpu utilization | -3.41 | [-59.73, +52.91] | 1 | (metrics) (profiles) (logs) |
Bounds Checks: ✅ Passed
| perf | experiment | bounds_check_name | replicates_passed | observed_value | links |
|---|---|---|---|---|---|
| ✅ | quality_gates_rss_dsd_heavy | memory_usage | 10/10 | 112.64MiB ≤ 140MiB | (metrics) (profiles) (logs) |
| ✅ | quality_gates_rss_dsd_low | memory_usage | 10/10 | 33.61MiB ≤ 50MiB | (metrics) (profiles) (logs) |
| ✅ | quality_gates_rss_dsd_medium | memory_usage | 10/10 | 52.74MiB ≤ 75MiB | (metrics) (profiles) (logs) |
| ✅ | quality_gates_rss_dsd_ultraheavy | memory_usage | 10/10 | 166.56MiB ≤ 200MiB | (metrics) (profiles) (logs) |
| ✅ | quality_gates_rss_idle | memory_usage | 10/10 | 20.91MiB ≤ 40MiB | (metrics) (profiles) (logs) |
Explanation
Confidence level: 90.00%
Effect size tolerance: |Δ mean %| ≥ 5.00%
Performance changes are noted in the perf column of each table:
- ✅ = significantly better comparison variant performance
- ❌ = significantly worse comparison variant performance
- ➖ = no significant change in performance
A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".
For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:
-
Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
-
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
-
Its configuration does not mark it "erratic".
dd-octo-sts bot
pushed a commit
that referenced
this pull request
Mar 16, 2026
## Summary
This PR restructures how OTTL path expressions are represented and
resolved, moving index keys from a flat list on the entire path to
per-field storage. It also fixes several lexer rules and aligns the EBNF
grammar documentation with the actual implementation and upstream
OpenTelemetry specification.
### What changed
*Path representation refactor (core change)*
* Introduced a Field struct that pairs a field name with its own index
keys (e.g., attributes["key"] becomes Field { name: "attributes", keys:
[String("key")] }).
* PathExpr now holds Vec<Field> instead of Vec<String> segments + a flat
Vec<IndexExpr>.
* ResolvedPath stores Vec<Field> instead of a full_path string and
separate indexes vector.
* PathAccessor::get/set now receive &[Field] instead of (&str,
&[IndexExpr]), giving integrators full structured access to the path
with per-field keys.
* Path resolver lookup key is still built by joining field names with
dots (ignoring keys), so existing PathResolverMap registrations remain
compatible.
* Updated both SpanFilterContext and SpanTransformContext (filter and
transform processors) to use the new &[Field] API.
*Lexer improvements*
* Whitespace rule now includes \n and \r in addition to spaces and tabs.
* Float literals support scientific notation (e.g., 1.0e10, .5E-3).
* String literal regex documented with EBNF-style escape sequence
semantics.
*EBNF grammar documentation*
* Aligned PATH rule to [LOWER_IDENT, "."], FIELD, {".", FIELD} with
FIELD = LOWER_IDENT, {INDEX}.
* Added ESCAPE_SEQ, STRING_CHAR productions for string literals.
* Added scientific notation to FLOAT_LITERAL.
* Added BYTES_LITERAL production, updated WS to include newlines.
*Tests*
Added ~390 new test lines covering: per-field index key correctness,
whitespace handling (newlines, tabs), float scientific notation, string
edge cases (empty strings, escape sequences, unicode), and comprehensive
path expression parsing.
## Change Type
- [ ] Bug fix
- [x] New feature
- [x] Non-functional (chore, refactoring, docs)
- [ ] Performance
## How did you test this PR?
Unit tests, CICD df99695
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR restructures how OTTL path expressions are represented and resolved, moving index keys from a flat list on the entire path to per-field storage. It also fixes several lexer rules and aligns the EBNF grammar documentation with the actual implementation and upstream OpenTelemetry specification.
What changed
Path representation refactor (core change)
Lexer improvements
EBNF grammar documentation
Tests
Added ~390 new test lines covering: per-field index key correctness, whitespace handling (newlines, tabs), float scientific notation, string edge cases (empty strings, escape sequences, unicode), and comprehensive path expression parsing.
Change Type
How did you test this PR?
Unit tests, CICD