Add technical doc for dynamic fields

[ykmr1224]

Description

• Add technical doc for dynamic fields

Related Issues

#4984

Check List

• New functionality includes testing.
• New functionality has been documented.
• New functionality has javadoc added.
• New functionality has a user manual doc added.
• New PPL command checklist all confirmed.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff or -s.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[coderabbitai]

📝 Walkthrough

Summary by CodeRabbit

• Documentation
  • Added a comprehensive end-to-end guide for Dynamic Fields in PPL covering concepts, architecture, runtime workflow, schema expansion, type normalization, usage examples, limitations, performance considerations, testing guidance, extension steps, and best practices.
  • Updated PPL commands documentation to reference the new Dynamic Fields guide for clearer cross-references and guidance when extending command support.

Walkthrough

Adds a new developer guide docs/dev/dynamic-fields.md describing the Dynamic Fields feature (architecture, components, runtime workflow, expansion, integration steps, limitations, examples). Also updates docs/dev/ppl-commands.md to reference the new guide from the FieldResolutionVisitor / spath note.

Changes

Cohort / File(s)	Summary
Dynamic Fields Documentation docs/dev/dynamic-fields.md	New comprehensive developer guide documenting dynamic fields: concepts (static vs dynamic, _MAP), components (FieldResolutionVisitor, FieldResolutionResult, CalciteRelNodeVisitor, DynamicFieldsHelper, DynamicFieldsResultProcessor), runtime workflow, dynamic field expansion, type normalization, ordering guarantees, integration steps for adding command support, tests, limitations, examples, and future notes.
Docs Cross-Reference Update docs/dev/ppl-commands.md	Minor comment edit to the FieldResolutionVisitor / spath support note to add an explicit reference to the new (docs/dev/dynamic-fields.md) guide.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Add technical doc for dynamic fields' directly and clearly describes the main change—adding documentation for the dynamic fields feature. It aligns with the primary objective of the PR.
Description check	✅ Passed	The description explicitly states 'Add technical doc for dynamic fields' and references the related issue, matching the changeset which adds comprehensive documentation for the dynamic fields feature.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Fix all issues with AI agents

In `@docs/dev/dynamic-fields.md`:
- Around line 6-7: Edit the sentence in docs/dev/dynamic-fields.md so it reads:
"The key idea is to map fields that are directly referred to static fields, and
others to dynamic fields. Most of the command implementations only consider
static fields, while a very few commands such as join need to handle dynamic
fields." Replace "which is directly referred" with "that are directly referred",
change "most of the command implementation" to "Most of the command
implementations", and change "very few command such as join needs" to "a very
few commands such as join need" to fix subject-verb agreement and clarity.
- Line 34: Update the documentation entry that incorrectly references
DynamicFieldsHelper: replace the wrong file path
`core/src/main/java/org/opensearch/sql/calcite/plan/DynamicFieldsConstants.java`
with the correct component path
`core/src/main/java/org/opensearch/sql/calcite/DynamicFieldsHelper.java`,
ensuring the term DynamicFieldsHelper remains and not DynamicFieldsConstants so
the doc points to the actual class.

🧹 Nitpick comments (2)

docs/dev/dynamic-fields.md (2)

49-65: Consider adding language identifiers to code blocks.

Several code blocks are missing language identifiers (lines 49, 147, 283). While not critical, adding identifiers improves syntax highlighting and readability.

💡 Suggested language identifiers

For line 49 (workflow/pseudo-code):

-```
+```text
Query: source=logs | spath input=data | fields a, b, prefix*

For line 147 (table example):

-```
+```text
Before expansion (internal representation):

For line 283 (execution plan):

-```
+```text
Project(a, b, _MAP)

Also applies to: 147-159, 283-289

---

205-206: Fix list indentation for consistency.

The bullet points are indented with 2 spaces instead of 0, which is inconsistent with markdown standards.

🔧 Proposed indentation fix

 **Note**: Command implementation does not need to handle dynamic fields unless:
-  - The command takes multiple inputs (e.g. join, append)
-  - The command works against all the fields
+- The command takes multiple inputs (e.g. join, append)
+- The command works against all the fields

[coderabbitai]

Actionable comments posted: 2

🤖 Fix all issues with AI agents

In `@docs/dev/dynamic-fields.md`:
- Line 10: Fix the grammatical error in the “Field Resolution” bullet by
correcting subject-verb agreement. In the docs section labeled “Field
Resolution,” replace the phrase “Analyze query and determines” with “Analyzes
query and determines” to maintain consistency in the description.
- Line 34: The file path for DynamicFieldsHelper in the References section is
incorrect; it includes an extra /plan subdirectory. Locate the Reference section
where DynamicFieldsHelper is mentioned and update the path from
core/src/main/java/org/opensearch/sql/calcite/plan/DynamicFieldsHelper.java to
core/src/main/java/org/opensearch/sql/calcite/DynamicFieldsHelper.java to
reflect the correct file location.

[coderabbitai]

Actionable comments posted: 2

🤖 Fix all issues with AI agents

In `@docs/dev/dynamic-fields.md`:
- Around line 42-44: The docs mix "STRING" and "VARCHAR" for dynamic field
values; standardize terminology to "VARCHAR" (to match Calcite/PPL SQL type)
across the document — update the phrases in the "Converts dynamic field values
to STRING type for consistency" and "Creates expanded schema with both static
and dynamic fields" sections and the occurrences referenced (around lines
143-144 and 253), and run a global search in docs/dev/dynamic-fields.md to
replace any remaining "STRING" or "VARCHAR" mismatches so all references to
dynamic field value types consistently use "VARCHAR".
- Around line 49-65: Update the documentation examples that reference the
non-existent test indices 'logs', 'logs1', and 'logs2' so they use canonical
test indices (e.g., 'accounts', 'people', or 'events') or add matching test
fixtures; specifically, change the example queries and any accompanying sample
output in dynamic-fields.md to use an existing index name or create
corresponding integ-test/sample data for 'logs*' entries and ensure tests cover
the examples so they run against the repo's fixture set.