filebeat/input/{v2,logv2}: add Redirector interface for general input migration

[efd6]

Proposed commit message

filebeat/input/{v2,logv2}: add Redirector interface for general input migration

Add an optional Redirector interface to the v2 input system so the
Loader can transparently redirect one input type to another. This
replaces the bespoke approach where logv2 directly imported and
instantiated the filestream plugin.

The Loader checks for Redirector before calling Create. If Redirect
returns a non-empty target type, the Loader resolves the target from
its plugin registry and calls its Create with the translated config.
Only one redirect hop is allowed. Delete follows the same redirect
path so CheckConfig cleanup operates on the correct target.

Refactor the logv2 manager to implement Redirector instead of holding
a direct reference to the filestream manager. This removes logv2's
import of the filestream package, eliminates a duplicate filestream
manager instance (and its redundant Init call), and makes the pattern
reusable for future V2-to-V2 migrations such as httpjson to cel.

Note

It may be worth reading the partner PR, #49614, to this to see its operation in context.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5d225aed-5444-4938-80c4-5ef228932bef

📥 Commits

Reviewing files that changed from the base of the PR and between c0d3721 and acbaa75.

📒 Files selected for processing (9)

• changelog/fragments/1774300895-id6318-filebeat-director.yaml
• filebeat/input/default-inputs/inputs.go
• filebeat/input/logv2/README.md
• filebeat/input/logv2/input.go
• filebeat/input/logv2/input_test.go
• filebeat/input/v2/loader.go
• filebeat/input/v2/loader_test.go
• filebeat/input/v2/redirect.go
• filebeat/input/v2/util_test.go

✅ Files skipped from review due to trivial changes (4)

• changelog/fragments/1774300895-id6318-filebeat-director.yaml
• filebeat/input/v2/loader_test.go
• filebeat/input/v2/redirect.go
• filebeat/input/logv2/input.go

🚧 Files skipped from review as they are similar to previous changes (2)

• filebeat/input/default-inputs/inputs.go
• filebeat/input/logv2/README.md

---

📝 Walkthrough

Walkthrough

Adds a redirect mechanism for Filebeat inputs: a new exported Redirector interface lets input managers translate a requested input into a different target type and config. The logv2 inputs were refactored to remove direct filestream/statestore coupling and to use redirect semantics instead. The v2 loader now detects Redirector implementations and resolves single-hop redirects during Configure and Delete, invoking the resolved target plugin with the translated config. Function signatures for LogPluginV2 and ContainerPluginV2 no longer accept a statestore parameter. Tests covering manager and loader redirect and delete behaviors were added.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

• 🛠️ Update Documentation: Commit on current branch
• 🛠️ Update Documentation: Create PR

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[chrisberkhout]

@efd6 Is it correct that we separately pursued an approach where we'd migrate to cel but let it get old httpjson input cursor data? What's the status of that? Was that unsuccessful, or just incomplete, or is it not the same use case as this?

[efd6]

This is essentially that, or at least an evolution of it (there was a shared state proposal that was abandoned, which may be what you are thinking of). There was an early, ugly, alternative approach that hid CEL in HTTP JSON. That has been abandoned as it was too ugly. There is also the non-cursor-state case that's handled by fleet; that is in parallel with this (I have notes for a KB article for how to use them together and when to do so).

The logic for the collection of HTTP JSON cursor state is in the partner PR.