Skip to content

docs: processors: tda doc for new processor plugin#2506

Merged
eschabell merged 3 commits intofluent:masterfrom
eschabell:erics_processors_tda_new
Mar 23, 2026
Merged

docs: processors: tda doc for new processor plugin#2506
eschabell merged 3 commits intofluent:masterfrom
eschabell:erics_processors_tda_new

Conversation

@eschabell
Copy link
Copy Markdown
Collaborator

@eschabell eschabell commented Mar 20, 2026

Topilogical Data Analysis: add TDA persistent homology processor page

  • New pipeline/processors/tda.md covering the TDA processor
  • Documents window_size, min_points, embed_dim, embed_delay, and threshold parameters
  • Notes all three output gauges: betti0, betti1, betti2
  • Adds SUMMARY.md entry between SQL and Filters as processors
  • Remove spaces around em dashes in Betti number table
  • Spell out ordinal "30th" as "thirtieth"
  • Wrap Ripser in backticks to suppress false spelling suggestions
  • Wrap TDA in backticks in H1 to suppress heading capitalization suggestion
  • SUMMARY.md: add TDA processor entry and sort processors alphabetically

Fixes #2497

Summary by CodeRabbit

Release Notes

  • Documentation
    • Added comprehensive documentation for a new Topological Data Analysis (TDA) processor, including behavior, output metrics, configuration parameters, delay-embedding options, and examples.
    • Reorganized the Processors documentation section with updated topic ordering.
    • Updated installation/build docs to list new options for Fluent Bit logs and the TDA processor (defaults: On).

@eschabell eschabell self-assigned this Mar 20, 2026
@eschabell eschabell requested a review from a team as a code owner March 20, 2026 20:52
@eschabell eschabell added waiting-on-review Waiting on a review from mainteners 5.0 labels Mar 20, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 20, 2026

Warning

Rate limit exceeded

@eschabell has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 5 minutes and 58 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2fcb67b8-9c79-447b-b6d2-b3316d69b3ce

📥 Commits

Reviewing files that changed from the base of the PR and between 9a35ec5 and 42ed312.

📒 Files selected for processing (1)
  • installation/downloads/source/build-and-install.md
📝 Walkthrough

Walkthrough

Adds a new documentation page for a Topological Data Analysis (TDA) metrics processor and updates the Processors table-of-contents ordering in SUMMARY.md; also documents two build-time options in the build/install docs.

Changes

Cohort / File(s) Summary
Processors TOC
SUMMARY.md
Reordered entries under Processors: inserted Topological data analysis (TDA) and moved the positions of Filters as processors and Conditional processing. (Docs-only; check broken/updated links.)
TDA processor docs
pipeline/processors/tda.md
New page describing TDA processor behavior: metric aggregation into feature vectors per (namespace, subsystem), sliding window, optional delay embedding (embed_dim/embed_delay), persistent homology via Ripser with threshold handling, and emitted gauge metrics fluentbit_tda_betti0/1/2. Includes config params and two YAML examples.
Build/install options
installation/downloads/source/build-and-install.md
Added build option entries for FLB_IN_FLUENTBIT_LOGS and FLB_PROCESSOR_TDA (both default On).

Sequence Diagram(s)

(omitted — changes are documentation and TOC updates)

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

Suggested reviewers

  • cosmo0920
  • lecaros
  • patrick-stephens

Poem

🐰 A curious rabbit hops through the docs tonight,
TDA flowers bloom in metric-light.
Betti beats count loops in a hop and a twirl,
Sliding windows stack — watch topology unfurl! ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: adding documentation for the new TDA processor plugin.
Linked Issues check ✅ Passed The PR fully addresses issue #2497 requirements: documents TDA processor behavior, all config parameters (window_size, min_points, embed_dim, embed_delay, threshold), outputs three gauge metrics (betti0, betti1, betti2), and adds SUMMARY.md entry.
Out of Scope Changes check ✅ Passed All changes are in-scope: TDA processor documentation, SUMMARY.md table-of-contents update, and build options documentation for related features, all directly supporting issue #2497 requirements.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@eschabell eschabell force-pushed the erics_processors_tda_new branch from 3a09e64 to ac16094 Compare March 20, 2026 20:56
  - Remove spaces around em dashes in Betti number table
  - Spell out ordinal "30th" as "thirtieth"
  - Wrap Ripser in backticks to suppress false spelling suggestions
  - Wrap TDA in backticks in H1 to suppress heading capitalization suggestion

  Applies to fluent#2497

Signed-off-by: Eric D. Schabell <eric@schabell.org>
…tically

  - Add Topological data analysis (TDA) entry under Processors section
  - Sort all processor entries alphabetically (Conditional processing and Filters as processors moved to correct positions)

  Applies to fluent#2497

Signed-off-by: Eric D. Schabell <eric@schabell.org>
@eschabell eschabell force-pushed the erics_processors_tda_new branch from ac16094 to bed5ee4 Compare March 20, 2026 21:00
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
pipeline/processors/tda.md (1)

3-3: Use neutral wording for the capability claim

Line 3 currently makes a broad comparative claim (“methods miss”), which reads promotional. Please keep it strictly technical and evidence-neutral.

Proposed wording
-This processor applies [Topological Data Analysis](https://en.wikipedia.org/wiki/Topological_data_analysis) (`TDA`) to incoming metrics using a sliding window and `Ripser` persistent homology. It computes Betti numbers that characterize the topological shape of the metric signal over time, which can surface structural patterns (such as recurring cycles or anomalies) that traditional statistical methods miss.
+This processor applies [Topological Data Analysis](https://en.wikipedia.org/wiki/Topological_data_analysis) (`TDA`) to incoming metrics using a sliding window and `Ripser` persistent homology. It computes Betti numbers that characterize the topological shape of the metric signal over time, which can help identify structural patterns such as recurring cycles or anomalies.

Based on learnings: “ensure all Markdown documentation remains technical and neutral… avoid promotional language.”

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pipeline/processors/tda.md` at line 3, The sentence describing Topological
Data Analysis (`TDA`) with Ripser and Betti numbers uses promotional phrasing
("that traditional statistical methods miss"); replace that comparative clause
with neutral, technical wording—e.g., change the ending to "which can reveal
structural patterns (such as recurring cycles or anomalies) in the metric
signal" or "which may not be captured by some standard statistical
techniques"—so the line mentioning Topological Data Analysis (`TDA`), Ripser,
and Betti numbers stays factual and evidence-neutral.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@pipeline/processors/tda.md`:
- Line 3: The sentence describing Topological Data Analysis (`TDA`) with Ripser
and Betti numbers uses promotional phrasing ("that traditional statistical
methods miss"); replace that comparative clause with neutral, technical
wording—e.g., change the ending to "which can reveal structural patterns (such
as recurring cycles or anomalies) in the metric signal" or "which may not be
captured by some standard statistical techniques"—so the line mentioning
Topological Data Analysis (`TDA`), Ripser, and Betti numbers stays factual and
evidence-neutral.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 97768797-aada-4cf0-832c-e37e82b34b27

📥 Commits

Reviewing files that changed from the base of the PR and between 3179471 and bed5ee4.

📒 Files selected for processing (2)
  • SUMMARY.md
  • pipeline/processors/tda.md

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@installation/downloads/source/build-and-install.md`:
- Line 236: Update the description for the FLB_PROCESSOR_TDA entry so the
acronym is expanded: replace the current text "Enable TDA processor" with
"Enable Topological Data Analysis (TDA) processor" (i.e., update the description
associated with FLB_PROCESSOR_TDA to spell out Topological Data Analysis and
include the TDA abbreviation in parentheses).

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 02579396-ccdb-44ca-89f9-05e42da1917e

📥 Commits

Reviewing files that changed from the base of the PR and between bed5ee4 and 9a35ec5.

📒 Files selected for processing (1)
  • installation/downloads/source/build-and-install.md

  - Add FLB_IN_FLUENTBIT_LOGS to input plugins table
  - Add FLB_PROCESSOR_TDA to processor plugins table

  Applies to fluent#2497

Signed-off-by: Eric D. Schabell <eric@schabell.org>
@eschabell eschabell force-pushed the erics_processors_tda_new branch from 9a35ec5 to 42ed312 Compare March 21, 2026 18:43
@eschabell
Copy link
Copy Markdown
Collaborator Author

@patrick-stephens ready for review!

@eschabell eschabell removed the waiting-on-review Waiting on a review from mainteners label Mar 23, 2026
@eschabell eschabell merged commit 847359a into fluent:master Mar 23, 2026
8 checks passed
@eschabell eschabell deleted the erics_processors_tda_new branch March 23, 2026 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feb 2026 audit - docs: processors: tda plugin doc missing (new)

2 participants