Skip to content

Feature/cel prefilter#759

Open
YakirOren wants to merge 5 commits intomainfrom
feature/cel-prefilter
Open

Feature/cel prefilter#759
YakirOren wants to merge 5 commits intomainfrom
feature/cel-prefilter

Conversation

@YakirOren
Copy link
Copy Markdown
Contributor

@YakirOren YakirOren commented Mar 29, 2026

Summary by CodeRabbit

Release Notes

  • New Features

    • Added prefiltering capability for rules to skip unnecessary evaluation based on event properties, including file paths, HTTP methods, network directions, and port numbers
  • Metrics & Observability

    • New metrics available to track and monitor prefiltered rule events

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 29, 2026

Warning

Rate limit exceeded

@YakirOren has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 19 minutes and 40 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 19 minutes and 40 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 76a95516-7953-4d73-962c-a91bcb13a9d3

📥 Commits

Reviewing files that changed from the base of the PR and between 9c33ec6 and 266a39c.

📒 Files selected for processing (14)
  • cmd/main.go
  • pkg/containerwatcher/v2/tracers/httpparse.go
  • pkg/metricsmanager/metrics_manager_interface.go
  • pkg/metricsmanager/metrics_manager_mock.go
  • pkg/metricsmanager/metrics_manager_noop.go
  • pkg/metricsmanager/prometheus/prometheus.go
  • pkg/rulebindingmanager/cache/cache.go
  • pkg/rulebindingmanager/cache/cache_test.go
  • pkg/rulebindingmanager/testdata/ruleparamsfiles/rule-ssh-params.yaml
  • pkg/rulemanager/extract_event_fields_test.go
  • pkg/rulemanager/prefilter/prefilter.go
  • pkg/rulemanager/prefilter/prefilter_test.go
  • pkg/rulemanager/rule_manager.go
  • pkg/rulemanager/types/v1/types.go
📝 Walkthrough

Walkthrough

A pre-filtering mechanism was implemented to skip events before CEL rule evaluation. This includes a new prefilter package with HTTP/SSH filtering logic, integration into rule binding and rule managers to populate and apply prefilters, and metrics tracking via a new Prometheus counter and no-op implementation.

Changes

Cohort / File(s) Summary
Metrics Manager Interface & Implementations
cmd/main.go, pkg/metricsmanager/metrics_manager_interface.go, pkg/metricsmanager/metrics_manager_mock.go, pkg/metricsmanager/metrics_manager_noop.go
Added ReportRulePrefiltered(ruleName string) interface method. Switched from NewMetricsMock() to NewMetricsNoop() when Prometheus exporter is disabled. Created new no-op implementation with stub methods for all lifecycle and reporting operations.
Prometheus Metrics
pkg/metricsmanager/prometheus/prometheus.go
Added new node_agent_rule_prefiltered_total counter metric (labeled by rule_id) with cached counter support to track prefiltered rule evaluations. Implemented ReportRulePrefiltered() method.
Prefilter Implementation
pkg/rulemanager/prefilter/prefilter.go
New package implementing lightweight HTTP/SSH pre-filtering layer with Direction and MethodMask types for HTTP metadata, EventFields struct for pre-extracted event values, and Params struct for parsed filter criteria. Includes ParseWithDefaults() for merging rule state and binding parameters, and ShouldSkip() for hot-path filtering decision logic.
Prefilter Tests
pkg/rulemanager/prefilter/prefilter_test.go
Comprehensive test suite covering ParseWithDefaults() behavior (nil handling, prefix normalization, parameter merging) and ShouldSkip() decision logic (prefix matching, direction/method filtering, port eligibility). Includes benchmarks for filtering performance.
Rule Manager Integration
pkg/rulemanager/rule_manager.go, pkg/rulemanager/extract_event_fields_test.go
Added extractEventFields() helper to lazily extract prefilterable fields per event type. Integrated prefiltering into ReportEnrichedEvent() to skip CEL evaluation for rules matching prefilter criteria and report prefiltered counts. Added test for event field extraction across event types (Open, Execve, HTTP, Network, SSH).
Rule Types & Rule Binding
pkg/rulemanager/types/v1/types.go, pkg/rulebindingmanager/cache/cache.go, pkg/rulebindingmanager/cache/cache_test.go
Added optional Prefilter field to Rule struct (excluded from serialization). Updated rule binding cache creation to populate prefilter from binding parameters via ParseWithDefaults(). Added test validating prefilter creation with proper prefix normalization and nil-parameter handling.

Sequence Diagram

sequenceDiagram
    participant Binding as Rule Binding Manager
    participant Cache as RuleCache
    participant Prefilter as Prefilter Package
    participant RuleManager as Rule Manager
    participant Metrics as Metrics Manager
    
    Binding->>Cache: createRule(bindingRule)
    Cache->>Prefilter: ParseWithDefaults(ruleState, parameters)
    Prefilter-->>Cache: Params{IgnorePrefixes, IncludePrefixes, AllowedPorts, Dir, MethodMask}
    Cache->>Cache: rule.Prefilter = Params
    Cache-->>Binding: Rule with Prefilter
    
    rect rgba(100, 150, 200, 0.5)
    note over RuleManager,Metrics: Event Processing Flow
    RuleManager->>RuleManager: extractEventFields(event)
    RuleManager->>RuleManager: for each enabled rule:
    alt rule.Prefilter != nil
        RuleManager->>Prefilter: ShouldSkip(extractedFields)
        Prefilter-->>RuleManager: bool (skip decision)
        RuleManager->>Metrics: ReportRulePrefiltered(ruleName)
    else rule.Prefilter == nil
        RuleManager->>RuleManager: Evaluate CEL rule
    end
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • remove toMap function  #693: Modifies pkg/rulemanager/rule_manager.go and event preparation logic, affecting similar event evaluation pathways.

Suggested reviewers

  • matthyx

Poem

🐰 Pre-filters hop through events with care,
Skipping those that don't belong there!
HTTP methods and paths take flight,
Events prefiltered before rule's sight.
Metrics track what passed the gate—
Efficient rules now seal their fate! 🌟

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 10.53% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title 'Feature/cel prefilter' is vague and uses non-descriptive branching convention naming rather than conveying meaningful information about the actual changes in the changeset. Revise the title to clearly describe the main feature being added, such as 'Add CEL rule pre-filtering for event optimization' or similar descriptive phrasing that explains what was implemented.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/cel-prefilter

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@YakirOren YakirOren force-pushed the feature/cel-prefilter branch from fde54c5 to 9c33ec6 Compare March 29, 2026 08:21
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (4)
pkg/metricsmanager/metrics_manager_mock.go (1)

63-63: Track prefiltered calls in the mock for testability.

ReportRulePrefiltered is currently a no-op, so tests can’t assert this new behavior path.

♻️ Proposed mock enhancement
 type MetricsMock struct {
 	FailedEventCounter   atomic.Int32
 	RuleProcessedCounter maps.SafeMap[string, int]
+	RulePrefilteredCounter maps.SafeMap[string, int]
 	RuleAlertCounter     maps.SafeMap[string, int]
 	EventCounter         maps.SafeMap[utils.EventType, int]
 	RuleEvaluationTime   maps.SafeMap[string, time.Duration] // key: "ruleID:eventType"
 }

-func (m *MetricsMock) ReportRulePrefiltered(ruleName string) {}
+func (m *MetricsMock) ReportRulePrefiltered(ruleName string) {
+	m.RulePrefilteredCounter.Set(ruleName, m.RulePrefilteredCounter.Get(ruleName)+1)
+}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/metricsmanager/metrics_manager_mock.go` at line 63, The
MetricsMock.ReportRulePrefiltered method is a no-op so tests cannot observe
prefiltered events; update the MetricsMock struct to track calls (e.g., add an
exported field like PrefilteredCalls map[string]int or Prefiltered []string plus
a sync.Mutex for concurrency safety) and implement ReportRulePrefiltered to
record/increment the entry for the provided ruleName; use the new exported field
in tests to assert the mock received the prefiltered report.
pkg/rulebindingmanager/cache/cache_test.go (1)

1045-1086: Expand this test to cover all changed createRule branches.

Current cases validate only the RuleID path. Please add cases for RuleName and RuleTags so all new hydration branches are covered.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/rulebindingmanager/cache/cache_test.go` around lines 1045 - 1086, Extend
TestCreateRulePrefilter to exercise the other createRule hydration branches: add
test cases where the binding uses RuleName (set
RuntimeAlertRuleBindingRule.RuleName to a string) and where it uses RuleTags
(set RuntimeAlertRuleBindingRule.RuleTags to a slice of strings) in addition to
the existing RuleID case; for each new case call
NewCacheMock("").createRule(binding), assert you get one rule and then validate
its Prefilter fields (IgnorePrefixes/IncludePrefixes) behave the same as the
RuleID case when Parameters are present and that Prefilter is nil when
Parameters are absent—this will cover the RuleName and RuleTags code paths in
createRule.
pkg/rulemanager/prefilter/prefilter.go (1)

243-249: ensureTrailingSlash mutates input slice in place.

This function modifies the input slice directly. While toStringSlice creates a copy before calling this function, if ensureTrailingSlash is ever called directly on a caller-owned slice, it would cause unexpected mutation. Consider returning a new slice for safety.

♻️ Safer implementation that creates a copy
 func ensureTrailingSlash(prefixes []string) []string {
+	result := make([]string, len(prefixes))
 	for i, p := range prefixes {
 		if !strings.HasSuffix(p, "/") {
-			prefixes[i] = p + "/"
+			result[i] = p + "/"
+		} else {
+			result[i] = p
 		}
 	}
-	return prefixes
+	return result
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/rulemanager/prefilter/prefilter.go` around lines 243 - 249,
ensureTrailingSlash currently mutates its input slice in place; change it to
allocate and return a new slice instead to avoid unexpected mutations (e.g.,
when callers pass a caller-owned slice). In the implementation of
ensureTrailingSlash, create a new slice of the same length, copy or transform
each element from the input into the new slice adding a trailing "/" when
needed, and return the new slice; update callers (e.g., where toStringSlice
currently copies) to use the returned slice but no caller-side mutation should
be required.
pkg/rulemanager/prefilter/prefilter_test.go (1)

66-80: Consider using assert.Equal directly on struct for cleaner assertions.

The field-by-field comparison works, but comparing the full struct would be more concise and catch any future fields that get added.

♻️ Suggested simplification
 		t.Run(tt.name, func(t *testing.T) {
 			got := ParseWithDefaults(tt.ruleState, tt.bindingParams)
 			if tt.expect == nil {
 				assert.Nil(t, got)
 			} else {
 				require.NotNil(t, got)
-				assert.Equal(t, tt.expect.IgnorePrefixes, got.IgnorePrefixes)
-				assert.Equal(t, tt.expect.IncludePrefixes, got.IncludePrefixes)
-				assert.Equal(t, tt.expect.AllowedPorts, got.AllowedPorts)
-				assert.Equal(t, tt.expect.Dir, got.Dir)
-				assert.Equal(t, tt.expect.MethodMask, got.MethodMask)
+				assert.Equal(t, tt.expect, got)
 			}
 		})
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/rulemanager/prefilter/prefilter_test.go` around lines 66 - 80, The test
currently compares fields of the ParseWithDefaults result one-by-one; replace
that with a single struct equality assertion to be more concise and
future-proof: inside the loop in prefiler_test.go, keep the nil-check branch (if
tt.expect == nil { assert.Nil(t, got) }) but in the non-nil branch remove
require.NotNil and the per-field asserts and call assert.Equal(t, tt.expect,
got) to compare the whole struct returned by ParseWithDefaults (variables: got
and tt.expect).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/rulemanager/prefilter/prefilter.go`:
- Around line 185-187: The check using p.AllowedPorts currently causes events to
be skipped when the destination port is present (slices.Contains(p.AllowedPorts,
e.DstPort)), which conflicts with the "Allowed" name; either rename AllowedPorts
to SkipPorts/IgnorePorts (and update usages and comments around the prefilter
logic where e.PortEligible and e.DstPort are evaluated) or invert the condition
so the prefilter only skips when the port is NOT in the allow-list (i.e., change
the logical test around p.AllowedPorts in the block referencing e.PortEligible,
p.AllowedPorts and e.DstPort to match the chosen semantics and update the
variable name/comment accordingly).

In `@pkg/rulemanager/rule_manager.go`:
- Around line 224-236: The prefilter is being skipped for non-K8s rules and also
runs before event-type filtering, causing overcounting; update the logic in the
rule evaluation loop so you first call rm.getRuleExpressions(rule, eventType)
and skip rules with zero expressions, then perform prefiltering only if a
prefilter exists or, if nil, attempt to hydrate the rule from the RBCache via
the same path used by CreateAllRules (e.g., call the rule-hydration method used
by RBCache.createRule) to populate rule.Prefilter before calling
rule.Prefilter.ShouldSkip(eventFields); keep using
extractEventFields(enrichedEvent.Event) when eventFields.Extracted is false and
continue to call rm.metrics.ReportRulePrefiltered(rule.Name) only when a
prefilter actually caused the skip.

---

Nitpick comments:
In `@pkg/metricsmanager/metrics_manager_mock.go`:
- Line 63: The MetricsMock.ReportRulePrefiltered method is a no-op so tests
cannot observe prefiltered events; update the MetricsMock struct to track calls
(e.g., add an exported field like PrefilteredCalls map[string]int or Prefiltered
[]string plus a sync.Mutex for concurrency safety) and implement
ReportRulePrefiltered to record/increment the entry for the provided ruleName;
use the new exported field in tests to assert the mock received the prefiltered
report.

In `@pkg/rulebindingmanager/cache/cache_test.go`:
- Around line 1045-1086: Extend TestCreateRulePrefilter to exercise the other
createRule hydration branches: add test cases where the binding uses RuleName
(set RuntimeAlertRuleBindingRule.RuleName to a string) and where it uses
RuleTags (set RuntimeAlertRuleBindingRule.RuleTags to a slice of strings) in
addition to the existing RuleID case; for each new case call
NewCacheMock("").createRule(binding), assert you get one rule and then validate
its Prefilter fields (IgnorePrefixes/IncludePrefixes) behave the same as the
RuleID case when Parameters are present and that Prefilter is nil when
Parameters are absent—this will cover the RuleName and RuleTags code paths in
createRule.

In `@pkg/rulemanager/prefilter/prefilter_test.go`:
- Around line 66-80: The test currently compares fields of the ParseWithDefaults
result one-by-one; replace that with a single struct equality assertion to be
more concise and future-proof: inside the loop in prefiler_test.go, keep the
nil-check branch (if tt.expect == nil { assert.Nil(t, got) }) but in the non-nil
branch remove require.NotNil and the per-field asserts and call assert.Equal(t,
tt.expect, got) to compare the whole struct returned by ParseWithDefaults
(variables: got and tt.expect).

In `@pkg/rulemanager/prefilter/prefilter.go`:
- Around line 243-249: ensureTrailingSlash currently mutates its input slice in
place; change it to allocate and return a new slice instead to avoid unexpected
mutations (e.g., when callers pass a caller-owned slice). In the implementation
of ensureTrailingSlash, create a new slice of the same length, copy or transform
each element from the input into the new slice adding a trailing "/" when
needed, and return the new slice; update callers (e.g., where toStringSlice
currently copies) to use the returned slice but no caller-side mutation should
be required.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f148052b-06e5-4673-8799-d2877da86e4c

📥 Commits

Reviewing files that changed from the base of the PR and between 2b05e16 and 9c33ec6.

📒 Files selected for processing (12)
  • cmd/main.go
  • pkg/metricsmanager/metrics_manager_interface.go
  • pkg/metricsmanager/metrics_manager_mock.go
  • pkg/metricsmanager/metrics_manager_noop.go
  • pkg/metricsmanager/prometheus/prometheus.go
  • pkg/rulebindingmanager/cache/cache.go
  • pkg/rulebindingmanager/cache/cache_test.go
  • pkg/rulemanager/extract_event_fields_test.go
  • pkg/rulemanager/prefilter/prefilter.go
  • pkg/rulemanager/prefilter/prefilter_test.go
  • pkg/rulemanager/rule_manager.go
  • pkg/rulemanager/types/v1/types.go

@YakirOren YakirOren force-pushed the feature/cel-prefilter branch from bed06bb to 9451661 Compare March 29, 2026 08:58
Signed-off-by: Yakir Oren <yakiroren@gmail.com>
Signed-off-by: Yakir Oren <yakiroren@gmail.com>
Signed-off-by: Yakir Oren <yakiroren@gmail.com>
…s first

Signed-off-by: Yakir Oren <yakiroren@gmail.com>
Signed-off-by: Yakir Oren <yakiroren@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: WIP

Development

Successfully merging this pull request may close these issues.

2 participants