feat(ae): reattribute DNS via kubescape process tree (fixes #80)#81
Open
entlein wants to merge 3 commits into
Open
feat(ae): reattribute DNS via kubescape process tree (fixes #80)#81entlein wants to merge 3 commits into
entlein wants to merge 3 commits into
Conversation
added 2 commits
July 2, 2026 09:41
…iew) Restacks the dx graph rename onto the k8sstormcenter#73 /query-runner tip. Renames across the AE forensic path: - table forensic_db.dx_attack_graph → dx_evidence_graph - view dx_attack_graph_malicious → dx_evidence_graph_malignant (medical terminology: malignant, not malicious) - endpoint /dx/attack_graph → /dx/evidence_graph - Go WriteAttackGraph/handleDXAttackGraph → *EvidenceGraph Done mechanically on k8sstormcenter#73 so it also covers order_query_test.go (absent from the original rename branch's lineage). Also fixes two pre-existing k8sstormcenter#73 lint nits surfaced by touching the package: gofumpt blank lines in order_query_test.go and the missing order_query_test.go entry in controller/BUILD.bazel's controller_test srcs (gazelle).
…manifest
Closes the AE-side wire gap for dx's §9 completeness contract (stacked on
the dx_evidence_graph rename). dx already POSTs manifests via
aeclient.WriteEvidenceManifest → /dx/evidence_manifest; aeprod26 404s that
path, so manifest_rows_exported=0 / graph_write_failures{kind=manifest}>0.
Adds, mirroring the evidence_graph path:
- control: manifestWriter iface + SetManifestWriter + POST /dx/evidence_manifest
handler. Accepts one manifest.Manifest per verdict; scalars map to typed
columns, nested collections (case_window/findings/orders/seeds/chain) are
rendered as JSON text so the JSONEachRow insert is CH-version independent.
- clickhouse: forensic_db.dx_evidence_manifest table (columns = manifest.Manifest
JSON tags; event_time nanos + hostname read-path like dx_evidence_graph),
WriteEvidenceManifest sink, KnownTables + OperatorOwnedTables (created on boot).
- main: wire SetManifestWriter(applier).
- tests: endpoint 501/202/502 + nested-as-text flattening; table-set guard.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
entlein
force-pushed
the
feat/ae-dns-ppid-attribution
branch
from
a1905f9 to
fcf76f5
Compare
Pixie drops pod attribution for short-lived processes — UDP DNS resolvers fork, do one lookup, and exit, so upid_to_pod_name yields "" at query time and dns_events land unattributed and get filtered out of the steered set. Kubescape captured those pids (and parents) at exec time with the owning pod; join on it. - kubescape.PIDIndex: pid -> "<ns>/<pod>" from RuntimeProcessDetails trees (top-level + children + ppid), built from the rows the trigger already reads. - pxl.QueryFor: expose df.pid; for ProcessTreeAttributed tables (dns_events) skip the in-pixie pod filter so empty-pod rows survive to the operator. - controller.enrichProcessTree: reattribute empty-pod rows via the index, then target-filter — with no index the unattributed rows are dropped, so a pulled-unfiltered DNS window never floods forensic_db. Wired in both the OrderQuery (/query) and pushPixieRows paths. - trigger ProcessTreeIndex(): bounded kubescape_logs read -> BuildPIDIndex; wired into controller.Config in main.go. Validated live on a SOC rig: pixie dns_events pid 882811 (pod="") resolves to observer via kubescape's tree. Unit + e2e tests cover index, enrichment, query shape, and no-flood. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
entlein
force-pushed
the
feat/ae-dns-ppid-attribution
branch
from
fcf76f5 to
e77894d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the recurring DNS attribution gap (#80): ~73% of
dns_eventsland with an empty pod and get dropped from the steered set.Root cause (validated on a live SOC rig)
UDP DNS is connectionless → no conn_open → pixie never sets
local_addr; the resolver process is short-lived, soupid_to_pod_nameyields""at query time. Pixie captures no ppid (grep -rniE '\bppid\b' src= 0;proc_parser.ccreads/proc/pid/statfields 20+22 but skips field 4). But kubescape already captured the pid — and its parent chain — at exec time, tagged with the owning pod. Join on it, no pixie/eBPF change.Live proof:
pixie dns_events pid 882811 pod=""→ kubescape treepid 882811 → observer→ attributed.Change
kubescape.PIDIndex—pid → "<ns>/<pod>"fromRuntimeProcessDetailstrees (top-level + children + ppid).pxl.QueryFor— exposedf.pid; forProcessTreeAttributedtables (dns_events) skip the in-pixie pod filter so empty-pod rows reach the operator.controller.enrichProcessTree— reattribute empty-pod rows via the index, then target-filter. No-flood guarantee: with no index the still-unattributed rows are dropped, so a pulled-unfiltered DNS window can never bloat forensic_db. Wired into bothOrderQueryandpushPixieRows.trigger.ProcessTreeIndex()— boundedkubescape_logsread →BuildPIDIndex; wired intocontroller.Configinmain.go.Tests
All 14 AE packages green. New unit tests:
pidindex(child+parent+host-pid),enrich(reattribution, numeric-shape coercion, no-flood), plus updatedqueryforline-count guards and an aligned e2e loadtest fixture.Stacked on
feat/ae-query-runner(#73). The pixie-native alternative (add ppid toproc_parser.ccfield 4 → walkpid→ppid) remains the 100%-coverage path noted on #80; this join needs no rebuild of pixie itself.🤖 Generated with Claude Code