Skill Being Reviewed
Skill name: detection-engineering
Skill path: skills/secops/detection-engineering/SKILL.md
False Positive Analysis
Benign code/configuration that can be over-flagged:
rule: impossible_travel
data_source: okta_signin
health: last_event_age_minutes < 10, parser_version pinned, sample count monitored
Why this is a false positive:
rule: impossible_travel can be safe when the missing compensating evidence is present, but the current skill does not ask reviewers to distinguish that evidence from the risky pattern. The review should require proof of the guardrail before escalating severity.
Coverage Gaps
Missed variant 1:
rule enabled: true
source: endpoint_process_events
# EDR connector last successful ingestion was 9 days ago
Why it should be caught:
This variant leaves a realistic attack path open while still satisfying the current high-level checklist language.
Missed variant 2:
sigma_rule maps field CommandLine
SIEM parser renamed field to process.command_line, rule silently matches zero events
Why it should be caught:
This is a common production edge case where policy exists on paper but does not bind the runtime behavior or evidence trail.
Edge Cases
A detection rule can be syntactically correct and mapped to ATT&CK while producing no alerts because telemetry is stale, parser mappings drifted, or expected event volume collapsed.
Remediation Quality
Comparison to Other Tools
| Tool |
Catches this? |
Notes |
| Sigma tooling |
Partial |
Validates rule syntax, not source health. |
| SIEM content packs |
Partial |
Assume mappings are current. |
| Detection-as-code tests |
Partial |
Good when they include replay/field mapping fixtures. |
Overall Assessment
Strengths:
The skill is strong on ATT&CK mapping and rule design.
Needs improvement:
It should require telemetry health evidence before rating a detection as effective.
Priority recommendations:
- Add a data-source health precondition to every detection review.
- Require replay tests or recent match evidence for critical detections.
- Flag schema/parser drift and zero-match periods as coverage failures, not just operational notes.
Bounty Info
Skill Being Reviewed
Skill name: detection-engineering
Skill path:
skills/secops/detection-engineering/SKILL.mdFalse Positive Analysis
Benign code/configuration that can be over-flagged:
Why this is a false positive:
rule: impossible_travel can be safe when the missing compensating evidence is present, but the current skill does not ask reviewers to distinguish that evidence from the risky pattern. The review should require proof of the guardrail before escalating severity.
Coverage Gaps
Missed variant 1:
Why it should be caught:
This variant leaves a realistic attack path open while still satisfying the current high-level checklist language.
Missed variant 2:
Why it should be caught:
This is a common production edge case where policy exists on paper but does not bind the runtime behavior or evidence trail.
Edge Cases
A detection rule can be syntactically correct and mapped to ATT&CK while producing no alerts because telemetry is stale, parser mappings drifted, or expected event volume collapsed.
Remediation Quality
Comparison to Other Tools
Overall Assessment
Strengths:
The skill is strong on ATT&CK mapping and rule design.
Needs improvement:
It should require telemetry health evidence before rating a detection as effective.
Priority recommendations:
Bounty Info