-
Notifications
You must be signed in to change notification settings - Fork 11
Add minimum required permissions documentation #18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
ad08aa3
Add minimum required permissions documentation
maxcold 17c393f
Add test:permissions tooling and fix index monitor privilege gap
maxcold a2601d3
Fix acknowledgeAlert wildcard bug and document backing-index privileges
maxcold 5c9c7e9
Add Quickstart asserted suite to test:permissions runner
maxcold 0b44d9b
Lead permissions doc with built-in role Quickstart
maxcold 5fd0cf7
Add acknowledgeDiscoveries privilege check (H1)
maxcold 1bbc70b
Add assessConfidence + getDiscoveryDetail privilege checks (M6)
maxcold 398653e
Add cleanupSampleData privilege checks for logs + alerts (H2)
maxcold 2662700
Fix README env vars to match runner.ts
maxcold c8ec693
Fix appendix: Generate sample data writes alerts too
maxcold d60b7df
Use admin grant API to mint user API keys
maxcold 6d9c241
Add getAlertContext privilege check (H3)
maxcold c87e7a2
Add getMapping privilege check (M3)
maxcold 872c1b4
Recognize embedded "status_code":403 in bulk-endpoint errors
maxcold 46eb5c8
Add bulkAction privilege check (H4)
maxcold 148b345
Add listExceptions + addException privilege checks (H5)
maxcold 6f76925
Doc updates from PR review feedback
maxcold b245303
Merge remote-tracking branch 'origin/main' into docs/minimum-required…
maxcold 9abbf7d
docs: add Dev Tools TL;DR section to permissions guide
maxcold 6b456cb
Merge remote-tracking branch 'origin/main' into docs/minimum-required…
maxcold af3e1cf
docs: align permissions.md with CLUSTERS_JSON config
maxcold 8bcfec4
scripts: port test-permissions to new service/client architecture
maxcold File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| # Permissions Test Tooling | ||
|
|
||
| Verifies that the role definitions documented in [`docs/permissions.md`](../../docs/permissions.md) actually work end-to-end against a real Elasticsearch + Kibana cluster. Provisions both documented roles, creates scoped API keys, and exercises every documented operation through the existing `src/elastic/*` business-logic modules. | ||
|
|
||
| ## Quick Start | ||
|
|
||
| ```bash | ||
| # Make sure .env has ELASTICSEARCH_URL, KIBANA_URL, ELASTIC_PASSWORD | ||
| # (and ELASTIC_USERNAME if not the default "elastic"). | ||
| # The runner authenticates as that user via Basic auth to bootstrap an | ||
| # admin API key, so the user must have manage_security plus enough | ||
| # privileges to seed sample data — `superuser` works for local dev. | ||
|
|
||
| npm run test:permissions | ||
| ``` | ||
|
|
||
| Exit code is `0` if every check passes (or is skipped); `1` otherwise. | ||
|
|
||
| ## Flags | ||
|
|
||
| | Flag | Description | | ||
| |---|---| | ||
| | `--role full\|readonly\|both` | Which role(s) to test (default: `both`). | | ||
| | `--cleanup-stale` | Delete leftover `mcp-app-test-*` roles and API keys before running. Useful after a crashed run. | | ||
| | `--no-cleanup` | Skip cleanup at the end and print the provisioned API keys so you can re-use them for manual debugging. | | ||
| | `--verbose`, `-v` | Print fixtures, stale-cleanup actions, and other debug info. | | ||
| | `-h`, `--help` | Show help. | | ||
|
|
||
| Pass flags via `--`, e.g. `npm run test:permissions -- --role readonly --verbose`. | ||
|
|
||
| ## What it does | ||
|
|
||
| 1. **Pre-flight.** Loads admin credentials from `.env`. Calls `checkExistingData()`; if the cluster has zero security alerts, calls `generateSampleData({ count: 50 })` to seed. | ||
| 2. **Capture fixtures.** Picks one alert, one case (creates one if none exist), and the first detection rule. | ||
| 3. **For each role (`full`, then `readonly`):** | ||
| - Creates a role `mcp-app-test-<role>-<suffix>` and an API key scoped to it. | ||
| - **Layer A** — calls `POST /_security/user/_has_privileges` as the scoped key, asserting that every privilege listed in the role descriptor is granted. | ||
| - **Layer B** — exercises one or more operations per group (alerts, cases, rules, attack-discovery, threat-hunt, sample-data) using the actual `src/elastic/*` functions. Reads must succeed for both roles. Writes must succeed for `full` and 403 for `readonly`. | ||
| 4. **Report** — prints per-role results grouped by operation group. Symbols: `✓` pass, `✗` fail, `→` skipped. | ||
| 5. **Cleanup** — always (in a `finally` block, and on `SIGINT`): invalidates API keys and deletes roles created by the run. | ||
|
|
||
| Created cases / rules / sample documents are tagged `mcp-app-test` and are **not** automatically deleted. If they accumulate, clean them up via Kibana or with a tag-scoped delete. | ||
|
|
||
| ## Output Interpretation | ||
|
|
||
| Each line in the report looks like: | ||
|
|
||
| ``` | ||
| ✓ acknowledgeAlert (expect 403) — denied (403/401) | ||
| ✗ createCase (expect 403) — expected 403 but call succeeded | ||
| → patchRule (expect 403) — no fixture available for this check | ||
| ``` | ||
|
|
||
| - `pass` (`✓`) — outcome matched the expectation (call succeeded when expected `ok`, or call returned 403/401 when expected `403`). | ||
| - `fail` (`✗`) — outcome diverged from the expectation. The detail explains what happened. | ||
| - `skipped` (`→`) — the cluster did not have a fixture for this op (e.g. no rule to patch). Skipped checks do **not** affect the exit code. | ||
|
|
||
| The summary line lists totals: `Summary: N passed, M failed, K skipped`. | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| **Read-only writes return `ok` instead of `403` (e.g. `expected 403 but call succeeded`).** | ||
| The read-only role is over-privileged. Compare `roles.ts` to the read-only role in [`docs/permissions.md`](../../docs/permissions.md) — usually a stale `"write"` snuck into `indices.privileges`, or an `*.all` Kibana feature privilege replaced an `*.read` one. | ||
|
|
||
| **Full-featured writes return `403`.** | ||
| A privilege documented in the full role is missing from `roles.ts`. Diff against the spec. | ||
|
|
||
| **Layer A reports `missing privileges: ...`.** | ||
| The role descriptor sent in the `PUT /_security/role` body doesn't include the listed privileges, or Elasticsearch rejected one of them (typo / removed feature). Check that the Kibana feature names match your stack version. The defaults target 9.4+ — see the version-specific tables in `docs/permissions.md`. | ||
|
|
||
| **`Fatal error: ELASTICSEARCH_URL, KIBANA_URL, and ELASTIC_PASSWORD must be set...`** | ||
| `.env` isn't loading or is missing one of `ELASTICSEARCH_URL`, `KIBANA_URL`, `ELASTIC_PASSWORD`. `ELASTIC_USERNAME` is optional (defaults to `elastic`). The script reads them via `dotenv/config`. | ||
|
|
||
| **`Seeding completed but no security alerts were created.`** | ||
| `generateSampleData` ran but didn't end up writing alerts. Usually means the admin key lacks `write` on `.alerts-security.alerts-default`. Use a key with at least the privileges in the full role. | ||
|
|
||
| **Leftover `mcp-app-test-*` roles or API keys.** | ||
| A previous run was killed before cleanup. Re-run with `--cleanup-stale` to wipe them, or delete manually via Kibana > Stack Management. | ||
|
|
||
| ## Architecture | ||
|
|
||
| ``` | ||
| scripts/test-permissions/ | ||
| ├── roles.ts # Role descriptors + operation check matrix | ||
| ├── elastic-admin.ts # PUT/DELETE role, POST/DELETE api_key, _has_privileges probe | ||
| ├── runner.ts # Orchestrator: provision → seed → test → report → cleanup | ||
| └── README.md # This file | ||
| ``` | ||
|
|
||
| The runner reuses business logic from `src/elastic/*` and swaps the API key on the singleton config (`setConfig`) between admin and each scoped role. This forces sequential per-role execution; that's fine at this scale. | ||
|
|
||
| ## Updating the Test Matrix | ||
|
|
||
| When you add or remove an operation from `src/elastic/*`, also update `operationChecks` in [`roles.ts`](./roles.ts): | ||
|
|
||
| - Add an entry naming the function and its group. | ||
| - Set `expect.full` to `"ok"` and `expect.readonly` to either `"ok"` (read) or `"403"` (write). | ||
| - If the call needs a fixture that may not exist, add `skipUnless` and gate it on the fixture. | ||
| - If the function silences errors internally (e.g. wraps each ES call in `try { ... } catch {}`), it's not testable here — either refactor the function to surface 403s or pick a different operation that hits the same privilege. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The MCP app uses an API key for auth - could we keep it consistent and use the same env variables? Then users dont need to maintain 2 sets of credentials
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The script is used for development and needs admin-level credentials that are distinct from the runtime MCP key:
PUT /_security/role/...(needsmanage_security),POST /_security/api_key/grantwithgrant_type: password(acts on behalf of a user), and seeds sample data intologs-*and.alerts-security.alerts-*.MCP ELASTICSEARCH_API_KEYis by design least-privilege over those alert/case indices and does not carrymanage_security. So even if we accepted the same env var name, users would have to maintain a second, a more privileged key under it to run the test runner.