diff --git a/docs/6.x/vulnerability-detection/schema-based-testing/docker-reference.md b/docs/6.x/vulnerability-detection/schema-based-testing/docker-reference.md new file mode 100644 index 0000000000..50404fa0fb --- /dev/null +++ b/docs/6.x/vulnerability-detection/schema-based-testing/docker-reference.md @@ -0,0 +1 @@ +--8<-- "latest/vulnerability-detection/schema-based-testing/docker-reference.md" diff --git a/docs/6.x/vulnerability-detection/schema-based-testing/strategies.md b/docs/6.x/vulnerability-detection/schema-based-testing/strategies.md new file mode 100644 index 0000000000..d28e1e17fe --- /dev/null +++ b/docs/6.x/vulnerability-detection/schema-based-testing/strategies.md @@ -0,0 +1 @@ +--8<-- "latest/vulnerability-detection/schema-based-testing/strategies.md" diff --git a/docs/7.x/vulnerability-detection/schema-based-testing/docker-reference.md b/docs/7.x/vulnerability-detection/schema-based-testing/docker-reference.md new file mode 100644 index 0000000000..50404fa0fb --- /dev/null +++ b/docs/7.x/vulnerability-detection/schema-based-testing/docker-reference.md @@ -0,0 +1 @@ +--8<-- "latest/vulnerability-detection/schema-based-testing/docker-reference.md" diff --git a/docs/7.x/vulnerability-detection/schema-based-testing/strategies.md b/docs/7.x/vulnerability-detection/schema-based-testing/strategies.md new file mode 100644 index 0000000000..d28e1e17fe --- /dev/null +++ b/docs/7.x/vulnerability-detection/schema-based-testing/strategies.md @@ -0,0 +1 @@ +--8<-- "latest/vulnerability-detection/schema-based-testing/strategies.md" diff --git a/docs/latest/about-wallarm/detecting-vulnerabilities.md b/docs/latest/about-wallarm/detecting-vulnerabilities.md index d50a524edb..63ebe8de2d 100644 --- a/docs/latest/about-wallarm/detecting-vulnerabilities.md +++ b/docs/latest/about-wallarm/detecting-vulnerabilities.md @@ -28,7 +28,7 @@ To detect vulnerabilities in the applications and APIs, Wallarm uses the followi !!! tip "" Available in the **Security Testing** [subscription](../about-wallarm/subscription-plans.md). Requires installed [**Wallarm node**](../about-wallarm/api-security-overview.md#how-wallarm-api-security-works). -* [**Schema-Based Testing (SBT)**](#schema-based-testing-sbt), part of [Wallarm's Security Testing suite](../vulnerability-detection/security-testing-overview.md): dynamic application security testing (DAST) solution that enables "shift-left" security - proactively identifies a wide range of vulnerabilities early in the development process; uses an API's schema (such as an OpenAPI specification or a Postman collection) as a blueprint to automatically generate and execute targeted security tests. +* [**Schema-Based Testing (SBT)**](#schema-based-testing-sbt), part of [Wallarm's Security Testing suite](../vulnerability-detection/security-testing-overview.md): dynamic application security testing (DAST) solution that enables "shift-left" security. SBT takes a Postman collection as its test basis and uses an AI-driven engine to generate and validate targeted security tests, identifying a wide range of vulnerabilities early in the development process. !!! tip "" Available in the **Security Testing** [subscription](../about-wallarm/subscription-plans.md). **Does not require** installed [Wallarm node](../about-wallarm/api-security-overview.md#how-wallarm-api-security-works). @@ -66,14 +66,14 @@ The Threat Replay Testing capabilities: ### Schema-Based Testing (SBT) -Wallarm's [Schema-Based Testing](../vulnerability-detection/schema-based-testing/overview.md) is a dynamic application security testing (DAST) solution that enables "shift-left" security. It uses an API's schema (such as an OpenAPI specification or a Postman collection) as a blueprint to automatically generate and execute targeted security tests. By integrating into CI/CD pipelines, Schema-Based Testing allows development teams to proactively identify a wide range of vulnerabilities—including OWASP API Top 10 risks, business logic flaws, and input validation issues—early in the development process, making them easier and cheaper to fix. +Wallarm's [Schema-Based Testing](../vulnerability-detection/schema-based-testing/overview.md) is a dynamic application security testing (DAST) solution that enables "shift-left" security. It takes a Postman collection — your existing functional tests — as a blueprint and runs an AI-driven scanner that analyzes the application context, generates vulnerability hypotheses, and validates each finding with an executable proof-of-exploit test. By integrating into CI/CD pipelines, Schema-Based Testing helps development teams proactively identify a wide range of vulnerabilities — including OWASP API Top 10 risks, business logic flaws, and input validation issues — early in the development process, making them easier and cheaper to fix. Schema-Based Testing capabilities: -* Tests based on provided application's OpenAPI specification or Postman collection. -* Deep, dynamic analysis of API endpoints. -* Detection of vulnerabilities in the application or API itself, as well as security misconfigurations in the underlying infrastructure or environment. -* Lightweight execution via Docker container. +* Tests built from your Postman collection: functional tests drive realistic, multi-request attack scenarios. +* Deep, dynamic analysis of API endpoints with AI-generated hypotheses validated by proof-of-exploit tests. +* Detection of OWASP API Top 10 risks, business logic and access control vulnerabilities (BOLA, BFLA, mass assignment, etc.), input validation issues, and traffic-observable misconfigurations. +* Lightweight execution via Docker container, with results streamed back to Wallarm Cloud. ### API Security Testing via Postman diff --git a/docs/latest/api-discovery/exploring.md b/docs/latest/api-discovery/exploring.md index c5dd2782e1..3b519ad8b2 100644 --- a/docs/latest/api-discovery/exploring.md +++ b/docs/latest/api-discovery/exploring.md @@ -273,7 +273,7 @@ Exporting to OAS lets you use the discovered API schema for protection, analysis * **Upload to API specifications** in Wallarm to [enforce requests](../api-specification-enforcement/overview.md) or enable rogue API detection (when available for your API Discovery version). * **Open in [Swagger Editor](https://editor.swagger.io/)** to inspect and edit the inventory in OpenAPI format. -* **Use in Wallarm's [Schema-Based Testing](../vulnerability-detection/schema-based-testing/overview.md)** to run automated API security tests, or **export to third-party platforms** (e.g. [Postman](https://www.postman.com/)) for documentation, testing, or further analysis. The specification helps with vulnerability testing and reviewing endpoints for sensitive data and undocumented parameters. +* **Export to third-party platforms** (for example, [Postman](https://www.postman.com/)) for documentation, testing, or further analysis. The specification helps with vulnerability testing and reviewing endpoints for sensitive data and undocumented parameters. After importing the OAS into Postman and adding functional tests, you can drive [Schema-Based Testing](../vulnerability-detection/schema-based-testing/overview.md) with the resulting collection. To download the OAS file: diff --git a/docs/latest/attacks-vulns-list.md b/docs/latest/attacks-vulns-list.md index 2f4885bfec..c8aa11a652 100644 --- a/docs/latest/attacks-vulns-list.md +++ b/docs/latest/attacks-vulns-list.md @@ -1133,7 +1133,7 @@ Complex business logic and access control vulnerabilities are defined by the str **Required configuration:** -Wallarm detects the complex business logic and access control vulnerabilities only with enabled [Schema-Based Testing (SBT)](vulnerability-detection/schema-based-testing/overview.md) where the [Postman-based testing](vulnerability-detection/schema-based-testing/setup.md#postman-collection-based-test-policies) is configured. +Wallarm detects the complex business logic and access control vulnerabilities with [Schema-Based Testing (SBT)](vulnerability-detection/schema-based-testing/overview.md), where the Active Scan mode runs the relevant [strategies](vulnerability-detection/schema-based-testing/strategies.md) against your Postman collection. Note that in Wallarm, you can also configure **AI Business logic abuse detection** mitigation control to detect [business logic abuse](#business-logic-abuse) at already running applications. diff --git a/docs/latest/user-guides/settings/api-tokens.md b/docs/latest/user-guides/settings/api-tokens.md index 062c8f8f9a..c95e687d83 100644 --- a/docs/latest/user-guides/settings/api-tokens.md +++ b/docs/latest/user-guides/settings/api-tokens.md @@ -28,7 +28,7 @@ The selected usage scope restricts how and where the token can be used: Appropriate for self-hosted Nodes only. * Wallarm API - select this option to use the token for making authenticated requests directly to the Wallarm API. -* Schema-Based Testing agent - [required](../../vulnerability-detection/schema-based-testing/) for work of [Schema-Based Testing](../../vulnerability-detection/schema-based-testing/setup.md#token). +* Schema-Based Testing agent - required for [Schema-Based Testing](../../vulnerability-detection/schema-based-testing/overview.md) to authenticate the Docker container against Wallarm Cloud. See [token prerequisites](../../vulnerability-detection/schema-based-testing/setup.md#prerequisites). ## Token expiration diff --git a/docs/latest/vulnerability-detection/api-security-testing-via-postman/overview.md b/docs/latest/vulnerability-detection/api-security-testing-via-postman/overview.md index ecbaf0e52a..d0620f50a8 100644 --- a/docs/latest/vulnerability-detection/api-security-testing-via-postman/overview.md +++ b/docs/latest/vulnerability-detection/api-security-testing-via-postman/overview.md @@ -20,18 +20,18 @@ API Security Testing looks for issues such as: Findings are summarized with explanations and remediation guidance, designed for developers rather than security specialists. For the full list of issue types that can be detected (including those found by ASTP), see [Vulnerability types](../../attacks-vulns-list.md#vulnerability-types). -## API Security Testing via Postman vs Schema-Based Testing (Postman) +## API Security Testing via Postman vs Schema-Based Testing -Wallarm also offers [Schema-Based Testing](../schema-based-testing/overview.md), which can use your Postman collection to run dynamic security tests (typically via Docker in CI/CD). Both options can use your Postman collections; choose based on how you work and how deep you need to go: +Wallarm also offers [Schema-Based Testing](../schema-based-testing/overview.md), which runs dynamic security tests against your application as a Docker container, typically in CI/CD. Both products take a Postman collection as input; choose based on how you work and how deep you need to go: -| | **API Security Testing via Postman** | **Schema-Based Testing** (Postman collection) | +| | **API Security Testing via Postman** | **Schema-Based Testing** | |---|---|---| -| **Use when** | You want a quick, conversational check inside Postman—ask in natural language and get results in the Agent chat in a few minutes. | You want automated, comprehensive DAST in CI/CD; you already have functional tests in Postman and want them to drive security tests. | -| **How it runs** | Passive, design-level analysis; no attack payloads, no traffic replay. | Dynamic testing: sends real requests, uses your collection's functional tests as a blueprint to generate and run security tests. | -| **Depth** | Auth gaps, data leaks, over-permissive endpoints, schema issues, basic BOLA/BOPLA—summarized for developers. | OWASP API Top 10, business logic, access control, input validation (injections, RCE, etc.), environment misconfigurations. | -| **Where** | Inside Postman (Agent Mode); results in chat and in Wallarm Cloud. | Docker-based; runs in your pipeline or locally; results in Wallarm Console (Test runs, Security Issues). | +| **Use when** | You want a quick, conversational check inside Postman — ask in natural language and get results in the Agent chat in a few minutes. | You want automated, comprehensive DAST embedded in CI/CD; you already have functional tests in Postman and want them to drive security tests. | +| **How it runs** | Passive, design-level analysis; no attack payloads, no traffic replay. | Dynamic testing as a Docker container: replays the collection, then generates and validates targeted security tests against the application. | +| **Depth** | Auth gaps, data leaks, over-permissive endpoints, schema issues, basic BOLA/BOPLA — summarized for developers. | OWASP API Top 10, business logic, access control, input validation (injections, RCE, etc.), and traffic-observable misconfigurations. | +| **Where** | Inside Postman (Agent Mode); results in chat and in Wallarm Cloud. | Docker container runs in your pipeline or locally; results in Wallarm Console (Test runs, Security Issues). | -In short: use **API Security Testing via Postman** for fast, in-Postman checks with minimal setup; use Schema-Based Testing with a Postman collection when you need full DAST and pipeline integration. +In short: use **API Security Testing via Postman** for fast, in-Postman checks with minimal setup; use **Schema-Based Testing** when you need full DAST and pipeline integration. ## Access via Postman diff --git a/docs/latest/vulnerability-detection/schema-based-testing/docker-reference.md b/docs/latest/vulnerability-detection/schema-based-testing/docker-reference.md new file mode 100644 index 0000000000..907c5fc391 --- /dev/null +++ b/docs/latest/vulnerability-detection/schema-based-testing/docker-reference.md @@ -0,0 +1,309 @@ +# Schema-Based Testing Docker Reference + +This reference covers everything that ships with the [Schema-Based Testing](overview.md) Docker container: the three ways to invoke it, environment variables, command-line flags, success criteria for CI/CD pipelines, report formats, request/response logs, mTLS, and policy management. For a guided first run, see [Setup](setup.md). + +## How runs are configured + +Schema-Based Testing always runs the same way — as a Docker container on your side that uses a Postman collection as the test basis, authenticates to Wallarm Cloud with a [token](setup.md#prerequisites), and reports results to **Test runs** and **Security Issues** (see [details](explore.md#test-runs-and-results)). + +What differs is how you provide the run parameters to the container: + +* **From a test policy**: you create a [test policy](setup.md#step-1-create-a-test-policy) in Wallarm Console; it stores the collection, environment file(s), and selected scan modes and strategies, and produces a Docker command linked to the policy. +* **Locally**: you build the Docker command yourself, passing the collection and other parameters as flags. The Wallarm Console wizard described in [Running without test policy](#running-without-test-policy) can compose this command for you. +* **Mixed**: you start from a policy's Docker command, but **override** individual parameters at run time (for example, point to a local collection file instead of the one stored in the policy). Anything not overridden is taken from the policy. + +## Requirements + +The Docker container needs network connectivity: + +* To the API being tested. +* To the Wallarm Cloud (US, EU, or ME): + + --8<-- "../include/cloud-ip-by-request.md" + +## Running with a test policy + +Once a test policy is [created](setup.md#step-1-create-a-test-policy), it provides a Docker run command that starts security testing for your application: + +1. Go to Wallarm Console → **Security Testing** → **Schema-Based** → **Test policies**. +1. Click your policy. The policy's Docker command is displayed. +1. If necessary, redefine the Docker log level and format, and the [test run success criteria](#test-run-success-criteria). These additions add extra [variables](#environment-variables) to the command. + + ![Schema-Based Testing - test policy Docker command](../../images/vulnerability-detection/sbt-policy-docker-command-new.png) + +1. Copy the command and run it, locally or from a CI/CD pipeline, on a machine that can reach the target application. + +## Running without test policy + +With a token, you can build the Docker run command yourself, defining all parameters locally. You can still use the Wallarm Console wizard to compose the command. + +To use the wizard: + +1. Go to **Security Testing** → **Schema-Based** and click **Generate test run command**. +1. Select the parameters; the wizard composes the Docker run command accordingly: + + * **Log level** — `Info` (default), `Debug`, `Warn`, `Error`. + * **Log format** — `Colored` (default), `JSON`, `Text`. + * **Fail on risk level** — `High` (default), `Critical`, `Medium`, `Low`, `Info`. See [Test run success criteria](#test-run-success-criteria). + +1. Copy the command. +1. Replace the placeholders with actual values. +1. Run the command on a machine that can reach the target application. + +If the wizard is left at default values, it does not add anything beyond the required environment variables to the Docker command. + +## Mixed: overriding policy parameters + +You can start from a policy's Docker command and override any parameter at run time — for example, use a local collection file instead of the one attached to the policy. Anything not overridden is taken from the policy. + +For overriding mechanics, see the Docker [`--help`](#available-flags) output and the **Configuration priority** line: CLI flags > environment variables > test policy > defaults. + +## Environment variables + +When you create and save a test policy in Wallarm, the resulting Docker run command includes all required `-e` variables and their values. The variables tell the Docker container which Wallarm Cloud account to connect to and which test policy to use as the basis for testing: + +Environment variable | Description| Required +--- | ---- | ---- +`WALLARM_API_HOST` | Wallarm API server: | Yes +`WALLARM_API_TOKEN` | Wallarm `Schema-Based Testing agent` API token. See details [here](setup.md#prerequisites). | Yes +`WALLARM_CLIENT_ID` | Numeric identifier of your Wallarm client account. The Console embeds it automatically when copying the command. | Yes +`WALLARM_TESTING_POLICY_ID` | ID of the test policy containing the test configuration. Required for runs linked to a policy. | Conditional +`FAIL_SEVERITY` | Defines [test run success criteria](#test-run-success-criteria) for runs that are part of a CI/CD pipeline. | No + +## Available flags + +You can use flags to modify the behavior of Schema-Based Testing running as a Docker container. To see the full list of available flags, use: + +``` + --help +``` + +The output looks like this: + +``` +Run Postman security testing workflow. + +This command performs automated API security testing using Postman collections, +executing comprehensive security tests against the target API and reporting vulnerabilities. + +OPERATING MODES: + +Standalone Mode: + Use local Postman collection without Wallarm platform integration. + Requires: --postman-collection-file flag + + Example: + docker run -v "$(pwd)/collection.json:/app/collection.json" \ + -e WALLARM_API_HOST="https://api.wallarm.com" \ + -e WALLARM_API_TOKEN="..." \ + wallarm/security-testing:v0.14.1 postman \ + --postman-collection-file /app/collection.json + +Cloud Mode: + Integrate with Wallarm platform using Test Policy. + Requires: WALLARM_TESTING_POLICY_ID environment variable or --testing-policy-id flag + + Example (with local files override): + docker run -v "$(pwd)/collection.json:/app/collection.json" \ + -v "$(pwd)/env.json:/app/env.json" \ + -e WALLARM_API_HOST="..." \ + -e WALLARM_API_TOKEN="..." \ + -e WALLARM_TESTING_POLICY_ID="..." \ + wallarm/security-testing:v0.14.1 postman \ + --postman-collection-file /app/collection.json \ + --postman-environment-files /app/env.json + + Example (uses collection from Test Policy): + docker run \ + -e WALLARM_API_HOST="..." \ + -e WALLARM_API_TOKEN="..." \ + -e WALLARM_TESTING_POLICY_ID="..." \ + wallarm/security-testing:v0.14.1 postman + +Configuration priority: CLI flags > Environment variables > Test Policy > Defaults + +Usage: +security-testing postman [flags] + +Flags: +-h, --help help for postman +--postman-collection-file string Path to local Postman collection file (JSON) +--postman-environment-files stringSlice + Path(s) to Postman environment file(s) (JSON, supports multiple files) + +Global Flags: +API: +--send-logs Send logs to Wallarm API (default true) +--testing-policy-id string Wallarm testing policy ID (optional, enables Test Policy integration) +--wallarm-api-host string Wallarm API host URL (required) +--wallarm-api-token string Wallarm API authentication token (required) + +GENERAL: +--export-files Upload local files (--postman-collection-file, --postman-environment-files) to Wallarm cloud +--fail-severity string Set fail severity level (critical, high, medium, low, info) (default "high") +--insecure Turn off SSL verification checks, and allow self-signed SSL certificates +--test-run-title string Custom title for the test run + +PREFLIGHT-CHECKS: +--preflight-check-timeout int Timeout in seconds for target URL connectivity checks (minimum: 10) (default "75") +--skip-target-url-connectivity-check + Skip checking target URL connectivity + +REPORTS: +--no-summary Disable summary output to stdout +-r, --report Enable report generation +--report-format string Report format (json, csv, junit) (default "json") +--report-output string Report output file path (default extension matches format: .json for JSON, .csv for CSV) (default "report/results.json") +--report-pretty Pretty-print JSON output (default true) + +REQUEST-LOG: +--request-log-output string Output file path for HAR request log (default "./results/request_log.har") +--send-request-logs Send request logs to Wallarm API (default true) + +LOGGING: +-f, --log-format string Set log format (json, colored, text) (default "colored") +-l, --log-level string Set log level (debug, info, warn, error) (default "info") + +MTLS: +--mtls-ca-cert string Path to CA certificate for server validation (PEM format) +--mtls-client-cert string Path to client certificate file (PEM format) +--mtls-client-key string Path to client private key file (PEM format, unencrypted) +``` + +See the explanation for the `--fail-severity` flag in the next section. + +## Test run success criteria + +When you include Schema-Based Testing in your CI/CD pipeline, define when a test run should be considered failed. Set **Fail on a risk level** in the test policy details to one of: + +* critical +* high (default) +* medium +* low +* info + +Selecting any value other than the default adds the `FAIL_SEVERITY` environment variable to the Docker run command. The selected value is also reflected in the policy's [Docker command panel](#running-with-a-test-policy) under **Fail on a risk level**. + +Example: + +``` +docker run -e WALLARM_API_HOST="us1.api.wallarm.com" -e WALLARM_API_TOKEN="" -e WALLARM_CLIENT_ID="" -e WALLARM_TESTING_POLICY_ID="" -e FAIL_SEVERITY="medium" --network="host" wallarm/security-testing:0.14.1 postman +``` + +You can also set the test run success criteria with the `--fail-severity` flag, for example: + +``` +docker run -e WALLARM_API_HOST="us1.api.wallarm.com" -e WALLARM_API_TOKEN="" -e WALLARM_CLIENT_ID="" -e WALLARM_TESTING_POLICY_ID="" --network="host" wallarm/security-testing:0.14.1 postman --fail-severity medium +``` + +If testing finds at least one medium (or high or critical) security issue, the test run exits with code `1` ("test failed"). + +## Report generation + +Schema-Based Testing can generate reports in multiple formats for integration with various tools and CI/CD pipelines. + +To enable report generation, use the `--report` flag along with `--report-format` and `--report-output`: + +``` +docker run -v "$(pwd)/reports:/app/report" \ + -e WALLARM_API_HOST="us1.api.wallarm.com" \ + -e WALLARM_API_TOKEN="" \ + -e WALLARM_CLIENT_ID="" \ + -e WALLARM_TESTING_POLICY_ID="" \ + --network="host" wallarm/security-testing:0.14.1 postman \ + --report --report-format json --report-output /app/report/results.json +``` + +Available report formats: + +| Format | Description | Use case | +| ------ | ----------- | -------- | +| `json` | JSON format with full vulnerability details | Custom integrations, detailed analysis | +| `csv` | Comma-separated values | Spreadsheet analysis, data processing | +| `junit` | JUnit XML format | CI/CD pipeline integration (Jenkins, GitLab CI, GitHub Actions, etc.) | + +### JUnit reports for CI/CD + +JUnit format is particularly useful for CI/CD integration, since most CI/CD platforms natively support JUnit test reports for visualizing test results. + +Example for a CI/CD pipeline: + +``` +docker run -v "$(pwd)/reports:/app/report" \ + -e WALLARM_API_HOST="us1.api.wallarm.com" \ + -e WALLARM_API_TOKEN="" \ + -e WALLARM_CLIENT_ID="" \ + -e WALLARM_TESTING_POLICY_ID="" \ + --network="host" wallarm/security-testing:0.14.1 postman \ + --report --report-format junit --report-output /app/report/security-tests.xml \ + --fail-severity high +``` + +This generates a JUnit XML report that CI/CD tools consume to display test results and track security issues over time. + +## Request/response log export + +Schema-Based Testing can export full HTTP request and response logs in HAR (HTTP Archive) format. This is useful for debugging, compliance, and detailed analysis of the security tests performed. + +By default, request logs are sent to the Wallarm API. You can also export them locally: + +``` +docker run -v "$(pwd)/logs:/app/results" \ + -e WALLARM_API_HOST="us1.api.wallarm.com" \ + -e WALLARM_API_TOKEN="" \ + -e WALLARM_CLIENT_ID="" \ + -e WALLARM_TESTING_POLICY_ID="" \ + --network="host" wallarm/security-testing:0.14.1 postman \ + --request-log-output /app/results/request_log.har +``` + +Request log flags: + +| Flag | Description | Default | +| ---- | ----------- | ------- | +| `--request-log-output` | Output file path for HAR request log | `./results/request_log.har` | +| `--send-request-logs` | Send request logs to Wallarm API | `true` | + +The HAR format is widely supported by browser developer tools and API debugging applications, making it easy to analyze the exact requests and responses generated during testing. + +## mTLS support + +Schema-Based Testing supports mutual TLS (mTLS) authentication for testing APIs that require client certificate authentication. + +To use mTLS, provide the client certificate, private key, and optionally the CA certificate: + +``` +docker run -v "$(pwd)/certs:/app/certs" \ + -e WALLARM_API_HOST="us1.api.wallarm.com" \ + -e WALLARM_API_TOKEN="" \ + -e WALLARM_CLIENT_ID="" \ + -e WALLARM_TESTING_POLICY_ID="" \ + --network="host" wallarm/security-testing:0.14.1 postman \ + --mtls-client-cert /app/certs/client.crt \ + --mtls-client-key /app/certs/client.key \ + --mtls-ca-cert /app/certs/ca.crt +``` + +mTLS flags: + +| Flag | Description | Format | +| ---- | ----------- | ------ | +| `--mtls-client-cert` | Path to client certificate file | PEM format | +| `--mtls-client-key` | Path to client private key file | PEM format, unencrypted | +| `--mtls-ca-cert` | Path to CA certificate for server validation | PEM format | + +!!! note "Self-signed certificates" + If your target API uses self-signed certificates without mTLS, you can use the `--insecure` flag to skip SSL verification checks. + +## Editing a policy + +You can edit a previously created policy. Click the policy to open its Docker command details, or use the edit button to open the edit dialog. + +## Deleting a policy + +You can delete a test policy. If you do so: + +* Information on previous test runs remains untouched. +* You will not be able to start a Docker run based on the deleted policy. +* If the policy's Docker containers are running, they continue to run and the testing continues. +* When the policy's Docker containers stop, you cannot re-run them. diff --git a/docs/latest/vulnerability-detection/schema-based-testing/explore.md b/docs/latest/vulnerability-detection/schema-based-testing/explore.md index 8facd64b08..f4bb99e977 100644 --- a/docs/latest/vulnerability-detection/schema-based-testing/explore.md +++ b/docs/latest/vulnerability-detection/schema-based-testing/explore.md @@ -1,12 +1,26 @@ # Exploring Test Run Results -Once [Schema-Based Testing](overview.md) is enabled and tests [are run](setup.md#docker-run), you can explore the test run results as described in this article. +Once [Schema-Based Testing](overview.md) is [set up](setup.md) and a test [has been run](docker-reference.md#running-with-a-test-policy), you can explore the results in Wallarm Console as described in this article. ## Test runs and results In Wallarm Console, go to **Security Testing** → **Schema-Based** → **Test runs** to view all test runs and their results. -Here you can review security issues detected during the run, health checks, and the Docker output for the run: +The **Test runs** tab lists every run started against any of your test policies. Each row shows: + +* Started / finished date and run duration. +* Run title and policy used. +* Target URL (taken from the Postman collection or environment). +* Number of replayed requests. +* **Credits** consumed by the run. +* Number of detected security issues grouped by risk level. +* Run status. + +Filter by policy, status, or test run title to narrow the list. + +![Schema-Based Testing - test runs list](../../images/vulnerability-detection/sbt-test-runs-row.png) + +For each completed run you can review the security issues detected, the health checks performed at the start of the run, and the Docker output captured during execution:
@@ -15,61 +29,101 @@ Here you can review security issues detected during the run, health checks, and
+Detected vulnerabilities are also published to the Wallarm Console's [**Security Issues**](../../api-attack-surface/security-issues.md) section with `Discovered by = Schema-Based Testing` and a link back to the originating test run, so they can be tracked, triaged, and resolved alongside vulnerabilities found by other Wallarm components. + ## Test run details -You can obtain test run details from Wallarm Cloud and from you Docker container output. +Click a test run to open its details page. From here you can also: + +* Click **Download Results** to export the run report. +* Click **Copy link** to share a direct link to this run with colleagues. + +The details page is organized into the following sections. + +![Schema-Based Testing - test run details](../../images/vulnerability-detection/sbt-test-run-details.png) + +### Health Checks + +Confirms that the inputs were valid and the run executed end-to-end: + +* **Postman collection validation** — the supplied collection was parsed successfully. +* **Execution success check** — the run completed without unrecoverable errors. + +### Errors & Warnings -### In Wallarm UI +Lists any non-fatal problems encountered during the run (for example, individual requests that failed to replay). An empty section means the run completed cleanly. -Click test run to expand and see its detailed technical information: +### Tests -![Schema-based security testing - test run details](../../images/vulnerability-detection/apitp-test-run-details.png) +Shows the engine's progress through the run as a sequence of stages: -You can check: +| Stage | What happens | +|---|---| +| **Generating** | The engine analyzes the collection and generates vulnerability hypotheses. | +| **Ready** | Hypotheses are prepared; test cases are about to be executed. | +| **Executing** | Generated tests are sent against the target application. | +| **Improving** | The engine retries inconclusive tests with refined inputs. | +| **Compiling** | Findings are compiled and mapped to security issues. | +| **Completed** | The run is finished. | -* Test run health: is specification/Postman collection valid and was the run successful. -* Errors and warnings (if any) occurred while running. -* Docker container output (with the **Copy to clipboard** capability). +When a run is in progress, the current stage is highlighted; previous stages are marked complete. -Also you can: +### Strategies -* Download JSON report for your test run. -* Copy link to the test run details to share with colleagues. +Below **Tests**, the run's results are grouped by [strategy](strategies.md). Each strategy block expands into the list of **hypotheses** the engine produced for that strategy and the verdict for each one: + +| Verdict | Meaning | +|---|---| +| **Vulnerable** | The validation step confirmed the hypothesis — a real, reproducible issue was found. A linked security issue is created in **Security Issues**. | +| **Not Vulnerable** | The validation step rejected the hypothesis. The application behaves as expected for this attack scenario. | + +Click any hypothesis to open its detail panel. The panel exposes four tabs: + +* **Hypothesis** — a description of what was tested: the targeted endpoint, the vulnerability under investigation, severity, and a step-by-step exploitation chain. +* **Source Code** — the test script the engine generated and executed for this hypothesis. Downloadable for offline reproduction. +* **Output** — the captured execution log of the test (requests, responses, intermediate decisions). +* **Finding** — for confirmed issues, the structured finding that was promoted to **Security Issues** (vulnerability type, evidence, request/response excerpts). + +![Schema-Based Testing - hypothesis detail panel](../../images/vulnerability-detection/sbt-hypothesis-detail.png) ### Docker container output -You can find test run results in your Docker container output directly, for example: +You can find test run progress and results in your Docker container output directly: + +![Schema-Based Testing - Docker container output](../../images/vulnerability-detection/sbt-docker-container-output.png) + +A typical run ends with a summary similar to: ``` -Sep 2 18:26:40.222 INF STARTING with arguments test_profile_id=60 client_id=5 wallarm_api_host=https://us1.api.wallarm.com wallarm_api_token=*****FWmup send_logs=true fail_severity=high -Sep 2 18:26:40.228 INF Running pre-flight checks -Sep 2 18:26:40.698 INF Successfully authenticated with Wallarm API -Sep 2 18:26:40.698 INF All pre-flight checks passed -Sep 2 18:26:40.698 INF Requesting test policy test_profile_id=60 -Sep 2 18:26:41.926 INF Test policy received -Sep 2 18:26:42.383 INF Running Postman workflow -Sep 2 18:26:42.383 INF Getting postman configuration files test_profile_id=60 -Sep 2 18:26:43.168 INF Running target URL connectivity check with Newman -Sep 2 18:27:40.723 INF Target URL connectivity check with Newman passed total_requests=47 failed_requests=9 failure_ratio=0.19148936170212766 threshold=0.75 -Sep 2 18:27:40.727 INF Starting Postman security testing workflow -Sep 2 18:33:51.069 INF Postman security testing completed successfully -Sep 2 18:33:51.069 INF Workflow completed successfully -Sep 2 18:33:57.283 INF Security issues summary total_issues=17 severity_counts.high=16 severity_counts.medium=1 -Sep 2 18:33:57.410 INF Container logs exportedto the cloud -Sep 2 18:33:57.411 ERR Exiting with code 1 due to security issues meeting fail-severity threshold fail_severity=high count=16 +INF Postman security testing completed successfully +INF Workflow completed successfully +INF Security issues summary total_issues=26 severity_counts.high=26 +ERR Exiting with code 1 due to security issues meeting fail-severity threshold fail_severity=high count=26 ``` -In the example, pay your attention to exit with code 1 (failed tests) due to set [test run success criteria](setup.md#test-run-success-criteria). +Note the exit with code `1` (failed tests) due to the configured [test run success criteria](docker-reference.md#test-run-success-criteria). + +## Reviewing detected security issues + +When the run is finished, the **Security issues** column on the **Test runs** tab shows the number of confirmed findings and their distribution by risk level. Click that number to open the [**Security Issues**](../../api-attack-surface/security-issues.md) page filtered to this run. + +On the filtered Security Issues page you can: + +* Use the top widgets to see **Top vulnerable hosts**, **Security issues by types** (BFLA, BOLA, Authentication bypass, Information exposure, SQLi, Denial of Service, etc.), and **Statistics** by severity. +* Refine by **Type**, **Risk**, **OWASP**, **Status**, **Incident**, or **Discovered by** filters. Schema-Based Testing findings carry `Discovered by = Schema-Based Testing` and the originating test run ID. +* Open any issue to inspect its full details — request/response evidence, the generated description, OWASP mapping, and the linked test case from the originating test run. + +![Schema-Based Testing - Security Issues filtered by test run](../../images/vulnerability-detection/sbt-security-issues-by-run.png) ## Downloading initial files -From the **Schema-Based Testing** page, you can download files that you used for creating OAS or Postman-based test policies: +From the **Schema-Based Testing** page, you can download the files used to create a test policy: -* OAS: its specification. -* Postman: Postman collection file, Postman environment file(s) if were used +* The Postman collection file. +* The Postman environment file(s), if attached. To download files: -1. In Wallarm Console, go to **API Security Testing** → **Schema-Based** → **Test policies** tab. -1. Hover your policy, and click the edit icon. Policy edit dialog is displayed. -1. In corresponding file field, click the download icon. +1. In Wallarm Console, go to **Security Testing** → **Schema-Based** → **Test policies**. +1. Hover over your policy and click the edit icon. The policy edit dialog opens. +1. In the corresponding file field, click the download icon. diff --git a/docs/latest/vulnerability-detection/schema-based-testing/overview.md b/docs/latest/vulnerability-detection/schema-based-testing/overview.md index 0a3201b164..8dc2598376 100644 --- a/docs/latest/vulnerability-detection/schema-based-testing/overview.md +++ b/docs/latest/vulnerability-detection/schema-based-testing/overview.md @@ -1,96 +1,53 @@ # Schema-Based Testing -Wallarm's Schema-Based Testing is a dynamic application security testing (DAST) solution that enables "shift-left" security. It uses an API's schema (such as an OpenAPI specification or a Postman collection) as a blueprint to automatically generate and execute targeted security tests. By integrating into CI/CD pipelines, Schema-Based Testing allows development teams to proactively identify a wide range of vulnerabilities—including OWASP API Top 10 risks, business logic flaws, and input validation issues—early in the development process, making them easier and cheaper to fix. +Wallarm's Schema-Based Testing is a dynamic application security testing (DAST) solution that enables "shift-left" security. It takes a [Postman collection](https://www.postman.com/product/collections/) — your existing functional tests — as a blueprint for security tests and runs them as a Docker container that fits into CI/CD pipelines next to functional tests, smoke tests, and other security testing. -Schema-Based Testing capabilities: +![Schema-Based Testing - test runs](../../images/vulnerability-detection/sbt-test-runs-row.png) -* Deep, dynamic analysis of API endpoints. -* Detection of vulnerabilities in the application or API itself, as well as security misconfigurations in the underlying infrastructure or environment. -* Visualization of found issues in the Wallarm Console's [**Security Issues**](../../api-attack-surface/security-issues.md) section. -* Lightweight execution via Docker container, enabling embedding into your CI/CD pipeline alongside functional tests, smoke test, and other security testing. +Use [Schema-Based Testing Setup](setup.md) to get from a fresh subscription to your first test run. -![Schema-Based Testing - test runs](../../images/vulnerability-detection/sbt-test-runs.png) +## AI-driven engine -## How it works +Schema-Based Testing is built on an AI-driven scanning engine that goes beyond signature-based scanners: -Use Schema-Based Testing by fulfilling the following steps: +* The engine analyzes the application context derived from the Postman collection, generates vulnerability hypotheses, builds attack chains, and validates each finding with an executable proof-of-exploit test before reporting it. +* This approach detects multi-step issues such as broken object level authorization (BOLA), broken function level authorization (BFLA), business logic abuse, and mass assignment — vulnerabilities that traditional payload-based DAST tools struggle to catch. +* Confirmed findings include a generated description, the test script that reproduces the issue, and the exploitation log captured during the run, so each result is reproducible by the development team. -1. **Create test policy**: specify the target application, provide its OpenAPI specification or Postman collection, base URL, and select the tests to run. -1. **Copy Docker command**: find your test policy on the **Test policies** tab, click it, and copy the provided Docker command. -1. **Run and monitor**: start the agent with the command. Track progress and view results on the **Test runs** tab. +## Scan modes -![Schema-Based Testing - how it works](../../images/vulnerability-detection/sbt-diagram.png) +Schema-Based Testing organizes its work into two scan modes that you enable independently in a test policy: -## Test basis +| Scan mode | Purpose | +|---|---| +| **Passive Scan** | Inspects the HTTP traffic captured by replaying the Postman collection, without sending additional payloads. Detects exposure of sensitive data, missing security headers, insecure cookies, debug output, and similar issues observable in normal responses. | +| **Active Scan** | Generates and sends targeted attack requests built from collection context, then validates each finding with an exploit test. Detects business logic, access control, injection, and other multi-step vulnerabilities. | -Schema-Based Testing can base its tests on: +Each scan mode is powered by one or more [strategies](strategies.md) — reusable scanning recipes, each targeting one vulnerability class. Wallarm ships a default catalog of Active and Passive strategies, and you can add custom ones. -* **OpenAPI specification (OAS)** - precise and machine-readable blueprint of your API allows to build efficient and reliable test suite for your application. You can use an OAS [exported from API Discovery](../../api-discovery/exploring.md#openapi-oas-export) or from any other source. OAS-based testing is focused on input validation, injection, and misconfiguration detection. -* **RAML specification** - if your API is documented using [RAML (RESTful API Modeling Language)](https://raml.org/), you can upload it directly. RAML files are automatically converted to OpenAPI specifications, enabling the same comprehensive security testing. See [details](setup.md#raml-based-test-policies). -* **Postman collection** - if you use the [**Postman**](https://www.postman.com/) API design platform, the **functional tests** from its [*collections*](https://www.postman.com/product/collections/) may be used to build security tests alongside. See [details](setup.md#postman-collection-based-test-policies). Postman collection-based testing is focused on complex business logic and access control vulnerabilities. +## Capabilities - !!! warning "Temporary limitation" - Creating new policies based on Postman collections is temporarily not available. Existing policies that were previously created using Postman collections continue to work as expected. +* Deep, dynamic analysis of API endpoints based on the supplied Postman collection. +* Detection of vulnerabilities listed in the [OWASP API Security Top 10](https://owasp.org/API-Security/editions/2023/en/0x11-t10/) and the OWASP Top 10, plus security misconfigurations exposed through the application's HTTP traffic. +* Visualization of found issues in the Wallarm Console's [**Security Issues**](../../api-attack-surface/security-issues.md) section, with the originating test run as the source. +* Lightweight execution via a Docker container, with run progress and aggregated results streamed back to Wallarm Cloud. -## Schema-Based Testing (Postman) vs API Security Testing via Postman +## Schema-Based Testing vs API Security Testing via Postman -Wallarm also offers [API Security Testing via Postman](../api-security-testing-via-postman/overview.md)—a lighter option that runs inside Postman Agent Mode with no Docker or CI/CD. Both options can use your Postman collections; choose based on how you work and how deep you need to go: +Wallarm also offers [API Security Testing via Postman](../api-security-testing-via-postman/overview.md) — a lighter solution that runs inside Postman Agent Mode with no Docker or CI/CD. Both products take a Postman collection as input; choose based on how you work and how deep you need to go: -| | **Schema-Based Testing** (Postman collection) | **API Security Testing via Postman** | +| | **Schema-Based Testing** | **API Security Testing via Postman** | |---|---|---| -| **Use when** | You want automated, comprehensive DAST in CI/CD; you already have functional tests in Postman and want them to drive security tests. | You want a quick, conversational check inside Postman—ask in natural language and get results in the Agent chat in a few minutes. | -| **How it runs** | Dynamic testing: sends real requests, uses your collection's functional tests as a blueprint to generate and run security tests. | Passive, design-level analysis; no attack payloads, no traffic replay. | -| **Depth** | OWASP API Top 10, business logic, access control, input validation (injections, RCE, etc.), environment misconfigurations. | Auth gaps, data leaks, over-permissive endpoints, schema issues, basic BOLA/BOPLA—summarized for developers. | -| **Where** | Docker-based; runs in your pipeline or locally; results in Wallarm Console (Test runs, Security Issues). | Inside Postman (Agent Mode); results in chat and in Wallarm Cloud. | +| **Use when** | You want automated, comprehensive DAST embedded in CI/CD; you already have functional tests in Postman and want them to drive security tests. | You want a quick, conversational check inside Postman — ask in natural language and get results in the Agent chat in a few minutes. | +| **How it runs** | Dynamic testing as a Docker container: replays the collection, then generates and validates targeted security tests against the application. | Passive, design-level analysis; no attack payloads, no traffic replay. | +| **Depth** | OWASP API Top 10, business logic, access control, input validation (injections, RCE, etc.), and traffic-observable misconfigurations. | Auth gaps, data leaks, over-permissive endpoints, schema issues, basic BOLA/BOPLA — summarized for developers. | +| **Where** | Docker container runs in your pipeline or locally; results in Wallarm Console (Test runs, Security Issues). | Inside Postman (Agent Mode); results in chat and in Wallarm Cloud. | -In short: use **Schema-Based Testing** with a Postman collection when you need full DAST and pipeline integration; use API Security Testing via Postman for fast, in-Postman checks with minimal setup. +In short: use **Schema-Based Testing** when you need full DAST and pipeline integration; use API Security Testing via Postman for fast, in-Postman checks with minimal setup. -## Test types +## Where to go next -For OpenAPI specification-based tests, Schema-Based Testing uses several types of tests to detect security issues: - -* **Authentication bypass** checks for risks of [bypassing](../../attacks-vulns-list.md#authentication-bypass) the authentication mechanisms or exploiting their weaknesses. -* **Environment misconfiguration tests** check for vulnerabilities and misconfigurations in the environment or infrastructure the application and APIs run on (not the API logic). Examples: - - * Exposed source code, backups, configuration files. - * Accessible `.git`, `.env`, or system files. - * Insecure web server settings (e.g., directory listing, weak TLS). - -* **GraphQL vulnerability detection** checks for 10 GraphQL most popular misconfigurations (API2, API4). - -* **Input parameter tests** check each input point (parameters, headers, etc.) defined in the OpenAPI specification for application-level vulnerabilities. Covered vulnerabilities: - - * Command injection - * CRLF injection - * LFI / RFI - * NoSQL injection - * Open redirect - * Path traversal - * Remote code execution (RCE) - * SQL injection - * SSRF - * SSTI - * XSS - * XXE - * Infoleak - -## Run types - -Any test run via Schema-Based Testing: - -* Is actually a Docker run -* Is based on some [schema](#test-basis) (OAS or Postman) -* Requires [token](setup.md#token) from Wallarm -* Is displayed in **Test runs**, results are displayed in **Security Issues** (see [details](explore.md#test-runs-and-results)) - -In the same time, you are free to choose whether to use Wallarm's UI: - -* **With UI**: you create test policy in Wallarm, it contains token, your schema and all testing parameters and provides you a Docker run command **linked** to it. -* **Without UI**: having token, you form Docker run command yourself defining all parameters locally. Note that you can still use Wallarm's [wizard](setup.md#running-without-test-policy) to help you with that. -* **Combination**: you create test policy in Wallarm, it contains token, your schema and all testing parameters and provides you a Docker run command linked to it, but **in this linked** command you **override** any parameters, like use local file instead of the one attached to the policy. What is not overridden, is taken from the policy. - -See details on each variant [here](setup.md#docker-run). - -## Enabling and setup - -To start using Schema-Based Testing, enable and configure it as described in [Schema-Based Testing Setup](setup.md). \ No newline at end of file +* [Setup](setup.md) — activate the subscription, create your first policy, and run a scan. +* [Strategies](strategies.md) — catalog of default strategies and how to manage custom ones. +* [Docker reference](docker-reference.md) — full Docker command, environment variables, CI/CD options, reports, mTLS. +* [Exploring test run results](explore.md) — how to read the **Test runs** page, run details, and the resulting security issues. diff --git a/docs/latest/vulnerability-detection/schema-based-testing/setup.md b/docs/latest/vulnerability-detection/schema-based-testing/setup.md index 0deeff0e57..2d1140a212 100644 --- a/docs/latest/vulnerability-detection/schema-based-testing/setup.md +++ b/docs/latest/vulnerability-detection/schema-based-testing/setup.md @@ -1,579 +1,61 @@ # Schema-Based Testing Setup -This article describes how to enable and configure Wallarm's [Schema-Based Testing](overview.md). - -## Enable - -Schema-Based Testing is disabled by default. To enable: - -1. Contact sales@wallarm.com to activate the [**Security Testing** subscription plan](../../about-wallarm/subscription-plans.md#core-subscription-plans) for your account. -1. Go to the **Security Testing** → **Schema-Based** → **Test policies** tab and create [at least one policy](#test-policy-types). +This article walks through the minimum steps required to go from a fresh account to your first Schema-Based Testing run. For advanced configuration, see [Docker reference](docker-reference.md); for the catalog of detection rules, see [Strategies](strategies.md). ## Prerequisites -### Token - -Schema-Based Testing requires a [token](../../user-guides/settings/api-tokens.md) for authorizing data exchange between running Docker container and Wallarm Cloud. The token can be created in two ways: - -* **Automatically** - Schema-Based Testing will create it automatically and include into the `docker run` command on first attempt to copy Docker command from any policy. Other policies will re-use already existing token. -* **Manually** - in Wallarm Console, go to **Settings** → **API Tokens**, click **New token**; on creation, set **Token usage** to `Schema-Based Testing agent`. All policies will use this token. - -### Whitelist - -When you are planning to run Wallarm's Schema-Based Testing against domains protected by some security tools, it is recommended to add the client IP (IP from where the Docker command is being run) to the allowlist, otherwise majority of requests will be blocked and the vulnerabilities will not be found. - -This includes the case when Wallarm itself is used as the protection tool for these domains - see details on Wallarm's allowlist [here](../../user-guides/ip-lists/overview.md). - -## Test policy types - -You can configure test policy [based](overview.md#test-basis) on OpenAPI specification (OAS), RAML specification, or Postman collection. Note that one test policy is aimed at only one type of testing. - -* OAS and RAML are more focused on input validation, injection, misconfiguration detection -* Postman - on complex business logic and access control vulnerabilities - -## OAS-based test policies - -OpenAPI specification (OAS)-based test policy defines persistently: - -* Application's **OpenAPI specification** -* Tests to run - -Besides persistent parameters that are the same for any test run, each test policy may optionally include parameters that can be overridden during each next test run (**Runtime parameters**). Re-defining the runtime parameters can be useful for embedding of Docker into the CI/CD pipelines: - -* Application's **Target URL** - - (although can be redefined during each run, some initial value is required) - -* Authentication parameters - -To configure test policy: - -1. Go to Wallarm Console → **Security Testing** → **Schema-Based** → **Test policies**. -1. Click **Add policy**, attach OpenAPI specification file. - - !!! info "Using OAS from API Discovery" - You can use an OpenAPI specification exported from [API Discovery](../../api-discovery/exploring.md#openapi-oas-export). In **API Discovery**, select a host, click **Download report** → **OpenAPI (OAS 3.1, JSON)** → **Generate OAS**, then attach the downloaded file when creating the test policy. -1. Select [test types](overview.md#test-types) to run. -1. Set **Target URL** (can be overridden dynamically during each test run). -1. Optionally, add authentication **Runtime parameters**. +Before you create a test policy, make sure the following is in place: - ![Schema-Based Testing - creating test policy based on OpenAPI specification](../../images/vulnerability-detection/sbt-policy-create.png) - -## RAML-based test policies - -If your API is documented using RAML (RESTful API Modeling Language), you can create test policies directly from RAML specifications. Wallarm automatically converts RAML files to OpenAPI specifications internally, enabling the same comprehensive security testing available for OAS-based policies. - -RAML-based test policy defines persistently: - -* Application's **RAML specification** -* Tests to run - -Besides persistent parameters, each test policy may include **Runtime parameters** that can be overridden during each test run: - -* Application's **Target URL** (although can be redefined during each run, some initial value is required) -* Authentication parameters - -To configure test policy: - -1. Go to Wallarm Console → **Security Testing** → **Schema-Based** → **Test policies**. -1. Click **Add policy**, select **RAML spec** as the source, and upload your RAML specification file. -1. Select [test types](overview.md#test-types) to run. -1. Set **Target URL** (can be overridden dynamically during each test run). -1. Optionally, add authentication **Runtime parameters**. - - ![Schema-Based Testing - creating test policy based on RAML specification](../../images/vulnerability-detection/sbt-policy-create-raml.png) - -## Postman collection-based test policies - -!!! warning "Temporary limitation" - Creating new policies based on Postman collections is temporarily not available. Existing policies that were previously created using Postman collections continue to work as expected - only the creation of new Postman-based policies is currently disabled. - -With Postman-based security testing you can automate security scans alongside your regular API tests, ensuring that each API run is thoroughly tested for vulnerabilities. - -!!! info "API functional tests as basis" - Wallarm's schema-based testing leverages your functional tests to generate security tests. The more complete your functional tests are, the broader the security coverage Wallarm can provide. More APIs, users, and requests mean richer and more effective security testing. - -### Pre-check of Postman collection - -Before using Postman data for schema-based security testing: - -1. Ensure that you collection and environment files contain: - - * Functional tests for API endpoints - * Location of the target application - * All environment variables set - * Necessary credentials to authenticate in the target application - -1. (Recommended) Check whether the Postman collection is working. For example, this can be done by running the command: +* **Security Testing subscription** — contact sales@wallarm.com to activate the [**Security Testing** subscription plan](../../about-wallarm/subscription-plans.md#core-subscription-plans) for your account. Once the subscription is assigned, the **Schema-Based** entry appears in the left menu under **Security Testing**. +* **Authentication token** — Wallarm Cloud uses a [token](../../user-guides/settings/api-tokens.md) to authorize data exchange between the Docker container and your account. You can let Schema-Based Testing create the token automatically the first time you copy a Docker command, or create it in advance under **Settings** → **API Tokens** with **Token usage** set to `Schema-Based Testing agent`. +* **IP allowlist (if applicable)** — when the target API is protected by Wallarm or other security tools, [allowlist](../../user-guides/ip-lists/overview.md) the IP from which the Docker command will run. Otherwise, attack requests are blocked and vulnerabilities go undetected. +* **Postman collection** — a working Postman collection (and optionally a Postman environment file) that describes how to call the target API. The collection must contain functional tests for the API endpoints, the location of the target application, all required environment variables, and credentials to authenticate. Verify it runs end-to-end, for example: ``` newman run my_collection.json -e my_environment.json ``` - This can tell you in advance whether there are any problems, for example, related to the quality of the Postman collection, the availability of the target URL, or that all the necessary variables are specified. - -!!! warning "Valid Postman collection" - If the Postman collection itself cannot work, then the schema-based security testing will not work either. - -### Configuring test policy in Wallarm + If the collection cannot run on its own, Schema-Based Testing cannot run either. -In Wallarm, Postman collection-based test policy defines: +## Step 1. Create a test policy -* Application's **Postman collection**. -* **Postman environment file(s)** (optional if all configuration is stored in the main collection file). +A test policy bundles the Postman collection, environment file(s), and selected scan modes into a single reusable configuration. - !!! info "Target URL and authentication" - Application's **Target URL** and authentication parameters are defined in the Postman collection or environment files. +1. Go to Wallarm Console → **Security Testing** → **Schema-Based** → **Test policies** and click **Create test policy**. -* Test case selection is not currently supported for security testing based on Postman collections. + ![Schema-Based Testing - test policies list](../../images/vulnerability-detection/sbt-policies-list.png) -To configure test policy: +1. Provide a **Policy name**. +1. Set **Source** to **Postman collection** and attach the Postman collection file. +1. Optionally, attach Postman environment file(s). Attaching two environment files runs testing twice with different variable values (for example, different credentials) and then compares results. +1. Under **Working Modes**, enable **Active Scan**, **Passive Scan**, or both. For each enabled mode, click **Advanced settings** to choose which [strategies](strategies.md) participate. By default, all strategies of the corresponding type are enabled — you can leave the defaults or narrow the selection. +1. Save the policy. -1. Go to Wallarm Console → **Security Testing** → **Schema-Based** → **Test policies**. -1. Click **Add policy**, set **Source** to **Postman collection**. -1. Attach Postman collection file. -1. Optionally, attach Postman environment file(s). Attaching 2 files allows running testing twice with different variable values (for example, different credentials) and then comparing results. + ![Schema-Based Testing - creating test policy based on Postman collection](../../images/vulnerability-detection/sbt-policy-create-postman-new.png) - ![Schema-Based Testing - creating test policy based on Postman collection](../../images/vulnerability-detection/sbt-policy-create-postman.png) +!!! info "Target URL and authentication" + For Postman-based policies, the **Target URL** and authentication parameters are taken from the Postman collection or environment file(s). They are not configured in the policy and cannot be redefined per Docker run. -### Business logic security testing +## Step 2. Copy the Docker command -For business logic testing ([OWASP API1:2023 Broken Object Level Authorization (BOLA)](https://github.com/OWASP/API-Security/blob/master/editions/2023/en/0xa1-broken-object-level-authorization.md) and [OWASP API5:2023 Broken Function Level Authorization (BFLA)](https://github.com/OWASP/API-Security/blob/master/editions/2023/en/0xa5-broken-function-level-authorization.md)) Wallarm's Schema-Based Security Testing requires API traffic from at least two different authenticated users, preferably with varying privileges (e.g., admin and regular users). This diversity enables more effective business logic security tests and provides a more thorough assessment. +1. On the **Test policies** tab, click the policy you just created. The policy's Docker command is displayed. +1. Copy the command. You can adjust the Docker log level, log format, and [success-criteria threshold](docker-reference.md#test-run-success-criteria) right above the command. -| Business logic vulnerability | Input requirements | -| ---- | ---- | -| [OWASP API1:2023 Broken Object Level Authorization (BOLA)](https://github.com/OWASP/API-Security/blob/master/editions/2023/en/0xa1-broken-object-level-authorization.md) | Testing for this vulnerability requires multiple authenticated users to demonstrate whether proper object-level authorization checks are in place. | -| [OWASP API5:2023 Broken Function Level Authorization (BFLA)](https://github.com/OWASP/API-Security/blob/master/editions/2023/en/0xa5-broken-function-level-authorization.md) | Testing for this vulnerability requires requests from users with different privilege levels to evaluate whether function-level authorization is enforced consistently. | + ![Schema-Based Testing - test policy Docker command](../../images/vulnerability-detection/sbt-policy-docker-command-new.png) -#### Example 1: Using Postman collections with requests from two users +## Step 3. Run the scan -Do the following: +Run the copied Docker command on a machine that has network connectivity: -1. Create a Postman collection containing requests from two authenticated users. For example, in the target application, include login and activity requests from `User A` and `User B` in the same collection. - - ![Postman collection - 2 users transactions](../../images/vulnerability-detection/posman-collection-2-users.png) - -1. Verify that all requests are executed correctly. -1. Create a test policy with the created Postman collection. - -#### Example 2: Using Postman environments for multiple users - -Do the following: - -1. Create two Postman environment files, each holding the credentials for a different authenticated user, e.g.: - - * `env1.json` for `User A` - * `env2.json` for `User B` - -1. Create a test policy that uses your Postman collection and both environment files. - - In this setup, the collection is executed twice - once with `env1.json` (`User A`) and once with `env2.json` (`User B`) - so each request runs under both user contexts. - -## Editing existing policy - -You can edit previously created policies: while clicking policy itself opens its Docker command info, you can click the edit button to access the edit dialog: - -![Schema-Based Testing - editing test policy](../../images/vulnerability-detection/sbt-policy-edit.png) - -## Docker run - -### Requirements - -The Docker container should have network connectivity: - -* To the API being tested -* To the Wallarm Cloud (US or EU): +* To the API being tested (`Target URL` from the Postman collection or environment). +* To Wallarm Cloud (US, EU, or ME): --8<-- "../include/cloud-ip-by-request.md" -### Running with test policy - -As test policy is [created](#test-policy-types), it provides you with the Docker run command which allows you start tests for your application: - -1. Go to Wallarm Console → **Security Testing** → **Schema-Based** → **Test policies**. -1. Click you policy. The policy's Docker command will be displayed. -1. If necessary, redefine Docker log level and format, and [test run success criteria](#test-run-success-criteria). This adds extra [variables](#environment-variables) to the command. - - ![Schema-Based Testing - test policy Docker command](../../images/vulnerability-detection/sbt-policy-docker-command.png) - -1. Copy command and run it or embed into your CI/CD pipeline. This will run security tests selected in the policy for your application. - - Remember that, for OAS-based run, you can override the policy's **Runtime parameters** on each run by adding the corresponding `-e` parameters to the Docker `run` command, for example: - - ``` - -e TARGET_URL="http://dvapi.st.wallarm.tools" - -e AUTH_HEADER="Authorization: Bearer " - ``` - - You can simplify re-defining these parameters by selecting the **Rewrite authentication data** and/or **Rewrite target URL** checkboxes that add to the command: - - ``` - -e AUTH_HEADER="${AUTH_HEADER}" -e TARGET_URL="${TARGET_URL}" - ``` - -1. View run statistics and [test run results](explore.md) on the **Test runs** tab. - -### Running without test policy - -Having token, you can form Docker run command yourself defining all parameters locally. Note that you can still use Wallarm's wizard to help you with that. - -To use wizard: - -1. Go to **Schema-Based Testing** section. -1. Click the **Generate test run command** button. -1. Select appropriate parameters, the wizard will compose the Docker run command correspondingly. - - ![Schema-Based Testing - generate test run command wizard](../../images/vulnerability-detection/sbt-generate-test-run-command-wizard.png) - -1. Copy the command. -1. Replace the placeholders with actual values. -1. You have the command. - -Note that: - -* If wizard is left with default values, it does not add anything additional to Docker command. -* **Export input data to the Cloud** if you use this, OAS or Postman collection file that you use locally to run tests, will be uploaded to **Test runs** for improved debugging if needed later. - -### Running combination - -You create test policy in Wallarm that will contains token, your schema and all testing parameters and provide you a Docker run command linked to it, but **in this linked** command you **override** any parameters, like use local file instead of the one attached to the policy. What is not overridden, is taken from the policy. - -See details on how to use the policy with local override in the Docker [`--help`](#available-flags). - -### Environment variables - -When you create and save a test policy in Wallarm, it automatically creates a Docker run command, adds all required `-e` variables and sets values for them. Generally, variables aim to tell Docker container to which account in Wallarm Cloud to connect and which test policy to take as the basis for testing of your application: - -Environment variable | Description| Required ---- | ---- | ---- -`WALLARM_API_HOST` | Wallarm API server:
  • `us1.api.wallarm.com` for the US Cloud
  • `api.wallarm.com` for the EU Cloud
  • `me1.api.wallarm.com` for the ME Cloud
| Yes -`WALLARM_API_TOKEN` | Wallarm's `Schema-Based Testing agent` API token. See details [here](#token). | Yes -`WALLARM_TESTING_POLICY_ID` | ID of [test policy](#test-policy-types) containing all testing configuration. | Yes -`FAIL_SEVERITY` | Defines [test run success criteria](#test-run-success-criteria) used when running tests as a part of a specific CI/CD pipeline. | No -`TARGET_URL` | For OAS-based run only. If set, overrides the target URL specified in test policy itself. | No -`AUTH_HEADER` | For OAS-based run only. If set, overrides the authentication data specified in test policy itself. | No - -### Available flags - -You can use flags to modify behavior of Wallarm's Schema-Based Testing running as Docker container. To see the full list of available flags, use: - -``` - --help -``` - -The outup will be something like: - -=== "OpenAPI specification (OAS)-based" - ``` - Run OpenAPI security testing workflow. - - This command performs automated API security testing using OpenAPI specifications, - executing comprehensive security tests against the target API and reporting vulnerabilities. - - OPERATING MODES: - - Standalone Mode: - Use local OpenAPI specification file without Wallarm platform integration. - Requires: --openapi-file and --target-url flags - - Example: - docker run -v "$(pwd)/api-spec.yaml:/app/api-spec.yaml" \ - -e WALLARM_API_HOST="https://api.wallarm.com" \ - -e WALLARM_API_TOKEN="..." \ - wallarm/security-testing:v0.11.0 openapi \ - --openapi-file /app/api-spec.yaml \ - --target-url https://api.example.com - - Cloud Mode: - Integrate with Wallarm platform using Test Policy. - Requires: WALLARM_TESTING_POLICY_ID environment variable or --testing-policy-id flag - - Example (with local file override): - docker run -v "$(pwd)/api-spec.yaml:/app/api-spec.yaml" \ - -e WALLARM_API_HOST="..." \ - -e WALLARM_API_TOKEN="..." \ - -e WALLARM_TESTING_POLICY_ID="..." \ - wallarm/security-testing:v0.11.0 openapi \ - --openapi-file /app/api-spec.yaml - - Example (uses specification from Test Policy): - docker run \ - -e WALLARM_API_HOST="..." \ - -e WALLARM_API_TOKEN="..." \ - -e WALLARM_TESTING_POLICY_ID="..." \ - wallarm/security-testing:v0.11.0 openapi - - Configuration priority: CLI flags > Environment variables > Test Policy > Defaults - - Usage: - security-testing openapi [flags] - - Flags: - --auth-header string Authentication header in 'Name: Value' format (can be provided by Test Policy) - -h, --help help for openapi - --openapi-file string Path to local OpenAPI specification file (JSON or YAML) - --target-url string Target application URL (can be provided by Test Policy) - --test-types stringSlice Comma-separated list of test types to run (environment_misconfiguration, sql_injection, command_injection, ssrf, xxe, open_redirect, xss, path_traversal, ssti, lfi_rfi, crlf_injection, infoleak, nosql_injection, rce, graphql_vulnerability_detection, auth_bypass, input_validation) - - Global Flags: - API: - --send-logs Send logs to Wallarm API (default true) - --testing-policy-id string Wallarm testing policy ID (optional, enables Test Policy integration) - --wallarm-api-host string Wallarm API host URL (required) - --wallarm-api-token string Wallarm API authentication token (required) - - GENERAL: - --export-files Upload local files (--openapi-file, --postman-collection-file, --postman-environment-files) to Wallarm cloud - --fail-severity string Set fail severity level (critical, high, medium, low, info) (default "high") - --insecure Turn off SSL verification checks, and allow self-signed SSL certificates - --test-run-title string Custom title for the test run - - PREFLIGHT-CHECKS: - --preflight-check-timeout int Timeout in seconds for target URL connectivity checks (minimum: 10) (default "75") - --skip-target-url-connectivity-check - Skip checking target URL connectivity - - REPORTS: - --no-summary Disable summary output to stdout - -r, --report Enable report generation - --report-format string Report format (json, csv, junit) (default "json") - --report-output string Report output file path (default extension matches format: .json for JSON, .csv for CSV) (default "report/results.json") - --report-pretty Pretty-print JSON output (default true) - - REQUEST-LOG: - --request-log-output string Output file path for HAR request log (default "./results/request_log.har") - --send-request-logs Send request logs to Wallarm API (default true) - - LOGGING: - -f, --log-format string Set log format (json, colored, text) (default "colored") - -l, --log-level string Set log level (debug, info, warn, error) (default "info") - - MTLS: - --mtls-ca-cert string Path to CA certificate for server validation (PEM format) - --mtls-client-cert string Path to client certificate file (PEM format) - --mtls-client-key string Path to client private key file (PEM format, unencrypted) - ``` -=== "Postman collection-based" - ``` - Run Postman security testing workflow. - - This command performs automated API security testing using Postman collections, - executing comprehensive security tests against the target API and reporting vulnerabilities. - - OPERATING MODES: - - Standalone Mode: - Use local Postman collection without Wallarm platform integration. - Requires: --postman-collection-file flag - - Example: - docker run -v "$(pwd)/collection.json:/app/collection.json" \ - -e WALLARM_API_HOST="https://api.wallarm.com" \ - -e WALLARM_API_TOKEN="..." \ - wallarm/security-testing:v0.11.0 postman \ - --postman-collection-file /app/collection.json - - Cloud Mode: - Integrate with Wallarm platform using Test Policy. - Requires: WALLARM_TESTING_POLICY_ID environment variable or --testing-policy-id flag - - Example (with local files override): - docker run -v "$(pwd)/collection.json:/app/collection.json" \ - -v "$(pwd)/env.json:/app/env.json" \ - -e WALLARM_API_HOST="..." \ - -e WALLARM_API_TOKEN="..." \ - -e WALLARM_TESTING_POLICY_ID="..." \ - wallarm/security-testing:v0.11.0 postman \ - --postman-collection-file /app/collection.json \ - --postman-environment-files /app/env.json - - Example (uses collection from Test Policy): - docker run \ - -e WALLARM_API_HOST="..." \ - -e WALLARM_API_TOKEN="..." \ - -e WALLARM_TESTING_POLICY_ID="..." \ - wallarm/security-testing:v0.11.0 postman - - Configuration priority: CLI flags > Environment variables > Test Policy > Defaults - - Usage: - security-testing postman [flags] - - Flags: - -h, --help help for postman - --postman-collection-file string Path to local Postman collection file (JSON) - --postman-environment-files stringSlice - Path(s) to Postman environment file(s) (JSON, supports multiple files) - - Global Flags: - API: - --send-logs Send logs to Wallarm API (default true) - --testing-policy-id string Wallarm testing policy ID (optional, enables Test Policy integration) - --wallarm-api-host string Wallarm API host URL (required) - --wallarm-api-token string Wallarm API authentication token (required) - - GENERAL: - --export-files Upload local files (--openapi-file, --postman-collection-file, --postman-environment-files) to Wallarm cloud - --fail-severity string Set fail severity level (critical, high, medium, low, info) (default "high") - --insecure Turn off SSL verification checks, and allow self-signed SSL certificates - --test-run-title string Custom title for the test run - - PREFLIGHT-CHECKS: - --preflight-check-timeout int Timeout in seconds for target URL connectivity checks (minimum: 10) (default "75") - --skip-target-url-connectivity-check - Skip checking target URL connectivity - - REPORTS: - --no-summary Disable summary output to stdout - -r, --report Enable report generation - --report-format string Report format (json, csv, junit) (default "json") - --report-output string Report output file path (default extension matches format: .json for JSON, .csv for CSV) (default "report/results.json") - --report-pretty Pretty-print JSON output (default true) - - REQUEST-LOG: - --request-log-output string Output file path for HAR request log (default "./results/request_log.har") - --send-request-logs Send request logs to Wallarm API (default true) - - LOGGING: - -f, --log-format string Set log format (json, colored, text) (default "colored") - -l, --log-level string Set log level (debug, info, warn, error) (default "info") - - MTLS: - --mtls-ca-cert string Path to CA certificate for server validation (PEM format) - --mtls-client-cert string Path to client certificate file (PEM format) - --mtls-client-key string Path to client private key file (PEM format, unencrypted) - ``` - -See the explanation for the `-fail-severity` flag in the next section. - -### Test run success criteria - -If your include Wallarm's Schema-Based test runs into your CI/CD pipeline, it is important to define the criteria of test run success. You can do that in the test policy details by setting -the **Fail on a risk level** to one of the values: - -* critical -* high (default) -* medium -* low -* info - -Selecting any value different from default adds the `FAIL_SEVERITY` environment variable to the Docker run command. - -![Schema-Based Testing - defining test run success criteria](../../images/vulnerability-detection/sbt-test-run-success-criteria-setup.png) - -Example: - -``` -docker run -e WALLARM_API_HOST="us1.api.wallarm.com" -e WALLARM_API_TOKEN="" -e WALLARM_TESTING_POLICY_ID="" -e FAIL_SEVERITY="medium" --network="host" wallarm/security-testing:0.11.0 -``` - - -You can also set test run success criteria with the `-fail-severity` flag, for example: - -``` -docker run -e WALLARM_API_HOST="us1.api.wallarm.com" -e WALLARM_API_TOKEN="" -e WALLARM_TESTING_POLICY_ID="" --network="host" wallarm/security-testing:0.11.0 --fail-severity medium -``` - -...will lead to the following: if testing finds at least 1 medium (or high or critical) security issue, the test run will exit with code 1 ("test failed"). - -### Report generation - -Schema-Based Testing can generate reports in multiple formats for integration with various tools and CI/CD pipelines. - -To enable report generation, use the `--report` flag along with `--report-format` and `--report-output`: - -``` -docker run -v "$(pwd)/reports:/app/report" \ - -e WALLARM_API_HOST="us1.api.wallarm.com" \ - -e WALLARM_API_TOKEN="" \ - -e WALLARM_TESTING_POLICY_ID="" \ - --network="host" wallarm/security-testing:0.11.0 openapi \ - --report --report-format json --report-output /app/report/results.json -``` - -Available report formats: - -| Format | Description | Use case | -| ------ | ----------- | -------- | -| `json` | JSON format with full vulnerability details | Custom integrations, detailed analysis | -| `csv` | Comma-separated values | Spreadsheet analysis, data processing | -| `junit` | JUnit XML format | CI/CD pipeline integration (Jenkins, GitLab CI, GitHub Actions, etc.) | - -#### JUnit reports for CI/CD - -JUnit format is particularly useful for CI/CD integration as most CI/CD platforms natively support JUnit test reports for visualizing test results. - -Example for CI/CD pipeline: - -``` -docker run -v "$(pwd)/reports:/app/report" \ - -e WALLARM_API_HOST="us1.api.wallarm.com" \ - -e WALLARM_API_TOKEN="" \ - -e WALLARM_TESTING_POLICY_ID="" \ - --network="host" wallarm/security-testing:0.11.0 openapi \ - --report --report-format junit --report-output /app/report/security-tests.xml \ - --fail-severity high -``` - -This generates a JUnit XML report that can be consumed by CI/CD tools to display test results and track security issues over time. - -### Request/response log export - -Schema-Based Testing can export full HTTP request and response logs in HAR (HTTP Archive) format. This is useful for debugging, compliance, and detailed analysis of the security tests performed. - -By default, request logs are sent to the Wallarm API. You can also export them locally: - -``` -docker run -v "$(pwd)/logs:/app/results" \ - -e WALLARM_API_HOST="us1.api.wallarm.com" \ - -e WALLARM_API_TOKEN="" \ - -e WALLARM_TESTING_POLICY_ID="" \ - --network="host" wallarm/security-testing:0.11.0 openapi \ - --request-log-output /app/results/request_log.har -``` - -Request log flags: - -| Flag | Description | Default | -| ---- | ----------- | ------- | -| `--request-log-output` | Output file path for HAR request log | `./results/request_log.har` | -| `--send-request-logs` | Send request logs to Wallarm API | `true` | - -The HAR format is widely supported by browser developer tools and API debugging applications, making it easy to analyze the exact requests and responses generated during testing. - -### mTLS support - -Schema-Based Testing supports mutual TLS (mTLS) authentication for testing APIs that require client certificate authentication. - -To use mTLS, provide the client certificate, private key, and optionally the CA certificate: - -``` -docker run -v "$(pwd)/certs:/app/certs" \ - -e WALLARM_API_HOST="us1.api.wallarm.com" \ - -e WALLARM_API_TOKEN="" \ - -e WALLARM_TESTING_POLICY_ID="" \ - --network="host" wallarm/security-testing:0.11.0 openapi \ - --mtls-client-cert /app/certs/client.crt \ - --mtls-client-key /app/certs/client.key \ - --mtls-ca-cert /app/certs/ca.crt -``` - -mTLS flags: - -| Flag | Description | Format | -| ---- | ----------- | ------ | -| `--mtls-client-cert` | Path to client certificate file | PEM format | -| `--mtls-client-key` | Path to client private key file | PEM format, unencrypted | -| `--mtls-ca-cert` | Path to CA certificate for server validation | PEM format | - -!!! note "Self-signed certificates" - If your target API uses self-signed certificates without mTLS, you can use the `--insecure` flag to skip SSL verification checks. +A typical run prints pre-flight checks, the workflow progress, and a final security-issues summary. For CI/CD integration, environment-variable reference, report generation, mTLS, and override patterns, see [Docker reference](docker-reference.md). -## Deleting policies +## Step 4. See results -You can delete a test policy. If you do so: +When the run finishes, the **Test runs** tab shows it with status, request count, credits consumed, and the number of detected security issues. Open the run to inspect health checks, the pipeline stages, and the per-strategy hypotheses with their verdicts. Confirmed findings are also promoted to **Security Issues** with `Discovered by = Schema-Based Testing`. -* Information on previous test runs will remain untouched -* You will not be able to run Docker's command based on the deleted policy -* If policy's Docker containers are running, they will continue to do so and the testing will continue -* When policy's Docker containers stop, you will not be able to re-run them \ No newline at end of file +See [Exploring test run results](explore.md) for a walkthrough. diff --git a/docs/latest/vulnerability-detection/schema-based-testing/strategies.md b/docs/latest/vulnerability-detection/schema-based-testing/strategies.md new file mode 100644 index 0000000000..4e74779205 --- /dev/null +++ b/docs/latest/vulnerability-detection/schema-based-testing/strategies.md @@ -0,0 +1,61 @@ +# Schema-Based Testing Strategies + +A **strategy** is a reusable scanning recipe used by [Schema-Based Testing](overview.md) during Active Scan or Passive Scan. Each strategy targets one vulnerability class and contains the detection logic the engine applies during a run. The set of strategies enabled in a test policy determines which vulnerability classes are checked. + +Strategies fall into two categories: + +* **Default strategies** — provided by Wallarm, kept up to date with new attack patterns, and available to all accounts that have Schema-Based Testing enabled. They cannot be edited, only viewed and toggled per policy. +* **Custom strategies** — created by you to extend the engine with checks specific to your application or threat model. A custom strategy is bound to your account and can be enabled in any of your test policies. + +Each strategy is either of type **Active** (sends attack requests during a run) or **Passive** (analyzes captured traffic only). The strategy's type determines which [scan mode](overview.md#scan-modes) it participates in. + +## Default passive strategies + +| Strategy | Purpose | +|---|---| +| **Data Exposure** | Detects unauthorized exposure of sensitive user data, authentication secrets, and third-party credentials in API responses. | +| **Security Headers Misconfiguration** | Flags missing browser security headers on rendered content and improper cache-control directives on responses carrying sensitive data. | +| **Secrets Management** | Detects credential-grade secrets transmitted over insecure channels and session cookies set without proper security attributes. | +| **Service Internals** | Detects exposure of internal server details through debug output, stack traces, version fingerprinting, and unintended static file serving. | +| **Web Vulnerabilities** | Identifies common web application vulnerabilities through pattern analysis, including open redirects, SSRF, injection syntax, and cross-site scripting markup in request inputs. | + +## Default active strategies + +| Strategy | Purpose | +|---|---| +| **BOLA (Broken Object Level Authorization)** | Checks whether one user can access or modify another user's resources by reusing object identifiers with their own credentials. | +| **BFLA (Broken Function Level Authorization)** | Verifies whether principals can invoke functions, workflow steps, or management actions reserved for higher-privilege roles or outside their tenant. | +| **Business Logic Abuse** | Detects flaws that allow attackers to gain unauthorized value, bypass workflow constraints, or manipulate quantities, states, and application-specific invariants. | +| **Excessive Data Exposure** | Identifies API responses that return sensitive, internal, or unnecessary fields beyond what the client needs. | +| **JWT Authentication Flaws** | Tests JSON Web Token implementations for algorithm manipulation, signature bypass, claim tampering, weak secret brute-force, missing expiration or audience/issuer validation, and replay of non-revocable tokens. | +| **Mass Assignment** | Detects APIs that bind all client-supplied fields directly to internal models, allowing unauthorized modification of restricted attributes (for example, roles, status, pricing). | +| **NoSQL Injection** | Sends operator-based payloads into query-driving fields to bypass authentication, expand result sets, or perform unauthorized data modifications. | +| **Resource Exhaustion** | Detects missing rate limits, brute-force exposure, and client-controlled amplification that lets attackers degrade availability or scale server-side work via unbounded inputs. | +| **SQL Injection** | Targets query-driving parameters with SQL syntax payloads to detect authentication bypass, unauthorized data access, data modification, and schema disclosure. | +| **SSRF (Server-Side Request Forgery)** | Identifies endpoints where user-controlled input can influence backend outbound network requests to attacker-controlled or internal destinations. | +| **Unauthenticated Access** | Detects endpoints that allow reading, modifying, or deleting business resources without requiring authentication, including cases where predictable selectors make objects directly addressable. | + +## Browsing strategies + +Strategies are managed on the **Security Testing** → **Schema-Based** → **Strategies** tab. Use the **All / Active / Passive** filter to narrow the list. Default strategies are read-only — open one to inspect its description and the vulnerability classes it covers. Custom strategies can be edited and deleted. + + + +## Creating a custom strategy + +To extend the engine with checks specific to your application or threat model: + +1. Go to **Security Testing** → **Schema-Based** → **Strategies** and click **Create strategy**. +1. Provide a **Name**. The slug is generated automatically; it must be lowercase, may contain digits and underscores, and must not collide with a reserved or already-existing slug. +1. Choose the strategy type: + + * **Passive** — analyzes captured traffic without sending additional requests. Provide the detection logic in the **Content** field. + * **Active** — sends targeted requests during the run. Provide the search prompt in the **Search content** field; the engine uses it to derive hypotheses to test against the application. + +1. Save the strategy. It becomes available immediately on the Strategies tab and can be enabled in any new or existing test policy. + +## Enabling strategies in a policy + +Open a test policy → **Working Modes** section. For each mode (Active Scan, Passive Scan), toggle the mode on, click **Advanced settings**, and select which strategies participate. Disabling a mode disables all of its strategies for that policy. + +By default, every strategy of the corresponding type is enabled when you turn on a scan mode. Narrow the selection if you want to focus a policy on a specific class of vulnerabilities, or if a particular check is noisy on your application. diff --git a/docs/latest/vulnerability-detection/security-testing-overview.md b/docs/latest/vulnerability-detection/security-testing-overview.md index c7f3bb9898..79bba777dd 100644 --- a/docs/latest/vulnerability-detection/security-testing-overview.md +++ b/docs/latest/vulnerability-detection/security-testing-overview.md @@ -4,7 +4,7 @@ Wallarm's Security Testing suite is a comprehensive platform designed to secure * **Threat Replay Testing (TRT)** takes a unique approach by converting real-world attack data into a continuous stream of security tests. Instead of relying on synthetic attacks, this tool automatically analyzes actual attack attempts targeting your production APIs, sanitizes the malicious payloads, and then "replays" them in a safe, non-production environment. This allows organizations to discover vulnerabilities exposed by active threats, ensuring their defenses are continuously updated against the latest attack patterns. -* **Schema-Based Testing** is a dynamic application security testing (DAST) solution that enables "shift-left" security. It uses an API's schema (such as an OpenAPI specification or a Postman collection) as a blueprint to automatically generate and execute targeted security tests. By integrating into CI/CD pipelines, Schema-Based Testing allows development teams to proactively identify a wide range of vulnerabilities—including OWASP API Top 10 risks, business logic flaws, and input validation issues—early in the development process, making them easier and cheaper to fix. +* **Schema-Based Testing** is a dynamic application security testing (DAST) solution that enables "shift-left" security. It takes a Postman collection — your existing functional tests — as a blueprint and runs an AI-driven scanner that analyzes the application context, generates vulnerability hypotheses, and validates each finding with a proof-of-exploit test. By integrating into CI/CD pipelines, Schema-Based Testing helps development teams proactively identify OWASP API Top 10 risks, business logic flaws, and input validation issues — early in the development process, making them easier and cheaper to fix. ## Threat Replay Testing @@ -32,20 +32,18 @@ Wallarm's [Schema-Based Testing](./schema-based-testing/overview.md) introduces **Key highlights** -Expanded vulnerability coverage: +Vulnerability coverage: * OWASP API Top 10 risks -* Business Logic flaws (BOLA, BFLA) -* Input validation issues (Injections, RCE, Path Traversal) -* Environment misconfigurations -* GraphQL misconfigurations +* Business logic flaws (BOLA, BFLA, mass assignment, business logic abuse) +* Input validation issues (injections, RCE, path traversal) +* Traffic-observable misconfigurations (security headers, secrets in transit, debug output) -Supported inputs: +Test basis: -* OpenAPI specifications -* Postman collections (for advanced testing of business logic scenarios and access control violations) +* Postman collection — your functional tests drive realistic, multi-request attack scenarios with authenticated users. -![Schema-Based Testing - test runs](../images/vulnerability-detection/sbt-test-runs.png) +![Schema-Based Testing - test runs](../images/vulnerability-detection/sbt-test-runs-row.png) Schema-Based Testing runs on a lightweight Docker-based agent, ensuring fast and isolated execution. It supports both one-time scans for quick assessments and continuous testing integrated into CI/CD pipelines, making it flexible for different stages of the development lifecycle. diff --git a/images/vulnerability-detection/sbt-docker-container-output.png b/images/vulnerability-detection/sbt-docker-container-output.png new file mode 100644 index 0000000000..1632d7ad66 Binary files /dev/null and b/images/vulnerability-detection/sbt-docker-container-output.png differ diff --git a/images/vulnerability-detection/sbt-hypothesis-detail.png b/images/vulnerability-detection/sbt-hypothesis-detail.png new file mode 100644 index 0000000000..f4b556cf2e Binary files /dev/null and b/images/vulnerability-detection/sbt-hypothesis-detail.png differ diff --git a/images/vulnerability-detection/sbt-policies-list.png b/images/vulnerability-detection/sbt-policies-list.png new file mode 100644 index 0000000000..262d1b31f2 Binary files /dev/null and b/images/vulnerability-detection/sbt-policies-list.png differ diff --git a/images/vulnerability-detection/sbt-policy-create-postman-new.png b/images/vulnerability-detection/sbt-policy-create-postman-new.png new file mode 100644 index 0000000000..804fab120b Binary files /dev/null and b/images/vulnerability-detection/sbt-policy-create-postman-new.png differ diff --git a/images/vulnerability-detection/sbt-policy-docker-command-new.png b/images/vulnerability-detection/sbt-policy-docker-command-new.png new file mode 100644 index 0000000000..9e8858f585 Binary files /dev/null and b/images/vulnerability-detection/sbt-policy-docker-command-new.png differ diff --git a/images/vulnerability-detection/sbt-security-issues-by-run.png b/images/vulnerability-detection/sbt-security-issues-by-run.png new file mode 100644 index 0000000000..81c1a2e034 Binary files /dev/null and b/images/vulnerability-detection/sbt-security-issues-by-run.png differ diff --git a/images/vulnerability-detection/sbt-test-run-details.png b/images/vulnerability-detection/sbt-test-run-details.png new file mode 100644 index 0000000000..9a66159340 Binary files /dev/null and b/images/vulnerability-detection/sbt-test-run-details.png differ diff --git a/images/vulnerability-detection/sbt-test-runs-row.png b/images/vulnerability-detection/sbt-test-runs-row.png new file mode 100644 index 0000000000..8c36f24c14 Binary files /dev/null and b/images/vulnerability-detection/sbt-test-runs-row.png differ diff --git a/mkdocs-6.x.yml b/mkdocs-6.x.yml index 50b6d7f686..282863102f 100644 --- a/mkdocs-6.x.yml +++ b/mkdocs-6.x.yml @@ -466,6 +466,8 @@ nav: - Schema-Based Testing: - Overview: vulnerability-detection/schema-based-testing/overview.md - Setup: vulnerability-detection/schema-based-testing/setup.md + - Strategies: vulnerability-detection/schema-based-testing/strategies.md + - Docker reference: vulnerability-detection/schema-based-testing/docker-reference.md - Exploring Results: vulnerability-detection/schema-based-testing/explore.md - API Security Testing via Postman: diff --git a/mkdocs-7.x.yml b/mkdocs-7.x.yml index 01ba621fee..c495131b53 100644 --- a/mkdocs-7.x.yml +++ b/mkdocs-7.x.yml @@ -468,6 +468,8 @@ nav: - Schema-Based Testing: - Overview: vulnerability-detection/schema-based-testing/overview.md - Setup: vulnerability-detection/schema-based-testing/setup.md + - Strategies: vulnerability-detection/schema-based-testing/strategies.md + - Docker reference: vulnerability-detection/schema-based-testing/docker-reference.md - Exploring Results: vulnerability-detection/schema-based-testing/explore.md - API Security Testing via Postman: