diff --git a/AGENTS.md b/AGENTS.md index 64279977619..ed261984c8a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -54,21 +54,26 @@ Modules in various stages of reorganization: |--------|-----------|-------------|-----|------|--------| | **url** | In module | N/A | Done | Done | **Complete** | | **location** | In module | N/A | N/A | Done | **Complete** | -| **product_type** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work | -| **test** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work | -| **engagement** | In dojo/models.py | Partial (32 lines) | Partial (views at root) | In dojo/api_v2/ | Needs work | -| **product** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work | -| **finding** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work | +| **product_type** | In module | N/A | Done | Done | **Complete** (#14970) | +| **test** | In module | N/A | Done | Done | **Complete** (#14971) | +| **engagement** | In module | In module | Done | Done | **Complete** (#14972) | +| **product** | In module | N/A | Done | Done | **Complete** (#14973) | +| **finding** | In module | N/A (helper.py) | Done | Done | **Complete** (#14974, incl. CWE + BurpRawRequestResponse) | +| **user / system_settings** | In module | N/A | Done | Done | **Complete** (#14981) | +| **endpoint / tool_type / tool_config / tool_product** | In module | N/A | Done | Done | **Complete** (#14982) | +| **survey / benchmark** | In module | N/A | Done | N/A (no API) | **Complete** (#14983) | +| **notes / note_type / file_uploads / reports / risk_acceptance** | In module | N/A | Done | Done | **Complete** (#14986) | +| **regulations / banner / announcement / development_environment / object** | In module | N/A | Done | Partial (API where one exists) | **Complete** (#14987) | ### Monolithic Files Being Decomposed -These files still contain code for multiple modules. Extract code to the target module's subdirectory and leave a re-export stub. +These files still contain code for multiple modules. Extract code to the target module's subdirectory and leave a re-export stub. (Counts reflect the completed Phase 10 stack; they shrink as branches merge to `dev`.) -- `dojo/models.py` (4,973 lines) — All model definitions -- `dojo/forms.py` (4,127 lines) — All Django forms -- `dojo/filters.py` (4,016 lines) — All UI and API filter classes -- `dojo/api_v2/serializers.py` (3,387 lines) — All DRF serializers -- `dojo/api_v2/views.py` (3,519 lines) — All API viewsets +- `dojo/models.py` (~645 lines) — re-export hub + the few models intentionally left here (`DojoMeta`, `Network_Locations`, `Sonarqube_Issue`/`Sonarqube_Issue_Transition`, `Check_List`, `Testing_Guide_Category`/`Testing_Guide`, `Language_Type`/`Languages`, `App_Analysis`, `SLA_Configuration`) plus shared utilities (`copy_model_util`, `get_current_date`, `tomorrow`, `UniqueUploadNameProvider`) +- `dojo/forms.py` (~914 lines) — remaining/shared Django forms +- `dojo/filters.py` (~1,376 lines) — remaining/shared filter classes + shared bases +- `dojo/api_v2/serializers.py` (~1,101 lines) — remaining serializers + re-exports for prefetcher discovery +- `dojo/api_v2/views.py` (~901 lines) — remaining viewsets + shared base classes/helpers --- @@ -117,21 +122,35 @@ grep -rn "from dojo.models import.*{Model}" dojo/ unittests/ 5. Remove original model code (keep re-export line) **Import rules for models.py:** -- Upward FKs (e.g., Test -> Engagement): import from `dojo.models` if not yet extracted, or `dojo.{module}.models` if already extracted -- Downward references (e.g., Product_Type querying Finding): use lazy imports inside method bodies -- Shared utilities (`copy_model_util`, `_manage_inherited_tags`, `get_current_date`, etc.): import from `dojo.models` +- **Prefer string FK refs to break circular imports.** Convert EVERY ForeignKey/ManyToMany/OneToOne whose target is NOT a class being moved into a string ref `"dojo."` (e.g. `models.ForeignKey(Engagement, ...)` → `models.ForeignKey("dojo.Engagement", ...)`). This lets the extracted `models.py` carry ZERO top-level `from dojo.models import ...`, which is what actually prevents circular imports. String refs produce identical migrations (Django resolves via the app registry) — `makemigrations --check` must still say "No changes detected". +- References AMONG the classes being moved together also use string refs, for uniformity and to avoid in-file ordering issues. +- Downward/other dojo references inside METHOD bodies: lazy imports inside the method. +- Shared utilities (`copy_model_util`, `_manage_inherited_tags`, `get_current_date`, `tomorrow`, etc.): import from `dojo.models`. CAVEAT: if a utility is used as a class-body field default (e.g. `default=get_current_date`), it must be imported (not redefined locally) so its `__module__` stays `dojo.models` — otherwise migration serialization changes and `makemigrations` flags a diff. These utils are defined early in `dojo.models` (before the re-export that loads your module), so a top-level `from dojo.models import get_current_date, tomorrow, copy_model_util` resolves correctly despite the partial circular load. - Do NOT set `app_label` in Meta — all models inherit `dojo` app_label automatically -**Verify:** +**Lint conventions (the repo pre-commit ruff is strict — match exactly):** +- Method-body lazy imports need `# noqa: PLC0415 -- lazy import, avoids circular dependency`. +- Mid-file / non-top re-exports in `dojo/models.py` need `# noqa: E402`, plus `# noqa: F401` ONLY on names not referenced elsewhere in `dojo/models.py` (a name still used by a remaining class body must NOT get F401). +- Self-check before committing: `/home/valentijn/.local/bin/ruff check --config ruff.toml ` (ruff is a host binary, NOT in the uwsgi container). Never let `ruff --fix` wrap a re-export into a parenthesized multiline — shorten the comment instead. + +**Re-export placement:** use ONE consolidated re-export block per module, placed at the earliest moved class's original position. A name referenced in a class-body FK at load-time must be re-exported BEFORE that line. + +**Constants:** single-source module-level constants in the extracted module and re-export from `dojo/models.py` (done for `IMPORT_ACTIONS`, `ENGAGEMENT_STATUS_CHOICES`). Do not duplicate. + +**Watch for load-bearing imports:** some imports in `dojo/models.py` exist for side effects, not the imported name (e.g. `from dojo.utils import parse_cvss_data` transitively registers `dojo.location` models for `apps.py:ready()`). If you remove the last consumer of such an import, keep it as a re-export or `apps.py` breaks. + +**Verify** (runs in docker; model imports need `manage.py shell -c`, not bare `python -c`): ```bash -python manage.py check -python manage.py makemigrations --check -python -c "from dojo.{module}.models import {Model}" -python -c "from dojo.models import {Model}" +docker compose exec -T uwsgi python manage.py check +docker compose exec -T uwsgi python manage.py makemigrations --check --dry-run # must say "No changes detected" +docker compose exec -T uwsgi python manage.py shell -c "from dojo.{module}.models import {Model}; print('ok')" +docker compose exec -T uwsgi python manage.py shell -c "from dojo.models import {Model}; print('ok')" ``` ### Phase 2: Extract Services +**This phase is conditional.** If the module's views are pure CRUD (form save/delete, simple field add/remove) with none of the "belongs in services" items below, there is NO `services.py` — skip the phase (the `url`/`location` reference modules have none). Don't invent a service just to have one. + Create `dojo/{module}/services.py` with business logic extracted from UI views. **What belongs in services.py:** @@ -173,8 +192,14 @@ Update UI views and API viewsets to call the service instead of containing logic ### Phase 4: Extract UI Filters to `ui/filters.py` 1. Create `dojo/{module}/ui/filters.py` — move module-specific filters from `dojo/filters.py` -2. Shared base classes (`DojoFilter`, `DateRangeFilter`, `ReportBooleanFilter`) stay in `dojo/filters.py` -3. Add re-exports in `dojo/filters.py` +2. Shared base classes (`DojoFilter`, `DateRangeFilter`, `ReportBooleanFilter`) stay in `dojo/filters.py`. **Keep the original base class** (`class XFilter(DojoFilter)`) — do NOT switch to `FilterSet` to dodge an import. +3. **Circular-import caveat**: a re-export in `dojo/filters.py` (`from dojo.{module}.ui.filters import XFilter`) while `ui/filters.py` imports `DojoFilter` back from `dojo.filters` creates a real cycle (fails when `ui/filters.py` loads first). Resolve per the re-export rule below — usually: **drop the `dojo/filters.py` re-export** when the filter's only consumer is the module's own view, and import the filter directly from `dojo.{module}.ui.filters` in that view (matches the `url` module). + +> **Re-export decisions (Phases 3,4,6,8) — decide per symbol, by actual remaining consumers:** +> - `grep -rn` the symbol across `dojo/` and `unittests/` first. Account for multi-line `from x import (\n ...\n)` blocks — a one-line grep misses them. +> - If a symbol is still referenced by code that REMAINS in the monolith (e.g. `ProductTypeSerializer` used by `ReportGenerateSerializer` in `api_v2/serializers.py`) → **keep** the re-export (`# noqa: E402` + `F401` as needed). +> - If the ONLY consumers are code you are moving/updating anyway (the module's own views/tests) → **omit** the re-export and point those consumers at the new path. This is required when a re-export would cycle (filter↔`dojo.filters`, `api_v2.views`↔`{module}.api.views`). +> - After dropping any re-export, run the module's real unit tests (not just `manage.py check`) — `check` won't catch a broken import in a test module. ### Phase 5: Move UI Views/URLs into `ui/` @@ -189,7 +214,13 @@ Update UI views and API viewsets to call the service instead of containing logic 1. Create `dojo/{module}/api/__init__.py` with `path = "{module}"` 2. Create `dojo/{module}/api/serializer.py` — move from `dojo/api_v2/serializers.py` -3. Add re-exports in `dojo/api_v2/serializers.py` +3. Re-export ONLY the serializers still referenced by code REMAINING in `api_v2/serializers.py` (e.g. one nested by `ReportGenerateSerializer` / used in a `RiskAcceptance` representation). Serializers consumed only by the viewset are imported by their new path in Phase 8, so omit those re-exports. + + **EXCEPTION — prefetcher discovery (re-export the FULL moved ModelSerializer set):** `dojo/api_v2/prefetch/prefetcher.py` builds its model→serializer map via `inspect.getmembers(sys.modules["dojo.api_v2.serializers"], ...)`. Any moved `ModelSerializer` that drops out of `api_v2/serializers.py`'s module members disappears from that map, so prefetch breaks (e.g. `test_detail_prefetch` / `test_list_prefetch` fail with `'' not found`) — and `manage.py check` does NOT catch it; only the `test_rest_framework` prefetch tests do. So re-export the ENTIRE set of moved `ModelSerializer`s (not just the ReportGenerate-nested ones), even nested/sub serializers with no other consumer. This re-export block is byte-identical module membership → zero behavior change. (Pure `serializers.Serializer` subclasses that aren't tied to a model and aren't referenced elsewhere can still be omitted.) This bit the finding module (18 serializers); revisit earlier modules if their prefetch tests ever regress. + +**Cycle-break for serializers that reference api_v2 serializers** (matches `dojo/test/api/serializer.py`, `dojo/engagement/api/serializer.py`): a moved serializer cannot import `NoteSerializer`/`FileSerializer`/`TagListSerializerField` etc. from `dojo.api_v2.serializers` at module level — that cycles once `api_v2/serializers.py` re-imports your serializer. Convert class-body field assignments (`tags = TagListSerializerField(...)`, `notes = NoteSerializer(many=True)`) into a lazy `get_fields()` override that imports inside the method (`# noqa: PLC0415`); `build_relational_field` lazy-imports the same way. The extracted module then carries ZERO top-level `dojo.api_v2.serializers` import. + +**`@extend_schema_field` decorators referencing api_v2 serializers also cycle** (their argument is evaluated eagerly at class-body load). A class-body `@extend_schema_field(RiskAcceptanceSerializer)` / `@extend_schema_field(BurpRawRequestResponseSerializer)` cannot stay. Drop the decorator and reapply the override at the bottom of the module via `drf_spectacular.utils.set_override(Cls.method, "field", LazyImportedSerializer)` inside a small `_apply_schema_overrides()` that lazy-imports the api_v2 serializer (`# noqa: PLC0415`). This preserves the generated schema with no top-level api_v2 reference. (Decorators whose argument is one of the MOVED serializers in the same file are fine as-is.) ### Phase 7: Extract API Filters to `api/filters.py` @@ -199,7 +230,9 @@ Update UI views and API viewsets to call the service instead of containing logic ### Phase 8: Extract API ViewSets to `api/views.py` 1. Create `dojo/{module}/api/views.py` — move from `dojo/api_v2/views.py` -2. Add re-exports in `dojo/api_v2/views.py` +2. Do NOT re-export the viewset in `dojo/api_v2/views.py` — it would cycle (`api_v2.views` ↔ `{module}.api.views`, because the viewset imports its base classes back from `api_v2.views`). Update the consumers instead: the `dojo/urls.py` registration (Phase 9) and `unittests/test_rest_framework.py`, which imports viewsets by name (a dropped re-export there is an ImportError that `manage.py check` won't catch — only the test run does). + +**Viewset import pattern (matches `dojo/test/api/views.py`, `dojo/engagement/api/views.py`):** `from dojo.api_v2.views import DojoModelViewSet, PrefetchDojoModelViewSet, report_generate, schema_with_prefetch` — base classes and helpers stay in the monolith. Requalify every `serializers.X` reference that stays in `api_v2` to `api_v2_serializers.X` via `from dojo.api_v2 import serializers as api_v2_serializers`; import the MOVED serializers by name from `dojo.{module}.api.serializer`. PRESERVE active class decorators such as `@extend_schema_view(**schema_with_prefetch())` — they are easy to drop when copying a viewset and silently change the generated schema. After moving, prune the now-unused engagement-specific imports left behind in `api_v2/views.py` (filter, services, queries, models) — ruff flags them. ### Phase 9: Extract API URL Registration @@ -214,12 +247,17 @@ Update UI views and API viewsets to call the service instead of containing logic ``` 2. Update `dojo/urls.py` — replace `v2_api.register(...)` with `add_{module}_urls(v2_api)` +**Preserve the exact route and basename** from the original `v2_api.register(...)` call. They often differ (e.g. route `product_types`, `basename="product_type"`); `path` in `api/__init__.py` should be the route string, and pass `basename=` explicitly if the original did. Changing either breaks DRF URL reversing and the API tests. Verify with `reverse('{basename}-list')`. + ### After Each Phase: Verify +**When copying a class/function out, capture through to the next top-level `class`/dedent.** A fixed-line-window read can silently truncate a long class (trailing fields + `Meta` + `__init__`), yielding a partial copy that still imports cleanly but drops behavior. Confirm the last line of the source class before deleting it from the monolith. + ```bash -python manage.py check -python manage.py makemigrations --check -python -m pytest unittests/ -x --timeout=120 +docker compose exec -T uwsgi python manage.py check +docker compose exec -T uwsgi python manage.py makemigrations --check --dry-run +# Tests run via the wrapper (NOT pytest/manage.py test directly); tee to capture output: +./run-unittest.sh --test-case unittests.{relevant_test_module} 2>&1 | tee /tmp/test.log ``` --- @@ -247,3 +285,70 @@ def critical_present(self): - **Signal registration**: Handled in `dojo/apps.py` via `import dojo.{module}.signals`. Already set up for test, engagement, product, product_type. - **Watson search**: Uses `self.get_model("Product")` in `apps.py` — works via Django's model registry regardless of file location. - **Admin registration**: Currently at the bottom of `dojo/models.py` (lines 4888-4973). Must be moved to `{module}/admin.py` and removed from `dojo/models.py` to avoid `AlreadyRegistered` errors. + +--- + +## Phase 10: Peripheral Model Modules — 10-PR Stack Continuation + +> **This section is the complete, self-contained brief for a fresh agent session (auto mode) to finish the reorganization.** The 5 core hierarchy modules (`product_type`, `test`, `engagement`, `product`, `finding`) are DONE — they are the templates. What remains is moving the ~45 *peripheral* model classes still defined in `dojo/models.py` into their domain modules, each as a **full vertical slice** (all 9 phases), reusing the playbook above. + +### Goal & scope + +`dojo/models.py` is now ~2,254 lines and still **defines** these peripheral model classes. Move each into its module (most module dirs already exist with `views.py`/`urls.py`/helpers but NO `models.py`/`admin.py` — only `dojo/url/` and `dojo/location/` are complete-with-models templates). Leave backward-compat re-exports in every monolith (`dojo/models.py`, `forms.py`, `filters.py`, `api_v2/serializers.py`, `api_v2/views.py`) per the rules above. + +**Decisions already locked with the user (do NOT relitigate):** +- **Full vertical slice per module** (Phases 1–9), not models-only. Skip a phase only when the module genuinely has no code for it (e.g. no API serializer/viewset exists → no `api/` layer; no module-specific form → no `ui/forms.py`). Follow the "Phase 2 is conditional" / re-export-by-actual-consumer rules above. +- **These models STAY in `dojo/models.py`** (no module worth creating — do NOT extract): `DojoMeta`, `Network_Locations`, `Sonarqube_Issue`, `Sonarqube_Issue_Transition`, `Check_List`, `Testing_Guide_Category`, `Testing_Guide`, `Language_Type`, `Languages`, `App_Analysis`. Leave them untouched. +- **`CWE` + `BurpRawRequestResponse` fold into `finding`** (they are finding-domain), and are done FIRST on the EXISTING finding PR (#14974), not a new PR. + +### The 10-PR stack + +The 5 core PRs already exist (stacked, merge bottom-up): `dev ← #14970 product_type ← #14971 test ← #14972 engagement ← #14973 product ← #14974 finding`. **The new work CONTINUES this stack on top of #14974.** All branches and PRs follow the same conventions as the existing 5. + +| PR | Branch (head) | Base | Contents | +|----|---------------|------|----------| +| 1–5 | existing | existing | DONE: product_type, test, engagement, product, finding | +| **5 (#14974)** | `reorg/finding-models` | `reorg/product-models` | **ADD `CWE` + `BurpRawRequestResponse` to `dojo/finding/`** (full slice). Existing PR — do NOT create a new one. | +| **6** | `reorg/peripheral-user` | `reorg/finding-models` | **Bundle A**: `user` (`Dojo_User`, `UserContactInfo`, `Contact`) + `system_settings` (`System_Settings`) | +| **7** | `reorg/peripheral-tools-endpoint` | `reorg/peripheral-user` | **Bundle B**: `endpoint` (`Endpoint_Params`, `Endpoint_Status`, `Endpoint`) + `tool_type` (`Tool_Type`) + `tool_config` (`Tool_Configuration`, + admin classes `ToolConfigForm_Admin`/`Tool_Configuration_Admin`) + `tool_product` (`Tool_Product_Settings`, `Tool_Product_History`) | +| **8** | `reorg/peripheral-survey-benchmark` | `reorg/peripheral-tools-endpoint` | **Bundle C**: `survey` (`Question`, `TextQuestion`, `Choice`, `ChoiceQuestion`, `Engagement_Survey`, `Answered_Survey`, `General_Survey`, `Answer`, `TextAnswer`, `ChoiceAnswer`) + `benchmark` (`Benchmark_Type`, `Benchmark_Category`, `Benchmark_Requirement`, `Benchmark_Product`, `Benchmark_Product_Summary`) | +| **9** | `reorg/peripheral-notes-files` | `reorg/peripheral-survey-benchmark` | **Bundle D**: `notes` (`NoteHistory`, `Notes`) + `note_type` (`Note_Type`) + `file_uploads` (`UniqueUploadNameProvider`, `FileUpload`, `FileAccessToken`) + `reports` (`Report_Type`) + `risk_acceptance` (`Risk_Acceptance`) | +| **10** | `reorg/peripheral-misc` | `reorg/peripheral-notes-files` | **Bundle E**: `regulations` (`Regulation`) + `banner` (`BannerConf`) + `announcement` (`Announcement`, `UserAnnouncement`) + `development_environment` (`Development_Environment`) + `object` (`Objects_Review`, `Objects_Product`) | + +**Bundle order is by FK direction**: `user` first (`Dojo_User` is an FK target almost everywhere); everything else references already-moved or string-ref'd models. Inside a bundle, FKs between same-bundle models are real class refs; FKs to anything OUTSIDE the bundle become string refs `"dojo."` (per the string-FK rule above — this keeps the extracted `models.py` free of top-level `from dojo.models import`). + +### Stack & PR mechanics (locked with user) + +- **Branches live on the `upstream` remote** (`git@github.com:DefectDojo/django-DefectDojo.git`), exactly like the existing 5 (their head branches are on upstream, e.g. `upstream/reorg/finding-models`). Push each new branch to `upstream`, and **force-push with `--force-with-lease`** on cascade (`git push --force-with-lease upstream :`). +- **The 5 new PRs are DRAFT PRs.** Create with `gh pr create --draft --repo DefectDojo/django-DefectDojo --base --head `. +- Each new branch is created from its predecessor's tip: `git checkout -b reorg/peripheral-user reorg/finding-models`, etc. Merge bottom-up. +- **PR descriptions**: every PR in the stack (all 10) must include a stack map listing all 10 PRs in order with checkboxes and the bottom-up merge note, so reviewers see the whole picture. Summary section only — NO test-plan section (see CLAUDE.local.md / PR rules). Format PR URLs as markdown links. Read an existing body with `gh pr view --json body -q '.body'` before editing; edit via `--body-file` or the REST `gh api -X PATCH` path (inline `--body` silently fails on this repo). +- **Cascade after editing a lower branch** (e.g. this AGENTS.md commit on #14970): `git rebase --onto ` up the chain, then force-push all with `--force-with-lease`. AGENTS.md edits always land on the bottom branch (#14970) and cascade. + +### Per-module execution = the 9-phase playbook above + +For EACH module in a bundle, run **Phase 0 pre-flight first** (the grep block above) to discover its exact forms/filters/serializers/viewsets/urls/admin/signals/consumers — do NOT trust a memorized list. Then Phases 1–9. Reference complete templates: `dojo/url/`, `dojo/location/` (models), and `dojo/finding/`, `dojo/product/`, `dojo/test/`, `dojo/engagement/` (full API+UI slices). Verify gates after each phase (`manage.py check`, `makemigrations --check --dry-run`, `./run-unittest.sh --test-case unittests. 2>&1 | tee /tmp/test.log`). All gates run in docker (`docker compose exec -T uwsgi ...`); model imports need `manage.py shell -c`. + +### Model line ranges in `dojo/models.py` (snapshot — re-grep before editing; line numbers shift as you extract) + +- **CWE** 1027–1031 · **BurpRawRequestResponse** 1563–1575 → `finding` (PR #14974) +- **Dojo_User** 174–209 · **UserContactInfo** 211–234 · **Contact** 605–612 · **System_Settings** 236–595 +- **Tool_Type** 940–949 · **Tool_Configuration** 951–979 · **ToolConfigForm_Admin/Tool_Configuration_Admin** 981–1010 · **Endpoint_Params** 1033–1039 · **Endpoint_Status** 1041–1093 · **Endpoint** 1095–1470 · **Tool_Product_Settings** 1765–1777 · **Tool_Product_History** 1779–1785 +- **Benchmark_Type** 1890–1905 · **Benchmark_Category** 1907–1921 · **Benchmark_Requirement** 1923–1939 · **Benchmark_Product** 1941–1957 · **Benchmark_Product_Summary** 1959–1989 · **Question** 1992–2012 · **TextQuestion** 2014–2024 · **Choice** 2026–2039 · **ChoiceQuestion** 2041–2058 · **Engagement_Survey** 2060–2076 · **Answered_Survey** 2078–2101 · **General_Survey** 2107–2123 · **Answer** 2126–2138 · **TextAnswer** 2140–2149 · **ChoiceAnswer** 2151–2253 +- **Note_Type** 614–623 · **NoteHistory** 625–636 · **Notes** 638–669 · **UniqueUploadNameProvider** 108–135 · **FileUpload** 671–749 · **FileAccessToken** 1679–1703 · **Report_Type** 751–753 · **Risk_Acceptance** 1577–1677 +- **Regulation** 136–168 · **Announcement** 1713–1725 · **UserAnnouncement** 1727–1730 · **BannerConf** 1732–1763 · **Development_Environment** 1472–1481 · **Objects_Review** 1829–1835 · **Objects_Product** 1837–1861 + +### Module-specific gotchas (beyond the generic playbook) + +- **`Question` / `Answer` (survey)**: base classes are defined inside a `with warnings.catch_warnings(): ...` block (polymorphic-model deprecation suppression). PRESERVE that block structure when moving to `dojo/survey/models.py` — don't flatten it. +- **survey & benchmark have NO serializers/viewsets in `api_v2`** (verified). So Bundle C likely has no `api/` layer — skip Phases 6–9 for those modules (confirm with Phase 0). They DO have UI views/urls/forms/filters. +- **`Benchmark_Requirement` → M2M `CWE`**: `CWE` moves to `finding` in PR #14974 (lands lower in the stack), so by the time Bundle C runs, use string ref `"dojo.CWE"` (the `dojo.models` re-export stays valid). Same for any other `CWE` reference. +- **`Risk_Acceptance`**: M2M `accepted_findings`→Finding, FK `owner`→Dojo_User, M2M `notes`→Notes — all cross-bundle → string refs. `dojo/risk_acceptance/` already has `api.py`/`helper.py`/`queries.py`/`signals.py` but no `models.py`; reconcile `api.py` vs the playbook's `api/` dir layout. +- **`Endpoint`**: references `Dojo_User`, `Finding`, `Product`, `Endpoint_Status` — string-ref everything except same-bundle `Endpoint_Params`/`Endpoint_Status`. `dojo/endpoint/` already has `queries.py`/`utils.py`/`signals.py`. +- **`tool_config` admin**: `ToolConfigForm_Admin` (a `forms.ModelForm`) and `Tool_Configuration_Admin` (an `admin.ModelAdmin`) currently sit in `dojo/models.py` — move them to `dojo/tool_config/admin.py` (form + admin), not `models.py`. +- **`CWE` / `BurpRawRequestResponse` are heavily imported** (20+ files across `dojo/` and `unittests/`, including tool parsers for CWE and importers for Burp). Run the Phase 0 consumer grep (`grep -rn "import.*\bCWE\b" dojo/ unittests/`, same for `BurpRawRequestResponse`) and rely on the `dojo.models` re-export for external consumers — only repoint finding's own code. +- **Shared bases (the `FindingTagStringFilter` trap)**: before moving any form/filter, grep for subclasses/consumers OUTSIDE the module. If a base form/filter is also used by a model staying in `dojo/models.py` or another module, KEEP it in the monolith and import it, rather than moving + back-importing (which cycles). The prefetcher full-re-export rule (Phase 6) applies to any moved `ModelSerializer`. + +### After the stack is built + +Update the **Current State** table above (mark the newly-completed modules **Complete**), and update the monolith line counts in "Monolithic Files Being Decomposed" (they are stale — `dojo/models.py` is ~2,254 lines now, not 4,973). diff --git a/dojo/announcement/__init__.py b/dojo/announcement/__init__.py index e69de29bb2d..d22235e108d 100644 --- a/dojo/announcement/__init__.py +++ b/dojo/announcement/__init__.py @@ -0,0 +1 @@ +import dojo.announcement.admin # noqa: F401 diff --git a/dojo/announcement/admin.py b/dojo/announcement/admin.py new file mode 100644 index 00000000000..23507595678 --- /dev/null +++ b/dojo/announcement/admin.py @@ -0,0 +1,6 @@ +from django.contrib import admin + +from dojo.announcement.models import Announcement, UserAnnouncement + +admin.site.register(Announcement) +admin.site.register(UserAnnouncement) diff --git a/dojo/announcement/api/__init__.py b/dojo/announcement/api/__init__.py new file mode 100644 index 00000000000..0e64dd504ef --- /dev/null +++ b/dojo/announcement/api/__init__.py @@ -0,0 +1 @@ +path = "announcements" # noqa: RUF067 diff --git a/dojo/announcement/api/serializer.py b/dojo/announcement/api/serializer.py new file mode 100644 index 00000000000..0022e2f22ef --- /dev/null +++ b/dojo/announcement/api/serializer.py @@ -0,0 +1,21 @@ +from django.db import IntegrityError +from rest_framework import serializers + +from dojo.announcement.models import Announcement + + +class AnnouncementSerializer(serializers.ModelSerializer): + + class Meta: + model = Announcement + fields = "__all__" + + def create(self, validated_data): + validated_data["id"] = 1 + try: + return super().create(validated_data) + except IntegrityError as e: + if 'duplicate key value violates unique constraint "dojo_announcement_pkey"' in str(e): + msg = "No more than one Announcement is allowed" + raise serializers.ValidationError(msg) + raise diff --git a/dojo/announcement/api/urls.py b/dojo/announcement/api/urls.py new file mode 100644 index 00000000000..1da04620311 --- /dev/null +++ b/dojo/announcement/api/urls.py @@ -0,0 +1,7 @@ +from dojo.announcement.api import path +from dojo.announcement.api.views import AnnouncementViewSet + + +def add_announcement_urls(router): + router.register(path, AnnouncementViewSet, basename="announcement") + return router diff --git a/dojo/announcement/api/views.py b/dojo/announcement/api/views.py new file mode 100644 index 00000000000..cf6a411adc9 --- /dev/null +++ b/dojo/announcement/api/views.py @@ -0,0 +1,20 @@ +from django_filters.rest_framework import DjangoFilterBackend + +from dojo.announcement.api.serializer import AnnouncementSerializer +from dojo.announcement.models import Announcement +from dojo.api_v2.views import DojoModelViewSet +from dojo.authorization import api_permissions as permissions + + +# Authorization: configuration +class AnnouncementViewSet( + DojoModelViewSet, +): + serializer_class = AnnouncementSerializer + queryset = Announcement.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = "__all__" + permission_classes = (permissions.UserHasConfigurationPermissionStaff,) + + def get_queryset(self): + return Announcement.objects.all().order_by("id") diff --git a/dojo/announcement/models.py b/dojo/announcement/models.py new file mode 100644 index 00000000000..9266b96c019 --- /dev/null +++ b/dojo/announcement/models.py @@ -0,0 +1,28 @@ +from django.db import models +from django.utils.translation import gettext as _ + +ANNOUNCEMENT_STYLE_CHOICES = ( + ("info", "Info"), + ("success", "Success"), + ("warning", "Warning"), + ("danger", "Danger"), +) + + +class Announcement(models.Model): + message = models.CharField(max_length=500, + help_text=_("This dismissable message will be displayed on all pages for authenticated users. It can contain basic html tags, for example https://example.com"), + default="") + style = models.CharField(max_length=64, choices=ANNOUNCEMENT_STYLE_CHOICES, default="info", + help_text=_("The style of banner to display. (info, success, warning, danger)")) + dismissable = models.BooleanField(default=False, + null=False, + blank=True, + verbose_name=_("Dismissable?"), + help_text=_("Ticking this box allows users to dismiss the current announcement"), + ) + + +class UserAnnouncement(models.Model): + announcement = models.ForeignKey("dojo.Announcement", null=True, editable=False, on_delete=models.CASCADE, related_name="user_announcement") + user = models.ForeignKey("dojo.Dojo_User", null=True, editable=False, on_delete=models.CASCADE) diff --git a/dojo/announcement/signals.py b/dojo/announcement/signals.py index c74fd0e5d50..677fafe93c2 100644 --- a/dojo/announcement/signals.py +++ b/dojo/announcement/signals.py @@ -1,7 +1,8 @@ from django.db.models.signals import post_save from django.dispatch import receiver -from dojo.models import Announcement, Dojo_User, UserAnnouncement +from dojo.announcement.models import Announcement, UserAnnouncement +from dojo.user.models import Dojo_User @receiver(post_save, sender=Dojo_User) diff --git a/dojo/announcement/ui/__init__.py b/dojo/announcement/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/announcement/ui/forms.py b/dojo/announcement/ui/forms.py new file mode 100644 index 00000000000..47e4f97721d --- /dev/null +++ b/dojo/announcement/ui/forms.py @@ -0,0 +1,17 @@ +from django import forms + +from dojo.announcement.models import Announcement + + +class AnnouncementCreateForm(forms.ModelForm): + class Meta: + model = Announcement + fields = "__all__" + + +class AnnouncementRemoveForm(AnnouncementCreateForm): + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.fields["dismissable"].disabled = True + self.fields["message"].disabled = True + self.fields["style"].disabled = True diff --git a/dojo/announcement/urls.py b/dojo/announcement/ui/urls.py similarity index 88% rename from dojo/announcement/urls.py rename to dojo/announcement/ui/urls.py index 9dc91187653..2ac2f2b8af0 100644 --- a/dojo/announcement/urls.py +++ b/dojo/announcement/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from dojo.announcement import views +from dojo.announcement.ui import views urlpatterns = [ re_path( diff --git a/dojo/announcement/views.py b/dojo/announcement/ui/views.py similarity index 95% rename from dojo/announcement/views.py rename to dojo/announcement/ui/views.py index 7afe915210b..f668901e86f 100644 --- a/dojo/announcement/views.py +++ b/dojo/announcement/ui/views.py @@ -7,8 +7,8 @@ from django.utils.translation import gettext from django.utils.translation import gettext_lazy as _ -from dojo.forms import AnnouncementCreateForm, AnnouncementRemoveForm -from dojo.models import Announcement, UserAnnouncement +from dojo.announcement.models import Announcement, UserAnnouncement +from dojo.announcement.ui.forms import AnnouncementCreateForm, AnnouncementRemoveForm from dojo.utils import add_breadcrumb logger = logging.getLogger(__name__) diff --git a/dojo/api_v2/serializers.py b/dojo/api_v2/serializers.py index eb0f67eb7be..1a239b6a447 100644 --- a/dojo/api_v2/serializers.py +++ b/dojo/api_v2/serializers.py @@ -1,5 +1,3 @@ -import base64 -import collections import json import logging import re @@ -10,85 +8,46 @@ import tagulous from django.conf import settings from django.contrib.auth.models import Permission -from django.contrib.auth.password_validation import validate_password -from django.core.exceptions import PermissionDenied, ValidationError +from django.core.exceptions import ValidationError from django.db import transaction from django.db.utils import IntegrityError -from django.urls import reverse from django.utils import timezone from django.utils.translation import gettext_lazy as _ from drf_spectacular.utils import extend_schema_field from rest_framework import serializers from rest_framework.exceptions import NotFound from rest_framework.exceptions import ValidationError as RestFrameworkValidationError -from rest_framework.fields import DictField - -import dojo.finding.helper as finding_helper -import dojo.risk_acceptance.helper as ra_helper -from dojo.authorization.authorization import user_has_permission -from dojo.celery_dispatch import dojo_dispatch_task -from dojo.endpoint.utils import endpoint_filter, endpoint_meta_import -from dojo.finding.helper import ( - save_endpoints_template, - save_vulnerability_ids, - save_vulnerability_ids_template, -) -from dojo.finding.queries import get_authorized_findings + from dojo.importers.auto_create_context import AutoCreateContextManager from dojo.importers.base_importer import BaseImporter from dojo.importers.default_importer import DefaultImporter from dojo.importers.default_reimporter import DefaultReImporter -from dojo.jira import services as jira_services -from dojo.location.models import Location, LocationFindingReference +from dojo.location.models import Location from dojo.models import ( IMPORT_ACTIONS, SEVERITIES, SEVERITY_CHOICES, STATS_FIELDS, - Announcement, App_Analysis, - BurpRawRequestResponse, - Check_List, Development_Environment, - Dojo_User, DojoMeta, Endpoint, - Endpoint_Params, - Endpoint_Status, Engagement, - Engagement_Presets, FileUpload, Finding, Finding_Group, - Finding_Template, Language_Type, Languages, Network_Locations, - Note_Type, - NoteHistory, Notes, Product, Product_API_Scan_Configuration, - Product_Type, - Regulation, - Risk_Acceptance, SLA_Configuration, Sonarqube_Issue, Sonarqube_Issue_Transition, - System_Settings, Test, - Test_Import, - Test_Import_Finding_Action, - Test_Type, - Tool_Configuration, - Tool_Product_Settings, - Tool_Type, User, - UserContactInfo, - Vulnerability_Id, - get_current_date, ) -from dojo.notifications.helper import async_create_notification from dojo.product_announcements import ( LargeScanSizeProductAnnouncement, ScanTypeProductAnnouncement, @@ -98,8 +57,6 @@ requires_file, requires_tool_type, ) -from dojo.user.queries import get_authorized_users -from dojo.user.utils import get_configuration_permissions_codenames from dojo.utils import is_scan_file_too_large from dojo.validators import ImporterFileExtensionValidator, tag_validator @@ -199,1613 +156,224 @@ def to_internal_value(self, data): logger.debug("data as json: %s", data) if not isinstance(data, list): - self.fail("not_a_list", input_type=type(data).__name__) - - data_safe = [] - for s in data: - # Ensure if the element in the list is string - if not isinstance(s, six.string_types): - self.fail("not_a_str") - # Run the children validation - self.child.run_validation(s) - # Split the tags up in any way we need to - substrings = re.findall(r'(?:"[^"]*"|[^",]+)', s) - # Validate the tag to ensure it doesn't contain invalid characters - for sub in substrings: - tag_validator(sub, exception_class=RestFrameworkValidationError) - data_safe.extend(substrings) - - logger.debug("result after rendering tags: %s", data_safe) - return data_safe - - def to_representation(self, value): - if not isinstance(value, list): - # we can't use isinstance because TagRelatedManager is non-existing class - # it cannot be imported or referenced, so we fallback to string - # comparison - if type(value).__name__ == "TagRelatedManager": - value = value.get_tag_list() - elif isinstance(value, str): - value = tagulous.utils.parse_tags(value) - else: - msg = f"unable to convert {type(value).__name__} into list of tags" - raise ValueError(msg) - return value - - -class RequestResponseDict(collections.UserList): - def __init__(self, *args, **kwargs): - pretty_print = kwargs.pop("pretty_print", True) - collections.UserList.__init__(self, *args, **kwargs) - self.pretty_print = pretty_print - - def __add__(self, rhs): - return RequestResponseDict(list.__add__(self, rhs)) - - def __getitem__(self, item): - result = list.__getitem__(self, item) - try: - return RequestResponseDict(result) - except TypeError: - return result - - def __str__(self): - if self.pretty_print: - return json.dumps( - self, sort_keys=True, indent=4, separators=(",", ": "), - ) - return json.dumps(self) - - -class RequestResponseSerializerField(serializers.ListSerializer): - child = DictField(child=serializers.CharField()) - default_error_messages = { - "not_a_list": _( - 'Expected a list of items but got type "{input_type}".', - ), - "invalid_json": _( - "Invalid json list. A tag list submitted in string" - " form must be valid json.", - ), - "not_a_dict": _( - "All list items must be of dict type with keys 'request' and 'response'", - ), - "not_a_str": _("All values in the dict must be of string type."), - } - order_by = None - - def __init__(self, **kwargs): - pretty_print = kwargs.pop("pretty_print", True) - - style = kwargs.pop("style", {}) - kwargs["style"] = {"base_template": "textarea.html"} - kwargs["style"].update(style) - - if "data" in kwargs: - data = kwargs["data"] - - if isinstance(data, list): - kwargs["many"] = True - - super().__init__(**kwargs) - - self.pretty_print = pretty_print - - def to_internal_value(self, data): - if isinstance(data, six.string_types): - if not data: - data = [] - try: - data = json.loads(data) - except ValueError: - self.fail("invalid_json") - - if not isinstance(data, list): - self.fail("not_a_list", input_type=type(data).__name__) - for s in data: - if not isinstance(s, dict): - self.fail("not_a_dict", input_type=type(s).__name__) - - request = s.get("request", None) - response = s.get("response", None) - - if not isinstance(request, str): - self.fail("not_a_str", input_type=type(request).__name__) - if not isinstance(response, str): - self.fail("not_a_str", input_type=type(request).__name__) - - self.child.run_validation(s) - return data - - def to_representation(self, value): - if not isinstance(value, RequestResponseDict): - if not isinstance(value, list): - # this will trigger when a queryset is found... - burps = value.all().order_by(*self.order_by) if self.order_by else value.all() - value = [ - { - "request": burp.get_request(), - "response": burp.get_response(), - } - for burp in burps - ] - - return value - - -class BurpRawRequestResponseSerializer(serializers.Serializer): - req_resp = RequestResponseSerializerField(required=True) - - -class BurpRawRequestResponseMultiSerializer(serializers.ModelSerializer): - burpRequestBase64 = serializers.CharField() - burpResponseBase64 = serializers.CharField() - - def to_representation(self, data): - return { - "id": data.id, - "finding": data.finding.id, - "burpRequestBase64": data.burpRequestBase64.decode("utf-8"), - "burpResponseBase64": data.burpResponseBase64.decode("utf-8"), - } - - def validate(self, data): - b64request = data.get("burpRequestBase64", None) - b64response = data.get("burpResponseBase64", None) - finding = data.get("finding", None) - # Make sure all fields are present - if not b64request or not b64response or not finding: - msg = "burpRequestBase64, burpResponseBase64, and finding are required." - raise ValidationError(msg) - # Verify we have true base64 decoding - try: - base64.b64decode(b64request, validate=True) - base64.b64decode(b64response, validate=True) - except Exception as e: - msg = "Inputs need to be valid base64 encodings" - raise ValidationError(msg) from e - # Encode the data in utf-8 to remove any bad characters - data["burpRequestBase64"] = b64request.encode("utf-8") - data["burpResponseBase64"] = b64response.encode("utf-8") - # Run the model validation - an ValidationError will be raised if there is an issue - BurpRawRequestResponse(finding=finding, burpRequestBase64=b64request, burpResponseBase64=b64response).clean() - - return data - - class Meta: - model = BurpRawRequestResponse - fields = "__all__" - - -class MetaSerializer(serializers.ModelSerializer): - product = serializers.PrimaryKeyRelatedField( - queryset=Product.objects.all(), - required=False, - default=None, - allow_null=True, - ) - endpoint = serializers.PrimaryKeyRelatedField( - queryset=Location.objects.all(), - required=False, - default=None, - allow_null=True, - ) - location = serializers.PrimaryKeyRelatedField( - queryset=Location.objects.all(), - required=False, - default=None, - allow_null=True, - ) - finding = serializers.PrimaryKeyRelatedField( - queryset=Finding.objects.all(), - required=False, - default=None, - allow_null=True, - ) - - def validate(self, data): - if settings.V3_FEATURE_LOCATIONS and "endpoint" in data: - data["location"] = data.pop("endpoint") - DojoMeta(**data).clean() - return data - - # TODO: Delete this after the move to Locations - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - if not settings.V3_FEATURE_LOCATIONS: - self.fields["endpoint"] = serializers.PrimaryKeyRelatedField( - queryset=Endpoint.objects.all(), - required=False, - default=None, - allow_null=True, - ) - - class Meta: - model = DojoMeta - fields = "__all__" - - -class MetadataSerializer(serializers.Serializer): - name = serializers.CharField(max_length=120) - value = serializers.CharField(max_length=300) - - -class MetaMainSerializer(serializers.Serializer): - id = serializers.IntegerField(read_only=True) - - product = serializers.PrimaryKeyRelatedField( - queryset=Product.objects.all(), - required=False, - default=None, - allow_null=True, - ) - endpoint = serializers.PrimaryKeyRelatedField( - queryset=Endpoint.objects.all(), - required=False, - default=None, - allow_null=True, - ) - finding = serializers.PrimaryKeyRelatedField( - queryset=Finding.objects.all(), - required=False, - default=None, - allow_null=True, - ) - metadata = MetadataSerializer(many=True) - - def validate(self, data): - product_id = data.get("product", None) - endpoint_id = data.get("endpoint", None) - finding_id = data.get("finding", None) - metadata = data.get("metadata") - - for item in metadata: - # this will only verify that one and only one of product, endpoint, or finding is passed... - DojoMeta(product=product_id, - endpoint=endpoint_id, - finding=finding_id, - name=item.get("name"), - value=item.get("value")).clean() - - return data - - -class ProductMetaSerializer(serializers.ModelSerializer): - class Meta: - model = DojoMeta - fields = ("name", "value") - - -class UserSerializer(serializers.ModelSerializer): - date_joined = serializers.DateTimeField(read_only=True) - last_login = serializers.DateTimeField(read_only=True, allow_null=True) - email = serializers.EmailField(required=True) - token_last_reset = serializers.SerializerMethodField() - password_last_reset = serializers.SerializerMethodField() - password = serializers.CharField( - write_only=True, - style={"input_type": "password"}, - required=False, - validators=[validate_password], - ) - configuration_permissions = serializers.PrimaryKeyRelatedField( - allow_null=True, - queryset=Permission.objects.filter( - codename__in=get_configuration_permissions_codenames(), - ), - many=True, - required=False, - source="user_permissions", - ) - - class Meta: - model = Dojo_User - fields = ( - "id", - "username", - "first_name", - "last_name", - "email", - "date_joined", - "last_login", - "is_active", - "is_staff", - "is_superuser", - "token_last_reset", - "password_last_reset", - "password", - "configuration_permissions", - ) - - @extend_schema_field(serializers.DateTimeField(allow_null=True)) - def get_token_last_reset(self, instance): - uci = getattr(instance, "usercontactinfo", None) - return getattr(uci, "token_last_reset", None) - - @extend_schema_field(serializers.DateTimeField(allow_null=True)) - def get_password_last_reset(self, instance): - uci = getattr(instance, "usercontactinfo", None) - return getattr(uci, "password_last_reset", None) - - def to_representation(self, instance): - ret = super().to_representation(instance) - - # This will show only "configuration_permissions" even if user has also - # other permissions - all_permissions = set(ret["configuration_permissions"]) - allowed_configuration_permissions = set( - self.fields[ - "configuration_permissions" - ].child_relation.queryset.values_list("id", flat=True), - ) - ret["configuration_permissions"] = list( - all_permissions.intersection(allowed_configuration_permissions), - ) - - return ret - - def update(self, instance, validated_data): - permissions_in_payload = None - new_configuration_permissions = None - if ( - "user_permissions" in validated_data - ): # This field was renamed from "configuration_permissions" in the meantime - permissions_in_payload = validated_data.pop("user_permissions") - new_configuration_permissions = set(permissions_in_payload) - - instance = super().update(instance, validated_data) - - # This will update only Permissions from category - # "configuration_permissions". Others will be untouched - if new_configuration_permissions: - allowed_configuration_permissions = set( - self.fields[ - "configuration_permissions" - ].child_relation.queryset.all(), - ) - non_configuration_permissions = ( - set(instance.user_permissions.all()) - - allowed_configuration_permissions - ) - new_permissions = non_configuration_permissions.union( - new_configuration_permissions, - ) - instance.user_permissions.set(new_permissions) - - # Clear all configuration permissions if an empty list is provided - if isinstance(permissions_in_payload, list) and len(permissions_in_payload) == 0: - instance.user_permissions.clear() - - return instance - - def create(self, validated_data): - password = validated_data.pop("password", None) - - new_configuration_permissions = None - if ( - "user_permissions" in validated_data - ): # This field was renamed from "configuration_permissions" in the meantime - new_configuration_permissions = set( - validated_data.pop("user_permissions"), - ) - - user = Dojo_User.objects.create(**validated_data) - - if password: - user.set_password(password) - else: - user.set_unusable_password() - - # This will create only Permissions from category - # "configuration_permissions". There are no other Permissions. - if new_configuration_permissions: - user.user_permissions.set(new_configuration_permissions) - - user.save() - return user - - def validate(self, data): - instance_is_superuser = self.instance.is_superuser if self.instance is not None else False - data_is_superuser = data.get("is_superuser", False) - if not self.context["request"].user.is_superuser and ( - instance_is_superuser or data_is_superuser - ): - msg = "Only superusers are allowed to add or edit superusers." - raise ValidationError(msg) - - instance_is_staff = self.instance.is_staff if self.instance is not None else False - data_is_staff = data.get("is_staff", instance_is_staff) - if not self.context["request"].user.is_superuser and data_is_staff != instance_is_staff: - msg = "Only superusers are allowed to add or edit staff users." - raise ValidationError(msg) - - if self.context["request"].method in {"PATCH", "PUT"} and "password" in data: - msg = "Update of password though API is not allowed" - raise ValidationError(msg) - if self.context["request"].method == "POST" and "password" not in data and settings.REQUIRE_PASSWORD_ON_USER: - msg = "Passwords must be supplied for new users" - raise ValidationError(msg) - return super().validate(data) - - -class UserContactInfoSerializer(serializers.ModelSerializer): - user_profile = UserSerializer(many=False, source="user", read_only=True) - - class Meta: - model = UserContactInfo - fields = "__all__" - - def validate(self, data): - user = data.get("user", None) or self.instance.user - if data.get("force_password_reset", False) and not user.has_usable_password(): - msg = "Password resets are not allowed for users authorized through SSO." - raise ValidationError(msg) - return super().validate(data) - - -class UserStubSerializer(serializers.ModelSerializer): - class Meta: - model = Dojo_User - fields = ("id", "username", "first_name", "last_name") - - -class AddUserSerializer(serializers.ModelSerializer): - class Meta: - model = User - fields = ("id", "username") - - -class NoteTypeSerializer(serializers.ModelSerializer): - class Meta: - model = Note_Type - fields = "__all__" - - -class NoteHistorySerializer(serializers.ModelSerializer): - current_editor = UserStubSerializer(read_only=True) - note_type = NoteTypeSerializer(read_only=True, many=False) - - class Meta: - model = NoteHistory - fields = "__all__" - - -class NoteSerializer(serializers.ModelSerializer): - author = UserStubSerializer(many=False, read_only=True) - editor = UserStubSerializer(read_only=True, many=False, allow_null=True) - history = NoteHistorySerializer(read_only=True, many=True) - note_type = NoteTypeSerializer(read_only=True, many=False) - - def update(self, instance, validated_data): - instance.entry = validated_data.get("entry") - instance.edited = True - instance.editor = self.context["request"].user - instance.edit_time = timezone.now() - history = NoteHistory( - data=instance.entry, - time=instance.edit_time, - current_editor=instance.editor, - ) - history.save() - instance.history.add(history) - instance.save() - return instance - - class Meta: - model = Notes - fields = "__all__" - - -class FileSerializer(serializers.ModelSerializer): - file = serializers.FileField(required=True) - - class Meta: - model = FileUpload - fields = "__all__" - - def validate(self, data): - if file := data.get("file"): - # the clean will validate the file extensions and raise a Validation error if the extensions are not accepted - FileUpload(title=file.name, file=file).clean() - return data - return None - - -class RawFileSerializer(serializers.ModelSerializer): - file = serializers.FileField(required=True) - - class Meta: - model = FileUpload - fields = ["file"] - - -class RiskAcceptanceProofSerializer(serializers.ModelSerializer): - path = serializers.FileField(required=True) - - class Meta: - model = Risk_Acceptance - fields = ["path"] - - -class ProductTypeSerializer(serializers.ModelSerializer): - class Meta: - model = Product_Type - fields = "__all__" - - -class EngagementSerializer(serializers.ModelSerializer): - tags = TagListSerializerField(required=False) - - class Meta: - model = Engagement - exclude = ("inherited_tags",) - - def validate(self, data): - if self.context["request"].method == "POST": - if data.get("target_start") > data.get("target_end"): - msg = "Your target start date exceeds your target end date" - raise serializers.ValidationError(msg) - return data - - def build_relational_field(self, field_name, relation_info): - if field_name == "notes": - return NoteSerializer, {"many": True, "read_only": True} - if field_name == "files": - return FileSerializer, {"many": True, "read_only": True} - return super().build_relational_field(field_name, relation_info) - - -class EngagementToNotesSerializer(serializers.Serializer): - engagement_id = serializers.PrimaryKeyRelatedField( - queryset=Engagement.objects.all(), many=False, allow_null=True, - ) - notes = NoteSerializer(many=True) - - -class RiskAcceptanceToNotesSerializer(serializers.Serializer): - risk_acceptance_id = serializers.PrimaryKeyRelatedField( - queryset=Risk_Acceptance.objects.all(), many=False, allow_null=True, - ) - notes = NoteSerializer(many=True) - - -class EngagementToFilesSerializer(serializers.Serializer): - engagement_id = serializers.PrimaryKeyRelatedField( - queryset=Engagement.objects.all(), many=False, allow_null=True, - ) - files = FileSerializer(many=True) - - def to_representation(self, data): - engagement = data.get("engagement_id") - files = data.get("files") - new_files = [{ - "id": file.id, - "file": "{site_url}/{file_access_url}".format( - site_url=settings.SITE_URL, - file_access_url=file.get_accessible_url( - engagement, engagement.id, - ), - ), - "title": file.title, - } for file in files] - return {"engagement_id": engagement.id, "files": new_files} - - -class EngagementCheckListSerializer(serializers.ModelSerializer): - class Meta: - model = Check_List - fields = "__all__" - - -class AppAnalysisSerializer(serializers.ModelSerializer): - tags = TagListSerializerField(required=False) - - class Meta: - model = App_Analysis - fields = "__all__" - - -class ToolTypeSerializer(serializers.ModelSerializer): - class Meta: - model = Tool_Type - fields = "__all__" - - def validate(self, data): - if self.context["request"].method == "POST": - name = data.get("name") - # Make sure this will not create a duplicate test type - if Tool_Type.objects.filter(name=name).count() > 0: - msg = "A Tool Type with the name already exists" - raise serializers.ValidationError(msg) - return data - - -class RegulationSerializer(serializers.ModelSerializer): - class Meta: - model = Regulation - fields = "__all__" - - -class ToolConfigurationSerializer(serializers.ModelSerializer): - class Meta: - model = Tool_Configuration - fields = "__all__" - extra_kwargs = { - "password": {"write_only": True}, - "ssh": {"write_only": True}, - "api_key": {"write_only": True}, - } - - -class ToolProductSettingsSerializer(serializers.ModelSerializer): - setting_url = serializers.CharField(source="url") - product = serializers.PrimaryKeyRelatedField( - queryset=Product.objects.all(), required=True, - ) - - class Meta: - model = Tool_Product_Settings - fields = "__all__" - - -class EndpointStatusSerializer(serializers.ModelSerializer): - class Meta: - model = Endpoint_Status - fields = "__all__" - - def run_validators(self, initial_data): - try: - return super().run_validators(initial_data) - except RestFrameworkValidationError as exc: - if "finding, endpoint must make a unique set" in str(exc): - msg = "This endpoint-finding relation already exists" - raise serializers.ValidationError(msg) from exc - raise - - def create(self, validated_data): - endpoint = validated_data.get("endpoint") - finding = validated_data.get("finding") - try: - status = Endpoint_Status.objects.create( - finding=finding, endpoint=endpoint, - ) - except IntegrityError as ie: - if "finding, endpoint must make a unique set" in str(ie): - msg = "This endpoint-finding relation already exists" - raise serializers.ValidationError(msg) - raise - status.mitigated = validated_data.get("mitigated", False) - status.false_positive = validated_data.get("false_positive", False) - status.out_of_scope = validated_data.get("out_of_scope", False) - status.risk_accepted = validated_data.get("risk_accepted", False) - status.date = validated_data.get("date", get_current_date()) - status.save() - return status - - def update(self, instance, validated_data): - try: - return super().update(instance, validated_data) - except IntegrityError as ie: - if "finding, endpoint must make a unique set" in str(ie): - msg = "This endpoint-finding relation already exists" - raise serializers.ValidationError(msg) - raise - - -class EndpointSerializer(serializers.ModelSerializer): - tags = TagListSerializerField(required=False) - active_finding_count = serializers.IntegerField(read_only=True) - - class Meta: - model = Endpoint - exclude = ("inherited_tags",) - - def validate(self, data): - - if self.context["request"].method != "PATCH": - if "product" not in data: - msg = "Product is required" - raise serializers.ValidationError(msg) - protocol = data.get("protocol") - userinfo = data.get("userinfo") - host = data.get("host") - port = data.get("port") - path = data.get("path") - query = data.get("query") - fragment = data.get("fragment") - product = data.get("product") - else: - protocol = data.get("protocol", self.instance.protocol) - userinfo = data.get("userinfo", self.instance.userinfo) - host = data.get("host", self.instance.host) - port = data.get("port", self.instance.port) - path = data.get("path", self.instance.path) - query = data.get("query", self.instance.query) - fragment = data.get("fragment", self.instance.fragment) - if "product" in data and data["product"] != self.instance.product: - msg = "Change of product is not possible" - raise serializers.ValidationError(msg) - product = self.instance.product - - endpoint_ins = Endpoint( - protocol=protocol, - userinfo=userinfo, - host=host, - port=port, - path=path, - query=query, - fragment=fragment, - product=product, - ) - endpoint_ins.clean() # Run standard validation and clean process; can raise errors - - endpoint = endpoint_filter( - protocol=endpoint_ins.protocol, - userinfo=endpoint_ins.userinfo, - host=endpoint_ins.host, - port=endpoint_ins.port, - path=endpoint_ins.path, - query=endpoint_ins.query, - fragment=endpoint_ins.fragment, - product=endpoint_ins.product, - ) - if ( - self.context["request"].method in {"PUT", "PATCH"} - and ( - (endpoint.count() > 1) - or ( - endpoint.count() == 1 - and endpoint.first().pk != self.instance.pk - ) - ) - ) or ( - self.context["request"].method == "POST" and endpoint.count() > 0 - ): - msg = ( - "It appears as though an endpoint with this data already " - "exists for this product." - ) - raise serializers.ValidationError(msg, code="invalid") - - # use clean data - data["protocol"] = endpoint_ins.protocol - data["userinfo"] = endpoint_ins.userinfo - data["host"] = endpoint_ins.host - data["port"] = endpoint_ins.port - data["path"] = endpoint_ins.path - data["query"] = endpoint_ins.query - data["fragment"] = endpoint_ins.fragment - data["product"] = endpoint_ins.product - - return data - - -class EndpointParamsSerializer(serializers.ModelSerializer): - class Meta: - model = Endpoint_Params - fields = "__all__" - - -from dojo.jira.api.serializers import ( # noqa: E402, F401 backward compat - JIRAInstanceSerializer, - JIRAIssueSerializer, - JIRAProjectSerializer, -) - - -class SonarqubeIssueSerializer(serializers.ModelSerializer): - class Meta: - model = Sonarqube_Issue - fields = "__all__" - - -class SonarqubeIssueTransitionSerializer(serializers.ModelSerializer): - class Meta: - model = Sonarqube_Issue_Transition - fields = "__all__" - - -class ProductAPIScanConfigurationSerializer(serializers.ModelSerializer): - class Meta: - model = Product_API_Scan_Configuration - fields = "__all__" - - -class DevelopmentEnvironmentSerializer(serializers.ModelSerializer): - class Meta: - model = Development_Environment - fields = "__all__" - - -class FindingGroupSerializer(serializers.ModelSerializer): - jira_issue = JIRAIssueSerializer(read_only=True, allow_null=True) - - class Meta: - model = Finding_Group - fields = ("id", "name", "test", "jira_issue") - - -class TestSerializer(serializers.ModelSerializer): - tags = TagListSerializerField(required=False) - test_type_name = serializers.ReadOnlyField() - finding_groups = FindingGroupSerializer( - source="finding_group_set", many=True, read_only=True, - ) - - class Meta: - model = Test - exclude = ("inherited_tags",) - - def build_relational_field(self, field_name, relation_info): - if field_name == "notes": - return NoteSerializer, {"many": True, "read_only": True} - if field_name == "files": - return FileSerializer, {"many": True, "read_only": True} - return super().build_relational_field(field_name, relation_info) - - -class TestCreateSerializer(serializers.ModelSerializer): - engagement = serializers.PrimaryKeyRelatedField( - queryset=Engagement.objects.all(), - ) - notes = serializers.PrimaryKeyRelatedField( - allow_null=True, - queryset=Notes.objects.all(), - many=True, - required=False, - ) - tags = TagListSerializerField(required=False) - - class Meta: - model = Test - exclude = ("inherited_tags",) - - -class TestTypeCreateSerializer(serializers.ModelSerializer): - - class Meta: - model = Test_Type - exclude = ("dynamically_generated",) - - -class TestTypeSerializer(serializers.ModelSerializer): - name = serializers.ReadOnlyField() - - class Meta: - model = Test_Type - exclude = ("dynamically_generated",) - - -class TestToNotesSerializer(serializers.Serializer): - test_id = serializers.PrimaryKeyRelatedField( - queryset=Test.objects.all(), many=False, allow_null=True, - ) - notes = NoteSerializer(many=True) - - -class TestToFilesSerializer(serializers.Serializer): - test_id = serializers.PrimaryKeyRelatedField( - queryset=Test.objects.all(), many=False, allow_null=True, - ) - files = FileSerializer(many=True) - - def to_representation(self, data): - test = data.get("test_id") - files = data.get("files") - new_files = [{ - "id": file.id, - "file": f"{settings.SITE_URL}/{file.get_accessible_url(test, test.id)}", - "title": file.title, - } for file in files] - return {"test_id": test.id, "files": new_files} - - -class TestImportFindingActionSerializer(serializers.ModelSerializer): - class Meta: - model = Test_Import_Finding_Action - fields = "__all__" - - -class TestImportSerializer(serializers.ModelSerializer): - # findings = TestImportFindingActionSerializer(source='test_import_finding_action', many=True, read_only=True) - test_import_finding_action_set = TestImportFindingActionSerializer( - many=True, read_only=True, - ) - - class Meta: - model = Test_Import - fields = "__all__" - - -class RiskAcceptanceSerializer(serializers.ModelSerializer): - path = serializers.SerializerMethodField() - - def create(self, validated_data): - instance = super().create(validated_data) - user = getattr(self.context.get("request", None), "user", None) - ra_helper.add_findings_to_risk_acceptance(user, instance, instance.accepted_findings.all()) - - # Add risk acceptance to engagement - # This is fine as Pro has its own model + relationshop to track links with engagements. - if instance.accepted_findings.exists(): - engagement = instance.accepted_findings.first().test.engagement - engagement.risk_acceptance.add(instance) - - return instance - - def update(self, instance, validated_data): - # Determine findings to risk accept, and findings to unaccept risk - existing_findings = Finding.objects.filter(risk_acceptance=self.instance.id) - new_findings_ids = [x.id for x in validated_data.get("accepted_findings", [])] - new_findings = Finding.objects.filter(id__in=new_findings_ids) - findings_to_add = set(new_findings) - set(existing_findings) - findings_to_remove = set(existing_findings) - set(new_findings) - findings_to_add = Finding.objects.filter(id__in=[x.id for x in findings_to_add]) - findings_to_remove = Finding.objects.filter(id__in=[x.id for x in findings_to_remove]) - # Make the update in the database - instance = super().update(instance, validated_data) - user = getattr(self.context.get("request", None), "user", None) - # Add the new findings - ra_helper.add_findings_to_risk_acceptance(user, instance, findings_to_add) - # Remove the ones that were not present in the payload - for finding in findings_to_remove: - ra_helper.remove_finding_from_risk_acceptance(user, instance, finding) - - # Handle orphaned risk acceptances: link to engagement if it now has findings - # This is fine as Pro has its own model + relationshop to track links with engagements. - if instance.accepted_findings.exists() and not instance.engagement: - engagement = instance.accepted_findings.first().test.engagement - engagement.risk_acceptance.add(instance) - - return instance - - @extend_schema_field(serializers.CharField()) - def get_path(self, obj): - engagement = Engagement.objects.filter( - risk_acceptance__id__in=[obj.id], - ).first() - path = "No proof has been supplied" - if engagement and obj.filename() is not None: - path = reverse( - "download_risk_acceptance", args=(engagement.id, obj.id), - ) - request = self.context.get("request") - if request: - path = request.build_absolute_uri(path) - return path - - @extend_schema_field(serializers.IntegerField()) - def get_engagement(self, obj): - engagement = Engagement.objects.filter( - risk_acceptance__id__in=[obj.id], - ).first() - return EngagementSerializer(read_only=True).to_representation( - engagement, - ) - - def validate(self, data): - def validate_findings_have_same_engagement(finding_objects: list[Finding]): - engagements = finding_objects.values_list("test__engagement__id", flat=True).distinct().count() - if engagements > 1: - msg = "You are not permitted to add findings from multiple engagements" - raise PermissionDenied(msg) - - findings = data.get("accepted_findings", []) - findings_ids = [x.id for x in findings] - finding_objects = Finding.objects.filter(id__in=findings_ids) - authed_findings = get_authorized_findings("edit").filter(id__in=findings_ids) - if len(findings) != len(authed_findings): - msg = "You are not permitted to add one or more selected findings to this risk acceptance" - raise PermissionDenied(msg) - if self.context["request"].method == "POST": - validate_findings_have_same_engagement(finding_objects) - - # Validate product allows full risk acceptance BEFORE creating instance - if finding_objects.exists(): - engagement = finding_objects.first().test.engagement - if not engagement.product.enable_full_risk_acceptance: - msg = "Full risk acceptance is not enabled for this product" - raise PermissionDenied(msg) - elif self.context["request"].method in {"PATCH", "PUT"}: - # Use the reverse relation instead of filtering - existing_findings = self.instance.accepted_findings.all() - existing_and_new_findings = existing_findings | finding_objects - validate_findings_have_same_engagement(existing_and_new_findings) - - # Explicit check to prevent engagement switching - risk_acceptance_engagement = self.instance.engagement - if risk_acceptance_engagement and finding_objects.exists(): - new_findings_engagement = finding_objects.first().test.engagement - if risk_acceptance_engagement.id != new_findings_engagement.id: - msg = f"Risk Acceptance belongs to engagement {risk_acceptance_engagement.id}. Cannot add findings from engagement {new_findings_engagement.id}" - raise ValidationError(msg) - return data - - class Meta: - model = Risk_Acceptance - fields = "__all__" - - -class FindingMetaSerializer(serializers.ModelSerializer): - class Meta: - model = DojoMeta - fields = ("name", "value") - - -class FindingProdTypeSerializer(serializers.ModelSerializer): - class Meta: - model = Product_Type - fields = ["id", "name"] - - -class FindingProductSerializer(serializers.ModelSerializer): - prod_type = FindingProdTypeSerializer(required=False) - - class Meta: - model = Product - fields = ["id", "name", "prod_type"] - - -class FindingEngagementSerializer(serializers.ModelSerializer): - product = FindingProductSerializer(required=False) - - class Meta: - model = Engagement - fields = [ - "id", - "name", - "description", - "product", - "target_start", - "target_end", - "branch_tag", - "engagement_type", - "build_id", - "commit_hash", - "version", - "created", - "updated", - ] - - -class FindingEnvironmentSerializer(serializers.ModelSerializer): - class Meta: - model = Development_Environment - fields = ["id", "name"] - - -class FindingTestTypeSerializer(serializers.ModelSerializer): - class Meta: - model = Test_Type - fields = ["id", "name"] - - -class FindingTestSerializer(serializers.ModelSerializer): - engagement = FindingEngagementSerializer(required=False) - environment = FindingEnvironmentSerializer(required=False) - test_type = FindingTestTypeSerializer(required=False) - - class Meta: - model = Test - fields = [ - "id", - "title", - "test_type", - "engagement", - "environment", - "branch_tag", - "build_id", - "commit_hash", - "version", - ] - - -class FindingRelatedFieldsSerializer(serializers.Serializer): - test = serializers.SerializerMethodField() - jira = serializers.SerializerMethodField() - - @extend_schema_field(FindingTestSerializer) - def get_test(self, obj): - return FindingTestSerializer(read_only=True).to_representation( - obj.test, - ) + self.fail("not_a_list", input_type=type(data).__name__) - @extend_schema_field(JIRAIssueSerializer) - def get_jira(self, obj): - issue = jira_services.get_issue(obj) - if issue is None: - return None - return JIRAIssueSerializer(read_only=True).to_representation(issue) + data_safe = [] + for s in data: + # Ensure if the element in the list is string + if not isinstance(s, six.string_types): + self.fail("not_a_str") + # Run the children validation + self.child.run_validation(s) + # Split the tags up in any way we need to + substrings = re.findall(r'(?:"[^"]*"|[^",]+)', s) + # Validate the tag to ensure it doesn't contain invalid characters + for sub in substrings: + tag_validator(sub, exception_class=RestFrameworkValidationError) + data_safe.extend(substrings) + logger.debug("result after rendering tags: %s", data_safe) + return data_safe -class VulnerabilityIdSerializer(serializers.ModelSerializer): - class Meta: - model = Vulnerability_Id - fields = ["vulnerability_id"] + def to_representation(self, value): + if not isinstance(value, list): + # we can't use isinstance because TagRelatedManager is non-existing class + # it cannot be imported or referenced, so we fallback to string + # comparison + if type(value).__name__ == "TagRelatedManager": + value = value.get_tag_list() + elif isinstance(value, str): + value = tagulous.utils.parse_tags(value) + else: + msg = f"unable to convert {type(value).__name__} into list of tags" + raise ValueError(msg) + return value -class FindingSerializer(serializers.ModelSerializer): - mitigated = serializers.DateTimeField(required=False, allow_null=True) - mitigated_by = serializers.PrimaryKeyRelatedField(required=False, allow_null=True, queryset=User.objects.all()) - tags = TagListSerializerField(required=False) - request_response = serializers.SerializerMethodField() - accepted_risks = serializers.SerializerMethodField() - push_to_jira = serializers.BooleanField(default=False) - found_by = serializers.PrimaryKeyRelatedField( - queryset=Test_Type.objects.all(), many=True, - ) - age = serializers.IntegerField(read_only=True) - sla_days_remaining = serializers.IntegerField(read_only=True, allow_null=True) - finding_meta = FindingMetaSerializer(read_only=True, many=True) - related_fields = serializers.SerializerMethodField(allow_null=True) - # for backwards compatibility - jira_creation = serializers.SerializerMethodField(read_only=True, allow_null=True) - jira_change = serializers.SerializerMethodField(read_only=True, allow_null=True) - display_status = serializers.SerializerMethodField() - finding_groups = FindingGroupSerializer( - source="finding_group_set", many=True, read_only=True, +class MetaSerializer(serializers.ModelSerializer): + product = serializers.PrimaryKeyRelatedField( + queryset=Product.objects.all(), + required=False, + default=None, + allow_null=True, ) - vulnerability_ids = VulnerabilityIdSerializer( - source="vulnerability_id_set", many=True, required=False, + endpoint = serializers.PrimaryKeyRelatedField( + queryset=Location.objects.all(), + required=False, + default=None, + allow_null=True, ) - reporter = serializers.PrimaryKeyRelatedField( - required=False, queryset=User.objects.all(), + location = serializers.PrimaryKeyRelatedField( + queryset=Location.objects.all(), + required=False, + default=None, + allow_null=True, ) - endpoints = serializers.PrimaryKeyRelatedField( - source="locations", - many=True, + finding = serializers.PrimaryKeyRelatedField( + queryset=Finding.objects.all(), required=False, - queryset=LocationFindingReference.objects.all(), + default=None, + allow_null=True, ) - class Meta: - model = Finding - exclude = ( - "cve", - "inherited_tags", - ) + def validate(self, data): + if settings.V3_FEATURE_LOCATIONS and "endpoint" in data: + data["location"] = data.pop("endpoint") + DojoMeta(**data).clean() + return data # TODO: Delete this after the move to Locations def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) if not settings.V3_FEATURE_LOCATIONS: - self.fields["endpoints"] = serializers.PrimaryKeyRelatedField( - many=True, required=False, queryset=Endpoint.objects.all(), - ) - - @extend_schema_field(RiskAcceptanceSerializer(many=True)) - def get_accepted_risks(self, obj): - request = self.context.get("request") - if request is None: - return [] - if not user_has_permission(request.user, obj, "edit"): - return [] - return RiskAcceptanceSerializer( - obj.risk_acceptance_set.all(), many=True, - ).data - - @extend_schema_field(serializers.DateTimeField()) - def get_jira_creation(self, obj): - return jira_services.get_creation(obj) - - @extend_schema_field(serializers.DateTimeField()) - def get_jira_change(self, obj): - return jira_services.get_change(obj) - - @extend_schema_field(FindingRelatedFieldsSerializer) - def get_related_fields(self, obj): - request = self.context.get("request", None) - if request is None: - return None - - query_params = request.query_params - if query_params.get("related_fields", "false") == "true": - return FindingRelatedFieldsSerializer( + self.fields["endpoint"] = serializers.PrimaryKeyRelatedField( + queryset=Endpoint.objects.all(), required=False, - ).to_representation(obj) - return None - - def get_display_status(self, obj) -> str: - return obj.status() - - def process_risk_acceptance(self, data): - is_risk_accepted = data.get("risk_accepted") - # Do not take any action if the `risk_accepted` was not passed - if not isinstance(is_risk_accepted, bool): - return - # Determine how to proceed based on the value of `risk_accepted` - if is_risk_accepted and not self.instance.risk_accepted and self.instance.test.engagement.product.enable_simple_risk_acceptance and not data.get("active", False): - ra_helper.simple_risk_accept(self.context["request"].user, self.instance) - elif not is_risk_accepted and self.instance.risk_accepted: # turning off risk_accepted - ra_helper.risk_unaccept(self.context["request"].user, self.instance) - - # Overriding this to push add Push to JIRA functionality - def update(self, instance, validated_data): - # push_all_issues already checked in api views.py - push_to_jira = validated_data.pop("push_to_jira") - - # Save vulnerability ids and pop them - parsed_vulnerability_ids = [] - if (vulnerability_ids := validated_data.pop("vulnerability_id_set", None)): - logger.debug("VULNERABILITY_ID_SET: %s", vulnerability_ids) - parsed_vulnerability_ids.extend(vulnerability_id["vulnerability_id"] for vulnerability_id in vulnerability_ids) - logger.debug("SETTING CVE FROM VULNERABILITY_ID_SET: %s", parsed_vulnerability_ids[0]) - validated_data["cve"] = parsed_vulnerability_ids[0] - - # Save the reporter on the finding - if reporter_id := validated_data.get("reporter"): - instance.reporter = reporter_id - - # Persist vulnerability IDs first so model save computes hash including them (if there is no hash yet) - # we can't pass unsaved_vulnerabilitiy_ids to super.update() - if parsed_vulnerability_ids: - save_vulnerability_ids(instance, parsed_vulnerability_ids) - - # Get found_by from validated_data - found_by = validated_data.pop("found_by", None) - # Handle updates to found_by data - if found_by: - instance.found_by.set(found_by) - # If there is no argument entered for found_by, the user would like to clear out the values on the Finding's found_by field - # Findings still maintain original found_by value associated with their test - # In the event the user does not supply the found_by field at all, we do not modify it - elif isinstance(found_by, list) and len(found_by) == 0: - instance.found_by.clear() - - locations = None - if settings.V3_FEATURE_LOCATIONS: - locations = validated_data.pop("locations", None) - - instance = super().update( - instance, validated_data, - ) - - if settings.V3_FEATURE_LOCATIONS and locations is not None: - for location_ref in instance.locations.all(): - location_ref.location.disassociate_from_finding(instance) - for location_ref in locations: - location_ref.location.associate_with_finding(instance) - - if push_to_jira or jira_services.is_keep_in_sync(instance): - # Push synchronously so that we can see jira errors in real time - success, message = jira_services.push(instance, force_sync=True) - if not success: - raise serializers.ValidationError(message) - - return instance - - def validate(self, data): - # Enforce mitigated metadata editability (only when non-null values are provided) - attempting_to_set_mitigated = any( - (field in data) and (data.get(field) is not None) - for field in ["mitigated", "mitigated_by"] - ) - user = getattr(self.context.get("request", None), "user", None) - if attempting_to_set_mitigated and not finding_helper.can_edit_mitigated_data(user): - errors = {} - if ("mitigated" in data) and (data.get("mitigated") is not None): - errors["mitigated"] = ["Editing mitigated timestamp is disabled (EDITABLE_MITIGATED_DATA=false)"] - if ("mitigated_by" in data) and (data.get("mitigated_by") is not None): - errors["mitigated_by"] = ["Editing mitigated_by is disabled (EDITABLE_MITIGATED_DATA=false)"] - if errors: - raise serializers.ValidationError(errors) - - if self.context["request"].method == "PATCH": - is_active = data.get("active", self.instance.active) - is_verified = data.get("verified", self.instance.verified) - is_duplicate = data.get("duplicate", self.instance.duplicate) - is_false_p = data.get("false_p", self.instance.false_p) - is_risk_accepted = data.get( - "risk_accepted", self.instance.risk_accepted, + default=None, + allow_null=True, ) - else: - is_active = data.get("active", True) - is_verified = data.get("verified", False) - is_duplicate = data.get("duplicate", False) - is_false_p = data.get("false_p", False) - is_risk_accepted = data.get("risk_accepted", False) - - if (is_active or is_verified) and is_duplicate: - msg = "Duplicate findings cannot be verified or active" - raise serializers.ValidationError(msg) - if is_false_p and is_verified: - msg = "False positive findings cannot be verified." - raise serializers.ValidationError(msg) - - if is_risk_accepted and not self.instance.risk_accepted: - if ( - not self.instance.test.engagement.product.enable_simple_risk_acceptance - ): - msg = "Simple risk acceptance is disabled for this product, use the UI to accept this finding." - raise serializers.ValidationError(msg) - - if is_active and is_risk_accepted: - msg = "Active findings cannot be risk accepted." - raise serializers.ValidationError(msg) - # assuming we made it past the validations,call risk acceptance properly to make sure notes, etc get created - # doing it here instead of in update because update doesn't know if the value changed - self.process_risk_acceptance(data) + class Meta: + model = DojoMeta + fields = "__all__" - return data - def validate_severity(self, value: str) -> str: - if value not in SEVERITIES: - msg = f"Severity must be one of the following: {SEVERITIES}" - raise serializers.ValidationError(msg) - return value +class MetadataSerializer(serializers.Serializer): + name = serializers.CharField(max_length=120) + value = serializers.CharField(max_length=300) - def build_relational_field(self, field_name, relation_info): - if field_name == "notes": - return NoteSerializer, {"many": True, "read_only": True} - return super().build_relational_field(field_name, relation_info) - - @extend_schema_field(BurpRawRequestResponseSerializer) - def get_request_response(self, obj): - # Not necessarily Burp scan specific - these are just any request/response pairs - burp_req_resp = obj.burprawrequestresponse_set.all() - var = settings.MAX_REQRESP_FROM_API - if var > -1: - burp_req_resp = burp_req_resp[:var] - burp_list = [] - for burp in burp_req_resp: - request = burp.get_request() - response = burp.get_response() - burp_list.append({"request": request, "response": response}) - serialized_burps = BurpRawRequestResponseSerializer( - {"req_resp": burp_list}, - ) - return serialized_burps.data +class MetaMainSerializer(serializers.Serializer): + id = serializers.IntegerField(read_only=True) -class FindingCreateSerializer(serializers.ModelSerializer): - mitigated = serializers.DateTimeField(required=False, allow_null=True) - mitigated_by = serializers.PrimaryKeyRelatedField(required=False, allow_null=True, queryset=User.objects.all()) - notes = serializers.PrimaryKeyRelatedField( - read_only=True, allow_null=True, required=False, many=True, - ) - test = serializers.PrimaryKeyRelatedField(queryset=Test.objects.all()) - thread_id = serializers.IntegerField(default=0) - found_by = serializers.PrimaryKeyRelatedField( - queryset=Test_Type.objects.all(), many=True, + product = serializers.PrimaryKeyRelatedField( + queryset=Product.objects.all(), + required=False, + default=None, + allow_null=True, ) - url = serializers.CharField(allow_null=True, default=None) - tags = TagListSerializerField(required=False) - push_to_jira = serializers.BooleanField(default=False) - vulnerability_ids = VulnerabilityIdSerializer( - source="vulnerability_id_set", many=True, required=False, + endpoint = serializers.PrimaryKeyRelatedField( + queryset=Endpoint.objects.all(), + required=False, + default=None, + allow_null=True, ) - reporter = serializers.PrimaryKeyRelatedField( - required=False, queryset=User.objects.all(), + finding = serializers.PrimaryKeyRelatedField( + queryset=Finding.objects.all(), + required=False, + default=None, + allow_null=True, ) - - class Meta: - model = Finding - exclude = ( - "cve", - "inherited_tags", - ) - extra_kwargs = { - "active": {"required": True}, - "verified": {"required": True}, - } - - # Overriding this to push add Push to JIRA functionality - def create(self, validated_data): - logger.debug("Creating finding with validated data: %s", validated_data) - push_to_jira = validated_data.pop("push_to_jira", False) - notes = validated_data.pop("notes", None) - found_by = validated_data.pop("found_by", None) - reviewers = validated_data.pop("reviewers", None) - # Process the vulnerability IDs specially - parsed_vulnerability_ids = [] - if (vulnerability_ids := validated_data.pop("vulnerability_id_set", None)): - logger.debug("VULNERABILITY_ID_SET: %s", vulnerability_ids) - parsed_vulnerability_ids.extend(vulnerability_id["vulnerability_id"] for vulnerability_id in vulnerability_ids) - logger.debug("PARSED_VULNERABILITY_IDST: %s", parsed_vulnerability_ids) - logger.debug("SETTING CVE FROM VULNERABILITY_ID_SET: %s", parsed_vulnerability_ids[0]) - validated_data["cve"] = parsed_vulnerability_ids[0] - # validated_data["unsaved_vulnerability_ids"] = parsed_vulnerability_ids - - # super.create() doesn't accept unsaved_vulnerability_ids or dedupe_option=False, so call save directly. - new_finding = Finding(**validated_data) - new_finding.unsaved_vulnerability_ids = parsed_vulnerability_ids or [] - new_finding.save() - - logger.debug(f"New finding CVE: {new_finding.cve}") - - # Deal with all of the many to many things - if notes: - new_finding.notes.set(notes) - if found_by: - new_finding.found_by.set(found_by) - if reviewers: - new_finding.reviewers.set(reviewers) - if parsed_vulnerability_ids: - save_vulnerability_ids(new_finding, parsed_vulnerability_ids) - - if push_to_jira: - jira_services.push(new_finding) - - # Create a notification - dojo_dispatch_task( - async_create_notification, - event="finding_added", - title=_("Addition of %s") % new_finding.title, - finding_id=new_finding.id, - description=_('Finding "%s" was added by %s') % (new_finding.title, new_finding.reporter), - url=reverse("view_finding", args=(new_finding.id,)), - icon="exclamation-triangle", - ) - - return new_finding + metadata = MetadataSerializer(many=True) def validate(self, data): - # Ensure mitigated fields are only set when editable is enabled (ignore nulls) - attempting_to_set_mitigated = any( - (field in data) and (data.get(field) is not None) - for field in ["mitigated", "mitigated_by"] - ) - user = getattr(getattr(self.context, "request", None), "user", None) - if attempting_to_set_mitigated and not finding_helper.can_edit_mitigated_data(user): - errors = {} - if ("mitigated" in data) and (data.get("mitigated") is not None): - errors["mitigated"] = ["Editing mitigated timestamp is disabled (EDITABLE_MITIGATED_DATA=false)"] - if ("mitigated_by" in data) and (data.get("mitigated_by") is not None): - errors["mitigated_by"] = ["Editing mitigated_by is disabled (EDITABLE_MITIGATED_DATA=false)"] - if errors: - raise serializers.ValidationError(errors) - - if "reporter" not in data: - request = self.context["request"] - data["reporter"] = request.user - - if (data.get("active") or data.get("verified")) and data.get( - "duplicate", - ): - msg = "Duplicate findings cannot be verified or active" - raise serializers.ValidationError(msg) - if data.get("false_p") and data.get("verified"): - msg = "False positive findings cannot be verified." - raise serializers.ValidationError(msg) - - if "risk_accepted" in data and data.get("risk_accepted"): - test = data.get("test") - # test = Test.objects.get(id=test_id) - if not test.engagement.product.enable_simple_risk_acceptance: - msg = "Simple risk acceptance is disabled for this product, use the UI to accept this finding." - raise serializers.ValidationError(msg) + product_id = data.get("product", None) + endpoint_id = data.get("endpoint", None) + finding_id = data.get("finding", None) + metadata = data.get("metadata") - if ( - data.get("active") - and "risk_accepted" in data - and data.get("risk_accepted") - ): - msg = "Active findings cannot be risk accepted." - raise serializers.ValidationError(msg) + for item in metadata: + # this will only verify that one and only one of product, endpoint, or finding is passed... + DojoMeta(product=product_id, + endpoint=endpoint_id, + finding=finding_id, + name=item.get("name"), + value=item.get("value")).clean() return data - def validate_severity(self, value: str) -> str: - if value not in SEVERITIES: - msg = f"Severity must be one of the following: {SEVERITIES}" - raise serializers.ValidationError(msg) - return value - -class FindingTemplateSerializer(serializers.ModelSerializer): - tags = TagListSerializerField(required=False) - vulnerability_ids = serializers.SerializerMethodField() - endpoints = serializers.SerializerMethodField() - - class Meta: - model = Finding_Template - exclude = ("cve", "vulnerability_ids_text") - - @extend_schema_field(serializers.ListField(child=serializers.CharField())) - def get_vulnerability_ids(self, obj): - """Return vulnerability IDs as a list of strings.""" - return obj.vulnerability_ids - - @extend_schema_field(serializers.ListField(child=serializers.CharField())) - def get_endpoints(self, obj): - """Return endpoints as a list of URL strings.""" - return obj.endpoints if hasattr(obj, "endpoints") else [] - - def create(self, validated_data): - - # Handle vulnerability_ids if provided as list - vulnerability_ids = None - if "vulnerability_ids" in self.initial_data: - vulnerability_ids = self.initial_data.get("vulnerability_ids", []) - if isinstance(vulnerability_ids, str): - # If it's a string, split by newlines - vulnerability_ids = [vid.strip() for vid in vulnerability_ids.split("\n") if vid.strip()] - elif not isinstance(vulnerability_ids, list): - vulnerability_ids = [] - - # Handle endpoints if provided as list - endpoint_urls = None - if "endpoints" in self.initial_data: - endpoint_urls = self.initial_data.get("endpoints", []) - if isinstance(endpoint_urls, str): - # If it's a string, split by newlines - endpoint_urls = [url.strip() for url in endpoint_urls.split("\n") if url.strip()] - elif not isinstance(endpoint_urls, list): - endpoint_urls = [] - - new_finding_template = super().create( - validated_data, - ) - - # Save vulnerability IDs using helper - if vulnerability_ids: - save_vulnerability_ids_template(new_finding_template, vulnerability_ids) +# Engagement serializers live in dojo/engagement/api/serializer.py. +# EngagementSerializer is re-exported here because ReportGenerateSerializer and +# RiskAcceptanceSerializer (below) still reference it. The other engagement +# serializers are imported directly from dojo.engagement.api by their consumers. +from dojo.engagement.api.serializer import ( # noqa: E402, F401 -- backward compat + EngagementCheckListSerializer, + EngagementSerializer, +) +from dojo.file_uploads.api.serializer import ( # noqa: E402, F401 -- re-export; prefetcher + lazy consumers in finding/test/engagement + FileSerializer, + RawFileSerializer, +) +from dojo.note_type.api.serializer import NoteTypeSerializer # noqa: E402, F401 -- re-export for prefetcher discovery +from dojo.notes.api.serializer import ( # noqa: E402, F401 -- re-export; prefetcher + RiskAcceptanceToNotesSerializer + lazy consumers + NoteHistorySerializer, + NoteSerializer, +) - # Save endpoints using helper - if endpoint_urls: - save_endpoints_template(new_finding_template, endpoint_urls) +# Product serializers live in dojo/product/api/serializer.py. ProductSerializer is +# re-exported because ReportGenerateSerializer (below) still references it; +# ProductMetaSerializer because dojo/asset/api/serializers.py imports it. +# ProductAPIScanConfigurationSerializer is imported directly from +# dojo.product.api.serializer by its only consumer (the viewset). +from dojo.product.api.serializer import ( # noqa: E402 -- backward compat + ProductMetaSerializer, # noqa: F401 -- backward compat + ProductSerializer, +) +from dojo.product_type.api.serializer import ProductTypeSerializer # noqa: E402 +from dojo.user.api.serializer import ( # noqa: E402, F401 -- backward compat + prefetcher discovery + AddUserSerializer, + UserContactInfoSerializer, + UserProfileSerializer, + UserSerializer, + UserStubSerializer, +) - return new_finding_template - def update(self, instance, validated_data): - # Handle vulnerability_ids if provided - if "vulnerability_ids" in self.initial_data: - vulnerability_ids = self.initial_data.get("vulnerability_ids", []) - if isinstance(vulnerability_ids, str): - vulnerability_ids = [vid.strip() for vid in vulnerability_ids.split("\n") if vid.strip()] - elif not isinstance(vulnerability_ids, list): - vulnerability_ids = [] - save_vulnerability_ids_template(instance, vulnerability_ids) +class AppAnalysisSerializer(serializers.ModelSerializer): + tags = TagListSerializerField(required=False) - # Handle endpoints if provided - if "endpoints" in self.initial_data: - endpoint_urls = self.initial_data.get("endpoints", []) - if isinstance(endpoint_urls, str): - endpoint_urls = [url.strip() for url in endpoint_urls.split("\n") if url.strip()] - elif not isinstance(endpoint_urls, list): - endpoint_urls = [] - save_endpoints_template(instance, endpoint_urls) + class Meta: + model = App_Analysis + fields = "__all__" - return super().update(instance, validated_data) +from dojo.endpoint.api.serializer import ( # noqa: E402, F401 -- re-export; prefetcher discovery requires all moved ModelSerializers here + EndpointParamsSerializer, + EndpointSerializer, + EndpointStatusSerializer, +) +from dojo.jira.api.serializers import ( # noqa: E402, F401 -- backward compat re-export + JIRAInstanceSerializer, + JIRAIssueSerializer, + JIRAProjectSerializer, +) +from dojo.regulations.api.serializer import RegulationSerializer # noqa: E402, F401 -- re-export; prefetcher discovery +from dojo.tool_config.api.serializer import ToolConfigurationSerializer # noqa: E402, F401 -- re-export +from dojo.tool_product.api.serializer import ToolProductSettingsSerializer # noqa: E402, F401 -- re-export +from dojo.tool_type.api.serializer import ToolTypeSerializer # noqa: E402, F401 -- re-export; prefetcher discovery -class ProductSerializer(serializers.ModelSerializer): - findings_count = serializers.SerializerMethodField() - findings_list = serializers.SerializerMethodField() - business_criticality = serializers.ChoiceField(choices=Product.BUSINESS_CRITICALITY_CHOICES, allow_blank=True, allow_null=True, required=False) - platform = serializers.ChoiceField(choices=Product.PLATFORM_CHOICES, allow_blank=True, allow_null=True, required=False) - lifecycle = serializers.ChoiceField(choices=Product.LIFECYCLE_CHOICES, allow_blank=True, allow_null=True, required=False) - origin = serializers.ChoiceField(choices=Product.ORIGIN_CHOICES, allow_blank=True, allow_null=True, required=False) +class SonarqubeIssueSerializer(serializers.ModelSerializer): + class Meta: + model = Sonarqube_Issue + fields = "__all__" - tags = TagListSerializerField(required=False) - product_meta = ProductMetaSerializer(read_only=True, many=True) +class SonarqubeIssueTransitionSerializer(serializers.ModelSerializer): class Meta: - model = Product - exclude = ( - "tid", - "updated", - "async_updating", - ) + model = Sonarqube_Issue_Transition + fields = "__all__" - def validate(self, data): - async_updating = getattr(self.instance, "async_updating", None) - if async_updating: - new_sla_config = data.get("sla_configuration", None) - old_sla_config = getattr(self.instance, "sla_configuration", None) - if new_sla_config and old_sla_config and new_sla_config != old_sla_config: - msg = "Finding SLA expiration dates are currently being recalculated. The SLA configuration for this product cannot be changed until the calculation is complete." - raise serializers.ValidationError(msg) - return data - def get_findings_count(self, obj) -> int: - return obj.findings_count +from dojo.development_environment.api.serializer import ( # noqa: E402 -- re-export; prefetcher discovery + DevelopmentEnvironmentSerializer, # noqa: F401 -- re-export; prefetcher discovery +) - # TODO: maybe extend_schema_field is needed here? - def get_findings_list(self, obj) -> list[int]: - return obj.open_findings_list() +# Risk acceptance serializers live in dojo/risk_acceptance/api/serializer.py. Re-exported here +# for backward compat: RiskAcceptanceSerializer is lazy-imported by dojo/finding/api/serializer.py +# (schema overrides); the ModelSerializers must also stay discoverable by the prefetcher. +from dojo.risk_acceptance.api.serializer import ( # noqa: E402 -- backward compat / prefetcher discovery + RiskAcceptanceProofSerializer, # noqa: F401 + RiskAcceptanceSerializer, # noqa: F401 -- lazy-imported by finding schema overrides + prefetcher + RiskAcceptanceToNotesSerializer, # noqa: F401 +) +from dojo.test.api.serializer import ( # noqa: E402, F401 -- backward compat re-export + TestCreateSerializer, + TestSerializer, + TestTypeCreateSerializer, + TestTypeSerializer, +) class CommonImportScanSerializer(serializers.Serializer): @@ -2297,71 +865,7 @@ def save(self, *, push_to_jira=False): self.process_scan(auto_create_manager, data, context) -class EndpointMetaImporterSerializer(serializers.Serializer): - file = serializers.FileField(required=True) - create_endpoints = serializers.BooleanField(default=True, required=False) - create_tags = serializers.BooleanField(default=True, required=False) - create_dojo_meta = serializers.BooleanField(default=False, required=False) - product_name = serializers.CharField(required=False) - product = serializers.PrimaryKeyRelatedField( - queryset=Product.objects.all(), required=False, - ) - # extra fields populated in response - # need to use the _id suffix as without the serializer framework gets - # confused - product_id = serializers.IntegerField(read_only=True) - - def validate(self, data): - file = data.get("file") - if file and is_scan_file_too_large(file): - msg = f"Report file is too large. Maximum supported size is {settings.SCAN_FILE_MAX_SIZE} MB" - raise serializers.ValidationError(msg) - - return data - - def save(self): - data = self.validated_data - file = data.get("file") - create_endpoints = data.get("create_endpoints", True) - create_tags = data.get("create_tags", True) - create_dojo_meta = data.get("create_dojo_meta", False) - auto_create = AutoCreateContextManager() - # Process the context to make an conversions needed. Catch any exceptions - # in this case and wrap them in a DRF exception - try: - auto_create.process_import_meta_data_from_dict(data) - # Get an existing product - product = auto_create.get_target_product_if_exists(**data) - if not product: - product = auto_create.get_target_product_by_id_if_exists(**data) - except (ValueError, TypeError) as e: - # Raise an explicit drf exception here - raise ValidationError(str(e)) - try: - if settings.V3_FEATURE_LOCATIONS: - endpoint_meta_import( - file, - product, - create_endpoints, - create_tags, - create_dojo_meta, - origin="API", - object_class=Location, - ) - else: - # TODO: Delete this after the move to Locations - endpoint_meta_import( - file, - product, - create_endpoints, - create_tags, - create_dojo_meta, - origin="API", - ) - except SyntaxError as se: - raise Exception(se) - except ValueError as ve: - raise Exception(ve) +from dojo.endpoint.api.serializer import EndpointMetaImporterSerializer # noqa: E402, F401 -- re-export class LanguageTypeSerializer(serializers.ModelSerializer): @@ -2460,87 +964,6 @@ class Meta: fields = "__all__" -class FindingToNotesSerializer(serializers.Serializer): - finding_id = serializers.PrimaryKeyRelatedField( - queryset=Finding.objects.all(), many=False, allow_null=True, - ) - notes = NoteSerializer(many=True) - - -class FindingToFilesSerializer(serializers.Serializer): - finding_id = serializers.PrimaryKeyRelatedField( - queryset=Finding.objects.all(), many=False, allow_null=True, - ) - files = FileSerializer(many=True) - - def to_representation(self, data): - finding = data.get("finding_id") - files = data.get("files") - new_files = [{ - "id": file.id, - "file": "{site_url}/{file_access_url}".format( - site_url=settings.SITE_URL, - file_access_url=file.get_accessible_url( - finding, finding.id, - ), - ), - "title": file.title, - } for file in files] - return {"finding_id": finding.id, "files": new_files} - - -class FindingCloseSerializer(serializers.ModelSerializer): - is_mitigated = serializers.BooleanField(required=False) - mitigated = serializers.DateTimeField(required=False) - false_p = serializers.BooleanField(required=False) - out_of_scope = serializers.BooleanField(required=False) - duplicate = serializers.BooleanField(required=False) - mitigated_by = serializers.PrimaryKeyRelatedField(required=False, allow_null=True, queryset=Dojo_User.objects.all()) - note = serializers.CharField(required=False, allow_blank=True) - note_type = serializers.PrimaryKeyRelatedField(required=False, allow_null=True, queryset=Note_Type.objects.all()) - - class Meta: - model = Finding - fields = ( - "is_mitigated", - "mitigated", - "false_p", - "out_of_scope", - "duplicate", - "mitigated_by", - "note", - "note_type", - ) - - def validate(self, data): - request = self.context.get("request") - request_user = getattr(request, "user", None) - - mitigated_by_user = data.get("mitigated_by") - if mitigated_by_user is not None: - # Require permission to edit mitigated metadata - if not (request_user and finding_helper.can_edit_mitigated_data(request_user)): - raise serializers.ValidationError({ - "mitigated_by": ["Not allowed to set mitigated_by."], - }) - - # Ensure selected user is authorized (Finding_Edit) - authorized_users = get_authorized_users("edit", user=request_user) - if not authorized_users.filter(id=mitigated_by_user.id).exists(): - raise serializers.ValidationError({ - "mitigated_by": [ - "Selected user is not authorized to be set as mitigated_by.", - ], - }) - - return data - - -class FindingVerifySerializer(serializers.Serializer): - note = serializers.CharField(required=False, allow_blank=True) - note_type = serializers.PrimaryKeyRelatedField(required=False, allow_null=True, queryset=Note_Type.objects.all()) - - class ReportGenerateOptionSerializer(serializers.Serializer): include_finding_notes = serializers.BooleanField(default=False) include_finding_images = serializers.BooleanField(default=False) @@ -2562,6 +985,31 @@ class ExecutiveSummarySerializer(serializers.Serializer): total_findings = serializers.IntegerField() +# Finding serializers live in dojo/finding/api/serializer.py. FindingSerializer and +# FindingToNotesSerializer are re-exported here because ReportGenerateSerializer +# (below) still references them. The remaining finding serializers are re-exported so +# they remain discoverable as members of this module by the prefetcher +# (dojo/api_v2/prefetch/prefetcher.py inspects this module to build its model->serializer +# map); changing that membership would silently change prefetch responses. +from dojo.finding.api.serializer import ( # noqa: E402 -- backward compat + BurpRawRequestResponseMultiSerializer, # noqa: F401 -- backward compat / prefetcher discovery + BurpRawRequestResponseSerializer, # noqa: F401 -- backward compat + FindingCloseSerializer, # noqa: F401 -- backward compat / prefetcher discovery + FindingCreateSerializer, # noqa: F401 -- backward compat / prefetcher discovery + FindingEngagementSerializer, # noqa: F401 -- backward compat / prefetcher discovery + FindingEnvironmentSerializer, # noqa: F401 -- backward compat / prefetcher discovery + FindingGroupSerializer, # noqa: F401 -- backward compat / prefetcher discovery + FindingMetaSerializer, # noqa: F401 -- backward compat / prefetcher discovery + FindingProdTypeSerializer, # noqa: F401 -- backward compat / prefetcher discovery + FindingProductSerializer, # noqa: F401 -- backward compat / prefetcher discovery + FindingSerializer, + FindingTemplateSerializer, # noqa: F401 -- backward compat / prefetcher discovery + FindingTestTypeSerializer, # noqa: F401 -- backward compat / prefetcher discovery + FindingToNotesSerializer, + VulnerabilityIdSerializer, # noqa: F401 -- backward compat / prefetcher discovery +) + + class ReportGenerateSerializer(serializers.Serializer): executive_summary = ExecutiveSummarySerializer(many=False, allow_null=True) product_type = ProductTypeSerializer(many=False, read_only=True) @@ -2592,10 +1040,7 @@ class TagSerializer(serializers.Serializer): tags = TagListSerializerField(required=True) -class SystemSettingsSerializer(serializers.ModelSerializer): - class Meta: - model = System_Settings - fields = "__all__" +from dojo.system_settings.api.serializer import SystemSettingsSerializer # noqa: E402, F401 -- backward compat class CeleryStatusSerializer(serializers.Serializer): @@ -2618,19 +1063,9 @@ class CeleryQueueTaskDetailSerializer(serializers.Serializer): latest_expires = serializers.CharField(allow_null=True, read_only=True) -class FindingNoteSerializer(serializers.Serializer): - note_id = serializers.IntegerField() - - from dojo.notifications.api.serializer import NotificationsSerializer # noqa: E402, F401 -- backward compat -class EngagementPresetsSerializer(serializers.ModelSerializer): - class Meta: - model = Engagement_Presets - fields = "__all__" - - class NetworkLocationsSerializer(serializers.ModelSerializer): class Meta: model = Network_Locations @@ -2656,11 +1091,6 @@ def validate(self, data): return data -class UserProfileSerializer(serializers.Serializer): - user = UserSerializer(many=False) - user_contact_info = UserContactInfoSerializer(many=False, required=False) - - class DeletePreviewSerializer(serializers.Serializer): model = serializers.CharField(read_only=True) id = serializers.IntegerField(read_only=True, allow_null=True) @@ -2673,21 +1103,7 @@ class Meta: exclude = ("content_type",) -class AnnouncementSerializer(serializers.ModelSerializer): - - class Meta: - model = Announcement - fields = "__all__" - - def create(self, validated_data): - validated_data["id"] = 1 - try: - return super().create(validated_data) - except IntegrityError as e: - if 'duplicate key value violates unique constraint "dojo_announcement_pkey"' in str(e): - msg = "No more than one Announcement is allowed" - raise serializers.ValidationError(msg) - raise - - +from dojo.announcement.api.serializer import ( # noqa: E402 -- re-export; prefetcher discovery + AnnouncementSerializer, # noqa: F401 -- re-export; prefetcher discovery +) from dojo.notifications.api.serializer import NotificationWebhooksSerializer # noqa: E402, F401 -- backward compat diff --git a/dojo/api_v2/views.py b/dojo/api_v2/views.py index 3f3663ecb7f..3f3070f2fcb 100644 --- a/dojo/api_v2/views.py +++ b/dojo/api_v2/views.py @@ -1,42 +1,29 @@ -import base64 import logging -import mimetypes from datetime import datetime -from pathlib import Path import pghistory -import tagulous -from crum import get_current_user from dateutil.relativedelta import relativedelta from django.conf import settings from django.contrib.auth.models import Permission from django.core.exceptions import ValidationError from django.db import IntegrityError -from django.db.models import OuterRef, Value -from django.db.models.functions import Coalesce from django.db.models.query import QuerySet as DjangoQuerySet -from django.http import FileResponse -from django.shortcuts import get_object_or_404 -from django.urls import reverse from django.utils import timezone from django_filters.rest_framework import DjangoFilterBackend from drf_spectacular.renderers import OpenApiJsonRenderer2 from drf_spectacular.types import OpenApiTypes from drf_spectacular.utils import ( OpenApiParameter, - OpenApiResponse, extend_schema, extend_schema_view, ) from drf_spectacular.views import SpectacularAPIView from rest_framework import mixins, status, viewsets from rest_framework.decorators import action -from rest_framework.generics import GenericAPIView from rest_framework.parsers import MultiPartParser from rest_framework.permissions import DjangoModelPermissions, IsAuthenticated from rest_framework.response import Response -import dojo.finding.helper as finding_helper from dojo.api_v2 import ( mixins as dojo_mixins, ) @@ -44,114 +31,53 @@ prefetch, serializers, ) -from dojo.api_v2.prefetch.prefetcher import _Prefetcher from dojo.authorization import api_permissions as permissions from dojo.authorization.authorization import user_has_permission_or_403 -from dojo.celery_dispatch import dojo_dispatch_task -from dojo.endpoint.queries import ( - get_authorized_endpoint_status, - get_authorized_endpoints, -) -from dojo.endpoint.views import get_endpoint_ids -from dojo.engagement.queries import get_authorized_engagements -from dojo.engagement.services import close_engagement, reopen_engagement +from dojo.endpoint.ui.views import get_endpoint_ids from dojo.filters import ( ApiAppAnalysisFilter, ApiDojoMetaFilter, - ApiEndpointFilter, - ApiEngagementFilter, - ApiFindingFilter, - ApiProductFilter, - ApiRiskAcceptanceFilter, - ApiTemplateFindingFilter, - ApiTestFilter, - ApiUserFilter, +) +from dojo.finding.ui.filters import ( ReportFindingFilter, ReportFindingFilterWithoutObjectLookups, - TestImportAPIFilter, -) -from dojo.finding.queries import ( - get_authorized_findings, -) -from dojo.finding.views import ( - duplicate_cluster, - reset_finding_duplicate_status_internal, - set_finding_as_original_internal, ) from dojo.importers.auto_create_context import AutoCreateContextManager from dojo.jira import services as jira_services from dojo.labels import get_labels from dojo.models import ( - Announcement, App_Analysis, - BurpRawRequestResponse, - Check_List, - Development_Environment, Dojo_User, DojoMeta, Endpoint, - Endpoint_Status, - Engagement, - Engagement_Presets, - FileUpload, Finding, - Finding_Template, Language_Type, Languages, Network_Locations, - Note_Type, - NoteHistory, - Notes, Product, - Product_API_Scan_Configuration, - Product_Type, - Regulation, - Risk_Acceptance, SLA_Configuration, Sonarqube_Issue, Sonarqube_Issue_Transition, System_Settings, Test, - Test_Import, - Test_Type, - Tool_Configuration, - Tool_Product_Settings, - Tool_Type, - User, - UserContactInfo, ) from dojo.product.queries import ( get_authorized_app_analysis, get_authorized_dojo_meta, - get_authorized_engagement_presets, get_authorized_languages, - get_authorized_product_api_scan_configurations, get_authorized_products, ) -from dojo.product_type.queries import ( - get_authorized_product_types, -) -from dojo.query_utils import build_count_subquery -from dojo.reports.views import ( +from dojo.reports.ui.views import ( prefetch_related_findings_for_report, report_url_resolver, ) -from dojo.risk_acceptance import api as ra_api -from dojo.risk_acceptance.helper import remove_finding_from_risk_acceptance -from dojo.risk_acceptance.queries import get_authorized_risk_acceptances -from dojo.test.queries import get_authorized_test_imports, get_authorized_tests -from dojo.tool_product.queries import get_authorized_tool_product_settings -from dojo.user.authentication import reset_token_for_user +from dojo.test.queries import get_authorized_tests from dojo.user.utils import get_configuration_permissions_codenames from dojo.utils import ( - async_delete, - generate_file_response, get_celery_queue_details, get_celery_queue_length, get_celery_worker_status, - get_setting, get_system_setting, - process_tag_notifications, purge_celery_queue, purge_celery_queue_by_task_name, ) @@ -197,2121 +123,196 @@ def get_indent(self, accepted_media_type, renderer_context): class DojoSpectacularAPIView(SpectacularAPIView): - renderer_classes = [DojoOpenApiJsonRenderer, *SpectacularAPIView.renderer_classes] - - -class DojoModelViewSet( - viewsets.ModelViewSet, - dojo_mixins.DeletePreviewModelMixin, -): - pass - - -class PrefetchDojoModelViewSet( - prefetch.PrefetchListMixin, - prefetch.PrefetchRetrieveMixin, - DojoModelViewSet, -): - pass - - -class DeprecationNoticeMixin: - - deprecated: bool | None = None - end_of_life_date: datetime | None = None - - def finalize_response(self, request, response, *args, **kwargs): - if self.deprecated is not None: - response["X-Deprecated"] = self.deprecated - if self.end_of_life_date is not None: - response["X-End-Of-Life-Date"] = self.end_of_life_date.isoformat() - return super().finalize_response(request, response, *args, **kwargs) - - -# Authorization: authenticated users -# Authorization: object-based -# @extend_schema_view(**schema_with_prefetch()) -# Nested models with prefetch make the response schema too long for Swagger UI -class EndPointViewSet( - PrefetchDojoModelViewSet, -): - serializer_class = serializers.EndpointSerializer - queryset = Endpoint.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_class = ApiEndpointFilter - - permission_classes = ( - IsAuthenticated, - permissions.UserHasEndpointPermission, - ) - - def get_queryset(self): - active_finding_subquery = build_count_subquery( - Finding.objects.filter(endpoints=OuterRef("pk"), active=True), - group_field="endpoints", - ) - return get_authorized_endpoints("view").annotate( - active_finding_count=Coalesce(active_finding_subquery, Value(0)), - ).distinct() - - @extend_schema( - request=serializers.ReportGenerateOptionSerializer, - responses={status.HTTP_200_OK: serializers.ReportGenerateSerializer}, - ) - @action( - detail=True, methods=["post"], - # IsAuthenticated only: report generation requires View permission, - # enforced by the permission-filtered get_queryset(). The viewset's - # permission_classes would check Edit (POST), which is too restrictive. - permission_classes=[IsAuthenticated], - ) - def generate_report(self, request, pk=None): - endpoint = self.get_object() - - options = {} - # prepare post data - report_options = serializers.ReportGenerateOptionSerializer( - data=request.data, - ) - if report_options.is_valid(): - options["include_finding_notes"] = report_options.validated_data[ - "include_finding_notes" - ] - options["include_finding_images"] = report_options.validated_data[ - "include_finding_images" - ] - options[ - "include_executive_summary" - ] = report_options.validated_data["include_executive_summary"] - options[ - "include_table_of_contents" - ] = report_options.validated_data["include_table_of_contents"] - else: - return Response( - report_options.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - data = report_generate(request, endpoint, options) - report = serializers.ReportGenerateSerializer(data) - return Response(report.data) - - -# Authorization: object-based -# @extend_schema_view(**schema_with_prefetch()) -# Nested models with prefetch make the response schema too long for Swagger UI -class EndpointStatusViewSet( - PrefetchDojoModelViewSet, -): - serializer_class = serializers.EndpointStatusSerializer - queryset = Endpoint_Status.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = [ - "mitigated", - "false_positive", - "out_of_scope", - "risk_accepted", - "mitigated_by", - "finding", - "endpoint", - ] - - permission_classes = ( - IsAuthenticated, - permissions.UserHasEndpointStatusPermission, - ) - - def get_queryset(self): - return get_authorized_endpoint_status( - "view", - ).distinct() - - -# Authorization: object-based -# @extend_schema_view(**schema_with_prefetch()) -# Nested models with prefetch make the response schema too long for Swagger UI -class EngagementViewSet( - # PrefetchDojoModelViewSet, - DojoModelViewSet, - ra_api.AcceptedRisksMixin, -): - serializer_class = serializers.EngagementSerializer - queryset = Engagement.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_class = ApiEngagementFilter - - permission_classes = ( - IsAuthenticated, - permissions.UserHasEngagementPermission, - ) - - @property - def risk_application_model_class(self): - return Engagement - - def destroy(self, request, *args, **kwargs): - instance = self.get_object() - if get_setting("ASYNC_OBJECT_DELETE"): - async_del = async_delete() - async_del.delete(instance) - else: - instance.delete() - return Response(status=status.HTTP_204_NO_CONTENT) - - def get_queryset(self): - return ( - get_authorized_engagements("view") - .prefetch_related("notes", "risk_acceptance", "files") - .distinct() - ) - - @extend_schema( - request=OpenApiTypes.NONE, responses={status.HTTP_200_OK: ""}, - ) - @action(detail=True, methods=["post"], permission_classes=(IsAuthenticated, permissions.UserHasEngagementRelatedObjectPermission)) - def close(self, request, pk=None): - eng = self.get_object() - close_engagement(eng) - return Response({}, status=status.HTTP_200_OK) - - @extend_schema( - request=OpenApiTypes.NONE, responses={status.HTTP_200_OK: ""}, - ) - @action(detail=True, methods=["post"], permission_classes=(IsAuthenticated, permissions.UserHasEngagementRelatedObjectPermission)) - def reopen(self, request, pk=None): - eng = self.get_object() - reopen_engagement(eng) - return Response({}, status=status.HTTP_200_OK) - - @extend_schema( - request=serializers.ReportGenerateOptionSerializer, - responses={status.HTTP_200_OK: serializers.ReportGenerateSerializer}, - ) - @action( - detail=True, methods=["post"], - # IsAuthenticated only: report generation requires View permission, - # enforced by the permission-filtered get_queryset(). The viewset's - # permission_classes would check Edit (POST), which is too restrictive. - permission_classes=[IsAuthenticated], - ) - def generate_report(self, request, pk=None): - engagement = self.get_object() - - options = {} - # prepare post data - report_options = serializers.ReportGenerateOptionSerializer( - data=request.data, - ) - if report_options.is_valid(): - options["include_finding_notes"] = report_options.validated_data[ - "include_finding_notes" - ] - options["include_finding_images"] = report_options.validated_data[ - "include_finding_images" - ] - options[ - "include_executive_summary" - ] = report_options.validated_data["include_executive_summary"] - options[ - "include_table_of_contents" - ] = report_options.validated_data["include_table_of_contents"] - else: - return Response( - report_options.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - data = report_generate(request, engagement, options) - report = serializers.ReportGenerateSerializer(data) - return Response(report.data) - - @extend_schema( - methods=["GET"], - responses={ - status.HTTP_200_OK: serializers.EngagementToNotesSerializer, - }, - ) - @extend_schema( - methods=["POST"], - request=serializers.AddNewNoteOptionSerializer, - responses={status.HTTP_201_CREATED: serializers.NoteSerializer}, - ) - @action(detail=True, methods=["get", "post"], permission_classes=[IsAuthenticated, permissions.UserHasEngagementNotePermission]) - def notes(self, request, pk=None): - engagement = self.get_object() - if request.method == "POST": - new_note = serializers.AddNewNoteOptionSerializer( - data=request.data, - ) - if new_note.is_valid(): - entry = new_note.validated_data["entry"] - private = new_note.validated_data.get("private", False) - note_type = new_note.validated_data.get("note_type", None) - else: - return Response( - new_note.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - notes = engagement.notes.filter(note_type=note_type).first() - if notes and note_type and note_type.is_single: - return Response("Only one instance of this note_type allowed on an engagement.", status=status.HTTP_400_BAD_REQUEST) - - author = request.user - note = Notes( - entry=entry, - author=author, - private=private, - note_type=note_type, - ) - note.save() - # Add an entry to the note history - history = NoteHistory.objects.create(data=note.entry, time=note.date, current_editor=note.author) - note.history.add(history) - # Now add the note to the object - engagement.notes.add(note) - # Determine if we need to send any notifications for user mentioned - process_tag_notifications( - request=request, - note=note, - parent_url=request.build_absolute_uri( - reverse("view_engagement", args=(engagement.id,)), - ), - parent_title=f"Engagement: {engagement.name}", - ) - - serialized_note = serializers.NoteSerializer( - {"author": author, "entry": entry, "private": private}, - ) - return Response( - serialized_note.data, status=status.HTTP_201_CREATED, - ) - notes = engagement.notes.all() - - serialized_notes = serializers.EngagementToNotesSerializer( - {"engagement_id": engagement, "notes": notes}, - ) - return Response(serialized_notes.data, status=status.HTTP_200_OK) - - @extend_schema( - methods=["GET"], - responses={ - status.HTTP_200_OK: serializers.EngagementToFilesSerializer, - }, - ) - @extend_schema( - methods=["POST"], - request=serializers.AddNewFileOptionSerializer, - responses={status.HTTP_201_CREATED: serializers.FileSerializer}, - ) - @action( - detail=True, methods=["get", "post"], parser_classes=(MultiPartParser,), permission_classes=[IsAuthenticated, permissions.UserHasEngagementRelatedObjectPermission], - ) - def files(self, request, pk=None): - engagement = self.get_object() - if request.method == "POST": - new_file = serializers.FileSerializer(data=request.data) - if new_file.is_valid(): - title = new_file.validated_data["title"] - file = new_file.validated_data["file"] - else: - return Response( - new_file.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - file = FileUpload(title=title, file=file) - file.save() - engagement.files.add(file) - - serialized_file = serializers.FileSerializer(file) - return Response( - serialized_file.data, status=status.HTTP_201_CREATED, - ) - - files = engagement.files.all() - serialized_files = serializers.EngagementToFilesSerializer( - {"engagement_id": engagement, "files": files}, - ) - return Response(serialized_files.data, status=status.HTTP_200_OK) - - @extend_schema( - methods=["POST"], - request=serializers.EngagementCheckListSerializer, - responses={ - status.HTTP_201_CREATED: serializers.EngagementCheckListSerializer, - }, - ) - @action(detail=True, methods=["get", "post"], permission_classes=[IsAuthenticated, permissions.UserHasEngagementRelatedObjectPermission]) - def complete_checklist(self, request, pk=None): - engagement = self.get_object() - check_lists = Check_List.objects.filter(engagement=engagement) - if request.method == "POST": - if check_lists.count() > 0: - return Response( - { - "message": "A completed checklist for this engagement already exists.", - }, - status=status.HTTP_400_BAD_REQUEST, - ) - check_list = serializers.EngagementCheckListSerializer( - data=request.data, - ) - if not check_list.is_valid(): - return Response( - check_list.errors, status=status.HTTP_400_BAD_REQUEST, - ) - check_list = Check_List(**check_list.data) - check_list.engagement = engagement - check_list.save() - serialized_check_list = serializers.EngagementCheckListSerializer( - check_list, - ) - return Response( - serialized_check_list.data, status=status.HTTP_201_CREATED, - ) - prefetch_params = request.GET.get("prefetch", "").split(",") - prefetcher = _Prefetcher() - entry = check_lists.first() - # Get the queried object representation - result = serializers.EngagementCheckListSerializer(entry).data - prefetcher._prefetch(entry, prefetch_params) - result["prefetch"] = prefetcher.prefetched_data - return Response(result, status=status.HTTP_200_OK) - - @extend_schema( - methods=["GET"], - responses={ - status.HTTP_200_OK: serializers.RawFileSerializer, - }, - ) - @action( - detail=True, - methods=["get"], - url_path=r"files/download/(?P\d+)", - permission_classes=[IsAuthenticated, permissions.UserHasEngagementRelatedObjectPermission], - ) - def download_file(self, request, file_id, pk=None): - engagement = self.get_object() - # Get the file object - file_object_qs = engagement.files.filter(id=file_id) - file_object = ( - file_object_qs.first() if len(file_object_qs) > 0 else None - ) - if file_object is None: - return Response( - {"error": "File ID not associated with Engagement"}, - status=status.HTTP_404_NOT_FOUND, - ) - # send file - return generate_file_response(file_object) - - @extend_schema( - request=serializers.EngagementUpdateJiraEpicSerializer, - responses={status.HTTP_200_OK: serializers.EngagementUpdateJiraEpicSerializer}, - ) - @action( - detail=True, methods=["post"], - permission_classes=(IsAuthenticated, permissions.UserHasEngagementRelatedObjectPermission), - ) - def update_jira_epic(self, request, pk=None): - engagement = self.get_object() - try: - if engagement.has_jira_issue: - task = jira_services.get_epic_task("update_epic") - if task: - dojo_dispatch_task(task, engagement.id, **request.data) - response = Response( - {"info": "Jira Epic update query sent"}, - status=status.HTTP_200_OK, - ) - else: - task = jira_services.get_epic_task("add_epic") - if task: - dojo_dispatch_task(task, engagement.id, **request.data) - response = Response( - {"info": "Jira Epic create query sent"}, - status=status.HTTP_200_OK, - ) - except ValidationError: - return Response( - {"error": "Bad Request!"}, - status=status.HTTP_400_BAD_REQUEST, - ) - return response - - -# @extend_schema_view(**schema_with_prefetch()) -# Nested models with prefetch make the response schema too long for Swagger UI -class RiskAcceptanceViewSet( - PrefetchDojoModelViewSet, -): - serializer_class = serializers.RiskAcceptanceSerializer - queryset = Risk_Acceptance.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_class = ApiRiskAcceptanceFilter - - permission_classes = ( - IsAuthenticated, - permissions.UserHasRiskAcceptancePermission, - ) - - def destroy(self, request, pk=None): - instance = self.get_object() - # Remove any findings on the risk acceptance - for finding in instance.accepted_findings.all(): - remove_finding_from_risk_acceptance(request.user, instance, finding) - # return the response of the object being deleted - return super().destroy(request, pk=pk) - - def get_queryset(self): - return ( - get_authorized_risk_acceptances("edit") - .prefetch_related( - "notes", "engagement_set", "owner", "accepted_findings", - ) - .distinct() - ) - - @extend_schema( - methods=["GET"], - responses={ - status.HTTP_200_OK: serializers.RiskAcceptanceToNotesSerializer, - }, - ) - @extend_schema( - methods=["POST"], - request=serializers.AddNewNoteOptionSerializer, - responses={status.HTTP_201_CREATED: serializers.NoteSerializer}, - ) - @action(detail=True, methods=["get", "post"], permission_classes=(IsAuthenticated, permissions.UserHasRiskAcceptanceRelatedObjectPermission)) - def notes(self, request, pk=None): - risk_acceptance = self.get_object() - if request.method == "POST": - new_note = serializers.AddNewNoteOptionSerializer(data=request.data) - if new_note.is_valid(): - entry = new_note.validated_data["entry"] - private = new_note.validated_data.get("private", False) - note_type = new_note.validated_data.get("note_type", None) - else: - return Response(new_note.errors, status=status.HTTP_400_BAD_REQUEST) - - notes = risk_acceptance.notes.filter(note_type=note_type).first() - if notes and note_type and note_type.is_single: - return Response("Only one instance of this note_type allowed on a risk acceptance.", status=status.HTTP_400_BAD_REQUEST) - - author = request.user - note = Notes(entry=entry, author=author, private=private, note_type=note_type) - note.save() - history = NoteHistory.objects.create(data=note.entry, time=note.date, current_editor=note.author) - note.history.add(history) - risk_acceptance.notes.add(note) - engagement = risk_acceptance.engagement - if engagement: - process_tag_notifications( - request=request, - note=note, - parent_url=request.build_absolute_uri( - reverse("view_risk_acceptance", args=(engagement.id, risk_acceptance.id)), - ), - parent_title=f"Risk Acceptance: {risk_acceptance.name}", - ) - - serialized_note = serializers.NoteSerializer( - {"author": author, "entry": entry, "private": private}, - ) - return Response(serialized_note.data, status=status.HTTP_201_CREATED) - - notes = risk_acceptance.notes.all() - serialized_notes = serializers.RiskAcceptanceToNotesSerializer( - {"risk_acceptance_id": risk_acceptance, "notes": notes}, - ) - return Response(serialized_notes.data, status=status.HTTP_200_OK) - - @extend_schema( - methods=["GET"], - responses={ - status.HTTP_200_OK: serializers.RiskAcceptanceProofSerializer, - }, - ) - @action(detail=True, methods=["get"], permission_classes=(IsAuthenticated, permissions.UserHasRiskAcceptanceRelatedObjectPermission)) - def download_proof(self, request, pk=None): - risk_acceptance = self.get_object() - # Get the file object - file_object = risk_acceptance.path - if file_object is None or risk_acceptance.filename() is None: - return Response( - {"error": "Proof has not provided to this risk acceptance..."}, - status=status.HTTP_404_NOT_FOUND, - ) - # Get the path of the file in media root - file_path = Path(settings.MEDIA_ROOT) / file_object.name - # NOTE: FileResponse takes ownership of closing the file handle when the response is closed. - # Explicitly register the closer to avoid potential resource leaks and satisfy static analyzers. - file_handle = file_path.open("rb") - # send file - response = FileResponse( - file_handle, - content_type=mimetypes.guess_type(str(file_path))[0] or "application/octet-stream", - status=status.HTTP_200_OK, - ) - if hasattr(response, "_resource_closers"): - response._resource_closers.append(file_handle.close) - response["Content-Length"] = file_object.size - response[ - "Content-Disposition" - ] = f'attachment; filename="{risk_acceptance.filename()}"' - - return response - - -# These are technologies in the UI and the API! -# Authorization: object-based -@extend_schema_view(**schema_with_prefetch()) -class AppAnalysisViewSet( - PrefetchDojoModelViewSet, -): - serializer_class = serializers.AppAnalysisSerializer - queryset = App_Analysis.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_class = ApiAppAnalysisFilter - - permission_classes = ( - IsAuthenticated, - permissions.UserHasAppAnalysisPermission, - ) - - def get_queryset(self): - return get_authorized_app_analysis("view") - - -# Authorization: configuration -class FindingTemplatesViewSet( - DojoModelViewSet, -): - serializer_class = serializers.FindingTemplateSerializer - queryset = Finding_Template.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_class = ApiTemplateFindingFilter - permission_classes = (permissions.UserHasConfigurationPermissionStaff,) - - def get_queryset(self): - return Finding_Template.objects.all().order_by("id") - - -# Authorization: object-based -@extend_schema_view( - list=extend_schema( - parameters=[ - OpenApiParameter( - "related_fields", - OpenApiTypes.BOOL, - OpenApiParameter.QUERY, - required=False, - description="Expand finding external relations (engagement, environment, product, \ - product_type, test, test_type)", - ), - OpenApiParameter( - "prefetch", - OpenApiTypes.STR, - OpenApiParameter.QUERY, - required=False, - description="List of fields for which to prefetch model instances and add those to the response", - ), - ], - ), - retrieve=extend_schema( - parameters=[ - OpenApiParameter( - "related_fields", - OpenApiTypes.BOOL, - OpenApiParameter.QUERY, - required=False, - description="Expand finding external relations (engagement, environment, product, \ - product_type, test, test_type)", - ), - OpenApiParameter( - "prefetch", - OpenApiTypes.STR, - OpenApiParameter.QUERY, - required=False, - description="List of fields for which to prefetch model instances and add those to the response", - ), - ], - ), -) -class FindingViewSet( - prefetch.PrefetchListMixin, - prefetch.PrefetchRetrieveMixin, - mixins.UpdateModelMixin, - mixins.DestroyModelMixin, - mixins.CreateModelMixin, - ra_api.AcceptedFindingsMixin, - viewsets.GenericViewSet, - dojo_mixins.DeletePreviewModelMixin, -): - serializer_class = serializers.FindingSerializer - queryset = Finding.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_class = ApiFindingFilter - permission_classes = ( - IsAuthenticated, - permissions.UserHasFindingPermission, - ) - - # Overriding mixins.UpdateModeMixin perform_update() method to grab push_to_jira - # data and add that as a parameter to .save() - def perform_update(self, serializer): - # IF JIRA is enabled and this product has a JIRA configuration - push_to_jira = serializer.validated_data.get("push_to_jira") - jira_project = jira_services.get_project(serializer.instance) - if get_system_setting("enable_jira") and jira_project: - push_to_jira = push_to_jira or jira_project.push_all_issues - - serializer.save(push_to_jira=push_to_jira) - - def get_queryset(self): - if settings.V3_FEATURE_LOCATIONS: - findings = get_authorized_findings( - "view", - ).prefetch_related( - "locations__location__url", - "reviewers", - "found_by", - "notes", - "risk_acceptance_set", - "test", - "tags", - "jira_issue", - "finding_group_set", - "files", - "burprawrequestresponse_set", - "status_finding", - "finding_meta", - "test__test_type", - "test__engagement", - "test__environment", - "test__engagement__product", - "test__engagement__product__prod_type", - ) - else: - # TODO: Delete this after the move to Locations - findings = get_authorized_findings( - "view", - ).prefetch_related( - "endpoints", - "reviewers", - "found_by", - "notes", - "risk_acceptance_set", - "test", - "tags", - "jira_issue", - "finding_group_set", - "files", - "burprawrequestresponse_set", - "status_finding", - "finding_meta", - "test__test_type", - "test__engagement", - "test__environment", - "test__engagement__product", - "test__engagement__product__prod_type", - ) - - return findings.distinct() - - def get_serializer_class(self): - if self.request and self.request.method == "POST": - return serializers.FindingCreateSerializer - return serializers.FindingSerializer - - @extend_schema( - methods=["POST"], - request=serializers.FindingCloseSerializer, - responses={status.HTTP_200_OK: serializers.FindingCloseSerializer}, - ) - @action(detail=True, methods=["post"], permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission)) - def close(self, request, pk=None): - finding = self.get_object() - - if request.method == "POST": - finding_close = serializers.FindingCloseSerializer( - data=request.data, - context={"request": request}, - ) - if finding_close.is_valid(): - # Remove the prefetched tags to avoid issues with delegating to celery - finding.tags._remove_prefetched_objects() - # Use shared helper to perform close operations - finding_helper.close_finding( - finding=finding, - user=request.user, - is_mitigated=finding_close.validated_data["is_mitigated"], - mitigated=(finding_close.validated_data.get("mitigated") if finding_helper.can_edit_mitigated_data(request.user) else timezone.now()), - mitigated_by=finding_close.validated_data.get("mitigated_by") or (request.user if not finding_helper.can_edit_mitigated_data(request.user) else None), - false_p=finding_close.validated_data.get("false_p", False), - out_of_scope=finding_close.validated_data.get("out_of_scope", False), - duplicate=finding_close.validated_data.get("duplicate", False), - note_entry=finding_close.validated_data.get("note"), - note_type=finding_close.validated_data.get("note_type"), - ) - else: - return Response( - finding_close.errors, status=status.HTTP_400_BAD_REQUEST, - ) - serialized_finding = serializers.FindingCloseSerializer(finding, context={"request": request}) - return Response(serialized_finding.data) - - @extend_schema( - methods=["POST"], - request=serializers.FindingVerifySerializer, - responses={status.HTTP_200_OK: serializers.FindingSerializer}, - ) - @action(detail=True, methods=["post"], permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission)) - def verify(self, request, pk=None): - finding = self.get_object() - - serializer = serializers.FindingVerifySerializer(data=request.data) - if not serializer.is_valid(): - return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST) - - # Remove prefetched tags to keep queryset state in sync - finding.tags._remove_prefetched_objects() - - finding_helper.verify_finding( - finding=finding, - user=request.user, - note_entry=serializer.validated_data.get("note"), - note_type=serializer.validated_data.get("note_type"), - ) - - serialized_finding = serializers.FindingSerializer(finding, context={"request": request}) - return Response(serialized_finding.data) - - @extend_schema( - methods=["GET"], - responses={status.HTTP_200_OK: serializers.TagSerializer}, - ) - @extend_schema( - methods=["POST"], - request=serializers.TagSerializer, - responses={status.HTTP_201_CREATED: serializers.TagSerializer}, - ) - @action(detail=True, methods=["get", "post"], permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission)) - def tags(self, request, pk=None): - finding = self.get_object() - - if request.method == "POST": - new_tags = serializers.TagSerializer(data=request.data) - if new_tags.is_valid(): - all_tags = finding.tags - all_tags = serializers.TagSerializer({"tags": all_tags}).data[ - "tags" - ] - for tag in new_tags.validated_data["tags"]: - for sub_tag in tagulous.utils.parse_tags(tag): - if sub_tag not in all_tags: - all_tags.append(sub_tag) - - new_tags = tagulous.utils.render_tags(all_tags) - - finding.tags = new_tags - finding.save() - else: - return Response( - new_tags.errors, status=status.HTTP_400_BAD_REQUEST, - ) - tags = finding.tags - serialized_tags = serializers.TagSerializer({"tags": tags}) - return Response(serialized_tags.data) - - @extend_schema( - methods=["GET"], - responses={ - status.HTTP_200_OK: serializers.BurpRawRequestResponseSerializer, - }, - ) - @extend_schema( - methods=["POST"], - request=serializers.BurpRawRequestResponseSerializer, - responses={ - status.HTTP_201_CREATED: serializers.BurpRawRequestResponseSerializer, - }, - ) - @action(detail=True, methods=["get", "post"], permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission)) - def request_response(self, request, pk=None): - finding = self.get_object() - - if request.method == "POST": - burps = serializers.BurpRawRequestResponseSerializer( - data=request.data, many=isinstance(request.data, list), - ) - if burps.is_valid(): - for pair in burps.validated_data["req_resp"]: - burp_rr = BurpRawRequestResponse( - finding=finding, - burpRequestBase64=base64.b64encode( - pair["request"].encode("utf-8"), - ), - burpResponseBase64=base64.b64encode( - pair["response"].encode("utf-8"), - ), - ) - burp_rr.clean() - burp_rr.save() - else: - return Response( - burps.errors, status=status.HTTP_400_BAD_REQUEST, - ) - # Not necessarily Burp scan specific - these are just any request/response pairs - burp_req_resp = BurpRawRequestResponse.objects.filter(finding=finding) - var = settings.MAX_REQRESP_FROM_API - if var > -1: - burp_req_resp = burp_req_resp[:var] - - burp_list = [] - for burp in burp_req_resp: - request = burp.get_request() - response = burp.get_response() - burp_list.append({"request": request, "response": response}) - serialized_burps = serializers.BurpRawRequestResponseSerializer( - {"req_resp": burp_list}, - ) - return Response(serialized_burps.data) - - @extend_schema( - methods=["GET"], - responses={status.HTTP_200_OK: serializers.FindingToNotesSerializer}, - ) - @extend_schema( - methods=["POST"], - request=serializers.AddNewNoteOptionSerializer, - responses={status.HTTP_201_CREATED: serializers.NoteSerializer}, - ) - @action(detail=True, methods=["get", "post"], permission_classes=(IsAuthenticated, permissions.UserHasFindingNotePermission)) - def notes(self, request, pk=None): - finding = self.get_object() - if request.method == "POST": - new_note = serializers.AddNewNoteOptionSerializer( - data=request.data, - ) - if new_note.is_valid(): - entry = new_note.validated_data["entry"] - private = new_note.validated_data.get("private", False) - note_type = new_note.validated_data.get("note_type", None) - else: - return Response( - new_note.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - if finding.notes: - notes = finding.notes.filter(note_type=note_type).first() - if notes and note_type and note_type.is_single: - return Response("Only one instance of this note_type allowed on a finding.", status=status.HTTP_400_BAD_REQUEST) - - author = request.user - note = Notes( - entry=entry, - author=author, - private=private, - note_type=note_type, - ) - note.save() - # Add an entry to the note history - history = NoteHistory.objects.create(data=note.entry, time=note.date, current_editor=note.author) - note.history.add(history) - # Now add the note to the object - finding.last_reviewed = note.date - finding.last_reviewed_by = author - finding.save(update_fields=["last_reviewed", "last_reviewed_by", "updated"]) - finding.notes.add(note) - # Determine if we need to send any notifications for user mentioned - process_tag_notifications( - request=request, - note=note, - parent_url=request.build_absolute_uri( - reverse("view_finding", args=(finding.id,)), - ), - parent_title=f"Finding: {finding.title}", - ) - - if finding.has_jira_issue: - jira_services.add_comment(finding, note) - elif finding.has_jira_group_issue: - jira_services.add_comment(finding.finding_group, note) - - serialized_note = serializers.NoteSerializer( - {"author": author, "entry": entry, "private": private}, - ) - return Response( - serialized_note.data, status=status.HTTP_201_CREATED, - ) - notes = finding.notes.all() - - serialized_notes = serializers.FindingToNotesSerializer( - {"finding_id": finding, "notes": notes}, - ) - return Response(serialized_notes.data, status=status.HTTP_200_OK) - - @extend_schema( - methods=["GET"], - responses={status.HTTP_200_OK: serializers.FindingToFilesSerializer}, - ) - @extend_schema( - methods=["POST"], - request=serializers.AddNewFileOptionSerializer, - responses={status.HTTP_201_CREATED: serializers.FileSerializer}, - ) - @action( - detail=True, methods=["get", "post"], parser_classes=(MultiPartParser,), permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission), - ) - def files(self, request, pk=None): - finding = self.get_object() - if request.method == "POST": - new_file = serializers.FileSerializer(data=request.data) - if new_file.is_valid(): - title = new_file.validated_data["title"] - file = new_file.validated_data["file"] - else: - return Response( - new_file.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - file = FileUpload(title=title, file=file) - file.save() - finding.files.add(file) - - serialized_file = serializers.FileSerializer(file) - return Response( - serialized_file.data, status=status.HTTP_201_CREATED, - ) - - files = finding.files.all() - serialized_files = serializers.FindingToFilesSerializer( - {"finding_id": finding, "files": files}, - ) - return Response(serialized_files.data, status=status.HTTP_200_OK) - - @extend_schema( - methods=["GET"], - responses={ - status.HTTP_200_OK: serializers.RawFileSerializer, - }, - ) - @action( - detail=True, - methods=["get"], - url_path=r"files/download/(?P\d+)", permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission), - ) - def download_file(self, request, file_id, pk=None): - finding = self.get_object() - # Get the file object - file_object_qs = finding.files.filter(id=file_id) - file_object = ( - file_object_qs.first() if len(file_object_qs) > 0 else None - ) - if file_object is None: - return Response( - {"error": "File ID not associated with Finding"}, - status=status.HTTP_404_NOT_FOUND, - ) - # send file - return generate_file_response(file_object) - - @extend_schema( - request=serializers.FindingNoteSerializer, - responses={status.HTTP_204_NO_CONTENT: ""}, - ) - @action(detail=True, methods=["patch"], permission_classes=(IsAuthenticated, permissions.UserHasFindingNotePermission)) - def remove_note(self, request, pk=None): - """Remove Note From Finding Note""" - finding = self.get_object() - notes = finding.notes.all() - if request.data["note_id"]: - note = get_object_or_404(Notes.objects, id=request.data["note_id"]) - if note not in notes: - return Response( - {"error": "Selected Note is not assigned to this Finding"}, - status=status.HTTP_400_BAD_REQUEST, - ) - else: - return Response( - {"error": "('note_id') parameter missing"}, - status=status.HTTP_400_BAD_REQUEST, - ) - if ( - note.author.username == request.user.username - or request.user.is_superuser - ): - finding.notes.remove(note) - note.delete() - else: - return Response( - {"error": "Delete Failed, You are not the Note's author"}, - status=status.HTTP_400_BAD_REQUEST, - ) - - return Response( - {"Success": "Selected Note has been Removed successfully"}, - status=status.HTTP_204_NO_CONTENT, - ) - - @extend_schema( - methods=["PUT", "PATCH"], - request=serializers.TagSerializer, - responses={status.HTTP_204_NO_CONTENT: ""}, - ) - @action(detail=True, methods=["put", "patch"], permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission)) - def remove_tags(self, request, pk=None): - """Remove Tag(s) from finding list of tags""" - finding = self.get_object() - delete_tags = serializers.TagSerializer(data=request.data) - if delete_tags.is_valid(): - all_tags = finding.tags - all_tags = serializers.TagSerializer({"tags": all_tags}).data[ - "tags" - ] - - # serializer turns it into a string, but we need a list - del_tags = delete_tags.validated_data["tags"] - if len(del_tags) < 1: - return Response( - {"error": "Empty Tag List Not Allowed"}, - status=status.HTTP_400_BAD_REQUEST, - ) - - for tag in del_tags: - if tag not in all_tags: - return Response( - { - "error": f"'{tag}' is not a valid tag in list '{all_tags}'", - }, - status=status.HTTP_400_BAD_REQUEST, - ) - all_tags.remove(tag) - new_tags = tagulous.utils.render_tags(all_tags) - finding.tags = new_tags - finding.save() - return Response( - {"success": "Tag(s) Removed"}, - status=status.HTTP_204_NO_CONTENT, - ) - return Response( - delete_tags.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - @extend_schema( - responses={ - status.HTTP_200_OK: serializers.FindingSerializer(many=True), - }, - ) - @action( - detail=True, - methods=["get"], - url_path=r"duplicate", - filter_backends=[], - pagination_class=None, - permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission), - ) - def get_duplicate_cluster(self, request, pk): - finding = self.get_object() - result = duplicate_cluster(request, finding) - serializer = serializers.FindingSerializer( - instance=result, many=True, context={"request": request}, - ) - return Response(serializer.data, status=status.HTTP_200_OK) - - @extend_schema( - request=OpenApiTypes.NONE, - responses={status.HTTP_204_NO_CONTENT: ""}, - ) - @action(detail=True, methods=["post"], url_path=r"duplicate/reset", permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission)) - def reset_finding_duplicate_status(self, request, pk): - self.get_object() - checked_duplicate_id = reset_finding_duplicate_status_internal( - request.user, pk, - ) - if checked_duplicate_id is None: - return Response(status=status.HTTP_400_BAD_REQUEST) - return Response(status=status.HTTP_204_NO_CONTENT) - - @extend_schema( - request=OpenApiTypes.NONE, - parameters=[ - OpenApiParameter( - "new_fid", OpenApiTypes.INT, OpenApiParameter.PATH, - ), - ], - responses={status.HTTP_204_NO_CONTENT: ""}, - ) - @action( - detail=True, methods=["post"], url_path=r"original/(?P\d+)", permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission), - ) - def set_finding_as_original(self, request, pk, new_fid): - self.get_object() - success = set_finding_as_original_internal(request.user, pk, new_fid) - if not success: - return Response(status=status.HTTP_400_BAD_REQUEST) - return Response(status=status.HTTP_204_NO_CONTENT) - - @extend_schema( - request=serializers.ReportGenerateOptionSerializer, - responses={status.HTTP_200_OK: serializers.ReportGenerateSerializer}, - ) - @action( - detail=False, methods=["post"], - # IsAuthenticated only: report generation requires View permission, - # enforced by the permission-filtered get_queryset(). The viewset's - # permission_classes would check Edit (POST), which is too restrictive. - permission_classes=[IsAuthenticated], - ) - def generate_report(self, request): - findings = self.get_queryset() - options = {} - # prepare post data - report_options = serializers.ReportGenerateOptionSerializer( - data=request.data, - ) - if report_options.is_valid(): - options["include_finding_notes"] = report_options.validated_data[ - "include_finding_notes" - ] - options["include_finding_images"] = report_options.validated_data[ - "include_finding_images" - ] - options[ - "include_executive_summary" - ] = report_options.validated_data["include_executive_summary"] - options[ - "include_table_of_contents" - ] = report_options.validated_data["include_table_of_contents"] - else: - return Response( - report_options.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - data = report_generate(request, findings, options) - report = serializers.ReportGenerateSerializer(data) - return Response(report.data) - - def _get_metadata(self, request, finding): - metadata = DojoMeta.objects.filter(finding=finding) - serializer = serializers.FindingMetaSerializer( - instance=metadata, many=True, - ) - return Response(serializer.data, status=status.HTTP_200_OK) - - def _edit_metadata(self, request, finding): - metadata_name = request.query_params.get("name", None) - if metadata_name is None: - return Response( - "Metadata name is required", status=status.HTTP_400_BAD_REQUEST, - ) - - try: - DojoMeta.objects.update_or_create( - name=metadata_name, - finding=finding, - defaults={ - "name": request.data.get("name"), - "value": request.data.get("value"), - }, - ) - - return Response(data=request.data, status=status.HTTP_200_OK) - except IntegrityError: - return Response( - "Update failed because the new name already exists", - status=status.HTTP_400_BAD_REQUEST, - ) - - def _add_metadata(self, request, finding): - metadata_data = serializers.FindingMetaSerializer(data=request.data) - - if metadata_data.is_valid(): - name = metadata_data.validated_data["name"] - value = metadata_data.validated_data["value"] - - metadata = DojoMeta(finding=finding, name=name, value=value) - try: - metadata.validate_unique() - metadata.save() - except ValidationError: - return Response( - "Create failed probably because the name of the metadata already exists", - status=status.HTTP_400_BAD_REQUEST, - ) - - return Response(data=metadata_data.data, status=status.HTTP_200_OK) - return Response( - metadata_data.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - def _remove_metadata(self, request, finding): - name = request.query_params.get("name", None) - if name is None: - return Response( - "A metadata name must be provided", - status=status.HTTP_400_BAD_REQUEST, - ) - - metadata = get_object_or_404( - DojoMeta.objects, finding=finding, name=name, - ) - metadata.delete() - - return Response("Metadata deleted", status=status.HTTP_200_OK) - - @extend_schema( - methods=["GET"], - responses={ - status.HTTP_200_OK: serializers.FindingMetaSerializer(many=True), - status.HTTP_404_NOT_FOUND: OpenApiResponse( - description="Returned if finding does not exist", - ), - }, - ) - @extend_schema( - methods=["DELETE"], - parameters=[ - OpenApiParameter( - "name", - OpenApiTypes.INT, - OpenApiParameter.QUERY, - required=True, - description="name of the metadata to retrieve. If name is empty, return all the \ - metadata associated with the finding", - ), - ], - responses={ - status.HTTP_200_OK: OpenApiResponse( - description="Returned if the metadata was correctly deleted", - ), - status.HTTP_404_NOT_FOUND: OpenApiResponse( - description="Returned if finding does not exist", - ), - status.HTTP_400_BAD_REQUEST: OpenApiResponse( - description="Returned if there was a problem with the metadata information", - ), - }, - ) - @extend_schema( - methods=["PUT"], - request=serializers.FindingMetaSerializer, - responses={ - status.HTTP_200_OK: serializers.FindingMetaSerializer, - status.HTTP_404_NOT_FOUND: OpenApiResponse( - description="Returned if finding does not exist", - ), - status.HTTP_400_BAD_REQUEST: OpenApiResponse( - description="Returned if there was a problem with the metadata information", - ), - }, - ) - @extend_schema( - methods=["POST"], - request=serializers.FindingMetaSerializer, - responses={ - status.HTTP_200_OK: serializers.FindingMetaSerializer, - status.HTTP_404_NOT_FOUND: OpenApiResponse( - description="Returned if finding does not exist", - ), - status.HTTP_400_BAD_REQUEST: OpenApiResponse( - description="Returned if there was a problem with the metadata information", - ), - }, - ) - @action( - detail=True, - methods=["post", "put", "delete", "get"], - filter_backends=[], - pagination_class=None, - permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission), - ) - def metadata(self, request, pk=None): - finding = self.get_object() - - if request.method == "GET": - return self._get_metadata(request, finding) - if request.method == "POST": - return self._add_metadata(request, finding) - if request.method in {"PUT", "PATCH"}: - return self._edit_metadata(request, finding) - if request.method == "DELETE": - return self._remove_metadata(request, finding) - - return Response( - {"error", "unsupported method"}, status=status.HTTP_400_BAD_REQUEST, - ) - - -# Authorization: configuration -from dojo.jira.api.views import ( # noqa: E402, F401 backward compat - JiraInstanceViewSet, - JiraIssuesViewSet, - JiraProjectViewSet, -) - - -# Authorization: superuser -class SonarqubeIssueViewSet( - DojoModelViewSet, -): - serializer_class = serializers.SonarqubeIssueSerializer - queryset = Sonarqube_Issue.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = ["id", "key", "status", "type"] - permission_classes = (permissions.IsSuperUser, DjangoModelPermissions) - - def get_queryset(self): - return Sonarqube_Issue.objects.all().order_by("id") - - -# Authorization: superuser -class SonarqubeIssueTransitionViewSet( - DojoModelViewSet, -): - serializer_class = serializers.SonarqubeIssueTransitionSerializer - queryset = Sonarqube_Issue_Transition.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = [ - "id", - "sonarqube_issue", - "finding_status", - "sonarqube_status", - "transitions", - ] - permission_classes = (permissions.IsSuperUser, DjangoModelPermissions) - - def get_queryset(self): - return Sonarqube_Issue_Transition.objects.all().order_by("id") - - -# Authorization: object-based -@extend_schema_view(**schema_with_prefetch()) -class ProductAPIScanConfigurationViewSet( - PrefetchDojoModelViewSet, -): - serializer_class = serializers.ProductAPIScanConfigurationSerializer - queryset = Product_API_Scan_Configuration.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = [ - "id", - "product", - "tool_configuration", - "service_key_1", - "service_key_2", - "service_key_3", - ] - permission_classes = ( - IsAuthenticated, - permissions.UserHasProductAPIScanConfigurationPermission, - ) - - def get_queryset(self): - return get_authorized_product_api_scan_configurations( - "view", - ) - - -# Authorization: object-based -# @extend_schema_view(**schema_with_prefetch()) -# Nested models with prefetch make the response schema too long for Swagger UI -class DojoMetaViewSet( - PrefetchDojoModelViewSet, -): - serializer_class = serializers.MetaSerializer - queryset = DojoMeta.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_class = ApiDojoMetaFilter - permission_classes = ( - IsAuthenticated, - permissions.UserHasDojoMetaPermission, - ) - - def get_queryset(self): - return get_authorized_dojo_meta("view") - - @extend_schema( - methods=["post", "patch"], - request=serializers.MetaMainSerializer, - responses={status.HTTP_200_OK: serializers.MetaMainSerializer}, - filters=False, - ) - @action( - detail=False, methods=["post", "patch"], pagination_class=None, - ) - def batch(self, request, pk=None): - serialized_data = serializers.MetaMainSerializer(data=request.data) - if serialized_data.is_valid(raise_exception=True): - if request.method == "POST": - self.process_post(request) - status_code = status.HTTP_201_CREATED - if request.method == "PATCH": - self.process_patch(request) - status_code = status.HTTP_200_OK - - return Response(status=status_code, data=serialized_data.data) - - def _fetch_and_authorize_parents(self, request, permission_map): - """Fetch parent objects and verify the user has the required permissions.""" - data = request.data - parents = {} - for field, (model, permission) in permission_map.items(): - obj = model.objects.filter(id=data.get(field)).first() - if obj: - user_has_permission_or_403(request.user, obj, permission) - parents[field] = obj - return parents - - def process_post(self, request): - data = request.data - parents = self._fetch_and_authorize_parents(request, { - "product": (Product, "edit"), - "finding": (Finding, "edit"), - "endpoint": (Endpoint, "edit"), - }) - metalist = data.get("metadata") - for metadata in metalist: - try: - DojoMeta.objects.create( - product=parents["product"], - finding=parents["finding"], - endpoint=parents["endpoint"], - name=metadata.get("name"), - value=metadata.get("value"), - ) - except (IntegrityError) as ex: # this should not happen as the data was validated in the batch call - raise ValidationError(str(ex)) - - def process_patch(self, request): - data = request.data - parents = self._fetch_and_authorize_parents(request, { - "product": (Product, "edit"), - "finding": (Finding, "edit"), - "endpoint": (Endpoint, "edit"), - }) - metalist = data.get("metadata") - for metadata in metalist: - dojometa = DojoMeta.objects.filter(product=parents["product"], finding=parents["finding"], endpoint=parents["endpoint"], name=metadata.get("name")) - if dojometa: - try: - dojometa.update( - name=metadata.get("name"), - value=metadata.get("value"), - ) - except (IntegrityError) as ex: - raise ValidationError(str(ex)) - else: - msg = f"Metadata {metadata.get('name')} not found for object." - raise ValidationError(msg) - - -@extend_schema_view(**schema_with_prefetch()) -class ProductViewSet( - prefetch.PrefetchListMixin, - prefetch.PrefetchRetrieveMixin, - mixins.CreateModelMixin, - mixins.DestroyModelMixin, - mixins.UpdateModelMixin, - viewsets.GenericViewSet, - dojo_mixins.DeletePreviewModelMixin, -): - serializer_class = serializers.ProductSerializer - queryset = Product.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_class = ApiProductFilter - permission_classes = ( - IsAuthenticated, - permissions.UserHasProductPermission, - ) - - def get_queryset(self): - return get_authorized_products("view").distinct() - - def destroy(self, request, *args, **kwargs): - instance = self.get_object() - if get_setting("ASYNC_OBJECT_DELETE"): - async_del = async_delete() - async_del.delete(instance) - else: - with Endpoint.allow_endpoint_init(): # TODO: Delete this after the move to Locations - instance.delete() - return Response(status=status.HTTP_204_NO_CONTENT) - - # def list(self, request): - # # Note the use of `get_queryset()` instead of `self.queryset` - # queryset = self.get_queryset() - # serializer = self.serializer_class(queryset, many=True) - # return Response(serializer.data) - - @extend_schema( - request=serializers.ReportGenerateOptionSerializer, - responses={status.HTTP_200_OK: serializers.ReportGenerateSerializer}, - ) - @action( - detail=True, methods=["post"], - # IsAuthenticated only: report generation requires View permission, - # enforced by the permission-filtered get_queryset(). The viewset's - # permission_classes would check Edit (POST), which is too restrictive. - permission_classes=[IsAuthenticated], - ) - def generate_report(self, request, pk=None): - product = self.get_object() - - options = {} - # prepare post data - report_options = serializers.ReportGenerateOptionSerializer( - data=request.data, - ) - if report_options.is_valid(): - options["include_finding_notes"] = report_options.validated_data[ - "include_finding_notes" - ] - options["include_finding_images"] = report_options.validated_data[ - "include_finding_images" - ] - options[ - "include_executive_summary" - ] = report_options.validated_data["include_executive_summary"] - options[ - "include_table_of_contents" - ] = report_options.validated_data["include_table_of_contents"] - else: - return Response( - report_options.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - data = report_generate(request, product, options) - report = serializers.ReportGenerateSerializer(data) - return Response(report.data) - - -# Authorization: object-based -@extend_schema_view(**schema_with_prefetch()) -# Authorization: object-based -@extend_schema_view(**schema_with_prefetch()) -class ProductTypeViewSet( - PrefetchDojoModelViewSet, -): - serializer_class = serializers.ProductTypeSerializer - queryset = Product_Type.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = [ - "id", - "name", - "critical_product", - "key_product", - "created", - "updated", - ] - permission_classes = ( - IsAuthenticated, - permissions.UserHasProductTypePermission, - ) - - def get_queryset(self): - return get_authorized_product_types( - "view", - ).distinct() - - def destroy(self, request, *args, **kwargs): - instance = self.get_object() - if get_setting("ASYNC_OBJECT_DELETE"): - async_del = async_delete() - async_del.delete(instance) - else: - with Endpoint.allow_endpoint_init(): # TODO: Delete this after the move to Locations - instance.delete() - return Response(status=status.HTTP_204_NO_CONTENT) - - @extend_schema( - request=serializers.ReportGenerateOptionSerializer, - responses={status.HTTP_200_OK: serializers.ReportGenerateSerializer}, - ) - @action( - detail=True, methods=["post"], - # IsAuthenticated only: report generation requires View permission, - # enforced by the permission-filtered get_queryset(). The viewset's - # permission_classes would check Edit (POST), which is too restrictive. - permission_classes=[IsAuthenticated], - ) - def generate_report(self, request, pk=None): - product_type = self.get_object() - - options = {} - # prepare post data - report_options = serializers.ReportGenerateOptionSerializer( - data=request.data, - ) - if report_options.is_valid(): - options["include_finding_notes"] = report_options.validated_data[ - "include_finding_notes" - ] - options["include_finding_images"] = report_options.validated_data[ - "include_finding_images" - ] - options[ - "include_executive_summary" - ] = report_options.validated_data["include_executive_summary"] - options[ - "include_table_of_contents" - ] = report_options.validated_data["include_table_of_contents"] - else: - return Response( - report_options.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - data = report_generate(request, product_type, options) - report = serializers.ReportGenerateSerializer(data) - return Response(report.data) - - -# Authorization: authenticated, configuration -class DevelopmentEnvironmentViewSet( - DojoModelViewSet, -): - serializer_class = serializers.DevelopmentEnvironmentSerializer - queryset = Development_Environment.objects.none() - filter_backends = (DjangoFilterBackend,) - permission_classes = (IsAuthenticated, permissions.UserHasDevelopmentEnvironmentPermission) - - def get_queryset(self): - return Development_Environment.objects.all().order_by("id") - - -# Authorization: object-based -# @extend_schema_view(**schema_with_prefetch()) -# Nested models with prefetch make the response schema too long for Swagger UI -class TestsViewSet( - PrefetchDojoModelViewSet, - ra_api.AcceptedRisksMixin, -): - serializer_class = serializers.TestSerializer - queryset = Test.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_class = ApiTestFilter - permission_classes = (IsAuthenticated, permissions.UserHasTestPermission) - - @property - def risk_application_model_class(self): - return Test - - def get_queryset(self): - return ( - get_authorized_tests("view") - .prefetch_related("notes", "files") - .distinct() - ) - - def destroy(self, request, *args, **kwargs): - instance = self.get_object() - if get_setting("ASYNC_OBJECT_DELETE"): - async_del = async_delete() - async_del.delete(instance) - else: - instance.delete() - return Response(status=status.HTTP_204_NO_CONTENT) - - def get_serializer_class(self): - if self.request and self.request.method == "POST": - if self.action == "accept_risks": - return ra_api.AcceptedRiskSerializer - return serializers.TestCreateSerializer - return serializers.TestSerializer - - @extend_schema( - request=serializers.ReportGenerateOptionSerializer, - responses={status.HTTP_200_OK: serializers.ReportGenerateSerializer}, - ) - @action( - detail=True, methods=["post"], - # IsAuthenticated only: report generation requires View permission, - # enforced by the permission-filtered get_queryset(). The viewset's - # permission_classes would check Edit (POST), which is too restrictive. - permission_classes=[IsAuthenticated], - ) - def generate_report(self, request, pk=None): - test = self.get_object() - - options = {} - # prepare post data - report_options = serializers.ReportGenerateOptionSerializer( - data=request.data, - ) - if report_options.is_valid(): - options["include_finding_notes"] = report_options.validated_data[ - "include_finding_notes" - ] - options["include_finding_images"] = report_options.validated_data[ - "include_finding_images" - ] - options[ - "include_executive_summary" - ] = report_options.validated_data["include_executive_summary"] - options[ - "include_table_of_contents" - ] = report_options.validated_data["include_table_of_contents"] - else: - return Response( - report_options.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - data = report_generate(request, test, options) - report = serializers.ReportGenerateSerializer(data) - return Response(report.data) - - @extend_schema( - methods=["GET"], - responses={status.HTTP_200_OK: serializers.TestToNotesSerializer}, - ) - @extend_schema( - methods=["POST"], - request=serializers.AddNewNoteOptionSerializer, - responses={status.HTTP_201_CREATED: serializers.NoteSerializer}, - ) - @action(detail=True, methods=["get", "post"], permission_classes=(IsAuthenticated, permissions.UserHasTestNotePermission)) - def notes(self, request, pk=None): - test = self.get_object() - if request.method == "POST": - new_note = serializers.AddNewNoteOptionSerializer( - data=request.data, - ) - if new_note.is_valid(): - entry = new_note.validated_data["entry"] - private = new_note.validated_data.get("private", False) - note_type = new_note.validated_data.get("note_type", None) - else: - return Response( - new_note.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - notes = test.notes.filter(note_type=note_type).first() - if notes and note_type and note_type.is_single: - return Response("Only one instance of this note_type allowed on a test.", status=status.HTTP_400_BAD_REQUEST) - - author = request.user - note = Notes( - entry=entry, - author=author, - private=private, - note_type=note_type, - ) - note.save() - # Add an entry to the note history - history = NoteHistory.objects.create(data=note.entry, time=note.date, current_editor=note.author) - note.history.add(history) - # Now add the note to the object - test.notes.add(note) - # Determine if we need to send any notifications for user mentioned - process_tag_notifications( - request=request, - note=note, - parent_url=request.build_absolute_uri( - reverse("view_test", args=(test.id,)), - ), - parent_title=f"Test: {test.title}", - ) - - serialized_note = serializers.NoteSerializer( - {"author": author, "entry": entry, "private": private}, - ) - return Response( - serialized_note.data, status=status.HTTP_201_CREATED, - ) - notes = test.notes.all() - - serialized_notes = serializers.TestToNotesSerializer( - {"test_id": test, "notes": notes}, - ) - return Response(serialized_notes.data, status=status.HTTP_200_OK) - - @extend_schema( - methods=["GET"], - responses={status.HTTP_200_OK: serializers.TestToFilesSerializer}, - ) - @extend_schema( - methods=["POST"], - request=serializers.AddNewFileOptionSerializer, - responses={status.HTTP_201_CREATED: serializers.FileSerializer}, - ) - @action( - detail=True, methods=["get", "post"], parser_classes=(MultiPartParser,), permission_classes=(IsAuthenticated, permissions.UserHasTestRelatedObjectPermission), - ) - def files(self, request, pk=None): - test = self.get_object() - if request.method == "POST": - new_file = serializers.FileSerializer(data=request.data) - if new_file.is_valid(): - title = new_file.validated_data["title"] - file = new_file.validated_data["file"] - else: - return Response( - new_file.errors, status=status.HTTP_400_BAD_REQUEST, - ) - - file = FileUpload(title=title, file=file) - file.save() - test.files.add(file) - - serialized_file = serializers.FileSerializer(file) - return Response( - serialized_file.data, status=status.HTTP_201_CREATED, - ) - - files = test.files.all() - serialized_files = serializers.TestToFilesSerializer( - {"test_id": test, "files": files}, - ) - return Response(serialized_files.data, status=status.HTTP_200_OK) - - @extend_schema( - methods=["GET"], - responses={ - status.HTTP_200_OK: serializers.RawFileSerializer, - }, - ) - @action( - detail=True, - methods=["get"], - url_path=r"files/download/(?P\d+)", - permission_classes=(IsAuthenticated, permissions.UserHasTestRelatedObjectPermission), - ) - def download_file(self, request, file_id, pk=None): - test = self.get_object() - # Get the file object - file_object_qs = test.files.filter(id=file_id) - file_object = ( - file_object_qs.first() if len(file_object_qs) > 0 else None - ) - if file_object is None: - return Response( - {"error": "File ID not associated with Test"}, - status=status.HTTP_404_NOT_FOUND, - ) - # send file - return generate_file_response(file_object) - - -# Authorization: authenticated, configuration -class TestTypesViewSet( - mixins.UpdateModelMixin, - mixins.CreateModelMixin, - viewsets.ReadOnlyModelViewSet, -): - serializer_class = serializers.TestTypeSerializer - queryset = Test_Type.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = [ - "name", - ] - permission_classes = (IsAuthenticated, DjangoModelPermissions) - - def get_queryset(self): - return Test_Type.objects.all().order_by("id") - - def get_serializer_class(self): - if self.action == "create": - return serializers.TestTypeCreateSerializer - return serializers.TestTypeSerializer + renderer_classes = [DojoOpenApiJsonRenderer, *SpectacularAPIView.renderer_classes] -# @extend_schema_view(**schema_with_prefetch()) -# Nested models with prefetch make the response schema too long for Swagger UI -class TestImportViewSet( - PrefetchDojoModelViewSet, +class DojoModelViewSet( + viewsets.ModelViewSet, + dojo_mixins.DeletePreviewModelMixin, ): - serializer_class = serializers.TestImportSerializer - queryset = Test_Import.objects.none() - filter_backends = (DjangoFilterBackend,) + pass - filterset_class = TestImportAPIFilter - permission_classes = ( - IsAuthenticated, - permissions.UserHasTestImportPermission, - ) +class PrefetchDojoModelViewSet( + prefetch.PrefetchListMixin, + prefetch.PrefetchRetrieveMixin, + DojoModelViewSet, +): + pass - def get_queryset(self): - return get_authorized_test_imports( - "view", - ).prefetch_related( - "test_import_finding_action_set", - "findings_affected", - "findings_affected__endpoints", - "findings_affected__status_finding", - "findings_affected__finding_meta", - "findings_affected__jira_issue", - "findings_affected__burprawrequestresponse_set", - "findings_affected__jira_issue", - "findings_affected__jira_issue", - "findings_affected__jira_issue", - "findings_affected__reviewers", - "findings_affected__notes", - "findings_affected__notes__author", - "findings_affected__notes__history", - "findings_affected__files", - "findings_affected__found_by", - "findings_affected__tags", - "findings_affected__risk_acceptance_set", - "test", - "test__tags", - "test__notes", - "test__notes__author", - "test__files", - "test__test_type", - "test__engagement", - "test__environment", - "test__engagement__product", - "test__engagement__product__prod_type", - ) +class DeprecationNoticeMixin: -# Authorization: configurations -@extend_schema_view(**schema_with_prefetch()) -class ToolConfigurationsViewSet( - PrefetchDojoModelViewSet, -): - serializer_class = serializers.ToolConfigurationSerializer - queryset = Tool_Configuration.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = [ - "id", - "name", - "tool_type", - "url", - "authentication_type", - ] - permission_classes = (permissions.UserHasConfigurationPermissionSuperuser,) + deprecated: bool | None = None + end_of_life_date: datetime | None = None - def get_queryset(self): - return Tool_Configuration.objects.all().order_by("id") + def finalize_response(self, request, response, *args, **kwargs): + if self.deprecated is not None: + response["X-Deprecated"] = self.deprecated + if self.end_of_life_date is not None: + response["X-End-Of-Life-Date"] = self.end_of_life_date.isoformat() + return super().finalize_response(request, response, *args, **kwargs) +# @extend_schema_view(**schema_with_prefetch()) +# Nested models with prefetch make the response schema too long for Swagger UI +# These are technologies in the UI and the API! # Authorization: object-based @extend_schema_view(**schema_with_prefetch()) -class ToolProductSettingsViewSet( +class AppAnalysisViewSet( PrefetchDojoModelViewSet, ): - serializer_class = serializers.ToolProductSettingsSerializer - queryset = Tool_Product_Settings.objects.none() + serializer_class = serializers.AppAnalysisSerializer + queryset = App_Analysis.objects.none() filter_backends = (DjangoFilterBackend,) - filterset_fields = [ - "id", - "name", - "product", - "tool_configuration", - "tool_project_id", - "url", - ] + filterset_class = ApiAppAnalysisFilter + permission_classes = ( IsAuthenticated, - permissions.UserHasToolProductSettingsPermission, + permissions.UserHasAppAnalysisPermission, ) def get_queryset(self): - return get_authorized_tool_product_settings("view") + return get_authorized_app_analysis("view") # Authorization: configuration -class ToolTypesViewSet( +from dojo.jira.api.views import ( # noqa: E402, F401 backward compat + JiraInstanceViewSet, + JiraIssuesViewSet, + JiraProjectViewSet, +) + + +# Authorization: superuser +class SonarqubeIssueViewSet( DojoModelViewSet, ): - serializer_class = serializers.ToolTypeSerializer - queryset = Tool_Type.objects.none() + serializer_class = serializers.SonarqubeIssueSerializer + queryset = Sonarqube_Issue.objects.none() filter_backends = (DjangoFilterBackend,) - filterset_fields = ["id", "name", "description"] - permission_classes = (permissions.UserHasConfigurationPermissionSuperuser,) + filterset_fields = ["id", "key", "status", "type"] + permission_classes = (permissions.IsSuperUser, DjangoModelPermissions) def get_queryset(self): - return Tool_Type.objects.all().order_by("id") + return Sonarqube_Issue.objects.all().order_by("id") -# Authorization: authenticated, configuration -class RegulationsViewSet( +# Authorization: superuser +class SonarqubeIssueTransitionViewSet( DojoModelViewSet, ): - serializer_class = serializers.RegulationSerializer - queryset = Regulation.objects.none() + serializer_class = serializers.SonarqubeIssueTransitionSerializer + queryset = Sonarqube_Issue_Transition.objects.none() filter_backends = (DjangoFilterBackend,) - filterset_fields = ["id", "name", "description"] - permission_classes = (IsAuthenticated, permissions.UserHasRegulationPermission) + filterset_fields = [ + "id", + "sonarqube_issue", + "finding_status", + "sonarqube_status", + "transitions", + ] + permission_classes = (permissions.IsSuperUser, DjangoModelPermissions) def get_queryset(self): - return Regulation.objects.all().order_by("id") + return Sonarqube_Issue_Transition.objects.all().order_by("id") -# Authorization: configuration -class UsersViewSet( - DojoModelViewSet, +# Authorization: object-based +# @extend_schema_view(**schema_with_prefetch()) +# Nested models with prefetch make the response schema too long for Swagger UI +class DojoMetaViewSet( + PrefetchDojoModelViewSet, ): - serializer_class = serializers.UserSerializer - queryset = User.objects.none() + serializer_class = serializers.MetaSerializer + queryset = DojoMeta.objects.none() filter_backends = (DjangoFilterBackend,) - filterset_class = ApiUserFilter - permission_classes = (permissions.UserHasConfigurationPermissionSuperuser,) + filterset_class = ApiDojoMetaFilter + permission_classes = ( + IsAuthenticated, + permissions.UserHasDojoMetaPermission, + ) def get_queryset(self): - return User.objects.all().order_by("id") - - def destroy(self, request, *args, **kwargs): - instance = self.get_object() - if request.user == instance: - return Response( - "Users may not delete themselves", - status=status.HTTP_400_BAD_REQUEST, - ) - self.perform_destroy(instance) - return Response(status=status.HTTP_204_NO_CONTENT) + return get_authorized_dojo_meta("view") + @extend_schema( + methods=["post", "patch"], + request=serializers.MetaMainSerializer, + responses={status.HTTP_200_OK: serializers.MetaMainSerializer}, + filters=False, + ) @action( - detail=True, - methods=["post"], - url_path="reset_api_token", - permission_classes=(IsAuthenticated, permissions.IsSuperUserOrGlobalOwner), - filter_backends=[], - pagination_class=None, + detail=False, methods=["post", "patch"], pagination_class=None, ) - def reset_api_token(self, request, pk=None): - target_user = self.get_object() - reset_token_for_user(acting_user=request.user, target_user=target_user) - return Response(status=status.HTTP_204_NO_CONTENT) + def batch(self, request, pk=None): + serialized_data = serializers.MetaMainSerializer(data=request.data) + if serialized_data.is_valid(raise_exception=True): + if request.method == "POST": + self.process_post(request) + status_code = status.HTTP_201_CREATED + if request.method == "PATCH": + self.process_patch(request) + status_code = status.HTTP_200_OK + return Response(status=status_code, data=serialized_data.data) -# Authorization: superuser -@extend_schema_view(**schema_with_prefetch()) -class UserContactInfoViewSet( - PrefetchDojoModelViewSet, -): - serializer_class = serializers.UserContactInfoSerializer - queryset = UserContactInfo.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = "__all__" - permission_classes = (permissions.IsSuperUser, DjangoModelPermissions) + def _fetch_and_authorize_parents(self, request, permission_map): + """Fetch parent objects and verify the user has the required permissions.""" + data = request.data + parents = {} + for field, (model, permission) in permission_map.items(): + obj = model.objects.filter(id=data.get(field)).first() + if obj: + user_has_permission_or_403(request.user, obj, permission) + parents[field] = obj + return parents - def get_queryset(self): - return UserContactInfo.objects.all().order_by("id") + def process_post(self, request): + data = request.data + parents = self._fetch_and_authorize_parents(request, { + "product": (Product, "edit"), + "finding": (Finding, "edit"), + "endpoint": (Endpoint, "edit"), + }) + metalist = data.get("metadata") + for metadata in metalist: + try: + DojoMeta.objects.create( + product=parents["product"], + finding=parents["finding"], + endpoint=parents["endpoint"], + name=metadata.get("name"), + value=metadata.get("value"), + ) + except (IntegrityError) as ex: # this should not happen as the data was validated in the batch call + raise ValidationError(str(ex)) + def process_patch(self, request): + data = request.data + parents = self._fetch_and_authorize_parents(request, { + "product": (Product, "edit"), + "finding": (Finding, "edit"), + "endpoint": (Endpoint, "edit"), + }) + metalist = data.get("metadata") + for metadata in metalist: + dojometa = DojoMeta.objects.filter(product=parents["product"], finding=parents["finding"], endpoint=parents["endpoint"], name=metadata.get("name")) + if dojometa: + try: + dojometa.update( + name=metadata.get("name"), + value=metadata.get("value"), + ) + except (IntegrityError) as ex: + raise ValidationError(str(ex)) + else: + msg = f"Metadata {metadata.get('name')} not found for object." + raise ValidationError(msg) -# Authorization: authenticated users -class UserProfileView(GenericAPIView): - permission_classes = (IsAuthenticated,) - pagination_class = None - serializer_class = serializers.UserProfileSerializer - @action( - detail=True, methods=["get"], filter_backends=[], pagination_class=None, - ) - def get(self, request, _=None): - user = get_current_user() - user_contact_info = ( - user.usercontactinfo if hasattr(user, "usercontactinfo") else None - ) - serializer = serializers.UserProfileSerializer( - { - "user": user, - "user_contact_info": user_contact_info, - }, - many=False, - ) - return Response(serializer.data) +# DevelopmentEnvironmentViewSet moved to dojo/development_environment/api/views.py +# RegulationsViewSet moved to dojo/regulations/api/views.py # Authorization: authenticated users, DjangoModelPermissions @@ -2386,38 +387,6 @@ def get_queryset(self): return get_authorized_tests("import") -# Authorization: authenticated users, DjangoModelPermissions -class EndpointMetaImporterView( - mixins.CreateModelMixin, viewsets.GenericViewSet, -): - - """ - Imports a CSV file into a product to propagate arbitrary meta and tags on endpoints. - - By Names: - - Provide `product_name` of existing product - - By ID: - - Provide the id of the product in the `product` parameter - - In this scenario Defect Dojo will look up the product by the provided details. - """ - - serializer_class = serializers.EndpointMetaImporterSerializer - parser_classes = [MultiPartParser] - queryset = Product.objects.none() - permission_classes = ( - IsAuthenticated, - permissions.UserHasMetaImportPermission, - ) - - def perform_create(self, serializer): - serializer.save() - - def get_queryset(self): - return get_authorized_products("edit") - - # Authorization: configuration class LanguageTypeViewSet( DojoModelViewSet, @@ -2546,76 +515,8 @@ def perform_create(self, serializer): pghistory.context(test_id=test_id_from_response) -# Authorization: configuration -class NoteTypeViewSet( - DojoModelViewSet, -): - serializer_class = serializers.NoteTypeSerializer - queryset = Note_Type.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = [ - "id", - "name", - "description", - "is_single", - "is_active", - "is_mandatory", - ] - permission_classes = (permissions.UserHasConfigurationPermissionSuperuser,) - - def get_queryset(self): - return Note_Type.objects.all().order_by("id") - - -class BurpRawRequestResponseViewSet( - DojoModelViewSet, -): - serializer_class = serializers.BurpRawRequestResponseMultiSerializer - queryset = BurpRawRequestResponse.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = ["finding"] - permission_classes = ( - IsAuthenticated, - permissions.UserHasBurpRawRequestResponsePermission, - ) - - def get_queryset(self): - return ( - BurpRawRequestResponse.objects.filter( - finding__in=get_authorized_findings( - "view", - ), - ) - .exclude( - burpRequestBase64__exact=b"", - burpResponseBase64__exact=b"", - ) - .order_by("id") - ) - - -# Authorization: superuser -class NotesViewSet( - mixins.UpdateModelMixin, - viewsets.ReadOnlyModelViewSet, -): - serializer_class = serializers.NoteSerializer - queryset = Notes.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = [ - "id", - "entry", - "author", - "private", - "date", - "edited", - "edit_time", - "editor", - ] - permission_classes = (permissions.IsSuperUser, DjangoModelPermissions) - - def get_queryset(self): - return Notes.objects.all().order_by("id") +from dojo.note_type.api.views import NoteTypeViewSet # noqa: E402, F401 -- re-export; urls.py imports by name +from dojo.notes.api.views import NotesViewSet # noqa: E402, F401 -- re-export; urls.py imports by name def report_generate(request, obj, options): @@ -2881,21 +782,6 @@ def report_generate(request, obj, options): return result -# Authorization: superuser -class SystemSettingsViewSet( - mixins.ListModelMixin, mixins.UpdateModelMixin, viewsets.GenericViewSet, -): - - """Basic control over System Settings. Use 'id' 1 for PUT, PATCH operations""" - - permission_classes = (permissions.IsSuperUser, DjangoModelPermissions) - serializer_class = serializers.SystemSettingsSerializer - queryset = System_Settings.objects.none() - - def get_queryset(self): - return System_Settings.objects.all().order_by("id") - - class CeleryViewSet(viewsets.ViewSet): permission_classes = (permissions.IsSuperUser, DjangoModelPermissions) queryset = System_Settings.objects.none() @@ -2969,23 +855,6 @@ def queue_task_purge(self, request): return Response({"purged": purged}) -@extend_schema_view(**schema_with_prefetch()) -class EngagementPresetsViewset( - PrefetchDojoModelViewSet, -): - serializer_class = serializers.EngagementPresetsSerializer - queryset = Engagement_Presets.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = ["id", "title", "product"] - permission_classes = ( - IsAuthenticated, - permissions.UserHasEngagementPresetPermission, - ) - - def get_queryset(self): - return get_authorized_engagement_presets("view") - - class NetworkLocationsViewset( DojoModelViewSet, ): @@ -3027,15 +896,47 @@ def get_queryset(self): return SLA_Configuration.objects.all().order_by("id") -# Authorization: configuration -class AnnouncementViewSet( - DojoModelViewSet, -): - serializer_class = serializers.AnnouncementSerializer - queryset = Announcement.objects.none() - filter_backends = (DjangoFilterBackend,) - filterset_fields = "__all__" - permission_classes = (permissions.UserHasConfigurationPermissionStaff,) - - def get_queryset(self): - return Announcement.objects.all().order_by("id") +# AnnouncementViewSet moved to dojo/announcement/api/views.py + +# Backward-compat re-exports for external consumers (e.g. dojo-pro) that still +# import (or attribute-access) ViewSets from dojo.api_v2.views. The viewsets +# moved into per-module api/views packages, and those modules import base +# classes back from dojo.api_v2.views at import time. Eagerly importing them +# here would create entry-order-dependent circular imports, so expose them +# lazily via PEP 562 __getattr__ instead. +import importlib # noqa: E402 + +_LAZY_VIEWSET_EXPORTS = { + "AnnouncementViewSet": "dojo.announcement.api.views", + "EndPointViewSet": "dojo.endpoint.api.views", + "EndpointMetaImporterView": "dojo.endpoint.api.views", + "EndpointStatusViewSet": "dojo.endpoint.api.views", + "EngagementViewSet": "dojo.engagement.api.views", + "EngagementPresetsViewset": "dojo.engagement.api.views", + "BurpRawRequestResponseViewSet": "dojo.finding.api.views", + "FindingViewSet": "dojo.finding.api.views", + "FindingTemplatesViewSet": "dojo.finding.api.views", + "ProductAPIScanConfigurationViewSet": "dojo.product.api.views", + "ProductTypeViewSet": "dojo.product_type.api.views", + "RiskAcceptanceViewSet": "dojo.risk_acceptance.api.views", + "SystemSettingsViewSet": "dojo.system_settings.api.views", + "TestsViewSet": "dojo.test.api.views", + "TestTypesViewSet": "dojo.test.api.views", + "TestImportViewSet": "dojo.test.api.views", + "ToolConfigurationsViewSet": "dojo.tool_config.api.views", + "ToolProductSettingsViewSet": "dojo.tool_product.api.views", + "ToolTypesViewSet": "dojo.tool_type.api.views", + "DevelopmentEnvironmentViewSet": "dojo.development_environment.api.views", + "RegulationsViewSet": "dojo.regulations.api.views", + "UserContactInfoViewSet": "dojo.user.api.views", + "UsersViewSet": "dojo.user.api.views", + "UserProfileView": "dojo.user.api.views", +} + + +def __getattr__(name): + module_path = _LAZY_VIEWSET_EXPORTS.get(name) + if module_path is None: + msg = f"module 'dojo.api_v2.views' has no attribute {name!r}" + raise AttributeError(msg) + return getattr(importlib.import_module(module_path), name) diff --git a/dojo/asset/urls.py b/dojo/asset/urls.py index 1b71a03dddf..d072060fb6e 100644 --- a/dojo/asset/urls.py +++ b/dojo/asset/urls.py @@ -1,8 +1,8 @@ from django.conf import settings from django.urls import re_path -from dojo.engagement import views as dojo_engagement_views -from dojo.product import views +from dojo.engagement.ui import views as dojo_engagement_views +from dojo.product.ui import views from dojo.utils import redirect_view # TODO: remove the else: branch once v3 migration is complete diff --git a/dojo/auditlog/__init__.py b/dojo/auditlog/__init__.py index b37e10499b4..bf33db6edac 100644 --- a/dojo/auditlog/__init__.py +++ b/dojo/auditlog/__init__.py @@ -14,6 +14,7 @@ "configure_pghistory_triggers": "dojo.auditlog.services", "register_django_pghistory_models": "dojo.auditlog.services", "process_events_for_display": "dojo.auditlog.helpers", + "TAG_MODEL_MAPPING": "dojo.auditlog.helpers", "get_tracked_models": "dojo.auditlog.backfill", "process_model_backfill": "dojo.auditlog.backfill", } diff --git a/dojo/authorization/url_permissions.py b/dojo/authorization/url_permissions.py index 70ac4ab20bb..e59aea15ebd 100644 --- a/dojo/authorization/url_permissions.py +++ b/dojo/authorization/url_permissions.py @@ -62,7 +62,7 @@ "delete_api_scan_configuration": [("object", Product_API_Scan_Configuration, "delete", "pascid")], # ----------------------------------------------------------------------- - # Engagement (dojo/engagement/views.py -> dojo/engagement/urls.py) + # Engagement (dojo/engagement/ui/views.py -> dojo/engagement/ui/urls.py) # ----------------------------------------------------------------------- "edit_engagement": [("object", Engagement, "edit", "eid")], "delete_engagement": [("object", Engagement, "delete", "eid")], diff --git a/dojo/banner/__init__.py b/dojo/banner/__init__.py index e69de29bb2d..b0433151e3a 100644 --- a/dojo/banner/__init__.py +++ b/dojo/banner/__init__.py @@ -0,0 +1 @@ +import dojo.banner.admin # noqa: F401 diff --git a/dojo/banner/admin.py b/dojo/banner/admin.py new file mode 100644 index 00000000000..02dc5f7d321 --- /dev/null +++ b/dojo/banner/admin.py @@ -0,0 +1,5 @@ +from django.contrib import admin + +from dojo.banner.models import BannerConf + +admin.site.register(BannerConf) diff --git a/dojo/banner/models.py b/dojo/banner/models.py new file mode 100644 index 00000000000..7c885b8ea34 --- /dev/null +++ b/dojo/banner/models.py @@ -0,0 +1,7 @@ +from django.db import models +from django.utils.translation import gettext as _ + + +class BannerConf(models.Model): + banner_enable = models.BooleanField(default=False, null=True, blank=True) + banner_message = models.CharField(max_length=500, help_text=_("This message will be displayed on the login page. It can contain basic html tags, for example https://example.com"), default="") diff --git a/dojo/banner/ui/__init__.py b/dojo/banner/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/banner/ui/forms.py b/dojo/banner/ui/forms.py new file mode 100644 index 00000000000..78a1fbbcf4b --- /dev/null +++ b/dojo/banner/ui/forms.py @@ -0,0 +1,18 @@ +from django import forms + + +class LoginBanner(forms.Form): + banner_enable = forms.BooleanField( + label="Enable login banner", + initial=False, + required=False, + help_text="Tick this box to enable a text banner on the login page", + ) + + banner_message = forms.CharField( + required=False, + label="Message to display on the login page", + ) + + def clean(self): + return super().clean() diff --git a/dojo/banner/urls.py b/dojo/banner/ui/urls.py similarity index 82% rename from dojo/banner/urls.py rename to dojo/banner/ui/urls.py index c0b75f1ff77..3751ac59d63 100644 --- a/dojo/banner/urls.py +++ b/dojo/banner/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from dojo.banner import views +from dojo.banner.ui import views urlpatterns = [ re_path( diff --git a/dojo/banner/views.py b/dojo/banner/ui/views.py similarity index 94% rename from dojo/banner/views.py rename to dojo/banner/ui/views.py index 1bdf8ce2e68..03646c7f914 100644 --- a/dojo/banner/views.py +++ b/dojo/banner/ui/views.py @@ -5,8 +5,8 @@ from django.shortcuts import get_object_or_404, render from django.urls import reverse -from dojo.forms import LoginBanner -from dojo.models import BannerConf +from dojo.banner.models import BannerConf +from dojo.banner.ui.forms import LoginBanner from dojo.utils import add_breadcrumb logger = logging.getLogger(__name__) diff --git a/dojo/benchmark/__init__.py b/dojo/benchmark/__init__.py index e69de29bb2d..08cfc4447d9 100644 --- a/dojo/benchmark/__init__.py +++ b/dojo/benchmark/__init__.py @@ -0,0 +1 @@ +import dojo.benchmark.admin # noqa: F401 diff --git a/dojo/benchmark/admin.py b/dojo/benchmark/admin.py new file mode 100644 index 00000000000..288569dc768 --- /dev/null +++ b/dojo/benchmark/admin.py @@ -0,0 +1,15 @@ +from django.contrib import admin + +from dojo.benchmark.models import ( + Benchmark_Category, + Benchmark_Product, + Benchmark_Product_Summary, + Benchmark_Requirement, + Benchmark_Type, +) + +admin.site.register(Benchmark_Type) +admin.site.register(Benchmark_Requirement) +admin.site.register(Benchmark_Category) +admin.site.register(Benchmark_Product) +admin.site.register(Benchmark_Product_Summary) diff --git a/dojo/benchmark/models.py b/dojo/benchmark/models.py new file mode 100644 index 00000000000..184e9dc9b2d --- /dev/null +++ b/dojo/benchmark/models.py @@ -0,0 +1,100 @@ +from django.db import models +from django.utils.translation import gettext as _ + + +class Benchmark_Type(models.Model): + name = models.CharField(max_length=300) + version = models.CharField(max_length=15) + source = (("PCI", "PCI"), + ("OWASP ASVS", "OWASP ASVS"), + ("OWASP Mobile ASVS", "OWASP Mobile ASVS")) + benchmark_source = models.CharField(max_length=20, blank=False, + null=True, choices=source, + default="OWASP ASVS") + created = models.DateTimeField(auto_now_add=True, null=False) + updated = models.DateTimeField(auto_now=True) + enabled = models.BooleanField(default=True) + + def __str__(self): + return self.name + " " + self.version + + +class Benchmark_Category(models.Model): + type = models.ForeignKey("dojo.Benchmark_Type", verbose_name=_("Benchmark Type"), on_delete=models.CASCADE) + name = models.CharField(max_length=300) + objective = models.TextField() + references = models.TextField(blank=True, null=True) + enabled = models.BooleanField(default=True) + created = models.DateTimeField(auto_now_add=True, null=False) + updated = models.DateTimeField(auto_now=True) + + class Meta: + ordering = ("name",) + + def __str__(self): + return self.name + ": " + self.type.name + + +class Benchmark_Requirement(models.Model): + category = models.ForeignKey("dojo.Benchmark_Category", on_delete=models.CASCADE) + objective_number = models.CharField(max_length=15, null=True, blank=True) + objective = models.TextField() + references = models.TextField(blank=True, null=True) + level_1 = models.BooleanField(default=False) + level_2 = models.BooleanField(default=False) + level_3 = models.BooleanField(default=False) + enabled = models.BooleanField(default=True) + cwe_mapping = models.ManyToManyField("dojo.CWE", blank=True) + testing_guide = models.ManyToManyField("dojo.Testing_Guide", blank=True) + created = models.DateTimeField(auto_now_add=True, null=False) + updated = models.DateTimeField(auto_now=True) + + def __str__(self): + return str(self.objective_number) + ": " + self.category.name + + +class Benchmark_Product(models.Model): + product = models.ForeignKey("dojo.Product", on_delete=models.CASCADE) + control = models.ForeignKey("dojo.Benchmark_Requirement", on_delete=models.CASCADE) + pass_fail = models.BooleanField(default=False, verbose_name=_("Pass"), + help_text=_("Does the product meet the requirement?")) + enabled = models.BooleanField(default=True, + help_text=_("Applicable for this specific product.")) + notes = models.ManyToManyField("dojo.Notes", blank=True, editable=False) + created = models.DateTimeField(auto_now_add=True, null=False) + updated = models.DateTimeField(auto_now=True) + + class Meta: + unique_together = [("product", "control")] + + def __str__(self): + return self.product.name + ": " + self.control.objective_number + ": " + self.control.category.name + + +class Benchmark_Product_Summary(models.Model): + product = models.ForeignKey("dojo.Product", on_delete=models.CASCADE) + benchmark_type = models.ForeignKey("dojo.Benchmark_Type", on_delete=models.CASCADE) + asvs_level = (("Level 1", "Level 1"), + ("Level 2", "Level 2"), + ("Level 3", "Level 3")) + desired_level = models.CharField(max_length=15, + null=False, choices=asvs_level, + default="Level 1") + current_level = models.CharField(max_length=15, blank=True, + null=True, choices=asvs_level, + default="None") + asvs_level_1_benchmark = models.IntegerField(null=False, default=0, help_text=_("Total number of active benchmarks for this application.")) + asvs_level_1_score = models.IntegerField(null=False, default=0, help_text=_("ASVS Level 1 Score")) + asvs_level_2_benchmark = models.IntegerField(null=False, default=0, help_text=_("Total number of active benchmarks for this application.")) + asvs_level_2_score = models.IntegerField(null=False, default=0, help_text=_("ASVS Level 2 Score")) + asvs_level_3_benchmark = models.IntegerField(null=False, default=0, help_text=_("Total number of active benchmarks for this application.")) + asvs_level_3_score = models.IntegerField(null=False, default=0, help_text=_("ASVS Level 3 Score")) + publish = models.BooleanField(default=False, help_text=_("Publish score to Product.")) + created = models.DateTimeField(auto_now_add=True, null=False) + updated = models.DateTimeField(auto_now=True) + + class Meta: + unique_together = [("product", "benchmark_type")] + + def __str__(self): + return self.product.name + ": " + self.benchmark_type.name diff --git a/dojo/benchmark/signals.py b/dojo/benchmark/signals.py index 6f87fa320cd..f6d997698a7 100644 --- a/dojo/benchmark/signals.py +++ b/dojo/benchmark/signals.py @@ -3,7 +3,7 @@ from django.db.models.signals import pre_delete from django.dispatch import receiver -from dojo.models import Benchmark_Product +from dojo.benchmark.models import Benchmark_Product from dojo.notes.helper import delete_related_notes logger = logging.getLogger(__name__) diff --git a/dojo/benchmark/ui/__init__.py b/dojo/benchmark/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/benchmark/ui/forms.py b/dojo/benchmark/ui/forms.py new file mode 100644 index 00000000000..c4416af53f9 --- /dev/null +++ b/dojo/benchmark/ui/forms.py @@ -0,0 +1,37 @@ +from django import forms + +from dojo.benchmark.models import ( + Benchmark_Product, + Benchmark_Product_Summary, + Benchmark_Requirement, +) + + +class Benchmark_Product_SummaryForm(forms.ModelForm): + + class Meta: + model = Benchmark_Product_Summary + exclude = ["product", "current_level", "benchmark_type", "asvs_level_1_benchmark", "asvs_level_1_score", "asvs_level_2_benchmark", "asvs_level_2_score", "asvs_level_3_benchmark", "asvs_level_3_score"] + + +class DeleteBenchmarkForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Benchmark_Product_Summary + fields = ["id"] + + +class BenchmarkForm(forms.ModelForm): + + class Meta: + model = Benchmark_Product + exclude = ["product", "control"] + + +class Benchmark_RequirementForm(forms.ModelForm): + + class Meta: + model = Benchmark_Requirement + exclude = [""] diff --git a/dojo/benchmark/urls.py b/dojo/benchmark/ui/urls.py similarity index 96% rename from dojo/benchmark/urls.py rename to dojo/benchmark/ui/urls.py index 849e83c603c..3581ce165ab 100644 --- a/dojo/benchmark/urls.py +++ b/dojo/benchmark/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from . import views +from dojo.benchmark.ui import views urlpatterns = [ re_path( diff --git a/dojo/benchmark/views.py b/dojo/benchmark/ui/views.py similarity index 98% rename from dojo/benchmark/views.py rename to dojo/benchmark/ui/views.py index b1dc065692e..40bfde471d7 100644 --- a/dojo/benchmark/views.py +++ b/dojo/benchmark/ui/views.py @@ -8,15 +8,15 @@ from django.urls import reverse from django.utils.translation import gettext as _ -from dojo.forms import Benchmark_Product_SummaryForm, DeleteBenchmarkForm -from dojo.models import ( +from dojo.benchmark.models import ( Benchmark_Category, Benchmark_Product, Benchmark_Product_Summary, Benchmark_Requirement, Benchmark_Type, - Product, ) +from dojo.benchmark.ui.forms import Benchmark_Product_SummaryForm, DeleteBenchmarkForm +from dojo.models import Product from dojo.templatetags.display_tags import asvs_level from dojo.utils import ( Product_Tab, diff --git a/dojo/components/views.py b/dojo/components/views.py index 28e6f720ea8..96f6bcbf4a0 100644 --- a/dojo/components/views.py +++ b/dojo/components/views.py @@ -5,8 +5,8 @@ from django.shortcuts import render from dojo.components.sql_group_concat import Sql_GroupConcat -from dojo.filters import ComponentFilter, ComponentFilterWithoutObjectLookups from dojo.finding.queries import get_authorized_findings +from dojo.product.ui.filters import ComponentFilter, ComponentFilterWithoutObjectLookups from dojo.utils import add_breadcrumb, get_page_items, get_system_setting diff --git a/dojo/development_environment/__init__.py b/dojo/development_environment/__init__.py index e69de29bb2d..adc8d51562b 100644 --- a/dojo/development_environment/__init__.py +++ b/dojo/development_environment/__init__.py @@ -0,0 +1 @@ +import dojo.development_environment.admin # noqa: F401 diff --git a/dojo/development_environment/admin.py b/dojo/development_environment/admin.py new file mode 100644 index 00000000000..a6ea8885e39 --- /dev/null +++ b/dojo/development_environment/admin.py @@ -0,0 +1,5 @@ +from django.contrib import admin + +from dojo.development_environment.models import Development_Environment + +admin.site.register(Development_Environment) diff --git a/dojo/development_environment/api/__init__.py b/dojo/development_environment/api/__init__.py new file mode 100644 index 00000000000..d48867a443d --- /dev/null +++ b/dojo/development_environment/api/__init__.py @@ -0,0 +1 @@ +path = "development_environments" # noqa: RUF067 diff --git a/dojo/development_environment/api/serializer.py b/dojo/development_environment/api/serializer.py new file mode 100644 index 00000000000..393c6ac2e98 --- /dev/null +++ b/dojo/development_environment/api/serializer.py @@ -0,0 +1,9 @@ +from rest_framework import serializers + +from dojo.development_environment.models import Development_Environment + + +class DevelopmentEnvironmentSerializer(serializers.ModelSerializer): + class Meta: + model = Development_Environment + fields = "__all__" diff --git a/dojo/development_environment/api/urls.py b/dojo/development_environment/api/urls.py new file mode 100644 index 00000000000..6d4937f37f7 --- /dev/null +++ b/dojo/development_environment/api/urls.py @@ -0,0 +1,7 @@ +from dojo.development_environment.api import path +from dojo.development_environment.api.views import DevelopmentEnvironmentViewSet + + +def add_development_environment_urls(router): + router.register(path, DevelopmentEnvironmentViewSet, basename="development_environment") + return router diff --git a/dojo/development_environment/api/views.py b/dojo/development_environment/api/views.py new file mode 100644 index 00000000000..e79748e89be --- /dev/null +++ b/dojo/development_environment/api/views.py @@ -0,0 +1,20 @@ +from django_filters.rest_framework import DjangoFilterBackend +from rest_framework.permissions import IsAuthenticated + +from dojo.api_v2.views import DojoModelViewSet +from dojo.authorization import api_permissions as permissions +from dojo.development_environment.api.serializer import DevelopmentEnvironmentSerializer +from dojo.development_environment.models import Development_Environment + + +# Authorization: authenticated, configuration +class DevelopmentEnvironmentViewSet( + DojoModelViewSet, +): + serializer_class = DevelopmentEnvironmentSerializer + queryset = Development_Environment.objects.none() + filter_backends = (DjangoFilterBackend,) + permission_classes = (IsAuthenticated, permissions.UserHasDevelopmentEnvironmentPermission) + + def get_queryset(self): + return Development_Environment.objects.all().order_by("id") diff --git a/dojo/development_environment/models.py b/dojo/development_environment/models.py new file mode 100644 index 00000000000..d8803d6e576 --- /dev/null +++ b/dojo/development_environment/models.py @@ -0,0 +1,13 @@ +from django.db import models +from django.urls import reverse + + +class Development_Environment(models.Model): + name = models.CharField(max_length=200) + + def __str__(self): + return self.name + + def get_breadcrumbs(self): + return [{"title": str(self), + "url": reverse("edit_dev_env", args=(self.id,))}] diff --git a/dojo/development_environment/ui/__init__.py b/dojo/development_environment/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/development_environment/ui/forms.py b/dojo/development_environment/ui/forms.py new file mode 100644 index 00000000000..4df9ceca798 --- /dev/null +++ b/dojo/development_environment/ui/forms.py @@ -0,0 +1,15 @@ +from django import forms + +from dojo.development_environment.models import Development_Environment + + +class Development_EnvironmentForm(forms.ModelForm): + class Meta: + model = Development_Environment + fields = ["name"] + + +class Delete_Dev_EnvironmentForm(forms.ModelForm): + class Meta: + model = Development_Environment + fields = ["id"] diff --git a/dojo/development_environment/urls.py b/dojo/development_environment/ui/urls.py similarity index 85% rename from dojo/development_environment/urls.py rename to dojo/development_environment/ui/urls.py index 1c1c60393d7..918789fcf79 100644 --- a/dojo/development_environment/urls.py +++ b/dojo/development_environment/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from dojo.development_environment import views +from dojo.development_environment.ui import views urlpatterns = [ # dev envs diff --git a/dojo/development_environment/views.py b/dojo/development_environment/ui/views.py similarity index 95% rename from dojo/development_environment/views.py rename to dojo/development_environment/ui/views.py index 8705fdd4c7c..8429c73bff0 100644 --- a/dojo/development_environment/views.py +++ b/dojo/development_environment/ui/views.py @@ -9,9 +9,9 @@ from django.urls import reverse from dojo.authorization.authorization import user_has_configuration_permission_or_403 +from dojo.development_environment.models import Development_Environment +from dojo.development_environment.ui.forms import Delete_Dev_EnvironmentForm, Development_EnvironmentForm from dojo.filters import DevelopmentEnvironmentFilter -from dojo.forms import Delete_Dev_EnvironmentForm, Development_EnvironmentForm -from dojo.models import Development_Environment from dojo.utils import add_breadcrumb, get_page_items logger = logging.getLogger(__name__) diff --git a/dojo/endpoint/__init__.py b/dojo/endpoint/__init__.py index e69de29bb2d..d774cc434b1 100644 --- a/dojo/endpoint/__init__.py +++ b/dojo/endpoint/__init__.py @@ -0,0 +1 @@ +import dojo.endpoint.admin # noqa: F401 diff --git a/dojo/endpoint/admin.py b/dojo/endpoint/admin.py new file mode 100644 index 00000000000..1a56f8d89d8 --- /dev/null +++ b/dojo/endpoint/admin.py @@ -0,0 +1,10 @@ +import tagulous.admin +from django.contrib import admin + +from dojo.endpoint.models import Endpoint, Endpoint_Params, Endpoint_Status + +admin.site.register(Endpoint_Params) +admin.site.register(Endpoint_Status) +admin.site.register(Endpoint) +tagulous.admin.register(Endpoint.tags) +tagulous.admin.register(Endpoint.inherited_tags) diff --git a/dojo/endpoint/api/__init__.py b/dojo/endpoint/api/__init__.py new file mode 100644 index 00000000000..0aa96944499 --- /dev/null +++ b/dojo/endpoint/api/__init__.py @@ -0,0 +1 @@ +path = "endpoints" # noqa: RUF067 diff --git a/dojo/endpoint/api/filters.py b/dojo/endpoint/api/filters.py new file mode 100644 index 00000000000..6dda92ab41a --- /dev/null +++ b/dojo/endpoint/api/filters.py @@ -0,0 +1,40 @@ +from django_filters import ( + BooleanFilter, + CharFilter, + OrderingFilter, +) + +from dojo.endpoint.models import Endpoint +from dojo.filters import CharFieldFilterANDExpression, CharFieldInFilter, DojoFilter + + +class ApiEndpointFilter(DojoFilter): + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Tag name contains") + tags = CharFieldInFilter( + field_name="tags__name", + lookup_expr="in", + help_text="Comma separated list of exact tags (uses OR for multiple values)") + tags__and = CharFieldFilterANDExpression( + field_name="tags__name", + help_text="Comma separated list of exact tags to match with an AND expression") + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Not Tag name contains", exclude="True") + not_tags = CharFieldInFilter(field_name="tags__name", lookup_expr="in", + help_text="Comma separated list of exact tags not present on model", exclude="True") + has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("host", "host"), + ("product", "product"), + ("id", "id"), + ("active_finding_count", "active_finding_count"), + ), + field_labels={ + "active_finding_count": "Active Findings Count", + }, + ) + + class Meta: + model = Endpoint + fields = ["id", "protocol", "userinfo", "host", "port", "path", "query", "fragment", "product"] diff --git a/dojo/endpoint/api/serializer.py b/dojo/endpoint/api/serializer.py new file mode 100644 index 00000000000..05e140d3f65 --- /dev/null +++ b/dojo/endpoint/api/serializer.py @@ -0,0 +1,230 @@ +import logging + +from django.conf import settings +from django.core.exceptions import ValidationError +from django.db.utils import IntegrityError +from rest_framework import serializers +from rest_framework.exceptions import ValidationError as RestFrameworkValidationError + +from dojo.endpoint.models import Endpoint, Endpoint_Params, Endpoint_Status +from dojo.endpoint.utils import endpoint_filter, endpoint_meta_import +from dojo.importers.auto_create_context import AutoCreateContextManager +from dojo.location.models import Location +from dojo.models import Product, get_current_date +from dojo.utils import is_scan_file_too_large + +logger = logging.getLogger(__name__) + + +class EndpointStatusSerializer(serializers.ModelSerializer): + class Meta: + model = Endpoint_Status + fields = "__all__" + + def run_validators(self, initial_data): + try: + return super().run_validators(initial_data) + except RestFrameworkValidationError as exc: + if "finding, endpoint must make a unique set" in str(exc): + msg = "This endpoint-finding relation already exists" + raise serializers.ValidationError(msg) from exc + raise + + def create(self, validated_data): + endpoint = validated_data.get("endpoint") + finding = validated_data.get("finding") + try: + status = Endpoint_Status.objects.create( + finding=finding, endpoint=endpoint, + ) + except IntegrityError as ie: + if "finding, endpoint must make a unique set" in str(ie): + msg = "This endpoint-finding relation already exists" + raise serializers.ValidationError(msg) + raise + status.mitigated = validated_data.get("mitigated", False) + status.false_positive = validated_data.get("false_positive", False) + status.out_of_scope = validated_data.get("out_of_scope", False) + status.risk_accepted = validated_data.get("risk_accepted", False) + status.date = validated_data.get("date", get_current_date()) + status.save() + return status + + def update(self, instance, validated_data): + try: + return super().update(instance, validated_data) + except IntegrityError as ie: + if "finding, endpoint must make a unique set" in str(ie): + msg = "This endpoint-finding relation already exists" + raise serializers.ValidationError(msg) + raise + + +class EndpointSerializer(serializers.ModelSerializer): + # tags field uses lazy get_fields() to break the import cycle: + # EndpointSerializer -> TagListSerializerField -> api_v2.serializers -> EndpointSerializer + active_finding_count = serializers.IntegerField(read_only=True) + + class Meta: + model = Endpoint + exclude = ("inherited_tags",) + + def get_fields(self): + fields = super().get_fields() + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + TagListSerializerField, + ) + fields["tags"] = TagListSerializerField(required=False) + return fields + + def validate(self, data): + + if self.context["request"].method != "PATCH": + if "product" not in data: + msg = "Product is required" + raise serializers.ValidationError(msg) + protocol = data.get("protocol") + userinfo = data.get("userinfo") + host = data.get("host") + port = data.get("port") + path = data.get("path") + query = data.get("query") + fragment = data.get("fragment") + product = data.get("product") + else: + protocol = data.get("protocol", self.instance.protocol) + userinfo = data.get("userinfo", self.instance.userinfo) + host = data.get("host", self.instance.host) + port = data.get("port", self.instance.port) + path = data.get("path", self.instance.path) + query = data.get("query", self.instance.query) + fragment = data.get("fragment", self.instance.fragment) + if "product" in data and data["product"] != self.instance.product: + msg = "Change of product is not possible" + raise serializers.ValidationError(msg) + product = self.instance.product + + endpoint_ins = Endpoint( + protocol=protocol, + userinfo=userinfo, + host=host, + port=port, + path=path, + query=query, + fragment=fragment, + product=product, + ) + endpoint_ins.clean() # Run standard validation and clean process; can raise errors + + endpoint = endpoint_filter( + protocol=endpoint_ins.protocol, + userinfo=endpoint_ins.userinfo, + host=endpoint_ins.host, + port=endpoint_ins.port, + path=endpoint_ins.path, + query=endpoint_ins.query, + fragment=endpoint_ins.fragment, + product=endpoint_ins.product, + ) + if ( + self.context["request"].method in {"PUT", "PATCH"} + and ( + (endpoint.count() > 1) + or ( + endpoint.count() == 1 + and endpoint.first().pk != self.instance.pk + ) + ) + ) or ( + self.context["request"].method == "POST" and endpoint.count() > 0 + ): + msg = ( + "It appears as though an endpoint with this data already " + "exists for this product." + ) + raise serializers.ValidationError(msg, code="invalid") + + # use clean data + data["protocol"] = endpoint_ins.protocol + data["userinfo"] = endpoint_ins.userinfo + data["host"] = endpoint_ins.host + data["port"] = endpoint_ins.port + data["path"] = endpoint_ins.path + data["query"] = endpoint_ins.query + data["fragment"] = endpoint_ins.fragment + data["product"] = endpoint_ins.product + + return data + + +class EndpointParamsSerializer(serializers.ModelSerializer): + class Meta: + model = Endpoint_Params + fields = "__all__" + + +class EndpointMetaImporterSerializer(serializers.Serializer): + file = serializers.FileField(required=True) + create_endpoints = serializers.BooleanField(default=True, required=False) + create_tags = serializers.BooleanField(default=True, required=False) + create_dojo_meta = serializers.BooleanField(default=False, required=False) + product_name = serializers.CharField(required=False) + product = serializers.PrimaryKeyRelatedField( + queryset=Product.objects.all(), required=False, + ) + # extra fields populated in response + # need to use the _id suffix as without the serializer framework gets + # confused + product_id = serializers.IntegerField(read_only=True) + + def validate(self, data): + file = data.get("file") + if file and is_scan_file_too_large(file): + msg = f"Report file is too large. Maximum supported size is {settings.SCAN_FILE_MAX_SIZE} MB" + raise serializers.ValidationError(msg) + + return data + + def save(self): + data = self.validated_data + file = data.get("file") + create_endpoints = data.get("create_endpoints", True) + create_tags = data.get("create_tags", True) + create_dojo_meta = data.get("create_dojo_meta", False) + auto_create = AutoCreateContextManager() + # Process the context to make an conversions needed. Catch any exceptions + # in this case and wrap them in a DRF exception + try: + auto_create.process_import_meta_data_from_dict(data) + # Get an existing product + product = auto_create.get_target_product_if_exists(**data) + if not product: + product = auto_create.get_target_product_by_id_if_exists(**data) + except (ValueError, TypeError) as e: + # Raise an explicit drf exception here + raise ValidationError(str(e)) + try: + if settings.V3_FEATURE_LOCATIONS: + endpoint_meta_import( + file, + product, + create_endpoints, + create_tags, + create_dojo_meta, + origin="API", + object_class=Location, + ) + else: + # TODO: Delete this after the move to Locations + endpoint_meta_import( + file, + product, + create_endpoints, + create_tags, + create_dojo_meta, + origin="API", + ) + except SyntaxError as se: + raise Exception(se) + except ValueError as ve: + raise Exception(ve) diff --git a/dojo/endpoint/api/urls.py b/dojo/endpoint/api/urls.py new file mode 100644 index 00000000000..4a138f3cbde --- /dev/null +++ b/dojo/endpoint/api/urls.py @@ -0,0 +1,20 @@ +from dojo.endpoint.api.views import EndpointMetaImporterView, EndpointStatusViewSet, EndPointViewSet + + +def add_endpoint_urls(router): + """ + Register endpoint/endpoint_status routes (non-V3 block only). + + endpoint_meta_import is always registered via register_endpoint_meta_import. + endpoints and endpoint_status are registered only when V3_FEATURE_LOCATIONS is OFF; + the V3 compat viewsets are registered by dojo/location/api/urls.py instead. + """ + router.register(r"endpoints", EndPointViewSet, basename="endpoint") + router.register(r"endpoint_status", EndpointStatusViewSet, basename="endpoint_status") + return router + + +def register_endpoint_meta_import(router): + """Register the unconditional endpoint_meta_import route.""" + router.register(r"endpoint_meta_import", EndpointMetaImporterView, basename="endpointmetaimport") + return router diff --git a/dojo/endpoint/api/views.py b/dojo/endpoint/api/views.py new file mode 100644 index 00000000000..02a6f65f31f --- /dev/null +++ b/dojo/endpoint/api/views.py @@ -0,0 +1,161 @@ +import logging + +from django.db.models import OuterRef, Value +from django.db.models.functions import Coalesce +from django_filters.rest_framework import DjangoFilterBackend +from drf_spectacular.utils import extend_schema +from rest_framework import mixins, status, viewsets +from rest_framework.decorators import action +from rest_framework.parsers import MultiPartParser +from rest_framework.permissions import IsAuthenticated +from rest_framework.response import Response + +from dojo.api_v2 import serializers as api_v2_serializers +from dojo.api_v2.views import PrefetchDojoModelViewSet, report_generate +from dojo.authorization import api_permissions as permissions +from dojo.endpoint.api.filters import ApiEndpointFilter +from dojo.endpoint.api.serializer import ( + EndpointMetaImporterSerializer, + EndpointSerializer, + EndpointStatusSerializer, +) +from dojo.endpoint.models import Endpoint, Endpoint_Status +from dojo.endpoint.queries import ( + get_authorized_endpoint_status, + get_authorized_endpoints, +) +from dojo.models import Finding +from dojo.product.queries import get_authorized_products +from dojo.query_utils import build_count_subquery + +logger = logging.getLogger(__name__) + + +# Authorization: authenticated users +# Authorization: object-based +# @extend_schema_view(**schema_with_prefetch()) +# Nested models with prefetch make the response schema too long for Swagger UI +class EndPointViewSet( + PrefetchDojoModelViewSet, +): + serializer_class = EndpointSerializer + queryset = Endpoint.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_class = ApiEndpointFilter + + permission_classes = ( + IsAuthenticated, + permissions.UserHasEndpointPermission, + ) + + def get_queryset(self): + active_finding_subquery = build_count_subquery( + Finding.objects.filter(endpoints=OuterRef("pk"), active=True), + group_field="endpoints", + ) + return get_authorized_endpoints("view").annotate( + active_finding_count=Coalesce(active_finding_subquery, Value(0)), + ).distinct() + + @extend_schema( + request=api_v2_serializers.ReportGenerateOptionSerializer, + responses={status.HTTP_200_OK: api_v2_serializers.ReportGenerateSerializer}, + ) + @action( + detail=True, methods=["post"], + # IsAuthenticated only: report generation requires View permission, + # enforced by the permission-filtered get_queryset(). The viewset's + # permission_classes would check Edit (POST), which is too restrictive. + permission_classes=[IsAuthenticated], + ) + def generate_report(self, request, pk=None): + endpoint = self.get_object() + + options = {} + # prepare post data + report_options = api_v2_serializers.ReportGenerateOptionSerializer( + data=request.data, + ) + if report_options.is_valid(): + options["include_finding_notes"] = report_options.validated_data[ + "include_finding_notes" + ] + options["include_finding_images"] = report_options.validated_data[ + "include_finding_images" + ] + options[ + "include_executive_summary" + ] = report_options.validated_data["include_executive_summary"] + options[ + "include_table_of_contents" + ] = report_options.validated_data["include_table_of_contents"] + else: + return Response( + report_options.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + data = report_generate(request, endpoint, options) + report = api_v2_serializers.ReportGenerateSerializer(data) + return Response(report.data) + + +# Authorization: object-based +# @extend_schema_view(**schema_with_prefetch()) +# Nested models with prefetch make the response schema too long for Swagger UI +class EndpointStatusViewSet( + PrefetchDojoModelViewSet, +): + serializer_class = EndpointStatusSerializer + queryset = Endpoint_Status.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = [ + "mitigated", + "false_positive", + "out_of_scope", + "risk_accepted", + "mitigated_by", + "finding", + "endpoint", + ] + + permission_classes = ( + IsAuthenticated, + permissions.UserHasEndpointStatusPermission, + ) + + def get_queryset(self): + return get_authorized_endpoint_status( + "view", + ).distinct() + + +# Authorization: authenticated users, DjangoModelPermissions +class EndpointMetaImporterView( + mixins.CreateModelMixin, viewsets.GenericViewSet, +): + + """ + Imports a CSV file into a product to propagate arbitrary meta and tags on endpoints. + + By Names: + - Provide `product_name` of existing product + + By ID: + - Provide the id of the product in the `product` parameter + + In this scenario Defect Dojo will look up the product by the provided details. + """ + + serializer_class = EndpointMetaImporterSerializer + parser_classes = [MultiPartParser] + queryset = Finding.objects.none() + permission_classes = ( + IsAuthenticated, + permissions.UserHasMetaImportPermission, + ) + + def perform_create(self, serializer): + serializer.save() + + def get_queryset(self): + return get_authorized_products("edit") diff --git a/dojo/endpoint/models.py b/dojo/endpoint/models.py new file mode 100644 index 00000000000..f55e3f5f8c0 --- /dev/null +++ b/dojo/endpoint/models.py @@ -0,0 +1,464 @@ +import contextlib +import logging +import re +from urllib.parse import urlparse + +import hyperlink +from django.conf import settings +from django.core.exceptions import ValidationError +from django.core.validators import validate_ipv46_address +from django.db import connection, models +from django.db.models import F, Q +from django.db.models.functions import Lower +from django.urls import reverse +from django.utils.translation import gettext as _ +from tagulous.models import TagField + +# get_current_date/get_current_datetime/copy_model_util are defined early in dojo.models, +# before the re-export that loads this module — resolves despite partial circular load. +# Must keep their dojo.models.* path for Django migration serialization. +from dojo.models import copy_model_util, get_current_date, get_current_datetime + +logger = logging.getLogger(__name__) + + +class Endpoint_Params(models.Model): + param = models.CharField(max_length=150) + value = models.CharField(max_length=150) + method_type = (("GET", "GET"), + ("POST", "POST")) + method = models.CharField(max_length=20, blank=False, null=True, choices=method_type) + + +class Endpoint_Status(models.Model): + date = models.DateField(default=get_current_date) + last_modified = models.DateTimeField(null=True, editable=False, default=get_current_datetime) + mitigated = models.BooleanField(default=False, blank=True) + mitigated_time = models.DateTimeField(editable=False, null=True, blank=True) + mitigated_by = models.ForeignKey("dojo.Dojo_User", editable=True, null=True, on_delete=models.RESTRICT) + false_positive = models.BooleanField(default=False, blank=True) + out_of_scope = models.BooleanField(default=False, blank=True) + risk_accepted = models.BooleanField(default=False, blank=True) + endpoint = models.ForeignKey("dojo.Endpoint", null=False, blank=False, on_delete=models.CASCADE, related_name="status_endpoint") + finding = models.ForeignKey("dojo.Finding", null=False, blank=False, on_delete=models.CASCADE, related_name="status_finding") + + class Meta: + indexes = [ + models.Index(fields=["finding", "mitigated"]), + models.Index(fields=["endpoint", "mitigated"]), + # Optimize frequent lookups of "active" statuses (mitigated/flags all False) + models.Index( + name="idx_eps_active_by_endpoint", + fields=["endpoint"], + condition=Q(mitigated=False, false_positive=False, out_of_scope=False, risk_accepted=False), + ), + models.Index( + name="idx_eps_active_by_finding", + fields=["finding"], + condition=Q(mitigated=False, false_positive=False, out_of_scope=False, risk_accepted=False), + ), + ] + constraints = [ + models.UniqueConstraint(fields=["finding", "endpoint"], name="endpoint-finding relation"), + ] + + def __str__(self): + with Endpoint.allow_endpoint_init(): # TODO: Delete this after the move to Locations + return f"'{self.finding}' on '{self.endpoint}'" + + def copy(self, finding=None): + copy = copy_model_util(self) + current_endpoint = self.endpoint + if finding: + copy.finding = finding + copy.endpoint = current_endpoint + copy.save() + + return copy + + @property + def age(self): + + diff = self.mitigated_time.date() - self.date if self.mitigated else get_current_date() - self.date + days = diff.days + return max(0, days) + + +class Endpoint(models.Model): + protocol = models.CharField(null=True, blank=True, max_length=20, + help_text=_("The communication protocol/scheme such as 'http', 'ftp', 'dns', etc.")) + userinfo = models.CharField(null=True, blank=True, max_length=500, + help_text=_("User info as 'alice', 'bob', etc.")) + host = models.CharField(null=True, blank=True, max_length=500, + help_text=_("The host name or IP address. It must not include the port number. " + "For example '127.0.0.1', 'localhost', 'yourdomain.com'.")) + port = models.IntegerField(null=True, blank=True, + help_text=_("The network port associated with the endpoint.")) + path = models.CharField(null=True, blank=True, max_length=500, + help_text=_("The location of the resource, it must not start with a '/'. For example " + "endpoint/420/edit")) + query = models.CharField(null=True, blank=True, max_length=1000, + help_text=_("The query string, the question mark should be omitted." + "For example 'group=4&team=8'")) + fragment = models.CharField(null=True, blank=True, max_length=500, + help_text=_("The fragment identifier which follows the hash mark. The hash mark should " + "be omitted. For example 'section-13', 'paragraph-2'.")) + product = models.ForeignKey("dojo.Product", null=True, blank=True, on_delete=models.CASCADE) + endpoint_params = models.ManyToManyField("dojo.Endpoint_Params", blank=True, editable=False) + findings = models.ManyToManyField("dojo.Finding", + blank=True, + verbose_name=_("Findings"), + through="dojo.Endpoint_Status") + + tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this endpoint. Choose from the list or add new tags. Press Enter key to add.")) + inherited_tags = TagField(blank=True, force_lowercase=True, help_text=_("Internal use tags sepcifically for maintaining parity with product. This field will be present as a subset in the tags field")) + + class Meta: + ordering = ["product", "host", "protocol", "port", "userinfo", "path", "query", "fragment"] + indexes = [ + models.Index(fields=["product"]), + # Fast case-insensitive equality on host within product scope + models.Index( + F("product"), + Lower("host"), + name="idx_ep_product_lower_host", + ), + ] + + def __init__(self, *args, **kwargs): + if settings.V3_FEATURE_LOCATIONS and not getattr(self, "_allow_v3_init", False): + msg = "Endpoint model is deprecated when V3_FEATURE_LOCATIONS is enabled" + raise NotImplementedError(msg) + super().__init__(*args, **kwargs) + + def __hash__(self): + return self.__str__().__hash__() + + def __eq__(self, other): + if isinstance(other, Endpoint): + contents_match = str(self) == str(other) + # Use product_id (cached integer) instead of self.product to avoid + # triggering a FK lookup on every comparison inside NestedObjects.add_edge. + if self.product_id is not None and other.product_id is not None: + return self.product_id == other.product_id and contents_match + return contents_match + + return NotImplemented + + def __str__(self): + try: + if self.host: + dummy_scheme = "dummy-scheme" # workaround for https://github.com/python-hyper/hyperlink/blob/b8c9152cd826bbe8e6cc125648f3738235019705/src/hyperlink/_url.py#L988 + url = hyperlink.EncodedURL( + scheme=self.protocol or dummy_scheme, + userinfo=self.userinfo or "", + host=self.host, + port=self.port, + path=tuple(self.path.split("/")) if self.path else (), + query=tuple( + ( + qe.split("=", 1) + if "=" in qe + else (qe, None) + ) + for qe in self.query.split("&") + ) if self.query else (), # inspired by https://github.com/python-hyper/hyperlink/blob/b8c9152cd826bbe8e6cc125648f3738235019705/src/hyperlink/_url.py#L1427 + fragment=self.fragment or "", + ) + # Return a normalized version of the URL to avoid differences where there shouldn't be any difference. + # Example: https://google.com and https://google.com:443 + normalize_path = self.path # it used to add '/' at the end of host + clean_url = url.normalize(scheme=True, host=True, path=normalize_path, query=True, fragment=True, userinfo=True, percents=True).to_uri().to_text() + if not self.protocol: + if clean_url[:len(dummy_scheme) + 3] == (dummy_scheme + "://"): + clean_url = clean_url[len(dummy_scheme) + 3:] + else: + msg = "hyperlink lib did not create URL as was expected" + raise ValueError(msg) + return clean_url + msg = "Missing host" + raise ValueError(msg) + except: + url = "" + if self.protocol: + url += f"{self.protocol}://" + if self.userinfo: + url += f"{self.userinfo}@" + if self.host: + url += self.host + if self.port: + url += f":{self.port}" + if self.path: + url += "{}{}".format("/" if self.path[0] != "/" else "", self.path) + if self.query: + url += f"?{self.query}" + if self.fragment: + url += f"#{self.fragment}" + return url + + def get_absolute_url(self): + return reverse("view_endpoint", args=[str(self.id)]) + + @classmethod + @contextlib.contextmanager + def allow_endpoint_init(cls): + # When migrating to Locations, Endpoints are not deleted (hooray backup!). Disallowing the initialization of + # Endpoints is a good way to catch where they might still be used (oops!). However, there are some circumstances + # -- object deletes -- where Django itself attempts to instantiate an Endpoint object. This, we need to allow: + # if a user wants to delete an object, including whatever Endpoints are attached to it, they should be able to. + # This context manager allows code to initialize Endpoints at our discretion. + old = getattr(cls, "_allow_v3_init", None) + cls._allow_v3_init = True + try: + yield + finally: + cls._allow_v3_init = old + + def clean(self): + errors = [] + null_char_list = ["0x00", "\x00"] + db_type = connection.vendor + if self.protocol is not None: + if not re.match(r"^[A-Za-z][A-Za-z0-9\.\-\+]+$", self.protocol): # https://tools.ietf.org/html/rfc3986#section-3.1 + errors.append(ValidationError(f'Protocol "{self.protocol}" has invalid format')) + if not self.protocol: + self.protocol = None + + if self.userinfo is not None: + if not re.match(r"^[A-Za-z0-9\.\-_~%\!\$&\'\(\)\*\+,;=:]+$", self.userinfo): # https://tools.ietf.org/html/rfc3986#section-3.2.1 + errors.append(ValidationError(f'Userinfo "{self.userinfo}" has invalid format')) + if not self.userinfo: + self.userinfo = None + + if self.host: + if not re.match(r"^[A-Za-z0-9_\-\+][A-Za-z0-9_\.\-\+]+$", self.host): + try: + validate_ipv46_address(self.host) + except ValidationError: + errors.append(ValidationError(f'Host "{self.host}" has invalid format')) + else: + errors.append(ValidationError("Host must not be empty")) + + if self.port is not None: + try: + int_port = int(self.port) + if not (0 <= int_port < 65536): + errors.append(ValidationError(f'Port "{self.port}" has invalid format - out of range')) + self.port = int_port + except ValueError: + errors.append(ValidationError(f'Port "{self.port}" has invalid format - it is not a number')) + + if self.path is not None: + while len(self.path) > 0 and self.path[0] == "/": # Endpoint store "root-less" path + self.path = self.path[1:] + if any(null_char in self.path for null_char in null_char_list): + old_value = self.path + if "postgres" in db_type: + action_string = "Postgres does not accept NULL character. Attempting to replace with %00..." + for remove_str in null_char_list: + self.path = self.path.replace(remove_str, "%00") + logger.error('Path "%s" has invalid format - It contains the NULL character. The following action was taken: %s', old_value, action_string) + if not self.path: + self.path = None + + if self.query is not None: + if len(self.query) > 0 and self.query[0] == "?": + self.query = self.query[1:] + if any(null_char in self.query for null_char in null_char_list): + old_value = self.query + if "postgres" in db_type: + action_string = "Postgres does not accept NULL character. Attempting to replace with %00..." + for remove_str in null_char_list: + self.query = self.query.replace(remove_str, "%00") + logger.error('Query "%s" has invalid format - It contains the NULL character. The following action was taken: %s', old_value, action_string) + if not self.query: + self.query = None + + if self.fragment is not None: + if len(self.fragment) > 0 and self.fragment[0] == "#": + self.fragment = self.fragment[1:] + if any(null_char in self.fragment for null_char in null_char_list): + old_value = self.fragment + if "postgres" in db_type: + action_string = "Postgres does not accept NULL character. Attempting to replace with %00..." + for remove_str in null_char_list: + self.fragment = self.fragment.replace(remove_str, "%00") + logger.error('Fragment "%s" has invalid format - It contains the NULL character. The following action was taken: %s', old_value, action_string) + if not self.fragment: + self.fragment = None + + if errors: + raise ValidationError(errors) + + @property + def is_broken(self): + try: + self.clean() + except: + return True + else: + return not self.product + + @property + def mitigated(self): + return not self.vulnerable + + @property + def vulnerable(self): + return Endpoint_Status.objects.filter( + endpoint=self, + mitigated=False, + false_positive=False, + out_of_scope=False, + risk_accepted=False, + ).count() > 0 + + @property + def findings_count(self): + return self.findings.all().count() + + def active_findings(self): + return self.findings.filter( + active=True, + out_of_scope=False, + mitigated__isnull=True, + false_p=False, + duplicate=False, + status_finding__false_positive=False, + status_finding__out_of_scope=False, + status_finding__risk_accepted=False, + ).order_by("numerical_severity") + + def active_verified_findings(self): + return self.findings.filter( + active=True, + verified=True, + out_of_scope=False, + mitigated__isnull=True, + false_p=False, + duplicate=False, + status_finding__false_positive=False, + status_finding__out_of_scope=False, + status_finding__risk_accepted=False, + ).order_by("numerical_severity") + + @property + def active_findings_count(self): + return self.active_findings().count() + + @property + def active_verified_findings_count(self): + return self.active_verified_findings().count() + + def host_endpoints(self): + return Endpoint.objects.filter(host=self.host, + product=self.product).distinct() + + @property + def host_endpoints_count(self): + return self.host_endpoints().count() + + def host_mitigated_endpoints(self): + meps = Endpoint_Status.objects \ + .filter(endpoint__in=self.host_endpoints()) \ + .filter(Q(mitigated=True) + | Q(false_positive=True) + | Q(out_of_scope=True) + | Q(risk_accepted=True) + | Q(finding__out_of_scope=True) + | Q(finding__mitigated__isnull=False) + | Q(finding__false_p=True) + | Q(finding__duplicate=True) + | Q(finding__active=False)) + return Endpoint.objects.filter(status_endpoint__in=meps).distinct() + + @property + def host_mitigated_endpoints_count(self): + return self.host_mitigated_endpoints().count() + + def host_findings(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + return Finding.objects.filter(endpoints__in=self.host_endpoints()).distinct() + + @property + def host_findings_count(self): + return self.host_findings().count() + + def host_active_findings(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + return Finding.objects.filter( + active=True, + out_of_scope=False, + mitigated__isnull=True, + false_p=False, + duplicate=False, + status_finding__false_positive=False, + status_finding__out_of_scope=False, + status_finding__risk_accepted=False, + endpoints__in=self.host_endpoints(), + ).order_by("numerical_severity") + + def host_active_verified_findings(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + return Finding.objects.filter( + active=True, + verified=True, + out_of_scope=False, + mitigated__isnull=True, + false_p=False, + duplicate=False, + status_finding__false_positive=False, + status_finding__out_of_scope=False, + status_finding__risk_accepted=False, + endpoints__in=self.host_endpoints(), + ).order_by("numerical_severity") + + @property + def host_active_findings_count(self): + return self.host_active_findings().count() + + @property + def host_active_verified_findings_count(self): + return self.host_active_verified_findings().count() + + def get_breadcrumbs(self): + bc = self.product.get_breadcrumbs() + bc += [{"title": self.host, + "url": reverse("view_endpoint", args=(self.id,))}] + return bc + + @staticmethod + def from_uri(uri): + try: + url = hyperlink.parse(url=uri) + except UnicodeDecodeError: + url = hyperlink.parse(url="//" + urlparse(uri).netloc) + except hyperlink.URLParseError as e: + msg = f"Invalid URL format: {e}" + raise ValidationError(msg) + + query_parts = [] # inspired by https://github.com/python-hyper/hyperlink/blob/b8c9152cd826bbe8e6cc125648f3738235019705/src/hyperlink/_url.py#L1768 + for k, v in url.query: + if v is None: + query_parts.append(k) + else: + query_parts.append(f"{k}={v}") + query_string = "&".join(query_parts) + + protocol = url.scheme or None + userinfo = ":".join(url.userinfo) if url.userinfo not in {(), ("",)} else None + host = url.host or None + port = url.port + path = "/".join(url.path)[:500] if url.path not in {None, (), ("",)} else None + query = query_string[:1000] if query_string is not None and query_string else None + fragment = url.fragment[:500] if url.fragment is not None and url.fragment else None + + return Endpoint( + protocol=protocol, + userinfo=userinfo, + host=host, + port=port, + path=path, + query=query, + fragment=fragment, + ) diff --git a/dojo/endpoint/signals.py b/dojo/endpoint/signals.py index aebc348c003..58f5e686d15 100644 --- a/dojo/endpoint/signals.py +++ b/dojo/endpoint/signals.py @@ -7,7 +7,7 @@ from django.urls import reverse from django.utils.translation import gettext as _ -from dojo.models import Endpoint +from dojo.endpoint.models import Endpoint from dojo.notifications.helper import create_notification from dojo.pghistory_models import DojoEvents diff --git a/dojo/endpoint/ui/__init__.py b/dojo/endpoint/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/endpoint/ui/filters.py b/dojo/endpoint/ui/filters.py new file mode 100644 index 00000000000..a4ccb35cce1 --- /dev/null +++ b/dojo/endpoint/ui/filters.py @@ -0,0 +1,260 @@ +from django.forms import HiddenInput +from django_filters import ( + CharFilter, + FilterSet, + ModelMultipleChoiceFilter, + NumberFilter, + OrderingFilter, +) + +from dojo.endpoint.models import Endpoint +from dojo.endpoint.queries import get_authorized_endpoints_for_queryset +from dojo.filters import DojoFilter +from dojo.labels import get_labels +from dojo.models import Engagement, Finding, Product, Test +from dojo.product.queries import get_authorized_products + +labels = get_labels() + + +class EndpointFilterHelper(FilterSet): + protocol = CharFilter(lookup_expr="icontains") + userinfo = CharFilter(lookup_expr="icontains") + host = CharFilter(lookup_expr="icontains") + port = NumberFilter() + path = CharFilter(lookup_expr="icontains") + query = CharFilter(lookup_expr="icontains") + fragment = CharFilter(lookup_expr="icontains") + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) + has_tags = CharFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("product", "product"), + ("host", "host"), + ("id", "id"), + ("active_finding_count", "active_finding_count"), + ), + field_labels={ + "active_finding_count": "Active Findings Count", + }, + ) + + +class EndpointFilter(EndpointFilterHelper, DojoFilter): + product = ModelMultipleChoiceFilter( + queryset=Product.objects.none(), + label=labels.ASSET_FILTERS_LABEL) + tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + label="Endpoint Tags", + queryset=Endpoint.tags.tag_model.objects.all().order_by("name")) + findings__tags = ModelMultipleChoiceFilter( + field_name="findings__tags__name", + to_field_name="name", + label="Finding Tags", + queryset=Finding.tags.tag_model.objects.all().order_by("name")) + findings__test__tags = ModelMultipleChoiceFilter( + field_name="findings__test__tags__name", + to_field_name="name", + label="Test Tags", + queryset=Test.tags.tag_model.objects.all().order_by("name")) + findings__test__engagement__tags = ModelMultipleChoiceFilter( + field_name="findings__test__engagement__tags__name", + to_field_name="name", + label="Engagement Tags", + queryset=Engagement.tags.tag_model.objects.all().order_by("name")) + findings__test__engagement__product__tags = ModelMultipleChoiceFilter( + field_name="findings__test__engagement__product__tags__name", + to_field_name="name", + label=labels.ASSET_FILTERS_TAGS_ASSET_LABEL, + queryset=Product.tags.tag_model.objects.all().order_by("name")) + not_tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + label="Not Endpoint Tags", + exclude=True, + queryset=Endpoint.tags.tag_model.objects.all().order_by("name")) + not_findings__tags = ModelMultipleChoiceFilter( + field_name="findings__tags__name", + to_field_name="name", + label="Not Finding Tags", + exclude=True, + queryset=Finding.tags.tag_model.objects.all().order_by("name")) + not_findings__test__tags = ModelMultipleChoiceFilter( + field_name="findings__test__tags__name", + to_field_name="name", + label="Not Test Tags", + exclude=True, + queryset=Test.tags.tag_model.objects.all().order_by("name")) + not_findings__test__engagement__tags = ModelMultipleChoiceFilter( + field_name="findings__test__engagement__tags__name", + to_field_name="name", + label="Not Engagement Tags", + exclude=True, + queryset=Engagement.tags.tag_model.objects.all().order_by("name")) + not_findings__test__engagement__product__tags = ModelMultipleChoiceFilter( + field_name="findings__test__engagement__product__tags__name", + to_field_name="name", + label=labels.ASSET_FILTERS_NOT_TAGS_ASSET_LABEL, + exclude=True, + queryset=Product.tags.tag_model.objects.all().order_by("name")) + + def __init__(self, *args, **kwargs): + self.user = None + if "user" in kwargs: + self.user = kwargs.pop("user") + super().__init__(*args, **kwargs) + self.form.fields["product"].queryset = get_authorized_products("view") + + @property + def qs(self): + parent = super().qs + return get_authorized_endpoints_for_queryset("view", parent) + + class Meta: + model = Endpoint + exclude = ["findings", "inherited_tags"] + + +class EndpointFilterWithoutObjectLookups(EndpointFilterHelper): + product = NumberFilter(widget=HiddenInput()) + product__name = CharFilter( + field_name="product__name", + lookup_expr="iexact", + label=labels.ASSET_FILTERS_NAME_LABEL, + help_text=labels.ASSET_FILTERS_NAME_HELP) + product__name_contains = CharFilter( + field_name="product__name", + lookup_expr="icontains", + label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL, + help_text=labels.ASSET_FILTERS_NAME_CONTAINS_HELP) + + tags_contains = CharFilter( + label="Endpoint Tag Contains", + field_name="tags__name", + lookup_expr="icontains", + help_text="Search for tags on a Endpoint that contain a given pattern") + tags = CharFilter( + label="Endpoint Tag", + field_name="tags__name", + lookup_expr="iexact", + help_text="Search for tags on a Endpoint that are an exact match") + findings__tags_contains = CharFilter( + label="Finding Tag Contains", + field_name="findings__tags__name", + lookup_expr="icontains", + help_text="Search for tags on a Finding that contain a given pattern") + findings__tags = CharFilter( + label="Finding Tag", + field_name="findings__tags__name", + lookup_expr="iexact", + help_text="Search for tags on a Finding that are an exact match") + findings__test__tags_contains = CharFilter( + label="Test Tag Contains", + field_name="findings__test__tags__name", + lookup_expr="icontains", + help_text="Search for tags on a Finding that contain a given pattern") + findings__test__tags = CharFilter( + label="Test Tag", + field_name="findings__test__tags__name", + lookup_expr="iexact", + help_text="Search for tags on a Finding that are an exact match") + findings__test__engagement__tags_contains = CharFilter( + label="Engagement Tag Contains", + field_name="findings__test__engagement__tags__name", + lookup_expr="icontains", + help_text="Search for tags on a Finding that contain a given pattern") + findings__test__engagement__tags = CharFilter( + label="Engagement Tag", + field_name="findings__test__engagement__tags__name", + lookup_expr="iexact", + help_text="Search for tags on a Finding that are an exact match") + findings__test__engagement__product__tags_contains = CharFilter( + label=labels.ASSET_FILTERS_TAG_ASSET_CONTAINS_LABEL, + field_name="findings__test__engagement__product__tags__name", + lookup_expr="icontains", + help_text=labels.ASSET_FILTERS_TAG_ASSET_CONTAINS_HELP) + findings__test__engagement__product__tags = CharFilter( + label=labels.ASSET_FILTERS_TAG_ASSET_LABEL, + field_name="findings__test__engagement__product__tags__name", + lookup_expr="iexact", + help_text=labels.ASSET_FILTERS_TAG_ASSET_HELP) + + not_tags_contains = CharFilter( + label="Endpoint Tag Does Not Contain", + field_name="tags__name", + lookup_expr="icontains", + help_text="Search for tags on a Endpoint that contain a given pattern, and exclude them", + exclude=True) + not_tags = CharFilter( + label="Not Endpoint Tag", + field_name="tags__name", + lookup_expr="iexact", + help_text="Search for tags on a Endpoint that are an exact match, and exclude them", + exclude=True) + not_findings__tags_contains = CharFilter( + label="Finding Tag Does Not Contain", + field_name="findings__tags__name", + lookup_expr="icontains", + help_text="Search for tags on a Finding that contain a given pattern, and exclude them", + exclude=True) + not_findings__tags = CharFilter( + label="Not Finding Tag", + field_name="findings__tags__name", + lookup_expr="iexact", + help_text="Search for tags on a Finding that are an exact match, and exclude them", + exclude=True) + not_findings__test__tags_contains = CharFilter( + label="Test Tag Does Not Contain", + field_name="findings__test__tags__name", + lookup_expr="icontains", + help_text="Search for tags on a Test that contain a given pattern, and exclude them", + exclude=True) + not_findings__test__tags = CharFilter( + label="Not Test Tag", + field_name="findings__test__tags__name", + lookup_expr="iexact", + help_text="Search for tags on a Test that are an exact match, and exclude them", + exclude=True) + not_findings__test__engagement__tags_contains = CharFilter( + label="Engagement Tag Does Not Contain", + field_name="findings__test__engagement__tags__name", + lookup_expr="icontains", + help_text="Search for tags on a Engagement that contain a given pattern, and exclude them", + exclude=True) + not_findings__test__engagement__tags = CharFilter( + label="Not Engagement Tag", + field_name="findings__test__engagement__tags__name", + lookup_expr="iexact", + help_text="Search for tags on a Engagement that are an exact match, and exclude them", + exclude=True) + not_findings__test__engagement__product__tags_contains = CharFilter( + label=labels.ASSET_FILTERS_TAG_NOT_CONTAIN_LABEL, + field_name="findings__test__engagement__product__tags__name", + lookup_expr="icontains", + help_text=labels.ASSET_FILTERS_TAG_NOT_CONTAIN_HELP, + exclude=True) + not_findings__test__engagement__product__tags = CharFilter( + label=labels.ASSET_FILTERS_TAG_NOT_LABEL, + field_name="findings__test__engagement__product__tags__name", + lookup_expr="iexact", + help_text=labels.ASSET_FILTERS_TAG_NOT_HELP, + exclude=True) + + def __init__(self, *args, **kwargs): + self.user = None + if "user" in kwargs: + self.user = kwargs.pop("user") + super().__init__(*args, **kwargs) + + @property + def qs(self): + parent = super().qs + return get_authorized_endpoints_for_queryset("view", parent) + + class Meta: + model = Endpoint + exclude = ["findings", "inherited_tags", "product"] diff --git a/dojo/endpoint/ui/forms.py b/dojo/endpoint/ui/forms.py new file mode 100644 index 00000000000..625cfc09e41 --- /dev/null +++ b/dojo/endpoint/ui/forms.py @@ -0,0 +1,164 @@ +from django import forms +from tagulous.forms import TagField + +from dojo.endpoint.models import Endpoint +from dojo.endpoint.utils import endpoint_filter, endpoint_get_or_create, validate_endpoints_to_add +from dojo.labels import get_labels +from dojo.models import Finding, Product +from dojo.product.queries import get_authorized_products +from dojo.validators import tag_validator + +labels = get_labels() + + +class ImportEndpointMetaForm(forms.Form): + file = forms.FileField(widget=forms.widgets.FileInput( + attrs={"accept": ".csv"}), + label="Choose meta file", + required=True) # Could not get required=True to actually accept the file as present + create_endpoints = forms.BooleanField( + label="Create nonexisting Endpoint", + initial=True, + required=False, + help_text="Create endpoints that do not already exist") + create_tags = forms.BooleanField( + label="Add Tags", + initial=True, + required=False, + help_text="Add meta from file as tags in the format key:value") + create_dojo_meta = forms.BooleanField( + label="Add Meta", + initial=False, + required=False, + help_text="Add data from file as Metadata. Metadata is used for displaying custom fields") + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + + +class EditEndpointForm(forms.ModelForm): + class Meta: + model = Endpoint + exclude = ["product", "inherited_tags"] + + def __init__(self, *args, **kwargs): + self.product = None + self.endpoint_instance = None + super().__init__(*args, **kwargs) + if "instance" in kwargs: + self.endpoint_instance = kwargs.pop("instance") + self.product = self.endpoint_instance.product + product_id = self.endpoint_instance.product.pk + findings = Finding.objects.filter(test__engagement__product__id=product_id) + self.fields["findings"].queryset = findings + + def clean(self): + + cleaned_data = super().clean() + + protocol = cleaned_data["protocol"] + userinfo = cleaned_data["userinfo"] + host = cleaned_data["host"] + port = cleaned_data["port"] + path = cleaned_data["path"] + query = cleaned_data["query"] + fragment = cleaned_data["fragment"] + + endpoint = endpoint_filter( + protocol=protocol, + userinfo=userinfo, + host=host, + port=port, + path=path, + query=query, + fragment=fragment, + product=self.product, + ) + if endpoint.count() > 1 or (endpoint.count() == 1 and endpoint.first().pk != self.endpoint_instance.pk): + msg = "It appears as though an endpoint with this data already exists for this product." + raise forms.ValidationError(msg, code="invalid") + + return cleaned_data + + def clean_tags(self): + tag_validator(self.cleaned_data.get("tags")) + return self.cleaned_data.get("tags") + + +class AddEndpointForm(forms.Form): + endpoint = forms.CharField(max_length=5000, required=True, label="Endpoint(s)", + help_text="The IP address, host name or full URL. You may enter one endpoint per line. " + "Each must be valid.", + widget=forms.widgets.Textarea(attrs={"rows": "15", "cols": "400"})) + product = forms.CharField(required=True, + label=labels.ASSET_LABEL, help_text=labels.ASSET_ENDPOINT_HELP, + widget=forms.widgets.HiddenInput()) + tags = TagField(required=False, + help_text="Add tags that help describe this endpoint. " + "Choose from the list or add new tags. Press Enter key to add.") + + def __init__(self, *args, **kwargs): + product = None + if "product" in kwargs: + product = kwargs.pop("product") + super().__init__(*args, **kwargs) + self.fields["product"] = forms.ModelChoiceField( + queryset=get_authorized_products("add"), + label=labels.ASSET_LABEL, + help_text=labels.ASSET_ENDPOINT_HELP) + if product is not None: + self.fields["product"].initial = product.id + + self.product = product + self.endpoints_to_process = [] + + def save(self): + processed_endpoints = [] + for e in self.endpoints_to_process: + endpoint, _created = endpoint_get_or_create( + protocol=e[0], + userinfo=e[1], + host=e[2], + port=e[3], + path=e[4], + query=e[5], + fragment=e[6], + product=self.product, + ) + processed_endpoints.append(endpoint) + return processed_endpoints + + def clean(self): + + cleaned_data = super().clean() + + if "endpoint" in cleaned_data and "product" in cleaned_data: + endpoint = cleaned_data["endpoint"] + product = cleaned_data["product"] + if isinstance(product, Product): + self.product = product + else: + self.product = Product.objects.get(id=int(product)) + else: + msg = "Please enter a valid URL or IP address." + raise forms.ValidationError(msg, code="invalid") + + endpoints_to_add_list, errors = validate_endpoints_to_add(endpoint) + if errors: + raise forms.ValidationError(errors) + self.endpoints_to_process = endpoints_to_add_list + + return cleaned_data + + def clean_tags(self): + tag_validator(self.cleaned_data.get("tags")) + return self.cleaned_data.get("tags") + + +class DeleteEndpointForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Endpoint + fields = ["id"] diff --git a/dojo/endpoint/urls.py b/dojo/endpoint/ui/urls.py similarity index 98% rename from dojo/endpoint/urls.py rename to dojo/endpoint/ui/urls.py index 94f6fbdcdb7..4b92af3e7b6 100644 --- a/dojo/endpoint/urls.py +++ b/dojo/endpoint/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from dojo.endpoint import views +from dojo.endpoint.ui import views urlpatterns = [ # endpoints diff --git a/dojo/endpoint/ui/views.py b/dojo/endpoint/ui/views.py new file mode 100644 index 00000000000..9e11a855118 --- /dev/null +++ b/dojo/endpoint/ui/views.py @@ -0,0 +1,506 @@ +import logging +from datetime import datetime + +from dateutil.relativedelta import relativedelta +from django.apps import apps +from django.conf import settings +from django.contrib import messages +from django.contrib.admin.utils import NestedObjects +from django.core.exceptions import PermissionDenied +from django.db import DEFAULT_DB_ALIAS +from django.db.models import OuterRef, QuerySet, Value +from django.db.models.functions import Coalesce +from django.http import HttpResponseRedirect +from django.shortcuts import get_object_or_404, render +from django.urls import reverse +from django.utils import timezone + +from dojo.authorization.authorization import user_has_permission_or_403 +from dojo.celery_dispatch import dojo_dispatch_task +from dojo.endpoint.queries import get_authorized_endpoints_for_queryset +from dojo.endpoint.ui.filters import EndpointFilter, EndpointFilterWithoutObjectLookups +from dojo.endpoint.utils import clean_hosts_run, endpoint_meta_import +from dojo.forms import ( + AddEndpointForm, + DeleteEndpointForm, + DojoMetaFormSet, + EditEndpointForm, + ImportEndpointMetaForm, +) +from dojo.models import DojoMeta, Endpoint, Endpoint_Status, Finding, Product +from dojo.query_utils import build_count_subquery +from dojo.reports.ui.views import generate_report +from dojo.utils import ( + Product_Tab, + add_breadcrumb, + add_error_message_to_response, + calculate_grade, + get_page_items, + get_period_counts, + get_setting, + get_system_setting, + is_scan_file_too_large, + redirect, +) + +logger = logging.getLogger(__name__) + + +def process_endpoints_view(request, *, host_view=False, vulnerable=False): + + if vulnerable: + endpoints = Endpoint.objects.filter( + status_endpoint__mitigated=False, + status_endpoint__false_positive=False, + status_endpoint__out_of_scope=False, + status_endpoint__risk_accepted=False) + else: + endpoints = Endpoint.objects.all() + + active_finding_subquery = build_count_subquery( + Finding.objects.filter(endpoints=OuterRef("pk"), active=True), + group_field="endpoints", + ) + endpoints = endpoints.prefetch_related("product", "product__tags", "tags").annotate( + active_finding_count=Coalesce(active_finding_subquery, Value(0)), + ).distinct() + endpoints = get_authorized_endpoints_for_queryset("view", endpoints, request.user) + filter_string_matching = get_system_setting("filter_string_matching", False) + filter_class = EndpointFilterWithoutObjectLookups if filter_string_matching else EndpointFilter + if host_view: + ids = get_endpoint_ids(filter_class(request.GET, queryset=endpoints, user=request.user).qs) + endpoints = filter_class(request.GET, queryset=endpoints.filter(id__in=ids), user=request.user) + else: + endpoints = filter_class(request.GET, queryset=endpoints, user=request.user) + + paged_endpoints = get_page_items(request, endpoints.qs, 25) + + view_name = "Vulnerable" if vulnerable else "All" + + if host_view: + view_name += " Hosts" + else: + view_name += " Endpoints" + + add_breadcrumb(title=view_name, top_level=not len(request.GET), request=request) + + product_tab = None + if "product" in request.GET: + p = request.GET.getlist("product", []) + if len(p) == 1: + product = get_object_or_404(Product, id=p[0]) + user_has_permission_or_403(request.user, product, "view") + product_tab = Product_Tab(product, view_name, tab="endpoints") + + return render( + request, "dojo/endpoints.html", { + "product_tab": product_tab, + "endpoints": paged_endpoints, + "filtered": endpoints, + "name": view_name, + "host_view": host_view, + }) + + +def get_endpoint_ids(endpoints): + hosts = [] + ids = [] + for e in endpoints: + key = f"{e.host}-{e.product.id}" + if key in hosts: + continue + hosts.append(key) + ids.append(e.id) + return ids + + +def all_endpoints(request): + return process_endpoints_view(request, host_view=False, vulnerable=False) + + +def all_endpoint_hosts(request): + return process_endpoints_view(request, host_view=True, vulnerable=False) + + +def vulnerable_endpoints(request): + return process_endpoints_view(request, host_view=False, vulnerable=True) + + +def vulnerable_endpoint_hosts(request): + return process_endpoints_view(request, host_view=True, vulnerable=True) + + +def process_endpoint_view(request, eid, *, host_view=False): + endpoint = get_object_or_404(Endpoint, id=eid) + + if host_view: + endpoints = endpoint.host_endpoints() + endpoint_metadata = None + all_findings = endpoint.host_findings() + active_findings = endpoint.host_active_findings() + else: + endpoints = None + endpoint_metadata = dict(endpoint.endpoint_meta.values_list("name", "value")) + all_findings = endpoint.findings.all() + active_findings = endpoint.active_findings() + + if all_findings: + start_date = timezone.make_aware(datetime.combine(all_findings.last().date, datetime.min.time())) + else: + start_date = timezone.now() + end_date = timezone.now() + + r = relativedelta(end_date, start_date) + months_between = (r.years * 12) + r.months + # include current month + months_between += 1 + + # closed_findings is needed as a parameter for get_periods_counts, but they are not relevant in the endpoint view + closed_findings = Finding.objects.none() + + monthly_counts = get_period_counts(all_findings, closed_findings, None, months_between, start_date, + relative_delta="months") + + paged_findings = get_page_items(request, active_findings, 25) + vulnerable = active_findings.count() != 0 + + product_tab = Product_Tab(endpoint.product, "Host" if host_view else "Endpoint", tab="endpoints") + return render(request, + "dojo/view_endpoint.html", + {"endpoint": endpoint, + "product_tab": product_tab, + "endpoints": endpoints, + "findings": paged_findings, + "all_findings": all_findings, + "opened_per_month": monthly_counts["opened_per_period"], + "endpoint_metadata": endpoint_metadata, + "vulnerable": vulnerable, + "host_view": host_view, + }) + + +def view_endpoint(request, eid): + return process_endpoint_view(request, eid, host_view=False) + + +def view_endpoint_host(request, eid): + return process_endpoint_view(request, eid, host_view=True) + + +def edit_endpoint(request, eid): + endpoint = get_object_or_404(Endpoint, id=eid) + + if request.method == "POST": + form = EditEndpointForm(request.POST, instance=endpoint) + if form.is_valid(): + logger.debug("saving endpoint") + endpoint = form.save() + messages.add_message(request, + messages.SUCCESS, + "Endpoint updated successfully.", + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_endpoint", args=(endpoint.id,))) + else: + add_breadcrumb(parent=endpoint, title="Edit", top_level=False, request=request) + form = EditEndpointForm(instance=endpoint) + + product_tab = Product_Tab(endpoint.product, "Endpoint", tab="endpoints") + + return render(request, + "dojo/edit_endpoint.html", + {"endpoint": endpoint, + "product_tab": product_tab, + "form": form, + }) + + +def delete_endpoint(request, eid): + endpoint = get_object_or_404(Endpoint, pk=eid) + product = endpoint.product + form = DeleteEndpointForm(instance=endpoint) + + if request.method == "POST": + if "id" in request.POST and str(endpoint.id) == request.POST["id"]: + form = DeleteEndpointForm(request.POST, instance=endpoint) + if form.is_valid(): + product = endpoint.product + endpoint.delete() + messages.add_message(request, + messages.SUCCESS, + "Endpoint and relationships removed.", + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_product", args=(product.id,))) + + rels = ["Previewing the relationships has been disabled.", ""] + display_preview = get_setting("DELETE_PREVIEW") + if display_preview: + collector = NestedObjects(using=DEFAULT_DB_ALIAS) + collector.collect([endpoint]) + rels = collector.nested() + + product_tab = Product_Tab(endpoint.product, "Delete Endpoint", tab="endpoints") + + return render(request, "dojo/delete_endpoint.html", + {"endpoint": endpoint, + "product_tab": product_tab, + "form": form, + "rels": rels, + }) + + +def add_endpoint(request, pid): + product = get_object_or_404(Product, id=pid) + template = "dojo/add_endpoint.html" + + form = AddEndpointForm(product=product) + if request.method == "POST": + form = AddEndpointForm(request.POST, product=product) + if form.is_valid(): + endpoints = form.save() + tags = request.POST.get("tags") + for e in endpoints: + e.tags = tags + e.save() + messages.add_message(request, + messages.SUCCESS, + "Endpoint added successfully.", + extra_tags="alert-success") + return HttpResponseRedirect(reverse("endpoint") + "?product=" + pid) + + product_tab = Product_Tab(product, "Add Endpoint", tab="endpoints") + + return render(request, template, { + "product_tab": product_tab, + "name": "Add Endpoint", + "form": form}) + + +def add_product_endpoint(request): + form = AddEndpointForm() + if request.method == "POST": + form = AddEndpointForm(request.POST) + if form.is_valid(): + user_has_permission_or_403(request.user, form.product, "add") + endpoints = form.save() + tags = request.POST.get("tags") + for e in endpoints: + e.tags = tags + e.save() + messages.add_message(request, + messages.SUCCESS, + "Endpoint added successfully.", + extra_tags="alert-success") + return HttpResponseRedirect(reverse("endpoint") + f"?product={form.product.id}") + add_breadcrumb(title="Add Endpoint", top_level=False, request=request) + return render(request, + "dojo/add_endpoint.html", + {"name": "Add Endpoint", + "form": form, + }) + + +def manage_meta_data(request, eid): + endpoint = Endpoint.objects.get(id=eid) + meta_data_query = DojoMeta.objects.filter(endpoint=endpoint) + form_mapping = {"endpoint": endpoint} + formset = DojoMetaFormSet(queryset=meta_data_query, form_kwargs={"fk_map": form_mapping}) + + if request.method == "POST": + formset = DojoMetaFormSet(request.POST, queryset=meta_data_query, form_kwargs={"fk_map": form_mapping}) + if formset.is_valid(): + formset.save() + messages.add_message( + request, messages.SUCCESS, "Metadata updated successfully.", extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_endpoint", args=(eid,))) + + add_breadcrumb(parent=endpoint, title="Manage Metadata", top_level=False, request=request) + product_tab = Product_Tab(endpoint.product, "Edit Metadata", tab="endpoints") + return render( + request, + "dojo/edit_metadata.html", + {"formset": formset, "product_tab": product_tab}, + ) + + +# bulk mitigate and delete are combined, so we can't have the nice user_is_authorized decorator +def endpoint_bulk_update_all(request, pid=None): + if request.method == "POST": + endpoints_to_update = request.POST.getlist("endpoints_to_update") + endpoints = Endpoint.objects.filter(id__in=endpoints_to_update).order_by("endpoint_meta__product__id") + total_endpoint_count = endpoints.count() + + if request.POST.get("delete_bulk_endpoints") and endpoints_to_update: + + if pid is not None: + product = get_object_or_404(Product, id=pid) + user_has_permission_or_403(request.user, product, "delete") + + endpoints = get_authorized_endpoints_for_queryset("delete", endpoints, request.user) + + skipped_endpoint_count = total_endpoint_count - endpoints.count() + deleted_endpoint_count = endpoints.count() + + product_calc = list(Product.objects.filter(endpoint__id__in=endpoints_to_update).distinct()) + endpoints.delete() + for prod in product_calc: + dojo_dispatch_task(calculate_grade, prod.id) + + if skipped_endpoint_count > 0: + add_error_message_to_response(f"Skipped deletion of {skipped_endpoint_count} endpoints because you are not authorized.") + + if deleted_endpoint_count > 0: + messages.add_message(request, + messages.SUCCESS, + f"Bulk delete of {deleted_endpoint_count} endpoints was successful.", + extra_tags="alert-success") + elif endpoints_to_update: + + if pid is not None: + product = get_object_or_404(Product, id=pid) + user_has_permission_or_403(request.user, product, "edit") + + endpoints = get_authorized_endpoints_for_queryset("edit", endpoints, request.user) + + skipped_endpoint_count = total_endpoint_count - endpoints.count() + updated_endpoint_count = endpoints.count() + + if skipped_endpoint_count > 0: + add_error_message_to_response(f"Skipped mitigation of {skipped_endpoint_count} endpoints because you are not authorized.") + + eps_count = Endpoint_Status.objects.filter(endpoint__in=endpoints).update( + mitigated=True, + mitigated_by=request.user, + mitigated_time=timezone.now(), + last_modified=timezone.now(), + ) + + if updated_endpoint_count > 0: + messages.add_message(request, + messages.SUCCESS, + f"Bulk mitigation of {updated_endpoint_count} endpoints ({eps_count} endpoint statuses) was successful.", + extra_tags="alert-success") + else: + messages.add_message(request, + messages.ERROR, + "Unable to process bulk update. Required fields were not selected.", + extra_tags="alert-danger") + return HttpResponseRedirect(reverse("endpoint", args=())) + + +def endpoint_status_bulk_update(request, fid): + if request.method == "POST": + post = request.POST + endpoints_to_update = post.getlist("endpoints_to_update") + status_list = ["active", "false_positive", "mitigated", "out_of_scope", "risk_accepted"] + enable = [item for item in status_list if item in list(post.keys())] + + if request.POST.get("remove_from_finding") and endpoints_to_update: + Endpoint_Status.objects.filter(finding_id=fid, endpoint_id__in=endpoints_to_update).delete() + messages.add_message( + request, + messages.SUCCESS, + "Selected endpoints have been removed from this finding.", + extra_tags="alert-success", + ) + elif endpoints_to_update and len(enable) > 0: + endpoints = Endpoint.objects.filter(id__in=endpoints_to_update).order_by("endpoint_meta__product__id") + for endpoint in endpoints: + endpoint_status = Endpoint_Status.objects.get( + endpoint=endpoint, + finding__id=fid) + for status in status_list: + if status in enable: + endpoint_status.__setattr__(status, True) # noqa: PLC2801 + if status == "mitigated": + endpoint_status.mitigated_by = request.user + endpoint_status.mitigated_time = timezone.now() + else: + endpoint_status.__setattr__(status, False) # noqa: PLC2801 + endpoint_status.last_modified = timezone.now() + endpoint_status.save() + messages.add_message(request, + messages.SUCCESS, + "Bulk edit of endpoints was successful. Check to make sure it is what you intended.", + extra_tags="alert-success") + else: + messages.add_message(request, + messages.ERROR, + "Unable to process bulk update. Required fields were not selected.", + extra_tags="alert-danger") + return redirect(request, post["return_url"]) + + +def prefetch_for_endpoints(endpoints): + if isinstance(endpoints, QuerySet): + endpoints = endpoints.prefetch_related("product", "tags", "product__tags") + active_finding_subquery = build_count_subquery( + Finding.objects.filter(endpoints=OuterRef("pk"), active=True), + group_field="endpoints", + ) + endpoints = endpoints.annotate(active_finding_count=Coalesce(active_finding_subquery, Value(0))) + else: + logger.debug("unable to prefetch because query was already executed") + + return endpoints + + +def migrate_endpoints_view(request): + + if not request.user.is_superuser: + raise PermissionDenied + + view_name = "Migrate endpoints" + + html_log = clean_hosts_run(apps=apps, change=(request.method == "POST")) + + return render( + request, "dojo/migrate_endpoints.html", { + "product_tab": None, + "name": view_name, + "html_log": html_log, + }) + + +def import_endpoint_meta(request, pid): + product = get_object_or_404(Product, id=pid) + form = ImportEndpointMetaForm() + if request.method == "POST": + form = ImportEndpointMetaForm(request.POST, request.FILES) + if form.is_valid(): + file = request.FILES.get("file", None) + # Make sure size is not too large + if file and is_scan_file_too_large(file): + messages.add_message( + request, + messages.ERROR, + f"Report file is too large. Maximum supported size is {settings.SCAN_FILE_MAX_SIZE} MB", + extra_tags="alert-danger") + + create_endpoints = form.cleaned_data["create_endpoints"] + create_tags = form.cleaned_data["create_tags"] + create_dojo_meta = form.cleaned_data["create_dojo_meta"] + + try: + endpoint_meta_import(file, product, create_endpoints, create_tags, create_dojo_meta, origin="UI", request=request) + except Exception as e: + logger.exception("An exception error occurred during the report import") + add_error_message_to_response(f"An exception error occurred during the report import:{e}") + return HttpResponseRedirect(reverse("endpoint") + "?product=" + pid) + + add_breadcrumb(title="Endpoint Meta Importer", top_level=False, request=request) + product_tab = Product_Tab(product, title="Endpoint Meta Importer", tab="endpoints") + return render(request, "dojo/endpoint_meta_importer.html", { + "product_tab": product_tab, + "form": form, + }) + + +def endpoint_report(request, eid): + endpoint = get_object_or_404(Endpoint, id=eid) + return generate_report(request, endpoint, host_view=False) + + +def endpoint_host_report(request, eid): + endpoint = get_object_or_404(Endpoint, id=eid) + return generate_report(request, endpoint, host_view=True) diff --git a/dojo/endpoint/views.py b/dojo/endpoint/views.py index 74c922f9ea7..3458a477812 100644 --- a/dojo/endpoint/views.py +++ b/dojo/endpoint/views.py @@ -1,506 +1,4 @@ -import logging -from datetime import datetime - -from dateutil.relativedelta import relativedelta -from django.apps import apps -from django.conf import settings -from django.contrib import messages -from django.contrib.admin.utils import NestedObjects -from django.core.exceptions import PermissionDenied -from django.db import DEFAULT_DB_ALIAS -from django.db.models import OuterRef, QuerySet, Value -from django.db.models.functions import Coalesce -from django.http import HttpResponseRedirect -from django.shortcuts import get_object_or_404, render -from django.urls import reverse -from django.utils import timezone - -from dojo.authorization.authorization import user_has_permission_or_403 -from dojo.celery_dispatch import dojo_dispatch_task -from dojo.endpoint.queries import get_authorized_endpoints_for_queryset -from dojo.endpoint.utils import clean_hosts_run, endpoint_meta_import -from dojo.filters import EndpointFilter, EndpointFilterWithoutObjectLookups -from dojo.forms import ( - AddEndpointForm, - DeleteEndpointForm, - DojoMetaFormSet, - EditEndpointForm, - ImportEndpointMetaForm, -) -from dojo.models import DojoMeta, Endpoint, Endpoint_Status, Finding, Product -from dojo.query_utils import build_count_subquery -from dojo.reports.views import generate_report -from dojo.utils import ( - Product_Tab, - add_breadcrumb, - add_error_message_to_response, - calculate_grade, - get_page_items, - get_period_counts, - get_setting, - get_system_setting, - is_scan_file_too_large, - redirect, -) - -logger = logging.getLogger(__name__) - - -def process_endpoints_view(request, *, host_view=False, vulnerable=False): - - if vulnerable: - endpoints = Endpoint.objects.filter( - status_endpoint__mitigated=False, - status_endpoint__false_positive=False, - status_endpoint__out_of_scope=False, - status_endpoint__risk_accepted=False) - else: - endpoints = Endpoint.objects.all() - - active_finding_subquery = build_count_subquery( - Finding.objects.filter(endpoints=OuterRef("pk"), active=True), - group_field="endpoints", - ) - endpoints = endpoints.prefetch_related("product", "product__tags", "tags").annotate( - active_finding_count=Coalesce(active_finding_subquery, Value(0)), - ).distinct() - endpoints = get_authorized_endpoints_for_queryset("view", endpoints, request.user) - filter_string_matching = get_system_setting("filter_string_matching", False) - filter_class = EndpointFilterWithoutObjectLookups if filter_string_matching else EndpointFilter - if host_view: - ids = get_endpoint_ids(filter_class(request.GET, queryset=endpoints, user=request.user).qs) - endpoints = filter_class(request.GET, queryset=endpoints.filter(id__in=ids), user=request.user) - else: - endpoints = filter_class(request.GET, queryset=endpoints, user=request.user) - - paged_endpoints = get_page_items(request, endpoints.qs, 25) - - view_name = "Vulnerable" if vulnerable else "All" - - if host_view: - view_name += " Hosts" - else: - view_name += " Endpoints" - - add_breadcrumb(title=view_name, top_level=not len(request.GET), request=request) - - product_tab = None - if "product" in request.GET: - p = request.GET.getlist("product", []) - if len(p) == 1: - product = get_object_or_404(Product, id=p[0]) - user_has_permission_or_403(request.user, product, "view") - product_tab = Product_Tab(product, view_name, tab="endpoints") - - return render( - request, "dojo/endpoints.html", { - "product_tab": product_tab, - "endpoints": paged_endpoints, - "filtered": endpoints, - "name": view_name, - "host_view": host_view, - }) - - -def get_endpoint_ids(endpoints): - hosts = [] - ids = [] - for e in endpoints: - key = f"{e.host}-{e.product.id}" - if key in hosts: - continue - hosts.append(key) - ids.append(e.id) - return ids - - -def all_endpoints(request): - return process_endpoints_view(request, host_view=False, vulnerable=False) - - -def all_endpoint_hosts(request): - return process_endpoints_view(request, host_view=True, vulnerable=False) - - -def vulnerable_endpoints(request): - return process_endpoints_view(request, host_view=False, vulnerable=True) - - -def vulnerable_endpoint_hosts(request): - return process_endpoints_view(request, host_view=True, vulnerable=True) - - -def process_endpoint_view(request, eid, *, host_view=False): - endpoint = get_object_or_404(Endpoint, id=eid) - - if host_view: - endpoints = endpoint.host_endpoints() - endpoint_metadata = None - all_findings = endpoint.host_findings() - active_findings = endpoint.host_active_findings() - else: - endpoints = None - endpoint_metadata = dict(endpoint.endpoint_meta.values_list("name", "value")) - all_findings = endpoint.findings.all() - active_findings = endpoint.active_findings() - - if all_findings: - start_date = timezone.make_aware(datetime.combine(all_findings.last().date, datetime.min.time())) - else: - start_date = timezone.now() - end_date = timezone.now() - - r = relativedelta(end_date, start_date) - months_between = (r.years * 12) + r.months - # include current month - months_between += 1 - - # closed_findings is needed as a parameter for get_periods_counts, but they are not relevant in the endpoint view - closed_findings = Finding.objects.none() - - monthly_counts = get_period_counts(all_findings, closed_findings, None, months_between, start_date, - relative_delta="months") - - paged_findings = get_page_items(request, active_findings, 25) - vulnerable = active_findings.count() != 0 - - product_tab = Product_Tab(endpoint.product, "Host" if host_view else "Endpoint", tab="endpoints") - return render(request, - "dojo/view_endpoint.html", - {"endpoint": endpoint, - "product_tab": product_tab, - "endpoints": endpoints, - "findings": paged_findings, - "all_findings": all_findings, - "opened_per_month": monthly_counts["opened_per_period"], - "endpoint_metadata": endpoint_metadata, - "vulnerable": vulnerable, - "host_view": host_view, - }) - - -def view_endpoint(request, eid): - return process_endpoint_view(request, eid, host_view=False) - - -def view_endpoint_host(request, eid): - return process_endpoint_view(request, eid, host_view=True) - - -def edit_endpoint(request, eid): - endpoint = get_object_or_404(Endpoint, id=eid) - - if request.method == "POST": - form = EditEndpointForm(request.POST, instance=endpoint) - if form.is_valid(): - logger.debug("saving endpoint") - endpoint = form.save() - messages.add_message(request, - messages.SUCCESS, - "Endpoint updated successfully.", - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_endpoint", args=(endpoint.id,))) - else: - add_breadcrumb(parent=endpoint, title="Edit", top_level=False, request=request) - form = EditEndpointForm(instance=endpoint) - - product_tab = Product_Tab(endpoint.product, "Endpoint", tab="endpoints") - - return render(request, - "dojo/edit_endpoint.html", - {"endpoint": endpoint, - "product_tab": product_tab, - "form": form, - }) - - -def delete_endpoint(request, eid): - endpoint = get_object_or_404(Endpoint, pk=eid) - product = endpoint.product - form = DeleteEndpointForm(instance=endpoint) - - if request.method == "POST": - if "id" in request.POST and str(endpoint.id) == request.POST["id"]: - form = DeleteEndpointForm(request.POST, instance=endpoint) - if form.is_valid(): - product = endpoint.product - endpoint.delete() - messages.add_message(request, - messages.SUCCESS, - "Endpoint and relationships removed.", - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_product", args=(product.id,))) - - rels = ["Previewing the relationships has been disabled.", ""] - display_preview = get_setting("DELETE_PREVIEW") - if display_preview: - collector = NestedObjects(using=DEFAULT_DB_ALIAS) - collector.collect([endpoint]) - rels = collector.nested() - - product_tab = Product_Tab(endpoint.product, "Delete Endpoint", tab="endpoints") - - return render(request, "dojo/delete_endpoint.html", - {"endpoint": endpoint, - "product_tab": product_tab, - "form": form, - "rels": rels, - }) - - -def add_endpoint(request, pid): - product = get_object_or_404(Product, id=pid) - template = "dojo/add_endpoint.html" - - form = AddEndpointForm(product=product) - if request.method == "POST": - form = AddEndpointForm(request.POST, product=product) - if form.is_valid(): - endpoints = form.save() - tags = request.POST.get("tags") - for e in endpoints: - e.tags = tags - e.save() - messages.add_message(request, - messages.SUCCESS, - "Endpoint added successfully.", - extra_tags="alert-success") - return HttpResponseRedirect(reverse("endpoint") + "?product=" + pid) - - product_tab = Product_Tab(product, "Add Endpoint", tab="endpoints") - - return render(request, template, { - "product_tab": product_tab, - "name": "Add Endpoint", - "form": form}) - - -def add_product_endpoint(request): - form = AddEndpointForm() - if request.method == "POST": - form = AddEndpointForm(request.POST) - if form.is_valid(): - user_has_permission_or_403(request.user, form.product, "add") - endpoints = form.save() - tags = request.POST.get("tags") - for e in endpoints: - e.tags = tags - e.save() - messages.add_message(request, - messages.SUCCESS, - "Endpoint added successfully.", - extra_tags="alert-success") - return HttpResponseRedirect(reverse("endpoint") + f"?product={form.product.id}") - add_breadcrumb(title="Add Endpoint", top_level=False, request=request) - return render(request, - "dojo/add_endpoint.html", - {"name": "Add Endpoint", - "form": form, - }) - - -def manage_meta_data(request, eid): - endpoint = Endpoint.objects.get(id=eid) - meta_data_query = DojoMeta.objects.filter(endpoint=endpoint) - form_mapping = {"endpoint": endpoint} - formset = DojoMetaFormSet(queryset=meta_data_query, form_kwargs={"fk_map": form_mapping}) - - if request.method == "POST": - formset = DojoMetaFormSet(request.POST, queryset=meta_data_query, form_kwargs={"fk_map": form_mapping}) - if formset.is_valid(): - formset.save() - messages.add_message( - request, messages.SUCCESS, "Metadata updated successfully.", extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_endpoint", args=(eid,))) - - add_breadcrumb(parent=endpoint, title="Manage Metadata", top_level=False, request=request) - product_tab = Product_Tab(endpoint.product, "Edit Metadata", tab="endpoints") - return render( - request, - "dojo/edit_metadata.html", - {"formset": formset, "product_tab": product_tab}, - ) - - -# bulk mitigate and delete are combined, so we can't have the nice user_is_authorized decorator -def endpoint_bulk_update_all(request, pid=None): - if request.method == "POST": - endpoints_to_update = request.POST.getlist("endpoints_to_update") - endpoints = Endpoint.objects.filter(id__in=endpoints_to_update).order_by("endpoint_meta__product__id") - total_endpoint_count = endpoints.count() - - if request.POST.get("delete_bulk_endpoints") and endpoints_to_update: - - if pid is not None: - product = get_object_or_404(Product, id=pid) - user_has_permission_or_403(request.user, product, "delete") - - endpoints = get_authorized_endpoints_for_queryset("delete", endpoints, request.user) - - skipped_endpoint_count = total_endpoint_count - endpoints.count() - deleted_endpoint_count = endpoints.count() - - product_calc = list(Product.objects.filter(endpoint__id__in=endpoints_to_update).distinct()) - endpoints.delete() - for prod in product_calc: - dojo_dispatch_task(calculate_grade, prod.id) - - if skipped_endpoint_count > 0: - add_error_message_to_response(f"Skipped deletion of {skipped_endpoint_count} endpoints because you are not authorized.") - - if deleted_endpoint_count > 0: - messages.add_message(request, - messages.SUCCESS, - f"Bulk delete of {deleted_endpoint_count} endpoints was successful.", - extra_tags="alert-success") - elif endpoints_to_update: - - if pid is not None: - product = get_object_or_404(Product, id=pid) - user_has_permission_or_403(request.user, product, "edit") - - endpoints = get_authorized_endpoints_for_queryset("edit", endpoints, request.user) - - skipped_endpoint_count = total_endpoint_count - endpoints.count() - updated_endpoint_count = endpoints.count() - - if skipped_endpoint_count > 0: - add_error_message_to_response(f"Skipped mitigation of {skipped_endpoint_count} endpoints because you are not authorized.") - - eps_count = Endpoint_Status.objects.filter(endpoint__in=endpoints).update( - mitigated=True, - mitigated_by=request.user, - mitigated_time=timezone.now(), - last_modified=timezone.now(), - ) - - if updated_endpoint_count > 0: - messages.add_message(request, - messages.SUCCESS, - f"Bulk mitigation of {updated_endpoint_count} endpoints ({eps_count} endpoint statuses) was successful.", - extra_tags="alert-success") - else: - messages.add_message(request, - messages.ERROR, - "Unable to process bulk update. Required fields were not selected.", - extra_tags="alert-danger") - return HttpResponseRedirect(reverse("endpoint", args=())) - - -def endpoint_status_bulk_update(request, fid): - if request.method == "POST": - post = request.POST - endpoints_to_update = post.getlist("endpoints_to_update") - status_list = ["active", "false_positive", "mitigated", "out_of_scope", "risk_accepted"] - enable = [item for item in status_list if item in list(post.keys())] - - if request.POST.get("remove_from_finding") and endpoints_to_update: - Endpoint_Status.objects.filter(finding_id=fid, endpoint_id__in=endpoints_to_update).delete() - messages.add_message( - request, - messages.SUCCESS, - "Selected endpoints have been removed from this finding.", - extra_tags="alert-success", - ) - elif endpoints_to_update and len(enable) > 0: - endpoints = Endpoint.objects.filter(id__in=endpoints_to_update).order_by("endpoint_meta__product__id") - for endpoint in endpoints: - endpoint_status = Endpoint_Status.objects.get( - endpoint=endpoint, - finding__id=fid) - for status in status_list: - if status in enable: - endpoint_status.__setattr__(status, True) # noqa: PLC2801 - if status == "mitigated": - endpoint_status.mitigated_by = request.user - endpoint_status.mitigated_time = timezone.now() - else: - endpoint_status.__setattr__(status, False) # noqa: PLC2801 - endpoint_status.last_modified = timezone.now() - endpoint_status.save() - messages.add_message(request, - messages.SUCCESS, - "Bulk edit of endpoints was successful. Check to make sure it is what you intended.", - extra_tags="alert-success") - else: - messages.add_message(request, - messages.ERROR, - "Unable to process bulk update. Required fields were not selected.", - extra_tags="alert-danger") - return redirect(request, post["return_url"]) - - -def prefetch_for_endpoints(endpoints): - if isinstance(endpoints, QuerySet): - endpoints = endpoints.prefetch_related("product", "tags", "product__tags") - active_finding_subquery = build_count_subquery( - Finding.objects.filter(endpoints=OuterRef("pk"), active=True), - group_field="endpoints", - ) - endpoints = endpoints.annotate(active_finding_count=Coalesce(active_finding_subquery, Value(0))) - else: - logger.debug("unable to prefetch because query was already executed") - - return endpoints - - -def migrate_endpoints_view(request): - - if not request.user.is_superuser: - raise PermissionDenied - - view_name = "Migrate endpoints" - - html_log = clean_hosts_run(apps=apps, change=(request.method == "POST")) - - return render( - request, "dojo/migrate_endpoints.html", { - "product_tab": None, - "name": view_name, - "html_log": html_log, - }) - - -def import_endpoint_meta(request, pid): - product = get_object_or_404(Product, id=pid) - form = ImportEndpointMetaForm() - if request.method == "POST": - form = ImportEndpointMetaForm(request.POST, request.FILES) - if form.is_valid(): - file = request.FILES.get("file", None) - # Make sure size is not too large - if file and is_scan_file_too_large(file): - messages.add_message( - request, - messages.ERROR, - f"Report file is too large. Maximum supported size is {settings.SCAN_FILE_MAX_SIZE} MB", - extra_tags="alert-danger") - - create_endpoints = form.cleaned_data["create_endpoints"] - create_tags = form.cleaned_data["create_tags"] - create_dojo_meta = form.cleaned_data["create_dojo_meta"] - - try: - endpoint_meta_import(file, product, create_endpoints, create_tags, create_dojo_meta, origin="UI", request=request) - except Exception as e: - logger.exception("An exception error occurred during the report import") - add_error_message_to_response(f"An exception error occurred during the report import:{e}") - return HttpResponseRedirect(reverse("endpoint") + "?product=" + pid) - - add_breadcrumb(title="Endpoint Meta Importer", top_level=False, request=request) - product_tab = Product_Tab(product, title="Endpoint Meta Importer", tab="endpoints") - return render(request, "dojo/endpoint_meta_importer.html", { - "product_tab": product_tab, - "form": form, - }) - - -def endpoint_report(request, eid): - endpoint = get_object_or_404(Endpoint, id=eid) - return generate_report(request, endpoint, host_view=False) - - -def endpoint_host_report(request, eid): - endpoint = get_object_or_404(Endpoint, id=eid) - return generate_report(request, endpoint, host_view=True) +# Backward-compat shim: the view logic moved to dojo.endpoint.ui.views during the +# module reorg. External consumers (e.g. dojo-pro) still import from +# dojo.endpoint.views, so re-export the public names from their new location. +from dojo.endpoint.ui.views import * # noqa: F403 -- backward compat re-export diff --git a/dojo/engagement/__init__.py b/dojo/engagement/__init__.py index e69de29bb2d..4dd8749c6cf 100644 --- a/dojo/engagement/__init__.py +++ b/dojo/engagement/__init__.py @@ -0,0 +1 @@ +import dojo.engagement.admin # noqa: F401 diff --git a/dojo/engagement/admin.py b/dojo/engagement/admin.py new file mode 100644 index 00000000000..921b7593b64 --- /dev/null +++ b/dojo/engagement/admin.py @@ -0,0 +1,15 @@ +from django.contrib import admin + +from dojo.engagement.models import Engagement, Engagement_Presets + + +@admin.register(Engagement_Presets) +class EngagementPresetsAdmin(admin.ModelAdmin): + + """Admin support for the Engagement_Presets model.""" + + +@admin.register(Engagement) +class EngagementAdmin(admin.ModelAdmin): + + """Admin support for the Engagement model.""" diff --git a/dojo/engagement/api/__init__.py b/dojo/engagement/api/__init__.py new file mode 100644 index 00000000000..60e35ed2e10 --- /dev/null +++ b/dojo/engagement/api/__init__.py @@ -0,0 +1 @@ +path = "engagements" # noqa: RUF067 diff --git a/dojo/engagement/api/filters.py b/dojo/engagement/api/filters.py new file mode 100644 index 00000000000..1015b019fd0 --- /dev/null +++ b/dojo/engagement/api/filters.py @@ -0,0 +1,69 @@ +from django_filters import ( + BooleanFilter, + CharFilter, + OrderingFilter, +) + +from dojo.filters import ( + CharFieldFilterANDExpression, + CharFieldInFilter, + DojoFilter, + NumberInFilter, +) +from dojo.labels import get_labels +from dojo.models import Engagement + +labels = get_labels() + + +class ApiEngagementFilter(DojoFilter): + product__prod_type = NumberInFilter(field_name="product__prod_type", lookup_expr="in") + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Tag name contains") + tags = CharFieldInFilter( + field_name="tags__name", + lookup_expr="in", + help_text="Comma separated list of exact tags (uses OR for multiple values)") + tags__and = CharFieldFilterANDExpression( + field_name="tags__name", + help_text="Comma separated list of exact tags to match with an AND expression") + product__tags = CharFieldInFilter( + field_name="product__tags__name", + lookup_expr="in", + help_text=labels.ASSET_FILTERS_CSV_TAGS_OR_HELP) + product__tags__and = CharFieldFilterANDExpression( + field_name="product__tags__name", + help_text=labels.ASSET_FILTERS_CSV_TAGS_AND_HELP) + + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Not Tag name contains", exclude="True") + not_tags = CharFieldInFilter(field_name="tags__name", lookup_expr="in", + help_text="Comma separated list of exact tags not present on model", exclude="True") + not_product__tags = CharFieldInFilter(field_name="product__tags__name", + lookup_expr="in", + help_text=labels.ASSET_FILTERS_CSV_TAGS_NOT_HELP, + exclude="True") + has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("name", "name"), + ("version", "version"), + ("target_start", "target_start"), + ("target_end", "target_end"), + ("status", "status"), + ("lead", "lead"), + ("created", "created"), + ("updated", "updated"), + ), + field_labels={ + "name": "Engagement Name", + }, + + ) + + class Meta: + model = Engagement + fields = ["id", "active", "target_start", + "target_end", "requester", "report_type", + "updated", "threat_model", "api_test", + "pen_test", "status", "product", "name", "version", "tags"] diff --git a/dojo/engagement/api/serializer.py b/dojo/engagement/api/serializer.py new file mode 100644 index 00000000000..7cae1b39485 --- /dev/null +++ b/dojo/engagement/api/serializer.py @@ -0,0 +1,87 @@ +from django.conf import settings +from rest_framework import serializers + +from dojo.models import Check_List, Engagement, Engagement_Presets + + +class EngagementSerializer(serializers.ModelSerializer): + class Meta: + model = Engagement + exclude = ("inherited_tags",) + + def get_fields(self): + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + TagListSerializerField, + ) + fields = super().get_fields() + fields["tags"] = TagListSerializerField(required=False) + return fields + + def validate(self, data): + if self.context["request"].method == "POST": + if data.get("target_start") > data.get("target_end"): + msg = "Your target start date exceeds your target end date" + raise serializers.ValidationError(msg) + return data + + def build_relational_field(self, field_name, relation_info): + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + FileSerializer, + NoteSerializer, + ) + if field_name == "notes": + return NoteSerializer, {"many": True, "read_only": True} + if field_name == "files": + return FileSerializer, {"many": True, "read_only": True} + return super().build_relational_field(field_name, relation_info) + + +class EngagementToNotesSerializer(serializers.Serializer): + engagement_id = serializers.PrimaryKeyRelatedField( + queryset=Engagement.objects.all(), many=False, allow_null=True, + ) + + def get_fields(self): + from dojo.api_v2.serializers import NoteSerializer # noqa: PLC0415 -- lazy import, avoids circular dependency + fields = super().get_fields() + fields["notes"] = NoteSerializer(many=True) + return fields + + +class EngagementToFilesSerializer(serializers.Serializer): + engagement_id = serializers.PrimaryKeyRelatedField( + queryset=Engagement.objects.all(), many=False, allow_null=True, + ) + + def get_fields(self): + from dojo.api_v2.serializers import FileSerializer # noqa: PLC0415 -- lazy import, avoids circular dependency + fields = super().get_fields() + fields["files"] = FileSerializer(many=True) + return fields + + def to_representation(self, data): + engagement = data.get("engagement_id") + files = data.get("files") + new_files = [{ + "id": file.id, + "file": "{site_url}/{file_access_url}".format( + site_url=settings.SITE_URL, + file_access_url=file.get_accessible_url( + engagement, engagement.id, + ), + ), + "title": file.title, + } for file in files] + return {"engagement_id": engagement.id, "files": new_files} + + +class EngagementCheckListSerializer(serializers.ModelSerializer): + class Meta: + model = Check_List + fields = "__all__" + + +class EngagementPresetsSerializer(serializers.ModelSerializer): + class Meta: + model = Engagement_Presets + fields = "__all__" diff --git a/dojo/engagement/api/urls.py b/dojo/engagement/api/urls.py new file mode 100644 index 00000000000..7c5ba0c2758 --- /dev/null +++ b/dojo/engagement/api/urls.py @@ -0,0 +1,7 @@ +from dojo.engagement.api.views import EngagementPresetsViewset, EngagementViewSet + + +def add_engagement_urls(router): + router.register("engagements", EngagementViewSet, basename="engagement") + router.register("engagement_presets", EngagementPresetsViewset, basename="engagement_presets") + return router diff --git a/dojo/engagement/api/views.py b/dojo/engagement/api/views.py new file mode 100644 index 00000000000..34ed278a358 --- /dev/null +++ b/dojo/engagement/api/views.py @@ -0,0 +1,371 @@ +from django.core.exceptions import ValidationError +from django.urls import reverse +from django_filters.rest_framework import DjangoFilterBackend +from drf_spectacular.types import OpenApiTypes +from drf_spectacular.utils import extend_schema, extend_schema_view +from rest_framework import status +from rest_framework.decorators import action +from rest_framework.parsers import MultiPartParser +from rest_framework.permissions import IsAuthenticated +from rest_framework.response import Response + +from dojo.api_v2 import serializers as api_v2_serializers +from dojo.api_v2.prefetch.prefetcher import _Prefetcher +from dojo.api_v2.views import DojoModelViewSet, PrefetchDojoModelViewSet, report_generate, schema_with_prefetch +from dojo.authorization import api_permissions as permissions +from dojo.celery_dispatch import dojo_dispatch_task +from dojo.engagement.api.filters import ApiEngagementFilter +from dojo.engagement.api.serializer import ( + EngagementCheckListSerializer, + EngagementPresetsSerializer, + EngagementSerializer, + EngagementToFilesSerializer, + EngagementToNotesSerializer, +) +from dojo.engagement.queries import get_authorized_engagements +from dojo.engagement.services import close_engagement, reopen_engagement +from dojo.jira import services as jira_services +from dojo.models import ( + Check_List, + Engagement, + Engagement_Presets, + FileUpload, + NoteHistory, + Notes, +) +from dojo.product.queries import get_authorized_engagement_presets +from dojo.risk_acceptance import api as ra_api +from dojo.utils import ( + async_delete, + generate_file_response, + get_setting, + process_tag_notifications, +) + + +# Authorization: object-based +# @extend_schema_view(**schema_with_prefetch()) +# Nested models with prefetch make the response schema too long for Swagger UI +class EngagementViewSet( + # PrefetchDojoModelViewSet, + DojoModelViewSet, + ra_api.AcceptedRisksMixin, +): + serializer_class = EngagementSerializer + queryset = Engagement.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_class = ApiEngagementFilter + + permission_classes = ( + IsAuthenticated, + permissions.UserHasEngagementPermission, + ) + + @property + def risk_application_model_class(self): + return Engagement + + def destroy(self, request, *args, **kwargs): + instance = self.get_object() + if get_setting("ASYNC_OBJECT_DELETE"): + async_del = async_delete() + async_del.delete(instance) + else: + instance.delete() + return Response(status=status.HTTP_204_NO_CONTENT) + + def get_queryset(self): + return ( + get_authorized_engagements("view") + .prefetch_related("notes", "risk_acceptance", "files") + .distinct() + ) + + @extend_schema( + request=OpenApiTypes.NONE, responses={status.HTTP_200_OK: ""}, + ) + @action(detail=True, methods=["post"], permission_classes=(IsAuthenticated, permissions.UserHasEngagementRelatedObjectPermission)) + def close(self, request, pk=None): + eng = self.get_object() + close_engagement(eng) + return Response({}, status=status.HTTP_200_OK) + + @extend_schema( + request=OpenApiTypes.NONE, responses={status.HTTP_200_OK: ""}, + ) + @action(detail=True, methods=["post"], permission_classes=(IsAuthenticated, permissions.UserHasEngagementRelatedObjectPermission)) + def reopen(self, request, pk=None): + eng = self.get_object() + reopen_engagement(eng) + return Response({}, status=status.HTTP_200_OK) + + @extend_schema( + request=api_v2_serializers.ReportGenerateOptionSerializer, + responses={status.HTTP_200_OK: api_v2_serializers.ReportGenerateSerializer}, + ) + @action( + detail=True, methods=["post"], + # IsAuthenticated only: report generation requires View permission, + # enforced by the permission-filtered get_queryset(). The viewset's + # permission_classes would check Edit (POST), which is too restrictive. + permission_classes=[IsAuthenticated], + ) + def generate_report(self, request, pk=None): + engagement = self.get_object() + + options = {} + # prepare post data + report_options = api_v2_serializers.ReportGenerateOptionSerializer( + data=request.data, + ) + if report_options.is_valid(): + options["include_finding_notes"] = report_options.validated_data[ + "include_finding_notes" + ] + options["include_finding_images"] = report_options.validated_data[ + "include_finding_images" + ] + options[ + "include_executive_summary" + ] = report_options.validated_data["include_executive_summary"] + options[ + "include_table_of_contents" + ] = report_options.validated_data["include_table_of_contents"] + else: + return Response( + report_options.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + data = report_generate(request, engagement, options) + report = api_v2_serializers.ReportGenerateSerializer(data) + return Response(report.data) + + @extend_schema( + methods=["GET"], + responses={ + status.HTTP_200_OK: EngagementToNotesSerializer, + }, + ) + @extend_schema( + methods=["POST"], + request=api_v2_serializers.AddNewNoteOptionSerializer, + responses={status.HTTP_201_CREATED: api_v2_serializers.NoteSerializer}, + ) + @action(detail=True, methods=["get", "post"], permission_classes=[IsAuthenticated, permissions.UserHasEngagementNotePermission]) + def notes(self, request, pk=None): + engagement = self.get_object() + if request.method == "POST": + new_note = api_v2_serializers.AddNewNoteOptionSerializer( + data=request.data, + ) + if new_note.is_valid(): + entry = new_note.validated_data["entry"] + private = new_note.validated_data.get("private", False) + note_type = new_note.validated_data.get("note_type", None) + else: + return Response( + new_note.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + notes = engagement.notes.filter(note_type=note_type).first() + if notes and note_type and note_type.is_single: + return Response("Only one instance of this note_type allowed on an engagement.", status=status.HTTP_400_BAD_REQUEST) + + author = request.user + note = Notes( + entry=entry, + author=author, + private=private, + note_type=note_type, + ) + note.save() + # Add an entry to the note history + history = NoteHistory.objects.create(data=note.entry, time=note.date, current_editor=note.author) + note.history.add(history) + # Now add the note to the object + engagement.notes.add(note) + # Determine if we need to send any notifications for user mentioned + process_tag_notifications( + request=request, + note=note, + parent_url=request.build_absolute_uri( + reverse("view_engagement", args=(engagement.id,)), + ), + parent_title=f"Engagement: {engagement.name}", + ) + + serialized_note = api_v2_serializers.NoteSerializer( + {"author": author, "entry": entry, "private": private}, + ) + return Response( + serialized_note.data, status=status.HTTP_201_CREATED, + ) + notes = engagement.notes.all() + + serialized_notes = EngagementToNotesSerializer( + {"engagement_id": engagement, "notes": notes}, + ) + return Response(serialized_notes.data, status=status.HTTP_200_OK) + + @extend_schema( + methods=["GET"], + responses={ + status.HTTP_200_OK: EngagementToFilesSerializer, + }, + ) + @extend_schema( + methods=["POST"], + request=api_v2_serializers.AddNewFileOptionSerializer, + responses={status.HTTP_201_CREATED: api_v2_serializers.FileSerializer}, + ) + @action( + detail=True, methods=["get", "post"], parser_classes=(MultiPartParser,), permission_classes=[IsAuthenticated, permissions.UserHasEngagementRelatedObjectPermission], + ) + def files(self, request, pk=None): + engagement = self.get_object() + if request.method == "POST": + new_file = api_v2_serializers.FileSerializer(data=request.data) + if new_file.is_valid(): + title = new_file.validated_data["title"] + file = new_file.validated_data["file"] + else: + return Response( + new_file.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + file = FileUpload(title=title, file=file) + file.save() + engagement.files.add(file) + + serialized_file = api_v2_serializers.FileSerializer(file) + return Response( + serialized_file.data, status=status.HTTP_201_CREATED, + ) + + files = engagement.files.all() + serialized_files = EngagementToFilesSerializer( + {"engagement_id": engagement, "files": files}, + ) + return Response(serialized_files.data, status=status.HTTP_200_OK) + + @extend_schema( + methods=["POST"], + request=EngagementCheckListSerializer, + responses={ + status.HTTP_201_CREATED: EngagementCheckListSerializer, + }, + ) + @action(detail=True, methods=["get", "post"], permission_classes=[IsAuthenticated, permissions.UserHasEngagementRelatedObjectPermission]) + def complete_checklist(self, request, pk=None): + engagement = self.get_object() + check_lists = Check_List.objects.filter(engagement=engagement) + if request.method == "POST": + if check_lists.count() > 0: + return Response( + { + "message": "A completed checklist for this engagement already exists.", + }, + status=status.HTTP_400_BAD_REQUEST, + ) + check_list = EngagementCheckListSerializer( + data=request.data, + ) + if not check_list.is_valid(): + return Response( + check_list.errors, status=status.HTTP_400_BAD_REQUEST, + ) + check_list = Check_List(**check_list.data) + check_list.engagement = engagement + check_list.save() + serialized_check_list = EngagementCheckListSerializer( + check_list, + ) + return Response( + serialized_check_list.data, status=status.HTTP_201_CREATED, + ) + prefetch_params = request.GET.get("prefetch", "").split(",") + prefetcher = _Prefetcher() + entry = check_lists.first() + # Get the queried object representation + result = EngagementCheckListSerializer(entry).data + prefetcher._prefetch(entry, prefetch_params) + result["prefetch"] = prefetcher.prefetched_data + return Response(result, status=status.HTTP_200_OK) + + @extend_schema( + methods=["GET"], + responses={ + status.HTTP_200_OK: api_v2_serializers.RawFileSerializer, + }, + ) + @action( + detail=True, + methods=["get"], + url_path=r"files/download/(?P\d+)", + permission_classes=[IsAuthenticated, permissions.UserHasEngagementRelatedObjectPermission], + ) + def download_file(self, request, file_id, pk=None): + engagement = self.get_object() + # Get the file object + file_object_qs = engagement.files.filter(id=file_id) + file_object = ( + file_object_qs.first() if len(file_object_qs) > 0 else None + ) + if file_object is None: + return Response( + {"error": "File ID not associated with Engagement"}, + status=status.HTTP_404_NOT_FOUND, + ) + # send file + return generate_file_response(file_object) + + @extend_schema( + request=api_v2_serializers.EngagementUpdateJiraEpicSerializer, + responses={status.HTTP_200_OK: api_v2_serializers.EngagementUpdateJiraEpicSerializer}, + ) + @action( + detail=True, methods=["post"], + permission_classes=(IsAuthenticated, permissions.UserHasEngagementRelatedObjectPermission), + ) + def update_jira_epic(self, request, pk=None): + engagement = self.get_object() + try: + if engagement.has_jira_issue: + task = jira_services.get_epic_task("update_epic") + if task: + dojo_dispatch_task(task, engagement.id, **request.data) + response = Response( + {"info": "Jira Epic update query sent"}, + status=status.HTTP_200_OK, + ) + else: + task = jira_services.get_epic_task("add_epic") + if task: + dojo_dispatch_task(task, engagement.id, **request.data) + response = Response( + {"info": "Jira Epic create query sent"}, + status=status.HTTP_200_OK, + ) + except ValidationError: + return Response( + {"error": "Bad Request!"}, + status=status.HTTP_400_BAD_REQUEST, + ) + return response + + +@extend_schema_view(**schema_with_prefetch()) +class EngagementPresetsViewset( + PrefetchDojoModelViewSet, +): + serializer_class = EngagementPresetsSerializer + queryset = Engagement_Presets.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = ["id", "title", "product"] + permission_classes = ( + IsAuthenticated, + permissions.UserHasEngagementPresetPermission, + ) + + def get_queryset(self): + return get_authorized_engagement_presets("view") diff --git a/dojo/engagement/models.py b/dojo/engagement/models.py new file mode 100644 index 00000000000..ec658e4f6f7 --- /dev/null +++ b/dojo/engagement/models.py @@ -0,0 +1,184 @@ +import logging +from contextlib import suppress + +from dateutil.relativedelta import relativedelta +from django.db import models +from django.urls import reverse +from django.utils import timezone +from django.utils.translation import gettext as _ +from tagulous.models import TagField + +from dojo.base_models.base import BaseModel + +logger = logging.getLogger(__name__) + + +class Engagement_Presets(models.Model): + title = models.CharField(max_length=500, default=None, help_text=_("Brief description of preset.")) + test_type = models.ManyToManyField("dojo.Test_Type", default=None, blank=True) + network_locations = models.ManyToManyField("dojo.Network_Locations", default=None, blank=True) + notes = models.CharField(max_length=2000, help_text=_("Description of what needs to be tested or setting up environment for testing"), null=True, blank=True) + scope = models.CharField(max_length=800, help_text=_("Scope of Engagement testing, IP's/Resources/URL's)"), default=None, blank=True) + product = models.ForeignKey("dojo.Product", on_delete=models.CASCADE) + created = models.DateTimeField(auto_now_add=True, null=False) + + class Meta: + ordering = ["title"] + + def __str__(self): + return self.title + + +ENGAGEMENT_STATUS_CHOICES = (("Not Started", "Not Started"), + ("Blocked", "Blocked"), + ("Cancelled", "Cancelled"), + ("Completed", "Completed"), + ("In Progress", "In Progress"), + ("On Hold", "On Hold"), + ("Scheduled", "Scheduled"), + ("Waiting for Resource", "Waiting for Resource")) + + +class Engagement(BaseModel): + name = models.CharField(max_length=300, null=True, blank=True) + description = models.CharField(max_length=2000, null=True, blank=True) + version = models.CharField(max_length=100, null=True, blank=True, help_text=_("Version of the product the engagement tested.")) + first_contacted = models.DateField(null=True, blank=True) + target_start = models.DateField(null=False, blank=False) + target_end = models.DateField(null=False, blank=False) + lead = models.ForeignKey("dojo.Dojo_User", editable=True, null=True, blank=True, on_delete=models.RESTRICT) + requester = models.ForeignKey("dojo.Contact", null=True, blank=True, on_delete=models.CASCADE) + preset = models.ForeignKey("dojo.Engagement_Presets", null=True, blank=True, help_text=_("Settings and notes for performing this engagement."), on_delete=models.CASCADE) + reason = models.CharField(max_length=2000, null=True, blank=True) + report_type = models.ForeignKey("dojo.Report_Type", null=True, blank=True, on_delete=models.CASCADE) + product = models.ForeignKey("dojo.Product", on_delete=models.CASCADE) + active = models.BooleanField(default=True, editable=False) + tracker = models.URLField(max_length=200, help_text=_("Link to epic or ticket system with changes to version."), editable=True, blank=True, null=True) + test_strategy = models.URLField(editable=True, blank=True, null=True) + threat_model = models.BooleanField(default=True) + api_test = models.BooleanField(default=True) + pen_test = models.BooleanField(default=True) + check_list = models.BooleanField(default=True) + notes = models.ManyToManyField("dojo.Notes", blank=True, editable=False) + files = models.ManyToManyField("dojo.FileUpload", blank=True, editable=False) + status = models.CharField(editable=True, max_length=2000, default="Not Started", + null=True, + choices=ENGAGEMENT_STATUS_CHOICES) + progress = models.CharField(max_length=100, + default="threat_model", editable=False) + tmodel_path = models.CharField(max_length=1000, default="none", + editable=False, blank=True, null=True) + risk_acceptance = models.ManyToManyField("dojo.Risk_Acceptance", + default=None, + editable=False, + blank=True) + done_testing = models.BooleanField(default=False, editable=False) + engagement_type = models.CharField(editable=True, max_length=30, default="Interactive", + null=True, + choices=(("Interactive", "Interactive"), + ("CI/CD", "CI/CD"))) + build_id = models.CharField(editable=True, max_length=150, + null=True, blank=True, help_text=_("Build ID of the product the engagement tested."), verbose_name=_("Build ID")) + commit_hash = models.CharField(editable=True, max_length=150, + null=True, blank=True, help_text=_("Commit hash from repo"), verbose_name=_("Commit Hash")) + branch_tag = models.CharField(editable=True, max_length=150, + null=True, blank=True, help_text=_("Tag or branch of the product the engagement tested."), verbose_name=_("Branch/Tag")) + build_server = models.ForeignKey("dojo.Tool_Configuration", verbose_name=_("Build Server"), help_text=_("Build server responsible for CI/CD test"), null=True, blank=True, related_name="build_server", on_delete=models.CASCADE) + source_code_management_server = models.ForeignKey("dojo.Tool_Configuration", null=True, blank=True, verbose_name=_("SCM Server"), help_text=_("Source code server for CI/CD test"), related_name="source_code_management_server", on_delete=models.CASCADE) + source_code_management_uri = models.URLField(max_length=600, null=True, blank=True, editable=True, verbose_name=_("Repo"), help_text=_("Resource link to source code")) + orchestration_engine = models.ForeignKey("dojo.Tool_Configuration", verbose_name=_("Orchestration Engine"), help_text=_("Orchestration service responsible for CI/CD test"), null=True, blank=True, related_name="orchestration", on_delete=models.CASCADE) + deduplication_on_engagement = models.BooleanField(default=False, verbose_name=_("Deduplication within this engagement only"), help_text=_("If enabled deduplication will only mark a finding in this engagement as duplicate of another finding if both findings are in this engagement. If disabled, deduplication is on the product level.")) + + tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this engagement. Choose from the list or add new tags. Press Enter key to add.")) + inherited_tags = TagField(blank=True, force_lowercase=True, help_text=_("Internal use tags sepcifically for maintaining parity with product. This field will be present as a subset in the tags field")) + + class Meta: + ordering = ["-target_start"] + indexes = [ + models.Index(fields=["product", "active"]), + ] + + def __str__(self): + return "Engagement {}: {} ({})".format(self.id if id else 0, self.name or "", + self.target_start.strftime( + "%b %d, %Y")) + + def get_absolute_url(self): + return reverse("view_engagement", args=[str(self.id)]) + + def copy(self): + from dojo.models import Test, copy_model_util # noqa: PLC0415 -- lazy import, avoids circular dependency + copy = copy_model_util(self) + # Save the necessary ManyToMany relationships + old_notes = list(self.notes.all()) + old_files = list(self.files.all()) + old_tags = list(self.tags.all()) + old_risk_acceptances = list(self.risk_acceptance.all()) + old_tests = list(Test.objects.filter(engagement=self)) + # Save the object before setting any ManyToMany relationships + copy.save() + # Copy the notes + for notes in old_notes: + copy.notes.add(notes.copy()) + # Copy the files + for files in old_files: + copy.files.add(files.copy()) + # Copy the tests + for test in old_tests: + test.copy(engagement=copy) + # Copy the risk_acceptances + for risk_acceptance in old_risk_acceptances: + copy.risk_acceptance.add(risk_acceptance.copy(engagement=copy)) + # Assign any tags + copy.tags.set(old_tags) + + return copy + + def is_overdue(self): + overdue_grace_days = 10 if self.engagement_type == "CI/CD" else 0 + + max_end_date = timezone.now() - relativedelta(days=overdue_grace_days) + + return self.target_end < max_end_date.date() + + def get_breadcrumbs(self): + bc = self.product.get_breadcrumbs() + bc += [{"title": str(self), + "url": reverse("view_engagement", args=(self.id,))}] + return bc + + # only used by bulk risk acceptance api + @property + def unaccepted_open_findings(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + from dojo.utils import get_system_setting # noqa: PLC0415 circular import + + findings = Finding.objects.filter(risk_accepted=False, active=True, duplicate=False, test__engagement=self) + if get_system_setting("enforce_verified_status", True) or get_system_setting("enforce_verified_status_metrics", True): + findings = findings.filter(verified=True) + + return findings + + def accept_risks(self, accepted_risks): + self.risk_acceptance.add(*accepted_risks) + + @property + def has_jira_issue(self): + from dojo.jira import services as jira_services # noqa: PLC0415 circular import + return jira_services.has_issue(self) + + @property + def is_ci_cd(self): + return self.engagement_type == "CI/CD" + + def delete(self, *args, **kwargs): + logger.debug("%d engagement delete", self.id) + from dojo.finding import helper as finding_helper # noqa: PLC0415 circular import + finding_helper.prepare_duplicates_for_delete(self) + super().delete(*args, **kwargs) + from dojo.models import Product # noqa: PLC0415 -- lazy import, avoids circular dependency + with suppress(Engagement.DoesNotExist, Product.DoesNotExist): + # Suppressing a potential issue created from async delete removing + # related objects in a separate task + from dojo.utils import perform_product_grading # noqa: PLC0415 circular import + perform_product_grading(self.product) diff --git a/dojo/engagement/services.py b/dojo/engagement/services.py index b78844fc6bc..6a5b7570103 100644 --- a/dojo/engagement/services.py +++ b/dojo/engagement/services.py @@ -3,10 +3,14 @@ from django.db.models.signals import pre_save from django.dispatch import receiver +from django.urls import reverse +from django.utils.translation import gettext as _ from dojo.celery_dispatch import dojo_dispatch_task from dojo.jira import services as jira_services from dojo.models import Engagement +from dojo.notifications.helper import create_notification +from dojo.utils import calculate_grade logger = logging.getLogger(__name__) @@ -28,6 +32,28 @@ def reopen_engagement(eng): eng.save() +def copy_engagement(engagement, user): + """ + Copy an engagement (and its tests/findings) within the same product, recalculate the + product grade, and notify. Returns the new engagement. + + HTTP-free so both the UI view and (eventually) the API can call it. + """ + product = engagement.product + engagement_copy = engagement.copy() + dojo_dispatch_task(calculate_grade, product.id) + create_notification( + event="engagement_copied", + title=_("Copying of %s") % engagement.name, + description=f'The engagement "{engagement.name}" was copied by {user}', + product=product, + url=reverse("view_engagement", args=(engagement_copy.id,)), + recipients=[engagement.lead], + icon="exclamation-triangle", + ) + return engagement_copy + + @receiver(pre_save, sender=Engagement) def set_name_if_none(sender, instance, *args, **kwargs): if not instance.name: diff --git a/dojo/engagement/ui/__init__.py b/dojo/engagement/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/engagement/ui/filters.py b/dojo/engagement/ui/filters.py new file mode 100644 index 00000000000..055da964976 --- /dev/null +++ b/dojo/engagement/ui/filters.py @@ -0,0 +1,399 @@ +from django.conf import settings +from django_filters import ( + BooleanFilter, + CharFilter, + FilterSet, + ModelChoiceFilter, + ModelMultipleChoiceFilter, + MultipleChoiceFilter, + OrderingFilter, +) + +from dojo.filters import DateRangeFilter, DojoFilter +from dojo.labels import get_labels +from dojo.models import ( + ENGAGEMENT_STATUS_CHOICES, + Dojo_User, + Engagement, + Product, + Product_API_Scan_Configuration, + Product_Type, + Test, + Test_Type, +) +from dojo.product_type.queries import get_authorized_product_types +from dojo.user.queries import get_authorized_users + +labels = get_labels() + + +class EngagementDirectFilterHelper(FilterSet): + name = CharFilter(lookup_expr="icontains", label="Engagement name contains") + version = CharFilter(field_name="version", lookup_expr="icontains", label="Engagement version") + test__version = CharFilter(field_name="test__version", lookup_expr="icontains", label="Test version") + product__name = CharFilter(lookup_expr="icontains", label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL) + status = MultipleChoiceFilter(choices=ENGAGEMENT_STATUS_CHOICES, label="Status") + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) + has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") + target_start = DateRangeFilter() + target_end = DateRangeFilter() + test__engagement__product__lifecycle = MultipleChoiceFilter( + choices=Product.LIFECYCLE_CHOICES, + label=labels.ASSET_LIFECYCLE_LABEL, + null_label="Empty") + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("target_start", "target_start"), + ("name", "name"), + ("product__name", "product__name"), + ("product__prod_type__name", "product__prod_type__name"), + ("lead__first_name", "lead__first_name"), + ), + field_labels={ + "target_start": "Start date", + "name": "Engagement", + "product__name": labels.ASSET_FILTERS_NAME_LABEL, + "product__prod_type__name": labels.ORG_FILTERS_LABEL, + "lead__first_name": "Lead", + }, + ) + + +class EngagementDirectFilter(EngagementDirectFilterHelper, DojoFilter): + lead = ModelChoiceFilter(queryset=Dojo_User.objects.none(), label="Lead") + product__prod_type = ModelMultipleChoiceFilter( + queryset=Product_Type.objects.none(), + label=labels.ORG_FILTERS_LABEL) + tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + queryset=Engagement.tags.tag_model.objects.all().order_by("name")) + not_tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + exclude=True, + queryset=Engagement.tags.tag_model.objects.all().order_by("name")) + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.form.fields["product__prod_type"].queryset = get_authorized_product_types("view") + self.form.fields["lead"].queryset = get_authorized_users("view") \ + .filter(engagement__lead__isnull=False).distinct() + + class Meta: + model = Engagement + fields = ["product__name", "product__prod_type"] + + +class EngagementDirectFilterWithoutObjectLookups(EngagementDirectFilterHelper): + lead = CharFilter( + field_name="lead__username", + lookup_expr="iexact", + label="Lead Username", + help_text="Search for Lead username that are an exact match") + lead_contains = CharFilter( + field_name="lead__username", + lookup_expr="icontains", + label="Lead Username Contains", + help_text="Search for Lead username that contain a given pattern") + product__prod_type__name = CharFilter( + field_name="product__prod_type__name", + lookup_expr="iexact", + label=labels.ORG_FILTERS_NAME_LABEL, + help_text=labels.ORG_FILTERS_NAME_HELP) + product__prod_type__name_contains = CharFilter( + field_name="product__prod_type__name", + lookup_expr="icontains", + label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, + help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) + + class Meta: + model = Engagement + fields = ["product__name"] + + +class EngagementFilterHelper(FilterSet): + name = CharFilter(lookup_expr="icontains", label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL) + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) + has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") + engagement__name = CharFilter(lookup_expr="icontains", label="Engagement name contains") + engagement__version = CharFilter(field_name="engagement__version", lookup_expr="icontains", label="Engagement version") + engagement__test__version = CharFilter(field_name="engagement__test__version", lookup_expr="icontains", label="Test version") + engagement__product__lifecycle = MultipleChoiceFilter( + choices=Product.LIFECYCLE_CHOICES, + label=labels.ASSET_LIFECYCLE_LABEL, + null_label="Empty") + engagement__status = MultipleChoiceFilter( + choices=ENGAGEMENT_STATUS_CHOICES, + label="Status") + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("name", "name"), + ("prod_type__name", "prod_type__name"), + ), + field_labels={ + "name": labels.ASSET_FILTERS_NAME_LABEL, + "prod_type__name": labels.ORG_FILTERS_LABEL, + }, + ) + + +class EngagementFilter(EngagementFilterHelper, DojoFilter): + engagement__lead = ModelChoiceFilter( + queryset=Dojo_User.objects.none(), + label="Lead") + prod_type = ModelMultipleChoiceFilter( + queryset=Product_Type.objects.none(), + label=labels.ORG_FILTERS_LABEL) + tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + queryset=Engagement.tags.tag_model.objects.all().order_by("name")) + not_tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + exclude=True, + queryset=Engagement.tags.tag_model.objects.all().order_by("name")) + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.form.fields["prod_type"].queryset = get_authorized_product_types("view") + self.form.fields["engagement__lead"].queryset = get_authorized_users("view") \ + .filter(engagement__lead__isnull=False).distinct() + self.form.fields["tags"].help_text = labels.ASSET_FILTERS_TAGS_HELP + self.form.fields["not_tags"].help_text = labels.ASSET_FILTERS_NOT_TAGS_HELP + + class Meta: + model = Product + fields = ["name", "prod_type"] + + +class ProductEngagementsFilter(DojoFilter): + engagement__name = CharFilter(field_name="name", lookup_expr="icontains", label="Engagement name contains") + engagement__lead = ModelChoiceFilter(field_name="lead", queryset=Dojo_User.objects.none(), label="Lead") + engagement__version = CharFilter(field_name="version", lookup_expr="icontains", label="Engagement version") + engagement__test__version = CharFilter(field_name="test__version", lookup_expr="icontains", label="Test version") + engagement__status = MultipleChoiceFilter(field_name="status", choices=ENGAGEMENT_STATUS_CHOICES, + label="Status") + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.form.fields["engagement__lead"].queryset = get_authorized_users("view") \ + .filter(engagement__lead__isnull=False).distinct() + + class Meta: + model = Engagement + fields = [] + + +class ProductEngagementsFilterWithoutObjectLookups(ProductEngagementsFilter): + engagement__lead = CharFilter( + field_name="lead__username", + lookup_expr="iexact", + label="Lead Username", + help_text="Search for Lead username that are an exact match") + + +class EngagementFilterWithoutObjectLookups(EngagementFilterHelper): + engagement__lead = CharFilter( + field_name="engagement__lead__username", + lookup_expr="iexact", + label="Lead Username", + help_text="Search for Lead username that are an exact match") + engagement__lead_contains = CharFilter( + field_name="engagement__lead__username", + lookup_expr="icontains", + label="Lead Username Contains", + help_text="Search for Lead username that contain a given pattern") + prod_type__name = CharFilter( + field_name="prod_type__name", + lookup_expr="iexact", + label=labels.ORG_FILTERS_LABEL, + help_text=labels.ORG_FILTERS_LABEL_HELP) + prod_type__name_contains = CharFilter( + field_name="prod_type__name", + lookup_expr="icontains", + label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, + help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) + + class Meta: + model = Product + fields = ["name"] + + +class ProductEngagementFilterHelper(FilterSet): + version = CharFilter(lookup_expr="icontains", label="Engagement version") + test__version = CharFilter(field_name="test__version", lookup_expr="icontains", label="Test version") + name = CharFilter(lookup_expr="icontains") + status = MultipleChoiceFilter(choices=ENGAGEMENT_STATUS_CHOICES, label="Status") + target_start = DateRangeFilter() + target_end = DateRangeFilter() + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("name", "name"), + ("version", "version"), + ("target_start", "target_start"), + ("target_end", "target_end"), + ("status", "status"), + ("lead", "lead"), + ), + field_labels={ + "name": "Engagement Name", + }, + ) + + class Meta: + model = Product + fields = ["name"] + + +class ProductEngagementFilter(ProductEngagementFilterHelper, DojoFilter): + lead = ModelChoiceFilter(queryset=Dojo_User.objects.none(), label="Lead") + tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + queryset=Engagement.tags.tag_model.objects.all().order_by("name")) + not_tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + exclude=True, + queryset=Engagement.tags.tag_model.objects.all().order_by("name")) + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.form.fields["lead"].queryset = get_authorized_users( + "view").filter(engagement__lead__isnull=False).distinct() + + +class ProductEngagementFilterWithoutObjectLookups(ProductEngagementFilterHelper, DojoFilter): + lead = CharFilter( + field_name="lead__username", + lookup_expr="iexact", + label="Lead Username", + help_text="Search for Lead username that are an exact match") + lead_contains = CharFilter( + field_name="lead__username", + lookup_expr="icontains", + label="Lead Username Contains", + help_text="Search for Lead username that contain a given pattern") + + +class EngagementTestFilterHelper(FilterSet): + version = CharFilter(lookup_expr="icontains", label="Version") + if settings.TRACK_IMPORT_HISTORY: + test_import__version = CharFilter(field_name="test_import__version", lookup_expr="icontains", label="Reimported Version") + target_start = DateRangeFilter() + target_end = DateRangeFilter() + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) + has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("title", "title"), + ("version", "version"), + ("target_start", "target_start"), + ("target_end", "target_end"), + ("lead", "lead"), + ("api_scan_configuration", "api_scan_configuration"), + ), + field_labels={ + "name": "Test Name", + }, + ) + + +class EngagementTestFilter(EngagementTestFilterHelper, DojoFilter): + lead = ModelChoiceFilter(queryset=Dojo_User.objects.none(), label="Lead") + api_scan_configuration = ModelChoiceFilter( + queryset=Product_API_Scan_Configuration.objects.none(), + label="API Scan Configuration") + tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + queryset=Test.tags.tag_model.objects.all().order_by("name")) + not_tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + exclude=True, + queryset=Test.tags.tag_model.objects.all().order_by("name")) + + class Meta: + model = Test + fields = [ + "title", "test_type", "target_start", + "target_end", "percent_complete", + "version", "api_scan_configuration", + ] + + def __init__(self, *args, **kwargs): + self.engagement = kwargs.pop("engagement") + super(DojoFilter, self).__init__(*args, **kwargs) + self.form.fields["test_type"].queryset = Test_Type.objects.filter(test__engagement=self.engagement).distinct().order_by("name") + self.form.fields["api_scan_configuration"].queryset = Product_API_Scan_Configuration.objects.filter(product=self.engagement.product).distinct() + self.form.fields["lead"].queryset = get_authorized_users("view") \ + .filter(test__lead__isnull=False).distinct() + + +class EngagementTestFilterWithoutObjectLookups(EngagementTestFilterHelper): + lead = CharFilter( + field_name="lead__username", + lookup_expr="iexact", + label="Lead Username", + help_text="Search for Lead username that are an exact match") + lead_contains = CharFilter( + field_name="lead__username", + lookup_expr="icontains", + label="Lead Username Contains", + help_text="Search for Lead username that contain a given pattern") + api_scan_configuration__tool_configuration__name = CharFilter( + field_name="api_scan_configuration__tool_configuration__name", + lookup_expr="iexact", + label="API Scan Configuration Name", + help_text="Search for Lead username that are an exact match") + api_scan_configuration__tool_configuration__name_contains = CharFilter( + field_name="api_scan_configuration__tool_configuration__name", + lookup_expr="icontains", + label="API Scan Configuration Name Contains", + help_text="Search for Lead username that contain a given pattern") + tags_contains = CharFilter( + label="Test Tag Contains", + field_name="tags__name", + lookup_expr="icontains", + help_text="Search for tags on a Test that contain a given pattern") + tags = CharFilter( + label="Test Tag", + field_name="tags__name", + lookup_expr="iexact", + help_text="Search for tags on a Test that are an exact match") + not_tags_contains = CharFilter( + label="Test Tag Does Not Contain", + field_name="tags__name", + lookup_expr="icontains", + help_text="Search for tags on a Test that contain a given pattern, and exclude them", + exclude=True) + not_tags = CharFilter( + label="Not Test Tag", + field_name="tags__name", + lookup_expr="iexact", + help_text="Search for tags on a Test that are an exact match, and exclude them", + exclude=True) + + class Meta: + model = Test + fields = [ + "title", "test_type", "target_start", + "target_end", "percent_complete", "version", + ] + + def __init__(self, *args, **kwargs): + self.engagement = kwargs.pop("engagement") + super().__init__(*args, **kwargs) + self.form.fields["test_type"].queryset = Test_Type.objects.filter(test__engagement=self.engagement).distinct().order_by("name") diff --git a/dojo/engagement/ui/forms.py b/dojo/engagement/ui/forms.py new file mode 100644 index 00000000000..a325e9a3d69 --- /dev/null +++ b/dojo/engagement/ui/forms.py @@ -0,0 +1,151 @@ +from django import forms + +from dojo.engagement.queries import get_authorized_engagements +from dojo.labels import get_labels +from dojo.models import Engagement, Engagement_Presets, Product +from dojo.product.queries import get_authorized_products +from dojo.user.queries import get_authorized_users, get_authorized_users_for_product_and_product_type +from dojo.utils import get_system_setting +from dojo.validators import tag_validator + +labels = get_labels() + + +class EngForm(forms.ModelForm): + name = forms.CharField( + max_length=300, required=False, + help_text=( + "Add a descriptive name to identify this engagement. " + "Without a name the target start date will be set." + )) + description = forms.CharField(widget=forms.Textarea(attrs={}), + required=False, help_text="Description of the engagement and details regarding the engagement.") + product = forms.ModelChoiceField(label=labels.ASSET_LABEL, + queryset=Product.objects.none(), + required=True) + target_start = forms.DateField(widget=forms.TextInput( + attrs={"class": "datepicker", "autocomplete": "off"})) + target_end = forms.DateField(widget=forms.TextInput( + attrs={"class": "datepicker", "autocomplete": "off"})) + lead = forms.ModelChoiceField( + queryset=None, + required=True, label="Testing Lead") + test_strategy = forms.URLField(required=False, label="Test Strategy URL") + + def __init__(self, *args, **kwargs): + cicd = False + product = None + if "cicd" in kwargs: + cicd = kwargs.pop("cicd") + + if "product" in kwargs: + product = kwargs.pop("product") + + self.user = None + if "user" in kwargs: + self.user = kwargs.pop("user") + + super().__init__(*args, **kwargs) + + if product: + self.fields["preset"] = forms.ModelChoiceField(help_text="Settings and notes for performing this engagement.", required=False, queryset=Engagement_Presets.objects.filter(product=product)) + self.fields["lead"].queryset = get_authorized_users_for_product_and_product_type(None, product, "view").filter(is_active=True) + else: + self.fields["lead"].queryset = get_authorized_users("view").filter(is_active=True) + + self.fields["product"].queryset = get_authorized_products("add") + + # Don't show CICD fields on a interactive engagement + if cicd is False: + del self.fields["build_id"] + del self.fields["commit_hash"] + del self.fields["branch_tag"] + del self.fields["build_server"] + del self.fields["source_code_management_server"] + # del self.fields['source_code_management_uri'] + del self.fields["orchestration_engine"] + else: + del self.fields["test_strategy"] + del self.fields["status"] + + def is_valid(self): + valid = super().is_valid() + + # we're done now if not valid + if not valid: + return valid + if self.cleaned_data["target_start"] > self.cleaned_data["target_end"]: + self.add_error("target_start", "Your target start date exceeds your target end date") + self.add_error("target_end", "Your target start date exceeds your target end date") + return False + return True + + def clean_tags(self): + tag_validator(self.cleaned_data.get("tags")) + return self.cleaned_data.get("tags") + + class Meta: + model = Engagement + exclude = ("first_contacted", "real_start", "engagement_type", "inherited_tags", + "real_end", "requester", "reason", "updated", "report_type", + "product", "threat_model", "api_test", "pen_test", "check_list") + + +class DeleteEngagementForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Engagement + fields = ["id"] + + +class EngagementPresetsForm(forms.ModelForm): + + notes = forms.CharField(widget=forms.Textarea(attrs={}), + required=False, help_text="Description of what needs to be tested or setting up environment for testing") + + scope = forms.CharField(widget=forms.Textarea(attrs={}), + required=False, help_text="Scope of Engagement testing, IP's/Resources/URL's)") + + class Meta: + model = Engagement_Presets + exclude = ["product"] + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + if disclaimer := get_system_setting("disclaimer_notes"): + self.disclaimer = disclaimer.strip() + + +class DeleteEngagementPresetsForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Engagement_Presets + fields = ["id"] + + +class AddEngagementForm(forms.Form): + product = forms.ModelChoiceField( + queryset=Product.objects.none(), + required=True, + widget=forms.widgets.Select(), + help_text="Select which product to attach Engagement") + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.fields["product"].queryset = get_authorized_products("add") + + +class ExistingEngagementForm(forms.Form): + engagement = forms.ModelChoiceField( + queryset=Engagement.objects.none(), + required=True, + widget=forms.widgets.Select(), + help_text="Select which Engagement to link the Questionnaire to") + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.fields["engagement"].queryset = get_authorized_engagements("edit").order_by("-target_start") diff --git a/dojo/engagement/urls.py b/dojo/engagement/ui/urls.py similarity index 98% rename from dojo/engagement/urls.py rename to dojo/engagement/ui/urls.py index 0f33c3aa697..0af9f481a87 100644 --- a/dojo/engagement/urls.py +++ b/dojo/engagement/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from dojo.engagement import views +from dojo.engagement.ui import views urlpatterns = [ # engagements and calendar diff --git a/dojo/engagement/ui/views.py b/dojo/engagement/ui/views.py new file mode 100644 index 00000000000..475ff9cf5f9 --- /dev/null +++ b/dojo/engagement/ui/views.py @@ -0,0 +1,1691 @@ +import csv +import logging +import mimetypes +import operator +import re +import time +from datetime import datetime, timedelta +from functools import partial, reduce +from pathlib import Path +from tempfile import NamedTemporaryFile +from time import strftime + +import pghistory +from django.conf import settings +from django.contrib import messages +from django.contrib.admin.utils import NestedObjects +from django.contrib.auth.models import User +from django.core.exceptions import PermissionDenied, ValidationError +from django.db import DEFAULT_DB_ALIAS +from django.db.models import OuterRef, Q, Value +from django.db.models.functions import Coalesce +from django.db.models.query import Prefetch, QuerySet +from django.http import HttpRequest, HttpResponse, HttpResponseRedirect, JsonResponse, QueryDict, StreamingHttpResponse +from django.shortcuts import get_object_or_404, render +from django.urls import Resolver404, reverse +from django.utils import timezone +from django.views import View +from django.views.decorators.cache import cache_page +from django.views.decorators.http import require_POST +from django.views.decorators.vary import vary_on_cookie +from openpyxl import Workbook +from openpyxl.styles import Font + +import dojo.risk_acceptance.helper as ra_helper +from dojo.authorization.authorization import user_has_permission_or_403 +from dojo.endpoint.utils import save_endpoints_to_add +from dojo.engagement.queries import get_authorized_engagements +from dojo.engagement.services import ( + close_engagement, + reopen_engagement, +) +from dojo.engagement.services import ( + copy_engagement as copy_engagement_service, +) +from dojo.engagement.ui.filters import ( + EngagementDirectFilter, + EngagementDirectFilterWithoutObjectLookups, + EngagementFilter, + EngagementFilterWithoutObjectLookups, + EngagementTestFilter, + EngagementTestFilterWithoutObjectLookups, + ProductEngagementsFilter, + ProductEngagementsFilterWithoutObjectLookups, +) +from dojo.engagement.ui.forms import DeleteEngagementForm, EngForm +from dojo.finding.helper import NOT_ACCEPTED_FINDINGS_QUERY +from dojo.finding.ui.views import find_available_notetypes +from dojo.forms import ( + AddFindingsRiskAcceptanceForm, + CheckForm, + DoneForm, + EditRiskAcceptanceForm, + ImportScanForm, + JIRAEngagementForm, + JIRAImportScanForm, + JIRAProjectForm, + NoteForm, + ReplaceRiskAcceptanceProofForm, + RiskAcceptanceForm, + TestForm, + TypedNoteForm, + UploadThreatForm, +) +from dojo.importers.base_importer import BaseImporter +from dojo.importers.default_importer import DefaultImporter +from dojo.jira import services as jira_services +from dojo.location.models import Location +from dojo.location.utils import save_locations_to_add +from dojo.models import ( + Check_List, + Development_Environment, + Dojo_User, + Endpoint, + Engagement, + Finding, + Note_Type, + Notes, + Product, + Product_API_Scan_Configuration, + Risk_Acceptance, + System_Settings, + Test, + Test_Import, +) +from dojo.notifications.helper import create_notification +from dojo.product.queries import get_authorized_products +from dojo.product_announcements import ( + ErrorPageProductAnnouncement, + LargeScanSizeProductAnnouncement, + ScanTypeProductAnnouncement, +) +from dojo.query_utils import build_count_subquery +from dojo.risk_acceptance.helper import prefetch_for_expiration +from dojo.tools.factory import get_scan_types_sorted +from dojo.user.queries import get_authorized_users +from dojo.utils import ( + FileIterWrapper, + Product_Tab, + add_breadcrumb, + add_error_message_to_response, + add_success_message_to_response, + async_delete, + generate_file_response_from_file_path, + get_cal_event, + get_page_items, + get_return_url, + get_setting, + get_system_setting, + handle_uploaded_threat, + redirect_to_return_url_or_else, +) + +logger = logging.getLogger(__name__) + + +@cache_page(60 * 5) # cache for 5 minutes +@vary_on_cookie +def engagement_calendar(request): + + if not get_system_setting("enable_calendar"): + raise Resolver404 + + if "lead" not in request.GET or "0" in request.GET.getlist("lead"): + engagements = get_authorized_engagements("view") + else: + filters = [] + leads = request.GET.getlist("lead", "") + if "-1" in request.GET.getlist("lead"): + leads.remove("-1") + filters.append(Q(lead__isnull=True)) + filters.append(Q(lead__in=leads)) + engagements = get_authorized_engagements("view").filter(reduce(operator.or_, filters)) + + engagements = engagements.select_related("lead") + engagements = engagements.prefetch_related("product") + + for e in engagements: + if e.target_end: + e.target_end += timedelta(days=1) + add_breadcrumb( + title="Engagement Calendar", top_level=True, request=request) + return render( + request, "dojo/calendar.html", { + "caltype": "engagements", + "leads": request.GET.getlist("lead", ""), + "engagements": engagements, + "users": get_authorized_users("view"), + }) + + +def get_filtered_engagements(request, view): + if view not in {"all", "active"}: + msg = f"View {view} is not allowed" + raise ValidationError(msg) + + engagements = get_authorized_engagements("view").order_by("-target_start") + + if view == "active": + engagements = engagements.filter(active=True) + + engagements = ( + engagements + .select_related("product", "product__prod_type") + .prefetch_related("lead", "tags", "product__tags") + ) + + if System_Settings.objects.get().enable_jira: + engagements = engagements.prefetch_related( + "jira_project__jira_instance", + "product__jira_project_set__jira_instance", + ) + + test_count_subquery = build_count_subquery( + Test.objects.filter(engagement=OuterRef("pk")), group_field="engagement_id", + ) + engagements = engagements.annotate(test_count=Coalesce(test_count_subquery, Value(0))) + + filter_string_matching = get_system_setting("filter_string_matching", False) + filter_class = EngagementDirectFilterWithoutObjectLookups if filter_string_matching else EngagementDirectFilter + return filter_class(request.GET, queryset=engagements) + + +def engagements(request, view): + if not view: + view = "active" + + filtered_engagements = get_filtered_engagements(request, view) + + engs = get_page_items(request, filtered_engagements.qs, 25) + product_name_words = sorted(get_authorized_products("view").values_list("name", flat=True)) + engagement_name_words = sorted(get_authorized_engagements("view").values_list("name", flat=True).distinct()) + + add_breadcrumb( + title=f"{view.capitalize()} Engagements", + top_level=not len(request.GET), + request=request) + + return render( + request, "dojo/engagement.html", { + "engagements": engs, + "filter_form": filtered_engagements.form, + "product_name_words": product_name_words, + "engagement_name_words": engagement_name_words, + "view": view.capitalize(), + }) + + +def engagements_all(request): + + products_with_engagements = get_authorized_products("view") + products_with_engagements = products_with_engagements.filter(~Q(engagement=None)).distinct() + + # count using prefetch instead of just using 'engagement__set_test_test` to avoid loading all test in memory just to count them + filter_string_matching = get_system_setting("filter_string_matching", False) + products_filter_class = ProductEngagementsFilterWithoutObjectLookups if filter_string_matching else ProductEngagementsFilter + test_count_subquery = build_count_subquery( + Test.objects.filter(engagement=OuterRef("pk")), + group_field="engagement_id", + ) + engagement_query = Engagement.objects.annotate(test_count=Coalesce(test_count_subquery, Value(0))) + filter_qs = products_with_engagements.prefetch_related( + Prefetch("engagement_set", queryset=products_filter_class(request.GET, engagement_query).qs), + ) + + filter_qs = filter_qs.prefetch_related( + "engagement_set__tags", + "prod_type", + "engagement_set__lead", + "tags", + ) + if System_Settings.objects.get().enable_jira: + filter_qs = filter_qs.prefetch_related( + "engagement_set__jira_project__jira_instance", + "jira_project_set__jira_instance", + ) + filter_class = EngagementFilterWithoutObjectLookups if filter_string_matching else EngagementFilter + filtered = filter_class( + request.GET, + queryset=filter_qs, + ) + + prods = get_page_items(request, filtered.qs, 25) + prods.paginator.count = sum(len(prod.engagement_set.all()) for prod in prods) + name_words = products_with_engagements.values_list("name", flat=True) + eng_words = get_authorized_engagements("view").values_list("name", flat=True).distinct() + + add_breadcrumb( + title="All Engagements", + top_level=not len(request.GET), + request=request) + + return render( + request, "dojo/engagements_all.html", { + "products": prods, + "filter_form": filtered.form, + "name_words": sorted(set(name_words)), + "eng_words": sorted(set(eng_words)), + "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), + }) + + +def edit_engagement(request, eid): + engagement = Engagement.objects.get(pk=eid) + is_ci_cd = engagement.engagement_type == "CI/CD" + jira_project_form = None + jira_epic_form = None + jira_project = None + + if request.method == "POST": + form = EngForm(request.POST, instance=engagement, cicd=is_ci_cd, product=engagement.product, user=request.user) + jira_project = jira_services.get_project(engagement, use_inheritance=False) + + if form.is_valid(): + # first save engagement details + new_status = form.cleaned_data.get("status") + if form.cleaned_data.get("product") != engagement.product: + user_has_permission_or_403( + request.user, + form.cleaned_data.get("product"), + "edit", + ) + engagement.product = form.cleaned_data.get("product") + engagement = form.save(commit=False) + if (new_status in {"Cancelled", "Completed"}): + engagement.active = False + else: + engagement.active = True + engagement.save() + form.save_m2m() + + messages.add_message( + request, + messages.SUCCESS, + "Engagement updated successfully.", + extra_tags="alert-success") + + success, jira_project_form = jira_services.process_project_form(request, instance=jira_project, target="engagement", engagement=engagement, product=engagement.product) + error = not success + + success, jira_epic_form = jira_services.process_epic_form(request, engagement=engagement) + error = error or not success + + if not error: + if "_Add Tests" in request.POST: + return HttpResponseRedirect( + reverse("add_tests", args=(engagement.id, ))) + return HttpResponseRedirect( + reverse("view_engagement", args=(engagement.id, ))) + else: + logger.debug(form.errors) + + else: + form = EngForm(initial={"product": engagement.product}, instance=engagement, cicd=is_ci_cd, product=engagement.product, user=request.user) + + jira_epic_form = None + if get_system_setting("enable_jira"): + jira_project = jira_services.get_project(engagement, use_inheritance=False) + jira_project_form = JIRAProjectForm(instance=jira_project, target="engagement", product=engagement.product) + logger.debug("showing jira-epic-form") + jira_epic_form = JIRAEngagementForm(instance=engagement) + + title = "Edit CI/CD Engagement" if is_ci_cd else "Edit Interactive Engagement" + + product_tab = Product_Tab(engagement.product, title=title, tab="engagements") + product_tab.setEngagement(engagement) + return render(request, "dojo/new_eng.html", { + "product_tab": product_tab, + "title": title, + "form": form, + "edit": True, + "jira_epic_form": jira_epic_form, + "jira_project_form": jira_project_form, + "engagement": engagement, + }) + + +def delete_engagement(request, eid): + engagement = get_object_or_404(Engagement, pk=eid) + product = engagement.product + form = DeleteEngagementForm(instance=engagement) + + if request.method == "POST": + if "id" in request.POST and str(engagement.id) == request.POST["id"]: + form = DeleteEngagementForm(request.POST, instance=engagement) + if form.is_valid(): + product = engagement.product + if get_setting("ASYNC_OBJECT_DELETE"): + async_del = async_delete() + async_del.delete(engagement) + message = "Engagement and relationships will be removed in the background." + else: + message = "Engagement and relationships removed." + engagement.delete() + messages.add_message( + request, + messages.SUCCESS, + message, + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_engagements", args=(product.id, ))) + + rels = ["Previewing the relationships has been disabled.", ""] + display_preview = get_setting("DELETE_PREVIEW") + if display_preview: + collector = NestedObjects(using=DEFAULT_DB_ALIAS) + collector.collect([engagement]) + rels = collector.nested() + + product_tab = Product_Tab(product, title="Delete Engagement", tab="engagements") + product_tab.setEngagement(engagement) + return render(request, "dojo/delete_engagement.html", { + "product_tab": product_tab, + "engagement": engagement, + "form": form, + "rels": rels, + }) + + +def copy_engagement(request, eid): + engagement = get_object_or_404(Engagement, id=eid) + product = engagement.product + form = DoneForm() + + if request.method == "POST": + form = DoneForm(request.POST) + if form.is_valid(): + copy_engagement_service(engagement, request.user) + messages.add_message( + request, + messages.SUCCESS, + "Engagement Copied successfully.", + extra_tags="alert-success") + return redirect_to_return_url_or_else(request, reverse("view_engagements", args=(product.id, ))) + messages.add_message( + request, + messages.ERROR, + "Unable to copy engagement, please try again.", + extra_tags="alert-danger") + + product_tab = Product_Tab(product, title="Copy Engagement", tab="engagements") + return render(request, "dojo/copy_object.html", { + "source": engagement, + "source_label": "Engagement", + "destination_label": "Product", + "product_tab": product_tab, + "form": form, + }) + + +class ViewEngagement(View): + + def get_template(self): + return "dojo/view_eng.html" + + def get_risks_accepted(self, eng): + accepted_findings_subquery = build_count_subquery( + Finding.objects.filter(risk_acceptance=OuterRef("pk")), + group_field="risk_acceptance", + ) + return eng.risk_acceptance.all().select_related("owner").annotate( + accepted_findings_count=Coalesce(accepted_findings_subquery, Value(0)), + ) + + def get_filtered_tests( + self, + request: HttpRequest, + queryset: list[Test], + engagement: Engagement, + ): + filter_string_matching = get_system_setting("filter_string_matching", False) + filter_class = EngagementTestFilterWithoutObjectLookups if filter_string_matching else EngagementTestFilter + return filter_class(request.GET, queryset=queryset, engagement=engagement) + + def get(self, request, eid, *args, **kwargs): + eng = get_object_or_404(Engagement, id=eid) + # Make sure the user is authorized + user_has_permission_or_403(request.user, eng, "view") + tests = eng.test_set.all().order_by("test_type__name", "-updated") + default_page_num = 10 + tests_filter = self.get_filtered_tests(request, tests, eng) + paged_tests = get_page_items(request, tests_filter.qs, default_page_num) + paged_tests.object_list = prefetch_for_view_tests(paged_tests.object_list) + prod = eng.product + risks_accepted = self.get_risks_accepted(eng) + preset_test_type = None + network = None + if eng.preset: + preset_test_type = eng.preset.test_type.all() + network = eng.preset.network_locations.all() + system_settings = System_Settings.objects.get() + + jissue = jira_services.get_issue(eng) + jira_project = jira_services.get_project(eng) + + try: + check = Check_List.objects.get(engagement=eng) + except: + check = None + notes = eng.notes.all() + note_type_activation = Note_Type.objects.filter(is_active=True).count() + if note_type_activation: + available_note_types = find_available_notetypes(notes) + form = DoneForm() + files = eng.files.all() + form = TypedNoteForm(available_note_types=available_note_types) if note_type_activation else NoteForm() + + add_breadcrumb(parent=eng, top_level=False, request=request) + + title = "" + if eng.engagement_type == "CI/CD": + title = " CI/CD" + product_tab = Product_Tab(prod, title="View" + title + " Engagement", tab="engagements") + product_tab.setEngagement(eng) + return render( + request, self.get_template(), { + "eng": eng, + "product_tab": product_tab, + "system_settings": system_settings, + "tests": paged_tests, + "filter": tests_filter, + "check": check, + "threat": eng.tmodel_path, + "form": form, + "notes": notes, + "files": files, + "risks_accepted": risks_accepted, + "jissue": jissue, + "jira_project": jira_project, + "network": network, + "preset_test_type": preset_test_type, + }) + + def post(self, request, eid, *args, **kwargs): + eng = get_object_or_404(Engagement, id=eid) + # Make sure the user is authorized + user_has_permission_or_403(request.user, eng, "view") + tests = eng.test_set.all().order_by("test_type__name", "-updated") + default_page_num = 10 + + tests_filter = self.get_filtered_tests(request, tests, eng) + paged_tests = get_page_items(request, tests_filter.qs, default_page_num) + # prefetch only after creating the filters to avoid https://code.djangoproject.com/ticket/23771 and https://code.djangoproject.com/ticket/25375 + paged_tests.object_list = prefetch_for_view_tests(paged_tests.object_list) + + prod = eng.product + risks_accepted = self.get_risks_accepted(eng) + preset_test_type = None + network = None + if eng.preset: + preset_test_type = eng.preset.test_type.all() + network = eng.preset.network_locations.all() + system_settings = System_Settings.objects.get() + + jissue = jira_services.get_issue(eng) + jira_project = jira_services.get_project(eng) + + try: + check = Check_List.objects.get(engagement=eng) + except: + check = None + notes = eng.notes.all() + note_type_activation = Note_Type.objects.filter(is_active=True).count() + if note_type_activation: + available_note_types = find_available_notetypes(notes) + form = DoneForm() + files = eng.files.all() + user_has_permission_or_403(request.user, eng, "add") + eng.progress = "check_list" + eng.save() + + if note_type_activation: + form = TypedNoteForm(request.POST, available_note_types=available_note_types) + else: + form = NoteForm(request.POST) + if form.is_valid(): + new_note = form.save(commit=False) + new_note.author = request.user + new_note.date = timezone.now() + new_note.save() + eng.notes.add(new_note) + form = TypedNoteForm(available_note_types=available_note_types) if note_type_activation else NoteForm() + title = f"Engagement: {eng.name} on {eng.product.name}" + messages.add_message(request, + messages.SUCCESS, + "Note added successfully.", + extra_tags="alert-success") + + add_breadcrumb(parent=eng, top_level=False, request=request) + + title = "" + if eng.engagement_type == "CI/CD": + title = " CI/CD" + product_tab = Product_Tab(prod, title="View" + title + " Engagement", tab="engagements") + product_tab.setEngagement(eng) + return render( + request, self.get_template(), { + "eng": eng, + "product_tab": product_tab, + "system_settings": system_settings, + "tests": paged_tests, + "filter": tests_filter, + "check": check, + "threat": eng.tmodel_path, + "form": form, + "notes": notes, + "files": files, + "risks_accepted": risks_accepted, + "jissue": jissue, + "jira_project": jira_project, + "network": network, + "preset_test_type": preset_test_type, + }) + + +def prefetch_for_view_tests(tests): + # old code can arrive here with prods being a list because the query was already executed + if not isinstance(tests, QuerySet): + logger.warning("unable to prefetch because query was already executed") + return tests + + prefetched = tests.select_related("lead", "test_type").prefetch_related("tags", "notes") + base_findings = Finding.objects.filter(test_id=OuterRef("pk")) + count_subquery = partial(build_count_subquery, group_field="test_id") + return prefetched.annotate( + count_findings_test_all=Coalesce(count_subquery(base_findings), Value(0)), + count_findings_test_active=Coalesce(count_subquery(base_findings.filter(active=True)), Value(0)), + count_findings_test_active_verified=Coalesce( + count_subquery(base_findings.filter(active=True, verified=True)), Value(0), + ), + count_findings_test_active_fix_available=Coalesce( + count_subquery(base_findings.filter(active=True, fix_available=True)), Value(0), + ), + count_findings_test_mitigated=Coalesce(count_subquery(base_findings.filter(is_mitigated=True)), Value(0)), + count_findings_test_dups=Coalesce(count_subquery(base_findings.filter(duplicate=True)), Value(0)), + total_reimport_count=Coalesce( + count_subquery(Test_Import.objects.filter(test_id=OuterRef("pk"), type=Test_Import.REIMPORT_TYPE)), + Value(0), + ), + ) + + +def add_tests(request, eid): + eng = Engagement.objects.get(id=eid) + + if request.method == "POST": + form = TestForm(request.POST, engagement=eng) + if form.is_valid(): + new_test = form.save(commit=False) + # set default scan_type as it's used in reimport + new_test.scan_type = new_test.test_type.name + new_test.engagement = eng + try: + new_test.lead = User.objects.get(id=form["lead"].value()) + except: + new_test.lead = None + + # Set status to in progress if a test is added + if eng.status != "In Progress" and eng.active is True: + eng.status = "In Progress" + eng.save() + + new_test.save() + + messages.add_message( + request, + messages.SUCCESS, + "Test added successfully.", + extra_tags="alert-success") + + create_notification( + event="test_added", + title=f"Test created for {new_test.engagement.product}: {new_test.engagement.name}: {new_test}", + test=new_test, + engagement=new_test.engagement, + product=new_test.engagement.product, + url=reverse("view_test", args=(new_test.id,)), + url_api=reverse("test-detail", args=(new_test.id,)), + ) + + if "_Add Another Test" in request.POST: + return HttpResponseRedirect( + reverse("add_tests", args=(eng.id, ))) + if "_Add Findings" in request.POST: + return HttpResponseRedirect( + reverse("add_findings", args=(new_test.id, ))) + if "_Finished" in request.POST: + return HttpResponseRedirect( + reverse("view_engagement", args=(eng.id, ))) + else: + form = TestForm(engagement=eng) + form.initial["target_start"] = eng.target_start + form.initial["target_end"] = eng.target_end + form.initial["lead"] = request.user + add_breadcrumb( + parent=eng, title="Add Tests", top_level=False, request=request) + product_tab = Product_Tab(eng.product, title="Add Tests", tab="engagements") + product_tab.setEngagement(eng) + return render(request, "dojo/add_tests.html", { + "product_tab": product_tab, + "form": form, + "eid": eid, + "eng": eng, + }) + + +class ImportScanResultsView(View): + def get_template(self) -> str: + """Returns the template that will be presented to the user""" + return "dojo/import_scan_results.html" + + def get_development_environment( + self, + environment_name: str = "Development", + ) -> Development_Environment | None: + """ + Get the development environment in two cases: + - GET: Environment "Development" by default + - POST: The label supplied by the user, with Development as a backup + """ + return Development_Environment.objects.filter(name=environment_name).first() + + def get_engagement_or_product( + self, + user: Dojo_User, + engagement_id: int | None = None, + product_id: int | None = None, + ) -> tuple[Engagement, Product, Product | Engagement]: + """Using the path parameters, either fetch the product or engagement""" + engagement = product = engagement_or_product = None + # Get the product if supplied + # Get the engagement if supplied + if engagement_id is not None: + engagement = get_object_or_404(Engagement, id=engagement_id) + engagement_or_product = engagement + elif product_id is not None: + product = get_object_or_404(Product, id=product_id) + engagement_or_product = product + else: + msg = "Either Engagement or Product has to be provided" + raise Exception(msg) + # Ensure the supplied user has access to import to the engagement or product + user_has_permission_or_403(user, engagement_or_product, "import") + + return engagement, product, engagement_or_product + + def get_form( + self, + request: HttpRequest, + **kwargs: dict, + ) -> ImportScanForm: + """Returns the default import form for importing findings""" + if request.method == "POST": + return ImportScanForm(request.POST, request.FILES, **kwargs) + return ImportScanForm(**kwargs) + + def get_jira_form( + self, + request: HttpRequest, + engagement_or_product: Engagement | Product, + ) -> tuple[JIRAImportScanForm | None, bool]: + """Returns a JiraImportScanForm if jira is enabled""" + jira_form = None + push_all_jira_issues = False + # Determine if jira issues should be pushed automatically + push_all_jira_issues = jira_services.is_push_all_issues(engagement_or_product) + # Only return the form if the jira is enabled on this engagement or product + if jira_services.get_project(engagement_or_product): + if request.method == "POST": + jira_form = JIRAImportScanForm( + request.POST, + push_all=push_all_jira_issues, + prefix="jiraform", + ) + else: + jira_form = JIRAImportScanForm( + push_all=push_all_jira_issues, + prefix="jiraform", + ) + return jira_form, push_all_jira_issues + + def get_product_tab( + self, + product: Product, + engagement: Engagement, + ) -> tuple[Product_Tab, dict]: + """ + Determine how the product tab will be rendered, and what tab will be selected + as currently active + """ + custom_breadcrumb = None + if engagement: + product_tab = Product_Tab(engagement.product, title="Import Scan Results", tab="engagements") + product_tab.setEngagement(engagement) + else: + custom_breadcrumb = {""} + product_tab = Product_Tab(product, title="Import Scan Results", tab="findings") + return product_tab, custom_breadcrumb + + def handle_request( + self, + request: HttpRequest, + engagement_id: int | None = None, + product_id: int | None = None, + ) -> tuple[HttpRequest, dict]: + """ + Process the common behaviors between request types, and then return + the request and context dict back to be rendered + """ + user = request.user + # Get the development environment + environment = self.get_development_environment() + # Get the product or engagement from the path parameters + engagement, product, engagement_or_product = self.get_engagement_or_product( + user, + engagement_id=engagement_id, + product_id=product_id, + ) + # Get the product tab and any additional custom breadcrumbs + product_tab, custom_breadcrumb = self.get_product_tab(product, engagement) + + if settings.V3_FEATURE_LOCATIONS: + endpoints = Location.objects.filter(products__product_id=product_tab.product.id) + else: + # TODO: Delete this after the move to Locations + endpoints = Endpoint.objects.filter(product__id=product_tab.product.id) + + # Get the import form with some initial data in place + form = self.get_form( + request, + environment=environment, + endpoints=endpoints, + api_scan_configuration=Product_API_Scan_Configuration.objects.filter(product__id=product_tab.product.id), + ) + # Get the jira form + jira_form, push_all_jira_issues = self.get_jira_form(request, engagement_or_product) + # Return the request and the context + return request, { + "user": user, + "lead": user, + "form": form, + "environment": environment, + "product_tab": product_tab, + "product": product, + "engagement": engagement, + "engagement_or_product": engagement_or_product, + "custom_breadcrumb": custom_breadcrumb, + "title": "Import Scan Results", + "jform": jira_form, + "scan_types": get_scan_types_sorted(), + "push_all_jira_issues": push_all_jira_issues, + } + + def validate_forms( + self, + context: dict, + ) -> bool: + """ + Validates each of the forms to ensure all errors from the form + level are bubbled up to the user first before we process too much + """ + form_validation_list = [] + for form_name in ["form", "jform"]: + if (form := context.get(form_name)) is not None: + if errors := form.errors: + form_validation_list.append(errors) + return form_validation_list + + def create_engagement( + self, + context: dict, + ) -> Engagement: + """ + Create an engagement if the import was triggered from the product level, + otherwise, return the existing engagement instead + """ + # Make sure an engagement does not exist already + engagement = context.get("engagement") + if engagement is None: + engagement = Engagement.objects.create( + name="AdHoc Import - " + strftime("%a, %d %b %Y %X", timezone.now().timetuple()), + threat_model=False, + api_test=False, + pen_test=False, + check_list=False, + active=True, + target_start=timezone.now().date(), + target_end=timezone.now().date(), + product=context.get("product"), + status="In Progress", + version=context.get("version"), + branch_tag=context.get("branch_tag"), + build_id=context.get("build_id"), + commit_hash=context.get("commit_hash"), + ) + # Update the engagement in the context + context["engagement"] = engagement + # Return the engagement + return engagement + + def get_importer( + self, + context: dict, + ) -> BaseImporter: + """Gets the importer to use""" + return DefaultImporter(**context) + + def import_findings( + self, + context: dict, + ) -> str | None: + """Attempt to import with all the supplied information""" + try: + # Log only user-entered form values, excluding internal objects + user_values = { + "scan_type": context.get("scan_type"), + "scan_date": context.get("scan_date"), + "minimum_severity": context.get("minimum_severity"), + "active": context.get("active"), + "verified": context.get("verified"), + "test_title": context.get("test_title"), + "tags": context.get("tags"), + "version": context.get("version"), + "branch_tag": context.get("branch_tag"), + "build_id": context.get("build_id"), + "commit_hash": context.get("commit_hash"), + "service": context.get("service"), + "close_old_findings": context.get("close_old_findings"), + "apply_tags_to_findings": context.get("apply_tags_to_findings"), + "apply_tags_to_endpoints": context.get("apply_tags_to_endpoints"), + "close_old_findings_product_scope": context.get("close_old_findings_product_scope"), + "group_by": context.get("group_by"), + "create_finding_groups_for_all_findings": context.get("create_finding_groups_for_all_findings"), + "push_to_jira": context.get("push_to_jira"), + "push_all_jira_issues": context.get("push_all_jira_issues"), + } + logger.debug(f"import_findings called with user values: {user_values}") + importer_client = self.get_importer(context) + context["test"], _, finding_count, closed_finding_count, _, _, _ = importer_client.process_scan( + context.pop("scan", None), + ) + # Add a message to the view for the user to see the results + add_success_message_to_response(importer_client.construct_imported_message( + finding_count=finding_count, + closed_finding_count=closed_finding_count, + )) + except Exception as e: + logger.exception("An exception error occurred during the report import") + return f"An exception error occurred during the report import: {e}" + return None + + def process_form( + self, + request: HttpRequest, + form: ImportScanForm, + context: dict, + ) -> str | None: + """Process the form and manipulate the input in any way that is appropriate""" + # Update the running context dict with cleaned form input + context.update({ + "scan": request.FILES.get("file", None), + "scan_date": form.cleaned_data.get("scan_date"), + "minimum_severity": form.cleaned_data.get("minimum_severity"), + "active": None, + "verified": None, + "scan_type": request.POST.get("scan_type"), + "test_title": form.cleaned_data.get("test_title") or None, + "tags": form.cleaned_data.get("tags"), + "version": form.cleaned_data.get("version") or None, + "branch_tag": form.cleaned_data.get("branch_tag") or None, + "build_id": form.cleaned_data.get("build_id") or None, + "commit_hash": form.cleaned_data.get("commit_hash") or None, + "api_scan_configuration": form.cleaned_data.get("api_scan_configuration") or None, + "service": form.cleaned_data.get("service") or None, + "close_old_findings": form.cleaned_data.get("close_old_findings", None), + "apply_tags_to_findings": form.cleaned_data.get("apply_tags_to_findings", False), + "apply_tags_to_endpoints": form.cleaned_data.get("apply_tags_to_endpoints", False), + "close_old_findings_product_scope": form.cleaned_data.get("close_old_findings_product_scope", None), + "group_by": form.cleaned_data.get("group_by") or None, + "create_finding_groups_for_all_findings": form.cleaned_data.get("create_finding_groups_for_all_findings", None), + "environment": self.get_development_environment(environment_name=form.cleaned_data.get("environment")), + }) + # Create the engagement if necessary + self.create_engagement(context) + # close_old_findings_product_scope is a modifier of close_old_findings. + # If it is selected, close_old_findings should also be selected. + if close_old_findings_product_scope := form.cleaned_data.get("close_old_findings_product_scope", None): + context["close_old_findings_product_scope"] = close_old_findings_product_scope + context["close_old_findings"] = True + if settings.V3_FEATURE_LOCATIONS: + # Save newly added locations + added_locations = save_locations_to_add(form.endpoints_to_add_list) + locations_from_form = [location.url for location in form.cleaned_data["endpoints"]] + context["endpoints_to_add"] = locations_from_form + added_locations + else: + # TODO: Delete this after the move to Locations + # Save newly added endpoints + added_endpoints = save_endpoints_to_add(form.endpoints_to_add_list, context.get("engagement").product) + endpoints_from_form = list(form.cleaned_data["endpoints"]) + context["endpoints_to_add"] = endpoints_from_form + added_endpoints + # Override the form values of active and verified + if activeChoice := form.cleaned_data.get("active", None): + if activeChoice == "force_to_true": + context["active"] = True + elif activeChoice == "force_to_false": + context["active"] = False + if verifiedChoice := form.cleaned_data.get("verified", None): + if verifiedChoice == "force_to_true": + context["verified"] = True + elif verifiedChoice == "force_to_false": + context["verified"] = False + return None + + def process_jira_form( + self, + request: HttpRequest, + form: JIRAImportScanForm, + context: dict, + ) -> str | None: + """ + Process the jira form by first making sure one was supplied + and then setting any values supplied by the user. An error + may be returned and will be bubbled up in the form of a message + """ + # Determine if push all issues is enabled + push_all_jira_issues = context.get("push_all_jira_issues", False) + context["push_to_jira"] = push_all_jira_issues or (form and form.cleaned_data.get("push_to_jira")) + return None + + def success_redirect( + self, + request: HttpRequest, + context: dict, + ) -> HttpResponseRedirect: + """Redirect the user to a place that indicates a successful import""" + duration = time.perf_counter() - request._start_time + LargeScanSizeProductAnnouncement(request=request, duration=duration) + ScanTypeProductAnnouncement(request=request, scan_type=context.get("scan_type")) + return HttpResponseRedirect(reverse("view_test", args=(context.get("test").id, ))) + + def failure_redirect( + self, + request: HttpRequest, + context: dict, + ) -> HttpResponseRedirect: + """Redirect the user to a place that indicates a failed import""" + ErrorPageProductAnnouncement(request=request) + if obj := context.get("engagement"): + url = "import_scan_results" + else: + obj = context.get("product") + url = "import_scan_results_prod" + return HttpResponseRedirect(reverse( + url, + args=(obj.id, ), + )) + + def get( + self, + request: HttpRequest, + engagement_id: int | None = None, + product_id: int | None = None, + ) -> HttpResponse: + """Process GET requests for the Import View""" + # process the request and path parameters + request, context = self.handle_request( + request, + engagement_id=engagement_id, + product_id=product_id, + ) + # Render the form + return render(request, self.get_template(), context) + + def post( + self, + request: HttpRequest, + engagement_id: int | None = None, + product_id: int | None = None, + ) -> HttpResponse: + """Process POST requests for the Import View""" + # process the request and path parameters + request, context = self.handle_request( + request, + engagement_id=engagement_id, + product_id=product_id, + ) + request._start_time = time.perf_counter() + # ensure all three forms are valid first before moving forward + if form_errors := self.validate_forms(context): + for form_error in form_errors: + add_error_message_to_response(form_error) + return self.failure_redirect(request, context) + # Process the jira form if it is present + if form_error := self.process_jira_form(request, context.get("jform"), context): + add_error_message_to_response(form_error) + return self.failure_redirect(request, context) + # Process the import form + if form_error := self.process_form(request, context.get("form"), context): + add_error_message_to_response(form_error) + return self.failure_redirect(request, context) + # Add pghistory context for audit trail (adds to existing middleware context) + pghistory.context( + source="import", + scan_type=context.get("scan_type"), + ) + # Kick off the import process + if import_error := self.import_findings(context): + add_error_message_to_response(import_error) + return self.failure_redirect(request, context) + # Add test_id to pghistory context now that test is created + if test := context.get("test"): + pghistory.context(test_id=test.id) + # Otherwise return the user back to the engagement (if present) or the product + return self.success_redirect(request, context) + + +def close_eng(request, eid): + eng = Engagement.objects.get(id=eid) + close_engagement(eng) + messages.add_message( + request, + messages.SUCCESS, + "Engagement closed successfully.", + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_engagements", args=(eng.product.id, ))) + + +@require_POST +def unlink_jira(request, eid): + eng = get_object_or_404(Engagement, id=eid) + logger.info("trying to unlink a linked jira epic from engagement %d:%s", eng.id, eng.name) + if eng.has_jira_issue: + try: + jira_services.unlink(request, eng) + messages.add_message( + request, + messages.SUCCESS, + "Link to JIRA epic successfully deleted", + extra_tags="alert-success", + ) + return JsonResponse({"result": "OK"}) + except Exception: + logger.exception("Link to JIRA epic could not be deleted") + messages.add_message( + request, + messages.ERROR, + "Link to JIRA epic could not be deleted, see alerts for details", + extra_tags="alert-danger", + ) + return HttpResponse(status=500) + else: + messages.add_message( + request, + messages.ERROR, + "Link to JIRA epic not found", + extra_tags="alert-danger", + ) + return HttpResponse(status=400) + + +def reopen_eng(request, eid): + eng = Engagement.objects.get(id=eid) + reopen_engagement(eng) + messages.add_message( + request, + messages.SUCCESS, + "Engagement reopened successfully.", + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_engagements", args=(eng.product.id, ))) + + +""" +Greg: +status: in production +method to complete checklists from the engagement view +""" + + +def complete_checklist(request, eid): + eng = get_object_or_404(Engagement, id=eid) + try: + checklist = Check_List.objects.get(engagement=eng) + except: + checklist = None + + add_breadcrumb( + parent=eng, + title="Complete checklist", + top_level=False, + request=request) + if request.method == "POST": + tests = Test.objects.filter(engagement=eng) + findings = Finding.objects.filter(test__in=tests).all() + form = CheckForm(request.POST, instance=checklist, findings=findings) + if form.is_valid(): + cl = form.save(commit=False) + try: + check_l = Check_List.objects.get(engagement=eng) + cl.id = check_l.id + cl.save() + form.save_m2m() + except: + cl.engagement = eng + cl.save() + form.save_m2m() + messages.add_message( + request, + messages.SUCCESS, + "Checklist saved.", + extra_tags="alert-success") + return HttpResponseRedirect( + reverse("view_engagement", args=(eid, ))) + else: + tests = Test.objects.filter(engagement=eng) + findings = Finding.objects.filter(test__in=tests).all() + form = CheckForm(instance=checklist, findings=findings) + + product_tab = Product_Tab(eng.product, title="Checklist", tab="engagements") + product_tab.setEngagement(eng) + return render(request, "dojo/checklist.html", { + "form": form, + "product_tab": product_tab, + "eid": eng.id, + "findings": findings, + }) + + +def add_risk_acceptance(request, eid, fid=None): + eng = get_object_or_404(Engagement, id=eid) + finding = None + if fid: + finding = get_object_or_404(Finding, id=fid) + + if not eng.product.enable_full_risk_acceptance: + raise PermissionDenied + + if request.method == "POST": + form = RiskAcceptanceForm(request.POST, request.FILES) + if form.is_valid(): + # first capture notes param as it cannot be saved directly as m2m + notes = None + if form.cleaned_data["notes"]: + notes = Notes( + entry=form.cleaned_data["notes"], + author=request.user, + date=timezone.now()) + notes.save() + + del form.cleaned_data["notes"] + + try: + # we sometimes see a weird exception here, but are unable to reproduce. + # we add some logging in case it happens + risk_acceptance = form.save() + except Exception: + logger.debug(vars(request.POST)) + logger.error(vars(form)) + logger.exception("Creation of Risk Acc. is not possible") + raise + + # attach note to risk acceptance object now in database + if notes: + risk_acceptance.notes.add(notes) + + eng.risk_acceptance.add(risk_acceptance) + + findings = form.cleaned_data["accepted_findings"] + + risk_acceptance = ra_helper.add_findings_to_risk_acceptance(request.user, risk_acceptance, findings) + + messages.add_message( + request, + messages.SUCCESS, + "Risk acceptance saved.", + extra_tags="alert-success") + + return redirect_to_return_url_or_else(request, reverse("view_engagement", args=(eid, ))) + else: + risk_acceptance_title_suggestion = f"Accept: {finding}" + form = RiskAcceptanceForm(initial={"owner": request.user, "name": risk_acceptance_title_suggestion}) + + finding_choices = Finding.objects.filter(duplicate=False, test__engagement=eng).filter(NOT_ACCEPTED_FINDINGS_QUERY).prefetch_related("test", "finding_group_set").order_by("test__id", "numerical_severity", "title") + + form.fields["accepted_findings"].queryset = finding_choices + if fid: + # Set the initial selected finding + form.fields["accepted_findings"].initial = {fid} + # Change the label for each finding in the dropdown + form.fields["accepted_findings"].label_from_instance = lambda obj: f"({obj.test.scan_type}) - ({obj.severity}) - {obj.title} - {obj.date} - {obj.status()} - {obj.finding_group})" + product_tab = Product_Tab(eng.product, title="Risk Acceptance", tab="engagements") + product_tab.setEngagement(eng) + + return render(request, "dojo/add_risk_acceptance.html", { + "eng": eng, + "product_tab": product_tab, + "form": form, + }) + + +def view_risk_acceptance(request, eid, raid): + return view_edit_risk_acceptance(request, eid=eid, raid=raid, edit_mode=False) + + +def edit_risk_acceptance(request, eid, raid): + return view_edit_risk_acceptance(request, eid=eid, raid=raid, edit_mode=True) + + +# will only be called by view_risk_acceptance and edit_risk_acceptance +def view_edit_risk_acceptance(request, eid, raid, *, edit_mode=False): + risk_acceptance = get_object_or_404(Risk_Acceptance, pk=raid) + eng = get_object_or_404(Engagement.objects.filter(risk_acceptance=risk_acceptance), pk=eid) + + if edit_mode and not eng.product.enable_full_risk_acceptance: + raise PermissionDenied + + risk_acceptance_form = None + errors = False + + if request.method == "POST": + # deleting before instantiating the form otherwise django messes up and we end up with an empty path value + if len(request.FILES) > 0: + logger.debug("new proof uploaded") + risk_acceptance.path.delete() + + if "decision" in request.POST: + old_expiration_date = risk_acceptance.expiration_date + risk_acceptance_form = EditRiskAcceptanceForm(request.POST, request.FILES, instance=risk_acceptance) + errors = errors or not risk_acceptance_form.is_valid() + if not errors: + logger.debug(f"path: {risk_acceptance_form.cleaned_data['path']}") + + risk_acceptance_form.save() + + if risk_acceptance.expiration_date != old_expiration_date: + # risk acceptance was changed, check if risk acceptance needs to be reinstated and findings made accepted again + ra_helper.reinstate(risk_acceptance, old_expiration_date) + + messages.add_message( + request, + messages.SUCCESS, + "Risk Acceptance saved successfully.", + extra_tags="alert-success") + + if "entry" in request.POST: + note_form = NoteForm(request.POST) + errors = errors or not note_form.is_valid() + if not errors: + new_note = note_form.save(commit=False) + new_note.author = request.user + new_note.date = timezone.now() + new_note.save() + risk_acceptance.notes.add(new_note) + messages.add_message( + request, + messages.SUCCESS, + "Note added successfully.", + extra_tags="alert-success") + + if "delete_note" in request.POST: + note = get_object_or_404(Notes, pk=request.POST["delete_note_id"]) + if note.author.username == request.user.username: + risk_acceptance.notes.remove(note) + note.delete() + messages.add_message( + request, + messages.SUCCESS, + "Note deleted successfully.", + extra_tags="alert-success") + else: + messages.add_message( + request, + messages.ERROR, + "Since you are not the note's author, it was not deleted.", + extra_tags="alert-danger") + + if edit_mode and "remove_finding" in request.POST: + finding = get_object_or_404( + risk_acceptance.accepted_findings, + pk=request.POST["remove_finding_id"]) + + ra_helper.remove_finding_from_risk_acceptance(request.user, risk_acceptance, finding) + + messages.add_message( + request, + messages.SUCCESS, + "Finding removed successfully from risk acceptance.", + extra_tags="alert-success") + + if "replace_file" in request.POST: + replace_form = ReplaceRiskAcceptanceProofForm( + request.POST, request.FILES, instance=risk_acceptance) + + errors = errors or not replace_form.is_valid() + if not errors: + replace_form.save() + + messages.add_message( + request, + messages.SUCCESS, + "New Proof uploaded successfully.", + extra_tags="alert-success") + else: + logger.error(replace_form.errors) + + if "add_findings" in request.POST: + add_findings_form = AddFindingsRiskAcceptanceForm( + request.POST, request.FILES, instance=risk_acceptance) + errors = errors or not add_findings_form.is_valid() + if not errors: + findings = add_findings_form.cleaned_data["accepted_findings"] + + ra_helper.add_findings_to_risk_acceptance(request.user, risk_acceptance, findings) + + messages.add_message( + request, + messages.SUCCESS, + f"Finding{'s' if len(findings) > 1 else ''} added successfully.", + extra_tags="alert-success") + if not errors: + logger.debug("redirecting to return_url") + return redirect_to_return_url_or_else(request, reverse("view_risk_acceptance", args=(eid, raid))) + logger.error("errors found") + + elif edit_mode: + risk_acceptance_form = EditRiskAcceptanceForm(instance=risk_acceptance) + + note_form = NoteForm() + replace_form = ReplaceRiskAcceptanceProofForm(instance=risk_acceptance) + add_findings_form = AddFindingsRiskAcceptanceForm(instance=risk_acceptance) + + accepted_findings = risk_acceptance.accepted_findings.order_by("numerical_severity") + fpage = get_page_items(request, accepted_findings, 15) + + unaccepted_findings = Finding.objects.filter(test__in=eng.test_set.all(), risk_accepted=False) \ + .exclude(id__in=accepted_findings).order_by("title") + add_fpage = get_page_items(request, unaccepted_findings, 25, "apage") + # on this page we need to add unaccepted findings as possible findings to add as accepted + + add_findings_form.fields[ + "accepted_findings"].queryset = add_fpage.object_list + + add_findings_form.fields["accepted_findings"].widget.request = request + add_findings_form.fields["accepted_findings"].widget.findings = unaccepted_findings + add_findings_form.fields["accepted_findings"].widget.page_number = add_fpage.number + + product_tab = Product_Tab(eng.product, title="Risk Acceptance", tab="engagements") + product_tab.setEngagement(eng) + return render( + request, "dojo/view_risk_acceptance.html", { + "risk_acceptance": risk_acceptance, + "engagement": eng, + "product_tab": product_tab, + "accepted_findings": fpage, + "notes": risk_acceptance.notes.all(), + "eng": eng, + "edit_mode": edit_mode, + "risk_acceptance_form": risk_acceptance_form, + "note_form": note_form, + "replace_form": replace_form, + "add_findings_form": add_findings_form, + # 'show_add_findings_form': len(unaccepted_findings), + "request": request, + "add_findings": add_fpage, + "return_url": get_return_url(request), + "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), + }) + + +def expire_risk_acceptance(request, eid, raid): + risk_acceptance = get_object_or_404(prefetch_for_expiration(Risk_Acceptance.objects.all()), pk=raid) + # Validate the engagement ID exists before moving forward + get_object_or_404(Engagement.objects.filter(risk_acceptance=risk_acceptance), pk=eid) + + ra_helper.expire_now(risk_acceptance) + + return redirect_to_return_url_or_else(request, reverse("view_risk_acceptance", args=(eid, raid))) + + +def reinstate_risk_acceptance(request, eid, raid): + risk_acceptance = get_object_or_404(prefetch_for_expiration(Risk_Acceptance.objects.all()), pk=raid) + eng = get_object_or_404(Engagement.objects.filter(risk_acceptance=risk_acceptance), pk=eid) + if not eng.product.enable_full_risk_acceptance: + raise PermissionDenied + + ra_helper.reinstate(risk_acceptance, risk_acceptance.expiration_date) + + return redirect_to_return_url_or_else(request, reverse("view_risk_acceptance", args=(eid, raid))) + + +def delete_risk_acceptance(request, eid, raid): + risk_acceptance = get_object_or_404(Risk_Acceptance, pk=raid) + eng = get_object_or_404(Engagement.objects.filter(risk_acceptance=risk_acceptance), pk=eid) + ra_helper.delete(eng, risk_acceptance) + + messages.add_message( + request, + messages.SUCCESS, + "Risk acceptance deleted successfully.", + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_engagement", args=(eng.id, ))) + + +def download_risk_acceptance(request, eid, raid): + mimetypes.init() + risk_acceptance = get_object_or_404(Risk_Acceptance, pk=raid) + # Ensure the risk acceptance is under the supplied engagement + if not Engagement.objects.filter(risk_acceptance=risk_acceptance, id=eid).exists(): + raise PermissionDenied + file_handle = (Path(settings.MEDIA_ROOT) / risk_acceptance.path.name).open(mode="rb") + response = StreamingHttpResponse(FileIterWrapper(file_handle)) + if hasattr(response, "_resource_closers"): + response._resource_closers.append(file_handle.close) + response["Content-Disposition"] = f'attachment; filename="{risk_acceptance.filename()}"' + mimetype, _encoding = mimetypes.guess_type(risk_acceptance.path.name) + response["Content-Type"] = mimetype or "application/octet-stream" + return response + + +""" +Greg +status: in production +Upload a threat model at the engagement level. Threat models are stored +under media folder +""" + + +def upload_threatmodel(request, eid): + eng = Engagement.objects.get(id=eid) + add_breadcrumb( + parent=eng, + title="Upload a threat model", + top_level=False, + request=request) + + if request.method == "POST": + form = UploadThreatForm(request.POST, request.FILES) + if form.is_valid(): + handle_uploaded_threat(request.FILES["file"], eng) + eng.progress = "other" + eng.threat_model = True + eng.save() + messages.add_message( + request, + messages.SUCCESS, + "Threat model saved.", + extra_tags="alert-success") + return HttpResponseRedirect( + reverse("view_engagement", args=(eid, ))) + else: + form = UploadThreatForm() + product_tab = Product_Tab(eng.product, title="Upload Threat Model", tab="engagements") + return render(request, "dojo/up_threat.html", { + "form": form, + "product_tab": product_tab, + "eng": eng, + }) + + +def view_threatmodel(request, eid): + eng = get_object_or_404(Engagement, pk=eid) + return generate_file_response_from_file_path(eng.tmodel_path) + + +def engagement_ics(request, eid): + eng = get_object_or_404(Engagement, id=eid) + start_date = datetime.combine(eng.target_start, datetime.min.time()) + end_date = datetime.combine(eng.target_end, datetime.max.time()) + if timezone.is_naive(start_date): + start_date = timezone.make_aware(start_date) + if timezone.is_naive(end_date): + end_date = timezone.make_aware(end_date) + uid = f"dojo_eng_{eng.id}_{eng.product.id}" + cal = get_cal_event( + start_date, + end_date, + f"Engagement: {eng.name} ({eng.product.name})", + ( + f"Set aside for engagement {eng.name}, on product {eng.product.name}. " + f"Additional detail can be found at {request.build_absolute_uri(reverse('view_engagement', args=(eng.id, )))}" + ), + uid, + ) + output = cal.serialize() + response = HttpResponse(content=output) + response["Content-Type"] = "text/calendar" + response["Content-Disposition"] = f"attachment; filename={eng.name}.ics" + return response + + +def get_list_index(full_list, index): + try: + element = full_list[index] + except Exception: + element = None + return element + + +def get_engagements(request): + url = request.META.get("QUERY_STRING") + if not url: + msg = "Please use the export button when exporting engagements" + raise ValidationError(msg) + url = url.removeprefix("url=") + + path_items = list(filter(None, re.split(r"/|\?", url))) + + if not path_items or path_items[0] != "engagement": + msg = "URL is not an engagement view" + raise ValidationError(msg) + + view = query = None + if get_list_index(path_items, 1) in {"active", "all"}: + view = get_list_index(path_items, 1) + query = get_list_index(path_items, 2) + else: + view = "active" + query = get_list_index(path_items, 1) + + request.GET = QueryDict(query) + return get_filtered_engagements(request, view).qs + + +def get_excludes(): + return [ + "is_ci_cd", + "jira_issue", + "jira_project", + "objects", + "unaccepted_open_findings", + "test_count", # already exported separately as “tests” + ] + + +def get_foreign_keys(): + return ["build_server", "lead", "orchestration_engine", "preset", "product", + "report_type", "requester", "source_code_management_server"] + + +def csv_export(request): + logger.debug("starting csv export") + engagements = get_engagements(request) + + response = HttpResponse(content_type="text/csv") + response["Content-Disposition"] = "attachment; filename=engagements.csv" + + writer = csv.writer(response) + + first_row = True + for engagement in engagements: + if first_row: + fields = [key for key in dir(engagement) + if key not in get_excludes() and not callable(getattr(engagement, key)) and not key.startswith("_")] + fields.append("tests") + + writer.writerow(fields) + + first_row = False + if not first_row: + fields = [] + for key in dir(engagement): + if key not in get_excludes() and not callable(getattr(engagement, key)) and not key.startswith("_"): + value = engagement.__dict__.get(key) + if key in get_foreign_keys() and getattr(engagement, key): + value = str(getattr(engagement, key)) + if value and isinstance(value, str): + value = value.replace("\n", " NEWLINE ").replace("\r", "") + fields.append(value) + fields.append(getattr(engagement, "test_count", 0)) + + writer.writerow(fields) + logger.debug("done with csv export") + return response + + +def excel_export(request): + logger.debug("starting excel export") + engagements = get_engagements(request) + + workbook = Workbook() + workbook.iso_dates = True + worksheet = workbook.active + worksheet.title = "Engagements" + + font_bold = Font(bold=True) + + row_num = 1 + for engagement in engagements: + if row_num == 1: + col_num = 1 + for key in dir(engagement): + if key not in get_excludes() and not callable(getattr(engagement, key)) and not key.startswith("_"): + cell = worksheet.cell(row=row_num, column=col_num, value=key) + cell.font = font_bold + col_num += 1 + cell = worksheet.cell(row=row_num, column=col_num, value="tests") + cell.font = font_bold + row_num = 2 + if row_num > 1: + col_num = 1 + for key in dir(engagement): + if key not in get_excludes() and not callable(getattr(engagement, key)) and not key.startswith("_"): + value = engagement.__dict__.get(key) + if key in get_foreign_keys() and getattr(engagement, key): + value = str(getattr(engagement, key)) + if value and isinstance(value, datetime): + value = value.replace(tzinfo=None) + worksheet.cell(row=row_num, column=col_num, value=value) + col_num += 1 + worksheet.cell(row=row_num, column=col_num, value=getattr(engagement, "test_count", 0)) + row_num += 1 + + with NamedTemporaryFile() as tmp: + workbook.save(tmp.name) + tmp.seek(0) + stream = tmp.read() + + response = HttpResponse( + content=stream, + content_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", + ) + response["Content-Disposition"] = "attachment; filename=engagements.xlsx" + logger.debug("done with excel export") + return response diff --git a/dojo/engagement/views.py b/dojo/engagement/views.py index 0154aa5d336..af84d6b39e1 100644 --- a/dojo/engagement/views.py +++ b/dojo/engagement/views.py @@ -1,1697 +1,4 @@ -import csv -import logging -import mimetypes -import operator -import re -import time -from datetime import datetime, timedelta -from functools import partial, reduce -from pathlib import Path -from tempfile import NamedTemporaryFile -from time import strftime - -import pghistory -from django.conf import settings -from django.contrib import messages -from django.contrib.admin.utils import NestedObjects -from django.contrib.auth.models import User -from django.core.exceptions import PermissionDenied, ValidationError -from django.db import DEFAULT_DB_ALIAS -from django.db.models import OuterRef, Q, Value -from django.db.models.functions import Coalesce -from django.db.models.query import Prefetch, QuerySet -from django.http import HttpRequest, HttpResponse, HttpResponseRedirect, JsonResponse, QueryDict, StreamingHttpResponse -from django.shortcuts import get_object_or_404, render -from django.urls import Resolver404, reverse -from django.utils import timezone -from django.utils.translation import gettext as _ -from django.views import View -from django.views.decorators.cache import cache_page -from django.views.decorators.http import require_POST -from django.views.decorators.vary import vary_on_cookie -from openpyxl import Workbook -from openpyxl.styles import Font - -import dojo.risk_acceptance.helper as ra_helper -from dojo.authorization.authorization import user_has_permission_or_403 -from dojo.celery_dispatch import dojo_dispatch_task -from dojo.endpoint.utils import save_endpoints_to_add -from dojo.engagement.queries import get_authorized_engagements -from dojo.engagement.services import close_engagement, reopen_engagement -from dojo.filters import ( - EngagementDirectFilter, - EngagementDirectFilterWithoutObjectLookups, - EngagementFilter, - EngagementFilterWithoutObjectLookups, - EngagementTestFilter, - EngagementTestFilterWithoutObjectLookups, - ProductEngagementsFilter, - ProductEngagementsFilterWithoutObjectLookups, -) -from dojo.finding.helper import NOT_ACCEPTED_FINDINGS_QUERY -from dojo.finding.views import find_available_notetypes -from dojo.forms import ( - AddFindingsRiskAcceptanceForm, - CheckForm, - DeleteEngagementForm, - DoneForm, - EditRiskAcceptanceForm, - EngForm, - ImportScanForm, - JIRAEngagementForm, - JIRAImportScanForm, - JIRAProjectForm, - NoteForm, - ReplaceRiskAcceptanceProofForm, - RiskAcceptanceForm, - TestForm, - TypedNoteForm, - UploadThreatForm, -) -from dojo.importers.base_importer import BaseImporter -from dojo.importers.default_importer import DefaultImporter -from dojo.jira import services as jira_services -from dojo.location.models import Location -from dojo.location.utils import save_locations_to_add -from dojo.models import ( - Check_List, - Development_Environment, - Dojo_User, - Endpoint, - Engagement, - Finding, - Note_Type, - Notes, - Product, - Product_API_Scan_Configuration, - Risk_Acceptance, - System_Settings, - Test, - Test_Import, -) -from dojo.notifications.helper import create_notification -from dojo.product.queries import get_authorized_products -from dojo.product_announcements import ( - ErrorPageProductAnnouncement, - LargeScanSizeProductAnnouncement, - ScanTypeProductAnnouncement, -) -from dojo.query_utils import build_count_subquery -from dojo.risk_acceptance.helper import prefetch_for_expiration -from dojo.tools.factory import get_scan_types_sorted -from dojo.user.queries import get_authorized_users -from dojo.utils import ( - FileIterWrapper, - Product_Tab, - add_breadcrumb, - add_error_message_to_response, - add_success_message_to_response, - async_delete, - calculate_grade, - generate_file_response_from_file_path, - get_cal_event, - get_page_items, - get_return_url, - get_setting, - get_system_setting, - handle_uploaded_threat, - redirect_to_return_url_or_else, -) - -logger = logging.getLogger(__name__) - - -@cache_page(60 * 5) # cache for 5 minutes -@vary_on_cookie -def engagement_calendar(request): - - if not get_system_setting("enable_calendar"): - raise Resolver404 - - if "lead" not in request.GET or "0" in request.GET.getlist("lead"): - engagements = get_authorized_engagements("view") - else: - filters = [] - leads = request.GET.getlist("lead", "") - if "-1" in request.GET.getlist("lead"): - leads.remove("-1") - filters.append(Q(lead__isnull=True)) - filters.append(Q(lead__in=leads)) - engagements = get_authorized_engagements("view").filter(reduce(operator.or_, filters)) - - engagements = engagements.select_related("lead") - engagements = engagements.prefetch_related("product") - - for e in engagements: - if e.target_end: - e.target_end += timedelta(days=1) - add_breadcrumb( - title="Engagement Calendar", top_level=True, request=request) - return render( - request, "dojo/calendar.html", { - "caltype": "engagements", - "leads": request.GET.getlist("lead", ""), - "engagements": engagements, - "users": get_authorized_users("view"), - }) - - -def get_filtered_engagements(request, view): - if view not in {"all", "active"}: - msg = f"View {view} is not allowed" - raise ValidationError(msg) - - engagements = get_authorized_engagements("view").order_by("-target_start") - - if view == "active": - engagements = engagements.filter(active=True) - - engagements = ( - engagements - .select_related("product", "product__prod_type") - .prefetch_related("lead", "tags", "product__tags") - ) - - if System_Settings.objects.get().enable_jira: - engagements = engagements.prefetch_related( - "jira_project__jira_instance", - "product__jira_project_set__jira_instance", - ) - - test_count_subquery = build_count_subquery( - Test.objects.filter(engagement=OuterRef("pk")), group_field="engagement_id", - ) - engagements = engagements.annotate(test_count=Coalesce(test_count_subquery, Value(0))) - - filter_string_matching = get_system_setting("filter_string_matching", False) - filter_class = EngagementDirectFilterWithoutObjectLookups if filter_string_matching else EngagementDirectFilter - return filter_class(request.GET, queryset=engagements) - - -def engagements(request, view): - if not view: - view = "active" - - filtered_engagements = get_filtered_engagements(request, view) - - engs = get_page_items(request, filtered_engagements.qs, 25) - product_name_words = sorted(get_authorized_products("view").values_list("name", flat=True)) - engagement_name_words = sorted(get_authorized_engagements("view").values_list("name", flat=True).distinct()) - - add_breadcrumb( - title=f"{view.capitalize()} Engagements", - top_level=not len(request.GET), - request=request) - - return render( - request, "dojo/engagement.html", { - "engagements": engs, - "filter_form": filtered_engagements.form, - "product_name_words": product_name_words, - "engagement_name_words": engagement_name_words, - "view": view.capitalize(), - }) - - -def engagements_all(request): - - products_with_engagements = get_authorized_products("view") - products_with_engagements = products_with_engagements.filter(~Q(engagement=None)).distinct() - - # count using prefetch instead of just using 'engagement__set_test_test` to avoid loading all test in memory just to count them - filter_string_matching = get_system_setting("filter_string_matching", False) - products_filter_class = ProductEngagementsFilterWithoutObjectLookups if filter_string_matching else ProductEngagementsFilter - test_count_subquery = build_count_subquery( - Test.objects.filter(engagement=OuterRef("pk")), - group_field="engagement_id", - ) - engagement_query = Engagement.objects.annotate(test_count=Coalesce(test_count_subquery, Value(0))) - filter_qs = products_with_engagements.prefetch_related( - Prefetch("engagement_set", queryset=products_filter_class(request.GET, engagement_query).qs), - ) - - filter_qs = filter_qs.prefetch_related( - "engagement_set__tags", - "prod_type", - "engagement_set__lead", - "tags", - ) - if System_Settings.objects.get().enable_jira: - filter_qs = filter_qs.prefetch_related( - "engagement_set__jira_project__jira_instance", - "jira_project_set__jira_instance", - ) - filter_class = EngagementFilterWithoutObjectLookups if filter_string_matching else EngagementFilter - filtered = filter_class( - request.GET, - queryset=filter_qs, - ) - - prods = get_page_items(request, filtered.qs, 25) - prods.paginator.count = sum(len(prod.engagement_set.all()) for prod in prods) - name_words = products_with_engagements.values_list("name", flat=True) - eng_words = get_authorized_engagements("view").values_list("name", flat=True).distinct() - - add_breadcrumb( - title="All Engagements", - top_level=not len(request.GET), - request=request) - - return render( - request, "dojo/engagements_all.html", { - "products": prods, - "filter_form": filtered.form, - "name_words": sorted(set(name_words)), - "eng_words": sorted(set(eng_words)), - "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), - }) - - -def edit_engagement(request, eid): - engagement = Engagement.objects.get(pk=eid) - is_ci_cd = engagement.engagement_type == "CI/CD" - jira_project_form = None - jira_epic_form = None - jira_project = None - - if request.method == "POST": - form = EngForm(request.POST, instance=engagement, cicd=is_ci_cd, product=engagement.product, user=request.user) - jira_project = jira_services.get_project(engagement, use_inheritance=False) - - if form.is_valid(): - # first save engagement details - new_status = form.cleaned_data.get("status") - if form.cleaned_data.get("product") != engagement.product: - user_has_permission_or_403( - request.user, - form.cleaned_data.get("product"), - "edit", - ) - engagement.product = form.cleaned_data.get("product") - engagement = form.save(commit=False) - if (new_status in {"Cancelled", "Completed"}): - engagement.active = False - else: - engagement.active = True - engagement.save() - form.save_m2m() - - messages.add_message( - request, - messages.SUCCESS, - "Engagement updated successfully.", - extra_tags="alert-success") - - success, jira_project_form = jira_services.process_project_form(request, instance=jira_project, target="engagement", engagement=engagement, product=engagement.product) - error = not success - - success, jira_epic_form = jira_services.process_epic_form(request, engagement=engagement) - error = error or not success - - if not error: - if "_Add Tests" in request.POST: - return HttpResponseRedirect( - reverse("add_tests", args=(engagement.id, ))) - return HttpResponseRedirect( - reverse("view_engagement", args=(engagement.id, ))) - else: - logger.debug(form.errors) - - else: - form = EngForm(initial={"product": engagement.product}, instance=engagement, cicd=is_ci_cd, product=engagement.product, user=request.user) - - jira_epic_form = None - if get_system_setting("enable_jira"): - jira_project = jira_services.get_project(engagement, use_inheritance=False) - jira_project_form = JIRAProjectForm(instance=jira_project, target="engagement", product=engagement.product) - logger.debug("showing jira-epic-form") - jira_epic_form = JIRAEngagementForm(instance=engagement) - - title = "Edit CI/CD Engagement" if is_ci_cd else "Edit Interactive Engagement" - - product_tab = Product_Tab(engagement.product, title=title, tab="engagements") - product_tab.setEngagement(engagement) - return render(request, "dojo/new_eng.html", { - "product_tab": product_tab, - "title": title, - "form": form, - "edit": True, - "jira_epic_form": jira_epic_form, - "jira_project_form": jira_project_form, - "engagement": engagement, - }) - - -def delete_engagement(request, eid): - engagement = get_object_or_404(Engagement, pk=eid) - product = engagement.product - form = DeleteEngagementForm(instance=engagement) - - if request.method == "POST": - if "id" in request.POST and str(engagement.id) == request.POST["id"]: - form = DeleteEngagementForm(request.POST, instance=engagement) - if form.is_valid(): - product = engagement.product - if get_setting("ASYNC_OBJECT_DELETE"): - async_del = async_delete() - async_del.delete(engagement) - message = "Engagement and relationships will be removed in the background." - else: - message = "Engagement and relationships removed." - engagement.delete() - messages.add_message( - request, - messages.SUCCESS, - message, - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_engagements", args=(product.id, ))) - - rels = ["Previewing the relationships has been disabled.", ""] - display_preview = get_setting("DELETE_PREVIEW") - if display_preview: - collector = NestedObjects(using=DEFAULT_DB_ALIAS) - collector.collect([engagement]) - rels = collector.nested() - - product_tab = Product_Tab(product, title="Delete Engagement", tab="engagements") - product_tab.setEngagement(engagement) - return render(request, "dojo/delete_engagement.html", { - "product_tab": product_tab, - "engagement": engagement, - "form": form, - "rels": rels, - }) - - -def copy_engagement(request, eid): - engagement = get_object_or_404(Engagement, id=eid) - product = engagement.product - form = DoneForm() - - if request.method == "POST": - form = DoneForm(request.POST) - if form.is_valid(): - engagement_copy = engagement.copy() - dojo_dispatch_task(calculate_grade, product.id) - messages.add_message( - request, - messages.SUCCESS, - "Engagement Copied successfully.", - extra_tags="alert-success") - create_notification(event="engagement_copied", # TODO: - if 'copy' functionality will be supported by API as well, 'create_notification' needs to be migrated to place where it will be able to cover actions from both interfaces - title=_("Copying of %s") % engagement.name, - description=f'The engagement "{engagement.name}" was copied by {request.user}', - product=product, - url=request.build_absolute_uri(reverse("view_engagement", args=(engagement_copy.id, ))), - recipients=[engagement.lead], - icon="exclamation-triangle") - return redirect_to_return_url_or_else(request, reverse("view_engagements", args=(product.id, ))) - messages.add_message( - request, - messages.ERROR, - "Unable to copy engagement, please try again.", - extra_tags="alert-danger") - - product_tab = Product_Tab(product, title="Copy Engagement", tab="engagements") - return render(request, "dojo/copy_object.html", { - "source": engagement, - "source_label": "Engagement", - "destination_label": "Product", - "product_tab": product_tab, - "form": form, - }) - - -class ViewEngagement(View): - - def get_template(self): - return "dojo/view_eng.html" - - def get_risks_accepted(self, eng): - accepted_findings_subquery = build_count_subquery( - Finding.objects.filter(risk_acceptance=OuterRef("pk")), - group_field="risk_acceptance", - ) - return eng.risk_acceptance.all().select_related("owner").annotate( - accepted_findings_count=Coalesce(accepted_findings_subquery, Value(0)), - ) - - def get_filtered_tests( - self, - request: HttpRequest, - queryset: list[Test], - engagement: Engagement, - ): - filter_string_matching = get_system_setting("filter_string_matching", False) - filter_class = EngagementTestFilterWithoutObjectLookups if filter_string_matching else EngagementTestFilter - return filter_class(request.GET, queryset=queryset, engagement=engagement) - - def get(self, request, eid, *args, **kwargs): - eng = get_object_or_404(Engagement, id=eid) - # Make sure the user is authorized - user_has_permission_or_403(request.user, eng, "view") - tests = eng.test_set.all().order_by("test_type__name", "-updated") - default_page_num = 10 - tests_filter = self.get_filtered_tests(request, tests, eng) - paged_tests = get_page_items(request, tests_filter.qs, default_page_num) - paged_tests.object_list = prefetch_for_view_tests(paged_tests.object_list) - prod = eng.product - risks_accepted = self.get_risks_accepted(eng) - preset_test_type = None - network = None - if eng.preset: - preset_test_type = eng.preset.test_type.all() - network = eng.preset.network_locations.all() - system_settings = System_Settings.objects.get() - - jissue = jira_services.get_issue(eng) - jira_project = jira_services.get_project(eng) - - try: - check = Check_List.objects.get(engagement=eng) - except: - check = None - notes = eng.notes.all() - note_type_activation = Note_Type.objects.filter(is_active=True).count() - if note_type_activation: - available_note_types = find_available_notetypes(notes) - form = DoneForm() - files = eng.files.all() - form = TypedNoteForm(available_note_types=available_note_types) if note_type_activation else NoteForm() - - add_breadcrumb(parent=eng, top_level=False, request=request) - - title = "" - if eng.engagement_type == "CI/CD": - title = " CI/CD" - product_tab = Product_Tab(prod, title="View" + title + " Engagement", tab="engagements") - product_tab.setEngagement(eng) - return render( - request, self.get_template(), { - "eng": eng, - "product_tab": product_tab, - "system_settings": system_settings, - "tests": paged_tests, - "filter": tests_filter, - "check": check, - "threat": eng.tmodel_path, - "form": form, - "notes": notes, - "files": files, - "risks_accepted": risks_accepted, - "jissue": jissue, - "jira_project": jira_project, - "network": network, - "preset_test_type": preset_test_type, - }) - - def post(self, request, eid, *args, **kwargs): - eng = get_object_or_404(Engagement, id=eid) - # Make sure the user is authorized - user_has_permission_or_403(request.user, eng, "view") - tests = eng.test_set.all().order_by("test_type__name", "-updated") - default_page_num = 10 - - tests_filter = self.get_filtered_tests(request, tests, eng) - paged_tests = get_page_items(request, tests_filter.qs, default_page_num) - # prefetch only after creating the filters to avoid https://code.djangoproject.com/ticket/23771 and https://code.djangoproject.com/ticket/25375 - paged_tests.object_list = prefetch_for_view_tests(paged_tests.object_list) - - prod = eng.product - risks_accepted = self.get_risks_accepted(eng) - preset_test_type = None - network = None - if eng.preset: - preset_test_type = eng.preset.test_type.all() - network = eng.preset.network_locations.all() - system_settings = System_Settings.objects.get() - - jissue = jira_services.get_issue(eng) - jira_project = jira_services.get_project(eng) - - try: - check = Check_List.objects.get(engagement=eng) - except: - check = None - notes = eng.notes.all() - note_type_activation = Note_Type.objects.filter(is_active=True).count() - if note_type_activation: - available_note_types = find_available_notetypes(notes) - form = DoneForm() - files = eng.files.all() - user_has_permission_or_403(request.user, eng, "add") - eng.progress = "check_list" - eng.save() - - if note_type_activation: - form = TypedNoteForm(request.POST, available_note_types=available_note_types) - else: - form = NoteForm(request.POST) - if form.is_valid(): - new_note = form.save(commit=False) - new_note.author = request.user - new_note.date = timezone.now() - new_note.save() - eng.notes.add(new_note) - form = TypedNoteForm(available_note_types=available_note_types) if note_type_activation else NoteForm() - title = f"Engagement: {eng.name} on {eng.product.name}" - messages.add_message(request, - messages.SUCCESS, - "Note added successfully.", - extra_tags="alert-success") - - add_breadcrumb(parent=eng, top_level=False, request=request) - - title = "" - if eng.engagement_type == "CI/CD": - title = " CI/CD" - product_tab = Product_Tab(prod, title="View" + title + " Engagement", tab="engagements") - product_tab.setEngagement(eng) - return render( - request, self.get_template(), { - "eng": eng, - "product_tab": product_tab, - "system_settings": system_settings, - "tests": paged_tests, - "filter": tests_filter, - "check": check, - "threat": eng.tmodel_path, - "form": form, - "notes": notes, - "files": files, - "risks_accepted": risks_accepted, - "jissue": jissue, - "jira_project": jira_project, - "network": network, - "preset_test_type": preset_test_type, - }) - - -def prefetch_for_view_tests(tests): - # old code can arrive here with prods being a list because the query was already executed - if not isinstance(tests, QuerySet): - logger.warning("unable to prefetch because query was already executed") - return tests - - prefetched = tests.select_related("lead", "test_type").prefetch_related("tags", "notes") - base_findings = Finding.objects.filter(test_id=OuterRef("pk")) - count_subquery = partial(build_count_subquery, group_field="test_id") - return prefetched.annotate( - count_findings_test_all=Coalesce(count_subquery(base_findings), Value(0)), - count_findings_test_active=Coalesce(count_subquery(base_findings.filter(active=True)), Value(0)), - count_findings_test_active_verified=Coalesce( - count_subquery(base_findings.filter(active=True, verified=True)), Value(0), - ), - count_findings_test_active_fix_available=Coalesce( - count_subquery(base_findings.filter(active=True, fix_available=True)), Value(0), - ), - count_findings_test_mitigated=Coalesce(count_subquery(base_findings.filter(is_mitigated=True)), Value(0)), - count_findings_test_dups=Coalesce(count_subquery(base_findings.filter(duplicate=True)), Value(0)), - total_reimport_count=Coalesce( - count_subquery(Test_Import.objects.filter(test_id=OuterRef("pk"), type=Test_Import.REIMPORT_TYPE)), - Value(0), - ), - ) - - -def add_tests(request, eid): - eng = Engagement.objects.get(id=eid) - - if request.method == "POST": - form = TestForm(request.POST, engagement=eng) - if form.is_valid(): - new_test = form.save(commit=False) - # set default scan_type as it's used in reimport - new_test.scan_type = new_test.test_type.name - new_test.engagement = eng - try: - new_test.lead = User.objects.get(id=form["lead"].value()) - except: - new_test.lead = None - - # Set status to in progress if a test is added - if eng.status != "In Progress" and eng.active is True: - eng.status = "In Progress" - eng.save() - - new_test.save() - - messages.add_message( - request, - messages.SUCCESS, - "Test added successfully.", - extra_tags="alert-success") - - create_notification( - event="test_added", - title=f"Test created for {new_test.engagement.product}: {new_test.engagement.name}: {new_test}", - test=new_test, - engagement=new_test.engagement, - product=new_test.engagement.product, - url=reverse("view_test", args=(new_test.id,)), - url_api=reverse("test-detail", args=(new_test.id,)), - ) - - if "_Add Another Test" in request.POST: - return HttpResponseRedirect( - reverse("add_tests", args=(eng.id, ))) - if "_Add Findings" in request.POST: - return HttpResponseRedirect( - reverse("add_findings", args=(new_test.id, ))) - if "_Finished" in request.POST: - return HttpResponseRedirect( - reverse("view_engagement", args=(eng.id, ))) - else: - form = TestForm(engagement=eng) - form.initial["target_start"] = eng.target_start - form.initial["target_end"] = eng.target_end - form.initial["lead"] = request.user - add_breadcrumb( - parent=eng, title="Add Tests", top_level=False, request=request) - product_tab = Product_Tab(eng.product, title="Add Tests", tab="engagements") - product_tab.setEngagement(eng) - return render(request, "dojo/add_tests.html", { - "product_tab": product_tab, - "form": form, - "eid": eid, - "eng": eng, - }) - - -class ImportScanResultsView(View): - def get_template(self) -> str: - """Returns the template that will be presented to the user""" - return "dojo/import_scan_results.html" - - def get_development_environment( - self, - environment_name: str = "Development", - ) -> Development_Environment | None: - """ - Get the development environment in two cases: - - GET: Environment "Development" by default - - POST: The label supplied by the user, with Development as a backup - """ - return Development_Environment.objects.filter(name=environment_name).first() - - def get_engagement_or_product( - self, - user: Dojo_User, - engagement_id: int | None = None, - product_id: int | None = None, - ) -> tuple[Engagement, Product, Product | Engagement]: - """Using the path parameters, either fetch the product or engagement""" - engagement = product = engagement_or_product = None - # Get the product if supplied - # Get the engagement if supplied - if engagement_id is not None: - engagement = get_object_or_404(Engagement, id=engagement_id) - engagement_or_product = engagement - elif product_id is not None: - product = get_object_or_404(Product, id=product_id) - engagement_or_product = product - else: - msg = "Either Engagement or Product has to be provided" - raise Exception(msg) - # Ensure the supplied user has access to import to the engagement or product - user_has_permission_or_403(user, engagement_or_product, "import") - - return engagement, product, engagement_or_product - - def get_form( - self, - request: HttpRequest, - **kwargs: dict, - ) -> ImportScanForm: - """Returns the default import form for importing findings""" - if request.method == "POST": - return ImportScanForm(request.POST, request.FILES, **kwargs) - return ImportScanForm(**kwargs) - - def get_jira_form( - self, - request: HttpRequest, - engagement_or_product: Engagement | Product, - ) -> tuple[JIRAImportScanForm | None, bool]: - """Returns a JiraImportScanForm if jira is enabled""" - jira_form = None - push_all_jira_issues = False - # Determine if jira issues should be pushed automatically - push_all_jira_issues = jira_services.is_push_all_issues(engagement_or_product) - # Only return the form if the jira is enabled on this engagement or product - if jira_services.get_project(engagement_or_product): - if request.method == "POST": - jira_form = JIRAImportScanForm( - request.POST, - push_all=push_all_jira_issues, - prefix="jiraform", - ) - else: - jira_form = JIRAImportScanForm( - push_all=push_all_jira_issues, - prefix="jiraform", - ) - return jira_form, push_all_jira_issues - - def get_product_tab( - self, - product: Product, - engagement: Engagement, - ) -> tuple[Product_Tab, dict]: - """ - Determine how the product tab will be rendered, and what tab will be selected - as currently active - """ - custom_breadcrumb = None - if engagement: - product_tab = Product_Tab(engagement.product, title="Import Scan Results", tab="engagements") - product_tab.setEngagement(engagement) - else: - custom_breadcrumb = {""} - product_tab = Product_Tab(product, title="Import Scan Results", tab="findings") - return product_tab, custom_breadcrumb - - def handle_request( - self, - request: HttpRequest, - engagement_id: int | None = None, - product_id: int | None = None, - ) -> tuple[HttpRequest, dict]: - """ - Process the common behaviors between request types, and then return - the request and context dict back to be rendered - """ - user = request.user - # Get the development environment - environment = self.get_development_environment() - # Get the product or engagement from the path parameters - engagement, product, engagement_or_product = self.get_engagement_or_product( - user, - engagement_id=engagement_id, - product_id=product_id, - ) - # Get the product tab and any additional custom breadcrumbs - product_tab, custom_breadcrumb = self.get_product_tab(product, engagement) - - if settings.V3_FEATURE_LOCATIONS: - endpoints = Location.objects.filter(products__product_id=product_tab.product.id) - else: - # TODO: Delete this after the move to Locations - endpoints = Endpoint.objects.filter(product__id=product_tab.product.id) - - # Get the import form with some initial data in place - form = self.get_form( - request, - environment=environment, - endpoints=endpoints, - api_scan_configuration=Product_API_Scan_Configuration.objects.filter(product__id=product_tab.product.id), - ) - # Get the jira form - jira_form, push_all_jira_issues = self.get_jira_form(request, engagement_or_product) - # Return the request and the context - return request, { - "user": user, - "lead": user, - "form": form, - "environment": environment, - "product_tab": product_tab, - "product": product, - "engagement": engagement, - "engagement_or_product": engagement_or_product, - "custom_breadcrumb": custom_breadcrumb, - "title": "Import Scan Results", - "jform": jira_form, - "scan_types": get_scan_types_sorted(), - "push_all_jira_issues": push_all_jira_issues, - } - - def validate_forms( - self, - context: dict, - ) -> bool: - """ - Validates each of the forms to ensure all errors from the form - level are bubbled up to the user first before we process too much - """ - form_validation_list = [] - for form_name in ["form", "jform"]: - if (form := context.get(form_name)) is not None: - if errors := form.errors: - form_validation_list.append(errors) - return form_validation_list - - def create_engagement( - self, - context: dict, - ) -> Engagement: - """ - Create an engagement if the import was triggered from the product level, - otherwise, return the existing engagement instead - """ - # Make sure an engagement does not exist already - engagement = context.get("engagement") - if engagement is None: - engagement = Engagement.objects.create( - name="AdHoc Import - " + strftime("%a, %d %b %Y %X", timezone.now().timetuple()), - threat_model=False, - api_test=False, - pen_test=False, - check_list=False, - active=True, - target_start=timezone.now().date(), - target_end=timezone.now().date(), - product=context.get("product"), - status="In Progress", - version=context.get("version"), - branch_tag=context.get("branch_tag"), - build_id=context.get("build_id"), - commit_hash=context.get("commit_hash"), - ) - # Update the engagement in the context - context["engagement"] = engagement - # Return the engagement - return engagement - - def get_importer( - self, - context: dict, - ) -> BaseImporter: - """Gets the importer to use""" - return DefaultImporter(**context) - - def import_findings( - self, - context: dict, - ) -> str | None: - """Attempt to import with all the supplied information""" - try: - # Log only user-entered form values, excluding internal objects - user_values = { - "scan_type": context.get("scan_type"), - "scan_date": context.get("scan_date"), - "minimum_severity": context.get("minimum_severity"), - "active": context.get("active"), - "verified": context.get("verified"), - "test_title": context.get("test_title"), - "tags": context.get("tags"), - "version": context.get("version"), - "branch_tag": context.get("branch_tag"), - "build_id": context.get("build_id"), - "commit_hash": context.get("commit_hash"), - "service": context.get("service"), - "close_old_findings": context.get("close_old_findings"), - "apply_tags_to_findings": context.get("apply_tags_to_findings"), - "apply_tags_to_endpoints": context.get("apply_tags_to_endpoints"), - "close_old_findings_product_scope": context.get("close_old_findings_product_scope"), - "group_by": context.get("group_by"), - "create_finding_groups_for_all_findings": context.get("create_finding_groups_for_all_findings"), - "push_to_jira": context.get("push_to_jira"), - "push_all_jira_issues": context.get("push_all_jira_issues"), - } - logger.debug(f"import_findings called with user values: {user_values}") - importer_client = self.get_importer(context) - context["test"], _, finding_count, closed_finding_count, _, _, _ = importer_client.process_scan( - context.pop("scan", None), - ) - # Add a message to the view for the user to see the results - add_success_message_to_response(importer_client.construct_imported_message( - finding_count=finding_count, - closed_finding_count=closed_finding_count, - )) - except Exception as e: - logger.exception("An exception error occurred during the report import") - return f"An exception error occurred during the report import: {e}" - return None - - def process_form( - self, - request: HttpRequest, - form: ImportScanForm, - context: dict, - ) -> str | None: - """Process the form and manipulate the input in any way that is appropriate""" - # Update the running context dict with cleaned form input - context.update({ - "scan": request.FILES.get("file", None), - "scan_date": form.cleaned_data.get("scan_date"), - "minimum_severity": form.cleaned_data.get("minimum_severity"), - "active": None, - "verified": None, - "scan_type": request.POST.get("scan_type"), - "test_title": form.cleaned_data.get("test_title") or None, - "tags": form.cleaned_data.get("tags"), - "version": form.cleaned_data.get("version") or None, - "branch_tag": form.cleaned_data.get("branch_tag") or None, - "build_id": form.cleaned_data.get("build_id") or None, - "commit_hash": form.cleaned_data.get("commit_hash") or None, - "api_scan_configuration": form.cleaned_data.get("api_scan_configuration") or None, - "service": form.cleaned_data.get("service") or None, - "close_old_findings": form.cleaned_data.get("close_old_findings", None), - "apply_tags_to_findings": form.cleaned_data.get("apply_tags_to_findings", False), - "apply_tags_to_endpoints": form.cleaned_data.get("apply_tags_to_endpoints", False), - "close_old_findings_product_scope": form.cleaned_data.get("close_old_findings_product_scope", None), - "group_by": form.cleaned_data.get("group_by") or None, - "create_finding_groups_for_all_findings": form.cleaned_data.get("create_finding_groups_for_all_findings", None), - "environment": self.get_development_environment(environment_name=form.cleaned_data.get("environment")), - }) - # Create the engagement if necessary - self.create_engagement(context) - # close_old_findings_product_scope is a modifier of close_old_findings. - # If it is selected, close_old_findings should also be selected. - if close_old_findings_product_scope := form.cleaned_data.get("close_old_findings_product_scope", None): - context["close_old_findings_product_scope"] = close_old_findings_product_scope - context["close_old_findings"] = True - if settings.V3_FEATURE_LOCATIONS: - # Save newly added locations - added_locations = save_locations_to_add(form.endpoints_to_add_list) - locations_from_form = [location.url for location in form.cleaned_data["endpoints"]] - context["endpoints_to_add"] = locations_from_form + added_locations - else: - # TODO: Delete this after the move to Locations - # Save newly added endpoints - added_endpoints = save_endpoints_to_add(form.endpoints_to_add_list, context.get("engagement").product) - endpoints_from_form = list(form.cleaned_data["endpoints"]) - context["endpoints_to_add"] = endpoints_from_form + added_endpoints - # Override the form values of active and verified - if activeChoice := form.cleaned_data.get("active", None): - if activeChoice == "force_to_true": - context["active"] = True - elif activeChoice == "force_to_false": - context["active"] = False - if verifiedChoice := form.cleaned_data.get("verified", None): - if verifiedChoice == "force_to_true": - context["verified"] = True - elif verifiedChoice == "force_to_false": - context["verified"] = False - return None - - def process_jira_form( - self, - request: HttpRequest, - form: JIRAImportScanForm, - context: dict, - ) -> str | None: - """ - Process the jira form by first making sure one was supplied - and then setting any values supplied by the user. An error - may be returned and will be bubbled up in the form of a message - """ - # Determine if push all issues is enabled - push_all_jira_issues = context.get("push_all_jira_issues", False) - context["push_to_jira"] = push_all_jira_issues or (form and form.cleaned_data.get("push_to_jira")) - return None - - def success_redirect( - self, - request: HttpRequest, - context: dict, - ) -> HttpResponseRedirect: - """Redirect the user to a place that indicates a successful import""" - duration = time.perf_counter() - request._start_time - LargeScanSizeProductAnnouncement(request=request, duration=duration) - ScanTypeProductAnnouncement(request=request, scan_type=context.get("scan_type")) - return HttpResponseRedirect(reverse("view_test", args=(context.get("test").id, ))) - - def failure_redirect( - self, - request: HttpRequest, - context: dict, - ) -> HttpResponseRedirect: - """Redirect the user to a place that indicates a failed import""" - ErrorPageProductAnnouncement(request=request) - if obj := context.get("engagement"): - url = "import_scan_results" - else: - obj = context.get("product") - url = "import_scan_results_prod" - return HttpResponseRedirect(reverse( - url, - args=(obj.id, ), - )) - - def get( - self, - request: HttpRequest, - engagement_id: int | None = None, - product_id: int | None = None, - ) -> HttpResponse: - """Process GET requests for the Import View""" - # process the request and path parameters - request, context = self.handle_request( - request, - engagement_id=engagement_id, - product_id=product_id, - ) - # Render the form - return render(request, self.get_template(), context) - - def post( - self, - request: HttpRequest, - engagement_id: int | None = None, - product_id: int | None = None, - ) -> HttpResponse: - """Process POST requests for the Import View""" - # process the request and path parameters - request, context = self.handle_request( - request, - engagement_id=engagement_id, - product_id=product_id, - ) - request._start_time = time.perf_counter() - # ensure all three forms are valid first before moving forward - if form_errors := self.validate_forms(context): - for form_error in form_errors: - add_error_message_to_response(form_error) - return self.failure_redirect(request, context) - # Process the jira form if it is present - if form_error := self.process_jira_form(request, context.get("jform"), context): - add_error_message_to_response(form_error) - return self.failure_redirect(request, context) - # Process the import form - if form_error := self.process_form(request, context.get("form"), context): - add_error_message_to_response(form_error) - return self.failure_redirect(request, context) - # Add pghistory context for audit trail (adds to existing middleware context) - pghistory.context( - source="import", - scan_type=context.get("scan_type"), - ) - # Kick off the import process - if import_error := self.import_findings(context): - add_error_message_to_response(import_error) - return self.failure_redirect(request, context) - # Add test_id to pghistory context now that test is created - if test := context.get("test"): - pghistory.context(test_id=test.id) - # Otherwise return the user back to the engagement (if present) or the product - return self.success_redirect(request, context) - - -def close_eng(request, eid): - eng = Engagement.objects.get(id=eid) - close_engagement(eng) - messages.add_message( - request, - messages.SUCCESS, - "Engagement closed successfully.", - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_engagements", args=(eng.product.id, ))) - - -@require_POST -def unlink_jira(request, eid): - eng = get_object_or_404(Engagement, id=eid) - logger.info("trying to unlink a linked jira epic from engagement %d:%s", eng.id, eng.name) - if eng.has_jira_issue: - try: - jira_services.unlink(request, eng) - messages.add_message( - request, - messages.SUCCESS, - "Link to JIRA epic successfully deleted", - extra_tags="alert-success", - ) - return JsonResponse({"result": "OK"}) - except Exception: - logger.exception("Link to JIRA epic could not be deleted") - messages.add_message( - request, - messages.ERROR, - "Link to JIRA epic could not be deleted, see alerts for details", - extra_tags="alert-danger", - ) - return HttpResponse(status=500) - else: - messages.add_message( - request, - messages.ERROR, - "Link to JIRA epic not found", - extra_tags="alert-danger", - ) - return HttpResponse(status=400) - - -def reopen_eng(request, eid): - eng = Engagement.objects.get(id=eid) - reopen_engagement(eng) - messages.add_message( - request, - messages.SUCCESS, - "Engagement reopened successfully.", - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_engagements", args=(eng.product.id, ))) - - -""" -Greg: -status: in production -method to complete checklists from the engagement view -""" - - -def complete_checklist(request, eid): - eng = get_object_or_404(Engagement, id=eid) - try: - checklist = Check_List.objects.get(engagement=eng) - except: - checklist = None - - add_breadcrumb( - parent=eng, - title="Complete checklist", - top_level=False, - request=request) - if request.method == "POST": - tests = Test.objects.filter(engagement=eng) - findings = Finding.objects.filter(test__in=tests).all() - form = CheckForm(request.POST, instance=checklist, findings=findings) - if form.is_valid(): - cl = form.save(commit=False) - try: - check_l = Check_List.objects.get(engagement=eng) - cl.id = check_l.id - cl.save() - form.save_m2m() - except: - cl.engagement = eng - cl.save() - form.save_m2m() - messages.add_message( - request, - messages.SUCCESS, - "Checklist saved.", - extra_tags="alert-success") - return HttpResponseRedirect( - reverse("view_engagement", args=(eid, ))) - else: - tests = Test.objects.filter(engagement=eng) - findings = Finding.objects.filter(test__in=tests).all() - form = CheckForm(instance=checklist, findings=findings) - - product_tab = Product_Tab(eng.product, title="Checklist", tab="engagements") - product_tab.setEngagement(eng) - return render(request, "dojo/checklist.html", { - "form": form, - "product_tab": product_tab, - "eid": eng.id, - "findings": findings, - }) - - -def add_risk_acceptance(request, eid, fid=None): - eng = get_object_or_404(Engagement, id=eid) - finding = None - if fid: - finding = get_object_or_404(Finding, id=fid) - - if not eng.product.enable_full_risk_acceptance: - raise PermissionDenied - - if request.method == "POST": - form = RiskAcceptanceForm(request.POST, request.FILES) - if form.is_valid(): - # first capture notes param as it cannot be saved directly as m2m - notes = None - if form.cleaned_data["notes"]: - notes = Notes( - entry=form.cleaned_data["notes"], - author=request.user, - date=timezone.now()) - notes.save() - - del form.cleaned_data["notes"] - - try: - # we sometimes see a weird exception here, but are unable to reproduce. - # we add some logging in case it happens - risk_acceptance = form.save() - except Exception: - logger.debug(vars(request.POST)) - logger.error(vars(form)) - logger.exception("Creation of Risk Acc. is not possible") - raise - - # attach note to risk acceptance object now in database - if notes: - risk_acceptance.notes.add(notes) - - eng.risk_acceptance.add(risk_acceptance) - - findings = form.cleaned_data["accepted_findings"] - - risk_acceptance = ra_helper.add_findings_to_risk_acceptance(request.user, risk_acceptance, findings) - - messages.add_message( - request, - messages.SUCCESS, - "Risk acceptance saved.", - extra_tags="alert-success") - - return redirect_to_return_url_or_else(request, reverse("view_engagement", args=(eid, ))) - else: - risk_acceptance_title_suggestion = f"Accept: {finding}" - form = RiskAcceptanceForm(initial={"owner": request.user, "name": risk_acceptance_title_suggestion}) - - finding_choices = Finding.objects.filter(duplicate=False, test__engagement=eng).filter(NOT_ACCEPTED_FINDINGS_QUERY).prefetch_related("test", "finding_group_set").order_by("test__id", "numerical_severity", "title") - - form.fields["accepted_findings"].queryset = finding_choices - if fid: - # Set the initial selected finding - form.fields["accepted_findings"].initial = {fid} - # Change the label for each finding in the dropdown - form.fields["accepted_findings"].label_from_instance = lambda obj: f"({obj.test.scan_type}) - ({obj.severity}) - {obj.title} - {obj.date} - {obj.status()} - {obj.finding_group})" - product_tab = Product_Tab(eng.product, title="Risk Acceptance", tab="engagements") - product_tab.setEngagement(eng) - - return render(request, "dojo/add_risk_acceptance.html", { - "eng": eng, - "product_tab": product_tab, - "form": form, - }) - - -def view_risk_acceptance(request, eid, raid): - return view_edit_risk_acceptance(request, eid=eid, raid=raid, edit_mode=False) - - -def edit_risk_acceptance(request, eid, raid): - return view_edit_risk_acceptance(request, eid=eid, raid=raid, edit_mode=True) - - -# will only be called by view_risk_acceptance and edit_risk_acceptance -def view_edit_risk_acceptance(request, eid, raid, *, edit_mode=False): - risk_acceptance = get_object_or_404(Risk_Acceptance, pk=raid) - eng = get_object_or_404(Engagement.objects.filter(risk_acceptance=risk_acceptance), pk=eid) - - if edit_mode and not eng.product.enable_full_risk_acceptance: - raise PermissionDenied - - risk_acceptance_form = None - errors = False - - if request.method == "POST": - # deleting before instantiating the form otherwise django messes up and we end up with an empty path value - if len(request.FILES) > 0: - logger.debug("new proof uploaded") - risk_acceptance.path.delete() - - if "decision" in request.POST: - old_expiration_date = risk_acceptance.expiration_date - risk_acceptance_form = EditRiskAcceptanceForm(request.POST, request.FILES, instance=risk_acceptance) - errors = errors or not risk_acceptance_form.is_valid() - if not errors: - logger.debug(f"path: {risk_acceptance_form.cleaned_data['path']}") - - risk_acceptance_form.save() - - if risk_acceptance.expiration_date != old_expiration_date: - # risk acceptance was changed, check if risk acceptance needs to be reinstated and findings made accepted again - ra_helper.reinstate(risk_acceptance, old_expiration_date) - - messages.add_message( - request, - messages.SUCCESS, - "Risk Acceptance saved successfully.", - extra_tags="alert-success") - - if "entry" in request.POST: - note_form = NoteForm(request.POST) - errors = errors or not note_form.is_valid() - if not errors: - new_note = note_form.save(commit=False) - new_note.author = request.user - new_note.date = timezone.now() - new_note.save() - risk_acceptance.notes.add(new_note) - messages.add_message( - request, - messages.SUCCESS, - "Note added successfully.", - extra_tags="alert-success") - - if "delete_note" in request.POST: - note = get_object_or_404(Notes, pk=request.POST["delete_note_id"]) - if note.author.username == request.user.username: - risk_acceptance.notes.remove(note) - note.delete() - messages.add_message( - request, - messages.SUCCESS, - "Note deleted successfully.", - extra_tags="alert-success") - else: - messages.add_message( - request, - messages.ERROR, - "Since you are not the note's author, it was not deleted.", - extra_tags="alert-danger") - - if edit_mode and "remove_finding" in request.POST: - finding = get_object_or_404( - risk_acceptance.accepted_findings, - pk=request.POST["remove_finding_id"]) - - ra_helper.remove_finding_from_risk_acceptance(request.user, risk_acceptance, finding) - - messages.add_message( - request, - messages.SUCCESS, - "Finding removed successfully from risk acceptance.", - extra_tags="alert-success") - - if "replace_file" in request.POST: - replace_form = ReplaceRiskAcceptanceProofForm( - request.POST, request.FILES, instance=risk_acceptance) - - errors = errors or not replace_form.is_valid() - if not errors: - replace_form.save() - - messages.add_message( - request, - messages.SUCCESS, - "New Proof uploaded successfully.", - extra_tags="alert-success") - else: - logger.error(replace_form.errors) - - if "add_findings" in request.POST: - add_findings_form = AddFindingsRiskAcceptanceForm( - request.POST, request.FILES, instance=risk_acceptance) - errors = errors or not add_findings_form.is_valid() - if not errors: - findings = add_findings_form.cleaned_data["accepted_findings"] - - ra_helper.add_findings_to_risk_acceptance(request.user, risk_acceptance, findings) - - messages.add_message( - request, - messages.SUCCESS, - f"Finding{'s' if len(findings) > 1 else ''} added successfully.", - extra_tags="alert-success") - if not errors: - logger.debug("redirecting to return_url") - return redirect_to_return_url_or_else(request, reverse("view_risk_acceptance", args=(eid, raid))) - logger.error("errors found") - - elif edit_mode: - risk_acceptance_form = EditRiskAcceptanceForm(instance=risk_acceptance) - - note_form = NoteForm() - replace_form = ReplaceRiskAcceptanceProofForm(instance=risk_acceptance) - add_findings_form = AddFindingsRiskAcceptanceForm(instance=risk_acceptance) - - accepted_findings = risk_acceptance.accepted_findings.order_by("numerical_severity") - fpage = get_page_items(request, accepted_findings, 15) - - unaccepted_findings = Finding.objects.filter(test__in=eng.test_set.all(), risk_accepted=False) \ - .exclude(id__in=accepted_findings).order_by("title") - add_fpage = get_page_items(request, unaccepted_findings, 25, "apage") - # on this page we need to add unaccepted findings as possible findings to add as accepted - - add_findings_form.fields[ - "accepted_findings"].queryset = add_fpage.object_list - - add_findings_form.fields["accepted_findings"].widget.request = request - add_findings_form.fields["accepted_findings"].widget.findings = unaccepted_findings - add_findings_form.fields["accepted_findings"].widget.page_number = add_fpage.number - - product_tab = Product_Tab(eng.product, title="Risk Acceptance", tab="engagements") - product_tab.setEngagement(eng) - return render( - request, "dojo/view_risk_acceptance.html", { - "risk_acceptance": risk_acceptance, - "engagement": eng, - "product_tab": product_tab, - "accepted_findings": fpage, - "notes": risk_acceptance.notes.all(), - "eng": eng, - "edit_mode": edit_mode, - "risk_acceptance_form": risk_acceptance_form, - "note_form": note_form, - "replace_form": replace_form, - "add_findings_form": add_findings_form, - # 'show_add_findings_form': len(unaccepted_findings), - "request": request, - "add_findings": add_fpage, - "return_url": get_return_url(request), - "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), - }) - - -def expire_risk_acceptance(request, eid, raid): - risk_acceptance = get_object_or_404(prefetch_for_expiration(Risk_Acceptance.objects.all()), pk=raid) - # Validate the engagement ID exists before moving forward - get_object_or_404(Engagement.objects.filter(risk_acceptance=risk_acceptance), pk=eid) - - ra_helper.expire_now(risk_acceptance) - - return redirect_to_return_url_or_else(request, reverse("view_risk_acceptance", args=(eid, raid))) - - -def reinstate_risk_acceptance(request, eid, raid): - risk_acceptance = get_object_or_404(prefetch_for_expiration(Risk_Acceptance.objects.all()), pk=raid) - eng = get_object_or_404(Engagement.objects.filter(risk_acceptance=risk_acceptance), pk=eid) - if not eng.product.enable_full_risk_acceptance: - raise PermissionDenied - - ra_helper.reinstate(risk_acceptance, risk_acceptance.expiration_date) - - return redirect_to_return_url_or_else(request, reverse("view_risk_acceptance", args=(eid, raid))) - - -def delete_risk_acceptance(request, eid, raid): - risk_acceptance = get_object_or_404(Risk_Acceptance, pk=raid) - eng = get_object_or_404(Engagement.objects.filter(risk_acceptance=risk_acceptance), pk=eid) - ra_helper.delete(eng, risk_acceptance) - - messages.add_message( - request, - messages.SUCCESS, - "Risk acceptance deleted successfully.", - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_engagement", args=(eng.id, ))) - - -def download_risk_acceptance(request, eid, raid): - mimetypes.init() - risk_acceptance = get_object_or_404(Risk_Acceptance, pk=raid) - # Ensure the risk acceptance is under the supplied engagement - if not Engagement.objects.filter(risk_acceptance=risk_acceptance, id=eid).exists(): - raise PermissionDenied - file_handle = (Path(settings.MEDIA_ROOT) / risk_acceptance.path.name).open(mode="rb") - response = StreamingHttpResponse(FileIterWrapper(file_handle)) - if hasattr(response, "_resource_closers"): - response._resource_closers.append(file_handle.close) - response["Content-Disposition"] = f'attachment; filename="{risk_acceptance.filename()}"' - mimetype, _encoding = mimetypes.guess_type(risk_acceptance.path.name) - response["Content-Type"] = mimetype or "application/octet-stream" - return response - - -""" -Greg -status: in production -Upload a threat model at the engagement level. Threat models are stored -under media folder -""" - - -def upload_threatmodel(request, eid): - eng = Engagement.objects.get(id=eid) - add_breadcrumb( - parent=eng, - title="Upload a threat model", - top_level=False, - request=request) - - if request.method == "POST": - form = UploadThreatForm(request.POST, request.FILES) - if form.is_valid(): - handle_uploaded_threat(request.FILES["file"], eng) - eng.progress = "other" - eng.threat_model = True - eng.save() - messages.add_message( - request, - messages.SUCCESS, - "Threat model saved.", - extra_tags="alert-success") - return HttpResponseRedirect( - reverse("view_engagement", args=(eid, ))) - else: - form = UploadThreatForm() - product_tab = Product_Tab(eng.product, title="Upload Threat Model", tab="engagements") - return render(request, "dojo/up_threat.html", { - "form": form, - "product_tab": product_tab, - "eng": eng, - }) - - -def view_threatmodel(request, eid): - eng = get_object_or_404(Engagement, pk=eid) - return generate_file_response_from_file_path(eng.tmodel_path) - - -def engagement_ics(request, eid): - eng = get_object_or_404(Engagement, id=eid) - start_date = datetime.combine(eng.target_start, datetime.min.time()) - end_date = datetime.combine(eng.target_end, datetime.max.time()) - if timezone.is_naive(start_date): - start_date = timezone.make_aware(start_date) - if timezone.is_naive(end_date): - end_date = timezone.make_aware(end_date) - uid = f"dojo_eng_{eng.id}_{eng.product.id}" - cal = get_cal_event( - start_date, - end_date, - f"Engagement: {eng.name} ({eng.product.name})", - ( - f"Set aside for engagement {eng.name}, on product {eng.product.name}. " - f"Additional detail can be found at {request.build_absolute_uri(reverse('view_engagement', args=(eng.id, )))}" - ), - uid, - ) - output = cal.serialize() - response = HttpResponse(content=output) - response["Content-Type"] = "text/calendar" - response["Content-Disposition"] = f"attachment; filename={eng.name}.ics" - return response - - -def get_list_index(full_list, index): - try: - element = full_list[index] - except Exception: - element = None - return element - - -def get_engagements(request): - url = request.META.get("QUERY_STRING") - if not url: - msg = "Please use the export button when exporting engagements" - raise ValidationError(msg) - url = url.removeprefix("url=") - - path_items = list(filter(None, re.split(r"/|\?", url))) - - if not path_items or path_items[0] != "engagement": - msg = "URL is not an engagement view" - raise ValidationError(msg) - - view = query = None - if get_list_index(path_items, 1) in {"active", "all"}: - view = get_list_index(path_items, 1) - query = get_list_index(path_items, 2) - else: - view = "active" - query = get_list_index(path_items, 1) - - request.GET = QueryDict(query) - return get_filtered_engagements(request, view).qs - - -def get_excludes(): - return [ - "is_ci_cd", - "jira_issue", - "jira_project", - "objects", - "unaccepted_open_findings", - "test_count", # already exported separately as “tests” - ] - - -def get_foreign_keys(): - return ["build_server", "lead", "orchestration_engine", "preset", "product", - "report_type", "requester", "source_code_management_server"] - - -def csv_export(request): - logger.debug("starting csv export") - engagements = get_engagements(request) - - response = HttpResponse(content_type="text/csv") - response["Content-Disposition"] = "attachment; filename=engagements.csv" - - writer = csv.writer(response) - - first_row = True - for engagement in engagements: - if first_row: - fields = [key for key in dir(engagement) - if key not in get_excludes() and not callable(getattr(engagement, key)) and not key.startswith("_")] - fields.append("tests") - - writer.writerow(fields) - - first_row = False - if not first_row: - fields = [] - for key in dir(engagement): - if key not in get_excludes() and not callable(getattr(engagement, key)) and not key.startswith("_"): - value = engagement.__dict__.get(key) - if key in get_foreign_keys() and getattr(engagement, key): - value = str(getattr(engagement, key)) - if value and isinstance(value, str): - value = value.replace("\n", " NEWLINE ").replace("\r", "") - fields.append(value) - fields.append(getattr(engagement, "test_count", 0)) - - writer.writerow(fields) - logger.debug("done with csv export") - return response - - -def excel_export(request): - logger.debug("starting excel export") - engagements = get_engagements(request) - - workbook = Workbook() - workbook.iso_dates = True - worksheet = workbook.active - worksheet.title = "Engagements" - - font_bold = Font(bold=True) - - row_num = 1 - for engagement in engagements: - if row_num == 1: - col_num = 1 - for key in dir(engagement): - if key not in get_excludes() and not callable(getattr(engagement, key)) and not key.startswith("_"): - cell = worksheet.cell(row=row_num, column=col_num, value=key) - cell.font = font_bold - col_num += 1 - cell = worksheet.cell(row=row_num, column=col_num, value="tests") - cell.font = font_bold - row_num = 2 - if row_num > 1: - col_num = 1 - for key in dir(engagement): - if key not in get_excludes() and not callable(getattr(engagement, key)) and not key.startswith("_"): - value = engagement.__dict__.get(key) - if key in get_foreign_keys() and getattr(engagement, key): - value = str(getattr(engagement, key)) - if value and isinstance(value, datetime): - value = value.replace(tzinfo=None) - worksheet.cell(row=row_num, column=col_num, value=value) - col_num += 1 - worksheet.cell(row=row_num, column=col_num, value=getattr(engagement, "test_count", 0)) - row_num += 1 - - with NamedTemporaryFile() as tmp: - workbook.save(tmp.name) - tmp.seek(0) - stream = tmp.read() - - response = HttpResponse( - content=stream, - content_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", - ) - response["Content-Disposition"] = "attachment; filename=engagements.xlsx" - logger.debug("done with excel export") - return response +# Backward-compat shim: the view logic moved to dojo.engagement.ui.views during the +# module reorg. External consumers (e.g. dojo-pro) still import from +# dojo.engagement.views, so re-export the public names from their new location. +from dojo.engagement.ui.views import * # noqa: F403 -- backward compat re-export diff --git a/dojo/file_uploads/__init__.py b/dojo/file_uploads/__init__.py index e69de29bb2d..4134e5e9d54 100644 --- a/dojo/file_uploads/__init__.py +++ b/dojo/file_uploads/__init__.py @@ -0,0 +1 @@ +import dojo.file_uploads.admin # noqa: F401 diff --git a/dojo/file_uploads/admin.py b/dojo/file_uploads/admin.py new file mode 100644 index 00000000000..0add1a26c56 --- /dev/null +++ b/dojo/file_uploads/admin.py @@ -0,0 +1,6 @@ +from django.contrib import admin + +from dojo.file_uploads.models import FileAccessToken, FileUpload + +admin.site.register(FileUpload) +admin.site.register(FileAccessToken) diff --git a/dojo/file_uploads/api/__init__.py b/dojo/file_uploads/api/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/file_uploads/api/serializer.py b/dojo/file_uploads/api/serializer.py new file mode 100644 index 00000000000..3f813dd7beb --- /dev/null +++ b/dojo/file_uploads/api/serializer.py @@ -0,0 +1,26 @@ +from rest_framework import serializers + +from dojo.file_uploads.models import FileUpload + + +class FileSerializer(serializers.ModelSerializer): + file = serializers.FileField(required=True) + + class Meta: + model = FileUpload + fields = "__all__" + + def validate(self, data): + if file := data.get("file"): + # the clean will validate the file extensions and raise a Validation error if the extensions are not accepted + FileUpload(title=file.name, file=file).clean() + return data + return None + + +class RawFileSerializer(serializers.ModelSerializer): + file = serializers.FileField(required=True) + + class Meta: + model = FileUpload + fields = ["file"] diff --git a/dojo/file_uploads/models.py b/dojo/file_uploads/models.py new file mode 100644 index 00000000000..e5fa8e5a66f --- /dev/null +++ b/dojo/file_uploads/models.py @@ -0,0 +1,103 @@ +from pathlib import Path +from uuid import uuid4 + +from django.conf import settings +from django.core.exceptions import ValidationError +from django.core.files.base import ContentFile +from django.db import models +from django.utils.translation import gettext_lazy as _ + +from dojo.models import ( # UniqueUploadNameProvider kept in dojo.models for migration upload_to path stability + UniqueUploadNameProvider, + copy_model_util, +) + + +class FileUpload(models.Model): + title = models.CharField(max_length=100, unique=True) + file = models.FileField(upload_to=UniqueUploadNameProvider("uploaded_files")) + + def delete(self, *args, **kwargs): + """Delete the model and remove the file from storage.""" + storage = self.file.storage + path = self.file.path + super().delete(*args, **kwargs) + if path and storage.exists(path): + storage.delete(path) + + def copy(self): + copy = copy_model_util(self) + # Add unique modifier to file name + # Truncate title to ensure it doesn't exceed max_length (100) when appending suffix + # Suffix " - clone-{8 chars}" is 17 characters, so truncate to 83 chars + clone_suffix = f" - clone-{str(uuid4())[:8]}" + max_title_length = 100 - len(clone_suffix) + truncated_title = self.title[:max_title_length] if len(self.title) > max_title_length else self.title + copy.title = f"{truncated_title}{clone_suffix}" + # Create new unique file name + current_url = self.file.url + _, current_full_filename = current_url.rsplit("/", 1) + _, extension = current_full_filename.split(".", 1) + new_file = ContentFile(self.file.read(), name=f"{uuid4()}.{extension}") + copy.file = new_file + copy.save() + + return copy + + def get_accessible_url(self, obj, obj_id): + from dojo.engagement.models import Engagement # noqa: PLC0415 -- lazy import, avoids circular dependency + from dojo.finding.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + from dojo.test.models import Test # noqa: PLC0415 -- lazy import, avoids circular dependency + if isinstance(obj, Engagement): + obj_type = "Engagement" + elif isinstance(obj, Test): + obj_type = "Test" + elif isinstance(obj, Finding): + obj_type = "Finding" + + return f"access_file/{self.id}/{obj_id}/{obj_type}" + + def clean(self): + if not self.title: + self.title = "" + + valid_extensions = settings.FILE_UPLOAD_TYPES + + # why does this not work with self.file.... + file_name = self.file.url if self.file else self.title + if Path(file_name).suffix.lower() not in valid_extensions: + if accepted_extensions := f"{', '.join(valid_extensions)}": + msg = ( + _("Unsupported extension. Supported extensions are as follows: %s") % accepted_extensions + ) + else: + msg = ( + _("File uploads are prohibited due to the list of acceptable file extensions being empty") + ) + raise ValidationError(msg) + + +class FileAccessToken(models.Model): + + """ + This will allow reports to request the images without exposing the + media root to the world without + authentication + """ + + user = models.ForeignKey("dojo.Dojo_User", null=False, blank=False, on_delete=models.CASCADE) + file = models.ForeignKey("dojo.FileUpload", null=False, blank=False, on_delete=models.CASCADE) + token = models.CharField(max_length=255) + size = models.CharField(max_length=9, + choices=( + ("small", "Small"), + ("medium", "Medium"), + ("large", "Large"), + ("thumbnail", "Thumbnail"), + ("original", "Original")), + default="medium") + + def save(self, *args, **kwargs): + if not self.token: + self.token = uuid4() + return super().save(*args, **kwargs) diff --git a/dojo/filters.py b/dojo/filters.py index 96184d92e2d..7b2e05b018b 100644 --- a/dojo/filters.py +++ b/dojo/filters.py @@ -1,27 +1,19 @@ import collections import decimal import logging -import warnings from datetime import datetime, timedelta import six import tagulous -from django import forms from django.apps import apps from django.conf import settings -from django.contrib.contenttypes.models import ContentType from django.db.models import Count, Q -from django.forms import HiddenInput from django.utils.timezone import now, tzinfo from django.utils.translation import gettext_lazy as _ from django_filters import ( - BooleanFilter, CharFilter, DateFilter, - DateFromToRangeFilter, - DateTimeFilter, FilterSet, - ModelChoiceFilter, ModelMultipleChoiceFilter, MultipleChoiceFilter, NumberFilter, @@ -30,13 +22,9 @@ ) from django_filters import rest_framework as filters from django_filters.filters import ChoiceFilter -from drf_spectacular.types import OpenApiTypes -from drf_spectacular.utils import extend_schema_field -from polymorphic.base import ManagerInheritanceWarning # from tagulous.forms import TagWidget # import tagulous -from dojo.endpoint.queries import get_authorized_endpoints_for_queryset from dojo.engagement.queries import get_authorized_engagements from dojo.finding.helper import ( ACCEPTED_FINDINGS_QUERY, @@ -50,47 +38,24 @@ VERIFIED_FINDINGS_QUERY, WAS_ACCEPTED_FINDINGS_QUERY, ) -from dojo.finding.queries import get_authorized_findings_for_queryset -from dojo.finding_group.queries import get_authorized_finding_groups_for_queryset from dojo.labels import get_labels -from dojo.location.status import FindingLocationStatus, ProductLocationStatus from dojo.models import ( - EFFORT_FOR_FIXING_CHOICES, - ENGAGEMENT_STATUS_CHOICES, - IMPORT_ACTIONS, SEVERITY_CHOICES, App_Analysis, - ChoiceQuestion, Development_Environment, - Dojo_User, DojoMeta, Endpoint, Endpoint_Status, Engagement, - Engagement_Survey, Finding, - Finding_Group, - Finding_Template, Note_Type, Product, - Product_API_Scan_Configuration, Product_Type, - Question, - Risk_Acceptance, Test, - Test_Import, - Test_Import_Finding_Action, - Test_Type, - TextQuestion, - User, Vulnerability_Id, ) -from dojo.product.queries import get_authorized_products from dojo.product_type.queries import get_authorized_product_types -from dojo.risk_acceptance.queries import get_authorized_risk_acceptances -from dojo.test.queries import get_authorized_tests -from dojo.user.queries import get_authorized_users -from dojo.utils import get_system_setting, get_visible_scan_types, is_finding_groups_enabled, truncate_timezone_aware +from dojo.utils import get_system_setting, is_finding_groups_enabled, truncate_timezone_aware logger = logging.getLogger(__name__) @@ -1000,505 +965,6 @@ def filter(self, qs, value): return self.options[value][1](self, qs, self.field_name) -class ProductComponentFilter(DojoFilter): - component_name = CharFilter(lookup_expr="icontains", label="Module Name") - component_version = CharFilter(lookup_expr="icontains", label="Module Version") - - o = OrderingFilter( - fields=( - ("component_name", "component_name"), - ("component_version", "component_version"), - ("active", "active"), - ("duplicate", "duplicate"), - ("total", "total"), - ), - field_labels={ - "component_name": "Component Name", - "component_version": "Component Version", - "active": "Active", - "duplicate": "Duplicate", - "total": "Total", - }, - ) - - -class ComponentFilterWithoutObjectLookups(ProductComponentFilter): - test__engagement__product__prod_type__name = CharFilter( - field_name="test__engagement__product__prod_type__name", - lookup_expr="iexact", - label=labels.ORG_FILTERS_NAME_LABEL, - help_text=labels.ORG_FILTERS_NAME_HELP) - test__engagement__product__prod_type__name_contains = CharFilter( - field_name="test__engagement__product__prod_type__name", - lookup_expr="icontains", - label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, - help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) - test__engagement__product__name = CharFilter( - field_name="test__engagement__product__name", - lookup_expr="iexact", - label=labels.ASSET_FILTERS_NAME_LABEL, - help_text=labels.ASSET_FILTERS_NAME_HELP) - test__engagement__product__name_contains = CharFilter( - field_name="test__engagement__product__name", - lookup_expr="icontains", - label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL, - help_text=labels.ASSET_FILTERS_NAME_CONTAINS_HELP) - - -class ComponentFilter(ProductComponentFilter): - test__engagement__product__prod_type = ModelMultipleChoiceFilter( - queryset=Product_Type.objects.none(), - label=labels.ORG_FILTERS_LABEL) - test__engagement__product = ModelMultipleChoiceFilter( - queryset=Product.objects.none(), - label=labels.ASSET_FILTERS_LABEL) - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.form.fields[ - "test__engagement__product__prod_type"].queryset = get_authorized_product_types("view") - self.form.fields[ - "test__engagement__product"].queryset = get_authorized_products("view") - - -class EngagementDirectFilterHelper(FilterSet): - name = CharFilter(lookup_expr="icontains", label="Engagement name contains") - version = CharFilter(field_name="version", lookup_expr="icontains", label="Engagement version") - test__version = CharFilter(field_name="test__version", lookup_expr="icontains", label="Test version") - product__name = CharFilter(lookup_expr="icontains", label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL) - status = MultipleChoiceFilter(choices=ENGAGEMENT_STATUS_CHOICES, label="Status") - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) - has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") - target_start = DateRangeFilter() - target_end = DateRangeFilter() - test__engagement__product__lifecycle = MultipleChoiceFilter( - choices=Product.LIFECYCLE_CHOICES, - label=labels.ASSET_LIFECYCLE_LABEL, - null_label="Empty") - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("target_start", "target_start"), - ("name", "name"), - ("product__name", "product__name"), - ("product__prod_type__name", "product__prod_type__name"), - ("lead__first_name", "lead__first_name"), - ), - field_labels={ - "target_start": "Start date", - "name": "Engagement", - "product__name": labels.ASSET_FILTERS_NAME_LABEL, - "product__prod_type__name": labels.ORG_FILTERS_LABEL, - "lead__first_name": "Lead", - }, - ) - - -class EngagementDirectFilter(EngagementDirectFilterHelper, DojoFilter): - lead = ModelChoiceFilter(queryset=Dojo_User.objects.none(), label="Lead") - product__prod_type = ModelMultipleChoiceFilter( - queryset=Product_Type.objects.none(), - label=labels.ORG_FILTERS_LABEL) - tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - queryset=Engagement.tags.tag_model.objects.all().order_by("name")) - not_tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - exclude=True, - queryset=Engagement.tags.tag_model.objects.all().order_by("name")) - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.form.fields["product__prod_type"].queryset = get_authorized_product_types("view") - self.form.fields["lead"].queryset = get_authorized_users("view") \ - .filter(engagement__lead__isnull=False).distinct() - - class Meta: - model = Engagement - fields = ["product__name", "product__prod_type"] - - -class EngagementDirectFilterWithoutObjectLookups(EngagementDirectFilterHelper): - lead = CharFilter( - field_name="lead__username", - lookup_expr="iexact", - label="Lead Username", - help_text="Search for Lead username that are an exact match") - lead_contains = CharFilter( - field_name="lead__username", - lookup_expr="icontains", - label="Lead Username Contains", - help_text="Search for Lead username that contain a given pattern") - product__prod_type__name = CharFilter( - field_name="product__prod_type__name", - lookup_expr="iexact", - label=labels.ORG_FILTERS_NAME_LABEL, - help_text=labels.ORG_FILTERS_NAME_HELP) - product__prod_type__name_contains = CharFilter( - field_name="product__prod_type__name", - lookup_expr="icontains", - label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, - help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) - - class Meta: - model = Engagement - fields = ["product__name"] - - -class EngagementFilterHelper(FilterSet): - name = CharFilter(lookup_expr="icontains", label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL) - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) - has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") - engagement__name = CharFilter(lookup_expr="icontains", label="Engagement name contains") - engagement__version = CharFilter(field_name="engagement__version", lookup_expr="icontains", label="Engagement version") - engagement__test__version = CharFilter(field_name="engagement__test__version", lookup_expr="icontains", label="Test version") - engagement__product__lifecycle = MultipleChoiceFilter( - choices=Product.LIFECYCLE_CHOICES, - label=labels.ASSET_LIFECYCLE_LABEL, - null_label="Empty") - engagement__status = MultipleChoiceFilter( - choices=ENGAGEMENT_STATUS_CHOICES, - label="Status") - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("name", "name"), - ("prod_type__name", "prod_type__name"), - ), - field_labels={ - "name": labels.ASSET_FILTERS_NAME_LABEL, - "prod_type__name": labels.ORG_FILTERS_LABEL, - }, - ) - - -class EngagementFilter(EngagementFilterHelper, DojoFilter): - engagement__lead = ModelChoiceFilter( - queryset=Dojo_User.objects.none(), - label="Lead") - prod_type = ModelMultipleChoiceFilter( - queryset=Product_Type.objects.none(), - label=labels.ORG_FILTERS_LABEL) - tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - queryset=Engagement.tags.tag_model.objects.all().order_by("name")) - not_tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - exclude=True, - queryset=Engagement.tags.tag_model.objects.all().order_by("name")) - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.form.fields["prod_type"].queryset = get_authorized_product_types("view") - self.form.fields["engagement__lead"].queryset = get_authorized_users("view") \ - .filter(engagement__lead__isnull=False).distinct() - self.form.fields["tags"].help_text = labels.ASSET_FILTERS_TAGS_HELP - self.form.fields["not_tags"].help_text = labels.ASSET_FILTERS_NOT_TAGS_HELP - - class Meta: - model = Product - fields = ["name", "prod_type"] - - -class ProductEngagementsFilter(DojoFilter): - engagement__name = CharFilter(field_name="name", lookup_expr="icontains", label="Engagement name contains") - engagement__lead = ModelChoiceFilter(field_name="lead", queryset=Dojo_User.objects.none(), label="Lead") - engagement__version = CharFilter(field_name="version", lookup_expr="icontains", label="Engagement version") - engagement__test__version = CharFilter(field_name="test__version", lookup_expr="icontains", label="Test version") - engagement__status = MultipleChoiceFilter(field_name="status", choices=ENGAGEMENT_STATUS_CHOICES, - label="Status") - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.form.fields["engagement__lead"].queryset = get_authorized_users("view") \ - .filter(engagement__lead__isnull=False).distinct() - - class Meta: - model = Engagement - fields = [] - - -class ProductEngagementsFilterWithoutObjectLookups(ProductEngagementsFilter): - engagement__lead = CharFilter( - field_name="lead__username", - lookup_expr="iexact", - label="Lead Username", - help_text="Search for Lead username that are an exact match") - - -class EngagementFilterWithoutObjectLookups(EngagementFilterHelper): - engagement__lead = CharFilter( - field_name="engagement__lead__username", - lookup_expr="iexact", - label="Lead Username", - help_text="Search for Lead username that are an exact match") - engagement__lead_contains = CharFilter( - field_name="engagement__lead__username", - lookup_expr="icontains", - label="Lead Username Contains", - help_text="Search for Lead username that contain a given pattern") - prod_type__name = CharFilter( - field_name="prod_type__name", - lookup_expr="iexact", - label=labels.ORG_FILTERS_LABEL, - help_text=labels.ORG_FILTERS_LABEL_HELP) - prod_type__name_contains = CharFilter( - field_name="prod_type__name", - lookup_expr="icontains", - label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, - help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) - - class Meta: - model = Product - fields = ["name"] - - -class ProductEngagementFilterHelper(FilterSet): - version = CharFilter(lookup_expr="icontains", label="Engagement version") - test__version = CharFilter(field_name="test__version", lookup_expr="icontains", label="Test version") - name = CharFilter(lookup_expr="icontains") - status = MultipleChoiceFilter(choices=ENGAGEMENT_STATUS_CHOICES, label="Status") - target_start = DateRangeFilter() - target_end = DateRangeFilter() - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("name", "name"), - ("version", "version"), - ("target_start", "target_start"), - ("target_end", "target_end"), - ("status", "status"), - ("lead", "lead"), - ), - field_labels={ - "name": "Engagement Name", - }, - ) - - class Meta: - model = Product - fields = ["name"] - - -class ProductEngagementFilter(ProductEngagementFilterHelper, DojoFilter): - lead = ModelChoiceFilter(queryset=Dojo_User.objects.none(), label="Lead") - tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - queryset=Engagement.tags.tag_model.objects.all().order_by("name")) - not_tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - exclude=True, - queryset=Engagement.tags.tag_model.objects.all().order_by("name")) - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.form.fields["lead"].queryset = get_authorized_users( - "view").filter(engagement__lead__isnull=False).distinct() - - -class ProductEngagementFilterWithoutObjectLookups(ProductEngagementFilterHelper, DojoFilter): - lead = CharFilter( - field_name="lead__username", - lookup_expr="iexact", - label="Lead Username", - help_text="Search for Lead username that are an exact match") - lead_contains = CharFilter( - field_name="lead__username", - lookup_expr="icontains", - label="Lead Username Contains", - help_text="Search for Lead username that contain a given pattern") - - -class ApiEngagementFilter(DojoFilter): - product__prod_type = NumberInFilter(field_name="product__prod_type", lookup_expr="in") - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Tag name contains") - tags = CharFieldInFilter( - field_name="tags__name", - lookup_expr="in", - help_text="Comma separated list of exact tags (uses OR for multiple values)") - tags__and = CharFieldFilterANDExpression( - field_name="tags__name", - help_text="Comma separated list of exact tags to match with an AND expression") - product__tags = CharFieldInFilter( - field_name="product__tags__name", - lookup_expr="in", - help_text=labels.ASSET_FILTERS_CSV_TAGS_OR_HELP) - product__tags__and = CharFieldFilterANDExpression( - field_name="product__tags__name", - help_text=labels.ASSET_FILTERS_CSV_TAGS_AND_HELP) - - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Not Tag name contains", exclude="True") - not_tags = CharFieldInFilter(field_name="tags__name", lookup_expr="in", - help_text="Comma separated list of exact tags not present on model", exclude="True") - not_product__tags = CharFieldInFilter(field_name="product__tags__name", - lookup_expr="in", - help_text=labels.ASSET_FILTERS_CSV_TAGS_NOT_HELP, - exclude="True") - has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("name", "name"), - ("version", "version"), - ("target_start", "target_start"), - ("target_end", "target_end"), - ("status", "status"), - ("lead", "lead"), - ("created", "created"), - ("updated", "updated"), - ), - field_labels={ - "name": "Engagement Name", - }, - - ) - - class Meta: - model = Engagement - fields = ["id", "active", "target_start", - "target_end", "requester", "report_type", - "updated", "threat_model", "api_test", - "pen_test", "status", "product", "name", "version", "tags"] - - -class ProductFilterHelper(FilterSet): - name = CharFilter(lookup_expr="icontains", label=labels.ASSET_FILTERS_NAME_LABEL) - name_exact = CharFilter(field_name="name", lookup_expr="iexact", label=labels.ASSET_FILTERS_NAME_EXACT_LABEL) - business_criticality = MultipleChoiceFilter(choices=Product.BUSINESS_CRITICALITY_CHOICES, null_label="Empty") - platform = MultipleChoiceFilter(choices=Product.PLATFORM_CHOICES, null_label="Empty") - lifecycle = MultipleChoiceFilter(choices=Product.LIFECYCLE_CHOICES, null_label="Empty") - origin = MultipleChoiceFilter(choices=Product.ORIGIN_CHOICES, null_label="Empty") - external_audience = BooleanFilter(field_name="external_audience") - internet_accessible = BooleanFilter(field_name="internet_accessible") - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag contains") - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) - outside_of_sla = ProductSLAFilter(label="Outside of SLA") - has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") - if settings.V3_FEATURE_LOCATIONS: - location_status = MultipleChoiceFilter( - field_name="locations__status", - choices=ProductLocationStatus.choices, - help_text="Status of the Location from the Products relationship", - ) - endpoints__host = CharFilter( - field_name="locations__location__url__host", method="filter_endpoints_host", label="Endpoint Host", - ) - endpoints = NumberFilter(field_name="locations__location", method="filter_endpoints", widget=HiddenInput()) - - def filter_endpoints_host(self, queryset, name, value): - return filter_endpoints_host_base( - queryset, - name, - value, - endpoint_id=self.data.get("endpoints"), - statuses=self.data.getlist("location_status"), - ) - - def filter_endpoints(self, queryset, name, value): - return filter_endpoints_base( - queryset, - name, - value, - statuses=self.data.getlist("location_status"), - host=self.data.get("endpoints__host"), - ) - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("name", "name"), - ("name_exact", "name_exact"), - ("prod_type__name", "prod_type__name"), - ("business_criticality", "business_criticality"), - ("platform", "platform"), - ("lifecycle", "lifecycle"), - ("origin", "origin"), - ("external_audience", "external_audience"), - ("internet_accessible", "internet_accessible"), - ("findings_count", "findings_count"), - ), - field_labels={ - "name": labels.ASSET_FILTERS_NAME_LABEL, - "name_exact": labels.ASSET_FILTERS_NAME_EXACT_LABEL, - "prod_type__name": labels.ORG_FILTERS_LABEL, - "business_criticality": "Business Criticality", - "platform": "Platform ", - "lifecycle": "Lifecycle ", - "origin": "Origin ", - "external_audience": "External Audience ", - "internet_accessible": "Internet Accessible ", - "findings_count": "Findings Count ", - }, - ) - - -class ProductFilter(ProductFilterHelper, DojoFilter): - prod_type = ModelMultipleChoiceFilter( - queryset=Product_Type.objects.none(), - label=labels.ORG_FILTERS_LABEL) - tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - queryset=Product.tags.tag_model.objects.all().order_by("name")) - not_tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - exclude=True, - queryset=Product.tags.tag_model.objects.all().order_by("name")) - - def __init__(self, *args, **kwargs): - self.user = None - if "user" in kwargs: - self.user = kwargs.pop("user") - super().__init__(*args, **kwargs) - self.form.fields["prod_type"].queryset = get_authorized_product_types("view") - self.form.fields["tags"].help_text = labels.ASSET_FILTERS_TAGS_HELP - self.form.fields["not_tags"].help_text = labels.ASSET_FILTERS_NOT_TAGS_HELP - - class Meta: - model = Product - fields = [ - "name", "name_exact", "prod_type", "business_criticality", - "platform", "lifecycle", "origin", "external_audience", - "internet_accessible", "tags", - ] - - -class ProductFilterWithoutObjectLookups(ProductFilterHelper): - prod_type__name = CharFilter( - field_name="prod_type__name", - lookup_expr="iexact", - label=labels.ORG_FILTERS_NAME_LABEL, - help_text=labels.ORG_FILTERS_NAME_HELP) - prod_type__name_contains = CharFilter( - field_name="prod_type__name", - lookup_expr="icontains", - label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, - help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) - - def __init__(self, *args, **kwargs): - kwargs.pop("user", None) - super().__init__(*args, **kwargs) - - class Meta: - model = Product - fields = [ - "name", "name_exact", "business_criticality", "platform", - "lifecycle", "origin", "external_audience", "internet_accessible", - ] - - class ApiDojoMetaFilter(DojoFilter): name_case_insensitive = CharFilter(field_name="name", lookup_expr="iexact") value_case_insensitive = CharFilter(field_name="value", lookup_expr="iexact") @@ -1523,81 +989,6 @@ class Meta: ] -class ApiProductFilter(DojoFilter): - # BooleanFilter - external_audience = BooleanFilter(field_name="external_audience") - internet_accessible = BooleanFilter(field_name="internet_accessible") - # CharFilter - name = CharFilter(lookup_expr="icontains") - name_exact = CharFilter(field_name="name", lookup_expr="iexact") - description = CharFilter(lookup_expr="icontains") - business_criticality = MultipleChoiceFilter(choices=Product.BUSINESS_CRITICALITY_CHOICES) - platform = MultipleChoiceFilter(choices=Product.PLATFORM_CHOICES) - lifecycle = MultipleChoiceFilter(choices=Product.LIFECYCLE_CHOICES) - origin = MultipleChoiceFilter(choices=Product.ORIGIN_CHOICES) - # NumberInFilter - id = NumberInFilter(field_name="id", lookup_expr="in") - product_manager = NumberInFilter(field_name="product_manager", lookup_expr="in") - technical_contact = NumberInFilter(field_name="technical_contact", lookup_expr="in") - team_manager = NumberInFilter(field_name="team_manager", lookup_expr="in") - prod_type = NumberInFilter(field_name="prod_type", lookup_expr="in") - tid = NumberInFilter(field_name="tid", lookup_expr="in") - prod_numeric_grade = NumberInFilter(field_name="prod_numeric_grade", lookup_expr="in") - user_records = NumberInFilter(field_name="user_records", lookup_expr="in") - regulations = NumberInFilter(field_name="regulations", lookup_expr="in") - - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") - tags = CharFieldInFilter( - field_name="tags__name", - lookup_expr="in", - help_text="Comma separated list of exact tags (uses OR for multiple values)") - tags__and = CharFieldFilterANDExpression( - field_name="tags__name", - help_text="Comma separated list of exact tags to match with an AND expression") - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Not Tag name contains", exclude="True") - not_tags = CharFieldInFilter(field_name="tags__name", lookup_expr="in", - help_text=labels.ASSET_FILTERS_CSV_TAGS_NOT_HELP, exclude="True") - has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") - outside_of_sla = extend_schema_field(OpenApiTypes.NUMBER)(ProductSLAFilter()) - - # DateRangeFilter - created = DateRangeFilter() - updated = DateRangeFilter() - # NumberFilter - revenue = NumberFilter() - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("id", "id"), - ("tid", "tid"), - ("name", "name"), - ("created", "created"), - ("prod_numeric_grade", "prod_numeric_grade"), - ("business_criticality", "business_criticality"), - ("platform", "platform"), - ("lifecycle", "lifecycle"), - ("origin", "origin"), - ("revenue", "revenue"), - ("external_audience", "external_audience"), - ("internet_accessible", "internet_accessible"), - ("product_manager", "product_manager"), - ("product_manager__first_name", "product_manager__first_name"), - ("product_manager__last_name", "product_manager__last_name"), - ("technical_contact", "technical_contact"), - ("technical_contact__first_name", "technical_contact__first_name"), - ("technical_contact__last_name", "technical_contact__last_name"), - ("team_manager", "team_manager"), - ("team_manager__first_name", "team_manager__first_name"), - ("team_manager__last_name", "team_manager__last_name"), - ("prod_type", "prod_type"), - ("prod_type__name", "prod_type__name"), - ("updated", "updated"), - ("user_records", "user_records"), - ), - ) - - class PercentageRangeFilter(RangeFilter): def filter(self, qs, value): if value is not None: @@ -1607,1750 +998,278 @@ def filter(self, qs, value): return super().filter(qs, value) -class ApiFindingFilter(DojoFilter): - # BooleanFilter - active = BooleanFilter(field_name="active") - duplicate = BooleanFilter(field_name="duplicate") - dynamic_finding = BooleanFilter(field_name="dynamic_finding") - false_p = BooleanFilter(field_name="false_p") - is_mitigated = BooleanFilter(field_name="is_mitigated") - out_of_scope = BooleanFilter(field_name="out_of_scope") - static_finding = BooleanFilter(field_name="static_finding") - under_defect_review = BooleanFilter(field_name="under_defect_review") - under_review = BooleanFilter(field_name="under_review") - verified = BooleanFilter(field_name="verified") - has_jira = BooleanFilter(field_name="jira_issue", lookup_expr="isnull", exclude=True) - fix_available = BooleanFilter(field_name="fix_available") - mitigation_available = BooleanFilter(method="filter_mitigation_available", label="Mitigation Available") - # CharFilter - component_version = CharFilter(lookup_expr="icontains") - component_name = CharFilter(lookup_expr="icontains") - vulnerability_id = CharFilter(method=custom_vulnerability_id_filter) - description = CharFilter(lookup_expr="icontains") - file_path = CharFilter(lookup_expr="icontains") - hash_code = CharFilter(lookup_expr="icontains") - impact = CharFilter(lookup_expr="icontains") - mitigation = CharFilter(lookup_expr="icontains") - numerical_severity = CharFilter(method=custom_filter, field_name="numerical_severity") - param = CharFilter(lookup_expr="icontains") - payload = CharFilter(lookup_expr="icontains") - references = CharFilter(lookup_expr="icontains") - severity = CharFilter(method=custom_filter, field_name="severity") - severity_justification = CharFilter(lookup_expr="icontains") - steps_to_reproduce = CharFilter(lookup_expr="icontains") - unique_id_from_tool = CharFilter(lookup_expr="icontains") - title = CharFilter(lookup_expr="icontains") - exact_title = CharFilter(field_name="title", lookup_expr="iexact", help_text="Finding title exact match (case-insensitive)") - product_name = CharFilter(lookup_expr="engagement__product__name__iexact", field_name="test", label=labels.ASSET_FILTERS_NAME_EXACT_LABEL) - product_name_contains = CharFilter(lookup_expr="engagement__product__name__icontains", field_name="test", label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL) - product_lifecycle = CharFilter(method=custom_filter, lookup_expr="engagement__product__lifecycle", - field_name="test__engagement__product__lifecycle", label=labels.ASSET_FILTERS_CSV_LIFECYCLES_LABEL) - # DateRangeFilter - created = DateRangeFilter() - date = DateRangeFilter() - discovered_on = DateFilter(field_name="date", lookup_expr="exact") - discovered_before = DateFilter(field_name="date", lookup_expr="lt") - discovered_after = DateFilter(field_name="date", lookup_expr="gt") - jira_creation = DateRangeFilter(field_name="jira_issue__jira_creation") - jira_change = DateRangeFilter(field_name="jira_issue__jira_change") - last_reviewed = DateRangeFilter() - mitigated = DateRangeFilter() - mitigated_on = DateTimeFilter(field_name="mitigated", lookup_expr="exact", method="filter_mitigated_on") - mitigated_before = DateTimeFilter(field_name="mitigated", lookup_expr="lt") - mitigated_after = DateTimeFilter(field_name="mitigated", lookup_expr="gt", label="Mitigated After", method="filter_mitigated_after") - # NumberInFilter - cwe = NumberInFilter(field_name="cwe", lookup_expr="in") - defect_review_requested_by = NumberInFilter(field_name="defect_review_requested_by", lookup_expr="in") - endpoints = NumberInFilter(field_name="endpoints", lookup_expr="in") - epss_score = PercentageRangeFilter( - field_name="epss_score", - label="EPSS score range", - help_text=( - "The range of EPSS score percentages to filter on; the min input is a lower bound, " - "the max is an upper bound. Leaving one empty will skip that bound (e.g., leaving " - "the min bound input empty will filter only on the max bound -- filtering on " - '"less than or equal"). Leading 0 required.' - )) - epss_percentile = PercentageRangeFilter( - field_name="epss_percentile", - label="EPSS percentile range", - help_text=( - "The range of EPSS percentiles to filter on; the min input is a lower bound, the max " - "is an upper bound. Leaving one empty will skip that bound (e.g., leaving the min bound " - 'input empty will filter only on the max bound -- filtering on "less than or equal"). Leading 0 required.' - )) - found_by = NumberInFilter(field_name="found_by", lookup_expr="in") - id = NumberInFilter(field_name="id", lookup_expr="in") - last_reviewed_by = NumberInFilter(field_name="last_reviewed_by", lookup_expr="in") - mitigated_by = NumberInFilter(field_name="mitigated_by", lookup_expr="in") - nb_occurences = NumberInFilter(field_name="nb_occurences", lookup_expr="in") - reporter = NumberInFilter(field_name="reporter", lookup_expr="in") - scanner_confidence = NumberInFilter(field_name="scanner_confidence", lookup_expr="in") - review_requested_by = NumberInFilter(field_name="review_requested_by", lookup_expr="in") - reviewers = NumberInFilter(field_name="reviewers", lookup_expr="in") - sast_source_line = NumberInFilter(field_name="sast_source_line", lookup_expr="in") - sonarqube_issue = NumberInFilter(field_name="sonarqube_issue", lookup_expr="in") - test__test_type = NumberInFilter(field_name="test__test_type", lookup_expr="in", label="Test Type") - test__engagement = NumberInFilter(field_name="test__engagement", lookup_expr="in") - test__engagement__product = NumberInFilter(field_name="test__engagement__product", lookup_expr="in") - test__engagement__product__prod_type = NumberInFilter(field_name="test__engagement__product__prod_type", lookup_expr="in") - finding_group = NumberInFilter(field_name="finding_group", lookup_expr="in") - - # ReportRiskAcceptanceFilter - risk_acceptance = extend_schema_field(OpenApiTypes.NUMBER)(ReportRiskAcceptanceFilter()) - - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Tag name contains") - tags = CharFieldInFilter( - field_name="tags__name", - lookup_expr="in", - help_text="Comma separated list of exact tags (uses OR for multiple values)") - tags__and = CharFieldFilterANDExpression( - field_name="tags__name", - help_text="Comma separated list of exact tags to match with an AND expression") - test__tags = CharFieldInFilter( - field_name="test__tags__name", - lookup_expr="in", - help_text="Comma separated list of exact tags present on test (uses OR for multiple values)") - test__tags__and = CharFieldFilterANDExpression( - field_name="test__tags__name", - help_text="Comma separated list of exact tags to match with an AND expression present on test") - test__engagement__tags = CharFieldInFilter( - field_name="test__engagement__tags__name", - lookup_expr="in", - help_text="Comma separated list of exact tags present on engagement (uses OR for multiple values)") - test__engagement__tags__and = CharFieldFilterANDExpression( - field_name="test__engagement__tags__name", - help_text="Comma separated list of exact tags to match with an AND expression present on engagement") - test__engagement__product__tags = CharFieldInFilter( - field_name="test__engagement__product__tags__name", - lookup_expr="in", - help_text=labels.ASSET_FILTERS_CSV_TAGS_OR_HELP) - test__engagement__product__tags__and = CharFieldFilterANDExpression( - field_name="test__engagement__product__tags__name", - help_text=labels.ASSET_FILTERS_CSV_TAGS_AND_HELP) - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Not Tag name contains", exclude="True") - not_tags = CharFieldInFilter(field_name="tags__name", lookup_expr="in", - help_text="Comma separated list of exact tags not present on model", exclude="True") - not_test__tags = CharFieldInFilter(field_name="test__tags__name", lookup_expr="in", exclude="True", help_text="Comma separated list of exact tags present on test") - not_test__engagement__tags = CharFieldInFilter(field_name="test__engagement__tags__name", lookup_expr="in", - help_text="Comma separated list of exact tags not present on engagement", - exclude="True") - not_test__engagement__product__tags = CharFieldInFilter( - field_name="test__engagement__product__tags__name", - lookup_expr="in", - help_text=labels.ASSET_FILTERS_CSV_TAGS_NOT_HELP, - exclude="True") - has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") - outside_of_sla = extend_schema_field(OpenApiTypes.NUMBER)(FindingSLAFilter()) - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("active", "active"), - ("component_name", "component_name"), - ("component_version", "component_version"), - ("created", "created"), - ("last_status_update", "last_status_update"), - ("last_reviewed", "last_reviewed"), - ("cwe", "cwe"), - ("date", "date"), - ("duplicate", "duplicate"), - ("dynamic_finding", "dynamic_finding"), - ("false_p", "false_p"), - ("found_by", "found_by"), - ("id", "id"), - ("is_mitigated", "is_mitigated"), - ("numerical_severity", "numerical_severity"), - ("out_of_scope", "out_of_scope"), - ("planned_remediation_date", "planned_remediation_date"), - ("severity", "severity"), - ("sla_expiration_date", "sla_expiration_date"), - ("reviewers", "reviewers"), - ("static_finding", "static_finding"), - ("test__engagement__product__name", "test__engagement__product__name"), - ("title", "title"), - ("under_defect_review", "under_defect_review"), - ("under_review", "under_review"), - ("verified", "verified"), - ), - ) - - class Meta: - model = Finding - exclude = ["url", "thread_id", "notes", "files", - "line", "cve"] - - def filter_mitigated_after(self, queryset, name, value): - if value.hour == 0 and value.minute == 0 and value.second == 0: - value = value.replace(hour=23, minute=59, second=59) - - return queryset.filter(mitigated__gt=value) - - def filter_mitigated_on(self, queryset, name, value): - if value.hour == 0 and value.minute == 0 and value.second == 0: - # we have a simple date without a time, lets get a range from this morning to tonight at 23:59:59:999 - nextday = value + timedelta(days=1) - return queryset.filter(mitigated__gte=value, mitigated__lt=nextday) - - return queryset.filter(mitigated=value) - - def filter_mitigation_available(self, queryset, name, value): - if value: - return queryset.exclude(mitigation__isnull=True).exclude(mitigation__exact="") - return queryset.filter(Q(mitigation__isnull=True) | Q(mitigation__exact="")) - - class PercentageFilter(NumberFilter): def __init__(self, *args, **kwargs): kwargs["method"] = self.filter_percentage - super().__init__(*args, **kwargs) - - def filter_percentage(self, queryset, name, value): - value /= decimal.Decimal("100.0") - # Provide some wiggle room for filtering since the UI rounds to two places (and because floats): - # a user may enter 0.15, but we'll return everything in [0.0015, 0.0016). - # To do this, add to our value 1^(whatever the exponent for our least significant digit place is), but ensure - # that the exponent is at MOST the ten thousandths place so we don't show a range of e.g. [0.2, 0.3). - exponent = min(value.normalize().as_tuple().exponent, -4) - max_val = value + decimal.Decimal(f"1E{exponent}") - lookup_kwargs = { - f"{name}__gte": value, - f"{name}__lt": max_val} - return queryset.filter(**lookup_kwargs) - - -class FindingFilterHelper(FilterSet): - title = CharFilter(lookup_expr="icontains") - date = DateRangeFilter(field_name="date", label="Date Discovered") - on = DateFilter(field_name="date", lookup_expr="exact", label="Discovered On") - before = DateFilter(field_name="date", lookup_expr="lt", label="Discovered Before") - after = DateFilter(field_name="date", lookup_expr="gt", label="Discovered After") - last_reviewed = DateRangeFilter() - last_status_update = DateRangeFilter() - cwe = MultipleChoiceFilter(choices=[]) - vulnerability_id = CharFilter(method=vulnerability_id_filter, label="Vulnerability Id") - severity = MultipleChoiceFilter(choices=SEVERITY_CHOICES) - duplicate = ReportBooleanFilter() - is_mitigated = ReportBooleanFilter() - fix_available = ReportBooleanFilter() - mitigation = CharFilter(lookup_expr="icontains") - mitigation_available = BooleanFilter(method="filter_mitigation_available", label="Mitigation Available") - mitigated = DateRangeFilter(field_name="mitigated", label="Mitigated Date") - mitigated_on = DateTimeFilter(field_name="mitigated", lookup_expr="exact", label="Mitigated On", method="filter_mitigated_on") - mitigated_before = DateTimeFilter(field_name="mitigated", lookup_expr="lt", label="Mitigated Before") - mitigated_after = DateTimeFilter(field_name="mitigated", lookup_expr="gt", label="Mitigated After", method="filter_mitigated_after") - planned_remediation_date = DateRangeOmniFilter() - planned_remediation_version = CharFilter(lookup_expr="icontains", label=_("Planned remediation version")) - file_path = CharFilter(lookup_expr="icontains") - param = CharFilter(lookup_expr="icontains") - payload = CharFilter(lookup_expr="icontains") - test__test_type = ModelMultipleChoiceFilter(queryset=Test_Type.objects.all(), label="Test Type") - service = CharFilter(lookup_expr="icontains") - test__engagement__version = CharFilter(lookup_expr="icontains", label="Engagement Version") - test__version = CharFilter(lookup_expr="icontains", label="Test Version") - risk_acceptance = ReportRiskAcceptanceFilter(label="Risk Accepted") - effort_for_fixing = MultipleChoiceFilter(choices=EFFORT_FOR_FIXING_CHOICES) - test_import_finding_action__test_import = NumberFilter(widget=HiddenInput()) - status = FindingStatusFilter(label="Status") - test__engagement__product__lifecycle = MultipleChoiceFilter( - choices=Product.LIFECYCLE_CHOICES, - label=labels.ASSET_LIFECYCLE_LABEL) - if settings.V3_FEATURE_LOCATIONS: - location_status = MultipleChoiceFilter( - field_name="locations__status", - choices=FindingLocationStatus.choices, - help_text="Status of the Location from the Findings relationship", - ) - endpoints__host = CharFilter( - field_name="locations__location__url__host", method="filter_endpoints_host", label="Endpoint Host", - ) - endpoints = NumberFilter(field_name="locations__location", method="filter_endpoints", widget=HiddenInput()) - - def filter_endpoints_host(self, queryset, name, value): - return filter_endpoints_host_base( - queryset, - name, - value, - endpoint_id=self.data.get("endpoints"), - statuses=self.data.getlist("location_status"), - ) - - def filter_endpoints(self, queryset, name, value): - return filter_endpoints_base( - queryset, - name, - value, - statuses=self.data.getlist("location_status"), - host=self.data.get("endpoints__host"), - ) - else: - # TODO: Delete this after the move to Locations - endpoints__host = CharFilter(lookup_expr="icontains", label="Endpoint Host") - endpoints = NumberFilter(widget=HiddenInput()) - - has_component = BooleanFilter( - field_name="component_name", - lookup_expr="isnull", - exclude=True, - label="Has Component") - has_notes = BooleanFilter( - field_name="notes", - lookup_expr="isnull", - exclude=True, - label="Has notes") - - if is_finding_groups_enabled(): - has_finding_group = BooleanFilter( - field_name="finding_group", - lookup_expr="isnull", - exclude=True, - label="Is Grouped") - - if get_system_setting("enable_jira"): - has_jira_issue = BooleanFilter( - field_name="jira_issue", - lookup_expr="isnull", - exclude=True, - label="Has JIRA") - jira_creation = DateRangeFilter(field_name="jira_issue__jira_creation", label="JIRA Creation") - jira_change = DateRangeFilter(field_name="jira_issue__jira_change", label="JIRA Updated") - jira_issue__jira_key = CharFilter(field_name="jira_issue__jira_key", lookup_expr="icontains", label="JIRA issue") - - if is_finding_groups_enabled(): - has_jira_group_issue = BooleanFilter( - field_name="finding_group__jira_issue", - lookup_expr="isnull", - exclude=True, - label="Has Group JIRA") - has_any_jira_issue = FindingHasJIRAFilter( - label="Has Any JIRA Issue", - help_text="Matches JIRA issues linked to the finding itself or to the finding's group.", - ) - - outside_of_sla = FindingSLAFilter(label="Outside of SLA") - has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") - epss_score = PercentageFilter(field_name="epss_score", label="EPSS score") - epss_score_range = PercentageRangeFilter( - field_name="epss_score", - label="EPSS score range", - help_text=( - "The range of EPSS score percentages to filter on; the left input is a lower bound, " - "the right is an upper bound. Leaving one empty will skip that bound (e.g., leaving " - "the lower bound input empty will filter only on the upper bound -- filtering on " - '"less than or equal").' - )) - epss_percentile = PercentageFilter(field_name="epss_percentile", label="EPSS percentile") - epss_percentile_range = PercentageRangeFilter( - field_name="epss_percentile", - label="EPSS percentile range", - help_text=( - "The range of EPSS percentiles to filter on; the left input is a lower bound, the right " - "is an upper bound. Leaving one empty will skip that bound (e.g., leaving the lower bound " - 'input empty will filter only on the upper bound -- filtering on "less than or equal").' - )) - kev_date = DateFilter(field_name="kev_date", lookup_expr="exact", label="Added to KEV On") - kev_before = DateFilter(field_name="kev_date", lookup_expr="lt", label="Added to KEV Before") - kev_after = DateFilter(field_name="kev_date", lookup_expr="gt", label="Added to KEV After") - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("numerical_severity", "numerical_severity"), - ("date", "date"), - ("mitigated", "mitigated"), - ("fix_available", "fix_available"), - ("risk_acceptance__created__date", - "risk_acceptance__created__date"), - ("last_reviewed", "last_reviewed"), - ("planned_remediation_date", "planned_remediation_date"), - ("planned_remediation_version", "planned_remediation_version"), - ("title", "title"), - ("test__engagement__product__name", - "test__engagement__product__name"), - ("service", "service"), - ("sla_age_days", "sla_age_days"), - ("epss_score", "epss_score"), - ("epss_percentile", "epss_percentile"), - ("known_exploited", "known_exploited"), - ("ransomware_used", "ransomware_used"), - ("kev_date", "kev_date"), - ), - field_labels={ - "numerical_severity": "Severity", - "date": "Date", - "risk_acceptance__created__date": "Acceptance Date", - "mitigated": "Mitigated Date", - "fix_available": "Fix Available", - "title": "Finding Name", - "test__engagement__product__name": labels.ASSET_FILTERS_NAME_LABEL, - "epss_score": "EPSS Score", - "epss_percentile": "EPSS Percentile", - "known_exploited": "Known Exploited", - "ransomware_used": "Ransomware Used", - "kev_date": "Date added to KEV", - "sla_age_days": "SLA age (days)", - "planned_remediation_date": "Planned Remediation", - "planned_remediation_version": "Planned remediation version", - }, - ) - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - if "test__test_type" in self.form.fields: - self.form.fields["test__test_type"].queryset = get_visible_scan_types() - - def set_date_fields(self, *args: list, **kwargs: dict): - date_input_widget = forms.DateInput(attrs={"class": "datepicker", "placeholder": "YYYY-MM-DD"}, format="%Y-%m-%d") - self.form.fields["on"].widget = date_input_widget - self.form.fields["before"].widget = date_input_widget - self.form.fields["after"].widget = date_input_widget - self.form.fields["kev_date"].widget = date_input_widget - self.form.fields["kev_before"].widget = date_input_widget - self.form.fields["kev_after"].widget = date_input_widget - self.form.fields["mitigated_on"].widget = date_input_widget - self.form.fields["mitigated_before"].widget = date_input_widget - self.form.fields["mitigated_after"].widget = date_input_widget - self.form.fields["cwe"].choices = cwe_options(self.queryset) - - def filter_mitigated_after(self, queryset, name, value): - if value.hour == 0 and value.minute == 0 and value.second == 0: - value = value.replace(hour=23, minute=59, second=59) - - return queryset.filter(mitigated__gt=value) - - def filter_mitigated_on(self, queryset, name, value): - if value.hour == 0 and value.minute == 0 and value.second == 0: - # we have a simple date without a time, lets get a range from this morning to tonight at 23:59:59:999 - nextday = value + timedelta(days=1) - return queryset.filter(mitigated__gte=value, mitigated__lt=nextday) - - return queryset.filter(mitigated=value) - - def filter_mitigation_available(self, queryset, name, value): - if value: - return queryset.exclude(mitigation__isnull=True).exclude(mitigation__exact="") - return queryset.filter(Q(mitigation__isnull=True) | Q(mitigation__exact="")) - - -def get_finding_group_queryset_for_context(pid=None, eid=None, tid=None): - """ - Helper function to build finding group queryset based on context hierarchy. - Context priority: test > engagement > product > global - - Args: - pid: Product ID (least specific) - eid: Engagement ID - tid: Test ID (most specific) - - Returns: - QuerySet of Finding_Group filtered by context - - """ - if tid is not None: - # Most specific: filter by test - return Finding_Group.objects.filter(test_id=tid).only("id", "name") - if eid is not None: - # Filter by engagement's tests - return Finding_Group.objects.filter(test__engagement_id=eid).only("id", "name") - if pid is not None: - # Filter by product's tests - return Finding_Group.objects.filter(test__engagement__product_id=pid).only("id", "name") - # Global: return all (authorization will be applied separately) - return Finding_Group.objects.all().only("id", "name") - - -class FindingFilterWithoutObjectLookups(FindingFilterHelper, FindingTagStringFilter): - test__engagement__product__prod_type = NumberFilter(widget=HiddenInput()) - test__engagement__product = NumberFilter(widget=HiddenInput()) - reporter = CharFilter( - field_name="reporter__username", - lookup_expr="iexact", - label="Reporter Username", - help_text="Search for Reporter names that are an exact match") - reporter_contains = CharFilter( - field_name="reporter__username", - lookup_expr="icontains", - label="Reporter Username Contains", - help_text="Search for Reporter names that contain a given pattern") - reviewers = CharFilter( - field_name="reviewers__username", - lookup_expr="iexact", - label="Reviewer Username", - help_text="Search for Reviewer names that are an exact match") - reviewers_contains = CharFilter( - field_name="reviewers__username", - lookup_expr="icontains", - label="Reviewer Username Contains", - help_text="Search for Reviewer usernames that contain a given pattern") - test__engagement__product__prod_type__name = CharFilter( - field_name="test__engagement__product__prod_type__name", - lookup_expr="iexact", - label=labels.ORG_FILTERS_NAME_LABEL, - help_text=labels.ORG_FILTERS_NAME_HELP) - test__engagement__product__prod_type__name_contains = CharFilter( - field_name="test__engagement__product__prod_type__name", - lookup_expr="icontains", - label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, - help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) - test__engagement__product__name = CharFilter( - field_name="test__engagement__product__name", - lookup_expr="iexact", - label=labels.ASSET_FILTERS_NAME_LABEL, - help_text=labels.ASSET_FILTERS_NAME_HELP) - test__engagement__product__name_contains = CharFilter( - field_name="test__engagement__product__name", - lookup_expr="icontains", - label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL, - help_text=labels.ASSET_FILTERS_NAME_CONTAINS_HELP) - test__engagement__name = CharFilter( - field_name="test__engagement__name", - lookup_expr="iexact", - label="Engagement Name", - help_text="Search for Engagement names that are an exact match") - test__engagement__name_contains = CharFilter( - field_name="test__engagement__name", - lookup_expr="icontains", - label="Engagement name Contains", - help_text="Search for Engagement names that contain a given pattern") - test__name = CharFilter( - field_name="test__title", - lookup_expr="iexact", - label="Test Name", - help_text="Search for Test names that are an exact match") - test__name_contains = CharFilter( - field_name="test__title", - lookup_expr="icontains", - label="Test name Contains", - help_text="Search for Test names that contain a given pattern") - - if is_finding_groups_enabled(): - finding_group__name = CharFilter( - field_name="finding_group__name", - lookup_expr="iexact", - label="Finding Group Name", - help_text="Search for Finding Group names that are an exact match") - finding_group__name_contains = CharFilter( - field_name="finding_group__name", - lookup_expr="icontains", - label="Finding Group Name Contains", - help_text="Search for Finding Group names that contain a given pattern") - - class Meta: - model = Finding - fields = get_finding_filterset_fields(filter_string_matching=True) - - exclude = ["url", "description", "mitigation", "impact", - "endpoints", "references", - "thread_id", "notes", "scanner_confidence", - "numerical_severity", "line", "duplicate_finding", - "hash_code", "reviewers", "created", "files", - "sla_start_date", "sla_expiration_date", "cvssv3", - "severity_justification", "steps_to_reproduce"] - - def __init__(self, *args, **kwargs): - self.user = None - self.pid = None - self.eid = None - self.tid = None - if "user" in kwargs: - self.user = kwargs.pop("user") - - if "pid" in kwargs: - self.pid = kwargs.pop("pid") - if "eid" in kwargs: - self.eid = kwargs.pop("eid") - if "tid" in kwargs: - self.tid = kwargs.pop("tid") - super().__init__(*args, **kwargs) - # Set some date fields - self.set_date_fields(*args, **kwargs) - # Don't show the product/engagement/test filter fields when in specific context - if self.tid or self.eid or self.pid: - if "test__engagement__product__name" in self.form.fields: - del self.form.fields["test__engagement__product__name"] - if "test__engagement__product__name_contains" in self.form.fields: - del self.form.fields["test__engagement__product__name_contains"] - if "test__engagement__product__prod_type__name" in self.form.fields: - del self.form.fields["test__engagement__product__prod_type__name"] - if "test__engagement__product__prod_type__name_contains" in self.form.fields: - del self.form.fields["test__engagement__product__prod_type__name_contains"] - # Also hide engagement and test fields if in test or engagement context - if self.tid: - if "test__engagement__name" in self.form.fields: - del self.form.fields["test__engagement__name"] - if "test__engagement__name_contains" in self.form.fields: - del self.form.fields["test__engagement__name_contains"] - if "test__name" in self.form.fields: - del self.form.fields["test__name"] - if "test__name_contains" in self.form.fields: - del self.form.fields["test__name_contains"] - elif self.eid: - if "test__engagement__name" in self.form.fields: - del self.form.fields["test__engagement__name"] - if "test__engagement__name_contains" in self.form.fields: - del self.form.fields["test__engagement__name_contains"] - - -class FindingFilter(FindingFilterHelper, FindingTagFilter): - reporter = ModelMultipleChoiceFilter(queryset=Dojo_User.objects.none()) - reviewers = ModelMultipleChoiceFilter(queryset=Dojo_User.objects.none()) - test__engagement__product__prod_type = ModelMultipleChoiceFilter( - queryset=Product_Type.objects.none(), - label=labels.ORG_FILTERS_LABEL) - test__engagement__product = ModelMultipleChoiceFilter( - queryset=Product.objects.none(), - label=labels.ASSET_FILTERS_LABEL) - test__engagement = ModelMultipleChoiceFilter( - queryset=Engagement.objects.none(), - label="Engagement") - test = ModelMultipleChoiceFilter( - queryset=Test.objects.none(), - label="Test") - - if is_finding_groups_enabled(): - finding_group = ModelMultipleChoiceFilter( - queryset=Finding_Group.objects.none(), - label="Finding Group") - - class Meta: - model = Finding - fields = get_finding_filterset_fields() - - exclude = ["url", "description", "mitigation", "impact", - "endpoints", "references", - "thread_id", "notes", "scanner_confidence", - "numerical_severity", "line", "duplicate_finding", - "hash_code", "reviewers", "created", "files", - "sla_start_date", "sla_expiration_date", "cvssv3", - "severity_justification", "steps_to_reproduce"] - - def __init__(self, *args, **kwargs): - self.user = None - self.pid = None - self.eid = None - self.tid = None - if "user" in kwargs: - self.user = kwargs.pop("user") - - if "pid" in kwargs: - self.pid = kwargs.pop("pid") - if "eid" in kwargs: - self.eid = kwargs.pop("eid") - if "tid" in kwargs: - self.tid = kwargs.pop("tid") - super().__init__(*args, **kwargs) - # Set some date fields - self.set_date_fields(*args, **kwargs) - # Don't show the product filter on the product finding view - self.set_related_object_fields(*args, **kwargs) - - def set_related_object_fields(self, *args: list, **kwargs: dict): - # Use helper to get contextual finding group queryset - finding_group_query = get_finding_group_queryset_for_context( - pid=self.pid, - eid=self.eid, - tid=self.tid, - ) - - # Filter by most specific context: test > engagement > product - if self.tid is not None: - # Test context: filter finding groups by test - if "test__engagement__product" in self.form.fields: - del self.form.fields["test__engagement__product"] - if "test__engagement__product__prod_type" in self.form.fields: - del self.form.fields["test__engagement__product__prod_type"] - if "test__engagement" in self.form.fields: - del self.form.fields["test__engagement"] - if "test" in self.form.fields: - del self.form.fields["test"] - elif self.eid is not None: - # Engagement context: filter finding groups by engagement - if "test__engagement__product" in self.form.fields: - del self.form.fields["test__engagement__product"] - if "test__engagement__product__prod_type" in self.form.fields: - del self.form.fields["test__engagement__product__prod_type"] - if "test__engagement" in self.form.fields: - del self.form.fields["test__engagement"] - # Filter tests by engagement - get_authorized_tests doesn't support engagement param - engagement = Engagement.objects.filter(id=self.eid).select_related("product").first() - if engagement: - self.form.fields["test"].queryset = get_authorized_tests("view", product=engagement.product).filter(engagement_id=self.eid).prefetch_related("test_type") - elif self.pid is not None: - # Product context: filter finding groups by product - if "test__engagement__product" in self.form.fields: - del self.form.fields["test__engagement__product"] - if "test__engagement__product__prod_type" in self.form.fields: - del self.form.fields["test__engagement__product__prod_type"] - # TODO: add authorized check to be sure - if "test__engagement" in self.form.fields: - self.form.fields["test__engagement"].queryset = Engagement.objects.filter( - product_id=self.pid, - ).all() - if "test" in self.form.fields: - self.form.fields["test"].queryset = get_authorized_tests("view", product=self.pid).prefetch_related("test_type") - else: - # Global context: show all authorized finding groups - self.form.fields[ - "test__engagement__product__prod_type"].queryset = get_authorized_product_types("view") - self.form.fields["test__engagement"].queryset = get_authorized_engagements("view") - if "test" in self.form.fields: - del self.form.fields["test"] - - if self.form.fields.get("test__engagement__product"): - self.form.fields["test__engagement__product"].queryset = get_authorized_products("view") - if self.form.fields.get("finding_group", None): - self.form.fields["finding_group"].queryset = get_authorized_finding_groups_for_queryset("view", finding_group_query, user=self.user) - self.form.fields["reporter"].queryset = get_authorized_users("view") - self.form.fields["reviewers"].queryset = self.form.fields["reporter"].queryset - - -class FindingGroupsFilter(FilterSet): - name = CharFilter(lookup_expr="icontains", label="Name") - severity = ChoiceFilter( - choices=[ - ("Low", "Low"), - ("Medium", "Medium"), - ("High", "High"), - ("Critical", "Critical"), - ], - label="Min Severity", - ) - engagement = ModelMultipleChoiceFilter(queryset=Engagement.objects.none(), label="Engagement") - product = ModelMultipleChoiceFilter(queryset=Product.objects.none(), label=labels.ASSET_LABEL) - - class Meta: - model = Finding - fields = ["name", "severity", "engagement", "product"] - - def __init__(self, *args, **kwargs): - self.user = kwargs.pop("user", None) - self.pid = kwargs.pop("pid", None) - super().__init__(*args, **kwargs) - self.set_related_object_fields() - - def set_related_object_fields(self): - if self.pid is not None: - self.form.fields["engagement"].queryset = Engagement.objects.filter(product_id=self.pid) - if "product" in self.form.fields: - del self.form.fields["product"] - else: - self.form.fields["product"].queryset = get_authorized_products("view") - self.form.fields["engagement"].queryset = get_authorized_engagements("view") - - -class AcceptedFindingFilter(FindingFilter): - risk_acceptance__created__date = DateRangeFilter(label="Acceptance Date") - risk_acceptance__owner = ModelMultipleChoiceFilter( - queryset=Dojo_User.objects.none(), - label="Risk Acceptance Owner") - risk_acceptance = ModelMultipleChoiceFilter( - queryset=Risk_Acceptance.objects.none(), - label="Accepted By") - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.form.fields["risk_acceptance__owner"].queryset = get_authorized_users("view") - self.form.fields["risk_acceptance"].queryset = get_authorized_risk_acceptances("edit") - - -class AcceptedFindingFilterWithoutObjectLookups(FindingFilterWithoutObjectLookups): - risk_acceptance__created__date = DateRangeFilter(label="Acceptance Date") - risk_acceptance__owner = CharFilter( - field_name="risk_acceptance__owner__username", - lookup_expr="iexact", - label="Risk Acceptance Owner Username", - help_text="Search for Risk Acceptance Owners username that are an exact match") - risk_acceptance__owner_contains = CharFilter( - field_name="risk_acceptance__owner__username", - lookup_expr="icontains", - label="Risk Acceptance Owner Username Contains", - help_text="Search for Risk Acceptance Owners username that contain a given pattern") - risk_acceptance__name = CharFilter( - field_name="risk_acceptance__name", - lookup_expr="iexact", - label="Risk Acceptance Name", - help_text="Search for Risk Acceptance name that are an exact match") - risk_acceptance__name_contains = CharFilter( - field_name="risk_acceptance__name", - lookup_expr="icontains", - label="Risk Acceptance Name", - help_text="Search for Risk Acceptance name contain a given pattern") - - -class SimilarFindingHelper(FilterSet): - hash_code = MultipleChoiceFilter() - vulnerability_ids = CharFilter(method=custom_vulnerability_id_filter, label="Vulnerability Ids") - - def update_data(self, data: dict, *args: list, **kwargs: dict): - # if filterset is bound, use initial values as defaults - # because of this, we can't rely on the self.form.has_changed - self.has_changed = True - if not data and self.finding: - # get a mutable copy of the QueryDict - data = data.copy() - - data["vulnerability_ids"] = ",".join(self.finding.vulnerability_ids) - data["cwe"] = self.finding.cwe - data["file_path"] = self.finding.file_path - data["line"] = self.finding.line - data["unique_id_from_tool"] = self.finding.unique_id_from_tool - data["test__test_type"] = self.finding.test.test_type - data["test__engagement__product"] = self.finding.test.engagement.product - data["test__engagement__product__prod_type"] = self.finding.test.engagement.product.prod_type - - self.has_changed = False - - def set_hash_codes(self, *args: list, **kwargs: dict): - if self.finding and self.finding.hash_code: - self.form.fields["hash_code"] = forms.MultipleChoiceField(choices=[(self.finding.hash_code, self.finding.hash_code[:24] + "...")], required=False, initial=[]) - - def filter_queryset(self, *args: list, **kwargs: dict): - queryset = super().filter_queryset(*args, **kwargs) - queryset = get_authorized_findings_for_queryset("view", queryset, self.user) - return queryset.exclude(pk=self.finding.pk) - - -class SimilarFindingFilter(FindingFilter, SimilarFindingHelper): - class Meta(FindingFilter.Meta): - model = Finding - # slightly different fields from FindingFilter, but keep the same ordering for UI consistency - fields = get_finding_filterset_fields(similar=True) - - def __init__(self, data=None, *args, **kwargs): - self.user = None - if "user" in kwargs: - self.user = kwargs.pop("user") - self.finding = None - if "finding" in kwargs: - self.finding = kwargs.pop("finding") - self.update_data(data, *args, **kwargs) - super().__init__(data, *args, **kwargs) - self.set_hash_codes(*args, **kwargs) - - -class SimilarFindingFilterWithoutObjectLookups(FindingFilterWithoutObjectLookups, SimilarFindingHelper): - class Meta(FindingFilterWithoutObjectLookups.Meta): - model = Finding - # slightly different fields from FindingFilter, but keep the same ordering for UI consistency - fields = get_finding_filterset_fields(similar=True, filter_string_matching=True) - - def __init__(self, data=None, *args, **kwargs): - self.user = None - if "user" in kwargs: - self.user = kwargs.pop("user") - self.finding = None - if "finding" in kwargs: - self.finding = kwargs.pop("finding") - self.update_data(data, *args, **kwargs) - super().__init__(data, *args, **kwargs) - self.set_hash_codes(*args, **kwargs) - - -class TemplateFindingFilter(DojoFilter): - title = CharFilter(lookup_expr="icontains") - cwe = MultipleChoiceFilter(choices=[]) - severity = MultipleChoiceFilter(choices=SEVERITY_CHOICES) - - tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - queryset=Finding.tags.tag_model.objects.all().order_by("name"), - # label='tags', # doesn't work with tagulous, need to set in __init__ below - ) - - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") - - not_tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - exclude=True, - queryset=Finding.tags.tag_model.objects.all().order_by("name"), - # label='tags', # doesn't work with tagulous, need to set in __init__ below - ) - - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("cwe", "cwe"), - ("title", "title"), - ("numerical_severity", "numerical_severity"), - ), - field_labels={ - "numerical_severity": "Severity", - }, - ) - - class Meta: - model = Finding_Template - exclude = ["description", "mitigation", "impact", - "references", "numerical_severity"] - - not_test__tags = ModelMultipleChoiceFilter( - field_name="test__tags__name", - to_field_name="name", - exclude=True, - label="Test without tags", - queryset=Test.tags.tag_model.objects.all().order_by("name"), - # label='tags', # doesn't work with tagulous, need to set in __init__ below - ) - - not_test__engagement__tags = ModelMultipleChoiceFilter( - field_name="test__engagement__tags__name", - to_field_name="name", - exclude=True, - label="Engagement without tags", - queryset=Engagement.tags.tag_model.objects.all().order_by("name"), - # label='tags', # doesn't work with tagulous, need to set in __init__ below - ) - - not_test__engagement__product__tags = ModelMultipleChoiceFilter( - field_name="test__engagement__product__tags__name", - to_field_name="name", - exclude=True, - label=labels.ASSET_FILTERS_WITHOUT_TAGS_LABEL, - queryset=Product.tags.tag_model.objects.all().order_by("name"), - # label='tags', # doesn't work with tagulous, need to set in __init__ below - ) - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.form.fields["cwe"].choices = cwe_options(self.queryset) - - -class ApiTemplateFindingFilter(DojoFilter): - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Tag name contains") - tags = CharFieldInFilter( - field_name="tags__name", - lookup_expr="in", - help_text="Comma separated list of exact tags (uses OR for multiple values)") - tags__and = CharFieldFilterANDExpression( - field_name="tags__name", - help_text="Comma separated list of exact tags to match with an AND expression") - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Not Tag name contains", exclude="True") - not_tags = CharFieldInFilter(field_name="tags__name", lookup_expr="in", - help_text="Comma separated list of exact tags not present on model", exclude="True") - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("title", "title"), - ("cwe", "cwe"), - ), - ) - - class Meta: - model = Finding_Template - fields = ["id", "title", "cwe", "severity", "description", - "mitigation"] - - -class MetricsFindingFilter(FindingFilter): - start_date = DateFilter(field_name="date", label="Start Date", lookup_expr=("gt")) - end_date = DateFilter(field_name="date", label="End Date", lookup_expr=("lt")) - date = MetricsDateRangeFilter() - vulnerability_id = CharFilter(method=vulnerability_id_filter, label="Vulnerability Id") - - not_tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - exclude=True, - queryset=Endpoint.tags.tag_model.objects.all().order_by("name"), - # label='tags', # doesn't work with tagulous, need to set in __init__ below - ) - - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) - - def __init__(self, *args, **kwargs): - if args[0]: - if args[0].get("start_date", "") or args[0].get("end_date", ""): - args[0]._mutable = True - args[0]["date"] = 8 - args[0]._mutable = False - - super().__init__(*args, **kwargs) - - class Meta(FindingFilter.Meta): - model = Finding - fields = get_finding_filterset_fields(metrics=True) - - -class MetricsFindingFilterWithoutObjectLookups(FindingFilterWithoutObjectLookups): - start_date = DateFilter(field_name="date", label="Start Date", lookup_expr=("gt")) - end_date = DateFilter(field_name="date", label="End Date", lookup_expr=("lt")) - date = MetricsDateRangeFilter() - vulnerability_id = CharFilter(method=vulnerability_id_filter, label="Vulnerability Id") - - not_tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - exclude=True, - queryset=Endpoint.tags.tag_model.objects.all().order_by("name"), - # label='tags', # doesn't work with tagulous, need to set in __init__ below - ) - - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) - - def __init__(self, *args, **kwargs): - if args[0]: - if args[0].get("start_date", "") or args[0].get("end_date", ""): - args[0]._mutable = True - args[0]["date"] = 8 - args[0]._mutable = False - - super().__init__(*args, **kwargs) - - class Meta(FindingFilterWithoutObjectLookups.Meta): - model = Finding - fields = get_finding_filterset_fields(metrics=True, filter_string_matching=True) - - -class MetricsEndpointFilterHelper(FilterSet): - start_date = DateFilter(field_name="date", label="Start Date", lookup_expr=("gt")) - end_date = DateFilter(field_name="date", label="End Date", lookup_expr=("lt")) - date = MetricsDateRangeFilter() - finding__test__engagement__version = CharFilter(lookup_expr="icontains", label="Engagement Version") - finding__severity = MultipleChoiceFilter(choices=SEVERITY_CHOICES, label="Severity") - endpoint__host = CharFilter(lookup_expr="icontains", label="Endpoint Host") - finding_title = CharFilter(lookup_expr="icontains", label="Finding Title") - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) - - -class MetricsEndpointFilter(MetricsEndpointFilterHelper): - finding__test__engagement__product__prod_type = ModelMultipleChoiceFilter( - queryset=Product_Type.objects.none(), - label=labels.ORG_FILTERS_LABEL) - finding__test__engagement = ModelMultipleChoiceFilter( - queryset=Engagement.objects.none(), - label="Engagement") - endpoint__tags = ModelMultipleChoiceFilter( - field_name="endpoint__tags__name", - to_field_name="name", - label="Endpoint tags", - queryset=Endpoint.tags.tag_model.objects.all().order_by("name")) - finding__tags = ModelMultipleChoiceFilter( - field_name="finding__tags__name", - to_field_name="name", - label="Finding tags", - queryset=Finding.tags.tag_model.objects.all().order_by("name")) - finding__test__tags = ModelMultipleChoiceFilter( - field_name="finding__test__tags__name", - to_field_name="name", - label="Test tags", - queryset=Test.tags.tag_model.objects.all().order_by("name")) - finding__test__engagement__tags = ModelMultipleChoiceFilter( - field_name="finding__test__engagement__tags__name", - to_field_name="name", - label="Engagement tags", - queryset=Engagement.tags.tag_model.objects.all().order_by("name")) - finding__test__engagement__product__tags = ModelMultipleChoiceFilter( - field_name="finding__test__engagement__product__tags__name", - to_field_name="name", - label=labels.ASSET_FILTERS_TAGS_ASSET_LABEL, - queryset=Product.tags.tag_model.objects.all().order_by("name")) - not_endpoint__tags = ModelMultipleChoiceFilter( - field_name="endpoint__tags__name", - to_field_name="name", - exclude=True, - label="Endpoint without tags", - queryset=Endpoint.tags.tag_model.objects.all().order_by("name")) - not_finding__tags = ModelMultipleChoiceFilter( - field_name="finding__tags__name", - to_field_name="name", - exclude=True, - label="Finding without tags", - queryset=Finding.tags.tag_model.objects.all().order_by("name")) - not_finding__test__tags = ModelMultipleChoiceFilter( - field_name="finding__test__tags__name", - to_field_name="name", - exclude=True, - label="Test without tags", - queryset=Test.tags.tag_model.objects.all().order_by("name")) - not_finding__test__engagement__tags = ModelMultipleChoiceFilter( - field_name="finding__test__engagement__tags__name", - to_field_name="name", - exclude=True, - label="Engagement without tags", - queryset=Engagement.tags.tag_model.objects.all().order_by("name")) - not_finding__test__engagement__product__tags = ModelMultipleChoiceFilter( - field_name="finding__test__engagement__product__tags__name", - to_field_name="name", - exclude=True, - label=labels.ASSET_FILTERS_WITHOUT_TAGS_LABEL, - queryset=Product.tags.tag_model.objects.all().order_by("name")) - - def __init__(self, *args, **kwargs): - if args[0]: - if args[0].get("start_date", "") or args[0].get("end_date", ""): - args[0]._mutable = True - args[0]["date"] = 8 - args[0]._mutable = False - - self.pid = None - if "pid" in kwargs: - self.pid = kwargs.pop("pid") - - super().__init__(*args, **kwargs) - if self.pid: - del self.form.fields["finding__test__engagement__product__prod_type"] - self.form.fields["finding__test__engagement"].queryset = Engagement.objects.filter( - product_id=self.pid, - ).all() - else: - self.form.fields["finding__test__engagement"].queryset = get_authorized_engagements("view").order_by("name") - - if "finding__test__engagement__product__prod_type" in self.form.fields: - self.form.fields[ - "finding__test__engagement__product__prod_type"].queryset = get_authorized_product_types("view") - - class Meta: - model = Endpoint_Status - exclude = ["last_modified", "endpoint", "finding"] - - -class MetricsEndpointFilterWithoutObjectLookups(MetricsEndpointFilterHelper, FindingTagStringFilter): - finding__test__engagement__product__prod_type = CharFilter( - field_name="finding__test__engagement__product__prod_type", - lookup_expr="iexact", - label=labels.ORG_FILTERS_NAME_LABEL, - help_text=labels.ORG_FILTERS_NAME_HELP) - finding__test__engagement__product__prod_type_contains = CharFilter( - field_name="finding__test__engagement__product__prod_type", - lookup_expr="icontains", - label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, - help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) - finding__test__engagement = CharFilter( - field_name="finding__test__engagement", - lookup_expr="iexact", - label="Engagement Name", - help_text="Search for Engagement names that are an exact match") - finding__test__engagement_contains = CharFilter( - field_name="finding__test__engagement", - lookup_expr="icontains", - label="Engagement Name Contains", - help_text="Search for Engagement names that contain a given pattern") - endpoint__tags_contains = CharFilter( - label="Endpoint Tag Contains", - field_name="endpoint__tags__name", - lookup_expr="icontains", - help_text="Search for tags on a Endpoint that contain a given pattern") - endpoint__tags = CharFilter( - label="Endpoint Tag", - field_name="endpoint__tags__name", - lookup_expr="iexact", - help_text="Search for tags on a Endpoint that are an exact match") - finding__tags_contains = CharFilter( - label="Finding Tag Contains", - field_name="finding__tags__name", - lookup_expr="icontains", - help_text="Search for tags on a Finding that contain a given pattern") - finding__tags = CharFilter( - label="Finding Tag", - field_name="finding__tags__name", - lookup_expr="iexact", - help_text="Search for tags on a Finding that are an exact match") - finding__test__tags_contains = CharFilter( - label="Test Tag Contains", - field_name="finding__test__tags__name", - lookup_expr="icontains", - help_text="Search for tags on a Finding that contain a given pattern") - finding__test__tags = CharFilter( - label="Test Tag", - field_name="finding__test__tags__name", - lookup_expr="iexact", - help_text="Search for tags on a Finding that are an exact match") - finding__test__engagement__tags_contains = CharFilter( - label="Engagement Tag Contains", - field_name="finding__test__engagement__tags__name", - lookup_expr="icontains", - help_text="Search for tags on a Finding that contain a given pattern") - finding__test__engagement__tags = CharFilter( - label="Engagement Tag", - field_name="finding__test__engagement__tags__name", - lookup_expr="iexact", - help_text="Search for tags on a Finding that are an exact match") - finding__test__engagement__product__tags_contains = CharFilter( - label=labels.ASSET_FILTERS_TAG_ASSET_CONTAINS_LABEL, - field_name="finding__test__engagement__product__tags__name", - lookup_expr="icontains", - help_text=labels.ASSET_FILTERS_TAG_ASSET_CONTAINS_HELP) - finding__test__engagement__product__tags = CharFilter( - label=labels.ASSET_FILTERS_TAG_ASSET_LABEL, - field_name="finding__test__engagement__product__tags__name", - lookup_expr="iexact", - help_text=labels.ASSET_FILTERS_TAG_ASSET_HELP) - - not_endpoint__tags_contains = CharFilter( - label="Endpoint Tag Does Not Contain", - field_name="endpoint__tags__name", - lookup_expr="icontains", - help_text="Search for tags on a Endpoint that contain a given pattern, and exclude them", - exclude=True) - not_endpoint__tags = CharFilter( - label="Not Endpoint Tag", - field_name="endpoint__tags__name", - lookup_expr="iexact", - help_text="Search for tags on a Endpoint that are an exact match, and exclude them", - exclude=True) - not_finding__tags_contains = CharFilter( - label="Finding Tag Does Not Contain", - field_name="finding__tags__name", - lookup_expr="icontains", - help_text="Search for tags on a Finding that contain a given pattern, and exclude them", - exclude=True) - not_finding__tags = CharFilter( - label="Not Finding Tag", - field_name="finding__tags__name", - lookup_expr="iexact", - help_text="Search for tags on a Finding that are an exact match, and exclude them", - exclude=True) - not_finding__test__tags_contains = CharFilter( - label="Test Tag Does Not Contain", - field_name="finding__test__tags__name", - lookup_expr="icontains", - help_text="Search for tags on a Test that contain a given pattern, and exclude them", - exclude=True) - not_finding__test__tags = CharFilter( - label="Not Test Tag", - field_name="finding__test__tags__name", - lookup_expr="iexact", - help_text="Search for tags on a Test that are an exact match, and exclude them", - exclude=True) - not_finding__test__engagement__tags_contains = CharFilter( - label="Engagement Tag Does Not Contain", - field_name="finding__test__engagement__tags__name", - lookup_expr="icontains", - help_text="Search for tags on a Engagement that contain a given pattern, and exclude them", - exclude=True) - not_finding__test__engagement__tags = CharFilter( - label="Not Engagement Tag", - field_name="finding__test__engagement__tags__name", - lookup_expr="iexact", - help_text="Search for tags on a Engagement that are an exact match, and exclude them", - exclude=True) - not_finding__test__engagement__product__tags_contains = CharFilter( - label=labels.ASSET_FILTERS_TAG_NOT_CONTAIN_LABEL, - field_name="finding__test__engagement__product__tags__name", - lookup_expr="icontains", - help_text=labels.ASSET_FILTERS_TAG_NOT_CONTAIN_HELP, - exclude=True) - not_finding__test__engagement__product__tags = CharFilter( - label=labels.ASSET_FILTERS_TAG_NOT_LABEL, - field_name="finding__test__engagement__product__tags__name", - lookup_expr="iexact", - help_text=labels.ASSET_FILTERS_TAG_NOT_HELP, - exclude=True) - - def __init__(self, *args, **kwargs): - if args[0]: - if args[0].get("start_date", "") or args[0].get("end_date", ""): - args[0]._mutable = True - args[0]["date"] = 8 - args[0]._mutable = False - self.pid = None - if "pid" in kwargs: - self.pid = kwargs.pop("pid") - super().__init__(*args, **kwargs) - if self.pid: - del self.form.fields["finding__test__engagement__product__prod_type"] + super().__init__(*args, **kwargs) - class Meta: - model = Endpoint_Status - exclude = ["last_modified", "endpoint", "finding"] + def filter_percentage(self, queryset, name, value): + value /= decimal.Decimal("100.0") + # Provide some wiggle room for filtering since the UI rounds to two places (and because floats): + # a user may enter 0.15, but we'll return everything in [0.0015, 0.0016). + # To do this, add to our value 1^(whatever the exponent for our least significant digit place is), but ensure + # that the exponent is at MOST the ten thousandths place so we don't show a range of e.g. [0.2, 0.3). + exponent = min(value.normalize().as_tuple().exponent, -4) + max_val = value + decimal.Decimal(f"1E{exponent}") + lookup_kwargs = { + f"{name}__gte": value, + f"{name}__lt": max_val} + return queryset.filter(**lookup_kwargs) -class EndpointFilterHelper(FilterSet): - protocol = CharFilter(lookup_expr="icontains") - userinfo = CharFilter(lookup_expr="icontains") - host = CharFilter(lookup_expr="icontains") - port = NumberFilter() - path = CharFilter(lookup_expr="icontains") - query = CharFilter(lookup_expr="icontains") - fragment = CharFilter(lookup_expr="icontains") +class MetricsEndpointFilterHelper(FilterSet): + start_date = DateFilter(field_name="date", label="Start Date", lookup_expr=("gt")) + end_date = DateFilter(field_name="date", label="End Date", lookup_expr=("lt")) + date = MetricsDateRangeFilter() + finding__test__engagement__version = CharFilter(lookup_expr="icontains", label="Engagement Version") + finding__severity = MultipleChoiceFilter(choices=SEVERITY_CHOICES, label="Severity") + endpoint__host = CharFilter(lookup_expr="icontains", label="Endpoint Host") + finding_title = CharFilter(lookup_expr="icontains", label="Finding Title") tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) - has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("product", "product"), - ("host", "host"), - ("id", "id"), - ("active_finding_count", "active_finding_count"), - ), - field_labels={ - "active_finding_count": "Active Findings Count", - }, - ) -class EndpointFilter(EndpointFilterHelper, DojoFilter): - product = ModelMultipleChoiceFilter( - queryset=Product.objects.none(), - label=labels.ASSET_FILTERS_LABEL) - tags = ModelMultipleChoiceFilter( - field_name="tags__name", +class MetricsEndpointFilter(MetricsEndpointFilterHelper): + finding__test__engagement__product__prod_type = ModelMultipleChoiceFilter( + queryset=Product_Type.objects.none(), + label=labels.ORG_FILTERS_LABEL) + finding__test__engagement = ModelMultipleChoiceFilter( + queryset=Engagement.objects.none(), + label="Engagement") + endpoint__tags = ModelMultipleChoiceFilter( + field_name="endpoint__tags__name", to_field_name="name", - label="Endpoint Tags", + label="Endpoint tags", queryset=Endpoint.tags.tag_model.objects.all().order_by("name")) - findings__tags = ModelMultipleChoiceFilter( - field_name="findings__tags__name", + finding__tags = ModelMultipleChoiceFilter( + field_name="finding__tags__name", to_field_name="name", - label="Finding Tags", + label="Finding tags", queryset=Finding.tags.tag_model.objects.all().order_by("name")) - findings__test__tags = ModelMultipleChoiceFilter( - field_name="findings__test__tags__name", + finding__test__tags = ModelMultipleChoiceFilter( + field_name="finding__test__tags__name", to_field_name="name", - label="Test Tags", + label="Test tags", queryset=Test.tags.tag_model.objects.all().order_by("name")) - findings__test__engagement__tags = ModelMultipleChoiceFilter( - field_name="findings__test__engagement__tags__name", + finding__test__engagement__tags = ModelMultipleChoiceFilter( + field_name="finding__test__engagement__tags__name", to_field_name="name", - label="Engagement Tags", + label="Engagement tags", queryset=Engagement.tags.tag_model.objects.all().order_by("name")) - findings__test__engagement__product__tags = ModelMultipleChoiceFilter( - field_name="findings__test__engagement__product__tags__name", + finding__test__engagement__product__tags = ModelMultipleChoiceFilter( + field_name="finding__test__engagement__product__tags__name", to_field_name="name", label=labels.ASSET_FILTERS_TAGS_ASSET_LABEL, queryset=Product.tags.tag_model.objects.all().order_by("name")) - not_tags = ModelMultipleChoiceFilter( - field_name="tags__name", + not_endpoint__tags = ModelMultipleChoiceFilter( + field_name="endpoint__tags__name", to_field_name="name", - label="Not Endpoint Tags", exclude=True, + label="Endpoint without tags", queryset=Endpoint.tags.tag_model.objects.all().order_by("name")) - not_findings__tags = ModelMultipleChoiceFilter( - field_name="findings__tags__name", + not_finding__tags = ModelMultipleChoiceFilter( + field_name="finding__tags__name", to_field_name="name", - label="Not Finding Tags", exclude=True, + label="Finding without tags", queryset=Finding.tags.tag_model.objects.all().order_by("name")) - not_findings__test__tags = ModelMultipleChoiceFilter( - field_name="findings__test__tags__name", + not_finding__test__tags = ModelMultipleChoiceFilter( + field_name="finding__test__tags__name", to_field_name="name", - label="Not Test Tags", exclude=True, + label="Test without tags", queryset=Test.tags.tag_model.objects.all().order_by("name")) - not_findings__test__engagement__tags = ModelMultipleChoiceFilter( - field_name="findings__test__engagement__tags__name", + not_finding__test__engagement__tags = ModelMultipleChoiceFilter( + field_name="finding__test__engagement__tags__name", to_field_name="name", - label="Not Engagement Tags", exclude=True, + label="Engagement without tags", queryset=Engagement.tags.tag_model.objects.all().order_by("name")) - not_findings__test__engagement__product__tags = ModelMultipleChoiceFilter( - field_name="findings__test__engagement__product__tags__name", + not_finding__test__engagement__product__tags = ModelMultipleChoiceFilter( + field_name="finding__test__engagement__product__tags__name", to_field_name="name", - label=labels.ASSET_FILTERS_NOT_TAGS_ASSET_LABEL, exclude=True, + label=labels.ASSET_FILTERS_WITHOUT_TAGS_LABEL, queryset=Product.tags.tag_model.objects.all().order_by("name")) def __init__(self, *args, **kwargs): - self.user = None - if "user" in kwargs: - self.user = kwargs.pop("user") + if args[0]: + if args[0].get("start_date", "") or args[0].get("end_date", ""): + args[0]._mutable = True + args[0]["date"] = 8 + args[0]._mutable = False + + self.pid = None + if "pid" in kwargs: + self.pid = kwargs.pop("pid") + super().__init__(*args, **kwargs) - self.form.fields["product"].queryset = get_authorized_products("view") + if self.pid: + del self.form.fields["finding__test__engagement__product__prod_type"] + self.form.fields["finding__test__engagement"].queryset = Engagement.objects.filter( + product_id=self.pid, + ).all() + else: + self.form.fields["finding__test__engagement"].queryset = get_authorized_engagements("view").order_by("name") - @property - def qs(self): - parent = super().qs - return get_authorized_endpoints_for_queryset("view", parent) + if "finding__test__engagement__product__prod_type" in self.form.fields: + self.form.fields[ + "finding__test__engagement__product__prod_type"].queryset = get_authorized_product_types("view") class Meta: - model = Endpoint - exclude = ["findings", "inherited_tags"] + model = Endpoint_Status + exclude = ["last_modified", "endpoint", "finding"] -class EndpointFilterWithoutObjectLookups(EndpointFilterHelper): - product = NumberFilter(widget=HiddenInput()) - product__name = CharFilter( - field_name="product__name", +class MetricsEndpointFilterWithoutObjectLookups(MetricsEndpointFilterHelper, FindingTagStringFilter): + finding__test__engagement__product__prod_type = CharFilter( + field_name="finding__test__engagement__product__prod_type", lookup_expr="iexact", - label=labels.ASSET_FILTERS_NAME_LABEL, - help_text=labels.ASSET_FILTERS_NAME_HELP) - product__name_contains = CharFilter( - field_name="product__name", + label=labels.ORG_FILTERS_NAME_LABEL, + help_text=labels.ORG_FILTERS_NAME_HELP) + finding__test__engagement__product__prod_type_contains = CharFilter( + field_name="finding__test__engagement__product__prod_type", lookup_expr="icontains", - label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL, - help_text=labels.ASSET_FILTERS_NAME_CONTAINS_HELP) - - tags_contains = CharFilter( + label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, + help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) + finding__test__engagement = CharFilter( + field_name="finding__test__engagement", + lookup_expr="iexact", + label="Engagement Name", + help_text="Search for Engagement names that are an exact match") + finding__test__engagement_contains = CharFilter( + field_name="finding__test__engagement", + lookup_expr="icontains", + label="Engagement Name Contains", + help_text="Search for Engagement names that contain a given pattern") + endpoint__tags_contains = CharFilter( label="Endpoint Tag Contains", - field_name="tags__name", + field_name="endpoint__tags__name", lookup_expr="icontains", help_text="Search for tags on a Endpoint that contain a given pattern") - tags = CharFilter( + endpoint__tags = CharFilter( label="Endpoint Tag", - field_name="tags__name", + field_name="endpoint__tags__name", lookup_expr="iexact", help_text="Search for tags on a Endpoint that are an exact match") - findings__tags_contains = CharFilter( + finding__tags_contains = CharFilter( label="Finding Tag Contains", - field_name="findings__tags__name", + field_name="finding__tags__name", lookup_expr="icontains", help_text="Search for tags on a Finding that contain a given pattern") - findings__tags = CharFilter( + finding__tags = CharFilter( label="Finding Tag", - field_name="findings__tags__name", + field_name="finding__tags__name", lookup_expr="iexact", help_text="Search for tags on a Finding that are an exact match") - findings__test__tags_contains = CharFilter( + finding__test__tags_contains = CharFilter( label="Test Tag Contains", - field_name="findings__test__tags__name", + field_name="finding__test__tags__name", lookup_expr="icontains", help_text="Search for tags on a Finding that contain a given pattern") - findings__test__tags = CharFilter( + finding__test__tags = CharFilter( label="Test Tag", - field_name="findings__test__tags__name", + field_name="finding__test__tags__name", lookup_expr="iexact", help_text="Search for tags on a Finding that are an exact match") - findings__test__engagement__tags_contains = CharFilter( + finding__test__engagement__tags_contains = CharFilter( label="Engagement Tag Contains", - field_name="findings__test__engagement__tags__name", + field_name="finding__test__engagement__tags__name", lookup_expr="icontains", help_text="Search for tags on a Finding that contain a given pattern") - findings__test__engagement__tags = CharFilter( + finding__test__engagement__tags = CharFilter( label="Engagement Tag", - field_name="findings__test__engagement__tags__name", + field_name="finding__test__engagement__tags__name", lookup_expr="iexact", help_text="Search for tags on a Finding that are an exact match") - findings__test__engagement__product__tags_contains = CharFilter( + finding__test__engagement__product__tags_contains = CharFilter( label=labels.ASSET_FILTERS_TAG_ASSET_CONTAINS_LABEL, - field_name="findings__test__engagement__product__tags__name", + field_name="finding__test__engagement__product__tags__name", lookup_expr="icontains", help_text=labels.ASSET_FILTERS_TAG_ASSET_CONTAINS_HELP) - findings__test__engagement__product__tags = CharFilter( + finding__test__engagement__product__tags = CharFilter( label=labels.ASSET_FILTERS_TAG_ASSET_LABEL, - field_name="findings__test__engagement__product__tags__name", + field_name="finding__test__engagement__product__tags__name", lookup_expr="iexact", help_text=labels.ASSET_FILTERS_TAG_ASSET_HELP) - not_tags_contains = CharFilter( + not_endpoint__tags_contains = CharFilter( label="Endpoint Tag Does Not Contain", - field_name="tags__name", - lookup_expr="icontains", - help_text="Search for tags on a Endpoint that contain a given pattern, and exclude them", - exclude=True) - not_tags = CharFilter( - label="Not Endpoint Tag", - field_name="tags__name", - lookup_expr="iexact", - help_text="Search for tags on a Endpoint that are an exact match, and exclude them", - exclude=True) - not_findings__tags_contains = CharFilter( - label="Finding Tag Does Not Contain", - field_name="findings__tags__name", - lookup_expr="icontains", - help_text="Search for tags on a Finding that contain a given pattern, and exclude them", - exclude=True) - not_findings__tags = CharFilter( - label="Not Finding Tag", - field_name="findings__tags__name", - lookup_expr="iexact", - help_text="Search for tags on a Finding that are an exact match, and exclude them", - exclude=True) - not_findings__test__tags_contains = CharFilter( - label="Test Tag Does Not Contain", - field_name="findings__test__tags__name", - lookup_expr="icontains", - help_text="Search for tags on a Test that contain a given pattern, and exclude them", - exclude=True) - not_findings__test__tags = CharFilter( - label="Not Test Tag", - field_name="findings__test__tags__name", - lookup_expr="iexact", - help_text="Search for tags on a Test that are an exact match, and exclude them", - exclude=True) - not_findings__test__engagement__tags_contains = CharFilter( - label="Engagement Tag Does Not Contain", - field_name="findings__test__engagement__tags__name", - lookup_expr="icontains", - help_text="Search for tags on a Engagement that contain a given pattern, and exclude them", - exclude=True) - not_findings__test__engagement__tags = CharFilter( - label="Not Engagement Tag", - field_name="findings__test__engagement__tags__name", - lookup_expr="iexact", - help_text="Search for tags on a Engagement that are an exact match, and exclude them", - exclude=True) - not_findings__test__engagement__product__tags_contains = CharFilter( - label=labels.ASSET_FILTERS_TAG_NOT_CONTAIN_LABEL, - field_name="findings__test__engagement__product__tags__name", - lookup_expr="icontains", - help_text=labels.ASSET_FILTERS_TAG_NOT_CONTAIN_HELP, - exclude=True) - not_findings__test__engagement__product__tags = CharFilter( - label=labels.ASSET_FILTERS_TAG_NOT_LABEL, - field_name="findings__test__engagement__product__tags__name", - lookup_expr="iexact", - help_text=labels.ASSET_FILTERS_TAG_NOT_HELP, - exclude=True) - - def __init__(self, *args, **kwargs): - self.user = None - if "user" in kwargs: - self.user = kwargs.pop("user") - super().__init__(*args, **kwargs) - - @property - def qs(self): - parent = super().qs - return get_authorized_endpoints_for_queryset("view", parent) - - class Meta: - model = Endpoint - exclude = ["findings", "inherited_tags", "product"] - - -class ApiEndpointFilter(DojoFilter): - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Tag name contains") - tags = CharFieldInFilter( - field_name="tags__name", - lookup_expr="in", - help_text="Comma separated list of exact tags (uses OR for multiple values)") - tags__and = CharFieldFilterANDExpression( - field_name="tags__name", - help_text="Comma separated list of exact tags to match with an AND expression") - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Not Tag name contains", exclude="True") - not_tags = CharFieldInFilter(field_name="tags__name", lookup_expr="in", - help_text="Comma separated list of exact tags not present on model", exclude="True") - has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("host", "host"), - ("product", "product"), - ("id", "id"), - ("active_finding_count", "active_finding_count"), - ), - field_labels={ - "active_finding_count": "Active Findings Count", - }, - ) - - class Meta: - model = Endpoint - fields = ["id", "protocol", "userinfo", "host", "port", "path", "query", "fragment", "product"] - - -class ApiRiskAcceptanceFilter(DojoFilter): - created = DateRangeFilter() - updated = DateRangeFilter() - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("name", "name"), - ("created", "created"), - ("updated", "updated"), - ), - ) - - class Meta: - model = Risk_Acceptance - fields = { - "name": ["exact", "icontains"], - "accepted_findings": ["exact"], - "recommendation": ["exact"], - "recommendation_details": ["exact", "icontains"], - "decision": ["exact"], - "decision_details": ["exact", "icontains"], - "accepted_by": ["exact", "icontains"], - "owner": ["exact"], - "expiration_date": ["exact", "gt", "lt", "gte", "lte"], - "expiration_date_warned": ["exact", "gt", "lt", "gte", "lte"], - "expiration_date_handled": ["exact", "gt", "lt", "gte", "lte"], - "reactivate_expired": ["exact"], - "restart_sla_expired": ["exact"], - "notes": ["exact"], - "created": ["exact", "gt", "lt", "gte", "lte"], - "updated": ["exact", "gt", "lt", "gte", "lte"], - } - - -class EngagementTestFilterHelper(FilterSet): - version = CharFilter(lookup_expr="icontains", label="Version") - if settings.TRACK_IMPORT_HISTORY: - test_import__version = CharFilter(field_name="test_import__version", lookup_expr="icontains", label="Reimported Version") - target_start = DateRangeFilter() - target_end = DateRangeFilter() - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) - has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("title", "title"), - ("version", "version"), - ("target_start", "target_start"), - ("target_end", "target_end"), - ("lead", "lead"), - ("api_scan_configuration", "api_scan_configuration"), - ), - field_labels={ - "name": "Test Name", - }, - ) - - -class EngagementTestFilter(EngagementTestFilterHelper, DojoFilter): - lead = ModelChoiceFilter(queryset=Dojo_User.objects.none(), label="Lead") - api_scan_configuration = ModelChoiceFilter( - queryset=Product_API_Scan_Configuration.objects.none(), - label="API Scan Configuration") - tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - queryset=Test.tags.tag_model.objects.all().order_by("name")) - not_tags = ModelMultipleChoiceFilter( - field_name="tags__name", - to_field_name="name", - exclude=True, - queryset=Test.tags.tag_model.objects.all().order_by("name")) - - class Meta: - model = Test - fields = [ - "title", "test_type", "target_start", - "target_end", "percent_complete", - "version", "api_scan_configuration", - ] - - def __init__(self, *args, **kwargs): - self.engagement = kwargs.pop("engagement") - super(DojoFilter, self).__init__(*args, **kwargs) - self.form.fields["test_type"].queryset = Test_Type.objects.filter(test__engagement=self.engagement).distinct().order_by("name") - self.form.fields["api_scan_configuration"].queryset = Product_API_Scan_Configuration.objects.filter(product=self.engagement.product).distinct() - self.form.fields["lead"].queryset = get_authorized_users("view") \ - .filter(test__lead__isnull=False).distinct() - - -class EngagementTestFilterWithoutObjectLookups(EngagementTestFilterHelper): - lead = CharFilter( - field_name="lead__username", - lookup_expr="iexact", - label="Lead Username", - help_text="Search for Lead username that are an exact match") - lead_contains = CharFilter( - field_name="lead__username", + field_name="endpoint__tags__name", lookup_expr="icontains", - label="Lead Username Contains", - help_text="Search for Lead username that contain a given pattern") - api_scan_configuration__tool_configuration__name = CharFilter( - field_name="api_scan_configuration__tool_configuration__name", + help_text="Search for tags on a Endpoint that contain a given pattern, and exclude them", + exclude=True) + not_endpoint__tags = CharFilter( + label="Not Endpoint Tag", + field_name="endpoint__tags__name", lookup_expr="iexact", - label="API Scan Configuration Name", - help_text="Search for Lead username that are an exact match") - api_scan_configuration__tool_configuration__name_contains = CharFilter( - field_name="api_scan_configuration__tool_configuration__name", - lookup_expr="icontains", - label="API Scan Configuration Name Contains", - help_text="Search for Lead username that contain a given pattern") - tags_contains = CharFilter( - label="Test Tag Contains", - field_name="tags__name", + help_text="Search for tags on a Endpoint that are an exact match, and exclude them", + exclude=True) + not_finding__tags_contains = CharFilter( + label="Finding Tag Does Not Contain", + field_name="finding__tags__name", lookup_expr="icontains", - help_text="Search for tags on a Test that contain a given pattern") - tags = CharFilter( - label="Test Tag", - field_name="tags__name", + help_text="Search for tags on a Finding that contain a given pattern, and exclude them", + exclude=True) + not_finding__tags = CharFilter( + label="Not Finding Tag", + field_name="finding__tags__name", lookup_expr="iexact", - help_text="Search for tags on a Test that are an exact match") - not_tags_contains = CharFilter( + help_text="Search for tags on a Finding that are an exact match, and exclude them", + exclude=True) + not_finding__test__tags_contains = CharFilter( label="Test Tag Does Not Contain", - field_name="tags__name", + field_name="finding__test__tags__name", lookup_expr="icontains", help_text="Search for tags on a Test that contain a given pattern, and exclude them", exclude=True) - not_tags = CharFilter( + not_finding__test__tags = CharFilter( label="Not Test Tag", - field_name="tags__name", + field_name="finding__test__tags__name", lookup_expr="iexact", help_text="Search for tags on a Test that are an exact match, and exclude them", exclude=True) - - class Meta: - model = Test - fields = [ - "title", "test_type", "target_start", - "target_end", "percent_complete", "version", - ] + not_finding__test__engagement__tags_contains = CharFilter( + label="Engagement Tag Does Not Contain", + field_name="finding__test__engagement__tags__name", + lookup_expr="icontains", + help_text="Search for tags on a Engagement that contain a given pattern, and exclude them", + exclude=True) + not_finding__test__engagement__tags = CharFilter( + label="Not Engagement Tag", + field_name="finding__test__engagement__tags__name", + lookup_expr="iexact", + help_text="Search for tags on a Engagement that are an exact match, and exclude them", + exclude=True) + not_finding__test__engagement__product__tags_contains = CharFilter( + label=labels.ASSET_FILTERS_TAG_NOT_CONTAIN_LABEL, + field_name="finding__test__engagement__product__tags__name", + lookup_expr="icontains", + help_text=labels.ASSET_FILTERS_TAG_NOT_CONTAIN_HELP, + exclude=True) + not_finding__test__engagement__product__tags = CharFilter( + label=labels.ASSET_FILTERS_TAG_NOT_LABEL, + field_name="finding__test__engagement__product__tags__name", + lookup_expr="iexact", + help_text=labels.ASSET_FILTERS_TAG_NOT_HELP, + exclude=True) def __init__(self, *args, **kwargs): - self.engagement = kwargs.pop("engagement") + if args[0]: + if args[0].get("start_date", "") or args[0].get("end_date", ""): + args[0]._mutable = True + args[0]["date"] = 8 + args[0]._mutable = False + self.pid = None + if "pid" in kwargs: + self.pid = kwargs.pop("pid") super().__init__(*args, **kwargs) - self.form.fields["test_type"].queryset = Test_Type.objects.filter(test__engagement=self.engagement).distinct().order_by("name") - - -class ApiTestFilter(DojoFilter): - tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Tag name contains") - tags = CharFieldInFilter( - field_name="tags__name", - lookup_expr="in", - help_text="Comma separated list of exact tags (uses OR for multiple values)") - tags__and = CharFieldFilterANDExpression( - field_name="tags__name", - help_text="Comma separated list of exact tags to match with an AND expression") - engagement__tags = CharFieldInFilter( - field_name="engagement__tags__name", - lookup_expr="in", - help_text="Comma separated list of exact tags present on engagement (uses OR for multiple values)") - engagement__tags__and = CharFieldFilterANDExpression( - field_name="engagement__tags__name", - help_text="Comma separated list of exact tags to match with an AND expression present on engagement") - engagement__product__tags = CharFieldInFilter( - field_name="engagement__product__tags__name", - lookup_expr="in", - help_text=labels.ASSET_FILTERS_CSV_TAGS_OR_HELP) - engagement__product__tags__and = CharFieldFilterANDExpression( - field_name="engagement__product__tags__name", - help_text=labels.ASSET_FILTERS_CSV_TAGS_AND_HELP) - - not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Not Tag name contains", exclude="True") - not_tags = CharFieldInFilter(field_name="tags__name", lookup_expr="in", - help_text="Comma separated list of exact tags not present on model", exclude="True") - not_engagement__tags = CharFieldInFilter(field_name="engagement__tags__name", lookup_expr="in", - help_text="Comma separated list of exact tags not present on engagement", - exclude="True") - not_engagement__product__tags = CharFieldInFilter(field_name="engagement__product__tags__name", - lookup_expr="in", - help_text=labels.ASSET_FILTERS_CSV_TAGS_NOT_HELP, - exclude="True") - has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("title", "title"), - ("version", "version"), - ("target_start", "target_start"), - ("target_end", "target_end"), - ("test_type", "test_type"), - ("lead", "lead"), - ("version", "version"), - ("branch_tag", "branch_tag"), - ("build_id", "build_id"), - ("commit_hash", "commit_hash"), - ("api_scan_configuration", "api_scan_configuration"), - ("engagement", "engagement"), - ("created", "created"), - ("updated", "updated"), - ), - field_labels={ - "name": "Test Name", - }, - ) + if self.pid: + del self.form.fields["finding__test__engagement__product__prod_type"] class Meta: - model = Test - fields = ["id", "title", "test_type", "target_start", - "target_end", "notes", "percent_complete", - "engagement", "version", - "branch_tag", "build_id", "commit_hash", - "api_scan_configuration", "scan_type"] + model = Endpoint_Status + exclude = ["last_modified", "endpoint", "finding"] class ApiAppAnalysisFilter(DojoFilter): @@ -3406,410 +1325,14 @@ class Meta: exclude = ["product"] -class ReportFindingFilterHelper(FilterSet): - title = CharFilter(lookup_expr="icontains", label="Name") - date = DateFromToRangeFilter(field_name="date", label="Date Discovered") - date_recent = DateRangeFilter(field_name="date", label="Relative Date") - severity = MultipleChoiceFilter(choices=SEVERITY_CHOICES) - active = ReportBooleanFilter() - is_mitigated = ReportBooleanFilter() - mitigated = DateRangeFilter(label="Mitigated Date") - verified = ReportBooleanFilter() - false_p = ReportBooleanFilter(label="False Positive") - risk_acceptance = ReportRiskAcceptanceFilter(label="Risk Accepted") - duplicate = ReportBooleanFilter() - out_of_scope = ReportBooleanFilter() - outside_of_sla = FindingSLAFilter(label="Outside of SLA") - file_path = CharFilter(lookup_expr="icontains") - mitigation_available = BooleanFilter(method="filter_mitigation_available", label="Mitigation Available") - - o = OrderingFilter( - fields=( - ("title", "title"), - ("date", "date"), - ("fix_available", "fix_available"), - ("numerical_severity", "numerical_severity"), - ("epss_score", "epss_score"), - ("epss_percentile", "epss_percentile"), - ("test__engagement__product__name", "test__engagement__product__name"), - ), - ) - - class Meta: - model = Finding - # exclude sonarqube issue as by default it will show all without checking permissions - exclude = ["date", "cwe", "url", "description", "mitigation", "impact", - "references", "sonarqube_issue", "duplicate_finding", - "thread_id", "notes", "inherited_tags", "endpoints", - "numerical_severity", "reporter", "last_reviewed", - "jira_creation", "jira_change", "files"] - - def filter_mitigation_available(self, queryset, name, value): - if value: - return queryset.exclude(mitigation__isnull=True).exclude(mitigation__exact="") - return queryset.filter(Q(mitigation__isnull=True) | Q(mitigation__exact="")) - - def manage_kwargs(self, kwargs): - self.prod_type = None - self.product = None - self.engagement = None - self.test = None - if "prod_type" in kwargs: - self.prod_type = kwargs.pop("prod_type") - if "product" in kwargs: - self.product = kwargs.pop("product") - if "engagement" in kwargs: - self.engagement = kwargs.pop("engagement") - if "test" in kwargs: - self.test = kwargs.pop("test") - - @property - def qs(self): - parent = super().qs - return get_authorized_findings_for_queryset("view", parent) - - -class ReportFindingFilter(ReportFindingFilterHelper, FindingTagFilter): - test__engagement__product = ModelMultipleChoiceFilter( - queryset=Product.objects.none(), label=labels.ASSET_FILTERS_LABEL) - test__engagement__product__prod_type = ModelMultipleChoiceFilter( - queryset=Product_Type.objects.none(), - label=labels.ORG_FILTERS_LABEL) - test__engagement__product__lifecycle = MultipleChoiceFilter(choices=Product.LIFECYCLE_CHOICES, label=labels.ASSET_LIFECYCLE_LABEL) - test__engagement = ModelMultipleChoiceFilter(queryset=Engagement.objects.none(), label="Engagement") - duplicate_finding = ModelChoiceFilter(queryset=Finding.objects.filter(original_finding__isnull=False).distinct()) - - def __init__(self, *args, **kwargs): - self.manage_kwargs(kwargs) - super().__init__(*args, **kwargs) - - # duplicate_finding queryset needs to restricted in line with permissions - # and inline with report scope to avoid a dropdown with 100K entries - duplicate_finding_query_set = self.form.fields["duplicate_finding"].queryset - duplicate_finding_query_set = get_authorized_findings_for_queryset("view", duplicate_finding_query_set) - - if self.test: - duplicate_finding_query_set = duplicate_finding_query_set.filter(test=self.test) - del self.form.fields["test__tags"] - del self.form.fields["test__engagement__tags"] - del self.form.fields["test__engagement__product__tags"] - if self.engagement: - duplicate_finding_query_set = duplicate_finding_query_set.filter(test__engagement=self.engagement) - del self.form.fields["test__engagement__tags"] - del self.form.fields["test__engagement__product__tags"] - elif self.product: - duplicate_finding_query_set = duplicate_finding_query_set.filter(test__engagement__product=self.product) - del self.form.fields["test__engagement__product"] - del self.form.fields["test__engagement__product__tags"] - elif self.prod_type: - duplicate_finding_query_set = duplicate_finding_query_set.filter(test__engagement__product__prod_type=self.prod_type) - del self.form.fields["test__engagement__product__prod_type"] - - self.form.fields["duplicate_finding"].queryset = duplicate_finding_query_set - - if "test__engagement__product__prod_type" in self.form.fields: - self.form.fields[ - "test__engagement__product__prod_type"].queryset = get_authorized_product_types("view") - if "test__engagement__product" in self.form.fields: - self.form.fields[ - "test__engagement__product"].queryset = get_authorized_products("view") - if "test__engagement" in self.form.fields: - self.form.fields["test__engagement"].queryset = get_authorized_engagements("view") - - -class ReportFindingFilterWithoutObjectLookups(ReportFindingFilterHelper, FindingTagStringFilter): - test__engagement__product__prod_type = NumberFilter(widget=HiddenInput()) - test__engagement__product = NumberFilter(widget=HiddenInput()) - test__engagement = NumberFilter(widget=HiddenInput()) - test = NumberFilter(widget=HiddenInput()) - endpoint = NumberFilter(widget=HiddenInput()) - reporter = CharFilter( - field_name="reporter__username", - lookup_expr="iexact", - label="Reporter Username", - help_text="Search for Reporter names that are an exact match") - reporter_contains = CharFilter( - field_name="reporter__username", - lookup_expr="icontains", - label="Reporter Username Contains", - help_text="Search for Reporter names that contain a given pattern") - reviewers = CharFilter( - field_name="reviewers__username", - lookup_expr="iexact", - label="Reviewer Username", - help_text="Search for Reviewer names that are an exact match") - reviewers_contains = CharFilter( - field_name="reviewers__username", - lookup_expr="icontains", - label="Reviewer Username Contains", - help_text="Search for Reviewer usernames that contain a given pattern") - last_reviewed_by = CharFilter( - field_name="last_reviewed_by__username", - lookup_expr="iexact", - label="Last Reviewed By Username", - help_text="Search for Last Reviewed By names that are an exact match") - last_reviewed_by_contains = CharFilter( - field_name="last_reviewed_by__username", - lookup_expr="icontains", - label="Last Reviewed By Username Contains", - help_text="Search for Last Reviewed By usernames that contain a given pattern") - review_requested_by = CharFilter( - field_name="review_requested_by__username", - lookup_expr="iexact", - label="Review Requested By Username", - help_text="Search for Review Requested By names that are an exact match") - review_requested_by_contains = CharFilter( - field_name="review_requested_by__username", - lookup_expr="icontains", - label="Review Requested By Username Contains", - help_text="Search for Review Requested By usernames that contain a given pattern") - mitigated_by = CharFilter( - field_name="mitigated_by__username", - lookup_expr="iexact", - label="Mitigator Username", - help_text="Search for Mitigator names that are an exact match") - mitigated_by_contains = CharFilter( - field_name="mitigated_by__username", - lookup_expr="icontains", - label="Mitigator Username Contains", - help_text="Search for Mitigator usernames that contain a given pattern") - defect_review_requested_by = CharFilter( - field_name="defect_review_requested_by__username", - lookup_expr="iexact", - label="Requester of Defect Review Username", - help_text="Search for Requester of Defect Review names that are an exact match") - defect_review_requested_by_contains = CharFilter( - field_name="defect_review_requested_by__username", - lookup_expr="icontains", - label="Requester of Defect Review Username Contains", - help_text="Search for Requester of Defect Review usernames that contain a given pattern") - test__engagement__product__prod_type__name = CharFilter( - field_name="test__engagement__product__prod_type__name", - lookup_expr="iexact", - label=labels.ORG_FILTERS_NAME_LABEL, - help_text=labels.ORG_FILTERS_NAME_HELP) - test__engagement__product__prod_type__name_contains = CharFilter( - field_name="test__engagement__product__prod_type__name", - lookup_expr="icontains", - label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, - help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) - test__engagement__product__name = CharFilter( - field_name="test__engagement__product__name", - lookup_expr="iexact", - label=labels.ASSET_FILTERS_NAME_LABEL, - help_text=labels.ASSET_FILTERS_NAME_HELP) - test__engagement__product__name_contains = CharFilter( - field_name="test__engagement__product__name", - lookup_expr="icontains", - label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL, - help_text=labels.ASSET_FILTERS_NAME_CONTAINS_HELP) - test__engagement__name = CharFilter( - field_name="test__engagement__name", - lookup_expr="iexact", - label="Engagement Name", - help_text="Search for Engagement names that are an exact match") - test__engagement__name_contains = CharFilter( - field_name="test__engagement__name", - lookup_expr="icontains", - label="Engagement name Contains", - help_text="Search for Engagement names that contain a given pattern") - test__name = CharFilter( - field_name="test__title", - lookup_expr="iexact", - label="Test Name", - help_text="Search for Test names that are an exact match") - test__name_contains = CharFilter( - field_name="test__title", - lookup_expr="icontains", - label="Test name Contains", - help_text="Search for Test names that contain a given pattern") - - def __init__(self, *args, **kwargs): - self.manage_kwargs(kwargs) - super().__init__(*args, **kwargs) - - product_type_refs = [ - "test__engagement__product__prod_type__name", - "test__engagement__product__prod_type__name_contains", - ] - product_refs = [ - "test__engagement__product__name", - "test__engagement__product__name_contains", - "test__engagement__product__tags", - "test__engagement__product__tags_contains", - "not_test__engagement__product__tags", - "not_test__engagement__product__tags_contains", - ] - engagement_refs = [ - "test__engagement__name", - "test__engagement__name_contains", - "test__engagement__tags", - "test__engagement__tags_contains", - "not_test__engagement__tags", - "not_test__engagement__tags_contains", - ] - test_refs = [ - "test__name", - "test__name_contains", - "test__tags", - "test__tags_contains", - "not_test__tags", - "not_test__tags_contains", - ] - - if self.test: - self.delete_tags_from_form(product_type_refs) - self.delete_tags_from_form(product_refs) - self.delete_tags_from_form(engagement_refs) - self.delete_tags_from_form(test_refs) - elif self.engagement: - self.delete_tags_from_form(product_type_refs) - self.delete_tags_from_form(product_refs) - self.delete_tags_from_form(engagement_refs) - elif self.product: - self.delete_tags_from_form(product_type_refs) - self.delete_tags_from_form(product_refs) - elif self.prod_type: - self.delete_tags_from_form(product_type_refs) - - -class UserFilter(DojoFilter): - first_name = CharFilter(lookup_expr="icontains") - last_name = CharFilter(lookup_expr="icontains") - username = CharFilter(lookup_expr="icontains") - email = CharFilter(lookup_expr="icontains") - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("username", "username"), - ("last_name", "last_name"), - ("first_name", "first_name"), - ("email", "email"), - ("is_active", "is_active"), - ("is_superuser", "is_superuser"), - ("is_staff", "is_staff"), - ("date_joined", "date_joined"), - ("last_login", "last_login"), - ), - field_labels={ - "username": "User Name", - "is_active": "Active", - "is_superuser": "Superuser", - "is_staff": "Staff", - }, - ) - - class Meta: - model = Dojo_User - fields = ["is_superuser", "is_staff", "is_active", "first_name", "last_name", "username", "email"] - - -# This class is used exclusively by Findings -class TestImportFilter(DojoFilter): - version = CharFilter(field_name="version", lookup_expr="icontains") - version_exact = CharFilter(field_name="version", lookup_expr="iexact", label="Version Exact") - branch_tag = CharFilter(lookup_expr="icontains", label="Branch/Tag") - build_id = CharFilter(lookup_expr="icontains", label="Build ID") - commit_hash = CharFilter(lookup_expr="icontains", label="Commit hash") - - findings_affected = BooleanFilter(field_name="findings_affected", lookup_expr="isnull", exclude=True, label="Findings affected") - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("date", "date"), - ("version", "version"), - ("branch_tag", "branch_tag"), - ("build_id", "build_id"), - ("commit_hash", "commit_hash"), - - ), - ) - - class Meta: - model = Test_Import - fields = [] - - -# This class is used exclusively by Findings -class TestImportFindingActionFilter(DojoFilter): - action = MultipleChoiceFilter(choices=IMPORT_ACTIONS) - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("action", "action"), - ), - ) - - class Meta: - model = Test_Import_Finding_Action - fields = [] - - -# Used within the TestImport API -class TestImportAPIFilter(DojoFilter): - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("id", "id"), - ("created", "created"), - ("modified", "modified"), - ("version", "version"), - ("branch_tag", "branch_tag"), - ("build_id", "build_id"), - ("commit_hash", "commit_hash"), - - ), - ) - - class Meta: - model = Test_Import - fields = ["test", - "findings_affected", - "version", - "branch_tag", - "build_id", - "commit_hash", - "test_import_finding_action__action", - "test_import_finding_action__finding", - "test_import_finding_action__created"] +# UserFilter lives in dojo/user/ui/filters.py — import from there directly. +# TestImportFilter and TestImportFindingActionFilter live in dojo/test/ui/filters.py and are +# re-exported at the bottom of this module for backward compatibility. # LogEntryFilter and PgHistoryFilter live in dojo/auditlog/filters.py and are # re-exported at the bottom of this module for backward compatibility. - - -class ProductTypeFilter(DojoFilter): - name = CharFilter(lookup_expr="icontains") - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("name", "name"), - ), - ) - - class Meta: - model = Product_Type - exclude = [] - include = ("name",) - - -class TestTypeFilter(DojoFilter): - name = CharFilter(lookup_expr="icontains") - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("name", "name"), - ), - ) - - class Meta: - model = Test_Type - exclude = [] - include = ("name",) +# TestTypeFilter lives in dojo/test/ui/filters.py and is re-exported below. class DevelopmentEnvironmentFilter(DojoFilter): @@ -3846,100 +1369,48 @@ class Meta: exclude = [] include = ("name", "is_single", "description") -# ============================== -# Defect Dojo Engaegment Surveys -# ============================== - - -class QuestionnaireFilter(FilterSet): - name = CharFilter(lookup_expr="icontains") - description = CharFilter(lookup_expr="icontains") - active = BooleanFilter() - - class Meta: - model = Engagement_Survey - exclude = ["questions"] - - survey_set = FilterSet - - -class QuestionTypeFilter(ChoiceFilter): - def any(self, qs, name): - return qs.all() - - def text_question(self, qs, name): - return qs.filter(polymorphic_ctype=ContentType.objects.get_for_model(TextQuestion)) - - def choice_question(self, qs, name): - return qs.filter(polymorphic_ctype=ContentType.objects.get_for_model(ChoiceQuestion)) - - options = { - None: (_("Any"), any), - 1: (_("Text Question"), text_question), - 2: (_("Choice Question"), choice_question), - } - - def __init__(self, *args, **kwargs): - kwargs["choices"] = [ - (key, value[0]) for key, value in six.iteritems(self.options)] - super().__init__(*args, **kwargs) - - def filter(self, qs, value): - try: - value = int(value) - except (ValueError, TypeError): - value = None - return self.options[value][1](self, qs, self.options[value][0]) - - -class ApiUserFilter(filters.FilterSet): - last_login = filters.DateFromToRangeFilter() - date_joined = filters.DateFromToRangeFilter() - is_active = filters.BooleanFilter() - is_superuser = filters.BooleanFilter() - username = filters.CharFilter(lookup_expr="icontains") - first_name = filters.CharFilter(lookup_expr="icontains") - last_name = filters.CharFilter(lookup_expr="icontains") - email = filters.CharFilter(lookup_expr="icontains") - class Meta: - model = User - fields = [ - "id", - "username", - "first_name", - "last_name", - "email", - "is_active", - "is_superuser", - "last_login", - "date_joined", - ] - - o = OrderingFilter( - # tuple-mapping retains order - fields=( - ("username", "username"), - ("last_name", "last_name"), - ("first_name", "first_name"), - ("email", "email"), - ("is_active", "is_active"), - ("is_superuser", "is_superuser"), - ("date_joined", "date_joined"), - ("last_login", "last_login"), - ), - ) - - -with warnings.catch_warnings(action="ignore", category=ManagerInheritanceWarning): - class QuestionFilter(FilterSet): - text = CharFilter(lookup_expr="icontains") - type = QuestionTypeFilter() +# ApiUserFilter lives in dojo/user/api/filters.py — import from there directly. +# QuestionnaireFilter, QuestionTypeFilter, QuestionFilter live in dojo/survey/ui/filters.py - class Meta: - model = Question - exclude = ["polymorphic_ctype", "created", "modified", "order"] - question_set = FilterSet +import importlib # noqa: E402 +# Backward-compat re-exports for external consumers (e.g. dojo-pro) that still +# import filters from dojo.filters. The filter classes themselves moved into the +# per-module ui/api filter packages. Those modules import dojo.filters base +# classes (DojoFilter, ...) at import time, so eagerly importing them back here +# creates circular imports. Instead expose them lazily via PEP 562 __getattr__, +# which only resolves the target module when the name is actually accessed. +from django_filters import BooleanFilter # noqa: E402, F401 -- backward compat re-export from dojo.auditlog.filters import LogEntryFilter, PgHistoryFilter # noqa: E402, F401 -- backward compat +from dojo.models import Product_API_Scan_Configuration # noqa: E402, F401 -- backward compat re-export + +_LAZY_FILTER_EXPORTS = { + "ApiEndpointFilter": "dojo.endpoint.api.filters", + "EndpointFilter": "dojo.endpoint.ui.filters", + "EndpointFilterWithoutObjectLookups": "dojo.endpoint.ui.filters", + "ApiEngagementFilter": "dojo.engagement.api.filters", + "EngagementDirectFilter": "dojo.engagement.ui.filters", + "EngagementTestFilter": "dojo.engagement.ui.filters", + "EngagementTestFilterWithoutObjectLookups": "dojo.engagement.ui.filters", + "ApiFindingFilter": "dojo.finding.api.filters", + "AcceptedFindingFilter": "dojo.finding.ui.filters", + "AcceptedFindingFilterWithoutObjectLookups": "dojo.finding.ui.filters", + "FindingFilter": "dojo.finding.ui.filters", + "FindingFilterWithoutObjectLookups": "dojo.finding.ui.filters", + "ReportFindingFilter": "dojo.finding.ui.filters", + "ReportFindingFilterWithoutObjectLookups": "dojo.finding.ui.filters", + "ApiProductFilter": "dojo.product.api.filters", + "ProductFilter": "dojo.product.ui.filters", + "ApiTestFilter": "dojo.test.api.filters", + "UserFilter": "dojo.user.ui.filters", +} + + +def __getattr__(name): + module_path = _LAZY_FILTER_EXPORTS.get(name) + if module_path is None: + msg = f"module 'dojo.filters' has no attribute {name!r}" + raise AttributeError(msg) + return getattr(importlib.import_module(module_path), name) diff --git a/dojo/finding/__init__.py b/dojo/finding/__init__.py index e69de29bb2d..83045d6089c 100644 --- a/dojo/finding/__init__.py +++ b/dojo/finding/__init__.py @@ -0,0 +1 @@ +import dojo.finding.admin # noqa: F401 diff --git a/dojo/finding/admin.py b/dojo/finding/admin.py new file mode 100644 index 00000000000..f10732d25f7 --- /dev/null +++ b/dojo/finding/admin.py @@ -0,0 +1,42 @@ +from django.contrib import admin + +from dojo.finding.models import ( + CWE, + BurpRawRequestResponse, + Finding, + Finding_Group, + Finding_Template, + Vulnerability_Id, +) + + +@admin.register(Finding) +class FindingAdmin(admin.ModelAdmin): + # TODO: Delete this after the move to Locations + # For efficiency with large databases, display many-to-many fields with raw + # IDs rather than multi-select + raw_id_fields = ( + "endpoints", + ) + + +@admin.register(Finding_Template) +class FindingTemplateAdmin(admin.ModelAdmin): + + """Admin support for the Finding_Template model.""" + + +@admin.register(Vulnerability_Id) +class VulnerabilityIdAdmin(admin.ModelAdmin): + + """Admin support for the Vulnerability_Id model.""" + + +@admin.register(Finding_Group) +class FindingGroupAdmin(admin.ModelAdmin): + + """Admin support for the Finding_Group model.""" + + +admin.site.register(CWE) +admin.site.register(BurpRawRequestResponse) diff --git a/dojo/finding/api/__init__.py b/dojo/finding/api/__init__.py new file mode 100644 index 00000000000..4d8feaaab6a --- /dev/null +++ b/dojo/finding/api/__init__.py @@ -0,0 +1 @@ +path = "findings" # noqa: RUF067 diff --git a/dojo/finding/api/filters.py b/dojo/finding/api/filters.py new file mode 100644 index 00000000000..f9450aa820f --- /dev/null +++ b/dojo/finding/api/filters.py @@ -0,0 +1,247 @@ +from datetime import timedelta + +from django.db.models import Q +from django_filters import ( + BooleanFilter, + CharFilter, + DateFilter, + DateTimeFilter, + OrderingFilter, +) +from drf_spectacular.types import OpenApiTypes +from drf_spectacular.utils import extend_schema_field + +from dojo.filters import ( + CharFieldFilterANDExpression, + CharFieldInFilter, + DateRangeFilter, + DojoFilter, + FindingSLAFilter, + NumberInFilter, + PercentageRangeFilter, + ReportRiskAcceptanceFilter, + custom_filter, + custom_vulnerability_id_filter, + labels, +) +from dojo.models import Finding, Finding_Template + + +class ApiFindingFilter(DojoFilter): + # BooleanFilter + active = BooleanFilter(field_name="active") + duplicate = BooleanFilter(field_name="duplicate") + dynamic_finding = BooleanFilter(field_name="dynamic_finding") + false_p = BooleanFilter(field_name="false_p") + is_mitigated = BooleanFilter(field_name="is_mitigated") + out_of_scope = BooleanFilter(field_name="out_of_scope") + static_finding = BooleanFilter(field_name="static_finding") + under_defect_review = BooleanFilter(field_name="under_defect_review") + under_review = BooleanFilter(field_name="under_review") + verified = BooleanFilter(field_name="verified") + has_jira = BooleanFilter(field_name="jira_issue", lookup_expr="isnull", exclude=True) + fix_available = BooleanFilter(field_name="fix_available") + mitigation_available = BooleanFilter(method="filter_mitigation_available", label="Mitigation Available") + # CharFilter + component_version = CharFilter(lookup_expr="icontains") + component_name = CharFilter(lookup_expr="icontains") + vulnerability_id = CharFilter(method=custom_vulnerability_id_filter) + description = CharFilter(lookup_expr="icontains") + file_path = CharFilter(lookup_expr="icontains") + hash_code = CharFilter(lookup_expr="icontains") + impact = CharFilter(lookup_expr="icontains") + mitigation = CharFilter(lookup_expr="icontains") + numerical_severity = CharFilter(method=custom_filter, field_name="numerical_severity") + param = CharFilter(lookup_expr="icontains") + payload = CharFilter(lookup_expr="icontains") + references = CharFilter(lookup_expr="icontains") + severity = CharFilter(method=custom_filter, field_name="severity") + severity_justification = CharFilter(lookup_expr="icontains") + steps_to_reproduce = CharFilter(lookup_expr="icontains") + unique_id_from_tool = CharFilter(lookup_expr="icontains") + title = CharFilter(lookup_expr="icontains") + exact_title = CharFilter(field_name="title", lookup_expr="iexact", help_text="Finding title exact match (case-insensitive)") + product_name = CharFilter(lookup_expr="engagement__product__name__iexact", field_name="test", label=labels.ASSET_FILTERS_NAME_EXACT_LABEL) + product_name_contains = CharFilter(lookup_expr="engagement__product__name__icontains", field_name="test", label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL) + product_lifecycle = CharFilter(method=custom_filter, lookup_expr="engagement__product__lifecycle", + field_name="test__engagement__product__lifecycle", label=labels.ASSET_FILTERS_CSV_LIFECYCLES_LABEL) + # DateRangeFilter + created = DateRangeFilter() + date = DateRangeFilter() + discovered_on = DateFilter(field_name="date", lookup_expr="exact") + discovered_before = DateFilter(field_name="date", lookup_expr="lt") + discovered_after = DateFilter(field_name="date", lookup_expr="gt") + jira_creation = DateRangeFilter(field_name="jira_issue__jira_creation") + jira_change = DateRangeFilter(field_name="jira_issue__jira_change") + last_reviewed = DateRangeFilter() + mitigated = DateRangeFilter() + mitigated_on = DateTimeFilter(field_name="mitigated", lookup_expr="exact", method="filter_mitigated_on") + mitigated_before = DateTimeFilter(field_name="mitigated", lookup_expr="lt") + mitigated_after = DateTimeFilter(field_name="mitigated", lookup_expr="gt", label="Mitigated After", method="filter_mitigated_after") + # NumberInFilter + cwe = NumberInFilter(field_name="cwe", lookup_expr="in") + defect_review_requested_by = NumberInFilter(field_name="defect_review_requested_by", lookup_expr="in") + endpoints = NumberInFilter(field_name="endpoints", lookup_expr="in") + epss_score = PercentageRangeFilter( + field_name="epss_score", + label="EPSS score range", + help_text=( + "The range of EPSS score percentages to filter on; the min input is a lower bound, " + "the max is an upper bound. Leaving one empty will skip that bound (e.g., leaving " + "the min bound input empty will filter only on the max bound -- filtering on " + '"less than or equal"). Leading 0 required.' + )) + epss_percentile = PercentageRangeFilter( + field_name="epss_percentile", + label="EPSS percentile range", + help_text=( + "The range of EPSS percentiles to filter on; the min input is a lower bound, the max " + "is an upper bound. Leaving one empty will skip that bound (e.g., leaving the min bound " + 'input empty will filter only on the max bound -- filtering on "less than or equal"). Leading 0 required.' + )) + found_by = NumberInFilter(field_name="found_by", lookup_expr="in") + id = NumberInFilter(field_name="id", lookup_expr="in") + last_reviewed_by = NumberInFilter(field_name="last_reviewed_by", lookup_expr="in") + mitigated_by = NumberInFilter(field_name="mitigated_by", lookup_expr="in") + nb_occurences = NumberInFilter(field_name="nb_occurences", lookup_expr="in") + reporter = NumberInFilter(field_name="reporter", lookup_expr="in") + scanner_confidence = NumberInFilter(field_name="scanner_confidence", lookup_expr="in") + review_requested_by = NumberInFilter(field_name="review_requested_by", lookup_expr="in") + reviewers = NumberInFilter(field_name="reviewers", lookup_expr="in") + sast_source_line = NumberInFilter(field_name="sast_source_line", lookup_expr="in") + sonarqube_issue = NumberInFilter(field_name="sonarqube_issue", lookup_expr="in") + test__test_type = NumberInFilter(field_name="test__test_type", lookup_expr="in", label="Test Type") + test__engagement = NumberInFilter(field_name="test__engagement", lookup_expr="in") + test__engagement__product = NumberInFilter(field_name="test__engagement__product", lookup_expr="in") + test__engagement__product__prod_type = NumberInFilter(field_name="test__engagement__product__prod_type", lookup_expr="in") + finding_group = NumberInFilter(field_name="finding_group", lookup_expr="in") + + # ReportRiskAcceptanceFilter + risk_acceptance = extend_schema_field(OpenApiTypes.NUMBER)(ReportRiskAcceptanceFilter()) + + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Tag name contains") + tags = CharFieldInFilter( + field_name="tags__name", + lookup_expr="in", + help_text="Comma separated list of exact tags (uses OR for multiple values)") + tags__and = CharFieldFilterANDExpression( + field_name="tags__name", + help_text="Comma separated list of exact tags to match with an AND expression") + test__tags = CharFieldInFilter( + field_name="test__tags__name", + lookup_expr="in", + help_text="Comma separated list of exact tags present on test (uses OR for multiple values)") + test__tags__and = CharFieldFilterANDExpression( + field_name="test__tags__name", + help_text="Comma separated list of exact tags to match with an AND expression present on test") + test__engagement__tags = CharFieldInFilter( + field_name="test__engagement__tags__name", + lookup_expr="in", + help_text="Comma separated list of exact tags present on engagement (uses OR for multiple values)") + test__engagement__tags__and = CharFieldFilterANDExpression( + field_name="test__engagement__tags__name", + help_text="Comma separated list of exact tags to match with an AND expression present on engagement") + test__engagement__product__tags = CharFieldInFilter( + field_name="test__engagement__product__tags__name", + lookup_expr="in", + help_text=labels.ASSET_FILTERS_CSV_TAGS_OR_HELP) + test__engagement__product__tags__and = CharFieldFilterANDExpression( + field_name="test__engagement__product__tags__name", + help_text=labels.ASSET_FILTERS_CSV_TAGS_AND_HELP) + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Not Tag name contains", exclude="True") + not_tags = CharFieldInFilter(field_name="tags__name", lookup_expr="in", + help_text="Comma separated list of exact tags not present on model", exclude="True") + not_test__tags = CharFieldInFilter(field_name="test__tags__name", lookup_expr="in", exclude="True", help_text="Comma separated list of exact tags present on test") + not_test__engagement__tags = CharFieldInFilter(field_name="test__engagement__tags__name", lookup_expr="in", + help_text="Comma separated list of exact tags not present on engagement", + exclude="True") + not_test__engagement__product__tags = CharFieldInFilter( + field_name="test__engagement__product__tags__name", + lookup_expr="in", + help_text=labels.ASSET_FILTERS_CSV_TAGS_NOT_HELP, + exclude="True") + has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") + outside_of_sla = extend_schema_field(OpenApiTypes.NUMBER)(FindingSLAFilter()) + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("active", "active"), + ("component_name", "component_name"), + ("component_version", "component_version"), + ("created", "created"), + ("last_status_update", "last_status_update"), + ("last_reviewed", "last_reviewed"), + ("cwe", "cwe"), + ("date", "date"), + ("duplicate", "duplicate"), + ("dynamic_finding", "dynamic_finding"), + ("false_p", "false_p"), + ("found_by", "found_by"), + ("id", "id"), + ("is_mitigated", "is_mitigated"), + ("numerical_severity", "numerical_severity"), + ("out_of_scope", "out_of_scope"), + ("planned_remediation_date", "planned_remediation_date"), + ("severity", "severity"), + ("sla_expiration_date", "sla_expiration_date"), + ("reviewers", "reviewers"), + ("static_finding", "static_finding"), + ("test__engagement__product__name", "test__engagement__product__name"), + ("title", "title"), + ("under_defect_review", "under_defect_review"), + ("under_review", "under_review"), + ("verified", "verified"), + ), + ) + + class Meta: + model = Finding + exclude = ["url", "thread_id", "notes", "files", + "line", "cve"] + + def filter_mitigated_after(self, queryset, name, value): + if value.hour == 0 and value.minute == 0 and value.second == 0: + value = value.replace(hour=23, minute=59, second=59) + + return queryset.filter(mitigated__gt=value) + + def filter_mitigated_on(self, queryset, name, value): + if value.hour == 0 and value.minute == 0 and value.second == 0: + # we have a simple date without a time, lets get a range from this morning to tonight at 23:59:59:999 + nextday = value + timedelta(days=1) + return queryset.filter(mitigated__gte=value, mitigated__lt=nextday) + + return queryset.filter(mitigated=value) + + def filter_mitigation_available(self, queryset, name, value): + if value: + return queryset.exclude(mitigation__isnull=True).exclude(mitigation__exact="") + return queryset.filter(Q(mitigation__isnull=True) | Q(mitigation__exact="")) + + +class ApiTemplateFindingFilter(DojoFilter): + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Tag name contains") + tags = CharFieldInFilter( + field_name="tags__name", + lookup_expr="in", + help_text="Comma separated list of exact tags (uses OR for multiple values)") + tags__and = CharFieldFilterANDExpression( + field_name="tags__name", + help_text="Comma separated list of exact tags to match with an AND expression") + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Not Tag name contains", exclude="True") + not_tags = CharFieldInFilter(field_name="tags__name", lookup_expr="in", + help_text="Comma separated list of exact tags not present on model", exclude="True") + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("title", "title"), + ("cwe", "cwe"), + ), + ) + + class Meta: + model = Finding_Template + fields = ["id", "title", "cwe", "severity", "description", + "mitigation"] diff --git a/dojo/finding/api/serializer.py b/dojo/finding/api/serializer.py new file mode 100644 index 00000000000..360f08b7926 --- /dev/null +++ b/dojo/finding/api/serializer.py @@ -0,0 +1,884 @@ +import base64 +import collections +import json +import logging + +import six +from django.conf import settings +from django.core.exceptions import ValidationError +from django.urls import reverse +from django.utils.translation import gettext_lazy as _ +from drf_spectacular.utils import extend_schema_field +from rest_framework import serializers +from rest_framework.fields import DictField + +import dojo.finding.helper as finding_helper +from dojo.authorization.authorization import user_has_permission +from dojo.celery_dispatch import dojo_dispatch_task +from dojo.finding.helper import ( + save_endpoints_template, + save_vulnerability_ids, + save_vulnerability_ids_template, +) +from dojo.finding.models import BurpRawRequestResponse +from dojo.jira import services as jira_services +from dojo.jira.api.serializers import JIRAIssueSerializer +from dojo.location.models import LocationFindingReference +from dojo.models import ( + SEVERITIES, + Development_Environment, + Dojo_User, + DojoMeta, + Endpoint, + Engagement, + Finding, + Finding_Group, + Finding_Template, + Note_Type, + Product, + Product_Type, + Test, + Test_Type, + User, + Vulnerability_Id, +) +from dojo.notifications.helper import async_create_notification +from dojo.user.queries import get_authorized_users + +logger = logging.getLogger(__name__) + + +class RequestResponseDict(collections.UserList): + def __init__(self, *args, **kwargs): + pretty_print = kwargs.pop("pretty_print", True) + collections.UserList.__init__(self, *args, **kwargs) + self.pretty_print = pretty_print + + def __add__(self, rhs): + return RequestResponseDict(list.__add__(self, rhs)) + + def __getitem__(self, item): + result = list.__getitem__(self, item) + try: + return RequestResponseDict(result) + except TypeError: + return result + + def __str__(self): + if self.pretty_print: + return json.dumps( + self, sort_keys=True, indent=4, separators=(",", ": "), + ) + return json.dumps(self) + + +class RequestResponseSerializerField(serializers.ListSerializer): + child = DictField(child=serializers.CharField()) + default_error_messages = { + "not_a_list": _( + 'Expected a list of items but got type "{input_type}".', + ), + "invalid_json": _( + "Invalid json list. A tag list submitted in string" + " form must be valid json.", + ), + "not_a_dict": _( + "All list items must be of dict type with keys 'request' and 'response'", + ), + "not_a_str": _("All values in the dict must be of string type."), + } + order_by = None + + def __init__(self, **kwargs): + pretty_print = kwargs.pop("pretty_print", True) + + style = kwargs.pop("style", {}) + kwargs["style"] = {"base_template": "textarea.html"} + kwargs["style"].update(style) + + if "data" in kwargs: + data = kwargs["data"] + + if isinstance(data, list): + kwargs["many"] = True + + super().__init__(**kwargs) + + self.pretty_print = pretty_print + + def to_internal_value(self, data): + if isinstance(data, six.string_types): + if not data: + data = [] + try: + data = json.loads(data) + except ValueError: + self.fail("invalid_json") + + if not isinstance(data, list): + self.fail("not_a_list", input_type=type(data).__name__) + for s in data: + if not isinstance(s, dict): + self.fail("not_a_dict", input_type=type(s).__name__) + + request = s.get("request", None) + response = s.get("response", None) + + if not isinstance(request, str): + self.fail("not_a_str", input_type=type(request).__name__) + if not isinstance(response, str): + self.fail("not_a_str", input_type=type(request).__name__) + + self.child.run_validation(s) + return data + + def to_representation(self, value): + if not isinstance(value, RequestResponseDict): + if not isinstance(value, list): + # this will trigger when a queryset is found... + burps = value.all().order_by(*self.order_by) if self.order_by else value.all() + value = [ + { + "request": burp.get_request(), + "response": burp.get_response(), + } + for burp in burps + ] + + return value + + +class BurpRawRequestResponseSerializer(serializers.Serializer): + req_resp = RequestResponseSerializerField(required=True) + + +class BurpRawRequestResponseMultiSerializer(serializers.ModelSerializer): + burpRequestBase64 = serializers.CharField() + burpResponseBase64 = serializers.CharField() + + def to_representation(self, data): + return { + "id": data.id, + "finding": data.finding.id, + "burpRequestBase64": data.burpRequestBase64.decode("utf-8"), + "burpResponseBase64": data.burpResponseBase64.decode("utf-8"), + } + + def validate(self, data): + b64request = data.get("burpRequestBase64", None) + b64response = data.get("burpResponseBase64", None) + finding = data.get("finding", None) + # Make sure all fields are present + if not b64request or not b64response or not finding: + msg = "burpRequestBase64, burpResponseBase64, and finding are required." + raise ValidationError(msg) + # Verify we have true base64 decoding + try: + base64.b64decode(b64request, validate=True) + base64.b64decode(b64response, validate=True) + except Exception as e: + msg = "Inputs need to be valid base64 encodings" + raise ValidationError(msg) from e + # Encode the data in utf-8 to remove any bad characters + data["burpRequestBase64"] = b64request.encode("utf-8") + data["burpResponseBase64"] = b64response.encode("utf-8") + # Run the model validation - an ValidationError will be raised if there is an issue + BurpRawRequestResponse(finding=finding, burpRequestBase64=b64request, burpResponseBase64=b64response).clean() + + return data + + class Meta: + model = BurpRawRequestResponse + fields = "__all__" + + +class FindingGroupSerializer(serializers.ModelSerializer): + jira_issue = JIRAIssueSerializer(read_only=True, allow_null=True) + + class Meta: + model = Finding_Group + fields = ("id", "name", "test", "jira_issue") + + +class FindingMetaSerializer(serializers.ModelSerializer): + class Meta: + model = DojoMeta + fields = ("name", "value") + + +class FindingProdTypeSerializer(serializers.ModelSerializer): + class Meta: + model = Product_Type + fields = ["id", "name"] + + +class FindingProductSerializer(serializers.ModelSerializer): + prod_type = FindingProdTypeSerializer(required=False) + + class Meta: + model = Product + fields = ["id", "name", "prod_type"] + + +class FindingEngagementSerializer(serializers.ModelSerializer): + product = FindingProductSerializer(required=False) + + class Meta: + model = Engagement + fields = [ + "id", + "name", + "description", + "product", + "target_start", + "target_end", + "branch_tag", + "engagement_type", + "build_id", + "commit_hash", + "version", + "created", + "updated", + ] + + +class FindingEnvironmentSerializer(serializers.ModelSerializer): + class Meta: + model = Development_Environment + fields = ["id", "name"] + + +class FindingTestTypeSerializer(serializers.ModelSerializer): + class Meta: + model = Test_Type + fields = ["id", "name"] + + +class FindingTestSerializer(serializers.ModelSerializer): + engagement = FindingEngagementSerializer(required=False) + environment = FindingEnvironmentSerializer(required=False) + test_type = FindingTestTypeSerializer(required=False) + + class Meta: + model = Test + fields = [ + "id", + "title", + "test_type", + "engagement", + "environment", + "branch_tag", + "build_id", + "commit_hash", + "version", + ] + + +class FindingRelatedFieldsSerializer(serializers.Serializer): + test = serializers.SerializerMethodField() + jira = serializers.SerializerMethodField() + + @extend_schema_field(FindingTestSerializer) + def get_test(self, obj): + return FindingTestSerializer(read_only=True).to_representation( + obj.test, + ) + + @extend_schema_field(JIRAIssueSerializer) + def get_jira(self, obj): + issue = jira_services.get_issue(obj) + if issue is None: + return None + return JIRAIssueSerializer(read_only=True).to_representation(issue) + + +class VulnerabilityIdSerializer(serializers.ModelSerializer): + class Meta: + model = Vulnerability_Id + fields = ["vulnerability_id"] + + +class FindingSerializer(serializers.ModelSerializer): + mitigated = serializers.DateTimeField(required=False, allow_null=True) + mitigated_by = serializers.PrimaryKeyRelatedField(required=False, allow_null=True, queryset=User.objects.all()) + request_response = serializers.SerializerMethodField() + accepted_risks = serializers.SerializerMethodField() + push_to_jira = serializers.BooleanField(default=False) + found_by = serializers.PrimaryKeyRelatedField( + queryset=Test_Type.objects.all(), many=True, + ) + age = serializers.IntegerField(read_only=True) + sla_days_remaining = serializers.IntegerField(read_only=True, allow_null=True) + finding_meta = FindingMetaSerializer(read_only=True, many=True) + related_fields = serializers.SerializerMethodField(allow_null=True) + # for backwards compatibility + jira_creation = serializers.SerializerMethodField(read_only=True, allow_null=True) + jira_change = serializers.SerializerMethodField(read_only=True, allow_null=True) + display_status = serializers.SerializerMethodField() + finding_groups = FindingGroupSerializer( + source="finding_group_set", many=True, read_only=True, + ) + vulnerability_ids = VulnerabilityIdSerializer( + source="vulnerability_id_set", many=True, required=False, + ) + reporter = serializers.PrimaryKeyRelatedField( + required=False, queryset=User.objects.all(), + ) + endpoints = serializers.PrimaryKeyRelatedField( + source="locations", + many=True, + required=False, + queryset=LocationFindingReference.objects.all(), + ) + + class Meta: + model = Finding + exclude = ( + "cve", + "inherited_tags", + ) + + def get_fields(self): + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + TagListSerializerField, + ) + fields = super().get_fields() + fields["tags"] = TagListSerializerField(required=False) + return fields + + # TODO: Delete this after the move to Locations + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + if not settings.V3_FEATURE_LOCATIONS: + self.fields["endpoints"] = serializers.PrimaryKeyRelatedField( + many=True, required=False, queryset=Endpoint.objects.all(), + ) + + def get_accepted_risks(self, obj): + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + RiskAcceptanceSerializer, + ) + # schema annotation applied lazily at module bottom (avoids circular import) + request = self.context.get("request") + if request is None: + return [] + if not user_has_permission(request.user, obj, "edit"): + return [] + return RiskAcceptanceSerializer( + obj.risk_acceptance_set.all(), many=True, + ).data + + @extend_schema_field(serializers.DateTimeField()) + def get_jira_creation(self, obj): + return jira_services.get_creation(obj) + + @extend_schema_field(serializers.DateTimeField()) + def get_jira_change(self, obj): + return jira_services.get_change(obj) + + @extend_schema_field(FindingRelatedFieldsSerializer) + def get_related_fields(self, obj): + request = self.context.get("request", None) + if request is None: + return None + + query_params = request.query_params + if query_params.get("related_fields", "false") == "true": + return FindingRelatedFieldsSerializer( + required=False, + ).to_representation(obj) + return None + + def get_display_status(self, obj) -> str: + return obj.status() + + def process_risk_acceptance(self, data): + import dojo.risk_acceptance.helper as ra_helper # noqa: PLC0415 -- lazy import, avoids circular dependency + is_risk_accepted = data.get("risk_accepted") + # Do not take any action if the `risk_accepted` was not passed + if not isinstance(is_risk_accepted, bool): + return + # Determine how to proceed based on the value of `risk_accepted` + if is_risk_accepted and not self.instance.risk_accepted and self.instance.test.engagement.product.enable_simple_risk_acceptance and not data.get("active", False): + ra_helper.simple_risk_accept(self.context["request"].user, self.instance) + elif not is_risk_accepted and self.instance.risk_accepted: # turning off risk_accepted + ra_helper.risk_unaccept(self.context["request"].user, self.instance) + + # Overriding this to push add Push to JIRA functionality + def update(self, instance, validated_data): + # push_all_issues already checked in api views.py + push_to_jira = validated_data.pop("push_to_jira") + + # Save vulnerability ids and pop them + parsed_vulnerability_ids = [] + if (vulnerability_ids := validated_data.pop("vulnerability_id_set", None)): + logger.debug("VULNERABILITY_ID_SET: %s", vulnerability_ids) + parsed_vulnerability_ids.extend(vulnerability_id["vulnerability_id"] for vulnerability_id in vulnerability_ids) + logger.debug("SETTING CVE FROM VULNERABILITY_ID_SET: %s", parsed_vulnerability_ids[0]) + validated_data["cve"] = parsed_vulnerability_ids[0] + + # Save the reporter on the finding + if reporter_id := validated_data.get("reporter"): + instance.reporter = reporter_id + + # Persist vulnerability IDs first so model save computes hash including them (if there is no hash yet) + # we can't pass unsaved_vulnerabilitiy_ids to super.update() + if parsed_vulnerability_ids: + save_vulnerability_ids(instance, parsed_vulnerability_ids) + + # Get found_by from validated_data + found_by = validated_data.pop("found_by", None) + # Handle updates to found_by data + if found_by: + instance.found_by.set(found_by) + # If there is no argument entered for found_by, the user would like to clear out the values on the Finding's found_by field + # Findings still maintain original found_by value associated with their test + # In the event the user does not supply the found_by field at all, we do not modify it + elif isinstance(found_by, list) and len(found_by) == 0: + instance.found_by.clear() + + locations = None + if settings.V3_FEATURE_LOCATIONS: + locations = validated_data.pop("locations", None) + + instance = super().update( + instance, validated_data, + ) + + if settings.V3_FEATURE_LOCATIONS and locations is not None: + for location_ref in instance.locations.all(): + location_ref.location.disassociate_from_finding(instance) + for location_ref in locations: + location_ref.location.associate_with_finding(instance) + + if push_to_jira or jira_services.is_keep_in_sync(instance): + # Push synchronously so that we can see jira errors in real time + success, message = jira_services.push(instance, force_sync=True) + if not success: + raise serializers.ValidationError(message) + + return instance + + def validate(self, data): + # Enforce mitigated metadata editability (only when non-null values are provided) + attempting_to_set_mitigated = any( + (field in data) and (data.get(field) is not None) + for field in ["mitigated", "mitigated_by"] + ) + user = getattr(self.context.get("request", None), "user", None) + if attempting_to_set_mitigated and not finding_helper.can_edit_mitigated_data(user): + errors = {} + if ("mitigated" in data) and (data.get("mitigated") is not None): + errors["mitigated"] = ["Editing mitigated timestamp is disabled (EDITABLE_MITIGATED_DATA=false)"] + if ("mitigated_by" in data) and (data.get("mitigated_by") is not None): + errors["mitigated_by"] = ["Editing mitigated_by is disabled (EDITABLE_MITIGATED_DATA=false)"] + if errors: + raise serializers.ValidationError(errors) + + if self.context["request"].method == "PATCH": + is_active = data.get("active", self.instance.active) + is_verified = data.get("verified", self.instance.verified) + is_duplicate = data.get("duplicate", self.instance.duplicate) + is_false_p = data.get("false_p", self.instance.false_p) + is_risk_accepted = data.get( + "risk_accepted", self.instance.risk_accepted, + ) + else: + is_active = data.get("active", True) + is_verified = data.get("verified", False) + is_duplicate = data.get("duplicate", False) + is_false_p = data.get("false_p", False) + is_risk_accepted = data.get("risk_accepted", False) + + if (is_active or is_verified) and is_duplicate: + msg = "Duplicate findings cannot be verified or active" + raise serializers.ValidationError(msg) + if is_false_p and is_verified: + msg = "False positive findings cannot be verified." + raise serializers.ValidationError(msg) + + if is_risk_accepted and not self.instance.risk_accepted: + if ( + not self.instance.test.engagement.product.enable_simple_risk_acceptance + ): + msg = "Simple risk acceptance is disabled for this product, use the UI to accept this finding." + raise serializers.ValidationError(msg) + + if is_active and is_risk_accepted: + msg = "Active findings cannot be risk accepted." + raise serializers.ValidationError(msg) + + # assuming we made it past the validations,call risk acceptance properly to make sure notes, etc get created + # doing it here instead of in update because update doesn't know if the value changed + self.process_risk_acceptance(data) + + return data + + def validate_severity(self, value: str) -> str: + if value not in SEVERITIES: + msg = f"Severity must be one of the following: {SEVERITIES}" + raise serializers.ValidationError(msg) + return value + + def build_relational_field(self, field_name, relation_info): + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + NoteSerializer, + ) + if field_name == "notes": + return NoteSerializer, {"many": True, "read_only": True} + return super().build_relational_field(field_name, relation_info) + + def get_request_response(self, obj): + # Not necessarily Burp scan specific - these are just any request/response pairs + burp_req_resp = obj.burprawrequestresponse_set.all() + var = settings.MAX_REQRESP_FROM_API + if var > -1: + burp_req_resp = burp_req_resp[:var] + burp_list = [] + for burp in burp_req_resp: + request = burp.get_request() + response = burp.get_response() + burp_list.append({"request": request, "response": response}) + serialized_burps = BurpRawRequestResponseSerializer( + {"req_resp": burp_list}, + ) + return serialized_burps.data + + +class FindingCreateSerializer(serializers.ModelSerializer): + mitigated = serializers.DateTimeField(required=False, allow_null=True) + mitigated_by = serializers.PrimaryKeyRelatedField(required=False, allow_null=True, queryset=User.objects.all()) + notes = serializers.PrimaryKeyRelatedField( + read_only=True, allow_null=True, required=False, many=True, + ) + test = serializers.PrimaryKeyRelatedField(queryset=Test.objects.all()) + thread_id = serializers.IntegerField(default=0) + found_by = serializers.PrimaryKeyRelatedField( + queryset=Test_Type.objects.all(), many=True, + ) + url = serializers.CharField(allow_null=True, default=None) + push_to_jira = serializers.BooleanField(default=False) + vulnerability_ids = VulnerabilityIdSerializer( + source="vulnerability_id_set", many=True, required=False, + ) + reporter = serializers.PrimaryKeyRelatedField( + required=False, queryset=User.objects.all(), + ) + + class Meta: + model = Finding + exclude = ( + "cve", + "inherited_tags", + ) + extra_kwargs = { + "active": {"required": True}, + "verified": {"required": True}, + } + + def get_fields(self): + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + TagListSerializerField, + ) + fields = super().get_fields() + fields["tags"] = TagListSerializerField(required=False) + return fields + + # Overriding this to push add Push to JIRA functionality + def create(self, validated_data): + logger.debug("Creating finding with validated data: %s", validated_data) + push_to_jira = validated_data.pop("push_to_jira", False) + notes = validated_data.pop("notes", None) + found_by = validated_data.pop("found_by", None) + reviewers = validated_data.pop("reviewers", None) + # Process the vulnerability IDs specially + parsed_vulnerability_ids = [] + if (vulnerability_ids := validated_data.pop("vulnerability_id_set", None)): + logger.debug("VULNERABILITY_ID_SET: %s", vulnerability_ids) + parsed_vulnerability_ids.extend(vulnerability_id["vulnerability_id"] for vulnerability_id in vulnerability_ids) + logger.debug("PARSED_VULNERABILITY_IDST: %s", parsed_vulnerability_ids) + logger.debug("SETTING CVE FROM VULNERABILITY_ID_SET: %s", parsed_vulnerability_ids[0]) + validated_data["cve"] = parsed_vulnerability_ids[0] + # validated_data["unsaved_vulnerability_ids"] = parsed_vulnerability_ids + + # super.create() doesn't accept unsaved_vulnerability_ids or dedupe_option=False, so call save directly. + new_finding = Finding(**validated_data) + new_finding.unsaved_vulnerability_ids = parsed_vulnerability_ids or [] + new_finding.save() + + logger.debug(f"New finding CVE: {new_finding.cve}") + + # Deal with all of the many to many things + if notes: + new_finding.notes.set(notes) + if found_by: + new_finding.found_by.set(found_by) + if reviewers: + new_finding.reviewers.set(reviewers) + if parsed_vulnerability_ids: + save_vulnerability_ids(new_finding, parsed_vulnerability_ids) + + if push_to_jira: + jira_services.push(new_finding) + + # Create a notification + dojo_dispatch_task( + async_create_notification, + event="finding_added", + title=_("Addition of %s") % new_finding.title, + finding_id=new_finding.id, + description=_('Finding "%s" was added by %s') % (new_finding.title, new_finding.reporter), + url=reverse("view_finding", args=(new_finding.id,)), + icon="exclamation-triangle", + ) + + return new_finding + + def validate(self, data): + # Ensure mitigated fields are only set when editable is enabled (ignore nulls) + attempting_to_set_mitigated = any( + (field in data) and (data.get(field) is not None) + for field in ["mitigated", "mitigated_by"] + ) + user = getattr(getattr(self.context, "request", None), "user", None) + if attempting_to_set_mitigated and not finding_helper.can_edit_mitigated_data(user): + errors = {} + if ("mitigated" in data) and (data.get("mitigated") is not None): + errors["mitigated"] = ["Editing mitigated timestamp is disabled (EDITABLE_MITIGATED_DATA=false)"] + if ("mitigated_by" in data) and (data.get("mitigated_by") is not None): + errors["mitigated_by"] = ["Editing mitigated_by is disabled (EDITABLE_MITIGATED_DATA=false)"] + if errors: + raise serializers.ValidationError(errors) + + if "reporter" not in data: + request = self.context["request"] + data["reporter"] = request.user + + if (data.get("active") or data.get("verified")) and data.get( + "duplicate", + ): + msg = "Duplicate findings cannot be verified or active" + raise serializers.ValidationError(msg) + if data.get("false_p") and data.get("verified"): + msg = "False positive findings cannot be verified." + raise serializers.ValidationError(msg) + + if "risk_accepted" in data and data.get("risk_accepted"): + test = data.get("test") + # test = Test.objects.get(id=test_id) + if not test.engagement.product.enable_simple_risk_acceptance: + msg = "Simple risk acceptance is disabled for this product, use the UI to accept this finding." + raise serializers.ValidationError(msg) + + if ( + data.get("active") + and "risk_accepted" in data + and data.get("risk_accepted") + ): + msg = "Active findings cannot be risk accepted." + raise serializers.ValidationError(msg) + + return data + + def validate_severity(self, value: str) -> str: + if value not in SEVERITIES: + msg = f"Severity must be one of the following: {SEVERITIES}" + raise serializers.ValidationError(msg) + return value + + +class FindingTemplateSerializer(serializers.ModelSerializer): + vulnerability_ids = serializers.SerializerMethodField() + endpoints = serializers.SerializerMethodField() + + class Meta: + model = Finding_Template + exclude = ("cve", "vulnerability_ids_text") + + def get_fields(self): + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + TagListSerializerField, + ) + fields = super().get_fields() + fields["tags"] = TagListSerializerField(required=False) + return fields + + @extend_schema_field(serializers.ListField(child=serializers.CharField())) + def get_vulnerability_ids(self, obj): + """Return vulnerability IDs as a list of strings.""" + return obj.vulnerability_ids + + @extend_schema_field(serializers.ListField(child=serializers.CharField())) + def get_endpoints(self, obj): + """Return endpoints as a list of URL strings.""" + return obj.endpoints if hasattr(obj, "endpoints") else [] + + def create(self, validated_data): + + # Handle vulnerability_ids if provided as list + vulnerability_ids = None + if "vulnerability_ids" in self.initial_data: + vulnerability_ids = self.initial_data.get("vulnerability_ids", []) + if isinstance(vulnerability_ids, str): + # If it's a string, split by newlines + vulnerability_ids = [vid.strip() for vid in vulnerability_ids.split("\n") if vid.strip()] + elif not isinstance(vulnerability_ids, list): + vulnerability_ids = [] + + # Handle endpoints if provided as list + endpoint_urls = None + if "endpoints" in self.initial_data: + endpoint_urls = self.initial_data.get("endpoints", []) + if isinstance(endpoint_urls, str): + # If it's a string, split by newlines + endpoint_urls = [url.strip() for url in endpoint_urls.split("\n") if url.strip()] + elif not isinstance(endpoint_urls, list): + endpoint_urls = [] + + new_finding_template = super().create( + validated_data, + ) + + # Save vulnerability IDs using helper + if vulnerability_ids: + save_vulnerability_ids_template(new_finding_template, vulnerability_ids) + + # Save endpoints using helper + if endpoint_urls: + save_endpoints_template(new_finding_template, endpoint_urls) + + return new_finding_template + + def update(self, instance, validated_data): + # Handle vulnerability_ids if provided + if "vulnerability_ids" in self.initial_data: + vulnerability_ids = self.initial_data.get("vulnerability_ids", []) + if isinstance(vulnerability_ids, str): + vulnerability_ids = [vid.strip() for vid in vulnerability_ids.split("\n") if vid.strip()] + elif not isinstance(vulnerability_ids, list): + vulnerability_ids = [] + save_vulnerability_ids_template(instance, vulnerability_ids) + + # Handle endpoints if provided + if "endpoints" in self.initial_data: + endpoint_urls = self.initial_data.get("endpoints", []) + if isinstance(endpoint_urls, str): + endpoint_urls = [url.strip() for url in endpoint_urls.split("\n") if url.strip()] + elif not isinstance(endpoint_urls, list): + endpoint_urls = [] + save_endpoints_template(instance, endpoint_urls) + + return super().update(instance, validated_data) + + +class FindingToNotesSerializer(serializers.Serializer): + finding_id = serializers.PrimaryKeyRelatedField( + queryset=Finding.objects.all(), many=False, allow_null=True, + ) + + def get_fields(self): + from dojo.api_v2.serializers import NoteSerializer # noqa: PLC0415 -- lazy import, avoids circular dependency + fields = super().get_fields() + fields["notes"] = NoteSerializer(many=True) + return fields + + +class FindingToFilesSerializer(serializers.Serializer): + finding_id = serializers.PrimaryKeyRelatedField( + queryset=Finding.objects.all(), many=False, allow_null=True, + ) + + def get_fields(self): + from dojo.api_v2.serializers import FileSerializer # noqa: PLC0415 -- lazy import, avoids circular dependency + fields = super().get_fields() + fields["files"] = FileSerializer(many=True) + return fields + + def to_representation(self, data): + finding = data.get("finding_id") + files = data.get("files") + new_files = [{ + "id": file.id, + "file": "{site_url}/{file_access_url}".format( + site_url=settings.SITE_URL, + file_access_url=file.get_accessible_url( + finding, finding.id, + ), + ), + "title": file.title, + } for file in files] + return {"finding_id": finding.id, "files": new_files} + + +class FindingCloseSerializer(serializers.ModelSerializer): + is_mitigated = serializers.BooleanField(required=False) + mitigated = serializers.DateTimeField(required=False) + false_p = serializers.BooleanField(required=False) + out_of_scope = serializers.BooleanField(required=False) + duplicate = serializers.BooleanField(required=False) + mitigated_by = serializers.PrimaryKeyRelatedField(required=False, allow_null=True, queryset=Dojo_User.objects.all()) + note = serializers.CharField(required=False, allow_blank=True) + note_type = serializers.PrimaryKeyRelatedField(required=False, allow_null=True, queryset=Note_Type.objects.all()) + + class Meta: + model = Finding + fields = ( + "is_mitigated", + "mitigated", + "false_p", + "out_of_scope", + "duplicate", + "mitigated_by", + "note", + "note_type", + ) + + def validate(self, data): + request = self.context.get("request") + request_user = getattr(request, "user", None) + + mitigated_by_user = data.get("mitigated_by") + if mitigated_by_user is not None: + # Require permission to edit mitigated metadata + if not (request_user and finding_helper.can_edit_mitigated_data(request_user)): + raise serializers.ValidationError({ + "mitigated_by": ["Not allowed to set mitigated_by."], + }) + + # Ensure selected user is authorized (Finding_Edit) + authorized_users = get_authorized_users("edit", user=request_user) + if not authorized_users.filter(id=mitigated_by_user.id).exists(): + raise serializers.ValidationError({ + "mitigated_by": [ + "Selected user is not authorized to be set as mitigated_by.", + ], + }) + + return data + + +class FindingVerifySerializer(serializers.Serializer): + note = serializers.CharField(required=False, allow_blank=True) + note_type = serializers.PrimaryKeyRelatedField(required=False, allow_null=True, queryset=Note_Type.objects.all()) + + +class FindingNoteSerializer(serializers.Serializer): + note_id = serializers.IntegerField() + + +def _apply_schema_overrides(): + # Apply @extend_schema_field annotations that reference serializers which remain + # in dojo.api_v2.serializers. These are applied here (rather than as class-body + # decorators) so the module carries no top-level dojo.api_v2.serializers import, + # which would create a circular dependency. drf-spectacular only reads these + # overrides at schema generation time, so applying them lazily on import is fine. + from drf_spectacular.utils import set_override # noqa: PLC0415 -- lazy import, avoids circular dependency + + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + RiskAcceptanceSerializer, + ) + set_override(FindingSerializer.get_accepted_risks, "field", RiskAcceptanceSerializer(many=True)) + set_override(FindingSerializer.get_request_response, "field", BurpRawRequestResponseSerializer) + + +_apply_schema_overrides() diff --git a/dojo/finding/api/urls.py b/dojo/finding/api/urls.py new file mode 100644 index 00000000000..c2d240d3a1a --- /dev/null +++ b/dojo/finding/api/urls.py @@ -0,0 +1,12 @@ +from dojo.finding.api.views import ( + BurpRawRequestResponseViewSet, + FindingTemplatesViewSet, + FindingViewSet, +) + + +def add_finding_urls(router): + router.register("finding_templates", FindingTemplatesViewSet, basename="finding_template") + router.register("findings", FindingViewSet, basename="finding") + router.register("request_response_pairs", BurpRawRequestResponseViewSet, basename="request_response_pairs") + return router diff --git a/dojo/finding/api/views.py b/dojo/finding/api/views.py new file mode 100644 index 00000000000..d40f72fd165 --- /dev/null +++ b/dojo/finding/api/views.py @@ -0,0 +1,862 @@ +import base64 +import logging + +import tagulous +from django.conf import settings +from django.core.exceptions import ValidationError +from django.db import IntegrityError +from django.shortcuts import get_object_or_404 +from django.urls import reverse +from django.utils import timezone +from django_filters.rest_framework import DjangoFilterBackend +from drf_spectacular.types import OpenApiTypes +from drf_spectacular.utils import ( + OpenApiParameter, + OpenApiResponse, + extend_schema, + extend_schema_view, +) +from rest_framework import mixins, status, viewsets +from rest_framework.decorators import action +from rest_framework.parsers import MultiPartParser +from rest_framework.permissions import IsAuthenticated +from rest_framework.response import Response + +import dojo.finding.helper as finding_helper +from dojo.api_v2 import ( + mixins as dojo_mixins, +) +from dojo.api_v2 import ( + prefetch, +) +from dojo.api_v2 import ( + serializers as api_v2_serializers, +) +from dojo.api_v2.views import DojoModelViewSet, report_generate +from dojo.authorization import api_permissions as permissions +from dojo.finding.api.filters import ApiFindingFilter, ApiTemplateFindingFilter +from dojo.finding.api.serializer import ( + BurpRawRequestResponseMultiSerializer, + BurpRawRequestResponseSerializer, + FindingCloseSerializer, + FindingCreateSerializer, + FindingMetaSerializer, + FindingNoteSerializer, + FindingSerializer, + FindingTemplateSerializer, + FindingToFilesSerializer, + FindingToNotesSerializer, + FindingVerifySerializer, +) +from dojo.finding.queries import get_authorized_findings +from dojo.finding.ui.views import ( + duplicate_cluster, + reset_finding_duplicate_status_internal, + set_finding_as_original_internal, +) +from dojo.jira import services as jira_services +from dojo.models import ( + BurpRawRequestResponse, + DojoMeta, + FileUpload, + Finding, + Finding_Template, + NoteHistory, + Notes, +) +from dojo.risk_acceptance import api as ra_api +from dojo.utils import ( + generate_file_response, + get_system_setting, + process_tag_notifications, +) + +logger = logging.getLogger(__name__) + + +# Authorization: configuration +class FindingTemplatesViewSet( + DojoModelViewSet, +): + serializer_class = FindingTemplateSerializer + queryset = Finding_Template.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_class = ApiTemplateFindingFilter + permission_classes = (permissions.UserHasConfigurationPermissionStaff,) + + def get_queryset(self): + return Finding_Template.objects.all().order_by("id") + + +# Authorization: object-based +@extend_schema_view( + list=extend_schema( + parameters=[ + OpenApiParameter( + "related_fields", + OpenApiTypes.BOOL, + OpenApiParameter.QUERY, + required=False, + description="Expand finding external relations (engagement, environment, product, \ + product_type, test, test_type)", + ), + OpenApiParameter( + "prefetch", + OpenApiTypes.STR, + OpenApiParameter.QUERY, + required=False, + description="List of fields for which to prefetch model instances and add those to the response", + ), + ], + ), + retrieve=extend_schema( + parameters=[ + OpenApiParameter( + "related_fields", + OpenApiTypes.BOOL, + OpenApiParameter.QUERY, + required=False, + description="Expand finding external relations (engagement, environment, product, \ + product_type, test, test_type)", + ), + OpenApiParameter( + "prefetch", + OpenApiTypes.STR, + OpenApiParameter.QUERY, + required=False, + description="List of fields for which to prefetch model instances and add those to the response", + ), + ], + ), +) +class FindingViewSet( + prefetch.PrefetchListMixin, + prefetch.PrefetchRetrieveMixin, + mixins.UpdateModelMixin, + mixins.DestroyModelMixin, + mixins.CreateModelMixin, + ra_api.AcceptedFindingsMixin, + viewsets.GenericViewSet, + dojo_mixins.DeletePreviewModelMixin, +): + serializer_class = FindingSerializer + queryset = Finding.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_class = ApiFindingFilter + permission_classes = ( + IsAuthenticated, + permissions.UserHasFindingPermission, + ) + + # Overriding mixins.UpdateModeMixin perform_update() method to grab push_to_jira + # data and add that as a parameter to .save() + def perform_update(self, serializer): + # IF JIRA is enabled and this product has a JIRA configuration + push_to_jira = serializer.validated_data.get("push_to_jira") + jira_project = jira_services.get_project(serializer.instance) + if get_system_setting("enable_jira") and jira_project: + push_to_jira = push_to_jira or jira_project.push_all_issues + + serializer.save(push_to_jira=push_to_jira) + + def get_queryset(self): + if settings.V3_FEATURE_LOCATIONS: + findings = get_authorized_findings( + "view", + ).prefetch_related( + "locations__location__url", + "reviewers", + "found_by", + "notes", + "risk_acceptance_set", + "test", + "tags", + "jira_issue", + "finding_group_set", + "files", + "burprawrequestresponse_set", + "status_finding", + "finding_meta", + "test__test_type", + "test__engagement", + "test__environment", + "test__engagement__product", + "test__engagement__product__prod_type", + ) + else: + # TODO: Delete this after the move to Locations + findings = get_authorized_findings( + "view", + ).prefetch_related( + "endpoints", + "reviewers", + "found_by", + "notes", + "risk_acceptance_set", + "test", + "tags", + "jira_issue", + "finding_group_set", + "files", + "burprawrequestresponse_set", + "status_finding", + "finding_meta", + "test__test_type", + "test__engagement", + "test__environment", + "test__engagement__product", + "test__engagement__product__prod_type", + ) + + return findings.distinct() + + def get_serializer_class(self): + if self.request and self.request.method == "POST": + return FindingCreateSerializer + return FindingSerializer + + @extend_schema( + methods=["POST"], + request=FindingCloseSerializer, + responses={status.HTTP_200_OK: FindingCloseSerializer}, + ) + @action(detail=True, methods=["post"], permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission)) + def close(self, request, pk=None): + finding = self.get_object() + + if request.method == "POST": + finding_close = FindingCloseSerializer( + data=request.data, + context={"request": request}, + ) + if finding_close.is_valid(): + # Remove the prefetched tags to avoid issues with delegating to celery + finding.tags._remove_prefetched_objects() + # Use shared helper to perform close operations + finding_helper.close_finding( + finding=finding, + user=request.user, + is_mitigated=finding_close.validated_data["is_mitigated"], + mitigated=(finding_close.validated_data.get("mitigated") if finding_helper.can_edit_mitigated_data(request.user) else timezone.now()), + mitigated_by=finding_close.validated_data.get("mitigated_by") or (request.user if not finding_helper.can_edit_mitigated_data(request.user) else None), + false_p=finding_close.validated_data.get("false_p", False), + out_of_scope=finding_close.validated_data.get("out_of_scope", False), + duplicate=finding_close.validated_data.get("duplicate", False), + note_entry=finding_close.validated_data.get("note"), + note_type=finding_close.validated_data.get("note_type"), + ) + else: + return Response( + finding_close.errors, status=status.HTTP_400_BAD_REQUEST, + ) + serialized_finding = FindingCloseSerializer(finding, context={"request": request}) + return Response(serialized_finding.data) + + @extend_schema( + methods=["POST"], + request=FindingVerifySerializer, + responses={status.HTTP_200_OK: FindingSerializer}, + ) + @action(detail=True, methods=["post"], permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission)) + def verify(self, request, pk=None): + finding = self.get_object() + + serializer = FindingVerifySerializer(data=request.data) + if not serializer.is_valid(): + return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST) + + # Remove prefetched tags to keep queryset state in sync + finding.tags._remove_prefetched_objects() + + finding_helper.verify_finding( + finding=finding, + user=request.user, + note_entry=serializer.validated_data.get("note"), + note_type=serializer.validated_data.get("note_type"), + ) + + serialized_finding = FindingSerializer(finding, context={"request": request}) + return Response(serialized_finding.data) + + @extend_schema( + methods=["GET"], + responses={status.HTTP_200_OK: api_v2_serializers.TagSerializer}, + ) + @extend_schema( + methods=["POST"], + request=api_v2_serializers.TagSerializer, + responses={status.HTTP_201_CREATED: api_v2_serializers.TagSerializer}, + ) + @action(detail=True, methods=["get", "post"], permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission)) + def tags(self, request, pk=None): + finding = self.get_object() + + if request.method == "POST": + new_tags = api_v2_serializers.TagSerializer(data=request.data) + if new_tags.is_valid(): + all_tags = finding.tags + all_tags = api_v2_serializers.TagSerializer({"tags": all_tags}).data[ + "tags" + ] + for tag in new_tags.validated_data["tags"]: + for sub_tag in tagulous.utils.parse_tags(tag): + if sub_tag not in all_tags: + all_tags.append(sub_tag) + + new_tags = tagulous.utils.render_tags(all_tags) + + finding.tags = new_tags + finding.save() + else: + return Response( + new_tags.errors, status=status.HTTP_400_BAD_REQUEST, + ) + tags = finding.tags + serialized_tags = api_v2_serializers.TagSerializer({"tags": tags}) + return Response(serialized_tags.data) + + @extend_schema( + methods=["GET"], + responses={ + status.HTTP_200_OK: BurpRawRequestResponseSerializer, + }, + ) + @extend_schema( + methods=["POST"], + request=BurpRawRequestResponseSerializer, + responses={ + status.HTTP_201_CREATED: BurpRawRequestResponseSerializer, + }, + ) + @action(detail=True, methods=["get", "post"], permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission)) + def request_response(self, request, pk=None): + finding = self.get_object() + + if request.method == "POST": + burps = BurpRawRequestResponseSerializer( + data=request.data, many=isinstance(request.data, list), + ) + if burps.is_valid(): + for pair in burps.validated_data["req_resp"]: + burp_rr = BurpRawRequestResponse( + finding=finding, + burpRequestBase64=base64.b64encode( + pair["request"].encode("utf-8"), + ), + burpResponseBase64=base64.b64encode( + pair["response"].encode("utf-8"), + ), + ) + burp_rr.clean() + burp_rr.save() + else: + return Response( + burps.errors, status=status.HTTP_400_BAD_REQUEST, + ) + # Not necessarily Burp scan specific - these are just any request/response pairs + burp_req_resp = BurpRawRequestResponse.objects.filter(finding=finding) + var = settings.MAX_REQRESP_FROM_API + if var > -1: + burp_req_resp = burp_req_resp[:var] + + burp_list = [] + for burp in burp_req_resp: + request = burp.get_request() + response = burp.get_response() + burp_list.append({"request": request, "response": response}) + serialized_burps = BurpRawRequestResponseSerializer( + {"req_resp": burp_list}, + ) + return Response(serialized_burps.data) + + @extend_schema( + methods=["GET"], + responses={status.HTTP_200_OK: FindingToNotesSerializer}, + ) + @extend_schema( + methods=["POST"], + request=api_v2_serializers.AddNewNoteOptionSerializer, + responses={status.HTTP_201_CREATED: api_v2_serializers.NoteSerializer}, + ) + @action(detail=True, methods=["get", "post"], permission_classes=(IsAuthenticated, permissions.UserHasFindingNotePermission)) + def notes(self, request, pk=None): + finding = self.get_object() + if request.method == "POST": + new_note = api_v2_serializers.AddNewNoteOptionSerializer( + data=request.data, + ) + if new_note.is_valid(): + entry = new_note.validated_data["entry"] + private = new_note.validated_data.get("private", False) + note_type = new_note.validated_data.get("note_type", None) + else: + return Response( + new_note.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + if finding.notes: + notes = finding.notes.filter(note_type=note_type).first() + if notes and note_type and note_type.is_single: + return Response("Only one instance of this note_type allowed on a finding.", status=status.HTTP_400_BAD_REQUEST) + + author = request.user + note = Notes( + entry=entry, + author=author, + private=private, + note_type=note_type, + ) + note.save() + # Add an entry to the note history + history = NoteHistory.objects.create(data=note.entry, time=note.date, current_editor=note.author) + note.history.add(history) + # Now add the note to the object + finding.last_reviewed = note.date + finding.last_reviewed_by = author + finding.save(update_fields=["last_reviewed", "last_reviewed_by", "updated"]) + finding.notes.add(note) + # Determine if we need to send any notifications for user mentioned + process_tag_notifications( + request=request, + note=note, + parent_url=request.build_absolute_uri( + reverse("view_finding", args=(finding.id,)), + ), + parent_title=f"Finding: {finding.title}", + ) + + if finding.has_jira_issue: + jira_services.add_comment(finding, note) + elif finding.has_jira_group_issue: + jira_services.add_comment(finding.finding_group, note) + + serialized_note = api_v2_serializers.NoteSerializer( + {"author": author, "entry": entry, "private": private}, + ) + return Response( + serialized_note.data, status=status.HTTP_201_CREATED, + ) + notes = finding.notes.all() + + serialized_notes = FindingToNotesSerializer( + {"finding_id": finding, "notes": notes}, + ) + return Response(serialized_notes.data, status=status.HTTP_200_OK) + + @extend_schema( + methods=["GET"], + responses={status.HTTP_200_OK: FindingToFilesSerializer}, + ) + @extend_schema( + methods=["POST"], + request=api_v2_serializers.AddNewFileOptionSerializer, + responses={status.HTTP_201_CREATED: api_v2_serializers.FileSerializer}, + ) + @action( + detail=True, methods=["get", "post"], parser_classes=(MultiPartParser,), permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission), + ) + def files(self, request, pk=None): + finding = self.get_object() + if request.method == "POST": + new_file = api_v2_serializers.FileSerializer(data=request.data) + if new_file.is_valid(): + title = new_file.validated_data["title"] + file = new_file.validated_data["file"] + else: + return Response( + new_file.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + file = FileUpload(title=title, file=file) + file.save() + finding.files.add(file) + + serialized_file = api_v2_serializers.FileSerializer(file) + return Response( + serialized_file.data, status=status.HTTP_201_CREATED, + ) + + files = finding.files.all() + serialized_files = FindingToFilesSerializer( + {"finding_id": finding, "files": files}, + ) + return Response(serialized_files.data, status=status.HTTP_200_OK) + + @extend_schema( + methods=["GET"], + responses={ + status.HTTP_200_OK: api_v2_serializers.RawFileSerializer, + }, + ) + @action( + detail=True, + methods=["get"], + url_path=r"files/download/(?P\d+)", permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission), + ) + def download_file(self, request, file_id, pk=None): + finding = self.get_object() + # Get the file object + file_object_qs = finding.files.filter(id=file_id) + file_object = ( + file_object_qs.first() if len(file_object_qs) > 0 else None + ) + if file_object is None: + return Response( + {"error": "File ID not associated with Finding"}, + status=status.HTTP_404_NOT_FOUND, + ) + # send file + return generate_file_response(file_object) + + @extend_schema( + request=FindingNoteSerializer, + responses={status.HTTP_204_NO_CONTENT: ""}, + ) + @action(detail=True, methods=["patch"], permission_classes=(IsAuthenticated, permissions.UserHasFindingNotePermission)) + def remove_note(self, request, pk=None): + """Remove Note From Finding Note""" + finding = self.get_object() + notes = finding.notes.all() + if request.data["note_id"]: + note = get_object_or_404(Notes.objects, id=request.data["note_id"]) + if note not in notes: + return Response( + {"error": "Selected Note is not assigned to this Finding"}, + status=status.HTTP_400_BAD_REQUEST, + ) + else: + return Response( + {"error": "('note_id') parameter missing"}, + status=status.HTTP_400_BAD_REQUEST, + ) + if ( + note.author.username == request.user.username + or request.user.is_superuser + ): + finding.notes.remove(note) + note.delete() + else: + return Response( + {"error": "Delete Failed, You are not the Note's author"}, + status=status.HTTP_400_BAD_REQUEST, + ) + + return Response( + {"Success": "Selected Note has been Removed successfully"}, + status=status.HTTP_204_NO_CONTENT, + ) + + @extend_schema( + methods=["PUT", "PATCH"], + request=api_v2_serializers.TagSerializer, + responses={status.HTTP_204_NO_CONTENT: ""}, + ) + @action(detail=True, methods=["put", "patch"], permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission)) + def remove_tags(self, request, pk=None): + """Remove Tag(s) from finding list of tags""" + finding = self.get_object() + delete_tags = api_v2_serializers.TagSerializer(data=request.data) + if delete_tags.is_valid(): + all_tags = finding.tags + all_tags = api_v2_serializers.TagSerializer({"tags": all_tags}).data[ + "tags" + ] + + # serializer turns it into a string, but we need a list + del_tags = delete_tags.validated_data["tags"] + if len(del_tags) < 1: + return Response( + {"error": "Empty Tag List Not Allowed"}, + status=status.HTTP_400_BAD_REQUEST, + ) + + for tag in del_tags: + if tag not in all_tags: + return Response( + { + "error": f"'{tag}' is not a valid tag in list '{all_tags}'", + }, + status=status.HTTP_400_BAD_REQUEST, + ) + all_tags.remove(tag) + new_tags = tagulous.utils.render_tags(all_tags) + finding.tags = new_tags + finding.save() + return Response( + {"success": "Tag(s) Removed"}, + status=status.HTTP_204_NO_CONTENT, + ) + return Response( + delete_tags.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + @extend_schema( + responses={ + status.HTTP_200_OK: FindingSerializer(many=True), + }, + ) + @action( + detail=True, + methods=["get"], + url_path=r"duplicate", + filter_backends=[], + pagination_class=None, + permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission), + ) + def get_duplicate_cluster(self, request, pk): + finding = self.get_object() + result = duplicate_cluster(request, finding) + serializer = FindingSerializer( + instance=result, many=True, context={"request": request}, + ) + return Response(serializer.data, status=status.HTTP_200_OK) + + @extend_schema( + request=OpenApiTypes.NONE, + responses={status.HTTP_204_NO_CONTENT: ""}, + ) + @action(detail=True, methods=["post"], url_path=r"duplicate/reset", permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission)) + def reset_finding_duplicate_status(self, request, pk): + self.get_object() + checked_duplicate_id = reset_finding_duplicate_status_internal( + request.user, pk, + ) + if checked_duplicate_id is None: + return Response(status=status.HTTP_400_BAD_REQUEST) + return Response(status=status.HTTP_204_NO_CONTENT) + + @extend_schema( + request=OpenApiTypes.NONE, + parameters=[ + OpenApiParameter( + "new_fid", OpenApiTypes.INT, OpenApiParameter.PATH, + ), + ], + responses={status.HTTP_204_NO_CONTENT: ""}, + ) + @action( + detail=True, methods=["post"], url_path=r"original/(?P\d+)", permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission), + ) + def set_finding_as_original(self, request, pk, new_fid): + self.get_object() + success = set_finding_as_original_internal(request.user, pk, new_fid) + if not success: + return Response(status=status.HTTP_400_BAD_REQUEST) + return Response(status=status.HTTP_204_NO_CONTENT) + + @extend_schema( + request=api_v2_serializers.ReportGenerateOptionSerializer, + responses={status.HTTP_200_OK: api_v2_serializers.ReportGenerateSerializer}, + ) + @action( + detail=False, methods=["post"], + # IsAuthenticated only: report generation requires View permission, + # enforced by the permission-filtered get_queryset(). The viewset's + # permission_classes would check Edit (POST), which is too restrictive. + permission_classes=[IsAuthenticated], + ) + def generate_report(self, request): + findings = self.get_queryset() + options = {} + # prepare post data + report_options = api_v2_serializers.ReportGenerateOptionSerializer( + data=request.data, + ) + if report_options.is_valid(): + options["include_finding_notes"] = report_options.validated_data[ + "include_finding_notes" + ] + options["include_finding_images"] = report_options.validated_data[ + "include_finding_images" + ] + options[ + "include_executive_summary" + ] = report_options.validated_data["include_executive_summary"] + options[ + "include_table_of_contents" + ] = report_options.validated_data["include_table_of_contents"] + else: + return Response( + report_options.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + data = report_generate(request, findings, options) + report = api_v2_serializers.ReportGenerateSerializer(data) + return Response(report.data) + + def _get_metadata(self, request, finding): + metadata = DojoMeta.objects.filter(finding=finding) + serializer = FindingMetaSerializer( + instance=metadata, many=True, + ) + return Response(serializer.data, status=status.HTTP_200_OK) + + def _edit_metadata(self, request, finding): + metadata_name = request.query_params.get("name", None) + if metadata_name is None: + return Response( + "Metadata name is required", status=status.HTTP_400_BAD_REQUEST, + ) + + try: + DojoMeta.objects.update_or_create( + name=metadata_name, + finding=finding, + defaults={ + "name": request.data.get("name"), + "value": request.data.get("value"), + }, + ) + + return Response(data=request.data, status=status.HTTP_200_OK) + except IntegrityError: + return Response( + "Update failed because the new name already exists", + status=status.HTTP_400_BAD_REQUEST, + ) + + def _add_metadata(self, request, finding): + metadata_data = FindingMetaSerializer(data=request.data) + + if metadata_data.is_valid(): + name = metadata_data.validated_data["name"] + value = metadata_data.validated_data["value"] + + metadata = DojoMeta(finding=finding, name=name, value=value) + try: + metadata.validate_unique() + metadata.save() + except ValidationError: + return Response( + "Create failed probably because the name of the metadata already exists", + status=status.HTTP_400_BAD_REQUEST, + ) + + return Response(data=metadata_data.data, status=status.HTTP_200_OK) + return Response( + metadata_data.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + def _remove_metadata(self, request, finding): + name = request.query_params.get("name", None) + if name is None: + return Response( + "A metadata name must be provided", + status=status.HTTP_400_BAD_REQUEST, + ) + + metadata = get_object_or_404( + DojoMeta.objects, finding=finding, name=name, + ) + metadata.delete() + + return Response("Metadata deleted", status=status.HTTP_200_OK) + + @extend_schema( + methods=["GET"], + responses={ + status.HTTP_200_OK: FindingMetaSerializer(many=True), + status.HTTP_404_NOT_FOUND: OpenApiResponse( + description="Returned if finding does not exist", + ), + }, + ) + @extend_schema( + methods=["DELETE"], + parameters=[ + OpenApiParameter( + "name", + OpenApiTypes.INT, + OpenApiParameter.QUERY, + required=True, + description="name of the metadata to retrieve. If name is empty, return all the \ + metadata associated with the finding", + ), + ], + responses={ + status.HTTP_200_OK: OpenApiResponse( + description="Returned if the metadata was correctly deleted", + ), + status.HTTP_404_NOT_FOUND: OpenApiResponse( + description="Returned if finding does not exist", + ), + status.HTTP_400_BAD_REQUEST: OpenApiResponse( + description="Returned if there was a problem with the metadata information", + ), + }, + ) + @extend_schema( + methods=["PUT"], + request=FindingMetaSerializer, + responses={ + status.HTTP_200_OK: FindingMetaSerializer, + status.HTTP_404_NOT_FOUND: OpenApiResponse( + description="Returned if finding does not exist", + ), + status.HTTP_400_BAD_REQUEST: OpenApiResponse( + description="Returned if there was a problem with the metadata information", + ), + }, + ) + @extend_schema( + methods=["POST"], + request=FindingMetaSerializer, + responses={ + status.HTTP_200_OK: FindingMetaSerializer, + status.HTTP_404_NOT_FOUND: OpenApiResponse( + description="Returned if finding does not exist", + ), + status.HTTP_400_BAD_REQUEST: OpenApiResponse( + description="Returned if there was a problem with the metadata information", + ), + }, + ) + @action( + detail=True, + methods=["post", "put", "delete", "get"], + filter_backends=[], + pagination_class=None, + permission_classes=(IsAuthenticated, permissions.UserHasFindingRelatedObjectPermission), + ) + def metadata(self, request, pk=None): + finding = self.get_object() + + if request.method == "GET": + return self._get_metadata(request, finding) + if request.method == "POST": + return self._add_metadata(request, finding) + if request.method in {"PUT", "PATCH"}: + return self._edit_metadata(request, finding) + if request.method == "DELETE": + return self._remove_metadata(request, finding) + + return Response( + {"error", "unsupported method"}, status=status.HTTP_400_BAD_REQUEST, + ) + + +class BurpRawRequestResponseViewSet( + DojoModelViewSet, +): + serializer_class = BurpRawRequestResponseMultiSerializer + queryset = BurpRawRequestResponse.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = ["finding"] + permission_classes = ( + IsAuthenticated, + permissions.UserHasBurpRawRequestResponsePermission, + ) + + def get_queryset(self): + return ( + BurpRawRequestResponse.objects.filter( + finding__in=get_authorized_findings( + "view", + ), + ) + .exclude( + burpRequestBase64__exact=b"", + burpResponseBase64__exact=b"", + ) + .order_by("id") + ) diff --git a/dojo/finding/models.py b/dojo/finding/models.py new file mode 100644 index 00000000000..337e2b97015 --- /dev/null +++ b/dojo/finding/models.py @@ -0,0 +1,1517 @@ +import base64 +import hashlib +import logging +import re +from contextlib import suppress +from datetime import datetime +from typing import TYPE_CHECKING + +import dateutil +from dateutil.parser import parse as datetutilsparse +from dateutil.relativedelta import relativedelta +from django.conf import settings +from django.core.validators import MaxValueValidator, MinValueValidator +from django.db import models +from django.urls import reverse +from django.utils import timezone +from django.utils.functional import cached_property +from django.utils.html import escape +from django.utils.translation import gettext as _ +from django_extensions.db.models import TimeStampedModel +from tagulous.models import TagField +from titlecase import titlecase + +from dojo.base_models.base import BaseModel + +# get_current_date/tomorrow/copy_model_util are defined early in dojo.models, before the +# re-export that loads this module — so this resolves despite the partial circular load, and +# keeps their dojo.models.* path for Django migration serialization (used as field defaults). +from dojo.models import copy_model_util, get_current_date, tomorrow +from dojo.validators import cvss3_validator, cvss4_validator + +if TYPE_CHECKING: + from dojo.importers.location_manager import UnsavedLocation + +logger = logging.getLogger(__name__) +deduplicationLogger = logging.getLogger("dojo.specific-loggers.deduplication") + + +class Finding(BaseModel): + # Fields loaded when performing deduplication (used by get_finding_models_for_deduplication + # and build_candidate_scope_queryset to restrict the SELECT to only what is needed). + # Covers the union of all deduplication algorithms so that a single queryset works + # regardless of which algorithm is in use. Large text fields (description, mitigation, + # impact, references, …) are intentionally excluded. + DEDUPLICATION_FIELDS = [ + "id", + # FK required for select_related("test") — must not be deferred + "test", + # Fields written by set_duplicate + "duplicate", + "active", + "verified", + "duplicate_finding", + # Guard checks in set_duplicate + "is_mitigated", + "mitigated", + "out_of_scope", + "false_p", + # Accessed by status() (debug logging only) + "under_review", + "risk_accepted", + # Used by hash-code and legacy algorithms for endpoint/location matching + "dynamic_finding", + "static_finding", + # Algorithm-specific matching fields + "hash_code", # hash_code, uid_or_hash, legacy + "unique_id_from_tool", # unique_id, uid_or_hash + "title", # legacy + "cwe", # legacy + "file_path", # legacy + "line", # legacy + ] + + # Large text fields deferred in build_candidate_scope_queryset. These are + # never accessed during deduplication or reimport candidate matching, so + # excluding them reduces the data loaded for every candidate finding. + DEDUPLICATION_DEFERRED_FIELDS = [ + "description", + "mitigation", + "impact", + "steps_to_reproduce", + "severity_justification", + "references", + "url", + "cvssv3", + "cvssv4", + ] + + title = models.CharField(max_length=511, + verbose_name=_("Title"), + help_text=_("A short description of the flaw.")) + date = models.DateField(default=get_current_date, + verbose_name=_("Date"), + help_text=_("The date the flaw was discovered.")) + sla_start_date = models.DateField( + blank=True, + null=True, + verbose_name=_("SLA Start Date"), + help_text=_("(readonly)The date used as start date for SLA calculation. Set by expiring risk acceptances. Empty by default, causing a fallback to 'date'.")) + sla_expiration_date = models.DateField( + blank=True, + null=True, + verbose_name=_("SLA Expiration Date"), + help_text=_("(readonly)The date SLA expires for this finding. Empty by default, causing a fallback to 'date'.")) + cwe = models.IntegerField(default=0, null=True, blank=True, + verbose_name=_("CWE"), + help_text=_("The CWE number associated with this flaw.")) + cve = models.CharField(max_length=50, + null=True, + blank=False, + verbose_name=_("Vulnerability Id"), + help_text=_("An id of a vulnerability in a security advisory associated with this finding. Can be a Common Vulnerabilities and Exposures (CVE) or from other sources.")) + epss_score = models.FloatField(default=None, null=True, blank=True, + verbose_name=_("EPSS Score"), + help_text=_("EPSS score for the CVE. Describes how likely it is the vulnerability will be exploited in the next 30 days."), + validators=[MinValueValidator(0.0), MaxValueValidator(1.0)]) + epss_percentile = models.FloatField(default=None, null=True, blank=True, + verbose_name=_("EPSS percentile"), + help_text=_("EPSS percentile for the CVE. Describes how many CVEs are scored at or below this one."), + validators=[MinValueValidator(0.0), MaxValueValidator(1.0)]) + known_exploited = models.BooleanField(default=False, + verbose_name=_("Known Exploited"), + help_text=_("Whether this vulnerability is known to have been exploited in the wild.")) + ransomware_used = models.BooleanField(default=False, + verbose_name=_("Used in Ransomware"), + help_text=_("Whether this vulnerability is known to have been leveraged as part of a ransomware campaign.")) + kev_date = models.DateField(null=True, blank=True, + verbose_name=_("KEV Date Added"), + help_text=_("The date the vulnerability was added to the KEV catalog."), + validators=[MaxValueValidator(tomorrow)]) + cvssv3 = models.TextField(validators=[cvss3_validator], + max_length=117, + null=True, + verbose_name=_("CVSS3 Vector"), + help_text=_("Common Vulnerability Scoring System version 3 (CVSS3) score associated with this finding.")) + cvssv3_score = models.FloatField(null=True, + blank=True, + verbose_name=_("CVSS3 Score"), + help_text=_("Numerical CVSSv3 score for the vulnerability. If the vector is given, the score is updated while saving the finding. The value must be between 0-10."), + validators=[MinValueValidator(0.0), MaxValueValidator(10.0)]) + + cvssv4 = models.TextField(validators=[cvss4_validator], + max_length=255, + null=True, + verbose_name=_("CVSS4 vector"), + help_text=_("Common Vulnerability Scoring System version 4 (CVSS4) score associated with this finding.")) + cvssv4_score = models.FloatField(null=True, + blank=True, + verbose_name=_("CVSSv4 Score"), + help_text=_("Numerical CVSSv4 score for the vulnerability. If the vector is given, the score is updated while saving the finding. The value must be between 0-10."), + validators=[MinValueValidator(0.0), MaxValueValidator(10.0)]) + + url = models.TextField(null=True, + blank=True, + editable=False, + verbose_name=_("URL"), + help_text=_("External reference that provides more information about this flaw.")) # not displayed and pretty much the same as references. To remove? + severity = models.CharField(max_length=200, + verbose_name=_("Severity"), + help_text=_("The severity level of this flaw (Critical, High, Medium, Low, Info).")) + description = models.TextField(verbose_name=_("Description"), + help_text=_("Longer more descriptive information about the flaw.")) + mitigation = models.TextField(verbose_name=_("Mitigation"), + null=True, + blank=True, + help_text=_("Text describing how to best fix the flaw.")) + fix_available = models.BooleanField(null=True, + default=None, + verbose_name=_("Fix Available"), + help_text=_("Denotes if there is a fix available for this flaw.")) + fix_version = models.CharField(null=True, + blank=True, + max_length=100, + verbose_name=_("Fix version"), + help_text=_("Version of the affected component in which the flaw is fixed.")) + impact = models.TextField(verbose_name=_("Impact"), + null=True, + blank=True, + help_text=_("Text describing the impact this flaw has on systems, products, enterprise, etc.")) + steps_to_reproduce = models.TextField(null=True, + blank=True, + verbose_name=_("Steps to Reproduce"), + help_text=_("Text describing the steps that must be followed in order to reproduce the flaw / bug.")) + severity_justification = models.TextField(null=True, + blank=True, + verbose_name=_("Severity Justification"), + help_text=_("Text describing why a certain severity was associated with this flaw.")) + # TODO: Delete this after the move to Locations + endpoints = models.ManyToManyField("dojo.Endpoint", + blank=True, + verbose_name=_("Endpoints"), + help_text=_("The hosts within the product that are susceptible to this flaw. + The status of the endpoint associated with this flaw (Vulnerable, Mitigated, ...)."), + through="dojo.Endpoint_Status") + references = models.TextField(null=True, + blank=True, + db_column="refs", + verbose_name=_("References"), + help_text=_("The external documentation available for this flaw.")) + test = models.ForeignKey("dojo.Test", + editable=False, + on_delete=models.CASCADE, + verbose_name=_("Test"), + help_text=_("The test that is associated with this flaw.")) + active = models.BooleanField(default=True, + verbose_name=_("Active"), + help_text=_("Denotes if this flaw is active or not.")) + # note that false positive findings cannot be verified + # in defectdojo verified means: "we have verified the finding and it turns out that it's not a false positive" + verified = models.BooleanField(default=False, + verbose_name=_("Verified"), + help_text=_("Denotes if this flaw has been manually verified by the tester.")) + false_p = models.BooleanField(default=False, + verbose_name=_("False Positive"), + help_text=_("Denotes if this flaw has been deemed a false positive by the tester.")) + duplicate = models.BooleanField(default=False, + verbose_name=_("Duplicate"), + help_text=_("Denotes if this flaw is a duplicate of other flaws reported.")) + duplicate_finding = models.ForeignKey("self", + editable=False, + null=True, + related_name="original_finding", + blank=True, on_delete=models.DO_NOTHING, + verbose_name=_("Duplicate Finding"), + help_text=_("Link to the original finding if this finding is a duplicate.")) + out_of_scope = models.BooleanField(default=False, + verbose_name=_("Out Of Scope"), + help_text=_("Denotes if this flaw falls outside the scope of the test and/or engagement.")) + risk_accepted = models.BooleanField(default=False, + verbose_name=_("Risk Accepted"), + help_text=_("Denotes if this finding has been marked as an accepted risk.")) + under_review = models.BooleanField(default=False, + verbose_name=_("Under Review"), + help_text=_("Denotes is this flaw is currently being reviewed.")) + + last_status_update = models.DateTimeField(editable=False, + null=True, + blank=True, + auto_now_add=True, + verbose_name=_("Last Status Update"), + help_text=_("Timestamp of latest status update (change in status related fields).")) + + review_requested_by = models.ForeignKey("dojo.Dojo_User", + null=True, + blank=True, + related_name="review_requested_by", + on_delete=models.RESTRICT, + verbose_name=_("Review Requested By"), + help_text=_("Documents who requested a review for this finding.")) + reviewers = models.ManyToManyField("dojo.Dojo_User", + blank=True, + verbose_name=_("Reviewers"), + help_text=_("Documents who reviewed the flaw.")) + + # Defect Tracking Review + under_defect_review = models.BooleanField(default=False, + verbose_name=_("Under Defect Review"), + help_text=_("Denotes if this finding is under defect review.")) + defect_review_requested_by = models.ForeignKey("dojo.Dojo_User", + null=True, + blank=True, + related_name="defect_review_requested_by", + on_delete=models.RESTRICT, + verbose_name=_("Defect Review Requested By"), + help_text=_("Documents who requested a defect review for this flaw.")) + is_mitigated = models.BooleanField(default=False, + verbose_name=_("Is Mitigated"), + help_text=_("Denotes if this flaw has been fixed.")) + thread_id = models.IntegerField(default=0, + editable=False, + verbose_name=_("Thread ID")) + mitigated = models.DateTimeField(editable=False, + null=True, + blank=True, + verbose_name=_("Mitigated"), + help_text=_("Denotes if this flaw has been fixed by storing the date it was fixed.")) + mitigated_by = models.ForeignKey("dojo.Dojo_User", + null=True, + editable=False, + related_name="mitigated_by", + on_delete=models.RESTRICT, + verbose_name=_("Mitigated By"), + help_text=_("Documents who has marked this flaw as fixed.")) + reporter = models.ForeignKey("dojo.Dojo_User", + editable=False, + default=1, + related_name="reporter", + on_delete=models.RESTRICT, + verbose_name=_("Reporter"), + help_text=_("Documents who reported the flaw.")) + notes = models.ManyToManyField("dojo.Notes", + blank=True, + editable=False, + verbose_name=_("Notes"), + help_text=_("Stores information pertinent to the flaw or the mitigation.")) + numerical_severity = models.CharField(max_length=4, + verbose_name=_("Numerical Severity"), + help_text=_("The numerical representation of the severity (S0, S1, S2, S3, S4).")) + last_reviewed = models.DateTimeField(null=True, + editable=False, + verbose_name=_("Last Reviewed"), + help_text=_("Provides the date the flaw was last 'touched' by a tester.")) + last_reviewed_by = models.ForeignKey("dojo.Dojo_User", + null=True, + editable=False, + related_name="last_reviewed_by", + on_delete=models.RESTRICT, + verbose_name=_("Last Reviewed By"), + help_text=_("Provides the person who last reviewed the flaw.")) + files = models.ManyToManyField("dojo.FileUpload", + blank=True, + editable=False, + verbose_name=_("Files"), + help_text=_("Files(s) related to the flaw.")) + param = models.TextField(null=True, + blank=True, + editable=False, + verbose_name=_("Parameter"), + help_text=_("Parameter used to trigger the issue (DAST).")) + payload = models.TextField(null=True, + blank=True, + editable=False, + verbose_name=_("Payload"), + help_text=_("Payload used to attack the service / application and trigger the bug / problem.")) + hash_code = models.CharField(null=True, + blank=True, + editable=False, + max_length=64, + verbose_name=_("Hash Code"), + help_text=_("A hash over a configurable set of fields that is used for findings deduplication.")) + line = models.IntegerField(null=True, + blank=True, + verbose_name=_("Line number"), + help_text=_("Source line number of the attack vector.")) + file_path = models.CharField(null=True, + blank=True, + max_length=4000, + verbose_name=_("File path"), + help_text=_("Identified file(s) containing the flaw.")) + component_name = models.CharField(null=True, + blank=True, + max_length=500, + verbose_name=_("Component name"), + help_text=_("Name of the affected component (library name, part of a system, ...).")) + component_version = models.CharField(null=True, + blank=True, + max_length=100, + verbose_name=_("Component version"), + help_text=_("Version of the affected component.")) + found_by = models.ManyToManyField("dojo.Test_Type", + editable=False, + verbose_name=_("Found by"), + help_text=_("The name of the scanner that identified the flaw.")) + static_finding = models.BooleanField(default=False, + verbose_name=_("Static finding (SAST)"), + help_text=_("Flaw has been detected from a Static Application Security Testing tool (SAST).")) + dynamic_finding = models.BooleanField(default=True, + verbose_name=_("Dynamic finding (DAST)"), + help_text=_("Flaw has been detected from a Dynamic Application Security Testing tool (DAST).")) + scanner_confidence = models.IntegerField(null=True, + blank=True, + default=None, + editable=False, + verbose_name=_("Scanner confidence"), + help_text=_("Confidence level of vulnerability which is supplied by the scanner.")) + sonarqube_issue = models.ForeignKey("dojo.Sonarqube_Issue", + null=True, + blank=True, + help_text=_("The SonarQube issue associated with this finding."), + verbose_name=_("SonarQube issue"), + on_delete=models.CASCADE) + unique_id_from_tool = models.CharField(null=True, + blank=True, + max_length=500, + verbose_name=_("Unique ID from tool"), + help_text=_("Vulnerability technical id from the source tool. Allows to track unique vulnerabilities over time across subsequent scans.")) + vuln_id_from_tool = models.CharField(null=True, + blank=True, + max_length=500, + verbose_name=_("Vulnerability ID from tool"), + help_text=_("Non-unique technical id from the source tool associated with the vulnerability type.")) + sast_source_object = models.CharField(null=True, + blank=True, + max_length=500, + verbose_name=_("SAST Source Object"), + help_text=_("Source object (variable, function...) of the attack vector.")) + sast_sink_object = models.CharField(null=True, + blank=True, + max_length=500, + verbose_name=_("SAST Sink Object"), + help_text=_("Sink object (variable, function...) of the attack vector.")) + sast_source_line = models.IntegerField(null=True, + blank=True, + verbose_name=_("SAST Source Line number"), + help_text=_("Source line number of the attack vector.")) + sast_source_file_path = models.CharField(null=True, + blank=True, + max_length=4000, + verbose_name=_("SAST Source File Path"), + help_text=_("Source file path of the attack vector.")) + nb_occurences = models.IntegerField(null=True, + blank=True, + verbose_name=_("Number of occurences"), + help_text=_("Number of occurences in the source tool when several vulnerabilites were found and aggregated by the scanner.")) + + # this is useful for vulnerabilities on dependencies : helps answer the question "Did I add this vulnerability or was it discovered recently?" + publish_date = models.DateField(null=True, + blank=True, + verbose_name=_("Publish date"), + help_text=_("Date when this vulnerability was made publicly available.")) + + # The service is used to generate the hash_code, so that it gets part of the deduplication of findings. + service = models.CharField(null=True, + blank=True, + max_length=200, + verbose_name=_("Service"), + help_text=_("A service is a self-contained piece of functionality within a Product. This is an optional field which is used in deduplication of findings when set.")) + + planned_remediation_date = models.DateField(null=True, + editable=True, + verbose_name=_("Planned Remediation Date"), + help_text=_("The date the flaw is expected to be remediated.")) + + planned_remediation_version = models.CharField(null=True, + blank=True, + max_length=99, + verbose_name=_("Planned remediation version"), + help_text=_("The target version when the vulnerability should be fixed / remediated")) + + effort_for_fixing = models.CharField(null=True, + blank=True, + max_length=99, + verbose_name=_("Effort for fixing"), + help_text=_("Effort for fixing / remediating the vulnerability (Low, Medium, High)")) + + tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this finding. Choose from the list or add new tags. Press Enter key to add.")) + inherited_tags = TagField(blank=True, force_lowercase=True, help_text=_("Internal use tags sepcifically for maintaining parity with product. This field will be present as a subset in the tags field")) + + SEVERITIES = {"Info": 4, "Low": 3, "Medium": 2, + "High": 1, "Critical": 0} + + class Meta: + ordering = ("numerical_severity", "-date", "title", "epss_score", "epss_percentile") + indexes = [ + models.Index(fields=["test", "active", "verified"]), + + models.Index(fields=["test", "is_mitigated"]), + models.Index(fields=["test", "duplicate"]), + models.Index(fields=["test", "out_of_scope"]), + models.Index(fields=["test", "false_p"]), + + models.Index(fields=["test", "unique_id_from_tool", "duplicate"]), + models.Index(fields=["test", "hash_code", "duplicate"]), + + models.Index(fields=["test", "component_name"]), + + models.Index(fields=["cve"]), + models.Index(fields=["epss_score"]), + models.Index(fields=["epss_percentile"]), + models.Index(fields=["cwe"]), + models.Index(fields=["out_of_scope"]), + models.Index(fields=["false_p"]), + models.Index(fields=["verified"]), + models.Index(fields=["mitigated"]), + models.Index(fields=["active"]), + models.Index(fields=["numerical_severity"]), + models.Index(fields=["date"]), + models.Index(fields=["title"]), + models.Index(fields=["hash_code"]), + models.Index(fields=["unique_id_from_tool"]), + # models.Index(fields=['file_path']), # can't add index because the field has max length 4000. + models.Index(fields=["line"]), + models.Index(fields=["component_name"]), + models.Index(fields=["duplicate"]), + models.Index(fields=["is_mitigated"]), + models.Index(fields=["duplicate_finding", "id"]), + models.Index(fields=["known_exploited"]), + models.Index(fields=["ransomware_used"]), + models.Index(fields=["kev_date"]), + ] + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + if settings.V3_FEATURE_LOCATIONS: + self.unsaved_locations: list[UnsavedLocation] = [] + else: + # TODO: Delete this after the move to Locations + self.unsaved_endpoints = [] + self.unsaved_request = None + self.unsaved_response = None + self.unsaved_tags = None + self.unsaved_files = None + self.unsaved_vulnerability_ids = None + + def __str__(self): + return self.title + + def save(self, dedupe_option=True, rules_option=True, product_grading_option=True, # noqa: FBT002 + issue_updater_option=True, push_to_jira=False, user=None, *args, **kwargs): # noqa: FBT002 - this is bit hard to fix nice have this universally fixed + logger.debug("Start saving finding of id " + str(self.id) + " dedupe_option:" + str(dedupe_option) + " (self.pk is %s)", "None" if self.pk is None else "not None") + from dojo.finding import helper as finding_helper # noqa: PLC0415 -- lazy import, avoids circular dependency + + is_new_finding = self.pk is None + + # if not isinstance(self.date, (datetime, date)): + # raise ValidationError(_("The 'date' field must be a valid date or datetime object.")) + + if not user: + from dojo.utils import get_current_user # noqa: PLC0415 -- lazy import, avoids circular dependency + user = get_current_user() + # Title Casing + self.title = titlecase(self.title[:511]) + # Set the date of the finding if nothing is supplied + if self.date is None: + self.date = timezone.now() + # Assign the numerical severity for correct sorting order + self.numerical_severity = Finding.get_numerical_severity(self.severity) + + # Synchronize cvssv3 score using cvssv3 vector + + if self.cvssv3: + try: + from dojo.utils import parse_cvss_data # noqa: PLC0415 -- lazy import, avoids circular dependency + cvss_data = parse_cvss_data(self.cvssv3) + if cvss_data: + self.cvssv3 = cvss_data.get("cvssv3") + self.cvssv3_score = cvss_data.get("cvssv3_score") + + except Exception as ex: + logger.warning("Can't compute cvssv3 score for finding id %i. Invalid cvssv3 vector found: '%s'. Exception: %s.", self.id, self.cvssv3, ex) + # remove invalid cvssv3 vector for new findings, or should we just throw a ValidationError? + if self.pk is None: + self.cvssv3 = None + + # behaviour for CVVS4 is slightly different. Extracting this into a method would lead to probably hard to read code + if self.cvssv4: + try: + from dojo.utils import parse_cvss_data # noqa: PLC0415 -- lazy import, avoids circular dependency + cvss_data = parse_cvss_data(self.cvssv4) + if cvss_data: + self.cvssv4 = cvss_data.get("cvssv4") + self.cvssv4_score = cvss_data.get("cvssv4_score") + + except Exception as ex: + logger.warning("Can't compute cvssv4 score for finding id %i. Invalid cvssv4 vector found: '%s'. Exception: %s.", self.id, self.cvssv4, ex) + self.cvssv4 = None + + self.set_hash_code(dedupe_option) + + if is_new_finding: + if settings.V3_FEATURE_LOCATIONS: + if (self.file_path is not None) and (len(self.unsaved_locations) == 0): + self.static_finding = True + self.dynamic_finding = False + elif (self.file_path is not None): + self.static_finding = True + # TODO: Delete this after the move to Locations + elif (self.file_path is not None) and (len(self.unsaved_endpoints) == 0): + self.static_finding = True + self.dynamic_finding = False + elif (self.file_path is not None): + self.static_finding = True + + # because we have reduced the number of (super()).save() calls, the helper is no longer called for new findings + # so we call it manually + finding_helper.update_finding_status(self, user, changed_fields={"id": (None, None)}) + + # logger.debug('setting static / dynamic in save') + # need to have an id/pk before we can access locations/endpoints + elif self.file_path is not None: + if settings.V3_FEATURE_LOCATIONS: + if not self.locations.exists(): + self.static_finding = True + self.dynamic_finding = False + else: + self.static_finding = True + # TODO: Delete this after the move to Locations + elif not self.endpoints.exists(): + self.static_finding = True + self.dynamic_finding = False + else: + self.static_finding = True + + # update the SLA expiration date last, after all other finding fields have been updated + self.set_sla_expiration_date() + + logger.debug("Saving finding of id " + str(self.id) + " dedupe_option:" + str(dedupe_option) + " (self.pk is %s)", "None" if self.pk is None else "not None") + # We cannot run the full_clean method here without issue, so we specify skip_validation + super().save(*args, **kwargs, skip_validation=True) + + # Only add to found_by for newly-created findings (avoid doing this on every update) + if is_new_finding: + self.found_by.add(self.test.test_type) + + # only perform post processing (in celery task) if needed. this check avoids submitting 1000s of tasks to celery that will do nothing + from dojo.models import System_Settings # noqa: PLC0415 -- lazy import, avoids circular dependency + system_settings = System_Settings.objects.get() + if dedupe_option or issue_updater_option or (product_grading_option and system_settings.enable_product_grade) or push_to_jira: + finding_helper.post_process_finding_save(self.id, dedupe_option=dedupe_option, rules_option=rules_option, product_grading_option=product_grading_option, + issue_updater_option=issue_updater_option, push_to_jira=push_to_jira, user=user, *args, **kwargs) + else: + logger.debug("no options selected that require finding post processing") + + def get_absolute_url(self): + return reverse("view_finding", args=[str(self.id)]) + + def copy(self, test=None): + copy = copy_model_util(self) + # Save the necessary ManyToMany relationships + old_notes = list(self.notes.all()) + old_files = list(self.files.all()) + old_reviewers = list(self.reviewers.all()) + old_found_by = list(self.found_by.all()) + old_tags = list(self.tags.all()) + # Wipe the IDs of the new object + if test: + copy.test = test + # Save the object before setting any ManyToMany relationships + copy.save() + # Copy the notes + for notes in old_notes: + copy.notes.add(notes.copy()) + # Copy the files + for files in old_files: + copy.files.add(files.copy()) + if settings.V3_FEATURE_LOCATIONS: + old_location_refs = self.locations.all() + for location_ref in old_location_refs: + location_ref.copy(copy) + else: + # TODO: Delete this after the move to Locations + # Copy the endpoint_status + old_status_findings = list(self.status_finding.all()) + for endpoint_status in old_status_findings: + endpoint_status.copy(finding=copy) # adding or setting is not necessary, link is created by Endpoint_Status.copy() + # Assign any reviewers + copy.reviewers.set(old_reviewers) + # Assign any found_by + copy.found_by.set(old_found_by) + # Assign any tags + copy.tags.set(old_tags) + + return copy + + def delete(self, *args, product_grading_option=True, **kwargs): + logger.debug("%d finding delete", self.id) + from dojo.finding import helper as finding_helper # noqa: PLC0415 -- lazy import, avoids circular dependency + finding_helper.finding_delete(self) + super().delete(*args, **kwargs) + if product_grading_option: + from dojo.models import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + Engagement, + Product, + Test, + ) + with suppress(Finding.DoesNotExist, Test.DoesNotExist, Engagement.DoesNotExist, Product.DoesNotExist): + # Suppressing a potential issue created from async delete removing + # related objects in a separate task + from dojo.utils import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + perform_product_grading, + ) + perform_product_grading(self.test.engagement.product) + + # only used by bulk risk acceptance api + @classmethod + def unaccepted_open_findings(cls): + from dojo.utils import get_system_setting # noqa: PLC0415 -- lazy import, avoids circular dependency + results = cls.objects.filter(active=True, duplicate=False, risk_accepted=False) + if get_system_setting("enforce_verified_status", True) or get_system_setting("enforce_verified_status_metrics", True): + results = results.filter(verified=True) + + return results + + @property + def risk_acceptance(self): + ras = self.risk_acceptance_set.all() + if ras: + return ras[0] + + return None + + def compute_hash_code(self): + # Allow Pro to overwrite compute hash_code which gets dedupe settings from a database instead of django.settings + from dojo.utils import get_custom_method # noqa: PLC0415 -- lazy import, avoids circular dependency + if compute_hash_code_method := get_custom_method("FINDING_COMPUTE_HASH_METHOD"): + deduplicationLogger.debug("using custom FINDING_COMPUTE_HASH_METHOD method") + return compute_hash_code_method(self) + + # Check if all needed settings are defined + if not hasattr(settings, "HASHCODE_FIELDS_PER_SCANNER") or not hasattr(settings, "HASHCODE_ALLOWS_NULL_CWE") or not hasattr(settings, "HASHCODE_ALLOWED_FIELDS"): + deduplicationLogger.debug("no or incomplete configuration per hash_code found; using legacy algorithm") + return self.compute_hash_code_legacy() + + hash_code_fields = self.test.hash_code_fields + + # Check if hash_code fields are found in the settings + if not hash_code_fields: + deduplicationLogger.debug( + "No configuration for hash_code computation found; using default fields for " + ("dynamic" if self.dynamic_finding else "static") + " scanners") + return self.compute_hash_code_legacy() + + # Check if all elements of HASHCODE_FIELDS_PER_SCANNER are in HASHCODE_ALLOWED_FIELDS + if not (all(elem in settings.HASHCODE_ALLOWED_FIELDS for elem in hash_code_fields)): + deduplicationLogger.debug( + "compute_hash_code - configuration error: some elements of HASHCODE_FIELDS_PER_SCANNER are not in the allowed list HASHCODE_ALLOWED_FIELDS. " + "Using default fields") + return self.compute_hash_code_legacy() + + # Make sure that we have a cwe if we need one + if self.cwe == 0 and not self.test.hash_code_allows_null_cwe: + deduplicationLogger.debug( + "Cannot compute hash_code based on configured fields because cwe is 0 for finding of title '" + self.title + "' found in file '" + str(self.file_path) + + "'. Fallback to legacy mode for this finding.") + return self.compute_hash_code_legacy() + + deduplicationLogger.debug("computing hash_code for finding id " + str(self.id) + " based on: " + ", ".join(hash_code_fields)) + + fields_to_hash = "" + for hashcodeField in hash_code_fields: + # Note: preserve this field label ("endpoints") for settings purposes through the Locations migration + if hashcodeField == "endpoints": + # For locations/endpoints, need to compute the field + locations = self.get_locations() + fields_to_hash += locations + deduplicationLogger.debug(hashcodeField + " : " + locations) + elif hashcodeField == "vulnerability_ids": + # For vulnerability_ids, need to compute the field + my_vulnerability_ids = self.get_vulnerability_ids() + fields_to_hash += my_vulnerability_ids + deduplicationLogger.debug(hashcodeField + " : " + my_vulnerability_ids) + else: + # Generically use the finding attribute having the same name, converts to str in case it's integer + fields_to_hash += str(getattr(self, hashcodeField)) + deduplicationLogger.debug(hashcodeField + " : " + str(getattr(self, hashcodeField))) + + # Log the hash_code fields that are always included (but are not part of the hash_code_fields list as they are inserted downtstream in self.hash_fields) + hash_code_fields_always = getattr(settings, "HASH_CODE_FIELDS_ALWAYS", []) + for hashcodeField in hash_code_fields_always: + if getattr(self, hashcodeField): + deduplicationLogger.debug(hashcodeField + " : " + str(getattr(self, hashcodeField))) + + deduplicationLogger.debug("compute_hash_code - fields_to_hash = " + fields_to_hash) + return self.hash_fields(fields_to_hash) + + def compute_hash_code_legacy(self): + fields_to_hash = self.title + str(self.cwe) + str(self.line) + str(self.file_path) + self.description + deduplicationLogger.debug("compute_hash_code_legacy - fields_to_hash = " + fields_to_hash) + return self.hash_fields(fields_to_hash) + + # Get vulnerability_ids to use for hash_code computation + def get_vulnerability_ids(self): + + def _get_unsaved_vulnerability_ids(finding) -> str: + if finding.unsaved_vulnerability_ids: + deduplicationLogger.debug("get_vulnerability_ids before the finding was saved") + # convert list of unsaved vulnerability_ids to the list of their canonical representation + vulnerability_id_str_list = [str(vulnerability_id) for vulnerability_id in finding.unsaved_vulnerability_ids] + # deduplicate (usually done upon saving finding) and sort endpoints + return "".join(sorted(dict.fromkeys(vulnerability_id_str_list))) + deduplicationLogger.debug("finding has no unsaved vulnerability references") + return "" + + def _get_saved_vulnerability_ids(finding) -> str: + if finding.id is not None: + vulnerability_ids = Vulnerability_Id.objects.filter(finding=finding) + deduplicationLogger.debug("get_vulnerability_ids after the finding was saved. Vulnerability references count: " + str(vulnerability_ids.count())) + # convert list of vulnerability_ids to the list of their canonical representation + vulnerability_id_str_list = [str(vulnerability_id) for vulnerability_id in vulnerability_ids.all()] + # sort vulnerability_ids strings + return "".join(sorted(vulnerability_id_str_list)) + return "" + + return _get_saved_vulnerability_ids(self) or _get_unsaved_vulnerability_ids(self) + + # Get locations/endpoints to use for hash_code computation + def get_locations(self): + # TODO: Delete this after the move to Locations + if not settings.V3_FEATURE_LOCATIONS: + # Get endpoints to use for hash_code computation + # (This sometimes reports "None") + def _get_unsaved_endpoints(finding) -> str: + if len(finding.unsaved_endpoints) > 0: + deduplicationLogger.debug("get_endpoints before the finding was saved") + # convert list of unsaved endpoints to the list of their canonical representation + endpoint_str_list = [str(endpoint) for endpoint in finding.unsaved_endpoints] + # deduplicate (usually done upon saving finding) and sort endpoints + return "".join(dict.fromkeys(endpoint_str_list)) + # we can get here when the parser defines static_finding=True but leaves dynamic_finding defaulted + # In this case, before saving the finding, both static_finding and dynamic_finding are True + # After saving dynamic_finding may be set to False probably during the saving process (observed on Bandit scan before forcing dynamic_finding=False at parser level) + deduplicationLogger.debug("trying to get endpoints on a finding before it was saved but no endpoints found (static parser wrongly identified as dynamic?") + return "" + + def _get_saved_endpoints(finding) -> str: + if finding.id is not None: + deduplicationLogger.debug("get_endpoints: after the finding was saved. Endpoints count: " + str(finding.endpoints.count())) + # convert list of endpoints to the list of their canonical representation + endpoint_str_list = [str(endpoint) for endpoint in finding.endpoints.all()] + # sort endpoints strings + return "".join(sorted(endpoint_str_list)) + return "" + + return _get_saved_endpoints(self) or _get_unsaved_endpoints(self) + + def _get_unsaved_locations(finding) -> str: + if len(finding.unsaved_locations) > 0: + deduplicationLogger.debug("get_locations before the finding was saved") + # convert list of unsaved locations to the list of their canonical representation + from dojo.importers.location_manager import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + LocationManager, + ) + unsaved_locations = LocationManager.clean_unsaved_locations(finding.unsaved_locations) + # deduplicate (usually done upon saving finding) and sort locations + locations = sorted({location.get_location_value() for location in unsaved_locations}) + return "".join(locations) + # we can get here when the parser defines static_finding=True but leaves dynamic_finding defaulted + # In this case, before saving the finding, both static_finding and dynamic_finding are True + # After saving dynamic_finding may be set to False probably during the saving process (observed on Bandit scan before forcing dynamic_finding=False at parser level) + deduplicationLogger.debug("trying to get locations on a finding before it was saved but no locations found (static parser wrongly identified as dynamic?") + return "" + + def _get_saved_locations(finding) -> str: + if finding.id is not None: + from dojo.url.models import URL # noqa: PLC0415 -- lazy import, avoids circular dependency + url_locations = finding.locations.filter(location__location_type=URL.get_location_type()) + deduplicationLogger.debug("get_locations: after the finding was saved. Locations count: " + str(url_locations.count())) + # convert list of locations to the list of their canonical representation + locations = sorted({location_ref.location.get_location_value() for location_ref in url_locations.all()}) + # sort locations strings + return "".join(sorted(locations)) + return "" + + return _get_saved_locations(self) or _get_unsaved_locations(self) + + # Compute the hash_code from the fields to hash + def hash_fields(self, fields_to_hash): + if hasattr(settings, "HASH_CODE_FIELDS_ALWAYS"): + for field in settings.HASH_CODE_FIELDS_ALWAYS: + if getattr(self, field): + deduplicationLogger.debug("adding HASH_CODE_FIELDS_ALWAYSfield %s to hash_fields: %s", field, getattr(self, field)) + fields_to_hash += str(getattr(self, field)) + + logger.debug("fields_to_hash : %s", fields_to_hash) + logger.debug("fields_to_hash lower: %s", fields_to_hash.lower()) + return hashlib.sha256(fields_to_hash.casefold().encode("utf-8").strip()).hexdigest() + + def duplicate_finding_set(self): + if self.duplicate: + if self.duplicate_finding is not None: + return Finding.objects.get( + id=self.duplicate_finding.id).original_finding.all().order_by("title") + return [] + return self.original_finding.all().order_by("title") + + def get_scanner_confidence_text(self): + if self.scanner_confidence and isinstance(self.scanner_confidence, int): + if self.scanner_confidence <= 2: + return "Certain" + if self.scanner_confidence >= 3 and self.scanner_confidence <= 5: + return "Firm" + return "Tentative" + return "" + + @staticmethod + def get_numerical_severity(severity): + if severity == "Critical": + return "S0" + if severity == "High": + return "S1" + if severity == "Medium": + return "S2" + if severity == "Low": + return "S3" + if severity == "Info": + return "S4" + return "S5" + + @staticmethod + def get_number_severity(severity): + if severity == "Critical": + return 4 + if severity == "High": + return 3 + if severity == "Medium": + return 2 + if severity == "Low": + return 1 + if severity == "Info": + return 0 + return 5 + + @staticmethod + def get_severity(num_severity): + severities = {0: "Info", 1: "Low", 2: "Medium", 3: "High", 4: "Critical"} + if num_severity in severities: + return severities[num_severity] + + return None + + def status(self): + status = [] + if self.under_review: + status += ["Under Review"] + if self.active: + status += ["Active"] + else: + status += ["Inactive"] + if self.verified: + status += ["Verified"] + if self.mitigated or self.is_mitigated: + status += ["Mitigated"] + if self.false_p: + status += ["False Positive"] + if self.out_of_scope: + status += ["Out Of Scope"] + if self.duplicate: + status += ["Duplicate"] + if self.risk_accepted: + status += ["Risk Accepted"] + if not len(status): + status += ["Initial"] + + return ", ".join([str(s) for s in status]) + + def _age(self, start_date): + if start_date and isinstance(start_date, str): + start_date = datetutilsparse(start_date).date() + + if isinstance(start_date, datetime): + start_date = start_date.date() + + if self.mitigated: + mitigated_date = self.mitigated + if isinstance(mitigated_date, datetime): + mitigated_date = self.mitigated.date() + diff = mitigated_date - start_date + else: + diff = get_current_date() - start_date + days = diff.days + return max(0, days) + + @property + def age(self): + return self._age(self.date) + + @property + def sla_age(self): + return self._age(self.get_sla_start_date()) + + def get_sla_start_date(self): + if self.sla_start_date: + return self.sla_start_date + return self.date + + def get_sla_configuration(self): + return self.test.engagement.product.sla_configuration + + def get_sla_period(self): + # Determine which method to use to calculate the SLA + from dojo.utils import get_custom_method # noqa: PLC0415 -- lazy import, avoids circular dependency + if method := get_custom_method("FINDING_SLA_PERIOD_METHOD"): + return method(self) + # Run the default method + sla_configuration = self.get_sla_configuration() + sla_period = getattr(sla_configuration, self.severity.lower(), None) + enforce_period = getattr(sla_configuration, str("enforce_" + self.severity.lower()), None) + return sla_period, enforce_period + + def set_sla_expiration_date(self): + # First check if SLA is enabled globally + from dojo.models import System_Settings # noqa: PLC0415 -- lazy import, avoids circular dependency + system_settings = System_Settings.objects.get() + if not system_settings.enable_finding_sla: + return + # Call the internal method to set the sla expiration date + self._set_sla_expiration_date() + + def _set_sla_expiration_date(self): + # some parsers provide date as a `str` instead of a `date` in which case we need to parse it #12299 on GitHub + sla_start_date = self.get_sla_start_date() + if sla_start_date and isinstance(sla_start_date, str): + sla_start_date = dateutil.parser.parse(sla_start_date).date() + + sla_period, enforce_period = self.get_sla_period() + if sla_period is not None and enforce_period: + self.sla_expiration_date = sla_start_date + relativedelta(days=sla_period) + else: + self.sla_expiration_date = None + + def sla_days_remaining(self): + if self.sla_expiration_date: + if self.mitigated: + mitigated_date = self.mitigated + if isinstance(mitigated_date, datetime): + mitigated_date = self.mitigated.date() + return (self.sla_expiration_date - mitigated_date).days + return (self.sla_expiration_date - get_current_date()).days + return None + + def sla_deadline(self): + return self.sla_expiration_date + + def github(self): + from dojo.github.models import GITHUB_Issue # noqa: PLC0415 -- lazy import, avoids circular dependency + try: + return self.github_issue + except GITHUB_Issue.DoesNotExist: + return None + + def has_github_issue(self): + from dojo.github.models import GITHUB_Issue # noqa: PLC0415 -- lazy import, avoids circular dependency + try: + # Attempt to access the github issue if it exists. If not, an exception will be caught + _ = self.github_issue + except GITHUB_Issue.DoesNotExist: + return False + return True + + def github_conf(self): + from dojo.github.models import GITHUB_PKey # noqa: PLC0415 -- lazy import, avoids circular dependency + try: + github_product_key = GITHUB_PKey.objects.get(product=self.test.engagement.product) + github_conf = github_product_key.conf + except: + github_conf = None + return github_conf + + # newer version that can work with prefetching + def github_conf_new(self): + try: + return self.test.engagement.product.github_pkey_set.all()[0].git_conf + except: + return None + + @property + def has_jira_issue(self): + from dojo.jira import services as jira_services # noqa: PLC0415 -- lazy import, avoids circular dependency + return jira_services.has_issue(self) + + @cached_property + def finding_group(self): + return self.finding_group_set.all().first() + # logger.debug('finding.finding_group: %s', group) + + @cached_property + def has_jira_group_issue(self): + if not self.has_finding_group: + return False + + from dojo.jira import services as jira_services # noqa: PLC0415 -- lazy import, avoids circular dependency + return jira_services.has_issue(self.finding_group) + + @property + def has_jira_configured(self): + from dojo.jira import services as jira_services # noqa: PLC0415 -- lazy import, avoids circular dependency + return jira_services.has_configured(self) + + @cached_property + def has_finding_group(self): + return self.finding_group is not None + + def save_no_options(self, *args, **kwargs): + logger.debug("save_no_options") + return self.save(dedupe_option=False, rules_option=False, product_grading_option=False, + issue_updater_option=False, push_to_jira=False, user=None, *args, **kwargs) + + # Check if a mandatory field is empty. If it's the case, fill it with "no given" + def clean(self): + no_check = ["test", "reporter"] + bigfields = ["description"] + for field_obj in self._meta.fields: + field = field_obj.name + if field not in no_check: + val = getattr(self, field) + if not val and field == "title": + setattr(self, field, "No title given") + if not val and field in bigfields: + setattr(self, field, f"No {field} given") + + def severity_display(self): + return self.severity + + def get_breadcrumbs(self): + bc = self.test.get_breadcrumbs() + bc += [{"title": str(self), + "url": reverse("view_finding", args=(self.id,))}] + return bc + + def get_valid_request_response_pairs(self): + empty_value = base64.b64encode(b"") + # Get a list of all req/resp pairs + all_req_resps = self.burprawrequestresponse_set.all() + # Filter away those that do not have any contents + return all_req_resps.exclude( + burpRequestBase64__exact=empty_value, + burpResponseBase64__exact=empty_value, + ) + + def get_report_requests(self): + # Get the list of request response pairs that are non empty + request_response_pairs = self.get_valid_request_response_pairs() + # Determine how many to return + if request_response_pairs.count() >= 3: + return request_response_pairs[0:3] + if request_response_pairs.count() > 0: + return request_response_pairs + return None + + def get_request(self): + # Get the list of request response pairs that are non empty + request_response_pairs = self.get_valid_request_response_pairs() + # Determine what to return + if request_response_pairs.count() > 0: + reqres = request_response_pairs.first() + return base64.b64decode(reqres.burpRequestBase64) + + def get_response(self): + # Get the list of request response pairs that are non empty + request_response_pairs = self.get_valid_request_response_pairs() + # Determine what to return + if request_response_pairs.count() > 0: + reqres = request_response_pairs.first() + res = base64.b64decode(reqres.burpResponseBase64) + # Removes all blank lines + return re.sub(r"\n\s*\n", "\n", res) + + def latest_note(self): + if self.notes.all(): + note = self.notes.all()[0] + return note.date.strftime("%Y-%m-%d %H:%M:%S") + ": " + note.author.get_full_name() + " : " + note.entry + + return "" + + def get_sast_source_file_path_with_link(self): + from dojo.utils import create_bleached_link # noqa: PLC0415 -- lazy import, avoids circular dependency + if self.sast_source_file_path is None: + return None + if self.test.engagement.source_code_management_uri is None: + return escape(self.sast_source_file_path) + link = self.test.engagement.source_code_management_uri + "/" + self.sast_source_file_path + if self.sast_source_line: + link = link + "#L" + str(self.sast_source_line) + return create_bleached_link(link, self.sast_source_file_path) + + def get_file_path_with_link(self): + from dojo.utils import create_bleached_link # noqa: PLC0415 -- lazy import, avoids circular dependency + if self.file_path is None: + return None + if self.test.engagement.source_code_management_uri is None: + return escape(self.file_path) + link = self.get_file_path_with_raw_link() + return create_bleached_link(link, self.file_path) + + def get_scm_type(self): + # extract scm type from product custom field 'scm-type' + + from dojo.models import DojoMeta # noqa: PLC0415 -- lazy import, avoids circular dependency + if hasattr(self.test.engagement, "product"): + dojo_meta = DojoMeta.objects.filter(product=self.test.engagement.product, name="scm-type").first() + if dojo_meta: + st = dojo_meta.value.strip() + if st: + return st.lower() + return "" + + def scm_public_prepare_base_link(self, uri): + # scm public (https://scm-domain.org) url template for browse is: + # https://scm-domain.org// + # but when you get repo url for git, its template is: + # https://scm-domain.org//.git + # so to create browser url - git url should be recomposed like below: + + parts_uri = uri.split(".git") + return parts_uri[0] + + def git_public_prepare_scm_link(self, uri, scm_type): + # if commit hash or branch/tag is set for engagement/test - + # hash or branch/tag should be appended to base browser link + intermediate_path = "/blob/" if scm_type in {"github", "gitlab"} else "/src/" + + link = self.scm_public_prepare_base_link(uri) + if self.test.commit_hash: + link += intermediate_path + self.test.commit_hash + "/" + self.file_path + elif self.test.engagement.commit_hash: + link += intermediate_path + self.test.engagement.commit_hash + "/" + self.file_path + elif self.test.branch_tag: + link += intermediate_path + self.test.branch_tag + "/" + self.file_path + elif self.test.engagement.branch_tag: + link += intermediate_path + self.test.engagement.branch_tag + "/" + self.file_path + else: + link += intermediate_path + "master/" + self.file_path + + return link + + def bitbucket_standalone_prepare_scm_base_link(self, uri): + # bitbucket onpremise/standalone url template for browse is: + # https://bb.example.com/projects//repos/ + # but when you get repo url for git, its template is: + # https://bb.example.com/scm//.git + # or for user public repo^ + # https://bb.example.com/users//repos/ + # but when you get repo url for git, its template is: + # https://bb.example.com/scm//.git (username often could be prefixed with ~) + # so to create borwser url - git url should be recomposed like below: + + parts_uri = uri.split(".git") + parts_scm = parts_uri[0].split("/scm/") + parts_project = parts_scm[1].split("/") + project = parts_project[0] + if project.startswith("~"): + return parts_scm[0] + "/users/" + parts_project[0][1:] + "/repos/" + parts_project[1] + "/browse" + return parts_scm[0] + "/projects/" + parts_project[0] + "/repos/" + parts_project[1] + "/browse" + + def bitbucket_standalone_prepare_scm_link(self, uri): + # if commit hash or branch/tag is set for engagement/test - + # hash or barnch/tag should be appended to base browser link + + link = self.bitbucket_standalone_prepare_scm_base_link(uri) + if self.test.commit_hash: + link += "/" + self.file_path + "?at=" + self.test.commit_hash + elif self.test.engagement.commit_hash: + link += "/" + self.file_path + "?at=" + self.test.engagement.commit_hash + elif self.test.branch_tag: + link += "/" + self.file_path + "?at=" + self.test.branch_tag + elif self.test.engagement.branch_tag: + link += "/" + self.file_path + "?at=" + self.test.engagement.branch_tag + else: + link += "/" + self.file_path + + return link + + def get_file_path_with_raw_link(self): + if self.file_path is None: + return None + + link = self.test.engagement.source_code_management_uri + scm_type = self.get_scm_type() + if (self.test.engagement.source_code_management_uri is not None): + if scm_type == "bitbucket-standalone": + link = self.bitbucket_standalone_prepare_scm_link(link) + elif scm_type in {"github", "gitlab", "gitea", "codeberg", "bitbucket"}: + link = self.git_public_prepare_scm_link(link, scm_type) + elif "https://github.com/" in self.test.engagement.source_code_management_uri: + link = self.git_public_prepare_scm_link(link, "github") + else: + link += "/" + self.file_path + else: + link += "/" + self.file_path + + # than - add line part to browser url + if self.line: + if scm_type in {"github", "gitlab", "gitea", "codeberg"} or "https://github.com/" in self.test.engagement.source_code_management_uri: + link = link + "#L" + str(self.line) + elif scm_type == "bitbucket-standalone": + link = link + "#" + str(self.line) + elif scm_type == "bitbucket": + link = link + "#lines-" + str(self.line) + return link + + def get_references_with_links(self): + from dojo.utils import create_bleached_link # noqa: PLC0415 -- lazy import, avoids circular dependency + if self.references is None: + return None + matches = re.findall(r"([\(|\[]?(https?):((//)|(\\\\))+([\w\d:#@%/;$~_?\+-=\\\.&](#!)?)*[\)|\]]?)", self.references) + + processed_matches = [] + for match in matches: + # Check if match isn't already a markdown link + # Only replace the same matches one time, otherwise the links will be corrupted + if not (match[0].startswith("[") or match[0].startswith("(")) and match[0] not in processed_matches: + self.references = self.references.replace(match[0], create_bleached_link(match[0], match[0]), 1) + processed_matches.append(match[0]) + + return self.references + + @cached_property + def vulnerability_ids(self): + # Get vulnerability ids from database and convert to list of strings + vulnerability_ids_model = self.vulnerability_id_set.all() + vulnerability_ids = [vulnerability_id.vulnerability_id for vulnerability_id in vulnerability_ids_model] + + # Synchronize the cve field with the unsaved_vulnerability_ids + # We do this to be as flexible as possible to handle the fields until + # the cve field is not needed anymore and can be removed. + if vulnerability_ids and self.cve: + # Make sure the first entry of the list is the value of the cve field + vulnerability_ids.insert(0, self.cve) + elif not vulnerability_ids and self.cve: + # If there is no list, make one with the value of the cve field + vulnerability_ids = [self.cve] + + # Remove duplicates + return list(dict.fromkeys(vulnerability_ids)) + + @property + def violates_sla(self): + return (self.sla_expiration_date and self.sla_expiration_date < timezone.now().date()) + + def set_hash_code(self, dedupe_option): + from dojo.utils import get_custom_method # noqa: PLC0415 -- lazy import, avoids circular dependency + if hash_method := get_custom_method("FINDING_HASH_METHOD"): + deduplicationLogger.debug("Using custom hash method") + hash_method(self, dedupe_option) + # Finding.save is called once from serializers.py with dedupe_option=False because the finding is not ready yet, for example the endpoints are not built + # It is then called a second time with dedupe_option defaulted to true; now we can compute the hash_code and run the deduplication + elif dedupe_option: + finding_id = self.id if self.id is not None else "unsaved" + if self.hash_code is not None: + deduplicationLogger.debug("Hash_code already computed for finding: %s", finding_id) + else: + self.hash_code = self.compute_hash_code() + deduplicationLogger.debug("Hash_code computed for finding: %s: %s", finding_id, self.hash_code) + + +class Vulnerability_Id(models.Model): + finding = models.ForeignKey("dojo.Finding", editable=False, on_delete=models.CASCADE) + vulnerability_id = models.TextField(max_length=50, blank=False, null=False) + + def __str__(self): + return self.vulnerability_id + + def get_absolute_url(self): + return reverse("view_finding", args=[str(self.finding.id)]) + + +class Finding_Group(TimeStampedModel): + + GROUP_BY_OPTIONS = [("component_name", "Component Name"), + ("component_name+component_version", "Component Name + Version"), + ("file_path", "File path"), + ("finding_title", "Finding Title"), + ("vuln_id_from_tool", "Vulnerability ID from Tool")] + + name = models.CharField(max_length=255, blank=False, null=False) + test = models.ForeignKey("dojo.Test", on_delete=models.CASCADE) + findings = models.ManyToManyField("dojo.Finding") + creator = models.ForeignKey("dojo.Dojo_User", on_delete=models.RESTRICT) + + def __str__(self): + return self.name + + @property + def has_jira_issue(self): + from dojo.jira import services as jira_services # noqa: PLC0415 -- lazy import, avoids circular dependency + return jira_services.has_issue(self) + + @cached_property + def severity(self): + if not self.findings.all(): + return None + max_number_severity = max(Finding.get_number_severity(find.severity) for find in self.findings.all()) + return Finding.get_severity(max_number_severity) + + @cached_property + def components(self): + components: dict[str, set[str | None]] = {} + for finding in self.findings.all(): + if finding.component_name is not None: + components.setdefault(finding.component_name, set()).add(finding.component_version) + return "; ".join(f"""{name}: {", ".join(map(str, versions))}""" for name, versions in components.items()) + + @property + def age(self): + if not self.findings.all(): + return None + + return max(find.age for find in self.findings.all()) + + @cached_property + def sla_days_remaining_internal(self): + if not self.findings.all(): + return None + + return min([find.sla_days_remaining() for find in self.findings.all() if find.sla_days_remaining()], default=None) + + def sla_days_remaining(self): + return self.sla_days_remaining_internal + + def sla_deadline(self): + if not self.findings.all(): + return None + + return min([find.sla_deadline() for find in self.findings.all() if find.sla_deadline()], default=None) + + def status(self): + if not self.findings.all(): + return None + + if any(find.active for find in self.findings.all()): + return "Active" + + if all(find.is_mitigated for find in self.findings.all()): + return "Mitigated" + + return "Inactive" + + @cached_property + def mitigated(self): + return all(find.mitigated is not None for find in self.findings.all()) + + def get_sla_start_date(self): + return min(find.get_sla_start_date() for find in self.findings.all()) + + def get_absolute_url(self): + return reverse("view_test", args=[str(self.test.id)]) + + class Meta: + ordering = ["id"] + + +class Finding_Template(models.Model): + title = models.TextField(max_length=1000) + cwe = models.IntegerField(default=None, null=True, blank=True) + cve = models.CharField(max_length=50, + null=True, + blank=False, + verbose_name="Vulnerability Id", + help_text="An id of a vulnerability in a security advisory associated with this finding. Can be a Common Vulnerabilities and Exposures (CVE) or from other sources.") + cvssv3 = models.TextField(help_text=_("Common Vulnerability Scoring System version 3 (CVSSv3) score associated with this finding."), validators=[cvss3_validator], max_length=117, null=True, verbose_name=_("CVSS v3 vector")) + cvssv3_score = models.FloatField(null=True, blank=True, help_text=_("CVSSv3 score")) + cvssv4 = models.TextField(help_text=_("Common Vulnerability Scoring System version 4 (CVSS4) score associated with this finding."), validators=[cvss4_validator], max_length=255, null=True, verbose_name=_("CVSS4 vector")) + cvssv4_score = models.FloatField(null=True, blank=True, help_text=_("CVSSv4 score")) + + severity = models.CharField(max_length=200, null=True, blank=True) + description = models.TextField(null=True, blank=True) + mitigation = models.TextField(null=True, blank=True) + impact = models.TextField(null=True, blank=True) + references = models.TextField(null=True, blank=True, db_column="refs") + last_used = models.DateTimeField(null=True, editable=False) + numerical_severity = models.CharField(max_length=4, null=True, blank=True, editable=False) + + # Remediation planning fields + fix_available = models.BooleanField(null=True, blank=True, help_text=_("Indicates if a fix is available for this vulnerability type")) + fix_version = models.CharField(max_length=100, null=True, blank=True, help_text=_("Version where fix is available")) + planned_remediation_version = models.CharField(max_length=99, null=True, blank=True, help_text=_("Target version for remediation")) + effort_for_fixing = models.CharField(max_length=99, null=True, blank=True, help_text=_("Effort estimate for fixing (e.g., Low/Medium/High)")) + + # Technical details fields + steps_to_reproduce = models.TextField(null=True, blank=True, help_text=_("Standard reproduction steps for this vulnerability type")) + severity_justification = models.TextField(null=True, blank=True, help_text=_("Explanation of why this severity level is appropriate")) + component_name = models.CharField(max_length=500, null=True, blank=True, help_text=_("Affected component name (e.g., library name)")) + component_version = models.CharField(max_length=100, null=True, blank=True, help_text=_("Affected component version")) + + # Notes field (single note content, not a list) + notes = models.TextField(null=True, blank=True, help_text=_("Note content to add when applying this template")) + + # String-based list fields (newline-separated) + vulnerability_ids_text = models.TextField(null=True, blank=True, help_text=_("Vulnerability IDs (one per line)")) + endpoints_text = models.TextField(null=True, blank=True, help_text=_("Endpoint URLs (one per line)")) + + tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this finding template. Choose from the list or add new tags. Press Enter key to add.")) + + SEVERITIES = {"Info": 4, "Low": 3, "Medium": 2, + "High": 1, "Critical": 0} + + class Meta: + ordering = ["-cwe"] + + def __str__(self): + return self.title + + def get_absolute_url(self): + return reverse("edit_template", args=[str(self.id)]) + + def get_breadcrumbs(self): + return [{"title": str(self), + "url": reverse("view_template", args=(self.id,))}] + + @property + def vulnerability_ids(self): + """Parse vulnerability IDs from TextField string (newline-separated).""" + vulnerability_ids = [] + + # Get from the TextField + if self.vulnerability_ids_text: + # Parse newline-separated string, remove empty lines + vulnerability_ids = [line.strip() for line in self.vulnerability_ids_text.split("\n") if line.strip()] + + # Synchronize the cve field with the vulnerability_ids + # We do this to be as flexible as possible to handle the fields until + # the cve field is not needed anymore and can be removed. + if vulnerability_ids and self.cve and self.cve not in vulnerability_ids: + # Make sure the first entry of the list is the value of the cve field + vulnerability_ids.insert(0, self.cve) + elif not vulnerability_ids and self.cve: + # If there is no list, make one with the value of the cve field + vulnerability_ids = [self.cve] + + # Remove duplicates + return list(dict.fromkeys(vulnerability_ids)) + + @property + def endpoints(self): + """Parse endpoint URLs from TextField string (newline-separated).""" + if not self.endpoints_text: + return [] + # Parse newline-separated string, remove empty lines + return [line.strip() for line in self.endpoints_text.split("\n") if line.strip()] + + +class CWE(models.Model): + url = models.CharField(max_length=1000) + description = models.CharField(max_length=2000) + number = models.IntegerField() + + +class BurpRawRequestResponse(models.Model): + finding = models.ForeignKey("dojo.Finding", blank=True, null=True, on_delete=models.CASCADE) + burpRequestBase64 = models.BinaryField() + burpResponseBase64 = models.BinaryField() + + def get_request(self): + return str(base64.b64decode(self.burpRequestBase64), errors="ignore") + + def get_response(self): + res = str(base64.b64decode(self.burpResponseBase64), errors="ignore") + # Removes all blank lines + return re.sub(r"\n\s*\n", "\n", res) diff --git a/dojo/finding/ui/__init__.py b/dojo/finding/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/finding/ui/filters.py b/dojo/finding/ui/filters.py new file mode 100644 index 00000000000..b886c9a5232 --- /dev/null +++ b/dojo/finding/ui/filters.py @@ -0,0 +1,1100 @@ +from datetime import timedelta + +from django import forms +from django.conf import settings +from django.db.models import Q +from django.forms import HiddenInput +from django.utils.translation import gettext_lazy as _ +from django_filters import ( + BooleanFilter, + CharFilter, + ChoiceFilter, + DateFilter, + DateFromToRangeFilter, + DateTimeFilter, + FilterSet, + ModelChoiceFilter, + ModelMultipleChoiceFilter, + MultipleChoiceFilter, + NumberFilter, + OrderingFilter, +) + +from dojo.engagement.queries import get_authorized_engagements +from dojo.filters import ( + DateRangeFilter, + DateRangeOmniFilter, + DojoFilter, + FindingHasJIRAFilter, + FindingSLAFilter, + FindingStatusFilter, + FindingTagFilter, + FindingTagStringFilter, + MetricsDateRangeFilter, + PercentageFilter, + PercentageRangeFilter, + ReportBooleanFilter, + ReportRiskAcceptanceFilter, + custom_vulnerability_id_filter, + cwe_options, + filter_endpoints_base, + filter_endpoints_host_base, + get_finding_filterset_fields, + vulnerability_id_filter, +) +from dojo.finding.queries import ( + get_authorized_findings_for_queryset, +) +from dojo.finding_group.queries import get_authorized_finding_groups_for_queryset +from dojo.labels import get_labels +from dojo.location.status import FindingLocationStatus +from dojo.models import ( + EFFORT_FOR_FIXING_CHOICES, + SEVERITY_CHOICES, + Dojo_User, + Endpoint, + Engagement, + Finding, + Finding_Group, + Finding_Template, + Product, + Product_Type, + Risk_Acceptance, + Test, + Test_Type, +) +from dojo.product.queries import get_authorized_products +from dojo.product_type.queries import get_authorized_product_types +from dojo.risk_acceptance.queries import get_authorized_risk_acceptances +from dojo.test.queries import get_authorized_tests +from dojo.user.queries import get_authorized_users +from dojo.utils import get_system_setting, get_visible_scan_types, is_finding_groups_enabled + +labels = get_labels() + + +class FindingFilterHelper(FilterSet): + title = CharFilter(lookup_expr="icontains") + date = DateRangeFilter(field_name="date", label="Date Discovered") + on = DateFilter(field_name="date", lookup_expr="exact", label="Discovered On") + before = DateFilter(field_name="date", lookup_expr="lt", label="Discovered Before") + after = DateFilter(field_name="date", lookup_expr="gt", label="Discovered After") + last_reviewed = DateRangeFilter() + last_status_update = DateRangeFilter() + cwe = MultipleChoiceFilter(choices=[]) + vulnerability_id = CharFilter(method=vulnerability_id_filter, label="Vulnerability Id") + severity = MultipleChoiceFilter(choices=SEVERITY_CHOICES) + duplicate = ReportBooleanFilter() + is_mitigated = ReportBooleanFilter() + fix_available = ReportBooleanFilter() + mitigation = CharFilter(lookup_expr="icontains") + mitigation_available = BooleanFilter(method="filter_mitigation_available", label="Mitigation Available") + mitigated = DateRangeFilter(field_name="mitigated", label="Mitigated Date") + mitigated_on = DateTimeFilter(field_name="mitigated", lookup_expr="exact", label="Mitigated On", method="filter_mitigated_on") + mitigated_before = DateTimeFilter(field_name="mitigated", lookup_expr="lt", label="Mitigated Before") + mitigated_after = DateTimeFilter(field_name="mitigated", lookup_expr="gt", label="Mitigated After", method="filter_mitigated_after") + planned_remediation_date = DateRangeOmniFilter() + planned_remediation_version = CharFilter(lookup_expr="icontains", label=_("Planned remediation version")) + file_path = CharFilter(lookup_expr="icontains") + param = CharFilter(lookup_expr="icontains") + payload = CharFilter(lookup_expr="icontains") + test__test_type = ModelMultipleChoiceFilter(queryset=Test_Type.objects.all(), label="Test Type") + service = CharFilter(lookup_expr="icontains") + test__engagement__version = CharFilter(lookup_expr="icontains", label="Engagement Version") + test__version = CharFilter(lookup_expr="icontains", label="Test Version") + risk_acceptance = ReportRiskAcceptanceFilter(label="Risk Accepted") + effort_for_fixing = MultipleChoiceFilter(choices=EFFORT_FOR_FIXING_CHOICES) + test_import_finding_action__test_import = NumberFilter(widget=HiddenInput()) + status = FindingStatusFilter(label="Status") + test__engagement__product__lifecycle = MultipleChoiceFilter( + choices=Product.LIFECYCLE_CHOICES, + label=labels.ASSET_LIFECYCLE_LABEL) + if settings.V3_FEATURE_LOCATIONS: + location_status = MultipleChoiceFilter( + field_name="locations__status", + choices=FindingLocationStatus.choices, + help_text="Status of the Location from the Findings relationship", + ) + endpoints__host = CharFilter( + field_name="locations__location__url__host", method="filter_endpoints_host", label="Endpoint Host", + ) + endpoints = NumberFilter(field_name="locations__location", method="filter_endpoints", widget=HiddenInput()) + + def filter_endpoints_host(self, queryset, name, value): + return filter_endpoints_host_base( + queryset, + name, + value, + endpoint_id=self.data.get("endpoints"), + statuses=self.data.getlist("location_status"), + ) + + def filter_endpoints(self, queryset, name, value): + return filter_endpoints_base( + queryset, + name, + value, + statuses=self.data.getlist("location_status"), + host=self.data.get("endpoints__host"), + ) + else: + # TODO: Delete this after the move to Locations + endpoints__host = CharFilter(lookup_expr="icontains", label="Endpoint Host") + endpoints = NumberFilter(widget=HiddenInput()) + + has_component = BooleanFilter( + field_name="component_name", + lookup_expr="isnull", + exclude=True, + label="Has Component") + has_notes = BooleanFilter( + field_name="notes", + lookup_expr="isnull", + exclude=True, + label="Has notes") + + if is_finding_groups_enabled(): + has_finding_group = BooleanFilter( + field_name="finding_group", + lookup_expr="isnull", + exclude=True, + label="Is Grouped") + + if get_system_setting("enable_jira"): + has_jira_issue = BooleanFilter( + field_name="jira_issue", + lookup_expr="isnull", + exclude=True, + label="Has JIRA") + jira_creation = DateRangeFilter(field_name="jira_issue__jira_creation", label="JIRA Creation") + jira_change = DateRangeFilter(field_name="jira_issue__jira_change", label="JIRA Updated") + jira_issue__jira_key = CharFilter(field_name="jira_issue__jira_key", lookup_expr="icontains", label="JIRA issue") + + if is_finding_groups_enabled(): + has_jira_group_issue = BooleanFilter( + field_name="finding_group__jira_issue", + lookup_expr="isnull", + exclude=True, + label="Has Group JIRA") + has_any_jira_issue = FindingHasJIRAFilter( + label="Has Any JIRA Issue", + help_text="Matches JIRA issues linked to the finding itself or to the finding's group.", + ) + + outside_of_sla = FindingSLAFilter(label="Outside of SLA") + has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") + epss_score = PercentageFilter(field_name="epss_score", label="EPSS score") + epss_score_range = PercentageRangeFilter( + field_name="epss_score", + label="EPSS score range", + help_text=( + "The range of EPSS score percentages to filter on; the left input is a lower bound, " + "the right is an upper bound. Leaving one empty will skip that bound (e.g., leaving " + "the lower bound input empty will filter only on the upper bound -- filtering on " + '"less than or equal").' + )) + epss_percentile = PercentageFilter(field_name="epss_percentile", label="EPSS percentile") + epss_percentile_range = PercentageRangeFilter( + field_name="epss_percentile", + label="EPSS percentile range", + help_text=( + "The range of EPSS percentiles to filter on; the left input is a lower bound, the right " + "is an upper bound. Leaving one empty will skip that bound (e.g., leaving the lower bound " + 'input empty will filter only on the upper bound -- filtering on "less than or equal").' + )) + kev_date = DateFilter(field_name="kev_date", lookup_expr="exact", label="Added to KEV On") + kev_before = DateFilter(field_name="kev_date", lookup_expr="lt", label="Added to KEV Before") + kev_after = DateFilter(field_name="kev_date", lookup_expr="gt", label="Added to KEV After") + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("numerical_severity", "numerical_severity"), + ("date", "date"), + ("mitigated", "mitigated"), + ("fix_available", "fix_available"), + ("risk_acceptance__created__date", + "risk_acceptance__created__date"), + ("last_reviewed", "last_reviewed"), + ("planned_remediation_date", "planned_remediation_date"), + ("planned_remediation_version", "planned_remediation_version"), + ("title", "title"), + ("test__engagement__product__name", + "test__engagement__product__name"), + ("service", "service"), + ("sla_age_days", "sla_age_days"), + ("epss_score", "epss_score"), + ("epss_percentile", "epss_percentile"), + ("known_exploited", "known_exploited"), + ("ransomware_used", "ransomware_used"), + ("kev_date", "kev_date"), + ), + field_labels={ + "numerical_severity": "Severity", + "date": "Date", + "risk_acceptance__created__date": "Acceptance Date", + "mitigated": "Mitigated Date", + "fix_available": "Fix Available", + "title": "Finding Name", + "test__engagement__product__name": labels.ASSET_FILTERS_NAME_LABEL, + "epss_score": "EPSS Score", + "epss_percentile": "EPSS Percentile", + "known_exploited": "Known Exploited", + "ransomware_used": "Ransomware Used", + "kev_date": "Date added to KEV", + "sla_age_days": "SLA age (days)", + "planned_remediation_date": "Planned Remediation", + "planned_remediation_version": "Planned remediation version", + }, + ) + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + if "test__test_type" in self.form.fields: + self.form.fields["test__test_type"].queryset = get_visible_scan_types() + + def set_date_fields(self, *args: list, **kwargs: dict): + date_input_widget = forms.DateInput(attrs={"class": "datepicker", "placeholder": "YYYY-MM-DD"}, format="%Y-%m-%d") + self.form.fields["on"].widget = date_input_widget + self.form.fields["before"].widget = date_input_widget + self.form.fields["after"].widget = date_input_widget + self.form.fields["kev_date"].widget = date_input_widget + self.form.fields["kev_before"].widget = date_input_widget + self.form.fields["kev_after"].widget = date_input_widget + self.form.fields["mitigated_on"].widget = date_input_widget + self.form.fields["mitigated_before"].widget = date_input_widget + self.form.fields["mitigated_after"].widget = date_input_widget + self.form.fields["cwe"].choices = cwe_options(self.queryset) + + def filter_mitigated_after(self, queryset, name, value): + if value.hour == 0 and value.minute == 0 and value.second == 0: + value = value.replace(hour=23, minute=59, second=59) + + return queryset.filter(mitigated__gt=value) + + def filter_mitigated_on(self, queryset, name, value): + if value.hour == 0 and value.minute == 0 and value.second == 0: + # we have a simple date without a time, lets get a range from this morning to tonight at 23:59:59:999 + nextday = value + timedelta(days=1) + return queryset.filter(mitigated__gte=value, mitigated__lt=nextday) + + return queryset.filter(mitigated=value) + + def filter_mitigation_available(self, queryset, name, value): + if value: + return queryset.exclude(mitigation__isnull=True).exclude(mitigation__exact="") + return queryset.filter(Q(mitigation__isnull=True) | Q(mitigation__exact="")) + + +def get_finding_group_queryset_for_context(pid=None, eid=None, tid=None): + """ + Helper function to build finding group queryset based on context hierarchy. + Context priority: test > engagement > product > global + + Args: + pid: Product ID (least specific) + eid: Engagement ID + tid: Test ID (most specific) + + Returns: + QuerySet of Finding_Group filtered by context + + """ + if tid is not None: + # Most specific: filter by test + return Finding_Group.objects.filter(test_id=tid).only("id", "name") + if eid is not None: + # Filter by engagement's tests + return Finding_Group.objects.filter(test__engagement_id=eid).only("id", "name") + if pid is not None: + # Filter by product's tests + return Finding_Group.objects.filter(test__engagement__product_id=pid).only("id", "name") + # Global: return all (authorization will be applied separately) + return Finding_Group.objects.all().only("id", "name") + + +class FindingFilterWithoutObjectLookups(FindingFilterHelper, FindingTagStringFilter): + test__engagement__product__prod_type = NumberFilter(widget=HiddenInput()) + test__engagement__product = NumberFilter(widget=HiddenInput()) + reporter = CharFilter( + field_name="reporter__username", + lookup_expr="iexact", + label="Reporter Username", + help_text="Search for Reporter names that are an exact match") + reporter_contains = CharFilter( + field_name="reporter__username", + lookup_expr="icontains", + label="Reporter Username Contains", + help_text="Search for Reporter names that contain a given pattern") + reviewers = CharFilter( + field_name="reviewers__username", + lookup_expr="iexact", + label="Reviewer Username", + help_text="Search for Reviewer names that are an exact match") + reviewers_contains = CharFilter( + field_name="reviewers__username", + lookup_expr="icontains", + label="Reviewer Username Contains", + help_text="Search for Reviewer usernames that contain a given pattern") + test__engagement__product__prod_type__name = CharFilter( + field_name="test__engagement__product__prod_type__name", + lookup_expr="iexact", + label=labels.ORG_FILTERS_NAME_LABEL, + help_text=labels.ORG_FILTERS_NAME_HELP) + test__engagement__product__prod_type__name_contains = CharFilter( + field_name="test__engagement__product__prod_type__name", + lookup_expr="icontains", + label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, + help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) + test__engagement__product__name = CharFilter( + field_name="test__engagement__product__name", + lookup_expr="iexact", + label=labels.ASSET_FILTERS_NAME_LABEL, + help_text=labels.ASSET_FILTERS_NAME_HELP) + test__engagement__product__name_contains = CharFilter( + field_name="test__engagement__product__name", + lookup_expr="icontains", + label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL, + help_text=labels.ASSET_FILTERS_NAME_CONTAINS_HELP) + test__engagement__name = CharFilter( + field_name="test__engagement__name", + lookup_expr="iexact", + label="Engagement Name", + help_text="Search for Engagement names that are an exact match") + test__engagement__name_contains = CharFilter( + field_name="test__engagement__name", + lookup_expr="icontains", + label="Engagement name Contains", + help_text="Search for Engagement names that contain a given pattern") + test__name = CharFilter( + field_name="test__title", + lookup_expr="iexact", + label="Test Name", + help_text="Search for Test names that are an exact match") + test__name_contains = CharFilter( + field_name="test__title", + lookup_expr="icontains", + label="Test name Contains", + help_text="Search for Test names that contain a given pattern") + + if is_finding_groups_enabled(): + finding_group__name = CharFilter( + field_name="finding_group__name", + lookup_expr="iexact", + label="Finding Group Name", + help_text="Search for Finding Group names that are an exact match") + finding_group__name_contains = CharFilter( + field_name="finding_group__name", + lookup_expr="icontains", + label="Finding Group Name Contains", + help_text="Search for Finding Group names that contain a given pattern") + + class Meta: + model = Finding + fields = get_finding_filterset_fields(filter_string_matching=True) + + exclude = ["url", "description", "mitigation", "impact", + "endpoints", "references", + "thread_id", "notes", "scanner_confidence", + "numerical_severity", "line", "duplicate_finding", + "hash_code", "reviewers", "created", "files", + "sla_start_date", "sla_expiration_date", "cvssv3", + "severity_justification", "steps_to_reproduce"] + + def __init__(self, *args, **kwargs): + self.user = None + self.pid = None + self.eid = None + self.tid = None + if "user" in kwargs: + self.user = kwargs.pop("user") + + if "pid" in kwargs: + self.pid = kwargs.pop("pid") + if "eid" in kwargs: + self.eid = kwargs.pop("eid") + if "tid" in kwargs: + self.tid = kwargs.pop("tid") + super().__init__(*args, **kwargs) + # Set some date fields + self.set_date_fields(*args, **kwargs) + # Don't show the product/engagement/test filter fields when in specific context + if self.tid or self.eid or self.pid: + if "test__engagement__product__name" in self.form.fields: + del self.form.fields["test__engagement__product__name"] + if "test__engagement__product__name_contains" in self.form.fields: + del self.form.fields["test__engagement__product__name_contains"] + if "test__engagement__product__prod_type__name" in self.form.fields: + del self.form.fields["test__engagement__product__prod_type__name"] + if "test__engagement__product__prod_type__name_contains" in self.form.fields: + del self.form.fields["test__engagement__product__prod_type__name_contains"] + # Also hide engagement and test fields if in test or engagement context + if self.tid: + if "test__engagement__name" in self.form.fields: + del self.form.fields["test__engagement__name"] + if "test__engagement__name_contains" in self.form.fields: + del self.form.fields["test__engagement__name_contains"] + if "test__name" in self.form.fields: + del self.form.fields["test__name"] + if "test__name_contains" in self.form.fields: + del self.form.fields["test__name_contains"] + elif self.eid: + if "test__engagement__name" in self.form.fields: + del self.form.fields["test__engagement__name"] + if "test__engagement__name_contains" in self.form.fields: + del self.form.fields["test__engagement__name_contains"] + + +class FindingFilter(FindingFilterHelper, FindingTagFilter): + reporter = ModelMultipleChoiceFilter(queryset=Dojo_User.objects.none()) + reviewers = ModelMultipleChoiceFilter(queryset=Dojo_User.objects.none()) + test__engagement__product__prod_type = ModelMultipleChoiceFilter( + queryset=Product_Type.objects.none(), + label=labels.ORG_FILTERS_LABEL) + test__engagement__product = ModelMultipleChoiceFilter( + queryset=Product.objects.none(), + label=labels.ASSET_FILTERS_LABEL) + test__engagement = ModelMultipleChoiceFilter( + queryset=Engagement.objects.none(), + label="Engagement") + test = ModelMultipleChoiceFilter( + queryset=Test.objects.none(), + label="Test") + + if is_finding_groups_enabled(): + finding_group = ModelMultipleChoiceFilter( + queryset=Finding_Group.objects.none(), + label="Finding Group") + + class Meta: + model = Finding + fields = get_finding_filterset_fields() + + exclude = ["url", "description", "mitigation", "impact", + "endpoints", "references", + "thread_id", "notes", "scanner_confidence", + "numerical_severity", "line", "duplicate_finding", + "hash_code", "reviewers", "created", "files", + "sla_start_date", "sla_expiration_date", "cvssv3", + "severity_justification", "steps_to_reproduce"] + + def __init__(self, *args, **kwargs): + self.user = None + self.pid = None + self.eid = None + self.tid = None + if "user" in kwargs: + self.user = kwargs.pop("user") + + if "pid" in kwargs: + self.pid = kwargs.pop("pid") + if "eid" in kwargs: + self.eid = kwargs.pop("eid") + if "tid" in kwargs: + self.tid = kwargs.pop("tid") + super().__init__(*args, **kwargs) + # Set some date fields + self.set_date_fields(*args, **kwargs) + # Don't show the product filter on the product finding view + self.set_related_object_fields(*args, **kwargs) + + def set_related_object_fields(self, *args: list, **kwargs: dict): + # Use helper to get contextual finding group queryset + finding_group_query = get_finding_group_queryset_for_context( + pid=self.pid, + eid=self.eid, + tid=self.tid, + ) + + # Filter by most specific context: test > engagement > product + if self.tid is not None: + # Test context: filter finding groups by test + if "test__engagement__product" in self.form.fields: + del self.form.fields["test__engagement__product"] + if "test__engagement__product__prod_type" in self.form.fields: + del self.form.fields["test__engagement__product__prod_type"] + if "test__engagement" in self.form.fields: + del self.form.fields["test__engagement"] + if "test" in self.form.fields: + del self.form.fields["test"] + elif self.eid is not None: + # Engagement context: filter finding groups by engagement + if "test__engagement__product" in self.form.fields: + del self.form.fields["test__engagement__product"] + if "test__engagement__product__prod_type" in self.form.fields: + del self.form.fields["test__engagement__product__prod_type"] + if "test__engagement" in self.form.fields: + del self.form.fields["test__engagement"] + # Filter tests by engagement - get_authorized_tests doesn't support engagement param + engagement = Engagement.objects.filter(id=self.eid).select_related("product").first() + if engagement: + self.form.fields["test"].queryset = get_authorized_tests("view", product=engagement.product).filter(engagement_id=self.eid).prefetch_related("test_type") + elif self.pid is not None: + # Product context: filter finding groups by product + if "test__engagement__product" in self.form.fields: + del self.form.fields["test__engagement__product"] + if "test__engagement__product__prod_type" in self.form.fields: + del self.form.fields["test__engagement__product__prod_type"] + # TODO: add authorized check to be sure + if "test__engagement" in self.form.fields: + self.form.fields["test__engagement"].queryset = Engagement.objects.filter( + product_id=self.pid, + ).all() + if "test" in self.form.fields: + self.form.fields["test"].queryset = get_authorized_tests("view", product=self.pid).prefetch_related("test_type") + else: + # Global context: show all authorized finding groups + self.form.fields[ + "test__engagement__product__prod_type"].queryset = get_authorized_product_types("view") + self.form.fields["test__engagement"].queryset = get_authorized_engagements("view") + if "test" in self.form.fields: + del self.form.fields["test"] + + if self.form.fields.get("test__engagement__product"): + self.form.fields["test__engagement__product"].queryset = get_authorized_products("view") + if self.form.fields.get("finding_group", None): + self.form.fields["finding_group"].queryset = get_authorized_finding_groups_for_queryset("view", finding_group_query, user=self.user) + self.form.fields["reporter"].queryset = get_authorized_users("view") + self.form.fields["reviewers"].queryset = self.form.fields["reporter"].queryset + + +class FindingGroupsFilter(FilterSet): + name = CharFilter(lookup_expr="icontains", label="Name") + severity = ChoiceFilter( + choices=[ + ("Low", "Low"), + ("Medium", "Medium"), + ("High", "High"), + ("Critical", "Critical"), + ], + label="Min Severity", + ) + engagement = ModelMultipleChoiceFilter(queryset=Engagement.objects.none(), label="Engagement") + product = ModelMultipleChoiceFilter(queryset=Product.objects.none(), label=labels.ASSET_LABEL) + + class Meta: + model = Finding + fields = ["name", "severity", "engagement", "product"] + + def __init__(self, *args, **kwargs): + self.user = kwargs.pop("user", None) + self.pid = kwargs.pop("pid", None) + super().__init__(*args, **kwargs) + self.set_related_object_fields() + + def set_related_object_fields(self): + if self.pid is not None: + self.form.fields["engagement"].queryset = Engagement.objects.filter(product_id=self.pid) + if "product" in self.form.fields: + del self.form.fields["product"] + else: + self.form.fields["product"].queryset = get_authorized_products("view") + self.form.fields["engagement"].queryset = get_authorized_engagements("view") + + +class AcceptedFindingFilter(FindingFilter): + risk_acceptance__created__date = DateRangeFilter(label="Acceptance Date") + risk_acceptance__owner = ModelMultipleChoiceFilter( + queryset=Dojo_User.objects.none(), + label="Risk Acceptance Owner") + risk_acceptance = ModelMultipleChoiceFilter( + queryset=Risk_Acceptance.objects.none(), + label="Accepted By") + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.form.fields["risk_acceptance__owner"].queryset = get_authorized_users("view") + self.form.fields["risk_acceptance"].queryset = get_authorized_risk_acceptances("edit") + + +class AcceptedFindingFilterWithoutObjectLookups(FindingFilterWithoutObjectLookups): + risk_acceptance__created__date = DateRangeFilter(label="Acceptance Date") + risk_acceptance__owner = CharFilter( + field_name="risk_acceptance__owner__username", + lookup_expr="iexact", + label="Risk Acceptance Owner Username", + help_text="Search for Risk Acceptance Owners username that are an exact match") + risk_acceptance__owner_contains = CharFilter( + field_name="risk_acceptance__owner__username", + lookup_expr="icontains", + label="Risk Acceptance Owner Username Contains", + help_text="Search for Risk Acceptance Owners username that contain a given pattern") + risk_acceptance__name = CharFilter( + field_name="risk_acceptance__name", + lookup_expr="iexact", + label="Risk Acceptance Name", + help_text="Search for Risk Acceptance name that are an exact match") + risk_acceptance__name_contains = CharFilter( + field_name="risk_acceptance__name", + lookup_expr="icontains", + label="Risk Acceptance Name", + help_text="Search for Risk Acceptance name contain a given pattern") + + +class SimilarFindingHelper(FilterSet): + hash_code = MultipleChoiceFilter() + vulnerability_ids = CharFilter(method=custom_vulnerability_id_filter, label="Vulnerability Ids") + + def update_data(self, data: dict, *args: list, **kwargs: dict): + # if filterset is bound, use initial values as defaults + # because of this, we can't rely on the self.form.has_changed + self.has_changed = True + if not data and self.finding: + # get a mutable copy of the QueryDict + data = data.copy() + + data["vulnerability_ids"] = ",".join(self.finding.vulnerability_ids) + data["cwe"] = self.finding.cwe + data["file_path"] = self.finding.file_path + data["line"] = self.finding.line + data["unique_id_from_tool"] = self.finding.unique_id_from_tool + data["test__test_type"] = self.finding.test.test_type + data["test__engagement__product"] = self.finding.test.engagement.product + data["test__engagement__product__prod_type"] = self.finding.test.engagement.product.prod_type + + self.has_changed = False + + def set_hash_codes(self, *args: list, **kwargs: dict): + if self.finding and self.finding.hash_code: + self.form.fields["hash_code"] = forms.MultipleChoiceField(choices=[(self.finding.hash_code, self.finding.hash_code[:24] + "...")], required=False, initial=[]) + + def filter_queryset(self, *args: list, **kwargs: dict): + queryset = super().filter_queryset(*args, **kwargs) + queryset = get_authorized_findings_for_queryset("view", queryset, self.user) + return queryset.exclude(pk=self.finding.pk) + + +class SimilarFindingFilter(FindingFilter, SimilarFindingHelper): + class Meta(FindingFilter.Meta): + model = Finding + # slightly different fields from FindingFilter, but keep the same ordering for UI consistency + fields = get_finding_filterset_fields(similar=True) + + def __init__(self, data=None, *args, **kwargs): + self.user = None + if "user" in kwargs: + self.user = kwargs.pop("user") + self.finding = None + if "finding" in kwargs: + self.finding = kwargs.pop("finding") + self.update_data(data, *args, **kwargs) + super().__init__(data, *args, **kwargs) + self.set_hash_codes(*args, **kwargs) + + +class SimilarFindingFilterWithoutObjectLookups(FindingFilterWithoutObjectLookups, SimilarFindingHelper): + class Meta(FindingFilterWithoutObjectLookups.Meta): + model = Finding + # slightly different fields from FindingFilter, but keep the same ordering for UI consistency + fields = get_finding_filterset_fields(similar=True, filter_string_matching=True) + + def __init__(self, data=None, *args, **kwargs): + self.user = None + if "user" in kwargs: + self.user = kwargs.pop("user") + self.finding = None + if "finding" in kwargs: + self.finding = kwargs.pop("finding") + self.update_data(data, *args, **kwargs) + super().__init__(data, *args, **kwargs) + self.set_hash_codes(*args, **kwargs) + + +class TemplateFindingFilter(DojoFilter): + title = CharFilter(lookup_expr="icontains") + cwe = MultipleChoiceFilter(choices=[]) + severity = MultipleChoiceFilter(choices=SEVERITY_CHOICES) + + tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + queryset=Finding.tags.tag_model.objects.all().order_by("name"), + # label='tags', # doesn't work with tagulous, need to set in __init__ below + ) + + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") + + not_tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + exclude=True, + queryset=Finding.tags.tag_model.objects.all().order_by("name"), + # label='tags', # doesn't work with tagulous, need to set in __init__ below + ) + + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("cwe", "cwe"), + ("title", "title"), + ("numerical_severity", "numerical_severity"), + ), + field_labels={ + "numerical_severity": "Severity", + }, + ) + + class Meta: + model = Finding_Template + exclude = ["description", "mitigation", "impact", + "references", "numerical_severity"] + + not_test__tags = ModelMultipleChoiceFilter( + field_name="test__tags__name", + to_field_name="name", + exclude=True, + label="Test without tags", + queryset=Test.tags.tag_model.objects.all().order_by("name"), + # label='tags', # doesn't work with tagulous, need to set in __init__ below + ) + + not_test__engagement__tags = ModelMultipleChoiceFilter( + field_name="test__engagement__tags__name", + to_field_name="name", + exclude=True, + label="Engagement without tags", + queryset=Engagement.tags.tag_model.objects.all().order_by("name"), + # label='tags', # doesn't work with tagulous, need to set in __init__ below + ) + + not_test__engagement__product__tags = ModelMultipleChoiceFilter( + field_name="test__engagement__product__tags__name", + to_field_name="name", + exclude=True, + label=labels.ASSET_FILTERS_WITHOUT_TAGS_LABEL, + queryset=Product.tags.tag_model.objects.all().order_by("name"), + # label='tags', # doesn't work with tagulous, need to set in __init__ below + ) + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.form.fields["cwe"].choices = cwe_options(self.queryset) + + +class MetricsFindingFilter(FindingFilter): + start_date = DateFilter(field_name="date", label="Start Date", lookup_expr=("gt")) + end_date = DateFilter(field_name="date", label="End Date", lookup_expr=("lt")) + date = MetricsDateRangeFilter() + vulnerability_id = CharFilter(method=vulnerability_id_filter, label="Vulnerability Id") + + not_tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + exclude=True, + queryset=Endpoint.tags.tag_model.objects.all().order_by("name"), + # label='tags', # doesn't work with tagulous, need to set in __init__ below + ) + + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) + + def __init__(self, *args, **kwargs): + if args[0]: + if args[0].get("start_date", "") or args[0].get("end_date", ""): + args[0]._mutable = True + args[0]["date"] = 8 + args[0]._mutable = False + + super().__init__(*args, **kwargs) + + class Meta(FindingFilter.Meta): + model = Finding + fields = get_finding_filterset_fields(metrics=True) + + +class MetricsFindingFilterWithoutObjectLookups(FindingFilterWithoutObjectLookups): + start_date = DateFilter(field_name="date", label="Start Date", lookup_expr=("gt")) + end_date = DateFilter(field_name="date", label="End Date", lookup_expr=("lt")) + date = MetricsDateRangeFilter() + vulnerability_id = CharFilter(method=vulnerability_id_filter, label="Vulnerability Id") + + not_tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + exclude=True, + queryset=Endpoint.tags.tag_model.objects.all().order_by("name"), + # label='tags', # doesn't work with tagulous, need to set in __init__ below + ) + + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) + + def __init__(self, *args, **kwargs): + if args[0]: + if args[0].get("start_date", "") or args[0].get("end_date", ""): + args[0]._mutable = True + args[0]["date"] = 8 + args[0]._mutable = False + + super().__init__(*args, **kwargs) + + class Meta(FindingFilterWithoutObjectLookups.Meta): + model = Finding + fields = get_finding_filterset_fields(metrics=True, filter_string_matching=True) + + +class ReportFindingFilterHelper(FilterSet): + title = CharFilter(lookup_expr="icontains", label="Name") + date = DateFromToRangeFilter(field_name="date", label="Date Discovered") + date_recent = DateRangeFilter(field_name="date", label="Relative Date") + severity = MultipleChoiceFilter(choices=SEVERITY_CHOICES) + active = ReportBooleanFilter() + is_mitigated = ReportBooleanFilter() + mitigated = DateRangeFilter(label="Mitigated Date") + verified = ReportBooleanFilter() + false_p = ReportBooleanFilter(label="False Positive") + risk_acceptance = ReportRiskAcceptanceFilter(label="Risk Accepted") + duplicate = ReportBooleanFilter() + out_of_scope = ReportBooleanFilter() + outside_of_sla = FindingSLAFilter(label="Outside of SLA") + file_path = CharFilter(lookup_expr="icontains") + mitigation_available = BooleanFilter(method="filter_mitigation_available", label="Mitigation Available") + + o = OrderingFilter( + fields=( + ("title", "title"), + ("date", "date"), + ("fix_available", "fix_available"), + ("numerical_severity", "numerical_severity"), + ("epss_score", "epss_score"), + ("epss_percentile", "epss_percentile"), + ("test__engagement__product__name", "test__engagement__product__name"), + ), + ) + + class Meta: + model = Finding + # exclude sonarqube issue as by default it will show all without checking permissions + exclude = ["date", "cwe", "url", "description", "mitigation", "impact", + "references", "sonarqube_issue", "duplicate_finding", + "thread_id", "notes", "inherited_tags", "endpoints", + "numerical_severity", "reporter", "last_reviewed", + "jira_creation", "jira_change", "files"] + + def filter_mitigation_available(self, queryset, name, value): + if value: + return queryset.exclude(mitigation__isnull=True).exclude(mitigation__exact="") + return queryset.filter(Q(mitigation__isnull=True) | Q(mitigation__exact="")) + + def manage_kwargs(self, kwargs): + self.prod_type = None + self.product = None + self.engagement = None + self.test = None + if "prod_type" in kwargs: + self.prod_type = kwargs.pop("prod_type") + if "product" in kwargs: + self.product = kwargs.pop("product") + if "engagement" in kwargs: + self.engagement = kwargs.pop("engagement") + if "test" in kwargs: + self.test = kwargs.pop("test") + + @property + def qs(self): + parent = super().qs + return get_authorized_findings_for_queryset("view", parent) + + +class ReportFindingFilter(ReportFindingFilterHelper, FindingTagFilter): + test__engagement__product = ModelMultipleChoiceFilter( + queryset=Product.objects.none(), label=labels.ASSET_FILTERS_LABEL) + test__engagement__product__prod_type = ModelMultipleChoiceFilter( + queryset=Product_Type.objects.none(), + label=labels.ORG_FILTERS_LABEL) + test__engagement__product__lifecycle = MultipleChoiceFilter(choices=Product.LIFECYCLE_CHOICES, label=labels.ASSET_LIFECYCLE_LABEL) + test__engagement = ModelMultipleChoiceFilter(queryset=Engagement.objects.none(), label="Engagement") + duplicate_finding = ModelChoiceFilter(queryset=Finding.objects.filter(original_finding__isnull=False).distinct()) + + def __init__(self, *args, **kwargs): + self.manage_kwargs(kwargs) + super().__init__(*args, **kwargs) + + # duplicate_finding queryset needs to restricted in line with permissions + # and inline with report scope to avoid a dropdown with 100K entries + duplicate_finding_query_set = self.form.fields["duplicate_finding"].queryset + duplicate_finding_query_set = get_authorized_findings_for_queryset("view", duplicate_finding_query_set) + + if self.test: + duplicate_finding_query_set = duplicate_finding_query_set.filter(test=self.test) + del self.form.fields["test__tags"] + del self.form.fields["test__engagement__tags"] + del self.form.fields["test__engagement__product__tags"] + if self.engagement: + duplicate_finding_query_set = duplicate_finding_query_set.filter(test__engagement=self.engagement) + del self.form.fields["test__engagement__tags"] + del self.form.fields["test__engagement__product__tags"] + elif self.product: + duplicate_finding_query_set = duplicate_finding_query_set.filter(test__engagement__product=self.product) + del self.form.fields["test__engagement__product"] + del self.form.fields["test__engagement__product__tags"] + elif self.prod_type: + duplicate_finding_query_set = duplicate_finding_query_set.filter(test__engagement__product__prod_type=self.prod_type) + del self.form.fields["test__engagement__product__prod_type"] + + self.form.fields["duplicate_finding"].queryset = duplicate_finding_query_set + + if "test__engagement__product__prod_type" in self.form.fields: + self.form.fields[ + "test__engagement__product__prod_type"].queryset = get_authorized_product_types("view") + if "test__engagement__product" in self.form.fields: + self.form.fields[ + "test__engagement__product"].queryset = get_authorized_products("view") + if "test__engagement" in self.form.fields: + self.form.fields["test__engagement"].queryset = get_authorized_engagements("view") + + +class ReportFindingFilterWithoutObjectLookups(ReportFindingFilterHelper, FindingTagStringFilter): + test__engagement__product__prod_type = NumberFilter(widget=HiddenInput()) + test__engagement__product = NumberFilter(widget=HiddenInput()) + test__engagement = NumberFilter(widget=HiddenInput()) + test = NumberFilter(widget=HiddenInput()) + endpoint = NumberFilter(widget=HiddenInput()) + reporter = CharFilter( + field_name="reporter__username", + lookup_expr="iexact", + label="Reporter Username", + help_text="Search for Reporter names that are an exact match") + reporter_contains = CharFilter( + field_name="reporter__username", + lookup_expr="icontains", + label="Reporter Username Contains", + help_text="Search for Reporter names that contain a given pattern") + reviewers = CharFilter( + field_name="reviewers__username", + lookup_expr="iexact", + label="Reviewer Username", + help_text="Search for Reviewer names that are an exact match") + reviewers_contains = CharFilter( + field_name="reviewers__username", + lookup_expr="icontains", + label="Reviewer Username Contains", + help_text="Search for Reviewer usernames that contain a given pattern") + last_reviewed_by = CharFilter( + field_name="last_reviewed_by__username", + lookup_expr="iexact", + label="Last Reviewed By Username", + help_text="Search for Last Reviewed By names that are an exact match") + last_reviewed_by_contains = CharFilter( + field_name="last_reviewed_by__username", + lookup_expr="icontains", + label="Last Reviewed By Username Contains", + help_text="Search for Last Reviewed By usernames that contain a given pattern") + review_requested_by = CharFilter( + field_name="review_requested_by__username", + lookup_expr="iexact", + label="Review Requested By Username", + help_text="Search for Review Requested By names that are an exact match") + review_requested_by_contains = CharFilter( + field_name="review_requested_by__username", + lookup_expr="icontains", + label="Review Requested By Username Contains", + help_text="Search for Review Requested By usernames that contain a given pattern") + mitigated_by = CharFilter( + field_name="mitigated_by__username", + lookup_expr="iexact", + label="Mitigator Username", + help_text="Search for Mitigator names that are an exact match") + mitigated_by_contains = CharFilter( + field_name="mitigated_by__username", + lookup_expr="icontains", + label="Mitigator Username Contains", + help_text="Search for Mitigator usernames that contain a given pattern") + defect_review_requested_by = CharFilter( + field_name="defect_review_requested_by__username", + lookup_expr="iexact", + label="Requester of Defect Review Username", + help_text="Search for Requester of Defect Review names that are an exact match") + defect_review_requested_by_contains = CharFilter( + field_name="defect_review_requested_by__username", + lookup_expr="icontains", + label="Requester of Defect Review Username Contains", + help_text="Search for Requester of Defect Review usernames that contain a given pattern") + test__engagement__product__prod_type__name = CharFilter( + field_name="test__engagement__product__prod_type__name", + lookup_expr="iexact", + label=labels.ORG_FILTERS_NAME_LABEL, + help_text=labels.ORG_FILTERS_NAME_HELP) + test__engagement__product__prod_type__name_contains = CharFilter( + field_name="test__engagement__product__prod_type__name", + lookup_expr="icontains", + label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, + help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) + test__engagement__product__name = CharFilter( + field_name="test__engagement__product__name", + lookup_expr="iexact", + label=labels.ASSET_FILTERS_NAME_LABEL, + help_text=labels.ASSET_FILTERS_NAME_HELP) + test__engagement__product__name_contains = CharFilter( + field_name="test__engagement__product__name", + lookup_expr="icontains", + label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL, + help_text=labels.ASSET_FILTERS_NAME_CONTAINS_HELP) + test__engagement__name = CharFilter( + field_name="test__engagement__name", + lookup_expr="iexact", + label="Engagement Name", + help_text="Search for Engagement names that are an exact match") + test__engagement__name_contains = CharFilter( + field_name="test__engagement__name", + lookup_expr="icontains", + label="Engagement name Contains", + help_text="Search for Engagement names that contain a given pattern") + test__name = CharFilter( + field_name="test__title", + lookup_expr="iexact", + label="Test Name", + help_text="Search for Test names that are an exact match") + test__name_contains = CharFilter( + field_name="test__title", + lookup_expr="icontains", + label="Test name Contains", + help_text="Search for Test names that contain a given pattern") + + def __init__(self, *args, **kwargs): + self.manage_kwargs(kwargs) + super().__init__(*args, **kwargs) + + product_type_refs = [ + "test__engagement__product__prod_type__name", + "test__engagement__product__prod_type__name_contains", + ] + product_refs = [ + "test__engagement__product__name", + "test__engagement__product__name_contains", + "test__engagement__product__tags", + "test__engagement__product__tags_contains", + "not_test__engagement__product__tags", + "not_test__engagement__product__tags_contains", + ] + engagement_refs = [ + "test__engagement__name", + "test__engagement__name_contains", + "test__engagement__tags", + "test__engagement__tags_contains", + "not_test__engagement__tags", + "not_test__engagement__tags_contains", + ] + test_refs = [ + "test__name", + "test__name_contains", + "test__tags", + "test__tags_contains", + "not_test__tags", + "not_test__tags_contains", + ] + + if self.test: + self.delete_tags_from_form(product_type_refs) + self.delete_tags_from_form(product_refs) + self.delete_tags_from_form(engagement_refs) + self.delete_tags_from_form(test_refs) + elif self.engagement: + self.delete_tags_from_form(product_type_refs) + self.delete_tags_from_form(product_refs) + self.delete_tags_from_form(engagement_refs) + elif self.product: + self.delete_tags_from_form(product_type_refs) + self.delete_tags_from_form(product_refs) + elif self.prod_type: + self.delete_tags_from_form(product_type_refs) diff --git a/dojo/finding/ui/forms.py b/dojo/finding/ui/forms.py new file mode 100644 index 00000000000..79f3e317059 --- /dev/null +++ b/dojo/finding/ui/forms.py @@ -0,0 +1,1083 @@ +import tagulous +from django import forms +from django.conf import settings +from django.core.exceptions import ValidationError +from django.utils.safestring import mark_safe +from django.utils.translation import gettext_lazy as _ +from tagulous.forms import TagField + +from dojo.endpoint.utils import validate_endpoints_to_add +from dojo.finding.queries import get_authorized_findings +from dojo.jira import services as jira_services +from dojo.location.models import Location +from dojo.location.utils import validate_locations_to_add +from dojo.models import ( + EFFORT_FOR_FIXING_CHOICES, + SEVERITY_CHOICES, + Dojo_User, + Endpoint, + Finding, + Finding_Group, + Finding_Template, + Notes, + Risk_Acceptance, + Test, +) +from dojo.user.queries import get_authorized_users, get_authorized_users_for_product_and_product_type +from dojo.utils import get_system_setting, is_finding_groups_enabled +from dojo.validators import cvss3_validator, cvss4_validator, tag_validator +from dojo.widgets import TableCheckboxWidget + +CVSS_CALCULATOR_URLS = { + "https://www.first.org/cvss/calculator/3-0": "CVSS3 Calculator by FIRST", + "https://www.first.org/cvss/calculator/4-0": "CVSS4 Calculator by FIRST", + "https://www.metaeffekt.com/security/cvss/calculator/": "CVSS2/3/4 Calculator by Metaeffekt", + } + + +vulnerability_ids_field = forms.CharField(max_length=5000, + required=False, + label="Vulnerability Ids", + help_text="Ids of vulnerabilities in security advisories associated with this finding. Can be Common Vulnerabilities and Exposures (CVE) or from other sources." + "You may enter one vulnerability id per line.", + widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) + +EFFORT_FOR_FIXING_INVALID_CHOICE = _("Select valid choice: Low,Medium,High") + + +class BulletListDisplayWidget(forms.Widget): + def __init__(self, urls_dict=None, *args, **kwargs): + self.urls_dict = urls_dict or {} + super().__init__(*args, **kwargs) + + def render(self, name, value, attrs=None, renderer=None): + if not self.urls_dict: + return "" + + html = '
    ' + for url, text in self.urls_dict.items(): + html += f'
  • {text}
    • ' + html += "
" + return mark_safe(html) + + +def hide_cvss_fields_if_disabled(form_instance): + """Hide CVSS fields based on system settings.""" + enable_cvss3 = get_system_setting("enable_cvss3_display", True) + enable_cvss4 = get_system_setting("enable_cvss4_display", True) + + # Hide CVSS3 fields if disabled + if not enable_cvss3: + if "cvssv3" in form_instance.fields: + del form_instance.fields["cvssv3"] + if "cvssv3_score" in form_instance.fields: + del form_instance.fields["cvssv3_score"] + + # Hide CVSS4 fields if disabled + if not enable_cvss4: + if "cvssv4" in form_instance.fields: + del form_instance.fields["cvssv4"] + if "cvssv4_score" in form_instance.fields: + del form_instance.fields["cvssv4_score"] + + # If both are disabled, hide all CVSS related fields + if not enable_cvss3 and not enable_cvss4: + if "cvss_info" in form_instance.fields: + del form_instance.fields["cvss_info"] + + +class EditFindingGroupForm(forms.ModelForm): + name = forms.CharField(max_length=255, required=True, label="Finding Group Name") + jira_issue = forms.CharField(max_length=255, required=False, label="Linked JIRA Issue", + help_text="Leave empty and check push to jira to create a new JIRA issue for this finding group.") + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.fields["push_to_jira"] = forms.BooleanField() + self.fields["push_to_jira"].required = False + self.fields["push_to_jira"].help_text = "Checking this will overwrite content of your JIRA issue, or create one." + + self.fields["push_to_jira"].label = "Push to JIRA" + + if hasattr(self.instance, "has_jira_issue") and self.instance.has_jira_issue: + jira_url = jira_services.get_url(self.instance) + self.fields["jira_issue"].initial = jira_url + self.fields["push_to_jira"].widget.attrs["checked"] = "checked" + + class Meta: + model = Finding_Group + fields = ["name"] + + +class DeleteFindingGroupForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Finding_Group + fields = ["id"] + + +class MergeFindings(forms.ModelForm): + FINDING_ACTION = (("", "Select an Action"), ("inactive", "Inactive"), ("delete", "Delete")) + + append_description = forms.BooleanField(label="Append Description", initial=True, required=False, + help_text="Description in all findings will be appended into the merged finding.") + + add_endpoints = forms.BooleanField(label="Add Endpoints", initial=True, required=False, + help_text="Endpoints in all findings will be merged into the merged finding.") + + dynamic_raw = forms.BooleanField(label="Dynamic Scanner Raw Requests", initial=True, required=False, + help_text="Dynamic scanner raw requests in all findings will be merged into the merged finding.") + + tag_finding = forms.BooleanField(label="Add Tags", initial=True, required=False, + help_text="Tags in all findings will be merged into the merged finding.") + + mark_tag_finding = forms.BooleanField(label="Tag Merged Finding", initial=True, required=False, + help_text="Creates a tag titled 'merged' for the finding that will be merged. If the 'Finding Action' is set to 'inactive' the inactive findings will be tagged with 'merged-inactive'.") + + append_reference = forms.BooleanField(label="Append Reference", initial=True, required=False, + help_text="Reference in all findings will be appended into the merged finding.") + + finding_action = forms.ChoiceField( + required=True, + choices=FINDING_ACTION, + label="Finding Action", + help_text="The action to take on the merged finding. Set the findings to inactive or delete the findings.") + + def __init__(self, *args, **kwargs): + _ = kwargs.pop("finding") + findings = kwargs.pop("findings") + super().__init__(*args, **kwargs) + + self.fields["finding_to_merge_into"] = forms.ModelChoiceField( + queryset=findings, initial=0, required="False", label="Finding to Merge Into", help_text="Findings selected below will be merged into this finding.") + + # Exclude the finding to merge into from the findings to merge into + self.fields["findings_to_merge"] = forms.ModelMultipleChoiceField( + queryset=findings, required=True, label="Findings to Merge", + widget=forms.widgets.SelectMultiple(attrs={"size": 10}), + help_text=("Select the findings to merge.")) + self.field_order = ["finding_to_merge_into", "findings_to_merge", "append_description", "add_endpoints", "append_reference"] + + class Meta: + model = Finding + fields = ["append_description", "add_endpoints", "append_reference"] + + +class AddFindingsRiskAcceptanceForm(forms.ModelForm): + + accepted_findings = forms.ModelMultipleChoiceField( + queryset=Finding.objects.none(), + required=True, + label="", + widget=TableCheckboxWidget(attrs={"size": 25}), + ) + + class Meta: + model = Risk_Acceptance + fields = ["accepted_findings"] + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.fields["accepted_findings"].queryset = get_authorized_findings("edit") + + +class AddFindingForm(forms.ModelForm): + title = forms.CharField(max_length=1000) + date = forms.DateField(required=True, + widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) + cwe = forms.IntegerField(required=False) + vulnerability_ids = vulnerability_ids_field + cvssv3 = forms.CharField(label="CVSS3 Vector", max_length=117, required=False, widget=forms.TextInput(attrs={"class": "cvsscalculator", "data-toggle": "dropdown", "aria-haspopup": "true", "aria-expanded": "false"})) + cvssv3_score = forms.FloatField(label="CVSS3 Score", required=False, max_value=10.0, min_value=0.0) + cvssv4 = forms.CharField(label="CVSS4 Vector", max_length=255, required=False) + cvssv4_score = forms.FloatField(label="CVSS4 Score", required=False, max_value=10.0, min_value=0.0) + description = forms.CharField(widget=forms.Textarea) + severity = forms.ChoiceField( + choices=SEVERITY_CHOICES, + error_messages={ + "required": "Select valid choice: In Progress, On Hold, Completed", + "invalid_choice": EFFORT_FOR_FIXING_INVALID_CHOICE}) + mitigation = forms.CharField(widget=forms.Textarea, required=False) + impact = forms.CharField(widget=forms.Textarea, required=False) + request = forms.CharField(widget=forms.Textarea, required=False) + response = forms.CharField(widget=forms.Textarea, required=False) + endpoints = forms.ModelMultipleChoiceField(Location.objects.none(), required=False, label="Systems / Endpoints") + endpoints_to_add = forms.CharField(max_length=5000, required=False, label="Endpoints to add", + help_text="The IP address, host name or full URL. You may enter one endpoint per line. " + "Each must be valid.", + widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) + references = forms.CharField(widget=forms.Textarea, required=False) + publish_date = forms.DateField(widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"}), required=False) + planned_remediation_date = forms.DateField(widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"}), required=False) + planned_remediation_version = forms.CharField(max_length=99, required=False) + effort_for_fixing = forms.ChoiceField( + required=False, + choices=EFFORT_FOR_FIXING_CHOICES, + error_messages={ + "invalid_choice": EFFORT_FOR_FIXING_INVALID_CHOICE}) + + # the only reliable way without hacking internal fields to get predicatble ordering is to make it explicit + field_order = ("title", "date", "cwe", "vulnerability_ids", "severity", "cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", "description", "mitigation", "impact", "request", "response", "steps_to_reproduce", + "severity_justification", "endpoints", "endpoints_to_add", "references", "active", "verified", "false_p", "duplicate", "out_of_scope", + "risk_accepted", "under_defect_review") + + def __init__(self, *args, **kwargs): + req_resp = kwargs.pop("req_resp") + + product = None + if "product" in kwargs: + product = kwargs.pop("product") + + super().__init__(*args, **kwargs) + + if settings.V3_FEATURE_LOCATIONS and product: + self.fields["endpoints"].queryset = Location.objects.filter(products__product=product) + # TODO: Delete this after the move to Locations + elif product: + self.fields["endpoints"].queryset = Endpoint.objects.filter(product=product) + else: + self.fields["endpoints"].queryset = Endpoint.objects.none() + + if req_resp: + self.fields["request"].initial = req_resp[0] + self.fields["response"].initial = req_resp[1] + + self.endpoints_to_add_list = [] + + # Hide CVSS fields based on system settings + hide_cvss_fields_if_disabled(self) + + def clean(self): + cleaned_data = super().clean() + if ((cleaned_data["active"] or cleaned_data["verified"]) and cleaned_data["duplicate"]): + msg = "Duplicate findings cannot be verified or active" + raise forms.ValidationError(msg) + if cleaned_data["false_p"] and cleaned_data["verified"]: + msg = "False positive findings cannot be verified." + raise forms.ValidationError(msg) + if cleaned_data["active"] and "risk_accepted" in cleaned_data and cleaned_data["risk_accepted"]: + msg = "Active findings cannot be risk accepted." + raise forms.ValidationError(msg) + + if settings.V3_FEATURE_LOCATIONS: + endpoints_to_add_list, errors = validate_locations_to_add(cleaned_data["endpoints_to_add"]) + else: + # TODO: Delete this after the move to Locations + endpoints_to_add_list, errors = validate_endpoints_to_add(cleaned_data["endpoints_to_add"]) + + if errors: + raise forms.ValidationError(errors) + self.endpoints_to_add_list = endpoints_to_add_list + + return cleaned_data + + def clean_tags(self): + tag_validator(self.cleaned_data.get("tags")) + return self.cleaned_data.get("tags") + + class Meta: + model = Finding + exclude = ("reporter", "url", "numerical_severity", "under_review", "reviewers", "cve", "inherited_tags", + "review_requested_by", "is_mitigated", "jira_creation", "jira_change", "endpoints", "sla_start_date") + + +class AdHocFindingForm(forms.ModelForm): + title = forms.CharField(max_length=1000) + date = forms.DateField(required=True, + widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) + cwe = forms.IntegerField(required=False) + vulnerability_ids = vulnerability_ids_field + + cvss_info = forms.CharField( + label="CVSS", + widget=BulletListDisplayWidget(CVSS_CALCULATOR_URLS), + required=False, + disabled=True) + + cvssv3 = forms.CharField(label="CVSS3 Vector", max_length=117, required=False, widget=forms.TextInput(attrs={"class": "cvsscalculator", "data-toggle": "dropdown", "aria-haspopup": "true", "aria-expanded": "false"})) + cvssv3_score = forms.FloatField(label="CVSS3 Score", required=False, max_value=10.0, min_value=0.0) + cvssv4 = forms.CharField(label="CVSS4 Vector", max_length=255, required=False) + cvssv4_score = forms.FloatField(label="CVSS4 Score", required=False, max_value=10.0, min_value=0.0) + description = forms.CharField(widget=forms.Textarea) + severity = forms.ChoiceField( + choices=SEVERITY_CHOICES, + error_messages={ + "required": "Select valid choice: In Progress, On Hold, Completed", + "invalid_choice": EFFORT_FOR_FIXING_INVALID_CHOICE}) + mitigation = forms.CharField(widget=forms.Textarea, required=False) + impact = forms.CharField(widget=forms.Textarea, required=False) + request = forms.CharField(widget=forms.Textarea, required=False) + response = forms.CharField(widget=forms.Textarea, required=False) + endpoints = forms.ModelMultipleChoiceField(queryset=Location.objects.all(), required=False, + label="Systems / Endpoints") + endpoints_to_add = forms.CharField(max_length=5000, required=False, label="Endpoints to add", + help_text="The IP address, host name or full URL. You may enter one endpoint per line. " + "Each must be valid.", + widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) + references = forms.CharField(widget=forms.Textarea, required=False) + publish_date = forms.DateField(widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"}), required=False) + planned_remediation_date = forms.DateField(widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"}), required=False) + planned_remediation_version = forms.CharField(max_length=99, required=False) + effort_for_fixing = forms.ChoiceField( + required=False, + choices=EFFORT_FOR_FIXING_CHOICES, + error_messages={ + "invalid_choice": EFFORT_FOR_FIXING_INVALID_CHOICE}) + + # the only reliable way without hacking internal fields to get predicatble ordering is to make it explicit + field_order = ("title", "date", "cwe", "vulnerability_ids", "severity", "cvss_info", "cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", "description", "mitigation", + "impact", "request", "response", "steps_to_reproduce", "severity_justification", "endpoints", "endpoints_to_add", "references", + "active", "verified", "false_p", "duplicate", "out_of_scope", "risk_accepted", "under_defect_review", "sla_start_date", "sla_expiration_date") + + def __init__(self, *args, **kwargs): + req_resp = kwargs.pop("req_resp") + + product = None + if "product" in kwargs: + product = kwargs.pop("product") + + super().__init__(*args, **kwargs) + + if settings.V3_FEATURE_LOCATIONS and product: + self.fields["endpoints"].queryset = Location.objects.filter(products__product=product) + # TODO: Delete this after the move to Locations + elif product: + self.fields["endpoints"].queryset = Endpoint.objects.filter(product=product) + else: + self.fields["endpoints"].queryset = Endpoint.objects.none() + + if req_resp: + self.fields["request"].initial = req_resp[0] + self.fields["response"].initial = req_resp[1] + + self.endpoints_to_add_list = [] + + # Hide CVSS fields based on system settings + hide_cvss_fields_if_disabled(self) + + def clean(self): + cleaned_data = super().clean() + if ((cleaned_data["active"] or cleaned_data["verified"]) and cleaned_data["duplicate"]): + msg = "Duplicate findings cannot be verified or active" + raise forms.ValidationError(msg) + if cleaned_data["false_p"] and cleaned_data["verified"]: + msg = "False positive findings cannot be verified." + raise forms.ValidationError(msg) + + if settings.V3_FEATURE_LOCATIONS: + endpoints_to_add_list, errors = validate_locations_to_add(cleaned_data["endpoints_to_add"]) + else: + # TODO: Delete this after the move to Locations + endpoints_to_add_list, errors = validate_endpoints_to_add(cleaned_data["endpoints_to_add"]) + + self.endpoints_to_add_list = endpoints_to_add_list + + if errors: + raise forms.ValidationError(errors) + + return cleaned_data + + def clean_tags(self): + tag_validator(self.cleaned_data.get("tags")) + return self.cleaned_data.get("tags") + + class Meta: + model = Finding + exclude = ("reporter", "url", "numerical_severity", "under_review", "reviewers", "cve", "inherited_tags", + "review_requested_by", "is_mitigated", "jira_creation", "jira_change", "endpoints", "sla_start_date", + "sla_expiration_date") + + +class PromoteFindingForm(forms.ModelForm): + title = forms.CharField(max_length=1000) + date = forms.DateField(required=True, + widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) + cwe = forms.IntegerField(required=False) + vulnerability_ids = vulnerability_ids_field + + cvss_info = forms.CharField( + label="CVSS", + widget=BulletListDisplayWidget(CVSS_CALCULATOR_URLS), + required=False, + disabled=True) + + cvssv3 = forms.CharField(label="CVSS3 Vector", max_length=117, required=False, widget=forms.TextInput(attrs={"class": "cvsscalculator", "data-toggle": "dropdown", "aria-haspopup": "true", "aria-expanded": "false"})) + cvssv3_score = forms.FloatField(label="CVSS3 Score", required=False, max_value=10.0, min_value=0.0) + cvssv4 = forms.CharField(label="CVSS4 Vector", max_length=255, required=False) + cvssv4_score = forms.FloatField(label="CVSS4 Score", required=False, max_value=10.0, min_value=0.0) + description = forms.CharField(widget=forms.Textarea) + severity = forms.ChoiceField( + choices=SEVERITY_CHOICES, + error_messages={ + "required": "Select valid choice: In Progress, On Hold, Completed", + "invalid_choice": "Select valid choice: Critical,High,Medium,Low"}) + mitigation = forms.CharField(widget=forms.Textarea, required=False) + impact = forms.CharField(widget=forms.Textarea, required=False) + endpoints = forms.ModelMultipleChoiceField(Location.objects.none(), required=False, label="Systems / Endpoints") + endpoints_to_add = forms.CharField(max_length=5000, required=False, label="Endpoints to add", + help_text="The IP address, host name or full URL. You may enter one endpoint per line. " + "Each must be valid.", + widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) + references = forms.CharField(widget=forms.Textarea, required=False) + + # the onyl reliable way without hacking internal fields to get predicatble ordering is to make it explicit + field_order = ("title", "group", "date", "sla_start_date", "sla_expiration_date", "cwe", "vulnerability_ids", "severity", "cvss_info", "cvssv3", + "cvssv3_score", "cvssv4", "cvssv4_score", "description", "mitigation", "impact", "request", "response", "steps_to_reproduce", + "severity_justification", "endpoints", "endpoints_to_add", "references", "active", "mitigated", "mitigated_by", "verified", + "false_p", "duplicate", "out_of_scope", "risk_accept", "under_defect_review") + + def __init__(self, *args, **kwargs): + product = None + if "product" in kwargs: + product = kwargs.pop("product") + + super().__init__(*args, **kwargs) + + if settings.V3_FEATURE_LOCATIONS and product: + self.fields["endpoints"].queryset = Location.objects.filter(products__product=product) + # TODO: Delete this after the move to Locations + elif product: + self.fields["endpoints"].queryset = Endpoint.objects.filter(product=product) + else: + self.fields["endpoints"].queryset = Endpoint.objects.none() + + self.endpoints_to_add_list = [] + + # Hide CVSS fields based on system settings + hide_cvss_fields_if_disabled(self) + + def clean(self): + cleaned_data = super().clean() + + if settings.V3_FEATURE_LOCATIONS: + endpoints_to_add_list, errors = validate_locations_to_add(cleaned_data["endpoints_to_add"]) + else: + # TODO: Delete this after the move to Locations + endpoints_to_add_list, errors = validate_endpoints_to_add(cleaned_data["endpoints_to_add"]) + + if errors: + raise forms.ValidationError(errors) + self.endpoints_to_add_list = endpoints_to_add_list + + return cleaned_data + + def clean_tags(self): + tag_validator(self.cleaned_data.get("tags")) + return self.cleaned_data.get("tags") + + class Meta: + model = Finding + exclude = ("reporter", "url", "numerical_severity", "active", "false_p", "verified", "endpoint_status", "cve", "inherited_tags", + "duplicate", "out_of_scope", "under_review", "reviewers", "review_requested_by", "is_mitigated", "jira_creation", "jira_change", "planned_remediation_date", "planned_remediation_version", "effort_for_fixing") + + +class FindingForm(forms.ModelForm): + title = forms.CharField(max_length=1000) + group = forms.ModelChoiceField(required=False, queryset=Finding_Group.objects.none(), help_text="The Finding Group to which this finding belongs, leave empty to remove the finding from the group. Groups can only be created via Bulk Edit for now.") + date = forms.DateField(required=True, + widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) + cwe = forms.IntegerField(required=False) + vulnerability_ids = vulnerability_ids_field + + cvss_info = forms.CharField( + label="CVSS", + widget=BulletListDisplayWidget(CVSS_CALCULATOR_URLS), + required=False, + disabled=True) + + cvssv3 = forms.CharField(label="CVSS3 Vector", max_length=117, required=False, widget=forms.TextInput(attrs={"class": "cvsscalculator", "data-toggle": "dropdown", "aria-haspopup": "true", "aria-expanded": "false"})) + cvssv3_score = forms.FloatField(label="CVSS3 Score", required=False, max_value=10.0, min_value=0.0) + cvssv4 = forms.CharField(label="CVSS4 Vector", max_length=255, required=False) + cvssv4_score = forms.FloatField(label="CVSS4 Score", required=False, max_value=10.0, min_value=0.0) + + description = forms.CharField(widget=forms.Textarea) + severity = forms.ChoiceField( + choices=SEVERITY_CHOICES, + error_messages={ + "required": "Select valid choice: In Progress, On Hold, Completed", + "invalid_choice": "Select valid choice: Critical,High,Medium,Low"}) + mitigation = forms.CharField(widget=forms.Textarea, required=False) + impact = forms.CharField(widget=forms.Textarea, required=False) + request = forms.CharField(widget=forms.Textarea, required=False) + response = forms.CharField(widget=forms.Textarea, required=False) + endpoints = forms.ModelMultipleChoiceField(queryset=Location.objects.none(), required=False, label="Systems / Endpoints") + endpoints_to_add = forms.CharField(max_length=5000, required=False, label="Endpoints to add", + help_text="The IP address, host name or full URL. You may enter one endpoint per line. " + "Each must be valid.", + widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) + references = forms.CharField(widget=forms.Textarea, required=False) + + mitigated = forms.DateField(required=False, help_text="Date and time when the flaw has been fixed", widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) + mitigated_by = forms.ModelChoiceField(required=False, queryset=Dojo_User.objects.none()) + + publish_date = forms.DateField(widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"}), required=False) + planned_remediation_date = forms.DateField(widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"}), required=False) + planned_remediation_version = forms.CharField(max_length=99, required=False) + effort_for_fixing = forms.ChoiceField( + required=False, + choices=EFFORT_FOR_FIXING_CHOICES, + error_messages={ + "invalid_choice": EFFORT_FOR_FIXING_INVALID_CHOICE}) + + # the only reliable way without hacking internal fields to get predicatble ordering is to make it explicit + field_order = ("title", "group", "date", "sla_start_date", "sla_expiration_date", "cwe", "vulnerability_ids", "severity", "cvss_info", "cvssv3", + "cvssv3_score", "cvssv4", "cvssv4_score", "description", "mitigation", "impact", "request", "response", "steps_to_reproduce", "severity_justification", + "endpoints", "endpoints_to_add", "references", "active", "mitigated", "mitigated_by", "verified", "false_p", "duplicate", + "out_of_scope", "risk_accept", "under_defect_review") + + def __init__(self, *args, **kwargs): + req_resp = None + if "req_resp" in kwargs: + req_resp = kwargs.pop("req_resp") + + self.can_edit_mitigated_data = kwargs.pop("can_edit_mitigated_data") if "can_edit_mitigated_data" in kwargs \ + else False + + super().__init__(*args, **kwargs) + + if settings.V3_FEATURE_LOCATIONS: + self.fields["endpoints"].queryset = Location.objects.filter(products__product=self.instance.test.engagement.product) + if self.instance and self.instance.pk: + self.fields["endpoints"].initial = Location.objects.filter(findings__finding=self.instance) + else: + # TODO: Delete this after the move to Locations + self.fields["endpoints"].queryset = Endpoint.objects.filter(product=self.instance.test.engagement.product) + if self.instance and self.instance.pk: + self.fields["endpoints"].initial = self.instance.endpoints.all() + + self.fields["mitigated_by"].queryset = get_authorized_users("edit") + + # do not show checkbox if finding is not accepted and simple risk acceptance is disabled + # if checked, always show to allow unaccept also with full risk acceptance enabled + # when adding from template, we don't have access to the test. quickfix for now to just hide simple risk acceptance + if not hasattr(self.instance, "test") or (not self.instance.risk_accepted and not self.instance.test.engagement.product.enable_simple_risk_acceptance): + del self.fields["risk_accepted"] + elif self.instance.risk_accepted: + self.fields["risk_accepted"].help_text = "Uncheck to unaccept the risk. Use full risk acceptance from the dropdown menu if you need advanced settings such as an expiry date." + elif self.instance.test.engagement.product.enable_simple_risk_acceptance: + self.fields["risk_accepted"].help_text = "Check to accept the risk. Use full risk acceptance from the dropdown menu if you need advanced settings such as an expiry date." + + # self.fields['tags'].widget.choices = t + if req_resp: + self.fields["request"].initial = req_resp[0] + self.fields["response"].initial = req_resp[1] + + if self.instance.duplicate: + self.fields["duplicate"].help_text = "Original finding that is being duplicated here (readonly). Use view finding page to manage duplicate relationships. Unchecking duplicate here will reset this findings duplicate status, but will trigger deduplication logic." + else: + self.fields["duplicate"].help_text = "You can mark findings as duplicate only from the view finding page." + + self.fields["sla_start_date"].disabled = True + self.fields["sla_expiration_date"].disabled = True + + if self.can_edit_mitigated_data: + if hasattr(self, "instance"): + self.fields["mitigated"].initial = self.instance.mitigated + self.fields["mitigated_by"].initial = self.instance.mitigated_by + else: + del self.fields["mitigated"] + del self.fields["mitigated_by"] + + if not is_finding_groups_enabled() or not hasattr(self.instance, "test"): + del self.fields["group"] + else: + self.fields["group"].queryset = self.instance.test.finding_group_set.all() + self.fields["group"].initial = self.instance.finding_group + + self.endpoints_to_add_list = [] + + # Hide CVSS fields based on system settings + hide_cvss_fields_if_disabled(self) + + def clean(self): + cleaned_data = super().clean() + + if (cleaned_data["active"] or cleaned_data["verified"]) and cleaned_data["duplicate"]: + msg = "Duplicate findings cannot be verified or active" + raise forms.ValidationError(msg) + if cleaned_data["false_p"] and cleaned_data["verified"]: + msg = "False positive findings cannot be verified." + raise forms.ValidationError(msg) + if cleaned_data["active"] and "risk_accepted" in cleaned_data and cleaned_data["risk_accepted"]: + msg = "Active findings cannot be risk accepted." + raise forms.ValidationError(msg) + + if settings.V3_FEATURE_LOCATIONS: + endpoints_to_add_list, errors = validate_locations_to_add(cleaned_data["endpoints_to_add"]) + else: + # TODO: Delete this after the move to Locations + endpoints_to_add_list, errors = validate_endpoints_to_add(cleaned_data["endpoints_to_add"]) + + self.endpoints_to_add_list = endpoints_to_add_list + + if errors: + raise forms.ValidationError(errors) + + return cleaned_data + + def clean_tags(self): + tag_validator(self.cleaned_data.get("tags")) + return self.cleaned_data.get("tags") + + def _post_clean(self): + super()._post_clean() + + if self.can_edit_mitigated_data: + opts = self.instance._meta + try: + opts.get_field("mitigated").save_form_data(self.instance, self.cleaned_data.get("mitigated")) + opts.get_field("mitigated_by").save_form_data(self.instance, self.cleaned_data.get("mitigated_by")) + except forms.ValidationError as e: + self._update_errors(e) + + class Meta: + model = Finding + exclude = ("reporter", "url", "numerical_severity", "under_review", "reviewers", "cve", "inherited_tags", + "review_requested_by", "is_mitigated", "jira_creation", "jira_change", "sonarqube_issue", + "endpoints", "endpoint_status") + + +class ApplyFindingTemplateForm(forms.Form): + + title = forms.CharField(max_length=1000, required=True) + + cwe = forms.IntegerField(label="CWE", required=False) + vulnerability_ids = vulnerability_ids_field + cvssv3 = forms.CharField(label="CVSSv3", max_length=117, required=False, widget=forms.TextInput(attrs={"class": "btn btn-secondary dropdown-toggle", "data-toggle": "dropdown", "aria-haspopup": "true", "aria-expanded": "false"})) + cvssv3_score = forms.FloatField(required=False, label="CVSSv3 Score") + cvssv4 = forms.CharField(label="CVSSv4", max_length=255, required=False) + cvssv4_score = forms.FloatField(required=False, label="CVSSv4 Score") + + severity = forms.ChoiceField(required=False, choices=SEVERITY_CHOICES, error_messages={"required": "Select valid choice: In Progress, On Hold, Completed", "invalid_choice": "Select valid choice: Critical,High,Medium,Low"}) + + description = forms.CharField(widget=forms.Textarea) + mitigation = forms.CharField(widget=forms.Textarea, required=False) + impact = forms.CharField(widget=forms.Textarea, required=False) + references = forms.CharField(widget=forms.Textarea, required=False) + + # Remediation planning fields + fix_available = forms.BooleanField(required=False) + fix_version = forms.CharField(max_length=100, required=False) + planned_remediation_version = forms.CharField(max_length=99, required=False) + effort_for_fixing = forms.CharField(max_length=99, required=False) + + # Technical details fields + steps_to_reproduce = forms.CharField(widget=forms.Textarea, required=False) + severity_justification = forms.CharField(widget=forms.Textarea, required=False) + component_name = forms.CharField(max_length=500, required=False) + component_version = forms.CharField(max_length=100, required=False) + + # Notes field + notes = forms.CharField(widget=forms.Textarea, required=False, help_text="Note content to add when applying template") + + # Endpoints field + endpoints = forms.CharField(max_length=5000, required=False, + help_text="Endpoint URLs (one per line)", + widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) + + tags = TagField(required=False, help_text="Add tags that help describe this finding template. Choose from the list or add new tags. Press Enter key to add.", initial=Finding.tags.tag_model.objects.all().order_by("name")) + + def __init__(self, template=None, *args, **kwargs): + super().__init__(*args, **kwargs) + self.fields["tags"].autocomplete_tags = Finding.tags.tag_model.objects.all().order_by("name") + self.template = template + if template: + # Populate vulnerability_ids field initial value + self.fields["vulnerability_ids"].initial = "\n".join(template.vulnerability_ids) + + # Populate CVSS fields from template + if hasattr(template, "cvssv3"): + self.fields["cvssv3"].initial = template.cvssv3 + if hasattr(template, "cvssv4"): + self.fields["cvssv4"].initial = template.cvssv4 + if hasattr(template, "cvssv3_score"): + self.fields["cvssv3_score"].initial = template.cvssv3_score + if hasattr(template, "cvssv4_score"): + self.fields["cvssv4_score"].initial = template.cvssv4_score + + # Populate all other new fields from template + for field_name in ["fix_available", "fix_version", "planned_remediation_version", + "effort_for_fixing", "steps_to_reproduce", "severity_justification", + "component_name", "component_version", "notes"]: + if hasattr(template, field_name): + value = getattr(template, field_name) + if value is not None: + self.fields[field_name].initial = value + + # Populate endpoints + if hasattr(template, "endpoints"): + endpoints_value = template.endpoints + if endpoints_value: + if isinstance(endpoints_value, list): + self.fields["endpoints"].initial = "\n".join(endpoints_value) + else: + self.fields["endpoints"].initial = endpoints_value + + # Hide CVSS fields based on system settings + hide_cvss_fields_if_disabled(self) + + def clean(self): + cleaned_data = super().clean() + + if "title" in cleaned_data: + if len(cleaned_data["title"]) <= 0: + msg = "The title is required." + raise forms.ValidationError(msg) + else: + msg = "The title is required." + raise forms.ValidationError(msg) + + return cleaned_data + + def clean_tags(self): + tag_validator(self.cleaned_data.get("tags")) + return self.cleaned_data.get("tags") + + class Meta: + fields = ["title", "cwe", "vulnerability_ids", "cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", + "severity", "description", "mitigation", "impact", "references", "tags", + "fix_available", "fix_version", "planned_remediation_version", "effort_for_fixing", + "steps_to_reproduce", "severity_justification", "component_name", "component_version", + "notes", "endpoints"] + order = ("title", "cwe", "vulnerability_ids", "cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", + "severity", "description", "impact", "steps_to_reproduce", "severity_justification", + "mitigation", "fix_available", "fix_version", "planned_remediation_version", + "effort_for_fixing", "component_name", "component_version", "references", "notes", + "endpoints", "tags") + + +class FindingTemplateForm(forms.ModelForm): + title = forms.CharField(max_length=1000, required=True) + + cwe = forms.IntegerField(label="CWE", required=False) + vulnerability_ids = vulnerability_ids_field + cvssv3 = forms.CharField(label="CVSS3 Vector", max_length=117, required=False, widget=forms.TextInput(attrs={"class": "btn btn-secondary dropdown-toggle", "data-toggle": "dropdown", "aria-haspopup": "true", "aria-expanded": "false"})) + cvssv3_score = forms.FloatField(required=False, label="CVSSv3 Score") + cvssv4 = forms.CharField(label="CVSS4 Vector", max_length=255, required=False) + cvssv4_score = forms.FloatField(required=False, label="CVSSv4 Score") + severity = forms.ChoiceField( + required=False, + choices=SEVERITY_CHOICES, + error_messages={ + "required": "Select valid choice: In Progress, On Hold, Completed", + "invalid_choice": "Select valid choice: Critical,High,Medium,Low"}) + + # Remediation planning fields + fix_available = forms.BooleanField(required=False) + fix_version = forms.CharField(max_length=100, required=False) + planned_remediation_version = forms.CharField(max_length=99, required=False) + effort_for_fixing = forms.CharField(max_length=99, required=False) + + # Technical details fields + steps_to_reproduce = forms.CharField(widget=forms.Textarea, required=False) + severity_justification = forms.CharField(widget=forms.Textarea, required=False) + component_name = forms.CharField(max_length=500, required=False) + component_version = forms.CharField(max_length=100, required=False) + + # Notes field + notes = forms.CharField(widget=forms.Textarea, required=False, help_text="Note content to add when applying template") + + # Endpoints field + endpoints = forms.CharField(max_length=5000, required=False, + help_text="Endpoint URLs (one per line)", + widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) + + field_order = ["title", "cwe", "vulnerability_ids", "severity", "cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", + "description", "impact", "steps_to_reproduce", "severity_justification", "mitigation", + "fix_available", "fix_version", "planned_remediation_version", "effort_for_fixing", + "component_name", "component_version", "references", "notes", "endpoints", "tags"] + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.fields["tags"].autocomplete_tags = Finding.tags.tag_model.objects.all().order_by("name") + + # Hide CVSS fields based on system settings + hide_cvss_fields_if_disabled(self) + + class Meta: + model = Finding_Template + order = ("title", "cwe", "vulnerability_ids", "cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", "severity", "description", "impact", + "steps_to_reproduce", "severity_justification", "mitigation", "fix_available", "fix_version", + "planned_remediation_version", "effort_for_fixing", "component_name", "component_version", + "references", "notes", "endpoints", "tags") + exclude = ("numerical_severity", "is_mitigated", "last_used", "endpoint_status", "cve", "vulnerability_ids_text") + + def clean_cvssv3(self): + value = self.cleaned_data.get("cvssv3") + if value: + try: + cvss3_validator(value) + except ValidationError as e: + raise forms.ValidationError(e.messages) + return value + + def clean_cvssv4(self): + value = self.cleaned_data.get("cvssv4") + if value: + try: + cvss4_validator(value) + except ValidationError as e: + raise forms.ValidationError(e.messages) + return value + + def clean_tags(self): + tag_validator(self.cleaned_data.get("tags")) + return self.cleaned_data.get("tags") + + +class DeleteFindingTemplateForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Finding_Template + fields = ["id"] + + +class FindingBulkUpdateForm(forms.ModelForm): + status = forms.BooleanField(required=False) + risk_acceptance = forms.BooleanField(required=False) + risk_accept = forms.BooleanField(required=False) + risk_unaccept = forms.BooleanField(required=False) + + date = forms.DateField(required=False, widget=forms.DateInput(attrs={"class": "datepicker"})) + planned_remediation_date = forms.DateField(required=False, widget=forms.DateInput(attrs={"class": "datepicker"})) + planned_remediation_version = forms.CharField(required=False, max_length=99, widget=forms.TextInput(attrs={"class": "form-control"})) + finding_group = forms.BooleanField(required=False) + finding_group_create = forms.BooleanField(required=False) + finding_group_create_name = forms.CharField(required=False) + finding_group_add = forms.BooleanField(required=False) + add_to_finding_group_id = forms.CharField(required=False) + finding_group_remove = forms.BooleanField(required=False) + finding_group_by = forms.BooleanField(required=False) + finding_group_by_option = forms.CharField(required=False) + + push_to_jira = forms.BooleanField(required=False) + # unlink_from_jira = forms.BooleanField(required=False) + push_to_github = forms.BooleanField(required=False) + tags = TagField(required=False, autocomplete_tags=Finding.tags.tag_model.objects.all().order_by("name")) + notes = forms.CharField(required=False, max_length=1024, widget=forms.TextInput(attrs={"class": "form-control"})) + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.fields["severity"].required = False + # we need to defer initialization to prevent multiple initializations if other forms are shown + self.fields["tags"].widget.tag_options = tagulous.models.options.TagOptions(autocomplete_settings={"width": "200px", "defer": True}) + if disclaimer := get_system_setting("disclaimer_notes"): + self.disclaimer = disclaimer.strip() + + def clean(self): + cleaned_data = super().clean() + + if (cleaned_data["active"] or cleaned_data["verified"]) and cleaned_data["duplicate"]: + msg = "Duplicate findings cannot be verified or active" + raise forms.ValidationError(msg) + if cleaned_data["false_p"] and cleaned_data["verified"]: + msg = "False positive findings cannot be verified." + raise forms.ValidationError(msg) + if cleaned_data["active"] and cleaned_data.get("risk_acceptance") and cleaned_data.get("risk_accept"): + msg = "Active findings cannot be risk accepted." + raise forms.ValidationError(msg) + return cleaned_data + + def clean_tags(self): + tag_validator(self.cleaned_data.get("tags")) + return self.cleaned_data.get("tags") + + class Meta: + model = Finding + fields = ("severity", "date", "planned_remediation_date", "active", "verified", "false_p", "duplicate", "out_of_scope", + "under_review", "is_mitigated") + + +class CloseFindingForm(forms.ModelForm): + entry = forms.CharField( + required=True, max_length=2400, + widget=forms.Textarea, label="Notes:", + error_messages={"required": ("The reason for closing a finding is " + "required, please use the text area " + "below to provide documentation.")}) + + mitigated = forms.DateField(required=False, help_text="Date and time when the flaw has been fixed", widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) + mitigated_by = forms.ModelChoiceField(required=False, queryset=Dojo_User.objects.none()) + false_p = forms.BooleanField(initial=False, required=False, label="False Positive") + out_of_scope = forms.BooleanField(initial=False, required=False, label="Out of Scope") + duplicate = forms.BooleanField(initial=False, required=False, label="Duplicate") + + def __init__(self, *args, **kwargs): + queryset = kwargs.pop("missing_note_types") + # must pop custom kwargs before calling parent __init__ to avoid unexpected kwarg errors + self.can_edit_mitigated_data = kwargs.pop("can_edit_mitigated_data") if "can_edit_mitigated_data" in kwargs \ + else False + super().__init__(*args, **kwargs) + if len(queryset) == 0: + self.fields["note_type"].widget = forms.HiddenInput() + else: + self.fields["note_type"] = forms.ModelChoiceField(queryset=queryset, label="Note Type", required=True) + + if self.can_edit_mitigated_data: + self.fields["mitigated_by"].queryset = get_authorized_users("edit") + self.fields["mitigated"].initial = self.instance.mitigated + self.fields["mitigated_by"].initial = self.instance.mitigated_by + if disclaimer := get_system_setting("disclaimer_notes"): + self.disclaimer = disclaimer.strip() + + def _post_clean(self): + super()._post_clean() + + if self.can_edit_mitigated_data: + opts = self.instance._meta + if not self.cleaned_data.get("active"): + try: + opts.get_field("mitigated").save_form_data(self.instance, self.cleaned_data.get("mitigated")) + opts.get_field("mitigated_by").save_form_data(self.instance, self.cleaned_data.get("mitigated_by")) + except forms.ValidationError as e: + self._update_errors(e) + + class Meta: + model = Notes + fields = ["note_type", "entry", "mitigated", "mitigated_by", "false_p", "out_of_scope", "duplicate"] + + +class EditPlannedRemediationDateFindingForm(forms.ModelForm): + def __init__(self, *args, **kwargs): + finding = None + if "finding" in kwargs: + finding = kwargs.pop("finding") + + super().__init__(*args, **kwargs) + + self.fields["planned_remediation_date"].required = True + self.fields["planned_remediation_date"].widget = forms.DateInput(attrs={"class": "datepicker"}) + + if finding is not None: + self.fields["planned_remediation_date"].initial = finding.planned_remediation_date + + class Meta: + model = Finding + fields = ["planned_remediation_date"] + + +class DefectFindingForm(forms.ModelForm): + CLOSE_CHOICES = (("Close Finding", "Close Finding"), ("Not Fixed", "Not Fixed")) + defect_choice = forms.ChoiceField(required=True, choices=CLOSE_CHOICES) + + entry = forms.CharField( + required=True, max_length=2400, + widget=forms.Textarea, label="Notes:", + error_messages={"required": ("The reason for closing a finding is " + "required, please use the text area " + "below to provide documentation.")}) + + class Meta: + model = Notes + fields = ["entry"] + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + if disclaimer := get_system_setting("disclaimer_notes"): + self.disclaimer = disclaimer.strip() + + +class ClearFindingReviewForm(forms.ModelForm): + entry = forms.CharField( + required=True, max_length=2400, + help_text="Please provide a message.", + widget=forms.Textarea, label="Notes:", + error_messages={"required": ("The reason for clearing a review is " + "required, please use the text area " + "below to provide documentation.")}) + + class Meta: + model = Finding + fields = ["active", "verified", "false_p", "out_of_scope", "duplicate", "is_mitigated"] + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + if disclaimer := get_system_setting("disclaimer_notes"): + self.disclaimer = disclaimer.strip() + + +class ReviewFindingForm(forms.Form): + reviewers = forms.MultipleChoiceField( + help_text=( + "Select all users who can review Finding. Only users with " + "at least write permission to this finding can be selected"), + required=False, + ) + entry = forms.CharField( + required=True, max_length=2400, + help_text="Please provide a message for reviewers.", + widget=forms.Textarea, label="Notes:", + error_messages={"required": ("The reason for requesting a review is " + "required, please use the text area " + "below to provide documentation.")}) + allow_all_reviewers = forms.BooleanField( + required=False, + label="Allow All Eligible Reviewers", + help_text=("Checking this box will allow any user in the drop down " + "above to provide a review for this finding")) + + def __init__(self, *args, **kwargs): + finding = kwargs.pop("finding", None) + kwargs.pop("user", None) + super().__init__(*args, **kwargs) + # Get the list of users + if finding is not None: + users = get_authorized_users_for_product_and_product_type(None, finding.test.engagement.product, "edit") + else: + users = get_authorized_users("edit").filter(is_active=True) + # Save a copy of the original query to be used in the validator + self.reviewer_queryset = users + # Set the users in the form + self.fields["reviewers"].choices = self._get_choices(self.reviewer_queryset) + if disclaimer := get_system_setting("disclaimer_notes"): + self.disclaimer = disclaimer.strip() + + @staticmethod + def _get_choices(queryset): + return [(item.pk, item.get_full_name()) for item in queryset] + + def clean(self): + cleaned_data = super().clean() + if cleaned_data.get("allow_all_reviewers", False): + cleaned_data["reviewers"] = [user.id for user in self.reviewer_queryset] + if len(cleaned_data.get("reviewers", [])) == 0: + msg = "Please select at least one user from the reviewers list" + raise ValidationError(msg) + return cleaned_data + + class Meta: + fields = ["reviewers", "entry", "allow_all_reviewers"] + + +class DeleteFindingForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Finding + fields = ["id"] + + +class CopyFindingForm(forms.Form): + test = forms.ModelChoiceField( + required=True, + queryset=Test.objects.none(), + error_messages={"required": "*"}) + + def __init__(self, *args, **kwargs): + authorized_lists = kwargs.pop("tests", None) + super().__init__(*args, **kwargs) + self.fields["test"].queryset = authorized_lists + + +class FindingFormID(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Finding + fields = ("id",) diff --git a/dojo/finding/urls.py b/dojo/finding/ui/urls.py similarity index 99% rename from dojo/finding/urls.py rename to dojo/finding/ui/urls.py index 96bceeec4e8..dd3929cf19b 100644 --- a/dojo/finding/urls.py +++ b/dojo/finding/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from dojo.finding import views +from dojo.finding.ui import views urlpatterns = [ # CRUD operations diff --git a/dojo/finding/ui/views.py b/dojo/finding/ui/views.py new file mode 100644 index 00000000000..e4ed5327e1c --- /dev/null +++ b/dojo/finding/ui/views.py @@ -0,0 +1,3408 @@ +# # findings +import base64 +import contextlib +import copy +import logging +import mimetypes +from collections import OrderedDict, defaultdict +from itertools import chain +from pathlib import Path + +import pghistory +from django.conf import settings +from django.contrib import messages +from django.core import serializers +from django.core.exceptions import PermissionDenied, ValidationError +from django.db import models +from django.db.models import Case, F, QuerySet, Value, When +from django.db.models.functions import Coalesce, ExtractDay, Length, TruncDate +from django.db.models.query import Prefetch +from django.http import Http404, HttpRequest, HttpResponse, HttpResponseRedirect, JsonResponse, StreamingHttpResponse +from django.shortcuts import get_object_or_404, render +from django.urls import reverse +from django.utils import timezone +from django.utils.safestring import mark_safe +from django.utils.translation import gettext as _ +from django.views import View +from django.views.decorators.http import require_POST +from imagekit import ImageSpec +from imagekit.processors import ResizeToFill + +import dojo.finding.helper as finding_helper +import dojo.risk_acceptance.helper as ra_helper +from dojo.authorization.authorization import user_has_global_permission_or_403, user_has_permission_or_403 +from dojo.celery_dispatch import dojo_dispatch_task +from dojo.endpoint.queries import get_authorized_endpoints +from dojo.finding.deduplication import ( + _fetch_fp_candidates_for_batch, + do_false_positive_history_batch, + match_finding_to_existing_findings, +) +from dojo.finding.queries import get_authorized_findings, get_authorized_findings_for_queryset, prefetch_for_findings +from dojo.finding.ui.filters import ( + AcceptedFindingFilter, + AcceptedFindingFilterWithoutObjectLookups, + FindingFilter, + FindingFilterWithoutObjectLookups, + SimilarFindingFilter, + SimilarFindingFilterWithoutObjectLookups, + TemplateFindingFilter, +) +from dojo.finding.ui.forms import ( + ApplyFindingTemplateForm, + ClearFindingReviewForm, + CloseFindingForm, + CopyFindingForm, + DefectFindingForm, + DeleteFindingForm, + DeleteFindingTemplateForm, + EditPlannedRemediationDateFindingForm, + FindingBulkUpdateForm, + FindingForm, + FindingTemplateForm, + MergeFindings, + ReviewFindingForm, +) +from dojo.forms import ( + GITHUBFindingForm, + JIRAFindingForm, + NoteForm, + TypedNoteForm, +) +from dojo.jira import services as jira_services +from dojo.location.queries import get_authorized_locations +from dojo.location.status import FindingLocationStatus +from dojo.models import ( + IMPORT_UNTOUCHED_FINDING, + BurpRawRequestResponse, + Dojo_User, + Endpoint_Status, + Engagement, + FileAccessToken, + Finding, + Finding_Group, + Finding_Template, + GITHUB_Issue, + GITHUB_PKey, + Note_Type, + NoteHistory, + Notes, + Product, + System_Settings, + Test, + Test_Import, + Test_Import_Finding_Action, + User, +) +from dojo.notifications.helper import create_notification +from dojo.tags.utils import bulk_add_tags_to_instances +from dojo.test.queries import get_authorized_tests +from dojo.test.ui.filters import TestImportFilter, TestImportFindingActionFilter +from dojo.tools import tool_issue_updater +from dojo.utils import ( + FileIterWrapper, + Product_Tab, + add_breadcrumb, + add_error_message_to_response, + add_external_issue, + add_field_errors_to_response, + add_success_message_to_response, + calculate_grade, + get_page_items, + get_page_items_and_count, + get_return_url, + get_system_setting, + get_visible_scan_types, + get_words_for_field, + process_tag_notifications, + redirect, + redirect_to_return_url_or_else, + reopen_external_issue, + update_external_issue, +) + +JFORM_PUSH_TO_JIRA_MESSAGE = "jform.push_to_jira: %s" + +logger = logging.getLogger(__name__) + + +def prefetch_for_similar_findings(findings): + prefetched_findings = findings + if isinstance( + findings, QuerySet, + ): # old code can arrive here with prods being a list because the query was already executed + prefetched_findings = prefetched_findings.prefetch_related("reporter") + prefetched_findings = prefetched_findings.prefetch_related( + "jira_issue__jira_project__jira_instance", + ) + prefetched_findings = prefetched_findings.prefetch_related("test__test_type") + prefetched_findings = prefetched_findings.prefetch_related( + "test__engagement__jira_project__jira_instance", + ) + prefetched_findings = prefetched_findings.prefetch_related( + "test__engagement__product__jira_project_set__jira_instance", + ) + prefetched_findings = prefetched_findings.prefetch_related("found_by") + prefetched_findings = prefetched_findings.prefetch_related( + "risk_acceptance_set", + ) + prefetched_findings = prefetched_findings.prefetch_related( + "risk_acceptance_set__accepted_findings", + ) + prefetched_findings = prefetched_findings.prefetch_related("original_finding") + prefetched_findings = prefetched_findings.prefetch_related("duplicate_finding") + # filter out noop reimport actions from finding status history + prefetched_findings = prefetched_findings.prefetch_related( + Prefetch( + "test_import_finding_action_set", + queryset=Test_Import_Finding_Action.objects.exclude( + action=IMPORT_UNTOUCHED_FINDING, + ), + ), + ) + """ + we could try to prefetch only the latest note with SubQuery and OuterRef, + but I'm getting that MySql doesn't support limits in subqueries. + """ + prefetched_findings = prefetched_findings.prefetch_related("notes") + prefetched_findings = prefetched_findings.prefetch_related("tags") + prefetched_findings = prefetched_findings.prefetch_related( + "vulnerability_id_set", + ) + else: + logger.debug("unable to prefetch because query was already executed") + + return prefetched_findings + + +class BaseListFindings: + def __init__( + self, + filter_name: str = "All", + product_id: int | None = None, + engagement_id: int | None = None, + test_id: int | None = None, + order_by: str = "numerical_severity", + prefetch_type: str = "all", + ): + self.filter_name = filter_name + self.product_id = product_id + self.engagement_id = engagement_id + self.test_id = test_id + self.order_by = order_by + self.prefetch_type = prefetch_type + + def get_filter_name(self): + if not hasattr(self, "filter_name"): + self.filter_name = "All" + return self.filter_name + + def get_order_by(self): + if not hasattr(self, "order_by"): + self.order_by = "numerical_severity" + return self.order_by + + def get_prefetch_type(self): + if not hasattr(self, "prefetch_type"): + self.prefetch_type = "all" + return self.prefetch_type + + def get_product_id(self): + if not hasattr(self, "product_id"): + self.product_id = None + return self.product_id + + def get_engagement_id(self): + if not hasattr(self, "engagement_id"): + self.engagement_id = None + return self.engagement_id + + def get_test_id(self): + if not hasattr(self, "test_id"): + self.test_id = None + return self.test_id + + def filter_findings_by_object(self, findings: QuerySet[Finding]): + if product_id := self.get_product_id(): + return findings.filter(test__engagement__product__id=product_id) + if engagement_id := self.get_engagement_id(): + return findings.filter(test__engagement=engagement_id) + if test_id := self.get_test_id(): + return findings.filter(test=test_id) + return findings + + def filter_findings_by_filter_name(self, findings: QuerySet[Finding]): + filter_name = self.get_filter_name() + if filter_name == "Open": + return findings.filter(finding_helper.OPEN_FINDINGS_QUERY) + if filter_name == "Verified": + return findings.filter(finding_helper.VERIFIED_FINDINGS_QUERY) + if filter_name == "Out of Scope": + return findings.filter(finding_helper.OUT_OF_SCOPE_FINDINGS_QUERY) + if filter_name == "False Positive": + return findings.filter(finding_helper.FALSE_POSITIVE_FINDINGS_QUERY) + if filter_name == "Inactive": + return findings.filter(finding_helper.INACTIVE_FINDINGS_QUERY) + if filter_name == "Accepted": + return findings.filter(finding_helper.ACCEPTED_FINDINGS_QUERY) + if filter_name == "Closed": + return findings.filter(finding_helper.CLOSED_FINDINGS_QUERY) + return findings + + def filter_findings_by_form(self, request: HttpRequest, findings: QuerySet[Finding]): + # Apply default ordering if no ordering parameter is provided + # This maintains backward compatibility with the previous behavior + if not request.GET.get("o"): + findings = findings.order_by(self.get_order_by()) + + # Set up the args for the form + args = [request.GET, findings] + # Set the initial form args + kwargs = { + "user": request.user, + "pid": self.get_product_id(), + "eid": self.get_engagement_id(), + "tid": self.get_test_id(), + } + + filter_string_matching = get_system_setting("filter_string_matching", False) + finding_filter_class = FindingFilterWithoutObjectLookups if filter_string_matching else FindingFilter + accepted_finding_filter_class = AcceptedFindingFilterWithoutObjectLookups if filter_string_matching else AcceptedFindingFilter + return ( + accepted_finding_filter_class(*args, **kwargs) + if self.get_filter_name() == "Accepted" + else finding_filter_class(*args, **kwargs) + ) + + def get_filtered_findings(self): + findings = get_authorized_findings("view") + # Annotate computed SLA age in days: sla_expiration_date - (sla_start_date or date) + # Handle NULL sla_expiration_date by using Coalesce to provide a large default value + # so NULLs sort last when sorting ascending (most urgent first) + findings = findings.annotate( + sla_age_days=Coalesce( + ExtractDay( + F("sla_expiration_date") - Coalesce(F("sla_start_date"), TruncDate("created")), + ), + Value(999999), # Large value to push NULLs to the end when sorting ascending + output_field=models.IntegerField(), + ), + ) + # Don't apply initial order_by here - let OrderingFilter handle it via request.GET['o'] + # This prevents conflicts between initial ordering and user-requested sorting + findings = self.filter_findings_by_object(findings) + return self.filter_findings_by_filter_name(findings) + + def get_fully_filtered_findings(self, request: HttpRequest): + findings = self.get_filtered_findings() + return self.filter_findings_by_form(request, findings) + + +class ListFindings(View, BaseListFindings): + def get_initial_context(self, request: HttpRequest): + context = { + "filter_name": self.get_filter_name(), + "show_product_column": True, + "custom_breadcrumb": None, + "product_tab": None, + "jira_project": None, + "github_config": None, + "bulk_edit_form": FindingBulkUpdateForm(request.GET), + "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), + "title_words": get_words_for_field(Finding, "title"), + "component_words": get_words_for_field(Finding, "component_name"), + "visible_test_types": get_visible_scan_types(), + } + # Look to see if the product was used + if product_id := self.get_product_id(): + product = get_object_or_404(Product, id=product_id) + user_has_permission_or_403(request.user, product, "view") + context["show_product_column"] = False + context["product_tab"] = Product_Tab(product, title="Findings", tab="findings") + context["jira_project"] = jira_services.get_project(product) + if github_config := GITHUB_PKey.objects.filter(product=product).first(): + context["github_config"] = github_config.git_conf_id + elif engagement_id := self.get_engagement_id(): + engagement = get_object_or_404(Engagement, id=engagement_id) + user_has_permission_or_403(request.user, engagement, "view") + context["show_product_column"] = False + context["product_tab"] = Product_Tab(engagement.product, title=engagement.name, tab="engagements") + context["jira_project"] = jira_services.get_project(engagement) + if github_config := GITHUB_PKey.objects.filter(product__engagement=engagement).first(): + context["github_config"] = github_config.git_conf_id + + return request, context + + def get_template(self): + return "dojo/findings_list.html" + + def add_breadcrumbs(self, request: HttpRequest, context: dict): + # show custom breadcrumb if user has filtered by exactly 1 endpoint + if "endpoints" in request.GET: + endpoint_ids = request.GET.getlist("endpoints", []) + if len(endpoint_ids) == 1 and endpoint_ids[0]: + endpoint_id = endpoint_ids[0] + # The findings `endpoints` filter is V3-gated (dojo/filters.py): + # under V3 it resolves against Location, otherwise against the + # legacy Endpoint model. Resolve the breadcrumb id against that + # same model so neither mode 404s a valid id. Scope to objects + # the user may view so the breadcrumb cannot disclose the + # existence/URL of one they cannot access (404 for both missing + # and unauthorized ids). + if settings.V3_FEATURE_LOCATIONS: + authorized = get_authorized_locations("view", user=request.user) + else: + authorized = get_authorized_endpoints("view", user=request.user) + endpoint = get_object_or_404(authorized, id=endpoint_id) + context["filter_name"] = "Vulnerable Endpoints" + context["custom_breadcrumb"] = OrderedDict( + [ + ("Endpoints", reverse("vulnerable_endpoints")), + (endpoint, reverse("view_endpoint", args=(endpoint.id,))), + ], + ) + # Show the "All findings" breadcrumb if nothing is coming from the product or engagement + elif not self.get_engagement_id() and not self.get_product_id(): + add_breadcrumb(title="Findings", top_level=not len(request.GET), request=request) + + return request, context + + def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): + # Store the product, engagement, and test ids + self.product_id = product_id + self.engagement_id = engagement_id + self.test_id = test_id + # Get the initial context + request, context = self.get_initial_context(request) + # Get the filtered findings + filtered_findings = self.get_fully_filtered_findings(request) + # trick to prefetch after paging to avoid huge join generated by select count(*) from Paginator + paged_findings = get_page_items(request, filtered_findings.qs, 25) + # prefetch the related objects in the findings + paged_findings.object_list = prefetch_for_findings( + paged_findings.object_list, + self.get_prefetch_type()) + # Add some breadcrumbs + request, context = self.add_breadcrumbs(request, context) + # Add the filtered and paged findings into the context + context |= { + "findings": paged_findings, + "filtered": filtered_findings, + } + # Render the view + return render(request, self.get_template(), context) + + +class ListOpenFindings(ListFindings): + def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): + self.filter_name = "Open" + return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) + + +class ListVerifiedFindings(ListFindings): + def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): + self.filter_name = "Verified" + return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) + + +class ListOutOfScopeFindings(ListFindings): + def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): + self.filter_name = "Out of Scope" + return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) + + +class ListFalsePositiveFindings(ListFindings): + def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): + self.filter_name = "False Positive" + return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) + + +class ListInactiveFindings(ListFindings): + def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): + self.filter_name = "Inactive" + return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) + + +class ListAcceptedFindings(ListFindings): + def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): + self.filter_name = "Accepted" + return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) + + +class ListClosedFindings(ListFindings): + def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): + self.filter_name = "Closed" + self.order_by = "-mitigated" + return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) + + +class ViewFinding(View): + def get_finding(self, finding_id: int): + finding_qs = prefetch_for_findings(Finding.objects.filter(id=finding_id), exclude_untouched=False) + return get_object_or_404(finding_qs, id=finding_id) + + def get_dojo_user(self, request: HttpRequest): + user = request.user + return get_object_or_404(Dojo_User, id=user.id) + + def get_previous_and_next_findings(self, finding: Finding): + # Get the whole list of findings in the current test + findings = ( + Finding.objects.filter(test=finding.test) + .order_by("numerical_severity") + .values_list("id", flat=True) + ) + logger.debug(findings) + # Set some reasonable defaults + next_finding_id = finding.id + prev_finding_id = finding.id + last_pos = (len(findings)) - 1 + # get the index of the current finding + current_finding_index = list(findings).index(finding.id) + # Try to get the previous ID + with contextlib.suppress(IndexError, ValueError): + prev_finding_id = findings[current_finding_index - 1] + # Try to get the next ID + with contextlib.suppress(IndexError, ValueError): + next_finding_id = findings[current_finding_index + 1] + + return { + "prev_finding_id": prev_finding_id, + "next_finding_id": next_finding_id, + "findings_list": findings, + "findings_list_lastElement": findings[last_pos], + } + + def get_request_response(self, finding: Finding): + request_response = None + burp_request = None + burp_response = None + try: + request_response = BurpRawRequestResponse.objects.filter(finding=finding).first() + if request_response is not None: + burp_request = base64.b64decode(request_response.burpRequestBase64) + burp_response = base64.b64decode(request_response.burpResponseBase64) + except Exception as e: + logger.debug("unsuspected error: %s", e) + + return { + "burp_request": burp_request, + "burp_response": burp_response, + } + + def get_test_import_data(self, request: HttpRequest, finding: Finding): + test_imports = Test_Import.objects.filter(findings_affected=finding) + test_import_filter = TestImportFilter(request.GET, test_imports) + + test_import_finding_actions = finding.test_import_finding_action_set + test_import_finding_actions_count = test_import_finding_actions.all().count() + test_import_finding_actions = test_import_finding_actions.filter(test_import__in=test_import_filter.qs) + test_import_finding_action_filter = TestImportFindingActionFilter(request.GET, test_import_finding_actions) + + paged_test_import_finding_actions = get_page_items_and_count(request, test_import_finding_action_filter.qs, 5, prefix="test_import_finding_actions") + paged_test_import_finding_actions.object_list = paged_test_import_finding_actions.object_list.prefetch_related("test_import") + + latest_test_import_finding_action = finding.test_import_finding_action_set.order_by("-created").first + + return { + "test_import_filter": test_import_filter, + "test_import_finding_action_filter": test_import_finding_action_filter, + "paged_test_import_finding_actions": paged_test_import_finding_actions, + "latest_test_import_finding_action": latest_test_import_finding_action, + "test_import_finding_actions_count": test_import_finding_actions_count, + } + + def get_similar_findings(self, request: HttpRequest, finding: Finding): + similar_findings_enabled = get_system_setting("enable_similar_findings", True) + if similar_findings_enabled is False: + return { + "similar_findings_enabled": similar_findings_enabled, + "duplicate_cluster": duplicate_cluster(request, finding), + "similar_findings": None, + "similar_findings_filter": None, + } + # add related actions for non-similar and non-duplicate cluster members + finding.related_actions = calculate_possible_related_actions_for_similar_finding( + request, finding, finding, + ) + if finding.duplicate_finding: + finding.duplicate_finding.related_actions = ( + calculate_possible_related_actions_for_similar_finding( + request, finding, finding.duplicate_finding, + ) + ) + filter_string_matching = get_system_setting("filter_string_matching", False) + finding_filter_class = SimilarFindingFilterWithoutObjectLookups if filter_string_matching else SimilarFindingFilter + similar_findings_filter = finding_filter_class( + request.GET, + queryset=get_authorized_findings("view") + .filter(test__engagement__product=finding.test.engagement.product) + .exclude(id=finding.id), + user=request.user, + finding=finding, + ) + logger.debug("similar query: %s", similar_findings_filter.qs.query) + similar_findings = get_page_items( + request, + similar_findings_filter.qs, + settings.SIMILAR_FINDINGS_MAX_RESULTS, + prefix="similar", + ) + similar_findings.object_list = prefetch_for_similar_findings( + similar_findings.object_list, + ) + for similar_finding in similar_findings: + similar_finding.related_actions = ( + calculate_possible_related_actions_for_similar_finding( + request, finding, similar_finding, + ) + ) + + return { + "similar_findings_enabled": similar_findings_enabled, + "duplicate_cluster": duplicate_cluster(request, finding), + "similar_findings": similar_findings, + "similar_findings_filter": similar_findings_filter, + } + + def get_jira_data(self, finding: Finding): + ( + can_be_pushed_to_jira, + can_be_pushed_to_jira_error, + error_code, + ) = jira_services.can_be_pushed(finding) + # Check the error code + if error_code: + logger.debug(error_code) + + return { + "can_be_pushed_to_jira": can_be_pushed_to_jira, + "can_be_pushed_to_jira_error": can_be_pushed_to_jira_error, + } + + def get_note_form(self, request: HttpRequest): + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = {} + + return NoteForm(*args, **kwargs) + + def get_typed_note_form(self, request: HttpRequest, context: dict): + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = { + "available_note_types": context.get("available_note_types"), + } + + return TypedNoteForm(*args, **kwargs) + + def get_form(self, request: HttpRequest, context: dict): + return ( + self.get_typed_note_form(request, context) + if context.get("note_type_activation") + else self.get_note_form(request) + ) + + def process_form(self, request: HttpRequest, finding: Finding, context: dict): + if context["form"].is_valid(): + # Create the note object + new_note = context["form"].save(commit=False) + new_note.author = request.user + new_note.date = timezone.now() + new_note.save() + # Add an entry to the note history + history = NoteHistory( + data=new_note.entry, time=new_note.date, current_editor=new_note.author, + ) + history.save() + new_note.history.add(history) + # Associate the note with the finding + finding.notes.add(new_note) + finding.last_reviewed = new_note.date + finding.last_reviewed_by = context["user"] + finding.save() + # Determine if the note should be sent to jira + if finding.has_jira_issue: + jira_services.add_comment(finding, new_note) + elif finding.has_jira_group_issue: + jira_services.add_comment(finding.finding_group, new_note) + # Send the notification of the note being added + url = request.build_absolute_uri( + reverse("view_finding", args=(finding.id,)), + ) + title = f"Finding: {finding.title}" + process_tag_notifications(request, new_note, url, title) + # Add a message to the request + messages.add_message( + request, messages.SUCCESS, "Note saved.", extra_tags="alert-success", + ) + + return request, True + + return request, False + + def get_initial_context(self, request: HttpRequest, finding: Finding, user: Dojo_User): + notes = finding.notes.all() + note_type_activation = Note_Type.objects.filter(is_active=True).count() + available_note_types = None + if note_type_activation: + available_note_types = find_available_notetypes(notes) + # Set the current context + context = { + "finding": finding, + "dojo_user": user, + "user": request.user, + "notes": notes, + "files": finding.files.all(), + "note_type_activation": note_type_activation, + "available_note_types": available_note_types, + "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), + "product_tab": Product_Tab( + finding.test.engagement.product, title="View Finding", tab="findings", + ), + } + # Set the form using the context, and then update the context + form = self.get_form(request, context) + context["form"] = form + + return context + + def get_template(self): + return "dojo/view_finding.html" + + def get(self, request: HttpRequest, finding_id: int): + # Get the initial objects + finding = self.get_finding(finding_id) + user = self.get_dojo_user(request) + # Make sure the user is authorized + user_has_permission_or_403(user, finding, "view") + # Set up the initial context + context = self.get_initial_context(request, finding, user) + # Add in the other extras + context |= self.get_previous_and_next_findings(finding) + # Add in more of the other extras + context |= self.get_request_response(finding) + context |= self.get_similar_findings(request, finding) + context |= self.get_test_import_data(request, finding) + context |= self.get_jira_data(finding) + # Render the form + return render(request, self.get_template(), context) + + def post(self, request: HttpRequest, finding_id): + # Get the initial objects + finding = self.get_finding(finding_id) + user = self.get_dojo_user(request) + # Make sure the user is authorized + user_has_permission_or_403(user, finding, "view") + # Quick perms check to determine if the user has access to add a note to the finding + user_has_permission_or_403(user, finding, "add") + # Set up the initial context + context = self.get_initial_context(request, finding, user) + # Determine the validity of the form + request, success = self.process_form(request, finding, context) + # Handle the case of a successful form + if success: + return HttpResponseRedirect(reverse("view_finding", args=(finding_id,))) + # Add in more of the other extras + context |= self.get_request_response(finding) + context |= self.get_similar_findings(request, finding) + context |= self.get_test_import_data(request, finding) + context |= self.get_jira_data(finding) + # Render the form + return render(request, self.get_template(), context) + + +class EditFinding(View): + def get_finding(self, finding_id: int): + return get_object_or_404(Finding, id=finding_id) + + def get_request_response(self, finding: Finding): + req_resp = None + if burp_rr := BurpRawRequestResponse.objects.filter(finding=finding).first(): + req_resp = (burp_rr.get_request(), burp_rr.get_response()) + + return req_resp + + def get_finding_form(self, request: HttpRequest, finding: Finding): + # Get the burp request if available + req_resp = self.get_request_response(finding) + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = { + "instance": finding, + "req_resp": req_resp, + "can_edit_mitigated_data": finding_helper.can_edit_mitigated_data(request.user), + "initial": {"vulnerability_ids": "\n".join(finding.vulnerability_ids)}, + } + + return FindingForm(*args, **kwargs) + + def get_jira_form(self, request: HttpRequest, finding: Finding, finding_form: FindingForm = None): + # Determine if jira should be used + if (jira_project := jira_services.get_project(finding)) is not None: + # Determine if push all findings is enabled + push_all_findings = jira_services.is_push_all_issues(finding) + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = { + "push_all": push_all_findings, + "prefix": "jiraform", + "instance": finding, + "jira_project": jira_project, + "finding_form": finding_form, + } + + return JIRAFindingForm(*args, **kwargs) + return None + + def get_github_form(self, request: HttpRequest, finding: Finding): + # Determine if github should be used + if get_system_setting("enable_github"): + # Ensure there is a github conf correctly configured for the product + config_present = GITHUB_PKey.objects.filter(product=finding.test.engagement.product) + if config_present := config_present.exclude(git_conf_id=None): + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = { + "enabled": finding.has_github_issue(), + "prefix": "githubform", + } + + return GITHUBFindingForm(*args, **kwargs) + return None + + def get_initial_context(self, request: HttpRequest, finding: Finding): + # Get the finding form first since it is used in another place + finding_form = self.get_finding_form(request, finding) + return { + "form": finding_form, + "finding": finding, + "jform": self.get_jira_form(request, finding, finding_form=finding_form), + "gform": self.get_github_form(request, finding), + "return_url": get_return_url(request), + "product_tab": Product_Tab( + finding.test.engagement.product, title="Edit Finding", tab="findings", + ), + } + + def validate_status_change(self, request: HttpRequest, finding: Finding, context: dict): + # If the finding is already not active, skip this extra validation + if not finding.active: + return request + # Validate the proper notes are added for mitigation + if (not context["form"]["active"].value() or context["form"]["false_p"].value() or context["form"]["out_of_scope"].value()) and not context["form"]["duplicate"].value(): + note_type_activation = Note_Type.objects.filter(is_active=True).count() + closing_disabled = 0 + if note_type_activation: + closing_disabled = len(get_missing_mandatory_notetypes(finding)) + if closing_disabled != 0: + error_inactive = ValidationError( + "Can not set a finding as inactive without adding all mandatory notes", + code="inactive_without_mandatory_notes", + ) + error_false_p = ValidationError( + "Can not set a finding as false positive without adding all mandatory notes", + code="false_p_without_mandatory_notes", + ) + error_out_of_scope = ValidationError( + "Can not set a finding as out of scope without adding all mandatory notes", + code="out_of_scope_without_mandatory_notes", + ) + if context["form"]["active"].value() is False: + context["form"].add_error("active", error_inactive) + if context["form"]["false_p"].value(): + context["form"].add_error("false_p", error_false_p) + if context["form"]["out_of_scope"].value(): + context["form"].add_error("out_of_scope", error_out_of_scope) + messages.add_message( + request, + messages.ERROR, + ("Can not set a finding as inactive, " + "false positive or out of scope without adding all mandatory notes"), + extra_tags="alert-danger", + ) + + return request + + def process_mitigated_data(self, request: HttpRequest, finding: Finding, context: dict): + # If active is not checked and CAN_EDIT_MITIGATED_DATA, + # mitigate the finding and the associated endpoints status + if finding_helper.can_edit_mitigated_data(request.user) and (( + context["form"]["active"].value() is False + or context["form"]["false_p"].value() + or context["form"]["out_of_scope"].value() + ) and context["form"]["duplicate"].value() is False): + now = timezone.now() + finding.is_mitigated = True + + if settings.V3_FEATURE_LOCATIONS: + for ref in finding.locations.all(): + ref.set_status( + FindingLocationStatus.Mitigated, + context["form"].cleaned_data.get("mitigated_by") or request.user, + context["form"].cleaned_data.get("mitigated") or now, + ) + else: + # TODO: Delete this after the move to Locations + endpoint_status = finding.status_finding.all() + for status in endpoint_status: + status.mitigated_by = ( + context["form"].cleaned_data.get("mitigated_by") or request.user + ) + status.mitigated_time = ( + context["form"].cleaned_data.get("mitigated") or now + ) + status.mitigated = True + status.last_modified = timezone.now() + status.save() + + def process_false_positive_history(self, finding: Finding, *, old_false_p: bool = False): + if get_system_setting("false_positive_history", False): + # If the finding is being marked as a false positive we dont need to call the + # fp history function because it will be called by the save function. + # If finding was a false positive and is being reactivated: retroactively reactivates all equal findings. + # old_false_p must be captured before form.save(commit=False) mutates the finding in place. + if old_false_p and not finding.false_p and get_system_setting("retroactive_false_positive_history"): + logger.debug("FALSE_POSITIVE_HISTORY: Reactivating existing findings based on: %s", finding) + # QuerySet.update() bypasses Django signals, which is intentional here — it mirrors + # the previous save_no_options() calls that also disabled all post-save processing. + # match_finding_to_existing_findings returns a lazy QS with no .only() applied, + # so any field can be added here without needing a corresponding .only() change in deduplication.py#_fetch_fp_candidates_for_batch. + match_finding_to_existing_findings( + finding, product=finding.test.engagement.product, + ).filter(false_p=True).update( + false_p=False, + active=finding.active, + verified=finding.verified, + out_of_scope=finding.out_of_scope, + is_mitigated=finding.is_mitigated, + ) + + def process_burp_request_response(self, finding: Finding, context: dict): + if "request" in context["form"].cleaned_data or "response" in context["form"].cleaned_data: + try: + burp_rr, _ = BurpRawRequestResponse.objects.get_or_create(finding=finding) + except BurpRawRequestResponse.MultipleObjectsReturned: + burp_rr = BurpRawRequestResponse.objects.filter(finding=finding).first() + burp_rr.burpRequestBase64 = base64.b64encode( + context["form"].cleaned_data["request"].encode(), + ) + burp_rr.burpResponseBase64 = base64.b64encode( + context["form"].cleaned_data["response"].encode(), + ) + burp_rr.clean() + burp_rr.save() + + def process_finding_form(self, request: HttpRequest, finding: Finding, context: dict): + if context["form"].is_valid(): + # process some of the easy stuff first + # Capture false_p before form.save(commit=False) mutates the finding in place, + # so process_false_positive_history can detect a false-positive → active transition. + old_false_p = finding.false_p + new_finding = context["form"].save(commit=False) + new_finding.test = finding.test + new_finding.numerical_severity = Finding.get_numerical_severity(new_finding.severity) + new_finding.last_reviewed = timezone.now() + new_finding.last_reviewed_by = request.user + new_finding.tags = context["form"].cleaned_data["tags"] + # Handle group related things + if "group" in context["form"].cleaned_data: + finding_group = context["form"].cleaned_data["group"] + finding_helper.update_finding_group(new_finding, finding_group) + # Handle risk exception related things + if "risk_accepted" in context["form"].cleaned_data and context["form"]["risk_accepted"].value(): + if new_finding.test.engagement.product.enable_simple_risk_acceptance: + ra_helper.simple_risk_accept(request.user, new_finding, perform_save=False) + elif new_finding.risk_accepted: + ra_helper.risk_unaccept(request.user, new_finding, perform_save=False) + # Save and add new locations; replace=True so deselected endpoints are removed + associated_locations = finding_helper.add_locations(new_finding, context["form"], replace=True) + # Remove unrelated endpoints + if settings.V3_FEATURE_LOCATIONS: + for ref in new_finding.locations.all(): + if ref.location not in associated_locations: + ref.location.disassociate_from_finding(new_finding) + else: + # TODO: Delete this after the move to Locations + endpoint_status_list = Endpoint_Status.objects.filter(finding=new_finding) + for endpoint_status in endpoint_status_list: + if endpoint_status.endpoint not in new_finding.endpoints.all(): + endpoint_status.delete() + # Handle some of the other steps + self.process_mitigated_data(request, new_finding, context) + self.process_false_positive_history(new_finding, old_false_p=old_false_p) + self.process_burp_request_response(new_finding, context) + # Save the vulnerability IDs + finding_helper.save_vulnerability_ids(new_finding, context["form"].cleaned_data["vulnerability_ids"].split()) + # Add a success message + messages.add_message( + request, + messages.SUCCESS, + "Finding saved successfully.", + extra_tags="alert-success", + ) + + return finding, request, True + add_error_message_to_response("The form has errors, please correct them below.") + add_field_errors_to_response(context["form"]) + + return finding, request, False + + def process_jira_form(self, request: HttpRequest, finding: Finding, context: dict): + # Capture case if the jira not being enabled + if context["jform"] is None: + return request, True, False + + if context["jform"] and context["jform"].is_valid(): + jira_message = None + logger.debug("jform.jira_issue: %s", context["jform"].cleaned_data.get("jira_issue")) + logger.debug(JFORM_PUSH_TO_JIRA_MESSAGE, context["jform"].cleaned_data.get("push_to_jira")) + # can't use helper as when push_all_jira_issues is True, the checkbox gets disabled and is always false + push_to_jira_checkbox = context["jform"].cleaned_data.get("push_to_jira") + push_all_jira_issues = jira_services.is_push_all_issues(finding) + push_to_jira = push_all_jira_issues or push_to_jira_checkbox or jira_services.is_keep_in_sync(finding) + logger.debug("push_to_jira: %s", push_to_jira) + logger.debug("push_all_jira_issues: %s", push_all_jira_issues) + logger.debug("has_jira_group_issue: %s", finding.has_jira_group_issue) + # if the jira issue key was changed, update database + new_jira_issue_key = context["jform"].cleaned_data.get("jira_issue") + # we only support linking / changing if there is no group issue + if not finding.has_jira_group_issue: + if finding.has_jira_issue: + """ + everything in DD around JIRA integration is based on the internal id + of the issue in JIRA instead of on the public jira issue key. + I have no idea why, but it means we have to retrieve the issue from JIRA + to get the internal JIRA id. we can assume the issue exist, + which is already checked in the validation of the form + """ + if not new_jira_issue_key: + jira_services.unlink_finding(request, finding) + jira_message = "Link to JIRA issue removed successfully." + elif new_jira_issue_key != finding.jira_issue.jira_key: + jira_services.unlink_finding(request, finding) + jira_services.link_finding(request, finding, new_jira_issue_key) + jira_message = "Changed JIRA link successfully." + elif new_jira_issue_key: + jira_services.link_finding(request, finding, new_jira_issue_key) + jira_message = "Linked a JIRA issue successfully." + # any existing finding should be updated + # Determine if a message should be added + if jira_message: + messages.add_message( + request, messages.SUCCESS, jira_message, extra_tags="alert-success", + ) + + return request, True, push_to_jira + add_field_errors_to_response(context["jform"]) + + return request, False, False + + def process_github_form(self, request: HttpRequest, finding: Finding, context: dict, old_status: str): + if "githubform-push_to_github" not in request.POST: + return request, True + + if context["gform"].is_valid(): + if GITHUB_Issue.objects.filter(finding=finding).exists(): + update_external_issue(finding.id, old_status, "github") + else: + add_external_issue(finding.id, "github") + + return request, True + add_field_errors_to_response(context["gform"]) + + return request, False + + def process_forms(self, request: HttpRequest, finding: Finding, context: dict): + form_success_list = [] + # Set vars for the completed forms + old_status = finding.status() + old_finding = copy.copy(finding) + # Validate finding mitigation + request = self.validate_status_change(request, finding, context) + # Check the validity of the form overall + new_finding, request, success = self.process_finding_form(request, finding, context) + form_success_list.append(success) + request, success, push_to_jira = self.process_jira_form(request, new_finding, context) + form_success_list.append(success) + request, success = self.process_github_form(request, new_finding, context, old_status) + form_success_list.append(success) + # Determine if all forms were successful + all_forms_valid = all(form_success_list) + # Check the validity of all the forms + if all_forms_valid: + # if we're removing the "duplicate" in the edit finding screen + # do not relaunch deduplication, otherwise, it's never taken into account + if old_finding.duplicate and not new_finding.duplicate: + new_finding.duplicate_finding = None + new_finding.save(push_to_jira=push_to_jira, dedupe_option=False) + else: + new_finding.save(push_to_jira=push_to_jira) + # we only push the group after storing the finding to make sure + # the updated data of the finding is pushed as part of the group + if push_to_jira and finding.finding_group: + jira_services.push(finding.finding_group) + + return request, all_forms_valid + + def get_template(self): + return "dojo/edit_finding.html" + + def get(self, request: HttpRequest, finding_id: int): + # Get the initial objects + finding = self.get_finding(finding_id) + # Make sure the user is authorized + user_has_permission_or_403(request.user, finding, "edit") + # Set up the initial context + context = self.get_initial_context(request, finding) + # Render the form + return render(request, self.get_template(), context) + + def post(self, request: HttpRequest, finding_id: int): + # Get the initial objects + finding = self.get_finding(finding_id) + # Make sure the user is authorized + user_has_permission_or_403(request.user, finding, "edit") + # Set up the initial context + context = self.get_initial_context(request, finding) + # Process the form + request, success = self.process_forms(request, finding, context) + # Handle the case of a successful form + if success: + return redirect_to_return_url_or_else(request, reverse("view_finding", args=(finding_id,))) + # Render the form + return render(request, self.get_template(), context) + + +class DeleteFinding(View): + def get_finding(self, finding_id: int): + return get_object_or_404(Finding, id=finding_id) + + def process_form(self, request: HttpRequest, finding: Finding, context: dict): + if context["form"].is_valid(): + product = finding.test.engagement.product + finding.delete() + # Update the grade of the product async + dojo_dispatch_task(calculate_grade, product.id) + # Add a message to the request that the finding was successfully deleted + messages.add_message( + request, + messages.SUCCESS, + "Finding deleted successfully.", + extra_tags="alert-success", + ) + + # Note: this notification has not be moved to "@receiver(post_delete, sender=Finding)" method as many other notifications + # Because it could generate too much noise, we keep it here only for findings created by hand in WebUI + # TODO: but same should be implemented for API endpoint + + # Send a notification that the finding had been deleted + create_notification( + event="finding_deleted", + title=f"Deletion of {finding.title}", + description=f'The finding "{finding.title}" was deleted by {request.user}', + product=product, + url=request.build_absolute_uri(reverse("all_findings")), + recipients=[finding.test.engagement.lead], + icon="exclamation-triangle", + ) + # return the request + return request, True + + # Add a failure message + messages.add_message( + request, + messages.ERROR, + "Unable to delete finding, please try again.", + extra_tags="alert-danger", + ) + + return request, False + + def post(self, request: HttpRequest, finding_id): + # Get the initial objects + finding = self.get_finding(finding_id) + # Make sure the user is authorized + user_has_permission_or_403(request.user, finding, "delete") + # Get the finding form + context = { + "form": DeleteFindingForm(request.POST, instance=finding), + } + # Process the form + request, success = self.process_form(request, finding, context) + # Handle the case of a successful form + if success: + return redirect_to_return_url_or_else(request, reverse("view_test", args=(finding.test.id,))) + raise PermissionDenied + + +def close_finding(request, fid): + finding = get_object_or_404(Finding, id=fid) + # in order to close a finding, we need to capture why it was closed + # we can do this with a Note + note_type_activation = Note_Type.objects.filter(is_active=True) + missing_note_types = get_missing_mandatory_notetypes(finding) if len(note_type_activation) else note_type_activation + form = CloseFindingForm( + instance=finding, + missing_note_types=missing_note_types, + can_edit_mitigated_data=finding_helper.can_edit_mitigated_data(request.user), + ) + if request.method == "POST": + form = CloseFindingForm( + request.POST, + instance=finding, + missing_note_types=missing_note_types, + can_edit_mitigated_data=finding_helper.can_edit_mitigated_data(request.user), + ) + + if form.is_valid(): + messages.add_message(request, messages.SUCCESS, "Note Saved.", extra_tags="alert-success") + + if len(missing_note_types) <= 1: + finding_helper.close_finding( + finding=finding, + user=request.user, + is_mitigated=True, + mitigated=form.cleaned_data.get("mitigated"), + mitigated_by=form.cleaned_data.get("mitigated_by") or request.user, + false_p=form.cleaned_data.get("false_p", False), + out_of_scope=form.cleaned_data.get("out_of_scope", False), + duplicate=form.cleaned_data.get("duplicate", False), + note_entry=form.cleaned_data.get("entry"), + note_type=form.cleaned_data.get("note_type"), + ) + + messages.add_message( + request, + messages.SUCCESS, + "Finding closed.", + extra_tags="alert-success", + ) + + # Notification sent by helper + return HttpResponseRedirect( + reverse("view_test", args=(finding.test.id,)), + ) + return HttpResponseRedirect( + reverse("close_finding", args=(finding.id,)), + ) + + product_tab = Product_Tab( + finding.test.engagement.product, title="Close", tab="findings", + ) + + return render( + request, + "dojo/close_finding.html", + { + "finding": finding, + "product_tab": product_tab, + "active_tab": "findings", + "user": request.user, + "form": form, + "note_types": missing_note_types, + }, + ) + + +def verify_finding(request, fid): + finding = get_object_or_404(Finding, id=fid) + + if finding.verified: + messages.add_message( + request, + messages.INFO, + "Finding already verified.", + extra_tags="alert-info", + ) + return redirect_to_return_url_or_else( + request, + reverse("view_finding", args=(finding.id,)), + ) + + form = NoteForm(data=request.POST or None) + form.fields["entry"].required = False + form.fields["entry"].label = _("Comment (optional)") + + if request.method == "POST" and form.is_valid(): + entry = form.cleaned_data.get("entry", "") + finding_helper.verify_finding( + finding=finding, + user=request.user, + note_entry=entry, + ) + + messages.add_message( + request, + messages.SUCCESS, + "Finding verified.", + extra_tags="alert-success", + ) + + return redirect_to_return_url_or_else( + request, + reverse("view_finding", args=(finding.id,)), + ) + + product_tab = Product_Tab( + finding.test.engagement.product, + title="Verify Finding", + tab="findings", + ) + + return render( + request, + "dojo/verify_finding.html", + { + "finding": finding, + "product_tab": product_tab, + "user": request.user, + "form": form, + "active_tab": "findings", + }, + ) + + +def defect_finding_review(request, fid): + finding = get_object_or_404(Finding, id=fid) + # in order to close a finding, we need to capture why it was closed + # we can do this with a Note + if request.method == "POST": + form = DefectFindingForm(request.POST) + if form.is_valid(): + now = timezone.now() + new_note = form.save(commit=False) + new_note.author = request.user + new_note.date = now + new_note.save() + finding.notes.add(new_note) + finding.under_review = False + defect_choice = form.cleaned_data["defect_choice"] + + if defect_choice == "Close Finding": + finding.active = False + finding.verified = True + finding.mitigated = now + finding.mitigated_by = request.user + finding.is_mitigated = True + finding.last_reviewed = finding.mitigated + finding.last_reviewed_by = request.user + finding.endpoints.clear() + else: + finding.active = True + finding.verified = True + finding.mitigated = None + finding.mitigated_by = None + finding.is_mitigated = False + finding.last_reviewed = now + finding.last_reviewed_by = request.user + + # Manage the jira status changes + push_to_jira = False + # Determine if the finding is in a group. if so, not push to jira + finding_in_group = finding.has_finding_group + # Check if there is a jira issue that needs to be updated + jira_issue_exists = finding.has_jira_issue or (finding.finding_group and finding.finding_group.has_jira_issue) + # Only push if the finding is not in a group + if jira_issue_exists: + # Determine if any automatic sync should occur + jira_instance = jira_services.get_instance(finding) + push_to_jira = jira_services.is_push_all_issues(finding) \ + or (jira_instance and jira_instance.finding_jira_sync) + # Add the closing note + if push_to_jira and not finding_in_group: + if defect_choice == "Close Finding": + new_note.entry += "\nJira issue set to resolved." + else: + new_note.entry += "\nJira issue re-opened." + jira_services.add_comment(finding, new_note, force_push=True) + # Save the finding + finding.save(push_to_jira=(push_to_jira and not finding_in_group)) + + # we only push the group after saving the finding to make sure + # the updated data of the finding is pushed as part of the group + if push_to_jira and finding_in_group: + jira_services.push(finding.finding_group) + + messages.add_message( + request, messages.SUCCESS, "Defect Reviewed", extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_test", args=(finding.test.id,))) + + else: + form = DefectFindingForm() + + product_tab = Product_Tab( + finding.test.engagement.product, title="Jira Status Review", tab="findings", + ) + + return render( + request, + "dojo/defect_finding_review.html", + { + "finding": finding, + "product_tab": product_tab, + "user": request.user, + "form": form, + }, + ) + + +def reopen_finding(request, fid): + finding = get_object_or_404(Finding, id=fid) + finding.active = True + finding.mitigated = None + finding.mitigated_by = request.user + finding.is_mitigated = False + finding.last_reviewed = finding.mitigated + finding.last_reviewed_by = request.user + finding.under_review = False + if settings.V3_FEATURE_LOCATIONS: + for ref in finding.locations.all(): + ref.set_status(FindingLocationStatus.Active, request.user, timezone.now()) + else: + endpoint_status = finding.status_finding.all() + for status in endpoint_status: + status.mitigated_by = None + status.mitigated_time = None + status.mitigated = False + status.last_modified = timezone.now() + status.save() + # Clear the risk acceptance, if present + ra_helper.risk_unaccept(request.user, finding) + finding.save(dedupe_option=False, push_to_jira=False) + if jira_services.is_push_all_issues(finding) or jira_services.is_keep_in_sync(finding): + jira_services.push(finding) + + reopen_external_issue(finding.id, "re-opened by defectdojo", "github") + + messages.add_message( + request, messages.SUCCESS, "Finding Reopened.", extra_tags="alert-success", + ) + + # Note: this notification has not be moved to "@receiver(pre_save, sender=Finding)" method as many other notifications + # Because it could generate too much noise, we keep it here only for findings created by hand in WebUI + # TODO: but same should be implemented for API endpoint + + create_notification( + event="finding_reopened", + title=_("Reopening of %s") % finding.title, + finding=finding, + description=f'The finding "{finding.title}" was reopened by {request.user}', + url=reverse("view_finding", args=(finding.id,)), + ) + return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) + + +def copy_finding(request, fid): + finding = get_object_or_404(Finding, id=fid) + product = finding.test.engagement.product + tests = get_authorized_tests("edit").filter( + engagement=finding.test.engagement, + ) + form = CopyFindingForm(tests=tests) + + if request.method == "POST": + form = CopyFindingForm(request.POST, tests=tests) + if form.is_valid(): + test = form.cleaned_data.get("test") + product = finding.test.engagement.product + finding_copy = finding.copy(test=test) + dojo_dispatch_task(calculate_grade, product.id) + messages.add_message( + request, + messages.SUCCESS, + "Finding Copied successfully.", + extra_tags="alert-success", + ) + create_notification( + event="finding_copied", # TODO: - if 'copy' functionality will be supported by API as well, 'create_notification' needs to be migrated to place where it will be able to cover actions from both interfaces + title=_("Copying of %s") % finding.title, + description=f'The finding "{finding.title}" was copied by {request.user} to {test.title}', + product=product, + url=request.build_absolute_uri( + reverse("copy_finding", args=(finding_copy.id,)), + ), + recipients=[finding.test.engagement.lead], + icon="exclamation-triangle", + ) + return redirect_to_return_url_or_else( + request, reverse("view_test", args=(test.id,)), + ) + messages.add_message( + request, + messages.ERROR, + "Unable to copy finding, please try again.", + extra_tags="alert-danger", + ) + + product_tab = Product_Tab(product, title="Copy Finding", tab="findings") + return render( + request, + "dojo/copy_object.html", + { + "source": finding, + "source_label": "Finding", + "destination_label": "Test", + "product_tab": product_tab, + "form": form, + }, + ) + + +def remediation_date(request, fid): + finding = get_object_or_404(Finding, id=fid) + user = get_object_or_404(Dojo_User, id=request.user.id) + + if request.method == "POST": + form = EditPlannedRemediationDateFindingForm(request.POST) + + if form.is_valid(): + finding.planned_remediation_date = request.POST.get( + "planned_remediation_date", "", + ) + finding.save() + messages.add_message( + request, + messages.SUCCESS, + "Finding Planned Remediation Date saved.", + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) + + else: + form = EditPlannedRemediationDateFindingForm(finding=finding) + + product_tab = Product_Tab( + finding.test.engagement.product, + title="Planned Remediation Date", + tab="findings", + ) + + return render( + request, + "dojo/remediation_date.html", + {"finding": finding, "product_tab": product_tab, "user": user, "form": form}, + ) + + +def touch_finding(request, fid): + finding = get_object_or_404(Finding, id=fid) + finding.last_reviewed = timezone.now() + finding.last_reviewed_by = request.user + finding.save() + return redirect_to_return_url_or_else( + request, reverse("view_finding", args=(finding.id,)), + ) + + +def simple_risk_accept(request, fid): + finding = get_object_or_404(Finding, id=fid) + + if not finding.test.engagement.product.enable_simple_risk_acceptance: + raise PermissionDenied + + ra_helper.simple_risk_accept(request.user, finding) + + messages.add_message( + request, messages.WARNING, "Finding risk accepted.", extra_tags="alert-success", + ) + + return redirect_to_return_url_or_else( + request, reverse("view_finding", args=(finding.id,)), + ) + + +def risk_unaccept(request, fid): + finding = get_object_or_404(Finding, id=fid) + ra_helper.risk_unaccept(request.user, finding) + + messages.add_message( + request, + messages.WARNING, + "Finding risk unaccepted.", + extra_tags="alert-success", + ) + + return redirect_to_return_url_or_else( + request, reverse("view_finding", args=(finding.id,)), + ) + + +def request_finding_review(request, fid): + finding = get_object_or_404(Finding, id=fid) + user = get_object_or_404(Dojo_User, id=request.user.id) + form = ReviewFindingForm(finding=finding, user=user) + # in order to review a finding, we need to capture why a review is needed + # we can do this with a Note + if request.method == "POST": + form = ReviewFindingForm(request.POST, finding=finding, user=user) + + if form.is_valid(): + now = timezone.now() + new_note = Notes() + new_note.entry = "Review Request: " + form.cleaned_data["entry"] + new_note.private = True + new_note.author = request.user + new_note.date = now + new_note.save() + finding.notes.add(new_note) + finding.active = True + finding.verified = False + finding.is_mitigated = False + finding.under_review = True + finding.review_requested_by = user + finding.last_reviewed = now + finding.last_reviewed_by = request.user + + reviewers = form.cleaned_data["reviewers"] + finding.reviewers.set(reviewers) + + # Manage the jira status changes + push_to_jira = False + # Determine if the finding is in a group. if so, not push to jira + finding_in_group = finding.has_finding_group + # Check if there is a jira issue that needs to be updated + jira_issue_exists = finding.has_jira_issue or (finding.finding_group and finding.finding_group.has_jira_issue) + # Only push if the finding is not in a group + if jira_issue_exists: + # Determine if any automatic sync should occur + jira_instance = jira_services.get_instance(finding) + push_to_jira = jira_services.is_push_all_issues(finding) \ + or (jira_instance and jira_instance.finding_jira_sync) + # Add the closing note + if push_to_jira and not finding_in_group: + jira_services.add_comment(finding, new_note, force_push=True) + # Save the finding + finding.save(push_to_jira=(push_to_jira and not finding_in_group)) + + # we only push the group after saving the finding to make sure + # the updated data of the finding is pushed as part of the group + if push_to_jira and finding_in_group: + jira_services.push(finding.finding_group) + + reviewers = Dojo_User.objects.filter(id__in=form.cleaned_data["reviewers"]) + reviewers_string = ", ".join([f"{user} ({user.id})" for user in reviewers]) + reviewers_usernames = [user.username for user in reviewers] + logger.debug("Asking %s for review", reviewers_string) + + create_notification( + event="review_requested", # TODO: - if 'review_requested' functionality will be supported by API as well, 'create_notification' needs to be migrated to place where it will be able to cover actions from both interfaces + title=f"Finding review requested for Test created for {finding.test.engagement.product}: {finding.test.engagement.name}: {finding.test} - {finding.title}", + requested_by=user, + note=new_note, + finding=finding, + reviewers=reviewers, + recipients=reviewers_usernames, + description=f'User {user.get_full_name()}({user.id}) has requested that user(s) {reviewers_string} review the finding "{finding.title}" for accuracy:\n\n{new_note}', + icon="check", + url=reverse("view_finding", args=(finding.id,)), + ) + + messages.add_message( + request, + messages.SUCCESS, + "Finding marked for review and reviewers notified.", + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) + + product_tab = Product_Tab( + finding.test.engagement.product, title="Review Finding", tab="findings", + ) + + return render( + request, + "dojo/review_finding.html", + {"finding": finding, "product_tab": product_tab, "user": user, "form": form, "enable_table_filtering": get_system_setting("enable_ui_table_based_searching")}, + ) + + +def clear_finding_review(request, fid): + finding = get_object_or_404(Finding, id=fid) + user = get_object_or_404(Dojo_User, id=request.user.id) + # If the user wanting to clear the review is not the user who requested + # the review or one of the users requested to provide the review, then + # do not allow the user to clear the review. + if user != finding.review_requested_by and user not in finding.reviewers.all(): + raise PermissionDenied + + # in order to clear a review for a finding, we need to capture why and how it was reviewed + # we can do this with a Note + if request.method == "POST": + form = ClearFindingReviewForm(request.POST, instance=finding) + + if form.is_valid(): + now = timezone.now() + new_note = Notes() + new_note.entry = "Review Cleared: " + form.cleaned_data["entry"] + new_note.author = request.user + new_note.date = now + new_note.save() + + finding = form.save(commit=False) + + if finding.is_mitigated: + finding.mitigated = now + finding.mitigated_by = request.user + finding.under_review = False + finding.last_reviewed = now + finding.last_reviewed_by = request.user + + finding.reviewers.set([]) + finding.notes.add(new_note) + + # Manage the jira status changes + push_to_jira = False + # Determine if the finding is in a group. if so, not push to jira + finding_in_group = finding.has_finding_group + # Check if there is a jira issue that needs to be updated + jira_issue_exists = finding.has_jira_issue or (finding.finding_group and finding.finding_group.has_jira_issue) + # Only push if the finding is not in a group + if jira_issue_exists: + # Determine if any automatic sync should occur + jira_instance = jira_services.get_instance(finding) + push_to_jira = jira_services.is_push_all_issues(finding) \ + or (jira_instance and jira_instance.finding_jira_sync) + # Add the closing note + if push_to_jira and not finding_in_group: + jira_services.add_comment(finding, new_note, force_push=True) + # Save the finding + finding.save(push_to_jira=(push_to_jira and not finding_in_group)) + + # we only push the group after saving the finding to make sure + # the updated data of the finding is pushed as part of the group + if push_to_jira and finding_in_group: + jira_services.push(finding.finding_group) + + messages.add_message( + request, + messages.SUCCESS, + "Finding review has been updated successfully.", + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) + + else: + form = ClearFindingReviewForm(instance=finding) + + product_tab = Product_Tab( + finding.test.engagement.product, title="Clear Finding Review", tab="findings", + ) + + return render( + request, + "dojo/clear_finding_review.html", + {"finding": finding, "product_tab": product_tab, "user": user, "form": form}, + ) + + +def mktemplate(request, fid): + user_has_global_permission_or_403(request.user, "add") + finding = get_object_or_404(Finding, id=fid) + templates = Finding_Template.objects.filter(title=finding.title) + if len(templates) > 0: + messages.add_message( + request, + messages.ERROR, + "A finding template with that title already exists.", + extra_tags="alert-danger", + ) + else: + template = Finding_Template( + title=finding.title, + cwe=finding.cwe, + cvssv3=finding.cvssv3, + cvssv3_score=finding.cvssv3_score, + cvssv4=finding.cvssv4, + cvssv4_score=finding.cvssv4_score, + severity=finding.severity, + description=finding.description, + mitigation=finding.mitigation, + impact=finding.impact, + references=finding.references, + numerical_severity=finding.numerical_severity, + fix_available=finding.fix_available, + fix_version=finding.fix_version, + planned_remediation_version=finding.planned_remediation_version, + effort_for_fixing=finding.effort_for_fixing, + steps_to_reproduce=finding.steps_to_reproduce, + severity_justification=finding.severity_justification, + component_name=finding.component_name, + component_version=finding.component_version, + tags=finding.tags.all(), + ) + template.save() + template.tags = finding.tags.all() + # Ensure template tags exist in Finding's tag model + # (They should already exist since they come from a finding, but ensure for consistency) + ensure_template_tags_in_finding_model(template) + + # Save vulnerability IDs using helper (handles both old and new format) + finding_helper.save_vulnerability_ids_template(template, finding.vulnerability_ids) + + # Copy endpoints if they exist + if finding.endpoints.exists(): + endpoint_urls = [str(ep) for ep in finding.endpoints.all()] + finding_helper.save_endpoints_template(template, endpoint_urls) + + messages.add_message( + request, + messages.SUCCESS, + mark_safe( + 'Finding template added successfully. You may edit it here.'.format(reverse("edit_template", args=(template.id,))), + ), + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) + + +def find_template_to_apply(request, fid): + # Templates may contain sensitive data from any product; require global permission + # to match the authorization level of the /template list view + user_has_global_permission_or_403(request.user, "edit") + finding = get_object_or_404(Finding, id=fid) + test = get_object_or_404(Test, id=finding.test.id) + templates_by_cve = ( + Finding_Template.objects.annotate( + cve_len=Length("cve"), order=models.Value(1, models.IntegerField()), + ) + .filter(cve=finding.cve, cve_len__gt=0) + .order_by("-last_used") + ) + if templates_by_cve.count() == 0: + templates_by_last_used = ( + Finding_Template.objects.all() + .order_by("-last_used") + .annotate( + cve_len=Length("cve"), order=models.Value(2, models.IntegerField()), + ) + ) + templates = templates_by_last_used + else: + templates_by_last_used = ( + Finding_Template.objects.all() + .exclude(cve=finding.cve) + .order_by("-last_used") + .annotate( + cve_len=Length("cve"), order=models.Value(2, models.IntegerField()), + ) + ) + union_queryset = templates_by_last_used.union(templates_by_cve).order_by( + "order", "-last_used", + ) + # Convert union queryset to regular queryset to avoid issues with distinct() in filters + # Get IDs from union queryset and create a new queryset filtered by those IDs + template_ids = list(union_queryset.values_list("id", flat=True)) + templates = Finding_Template.objects.filter(id__in=template_ids).annotate( + cve_len=Length("cve"), + order=Case( + *[When(id=template_id, then=models.Value(i + 1)) for i, template_id in enumerate(template_ids)], + default=models.Value(len(template_ids) + 1), + output_field=models.IntegerField(), + ), + ).order_by("order", "-last_used") + + templates = TemplateFindingFilter(request.GET, queryset=templates) + paged_templates = get_page_items(request, templates.qs, 25) + + # just query all templates as this weird ordering above otherwise breaks Django ORM + title_words = get_words_for_field(Finding_Template, "title") + product_tab = Product_Tab( + test.engagement.product, title="Apply Template to Finding", tab="findings", + ) + return render( + request, + "dojo/templates.html", + { + "templates": paged_templates, + "product_tab": product_tab, + "filtered": templates, + "title_words": title_words, + "tid": test.id, + "fid": fid, + "add_from_template": False, + "apply_template": True, + }, + ) + + +def choose_finding_template_options(request, tid, fid): + finding = get_object_or_404(Finding, id=fid) + user_has_permission_or_403(request.user, finding, "edit") + template = get_object_or_404(Finding_Template, id=tid) + data = finding.__dict__.copy() + # Remove tags and other non-serializable fields + data.pop("tags", None) + data.pop("_state", None) + data.pop("_tags_tagulous", None) + + # Populate from template for fields that exist on template + template_fields = ["cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", + "fix_available", "fix_version", "planned_remediation_version", + "effort_for_fixing", "steps_to_reproduce", "severity_justification", + "component_name", "component_version", "notes"] + for field in template_fields: + if hasattr(template, field): + value = getattr(template, field) + if value is not None: + data[field] = value + + # Handle vulnerability_ids and endpoints (convert lists to strings) + data["vulnerability_ids"] = "\n".join(finding.vulnerability_ids) + if hasattr(template, "endpoints") and template.endpoints: + endpoints_value = template.endpoints + if isinstance(endpoints_value, list): + data["endpoints"] = "\n".join(endpoints_value) + else: + data["endpoints"] = endpoints_value + + template_tag_names = [tag.name for tag in template.tags.all()] + # Add tags as comma-separated string for TagField + if template_tag_names: + data["tags"] = ", ".join(template_tag_names) + + form = ApplyFindingTemplateForm(data=data, template=template) + # Combine tags from both Finding_Template and Finding tag models + # This ensures we don't lose tags that exist on templates but may have been removed from findings + if "tags" in form.fields: + finding_tag_model = Finding.tags.tag_model + template_tag_model = Finding_Template.tags.tag_model + + # Get all tags from Finding_Template model + template_tags = set(template_tag_model.objects.values_list("name", flat=True)) + # Get all tags from Finding model + finding_tags = set(finding_tag_model.objects.values_list("name", flat=True)) + # Combine both sets to get all unique tag names + all_tag_names = template_tags | finding_tags + + # Ensure all tags from both models exist in Finding's tag model (where they'll be applied) + # Strictly speaking, creating tags here isn't necessary since TagField can create them on save, + # but it's the safest option to avoid tags getting lost or not getting rendered properly. + # This prevents tagulous from removing tags that only exist on templates and ensures + # TagField can display them correctly during form rendering. + # Store tag objects in a dict for reuse + tag_objects = {} + for tag_name in all_tag_names: + tag, _ = finding_tag_model.objects.get_or_create( + name=tag_name, + defaults={"name": tag_name, "protected": False}, + ) + tag_objects[tag_name] = tag + + # Update autocomplete_tags to include tags from both models + form.fields["tags"].autocomplete_tags = finding_tag_model.objects.all().order_by("name") + + # Set initial value using template tags (already created above) + if template_tag_names: + template_finding_tags = [tag_objects[tag_name] for tag_name in template_tag_names] + form.fields["tags"].initial = template_finding_tags + product_tab = Product_Tab( + finding.test.engagement.product, + title="Finding Template Options", + tab="findings", + ) + return render( + request, + "dojo/apply_finding_template.html", + { + "finding": finding, + "product_tab": product_tab, + "template": template, + "form": form, + "finding_tags": [tag.name for tag in finding.tags.all()], + }, + ) + + +def apply_template_to_finding(request, fid, tid): + finding = get_object_or_404(Finding, id=fid) + user_has_permission_or_403(request.user, finding, "edit") + template = get_object_or_404(Finding_Template, id=tid) + + if request.method == "POST": + form = ApplyFindingTemplateForm(data=request.POST) + + if form.is_valid(): + template.last_used = timezone.now() + template.save() + + # Apply basic fields (existing) + finding.title = form.cleaned_data["title"] + finding.cwe = form.cleaned_data["cwe"] + finding.severity = form.cleaned_data["severity"] + finding.description = form.cleaned_data["description"] + finding.mitigation = form.cleaned_data["mitigation"] + finding.impact = form.cleaned_data["impact"] + finding.references = form.cleaned_data["references"] + finding.tags = form.cleaned_data["tags"] + + # Copy template fields (using centralized helper) + finding_helper.copy_template_fields_to_finding( + finding=finding, + template=template, + form_data=form.cleaned_data, + user=request.user, + copy_vulnerability_ids=True, + copy_endpoints=True, + copy_notes=True, + ) + + # Update review fields + finding.last_reviewed = timezone.now() + finding.last_reviewed_by = request.user + + # Save finding (this will trigger CVSS score computation if vectors are set) + finding.save() + else: + messages.add_message( + request, + messages.ERROR, + "There appears to be errors on the form, please correct below.", + extra_tags="alert-danger", + ) + product_tab = Product_Tab( + finding.test.engagement.product, + title="Apply Finding Template", + tab="findings", + ) + return render( + request, + "dojo/apply_finding_template.html", + { + "finding": finding, + "product_tab": product_tab, + "template": template, + "form": form, + }, + ) + + return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) + return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) + + +def templates(request): + templates = Finding_Template.objects.all().order_by("cwe") + templates = TemplateFindingFilter(request.GET, queryset=templates) + paged_templates = get_page_items(request, templates.qs, 25) + + title_words = get_words_for_field(templates.qs, "title") + + add_breadcrumb(title="Template Listing", top_level=True, request=request) + return render( + request, + "dojo/templates.html", + { + "templates": paged_templates, + "filtered": templates, + "title_words": title_words, + }, + ) + + +def export_templates_to_json(request): + leads_as_json = serializers.serialize("json", Finding_Template.objects.all()) + return HttpResponse(leads_as_json, content_type="application/json") + + +def ensure_template_tags_in_finding_model(template): + """ + Ensure all tags on a Finding_Template also exist in Finding's tag model. + This prevents tags from being lost when tagulous cleans up unused tags and ensures + tags can be properly applied when templates are used. + """ + if not template or not template.pk: + return + + finding_tag_model = Finding.tags.tag_model + + # Get all tag names from the template + template_tag_names = [tag.name for tag in template.tags.all()] + + # Ensure each tag exists in Finding's tag model + for tag_name in template_tag_names: + finding_tag_model.objects.get_or_create( + name=tag_name, + defaults={"name": tag_name, "protected": False}, + ) + + +def apply_cwe_mitigation(apply_to_findings, template, *, update=True): + count = 0 + if apply_to_findings and template.template_match and template.cwe is not None: + # Update active, verified findings with the CWE template + # If CWE only match only update issues where there isn't a CWE + Title match + if template.template_match_title: + count = Finding.objects.filter( + active=True, + verified=True, + cwe=template.cwe, + title__icontains=template.title, + ).update( + mitigation=template.mitigation, + impact=template.impact, + references=template.references, + ) + else: + finding_templates = Finding_Template.objects.filter( + cwe=template.cwe, template_match=True, template_match_title=True, + ) + + finding_ids = None + result_list = None + # Exclusion list + for title_template in finding_templates: + finding_ids = Finding.objects.filter( + active=True, + verified=True, + cwe=title_template.cwe, + title__icontains=title_template.title, + ).values_list("id", flat=True) + result_list = finding_ids if result_list is None else list(chain(result_list, finding_ids)) + + # If result_list is None the filter exclude won't work + if result_list: + count = Finding.objects.filter( + active=True, verified=True, cwe=template.cwe, + ).exclude(id__in=result_list) + else: + count = Finding.objects.filter( + active=True, verified=True, cwe=template.cwe, + ) + + if update: + # MySQL won't allow an 'update in statement' so loop will have to do + for finding in count: + finding.mitigation = template.mitigation + finding.impact = template.impact + finding.references = template.references + template.last_used = timezone.now() + template.save() + new_note = Notes() + new_note.entry = ( + f"CWE remediation text applied to finding for CWE: {template.cwe} using template: {template.title}." + ) + new_note.author, _created = User.objects.get_or_create( + username="System", + ) + new_note.save() + finding.notes.add(new_note) + finding.save() + + count = count.count() + return count + + +def add_template(request): + form = FindingTemplateForm() + if request.method == "POST": + form = FindingTemplateForm(request.POST) + if form.is_valid(): + template = form.save(commit=False) + template.numerical_severity = Finding.get_numerical_severity( + template.severity, + ) + # Save vulnerability IDs using helper + finding_helper.save_vulnerability_ids_template( + template, form.cleaned_data["vulnerability_ids"].split(), + ) + # Save endpoints using helper + if form.cleaned_data.get("endpoints"): + endpoint_urls = [url.strip() for url in form.cleaned_data["endpoints"].split("\n") if url.strip()] + finding_helper.save_endpoints_template(template, endpoint_urls) + template.save() + form.save_m2m() + # Ensure template tags exist in Finding's tag model + ensure_template_tags_in_finding_model(template) + + messages.add_message( + request, + messages.SUCCESS, + "Template created successfully.", + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("templates")) + messages.add_message( + request, + messages.ERROR, + "Template form has error, please revise and try again.", + extra_tags="alert-danger", + ) + add_breadcrumb(title="Add Template", top_level=False, request=request) + return render( + request, "dojo/add_template.html", {"form": form, "name": "Add Template"}, + ) + + +def edit_template(request, tid): + template = get_object_or_404(Finding_Template, id=tid) + initial_data = {"vulnerability_ids": "\n".join(template.vulnerability_ids)} + # Add endpoints to initial data if they exist + if hasattr(template, "endpoints") and template.endpoints: + endpoints_value = template.endpoints + if isinstance(endpoints_value, list): + initial_data["endpoints"] = "\n".join(endpoints_value) + else: + initial_data["endpoints"] = endpoints_value + form = FindingTemplateForm( + instance=template, + initial=initial_data, + ) + + if request.method == "POST": + form = FindingTemplateForm(request.POST, instance=template) + if form.is_valid(): + template = form.save(commit=False) + template.numerical_severity = Finding.get_numerical_severity( + template.severity, + ) + # Save vulnerability IDs using helper + finding_helper.save_vulnerability_ids_template( + template, form.cleaned_data["vulnerability_ids"].split(), + ) + # Save endpoints using helper + if form.cleaned_data.get("endpoints"): + endpoint_urls = [url.strip() for url in form.cleaned_data["endpoints"].split("\n") if url.strip()] + finding_helper.save_endpoints_template(template, endpoint_urls) + template.save() + form.save_m2m() + # Ensure template tags exist in Finding's tag model + ensure_template_tags_in_finding_model(template) + + messages.add_message( + request, + messages.SUCCESS, + "Template updated successfully.", + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("templates")) + messages.add_message( + request, + messages.ERROR, + "Template form has error, please revise and try again.", + extra_tags="alert-danger", + ) + + add_breadcrumb(title="Edit Template", top_level=False, request=request) + return render( + request, + "dojo/add_template.html", + { + "form": form, + "name": "Edit Template", + "template": template, + }, + ) + + +def delete_template(request, tid): + template = get_object_or_404(Finding_Template, id=tid) + if request.method == "POST": + form = DeleteFindingTemplateForm(request.POST, instance=template) + if form.is_valid(): + template.delete() + messages.add_message( + request, + messages.SUCCESS, + "Finding Template deleted successfully.", + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("templates")) + messages.add_message( + request, + messages.ERROR, + "Unable to delete Template, please revise and try again.", + extra_tags="alert-danger", + ) + return None + raise PermissionDenied + + +def download_finding_pic(request, token): + class Thumbnail(ImageSpec): + processors = [ResizeToFill(100, 100)] + format = "JPEG" + options = {"quality": 70} + + class Small(ImageSpec): + processors = [ResizeToFill(640, 480)] + format = "JPEG" + options = {"quality": 100} + + class Medium(ImageSpec): + processors = [ResizeToFill(800, 600)] + format = "JPEG" + options = {"quality": 100} + + class Large(ImageSpec): + processors = [ResizeToFill(1024, 768)] + format = "JPEG" + options = {"quality": 100} + + class Original(ImageSpec): + format = "JPEG" + options = {"quality": 100} + + mimetypes.init() + + size_map = { + "thumbnail": Thumbnail, + "small": Small, + "medium": Medium, + "large": Large, + "original": Original, + } + + try: + access_token = FileAccessToken.objects.get(token=token) + if access_token.size not in list(size_map.keys()): + raise Http404 + size = access_token.size + access_token.delete() + except Exception: + raise PermissionDenied + + with Path(access_token.file.file.file.name).open("rb") as file: + file_name = file.name + image = size_map[size](source=file).generate() + response = StreamingHttpResponse(FileIterWrapper(image)) + response["Content-Disposition"] = "inline" + mimetype, _encoding = mimetypes.guess_type(file_name) + response["Content-Type"] = mimetype or "application/octet-stream" + return response + + +def merge_finding_product(request, pid): + product = get_object_or_404(Product, pk=pid) + finding_to_update = request.GET.getlist("finding_to_update") + findings = None + + if ( + request.GET.get("merge_findings") or request.method == "POST" + ) and finding_to_update: + finding = Finding.objects.get( + id=finding_to_update[0], test__engagement__product=product, + ) + findings = Finding.objects.filter( + id__in=finding_to_update, test__engagement__product=product, + ) + form = MergeFindings( + finding=finding, + findings=findings, + initial={"finding_to_merge_into": finding_to_update[0]}, + ) + + if request.method == "POST": + form = MergeFindings(request.POST, finding=finding, findings=findings) + if form.is_valid(): + finding_to_merge_into = form.cleaned_data["finding_to_merge_into"] + findings_to_merge = form.cleaned_data["findings_to_merge"] + finding_descriptions = "" + finding_references = "" + notes_entry = "" + static = False + dynamic = False + + if finding_to_merge_into not in findings_to_merge: + for finding in findings_to_merge.exclude( + pk=finding_to_merge_into.pk, + ): + notes_entry = f"{notes_entry}\n- {finding.title} ({finding.id})," + if finding.static_finding: + static = finding.static_finding + + if finding.dynamic_finding: + dynamic = finding.dynamic_finding + + if form.cleaned_data["append_description"]: + finding_descriptions = f"{finding_descriptions}\n{finding.description}" + # Workaround until file path is one to many + if finding.file_path: + finding_descriptions = f"{finding_descriptions}\n**File Path:** {finding.file_path}\n" + + # If checked merge the Reference + if ( + form.cleaned_data["append_reference"] + and finding.references is not None + ): + finding_references = f"{finding_references}\n{finding.references}" + + # if checked merge the endpoints + if form.cleaned_data["add_endpoints"]: + finding_to_merge_into.endpoints.add( + *finding.endpoints.all(), + ) + + # if checked merge the tags + if form.cleaned_data["tag_finding"]: + for tag in finding.tags.all(): + finding_to_merge_into.tags.add(tag) + + # if checked re-assign the burp requests to the merged finding + if form.cleaned_data["dynamic_raw"]: + BurpRawRequestResponse.objects.filter( + finding=finding, + ).update(finding=finding_to_merge_into) + + # Add merge finding information to the note if set to inactive + if form.cleaned_data["finding_action"] == "inactive": + single_finding_notes_entry = ("Finding has been set to inactive " + f"and merged with the finding: {finding_to_merge_into.title}.") + note = Notes( + entry=single_finding_notes_entry, author=request.user, + ) + note.save() + finding.notes.add(note) + + # If the merged finding should be tagged as merged-into + if form.cleaned_data["mark_tag_finding"]: + finding.tags.add("merged-inactive") + + # Update the finding to merge into + if finding_descriptions: + finding_to_merge_into.description = f"{finding_to_merge_into.description}\n\n{finding_descriptions}" + + if finding_to_merge_into.static_finding: + static = finding.static_finding + + if finding_to_merge_into.dynamic_finding: + dynamic = finding.dynamic_finding + + if finding_references: + finding_to_merge_into.references = f"{finding_to_merge_into.references}\n{finding_references}" + + finding_to_merge_into.static_finding = static + finding_to_merge_into.dynamic_finding = dynamic + + # Update the timestamp + finding_to_merge_into.last_reviewed = timezone.now() + finding_to_merge_into.last_reviewed_by = request.user + + # Save the data to the merged finding + finding_to_merge_into.save() + + # If the finding merged into should be tagged as merged + if form.cleaned_data["mark_tag_finding"]: + finding_to_merge_into.tags.add("merged") + + finding_action = "" + # Take action on the findings + if form.cleaned_data["finding_action"] == "inactive": + finding_action = "inactivated" + findings_to_merge.exclude(pk=finding_to_merge_into.pk).update( + active=False, + last_reviewed=timezone.now(), + last_reviewed_by=request.user, + ) + elif form.cleaned_data["finding_action"] == "delete": + finding_action = "deleted" + findings_to_merge.delete() + + notes_entry = ("Finding consists of merged findings from the following " + f"findings which have been {finding_action}: {notes_entry[:-1]}") + note = Notes(entry=notes_entry, author=request.user) + note.save() + finding_to_merge_into.notes.add(note) + + messages.add_message( + request, + messages.SUCCESS, + "Findings merged", + extra_tags="alert-success", + ) + return HttpResponseRedirect( + reverse("edit_finding", args=(finding_to_merge_into.id,)), + ) + messages.add_message( + request, + messages.ERROR, + "Unable to merge findings. Findings to merge contained in finding to merge into.", + extra_tags="alert-danger", + ) + else: + messages.add_message( + request, + messages.ERROR, + "Unable to merge findings. Required fields were not selected.", + extra_tags="alert-danger", + ) + + product_tab = Product_Tab( + finding.test.engagement.product, title="Merge Findings", tab="findings", + ) + custom_breadcrumb = { + "Open Findings": reverse( + "product_open_findings", args=(finding.test.engagement.product.id,), + ) + + "?test__engagement__product=" + + str(finding.test.engagement.product.id), + } + + return render( + request, + "dojo/merge_findings.html", + { + "form": form, + "name": "Merge Findings", + "finding": finding, + "product_tab": product_tab, + "title": product_tab.title, + "custom_breadcrumb": custom_breadcrumb, + }, + ) + +# bulk update and delete are combined, so we can't have the nice user_is_authorized decorator + + +def _bulk_delete_findings(request, pid, form, finding_to_update, finds, total_find_count): + """Helper function to handle bulk deletion of findings.""" + if form.is_valid() and finding_to_update: + if pid is not None: + product = get_object_or_404(Product, id=pid) + user_has_permission_or_403( + request.user, product, "delete", + ) + + finds = get_authorized_findings_for_queryset( + "delete", finds, + ).distinct() + + skipped_find_count = total_find_count - finds.count() + deleted_find_count = finds.count() + + for find in finds: + find.delete() + + if skipped_find_count > 0: + add_error_message_to_response( + f"Skipped deletion of {skipped_find_count} findings because you are not authorized.", + ) + + if deleted_find_count > 0: + messages.add_message( + request, + messages.SUCCESS, + f"Bulk delete of {deleted_find_count} findings was successful.", + extra_tags="alert-success", + ) + + +def _bulk_update_finding_status_and_severity(finds, form, request, system_settings, prods, now): + """Helper function to handle status and severity updates for findings.""" + skipped_duplicate_count = 0 + actually_updated_count = 0 + + if form.cleaned_data["severity"] or form.cleaned_data["status"]: + # Accumulate findings for batched FP-history processing after the per-finding loop + fp_findings = [] # findings being marked as FP + reactivation_findings = [] # findings being un-FP'd (retroactive reactivation) + + for find in finds: + old_find = copy.deepcopy(find) + + if form.cleaned_data["severity"]: + find.severity = form.cleaned_data["severity"] + find.numerical_severity = Finding.get_numerical_severity( + form.cleaned_data["severity"], + ) + find.last_reviewed = now + find.last_reviewed_by = request.user + + if form.cleaned_data["status"]: + # logger.debug('setting status from bulk edit form: %s', form) + # Check if finding is duplicate and user wants to set active/verified + if find.duplicate and (form.cleaned_data["active"] or form.cleaned_data["verified"]): + # Skip active/verified but allow other status changes + skipped_duplicate_count += 1 + # Set other fields but not active/verified + find.false_p = form.cleaned_data["false_p"] + find.out_of_scope = form.cleaned_data["out_of_scope"] + find.is_mitigated = form.cleaned_data["is_mitigated"] + find.under_review = form.cleaned_data["under_review"] + else: + # Apply all status changes normally + find.active = form.cleaned_data["active"] + find.verified = form.cleaned_data["verified"] + find.false_p = form.cleaned_data["false_p"] + find.out_of_scope = form.cleaned_data["out_of_scope"] + find.is_mitigated = form.cleaned_data["is_mitigated"] + find.under_review = form.cleaned_data["under_review"] + find.last_reviewed = timezone.now() + find.last_reviewed_by = request.user + + # use super to avoid all custom logic in our overriden save method + # it will trigger the pre_save signal + find.save_no_options() + actually_updated_count += 1 + + if system_settings.false_positive_history: + if find.false_p: + fp_findings.append(find) + elif old_find.false_p and not find.false_p: + reactivation_findings.append(find) + + # --- Batch FP history: one DB query per (product, algorithm) group instead of one per finding --- + if system_settings.false_positive_history and fp_findings: + groups: dict = defaultdict(list) + for find in fp_findings: + groups[find.test.engagement.product_id, find.test.deduplication_algorithm].append(find) + for group_findings in groups.values(): + do_false_positive_history_batch(group_findings) + + # --- Batch retroactive reactivation --- + if ( + system_settings.false_positive_history + and system_settings.retroactive_false_positive_history + and reactivation_findings + ): + all_fp_ids_to_reactivate: set = set() + groups = defaultdict(list) + for find in reactivation_findings: + groups[find.test.engagement.product_id, find.test.deduplication_algorithm].append(find) + for (_, dedup_alg), group_findings in groups.items(): + product = group_findings[0].test.engagement.product + candidates = _fetch_fp_candidates_for_batch(group_findings, product, dedup_alg) + for find in group_findings: + if dedup_alg == "unique_id_from_tool_or_hash_code": + by_uid, by_hash = candidates + uid_matches = by_uid.get(find.unique_id_from_tool, []) if find.unique_id_from_tool else [] + hash_matches = by_hash.get(find.hash_code, []) if find.hash_code else [] + seen: dict = {} + for ef in uid_matches + hash_matches: + seen.setdefault(ef.id, ef) + existing = list(seen.values()) + elif dedup_alg == "hash_code": + existing = candidates.get(find.hash_code, []) if find.hash_code else [] + elif dedup_alg == "unique_id_from_tool": + existing = candidates.get(find.unique_id_from_tool, []) if find.unique_id_from_tool else [] + elif dedup_alg == "legacy": + lookup_key = (find.title.lower(), find.severity) if find.title else None + existing = candidates.get(lookup_key, []) if lookup_key else [] + else: + existing = [] + for ef in existing: + if ef.false_p: + all_fp_ids_to_reactivate.add(ef.id) + + if all_fp_ids_to_reactivate: + logger.debug( + "FALSE_POSITIVE_HISTORY: Reactivating %i finding(s): %s", + len(all_fp_ids_to_reactivate), + sorted(all_fp_ids_to_reactivate), + ) + # All reactivation findings received the same form values, so a single bulk update covers all. + # QuerySet.update() bypasses Django signals, which is intentional here — it mirrors + # the previous save_no_options() calls that also disabled all post-save processing. + Finding.objects.filter(id__in=all_fp_ids_to_reactivate).update( + false_p=False, + active=form.cleaned_data["active"], + verified=form.cleaned_data["verified"], + out_of_scope=form.cleaned_data["out_of_scope"], + is_mitigated=form.cleaned_data["is_mitigated"], + ) + + for prod in prods: + calculate_grade(prod.id) + + if skipped_duplicate_count > 0: + messages.add_message( + request, + messages.WARNING, + f"Skipped status update of {skipped_duplicate_count} duplicate findings. Duplicate findings cannot be active or verified.", + extra_tags="alert-warning", + ) + + return skipped_duplicate_count, actually_updated_count + + +def _bulk_update_simple_fields(finds, form): + """Helper function to handle simple field updates (date, planned_remediation_date, etc.).""" + if form.cleaned_data["date"]: + for finding in finds: + finding.date = form.cleaned_data["date"] + finding.save_no_options() + + if form.cleaned_data["planned_remediation_date"]: + for finding in finds: + finding.planned_remediation_date = form.cleaned_data[ + "planned_remediation_date" + ] + finding.save_no_options() + + if form.cleaned_data["planned_remediation_version"]: + for finding in finds: + finding.planned_remediation_version = form.cleaned_data[ + "planned_remediation_version" + ] + finding.save_no_options() + + +def _bulk_update_risk_acceptance(finds, form, request, prods): + """Helper function to handle risk acceptance updates.""" + skipped_risk_accept_count = 0 + + if form.cleaned_data["risk_acceptance"]: + for finding in finds: + if form.cleaned_data["risk_accept"]: + if ( + not finding.test.engagement.product.enable_simple_risk_acceptance + ): + skipped_risk_accept_count += 1 + else: + ra_helper.simple_risk_accept(request.user, finding) + elif form.cleaned_data["risk_unaccept"]: + ra_helper.risk_unaccept(request.user, finding) + + for prod in prods: + calculate_grade(prod.id) + + if skipped_risk_accept_count > 0: + messages.add_message( + request, + messages.WARNING, + (f"Skipped simple risk acceptance of {skipped_risk_accept_count} findings, " + "simple risk acceptance is disabled on the related products"), + extra_tags="alert-warning", + ) + + return skipped_risk_accept_count + + +def _bulk_update_finding_groups(finds, form): + """Helper function to handle finding group operations.""" + return_url = None + + if form.cleaned_data["finding_group_create"]: + logger.debug("finding_group_create checked!") + finding_group_name = form.cleaned_data["finding_group_create_name"] + logger.debug("finding_group_create_name: %s", finding_group_name) + finding_group, added, skipped = finding_helper.create_finding_group( + finds, finding_group_name, + ) + + if added: + add_success_message_to_response( + f"Created finding group with {added} findings", + ) + return_url = reverse( + "view_finding_group", args=(finding_group.id,), + ) + + if skipped: + add_success_message_to_response( + f"Skipped {skipped} findings in group creation, findings already part of another group", + ) + + # refresh findings from db + finds = finds.all() + + if form.cleaned_data["finding_group_add"]: + logger.debug("finding_group_add checked!") + fgid = form.cleaned_data["add_to_finding_group_id"] + finding_group = Finding_Group.objects.get(id=fgid) + finding_group, added, skipped = finding_helper.add_to_finding_group( + finding_group, finds, + ) + + if added: + add_success_message_to_response( + f"Added {added} findings to finding group {finding_group.name}", + ) + return_url = reverse( + "view_finding_group", args=(finding_group.id,), + ) + + if skipped: + add_success_message_to_response( + f"Skipped {skipped} findings when adding to finding group {finding_group.name}, " + "findings already part of another group", + ) + + # refresh findings from db + finds = finds.all() + + if form.cleaned_data["finding_group_remove"]: + logger.debug("finding_group_remove checked!") + ( + finding_groups, + removed, + skipped, + ) = finding_helper.remove_from_finding_group(finds) + + if removed: + add_success_message_to_response( + "Removed {} findings from finding groups {}".format( + removed, + ",".join( + [ + finding_group.name + for finding_group in finding_groups + ], + ), + ), + ) + + if skipped: + add_success_message_to_response( + f"Skipped {skipped} findings when removing from any finding group, findings not part of any group", + ) + + # refresh findings from db + finds = finds.all() + + if form.cleaned_data["finding_group_by"]: + logger.debug("finding_group_by checked!") + logger.debug(form.cleaned_data) + finding_group_by_option = form.cleaned_data[ + "finding_group_by_option" + ] + logger.debug("finding_group_by_option: %s", finding_group_by_option) + + ( + finding_groups, + grouped, + skipped, + groups_created, + ) = finding_helper.group_findings_by(finds, finding_group_by_option) + + if grouped: + add_success_message_to_response( + f"Grouped {grouped} findings into {len(finding_groups)} ({groups_created} newly created) finding groups", + ) + + if skipped: + add_success_message_to_response( + f"Skipped {skipped} findings when grouping by {finding_group_by_option} as these findings " + "were already in an existing group", + ) + + # refresh findings from db + finds = finds.all() + + return return_url, finds + + +def _bulk_push_to_jira(finds, form, note): + """Helper function to handle JIRA push operations.""" + error_counts = defaultdict(lambda: 0) + success_count = 0 + finding_groups = set( # noqa: C401 + finding.finding_group + for finding in finds + if finding.has_finding_group + and ( + jira_services.is_push_all_issues(finding) + or jira_services.is_keep_in_sync(finding) + or form.cleaned_data.get("push_to_jira") + ) + ) + logger.debug("finding_groups: %s", finding_groups) + for group in finding_groups: + if ( + form.cleaned_data.get("push_to_jira") + or jira_services.is_push_all_issues(group) + or jira_services.is_keep_in_sync(group) + ): + ( + can_be_pushed_to_jira, + error_message, + _error_code, + ) = jira_services.can_be_pushed(group) + if not can_be_pushed_to_jira: + error_counts[error_message] += 1 + jira_services.log_cannot_be_pushed_reason(error_message, group) + else: + logger.debug( + "pushing to jira from finding.finding_bulk_update_all()", + ) + jira_services.push(group) + success_count += 1 + + for error_message, error_count in error_counts.items(): + add_error_message_to_response(f"{error_count} finding groups could not be pushed to JIRA: {error_message}") + + if success_count > 0: + add_success_message_to_response(f"{success_count} finding groups pushed to JIRA successfully") + + # refresh from db + finds = finds.all() + + error_counts = defaultdict(lambda: 0) + success_count = 0 + for finding in finds: + tool_issue_updater.async_tool_issue_update(finding) + + # not sure yet if we want to support bulk unlink, so leave as commented out for now + # if form.cleaned_data['unlink_from_jira']: + # if finding.has_jira_issue: + # jira_services.unlink_finding(request, finding) + + # Because we never call finding.save() in a bulk update, we need to actually + # push the JIRA stuff here, rather than in finding.save() + # can't use helper as when push_all_jira_issues is True, + # the checkbox gets disabled and is always false + # push_to_jira = jira_services.is_push_to_jira(new_finding, + # form.cleaned_data.get('push_to_jira')) + if ( + form.cleaned_data.get("push_to_jira") + or jira_services.is_push_all_issues(finding) + or jira_services.is_keep_in_sync(finding) + ) and not finding.has_finding_group: + ( + can_be_pushed_to_jira, + error_message, + _error_code, + ) = jira_services.can_be_pushed(finding) + if finding.has_jira_group_issue and not finding.has_jira_issue: + error_message = ( + "finding already pushed as part of Finding Group" + ) + error_counts[error_message] += 1 + jira_services.log_cannot_be_pushed_reason(error_message, finding) + elif not can_be_pushed_to_jira: + error_counts[error_message] += 1 + jira_services.log_cannot_be_pushed_reason(error_message, finding) + else: + logger.debug( + "pushing to jira from finding.finding_bulk_update_all()", + ) + jira_services.push(finding) + if note is not None and isinstance(note, Notes): + jira_services.add_comment(finding, note) + success_count += 1 + + for error_message, error_count in error_counts.items(): + add_error_message_to_response(f"{error_count} findings could not be pushed to JIRA: {error_message}") + + if success_count > 0: + add_success_message_to_response(f"{success_count} findings pushed to JIRA successfully") + + +def finding_bulk_update_all(request, pid=None): + system_settings = System_Settings.objects.get() + + logger.debug("bulk 10") + form = FindingBulkUpdateForm(request.POST) + now = timezone.now() + return_url = None + + if request.method == "POST": + logger.debug("bulk 20") + + finding_to_update = request.POST.getlist("finding_to_update") + # Add pghistory context for audit trail (adds to existing middleware context) + pghistory.context( + source="bulk_edit", + finding_count=len(finding_to_update), + ) + finds = Finding.objects.filter(id__in=finding_to_update).order_by("id") + total_find_count = finds.count() + prods = set(find.test.engagement.product for find in finds) # noqa: C401 + if request.POST.get("delete_bulk_findings"): + _bulk_delete_findings(request, pid, form, finding_to_update, finds, total_find_count) + elif form.is_valid() and finding_to_update: + if pid is not None: + product = get_object_or_404(Product, id=pid) + user_has_permission_or_403( + request.user, product, "edit", + ) + + # make sure users are not editing stuff they are not authorized for + finds = get_authorized_findings_for_queryset( + "edit", finds, + ).distinct() + + skipped_find_count = total_find_count - finds.count() + updated_find_count = finds.count() + + if skipped_find_count > 0: + add_error_message_to_response( + f"Skipped update of {skipped_find_count} findings because you are not authorized.", + ) + + finds = prefetch_for_findings(finds) + note = None + actually_updated_count = 0 + + _skipped_duplicate_count, actually_updated_count = _bulk_update_finding_status_and_severity( + finds, form, request, system_settings, prods, now, + ) + + _bulk_update_simple_fields(finds, form) + + _skipped_risk_accept_count = _bulk_update_risk_acceptance( + finds, form, request, prods, + ) + + group_return_url, finds = _bulk_update_finding_groups(finds, form) + if group_return_url: + return_url = group_return_url + + if form.cleaned_data["push_to_github"]: + logger.debug("push selected findings to github") + for finding in finds: + logger.debug("will push to GitHub finding: " + str(finding)) + old_status = finding.status() + if form.cleaned_data["push_to_github"]: + if GITHUB_Issue.objects.filter(finding=finding).exists(): + update_external_issue(finding.id, old_status, "github") + else: + add_external_issue(finding.id, "github") + + if form.cleaned_data["notes"]: + logger.debug("Setting bulk notes") + note = Notes( + entry=form.cleaned_data["notes"], + author=request.user, + date=timezone.now(), + ) + note.save() + history = NoteHistory( + data=note.entry, time=note.date, current_editor=note.author, + ) + history.save() + note.history.add(history) + for finding in finds: + finding.notes.add(note) + finding.save() + + if form.cleaned_data["tags"]: + tags = form.cleaned_data["tags"] + logger.debug("bulk_edit: adding tags to %d findings: %s", finds.count(), tags) + # Delegate parsing and handling of strings/iterables to helper + bulk_add_tags_to_instances(tag_or_tags=tags, instances=finds, tag_field_name="tags") + + _bulk_push_to_jira(finds, form, note) + + # Show success message if status/severity updates were made (using actually_updated_count) + # or if other updates were made (using updated_find_count) + if (form.cleaned_data["severity"] or form.cleaned_data["status"]) and actually_updated_count > 0: + messages.add_message( + request, + messages.SUCCESS, + f"Bulk update of {actually_updated_count} findings was successful.", + extra_tags="alert-success", + ) + elif updated_find_count > 0 and ( + form.cleaned_data["date"] or form.cleaned_data["planned_remediation_date"] + or form.cleaned_data["planned_remediation_version"] or form.cleaned_data["tags"] + or form.cleaned_data["notes"] or form.cleaned_data["risk_acceptance"] + or form.cleaned_data["finding_group_create"] or form.cleaned_data["finding_group_add"] + or form.cleaned_data["finding_group_remove"] or form.cleaned_data["finding_group_by"] + or form.cleaned_data["push_to_jira"] or form.cleaned_data["push_to_github"] + ): + messages.add_message( + request, + messages.SUCCESS, + f"Bulk update of {updated_find_count} findings was successful.", + extra_tags="alert-success", + ) + else: + messages.add_message( + request, + messages.ERROR, + "Unable to process bulk update. Required fields were not selected.", + extra_tags="alert-danger", + ) + + if return_url: + redirect(request, return_url) + + return redirect_to_return_url_or_else(request, None) + + +def find_available_notetypes(notes): + single_note_types = Note_Type.objects.filter( + is_single=True, is_active=True, + ).values_list("id", flat=True) + multiple_note_types = Note_Type.objects.filter( + is_single=False, is_active=True, + ).values_list("id", flat=True) + available_note_types = [] + for note_type_id in multiple_note_types: + available_note_types.append(note_type_id) + for note_type_id in single_note_types: + for note in notes: + if note_type_id == note.note_type_id: + break + else: + available_note_types.append(note_type_id) + return Note_Type.objects.filter(id__in=available_note_types).order_by("-id") + + +def get_missing_mandatory_notetypes(finding): + notes = finding.notes.all() + mandatory_note_types = Note_Type.objects.filter( + is_mandatory=True, is_active=True, + ).values_list("id", flat=True) + notes_to_be_added = [] + for note_type_id in mandatory_note_types: + for note in notes: + if note_type_id == note.note_type_id: + break + else: + notes_to_be_added.append(note_type_id) + return Note_Type.objects.filter(id__in=notes_to_be_added) + + +@require_POST +def mark_finding_duplicate(request, original_id, duplicate_id): + + original = get_object_or_404(Finding, id=original_id) + duplicate = get_object_or_404( + Finding.objects.filter(test__engagement__product=original.test.engagement.product), + id=duplicate_id, + ) + + if original.test.engagement != duplicate.test.engagement: + if (original.test.engagement.deduplication_on_engagement + or duplicate.test.engagement.deduplication_on_engagement): + messages.add_message( + request, + messages.ERROR, + ("Marking finding as duplicate/original failed as they are not in the same engagement " + "and deduplication_on_engagement is enabled for at least one of them"), + extra_tags="alert-danger", + ) + return redirect_to_return_url_or_else( + request, reverse("view_finding", args=(duplicate.id,)), + ) + + duplicate.duplicate = True + duplicate.active = False + duplicate.verified = False + # make sure we don't create circular or transitive duplicates + if original.duplicate: + duplicate.duplicate_finding = original.duplicate_finding + else: + duplicate.duplicate_finding = original + + logger.debug( + "marking finding %i as duplicate of %i", + duplicate.id, + duplicate.duplicate_finding.id, + ) + + duplicate.last_reviewed = timezone.now() + duplicate.last_reviewed_by = request.user + duplicate.save(dedupe_option=False) + original.found_by.add(duplicate.test.test_type) + original.save(dedupe_option=False) + + return redirect_to_return_url_or_else( + request, reverse("view_finding", args=(duplicate.id,)), + ) + + +def reset_finding_duplicate_status_internal(user, duplicate_id): + duplicate = get_object_or_404(Finding, id=duplicate_id) + + if not duplicate.duplicate: + return None + + logger.debug("resetting duplicate status of %i", duplicate.id) + duplicate.duplicate = False + duplicate.active = True + if duplicate.duplicate_finding: + # duplicate.duplicate_finding.original_finding.remove(duplicate) # shouldn't be needed + duplicate.duplicate_finding = None + duplicate.last_reviewed = timezone.now() + duplicate.last_reviewed_by = user + duplicate.save(dedupe_option=False) + + return duplicate.id + + +@require_POST +def reset_finding_duplicate_status(request, duplicate_id): + checked_duplicate_id = reset_finding_duplicate_status_internal( + request.user, duplicate_id, + ) + if checked_duplicate_id is None: + messages.add_message( + request, + messages.ERROR, + "Can't reset duplicate status of a finding that is not a duplicate", + extra_tags="alert-danger", + ) + return redirect_to_return_url_or_else( + request, reverse("view_finding", args=(duplicate_id,)), + ) + + return redirect_to_return_url_or_else( + request, reverse("view_finding", args=(checked_duplicate_id,)), + ) + + +def set_finding_as_original_internal(user, finding_id, new_original_id): + finding = get_object_or_404(Finding, id=finding_id) + new_original = get_object_or_404( + Finding.objects.filter(test__engagement__product=finding.test.engagement.product), + id=new_original_id, + ) + + if finding.test.engagement != new_original.test.engagement: + if (finding.test.engagement.deduplication_on_engagement + or new_original.test.engagement.deduplication_on_engagement): + return False + + if finding.duplicate or finding.original_finding.all(): + # existing cluster, so update all cluster members + + if finding.duplicate and finding.duplicate_finding: + logger.debug( + "setting old original %i as duplicate of %i", + finding.duplicate_finding.id, + new_original.id, + ) + finding.duplicate_finding.duplicate_finding = new_original + finding.duplicate_finding.duplicate = True + finding.duplicate_finding.save(dedupe_option=False) + + for cluster_member in finding.duplicate_finding_set(): + if cluster_member != new_original: + logger.debug( + "setting new original for %i to %i", + cluster_member.id, + new_original.id, + ) + cluster_member.duplicate_finding = new_original + cluster_member.save(dedupe_option=False) + + logger.debug( + "setting new original for old root %i to %i", finding.id, new_original.id, + ) + finding.duplicate = True + finding.duplicate_finding = new_original + finding.save(dedupe_option=False) + + else: + # creating a new cluster, so mark finding as duplicate + logger.debug("marking %i as duplicate of %i", finding.id, new_original.id) + finding.duplicate = True + finding.active = False + finding.duplicate_finding = new_original + finding.last_reviewed = timezone.now() + finding.last_reviewed_by = user + finding.save(dedupe_option=False) + + logger.debug("marking new original %i as not duplicate", new_original.id) + new_original.duplicate = False + new_original.duplicate_finding = None + new_original.save(dedupe_option=False) + + return True + + +@require_POST +def set_finding_as_original(request, finding_id, new_original_id): + success = set_finding_as_original_internal( + request.user, finding_id, new_original_id, + ) + if not success: + messages.add_message( + request, + messages.ERROR, + ("Marking finding as duplicate/original failed as they are not in the same engagement " + "and deduplication_on_engagement is enabled for at least one of them"), + extra_tags="alert-danger", + ) + + return redirect_to_return_url_or_else( + request, reverse("view_finding", args=(finding_id,)), + ) + + +@require_POST +def unlink_jira(request, fid): + finding = get_object_or_404(Finding, id=fid) + logger.info( + "trying to unlink a linked jira issue from %d:%s", finding.id, finding.title, + ) + if finding.has_jira_issue: + try: + jira_services.unlink_finding(request, finding) + + messages.add_message( + request, + messages.SUCCESS, + "Link to JIRA issue succesfully deleted", + extra_tags="alert-success", + ) + + return JsonResponse({"result": "OK"}) + except Exception: + logger.exception("Link to JIRA could not be deleted") + messages.add_message( + request, + messages.ERROR, + "Link to JIRA could not be deleted, see alerts for details", + extra_tags="alert-danger", + ) + + return HttpResponse(status=500) + else: + messages.add_message( + request, messages.ERROR, "Link to JIRA not found", extra_tags="alert-danger", + ) + return HttpResponse(status=400) + + +@require_POST +def push_to_jira(request, fid): + finding = get_object_or_404(Finding, id=fid) + try: + logger.info( + "trying to push %d:%s to JIRA to create or update JIRA issue", + finding.id, + finding.title, + ) + logger.debug("pushing to jira from finding.push_to-jira()") + + # it may look like succes here, but the push_to_jira are swallowing exceptions + # but cant't change too much now without having a test suite, + # so leave as is for now with the addition warning message + # to check alerts for background errors. + if jira_services.push(finding): + messages.add_message( + request, + messages.SUCCESS, + message="Action queued to create or update linked JIRA issue, check alerts for background errors.", + extra_tags="alert-success", + ) + else: + messages.add_message( + request, + messages.SUCCESS, + "Push to JIRA failed, check alerts on the top right for errors", + extra_tags="alert-danger", + ) + + return JsonResponse({"result": "OK"}) + except Exception: + logger.exception("Error pushing to JIRA") + messages.add_message( + request, messages.ERROR, "Error pushing to JIRA", extra_tags="alert-danger", + ) + return HttpResponse(status=500) + + +# precalculate because we need related_actions to be set +def duplicate_cluster(request, finding): + duplicate_cluster = finding.duplicate_finding_set() + + duplicate_cluster = prefetch_for_findings(duplicate_cluster) + + # populate actions for findings in duplicate cluster + for duplicate_member in duplicate_cluster: + duplicate_member.related_actions = ( + calculate_possible_related_actions_for_similar_finding( + request, finding, duplicate_member, + ) + ) + + return duplicate_cluster + + +# django doesn't allow much logic or even method calls with parameters in templates. +# so we have to use a function in this view to calculate the possible actions on a similar (or duplicate) finding. +# and we assign this dictionary to the finding so it can be accessed in the template. +# these actions are always calculated in the context of the finding the user is viewing +# because this determines which actions are possible +def calculate_possible_related_actions_for_similar_finding( + request, finding, similar_finding, +): + actions = [] + if similar_finding.test.engagement != finding.test.engagement and ( + similar_finding.test.engagement.deduplication_on_engagement + or finding.test.engagement.deduplication_on_engagement + ): + actions.append( + { + "action": "None", + "reason": ("This finding is in a different engagement and deduplication_inside_engagment " + "is enabled here or in that finding"), + }, + ) + elif finding.duplicate_finding == similar_finding: + actions.append( + { + "action": "None", + "reason": ("This finding is the root of the cluster, use an action on another row, " + "or the finding on top of the page to change the root of the cluser"), + }, + ) + elif similar_finding.original_finding.all(): + actions.append( + { + "action": "None", + "reason": ("This finding is similar, but is already an original in a different cluster. " + "Remove it from that cluster before you connect it to this cluster."), + }, + ) + elif similar_finding.duplicate_finding: + # reset duplicate status is always possible + actions.append( + { + "action": "reset_finding_duplicate_status", + "reason": ("This will remove the finding from the cluster, " + "effectively marking it no longer as duplicate. " + "Will not trigger deduplication logic after saving."), + }, + ) + + if similar_finding.duplicate_finding in {finding, finding.duplicate_finding}: + # duplicate inside the same cluster + actions.append( + { + "action": "set_finding_as_original", + "reason": ("Sets this finding as the Original for the whole cluster. " + "The existing Original will be downgraded to become a member of the cluster and, " + "together with the other members, will be marked as duplicate of the new Original."), + }, + ) + else: + # duplicate inside different cluster + actions.append( + { + "action": "mark_finding_duplicate", + "reason": ("Will mark this finding as duplicate of the root finding in this cluster, " + "effectively adding it to the cluster and removing it from the other cluster."), + }, + ) + # similar is not a duplicate yet + elif finding.duplicate or finding.original_finding.all(): + actions.extend(( + { + "action": "mark_finding_duplicate", + "reason": "Will mark this finding as duplicate of the root finding in this cluster", + }, { + "action": "set_finding_as_original", + "reason": ( + "Sets this finding as the Original for the whole cluster. " + "The existing Original will be downgraded to become a member of the cluster and, " + "together with the other members, will be marked as duplicate of the new Original." + ), + }, + )) + else: + # similar_finding is not an original/root of a cluster as per earlier if clause + actions.extend(( + { + "action": "mark_finding_duplicate", + "reason": "Will mark this finding as duplicate of the finding on this page.", + }, { + "action": "set_finding_as_original", + "reason": ( + "Sets this finding as the Original marking the finding " + "on this page as duplicate of this original." + ), + }, + )) + + return actions diff --git a/dojo/finding/views.py b/dojo/finding/views.py index e39a0a8fea8..41b9deba7ff 100644 --- a/dojo/finding/views.py +++ b/dojo/finding/views.py @@ -1,3407 +1,4 @@ -# # findings -import base64 -import contextlib -import copy -import logging -import mimetypes -from collections import OrderedDict, defaultdict -from itertools import chain -from pathlib import Path - -import pghistory -from django.conf import settings -from django.contrib import messages -from django.core import serializers -from django.core.exceptions import PermissionDenied, ValidationError -from django.db import models -from django.db.models import Case, F, QuerySet, Value, When -from django.db.models.functions import Coalesce, ExtractDay, Length, TruncDate -from django.db.models.query import Prefetch -from django.http import Http404, HttpRequest, HttpResponse, HttpResponseRedirect, JsonResponse, StreamingHttpResponse -from django.shortcuts import get_object_or_404, render -from django.urls import reverse -from django.utils import timezone -from django.utils.safestring import mark_safe -from django.utils.translation import gettext as _ -from django.views import View -from django.views.decorators.http import require_POST -from imagekit import ImageSpec -from imagekit.processors import ResizeToFill - -import dojo.finding.helper as finding_helper -import dojo.risk_acceptance.helper as ra_helper -from dojo.authorization.authorization import user_has_global_permission_or_403, user_has_permission_or_403 -from dojo.celery_dispatch import dojo_dispatch_task -from dojo.endpoint.queries import get_authorized_endpoints -from dojo.filters import ( - AcceptedFindingFilter, - AcceptedFindingFilterWithoutObjectLookups, - FindingFilter, - FindingFilterWithoutObjectLookups, - SimilarFindingFilter, - SimilarFindingFilterWithoutObjectLookups, - TemplateFindingFilter, - TestImportFilter, - TestImportFindingActionFilter, -) -from dojo.finding.deduplication import ( - _fetch_fp_candidates_for_batch, - do_false_positive_history_batch, - match_finding_to_existing_findings, -) -from dojo.finding.queries import get_authorized_findings, get_authorized_findings_for_queryset, prefetch_for_findings -from dojo.forms import ( - ApplyFindingTemplateForm, - ClearFindingReviewForm, - CloseFindingForm, - CopyFindingForm, - DefectFindingForm, - DeleteFindingForm, - DeleteFindingTemplateForm, - EditPlannedRemediationDateFindingForm, - FindingBulkUpdateForm, - FindingForm, - FindingTemplateForm, - GITHUBFindingForm, - JIRAFindingForm, - MergeFindings, - NoteForm, - ReviewFindingForm, - TypedNoteForm, -) -from dojo.jira import services as jira_services -from dojo.location.queries import get_authorized_locations -from dojo.location.status import FindingLocationStatus -from dojo.models import ( - IMPORT_UNTOUCHED_FINDING, - BurpRawRequestResponse, - Dojo_User, - Endpoint_Status, - Engagement, - FileAccessToken, - Finding, - Finding_Group, - Finding_Template, - GITHUB_Issue, - GITHUB_PKey, - Note_Type, - NoteHistory, - Notes, - Product, - System_Settings, - Test, - Test_Import, - Test_Import_Finding_Action, - User, -) -from dojo.notifications.helper import create_notification -from dojo.tags.utils import bulk_add_tags_to_instances -from dojo.test.queries import get_authorized_tests -from dojo.tools import tool_issue_updater -from dojo.utils import ( - FileIterWrapper, - Product_Tab, - add_breadcrumb, - add_error_message_to_response, - add_external_issue, - add_field_errors_to_response, - add_success_message_to_response, - calculate_grade, - get_page_items, - get_page_items_and_count, - get_return_url, - get_system_setting, - get_visible_scan_types, - get_words_for_field, - process_tag_notifications, - redirect, - redirect_to_return_url_or_else, - reopen_external_issue, - update_external_issue, -) - -JFORM_PUSH_TO_JIRA_MESSAGE = "jform.push_to_jira: %s" - -logger = logging.getLogger(__name__) - - -def prefetch_for_similar_findings(findings): - prefetched_findings = findings - if isinstance( - findings, QuerySet, - ): # old code can arrive here with prods being a list because the query was already executed - prefetched_findings = prefetched_findings.prefetch_related("reporter") - prefetched_findings = prefetched_findings.prefetch_related( - "jira_issue__jira_project__jira_instance", - ) - prefetched_findings = prefetched_findings.prefetch_related("test__test_type") - prefetched_findings = prefetched_findings.prefetch_related( - "test__engagement__jira_project__jira_instance", - ) - prefetched_findings = prefetched_findings.prefetch_related( - "test__engagement__product__jira_project_set__jira_instance", - ) - prefetched_findings = prefetched_findings.prefetch_related("found_by") - prefetched_findings = prefetched_findings.prefetch_related( - "risk_acceptance_set", - ) - prefetched_findings = prefetched_findings.prefetch_related( - "risk_acceptance_set__accepted_findings", - ) - prefetched_findings = prefetched_findings.prefetch_related("original_finding") - prefetched_findings = prefetched_findings.prefetch_related("duplicate_finding") - # filter out noop reimport actions from finding status history - prefetched_findings = prefetched_findings.prefetch_related( - Prefetch( - "test_import_finding_action_set", - queryset=Test_Import_Finding_Action.objects.exclude( - action=IMPORT_UNTOUCHED_FINDING, - ), - ), - ) - """ - we could try to prefetch only the latest note with SubQuery and OuterRef, - but I'm getting that MySql doesn't support limits in subqueries. - """ - prefetched_findings = prefetched_findings.prefetch_related("notes") - prefetched_findings = prefetched_findings.prefetch_related("tags") - prefetched_findings = prefetched_findings.prefetch_related( - "vulnerability_id_set", - ) - else: - logger.debug("unable to prefetch because query was already executed") - - return prefetched_findings - - -class BaseListFindings: - def __init__( - self, - filter_name: str = "All", - product_id: int | None = None, - engagement_id: int | None = None, - test_id: int | None = None, - order_by: str = "numerical_severity", - prefetch_type: str = "all", - ): - self.filter_name = filter_name - self.product_id = product_id - self.engagement_id = engagement_id - self.test_id = test_id - self.order_by = order_by - self.prefetch_type = prefetch_type - - def get_filter_name(self): - if not hasattr(self, "filter_name"): - self.filter_name = "All" - return self.filter_name - - def get_order_by(self): - if not hasattr(self, "order_by"): - self.order_by = "numerical_severity" - return self.order_by - - def get_prefetch_type(self): - if not hasattr(self, "prefetch_type"): - self.prefetch_type = "all" - return self.prefetch_type - - def get_product_id(self): - if not hasattr(self, "product_id"): - self.product_id = None - return self.product_id - - def get_engagement_id(self): - if not hasattr(self, "engagement_id"): - self.engagement_id = None - return self.engagement_id - - def get_test_id(self): - if not hasattr(self, "test_id"): - self.test_id = None - return self.test_id - - def filter_findings_by_object(self, findings: QuerySet[Finding]): - if product_id := self.get_product_id(): - return findings.filter(test__engagement__product__id=product_id) - if engagement_id := self.get_engagement_id(): - return findings.filter(test__engagement=engagement_id) - if test_id := self.get_test_id(): - return findings.filter(test=test_id) - return findings - - def filter_findings_by_filter_name(self, findings: QuerySet[Finding]): - filter_name = self.get_filter_name() - if filter_name == "Open": - return findings.filter(finding_helper.OPEN_FINDINGS_QUERY) - if filter_name == "Verified": - return findings.filter(finding_helper.VERIFIED_FINDINGS_QUERY) - if filter_name == "Out of Scope": - return findings.filter(finding_helper.OUT_OF_SCOPE_FINDINGS_QUERY) - if filter_name == "False Positive": - return findings.filter(finding_helper.FALSE_POSITIVE_FINDINGS_QUERY) - if filter_name == "Inactive": - return findings.filter(finding_helper.INACTIVE_FINDINGS_QUERY) - if filter_name == "Accepted": - return findings.filter(finding_helper.ACCEPTED_FINDINGS_QUERY) - if filter_name == "Closed": - return findings.filter(finding_helper.CLOSED_FINDINGS_QUERY) - return findings - - def filter_findings_by_form(self, request: HttpRequest, findings: QuerySet[Finding]): - # Apply default ordering if no ordering parameter is provided - # This maintains backward compatibility with the previous behavior - if not request.GET.get("o"): - findings = findings.order_by(self.get_order_by()) - - # Set up the args for the form - args = [request.GET, findings] - # Set the initial form args - kwargs = { - "user": request.user, - "pid": self.get_product_id(), - "eid": self.get_engagement_id(), - "tid": self.get_test_id(), - } - - filter_string_matching = get_system_setting("filter_string_matching", False) - finding_filter_class = FindingFilterWithoutObjectLookups if filter_string_matching else FindingFilter - accepted_finding_filter_class = AcceptedFindingFilterWithoutObjectLookups if filter_string_matching else AcceptedFindingFilter - return ( - accepted_finding_filter_class(*args, **kwargs) - if self.get_filter_name() == "Accepted" - else finding_filter_class(*args, **kwargs) - ) - - def get_filtered_findings(self): - findings = get_authorized_findings("view") - # Annotate computed SLA age in days: sla_expiration_date - (sla_start_date or date) - # Handle NULL sla_expiration_date by using Coalesce to provide a large default value - # so NULLs sort last when sorting ascending (most urgent first) - findings = findings.annotate( - sla_age_days=Coalesce( - ExtractDay( - F("sla_expiration_date") - Coalesce(F("sla_start_date"), TruncDate("created")), - ), - Value(999999), # Large value to push NULLs to the end when sorting ascending - output_field=models.IntegerField(), - ), - ) - # Don't apply initial order_by here - let OrderingFilter handle it via request.GET['o'] - # This prevents conflicts between initial ordering and user-requested sorting - findings = self.filter_findings_by_object(findings) - return self.filter_findings_by_filter_name(findings) - - def get_fully_filtered_findings(self, request: HttpRequest): - findings = self.get_filtered_findings() - return self.filter_findings_by_form(request, findings) - - -class ListFindings(View, BaseListFindings): - def get_initial_context(self, request: HttpRequest): - context = { - "filter_name": self.get_filter_name(), - "show_product_column": True, - "custom_breadcrumb": None, - "product_tab": None, - "jira_project": None, - "github_config": None, - "bulk_edit_form": FindingBulkUpdateForm(request.GET), - "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), - "title_words": get_words_for_field(Finding, "title"), - "component_words": get_words_for_field(Finding, "component_name"), - "visible_test_types": get_visible_scan_types(), - } - # Look to see if the product was used - if product_id := self.get_product_id(): - product = get_object_or_404(Product, id=product_id) - user_has_permission_or_403(request.user, product, "view") - context["show_product_column"] = False - context["product_tab"] = Product_Tab(product, title="Findings", tab="findings") - context["jira_project"] = jira_services.get_project(product) - if github_config := GITHUB_PKey.objects.filter(product=product).first(): - context["github_config"] = github_config.git_conf_id - elif engagement_id := self.get_engagement_id(): - engagement = get_object_or_404(Engagement, id=engagement_id) - user_has_permission_or_403(request.user, engagement, "view") - context["show_product_column"] = False - context["product_tab"] = Product_Tab(engagement.product, title=engagement.name, tab="engagements") - context["jira_project"] = jira_services.get_project(engagement) - if github_config := GITHUB_PKey.objects.filter(product__engagement=engagement).first(): - context["github_config"] = github_config.git_conf_id - - return request, context - - def get_template(self): - return "dojo/findings_list.html" - - def add_breadcrumbs(self, request: HttpRequest, context: dict): - # show custom breadcrumb if user has filtered by exactly 1 endpoint - if "endpoints" in request.GET: - endpoint_ids = request.GET.getlist("endpoints", []) - if len(endpoint_ids) == 1 and endpoint_ids[0]: - endpoint_id = endpoint_ids[0] - # The findings `endpoints` filter is V3-gated (dojo/filters.py): - # under V3 it resolves against Location, otherwise against the - # legacy Endpoint model. Resolve the breadcrumb id against that - # same model so neither mode 404s a valid id. Scope to objects - # the user may view so the breadcrumb cannot disclose the - # existence/URL of one they cannot access (404 for both missing - # and unauthorized ids). - if settings.V3_FEATURE_LOCATIONS: - authorized = get_authorized_locations("view", user=request.user) - else: - authorized = get_authorized_endpoints("view", user=request.user) - endpoint = get_object_or_404(authorized, id=endpoint_id) - context["filter_name"] = "Vulnerable Endpoints" - context["custom_breadcrumb"] = OrderedDict( - [ - ("Endpoints", reverse("vulnerable_endpoints")), - (endpoint, reverse("view_endpoint", args=(endpoint.id,))), - ], - ) - # Show the "All findings" breadcrumb if nothing is coming from the product or engagement - elif not self.get_engagement_id() and not self.get_product_id(): - add_breadcrumb(title="Findings", top_level=not len(request.GET), request=request) - - return request, context - - def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): - # Store the product, engagement, and test ids - self.product_id = product_id - self.engagement_id = engagement_id - self.test_id = test_id - # Get the initial context - request, context = self.get_initial_context(request) - # Get the filtered findings - filtered_findings = self.get_fully_filtered_findings(request) - # trick to prefetch after paging to avoid huge join generated by select count(*) from Paginator - paged_findings = get_page_items(request, filtered_findings.qs, 25) - # prefetch the related objects in the findings - paged_findings.object_list = prefetch_for_findings( - paged_findings.object_list, - self.get_prefetch_type()) - # Add some breadcrumbs - request, context = self.add_breadcrumbs(request, context) - # Add the filtered and paged findings into the context - context |= { - "findings": paged_findings, - "filtered": filtered_findings, - } - # Render the view - return render(request, self.get_template(), context) - - -class ListOpenFindings(ListFindings): - def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): - self.filter_name = "Open" - return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) - - -class ListVerifiedFindings(ListFindings): - def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): - self.filter_name = "Verified" - return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) - - -class ListOutOfScopeFindings(ListFindings): - def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): - self.filter_name = "Out of Scope" - return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) - - -class ListFalsePositiveFindings(ListFindings): - def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): - self.filter_name = "False Positive" - return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) - - -class ListInactiveFindings(ListFindings): - def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): - self.filter_name = "Inactive" - return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) - - -class ListAcceptedFindings(ListFindings): - def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): - self.filter_name = "Accepted" - return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) - - -class ListClosedFindings(ListFindings): - def get(self, request: HttpRequest, product_id: int | None = None, engagement_id: int | None = None, test_id: int | None = None): - self.filter_name = "Closed" - self.order_by = "-mitigated" - return super().get(request, product_id=product_id, engagement_id=engagement_id, test_id=test_id) - - -class ViewFinding(View): - def get_finding(self, finding_id: int): - finding_qs = prefetch_for_findings(Finding.objects.filter(id=finding_id), exclude_untouched=False) - return get_object_or_404(finding_qs, id=finding_id) - - def get_dojo_user(self, request: HttpRequest): - user = request.user - return get_object_or_404(Dojo_User, id=user.id) - - def get_previous_and_next_findings(self, finding: Finding): - # Get the whole list of findings in the current test - findings = ( - Finding.objects.filter(test=finding.test) - .order_by("numerical_severity") - .values_list("id", flat=True) - ) - logger.debug(findings) - # Set some reasonable defaults - next_finding_id = finding.id - prev_finding_id = finding.id - last_pos = (len(findings)) - 1 - # get the index of the current finding - current_finding_index = list(findings).index(finding.id) - # Try to get the previous ID - with contextlib.suppress(IndexError, ValueError): - prev_finding_id = findings[current_finding_index - 1] - # Try to get the next ID - with contextlib.suppress(IndexError, ValueError): - next_finding_id = findings[current_finding_index + 1] - - return { - "prev_finding_id": prev_finding_id, - "next_finding_id": next_finding_id, - "findings_list": findings, - "findings_list_lastElement": findings[last_pos], - } - - def get_request_response(self, finding: Finding): - request_response = None - burp_request = None - burp_response = None - try: - request_response = BurpRawRequestResponse.objects.filter(finding=finding).first() - if request_response is not None: - burp_request = base64.b64decode(request_response.burpRequestBase64) - burp_response = base64.b64decode(request_response.burpResponseBase64) - except Exception as e: - logger.debug("unsuspected error: %s", e) - - return { - "burp_request": burp_request, - "burp_response": burp_response, - } - - def get_test_import_data(self, request: HttpRequest, finding: Finding): - test_imports = Test_Import.objects.filter(findings_affected=finding) - test_import_filter = TestImportFilter(request.GET, test_imports) - - test_import_finding_actions = finding.test_import_finding_action_set - test_import_finding_actions_count = test_import_finding_actions.all().count() - test_import_finding_actions = test_import_finding_actions.filter(test_import__in=test_import_filter.qs) - test_import_finding_action_filter = TestImportFindingActionFilter(request.GET, test_import_finding_actions) - - paged_test_import_finding_actions = get_page_items_and_count(request, test_import_finding_action_filter.qs, 5, prefix="test_import_finding_actions") - paged_test_import_finding_actions.object_list = paged_test_import_finding_actions.object_list.prefetch_related("test_import") - - latest_test_import_finding_action = finding.test_import_finding_action_set.order_by("-created").first - - return { - "test_import_filter": test_import_filter, - "test_import_finding_action_filter": test_import_finding_action_filter, - "paged_test_import_finding_actions": paged_test_import_finding_actions, - "latest_test_import_finding_action": latest_test_import_finding_action, - "test_import_finding_actions_count": test_import_finding_actions_count, - } - - def get_similar_findings(self, request: HttpRequest, finding: Finding): - similar_findings_enabled = get_system_setting("enable_similar_findings", True) - if similar_findings_enabled is False: - return { - "similar_findings_enabled": similar_findings_enabled, - "duplicate_cluster": duplicate_cluster(request, finding), - "similar_findings": None, - "similar_findings_filter": None, - } - # add related actions for non-similar and non-duplicate cluster members - finding.related_actions = calculate_possible_related_actions_for_similar_finding( - request, finding, finding, - ) - if finding.duplicate_finding: - finding.duplicate_finding.related_actions = ( - calculate_possible_related_actions_for_similar_finding( - request, finding, finding.duplicate_finding, - ) - ) - filter_string_matching = get_system_setting("filter_string_matching", False) - finding_filter_class = SimilarFindingFilterWithoutObjectLookups if filter_string_matching else SimilarFindingFilter - similar_findings_filter = finding_filter_class( - request.GET, - queryset=get_authorized_findings("view") - .filter(test__engagement__product=finding.test.engagement.product) - .exclude(id=finding.id), - user=request.user, - finding=finding, - ) - logger.debug("similar query: %s", similar_findings_filter.qs.query) - similar_findings = get_page_items( - request, - similar_findings_filter.qs, - settings.SIMILAR_FINDINGS_MAX_RESULTS, - prefix="similar", - ) - similar_findings.object_list = prefetch_for_similar_findings( - similar_findings.object_list, - ) - for similar_finding in similar_findings: - similar_finding.related_actions = ( - calculate_possible_related_actions_for_similar_finding( - request, finding, similar_finding, - ) - ) - - return { - "similar_findings_enabled": similar_findings_enabled, - "duplicate_cluster": duplicate_cluster(request, finding), - "similar_findings": similar_findings, - "similar_findings_filter": similar_findings_filter, - } - - def get_jira_data(self, finding: Finding): - ( - can_be_pushed_to_jira, - can_be_pushed_to_jira_error, - error_code, - ) = jira_services.can_be_pushed(finding) - # Check the error code - if error_code: - logger.debug(error_code) - - return { - "can_be_pushed_to_jira": can_be_pushed_to_jira, - "can_be_pushed_to_jira_error": can_be_pushed_to_jira_error, - } - - def get_note_form(self, request: HttpRequest): - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = {} - - return NoteForm(*args, **kwargs) - - def get_typed_note_form(self, request: HttpRequest, context: dict): - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = { - "available_note_types": context.get("available_note_types"), - } - - return TypedNoteForm(*args, **kwargs) - - def get_form(self, request: HttpRequest, context: dict): - return ( - self.get_typed_note_form(request, context) - if context.get("note_type_activation") - else self.get_note_form(request) - ) - - def process_form(self, request: HttpRequest, finding: Finding, context: dict): - if context["form"].is_valid(): - # Create the note object - new_note = context["form"].save(commit=False) - new_note.author = request.user - new_note.date = timezone.now() - new_note.save() - # Add an entry to the note history - history = NoteHistory( - data=new_note.entry, time=new_note.date, current_editor=new_note.author, - ) - history.save() - new_note.history.add(history) - # Associate the note with the finding - finding.notes.add(new_note) - finding.last_reviewed = new_note.date - finding.last_reviewed_by = context["user"] - finding.save() - # Determine if the note should be sent to jira - if finding.has_jira_issue: - jira_services.add_comment(finding, new_note) - elif finding.has_jira_group_issue: - jira_services.add_comment(finding.finding_group, new_note) - # Send the notification of the note being added - url = request.build_absolute_uri( - reverse("view_finding", args=(finding.id,)), - ) - title = f"Finding: {finding.title}" - process_tag_notifications(request, new_note, url, title) - # Add a message to the request - messages.add_message( - request, messages.SUCCESS, "Note saved.", extra_tags="alert-success", - ) - - return request, True - - return request, False - - def get_initial_context(self, request: HttpRequest, finding: Finding, user: Dojo_User): - notes = finding.notes.all() - note_type_activation = Note_Type.objects.filter(is_active=True).count() - available_note_types = None - if note_type_activation: - available_note_types = find_available_notetypes(notes) - # Set the current context - context = { - "finding": finding, - "dojo_user": user, - "user": request.user, - "notes": notes, - "files": finding.files.all(), - "note_type_activation": note_type_activation, - "available_note_types": available_note_types, - "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), - "product_tab": Product_Tab( - finding.test.engagement.product, title="View Finding", tab="findings", - ), - } - # Set the form using the context, and then update the context - form = self.get_form(request, context) - context["form"] = form - - return context - - def get_template(self): - return "dojo/view_finding.html" - - def get(self, request: HttpRequest, finding_id: int): - # Get the initial objects - finding = self.get_finding(finding_id) - user = self.get_dojo_user(request) - # Make sure the user is authorized - user_has_permission_or_403(user, finding, "view") - # Set up the initial context - context = self.get_initial_context(request, finding, user) - # Add in the other extras - context |= self.get_previous_and_next_findings(finding) - # Add in more of the other extras - context |= self.get_request_response(finding) - context |= self.get_similar_findings(request, finding) - context |= self.get_test_import_data(request, finding) - context |= self.get_jira_data(finding) - # Render the form - return render(request, self.get_template(), context) - - def post(self, request: HttpRequest, finding_id): - # Get the initial objects - finding = self.get_finding(finding_id) - user = self.get_dojo_user(request) - # Make sure the user is authorized - user_has_permission_or_403(user, finding, "view") - # Quick perms check to determine if the user has access to add a note to the finding - user_has_permission_or_403(user, finding, "add") - # Set up the initial context - context = self.get_initial_context(request, finding, user) - # Determine the validity of the form - request, success = self.process_form(request, finding, context) - # Handle the case of a successful form - if success: - return HttpResponseRedirect(reverse("view_finding", args=(finding_id,))) - # Add in more of the other extras - context |= self.get_request_response(finding) - context |= self.get_similar_findings(request, finding) - context |= self.get_test_import_data(request, finding) - context |= self.get_jira_data(finding) - # Render the form - return render(request, self.get_template(), context) - - -class EditFinding(View): - def get_finding(self, finding_id: int): - return get_object_or_404(Finding, id=finding_id) - - def get_request_response(self, finding: Finding): - req_resp = None - if burp_rr := BurpRawRequestResponse.objects.filter(finding=finding).first(): - req_resp = (burp_rr.get_request(), burp_rr.get_response()) - - return req_resp - - def get_finding_form(self, request: HttpRequest, finding: Finding): - # Get the burp request if available - req_resp = self.get_request_response(finding) - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = { - "instance": finding, - "req_resp": req_resp, - "can_edit_mitigated_data": finding_helper.can_edit_mitigated_data(request.user), - "initial": {"vulnerability_ids": "\n".join(finding.vulnerability_ids)}, - } - - return FindingForm(*args, **kwargs) - - def get_jira_form(self, request: HttpRequest, finding: Finding, finding_form: FindingForm = None): - # Determine if jira should be used - if (jira_project := jira_services.get_project(finding)) is not None: - # Determine if push all findings is enabled - push_all_findings = jira_services.is_push_all_issues(finding) - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = { - "push_all": push_all_findings, - "prefix": "jiraform", - "instance": finding, - "jira_project": jira_project, - "finding_form": finding_form, - } - - return JIRAFindingForm(*args, **kwargs) - return None - - def get_github_form(self, request: HttpRequest, finding: Finding): - # Determine if github should be used - if get_system_setting("enable_github"): - # Ensure there is a github conf correctly configured for the product - config_present = GITHUB_PKey.objects.filter(product=finding.test.engagement.product) - if config_present := config_present.exclude(git_conf_id=None): - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = { - "enabled": finding.has_github_issue(), - "prefix": "githubform", - } - - return GITHUBFindingForm(*args, **kwargs) - return None - - def get_initial_context(self, request: HttpRequest, finding: Finding): - # Get the finding form first since it is used in another place - finding_form = self.get_finding_form(request, finding) - return { - "form": finding_form, - "finding": finding, - "jform": self.get_jira_form(request, finding, finding_form=finding_form), - "gform": self.get_github_form(request, finding), - "return_url": get_return_url(request), - "product_tab": Product_Tab( - finding.test.engagement.product, title="Edit Finding", tab="findings", - ), - } - - def validate_status_change(self, request: HttpRequest, finding: Finding, context: dict): - # If the finding is already not active, skip this extra validation - if not finding.active: - return request - # Validate the proper notes are added for mitigation - if (not context["form"]["active"].value() or context["form"]["false_p"].value() or context["form"]["out_of_scope"].value()) and not context["form"]["duplicate"].value(): - note_type_activation = Note_Type.objects.filter(is_active=True).count() - closing_disabled = 0 - if note_type_activation: - closing_disabled = len(get_missing_mandatory_notetypes(finding)) - if closing_disabled != 0: - error_inactive = ValidationError( - "Can not set a finding as inactive without adding all mandatory notes", - code="inactive_without_mandatory_notes", - ) - error_false_p = ValidationError( - "Can not set a finding as false positive without adding all mandatory notes", - code="false_p_without_mandatory_notes", - ) - error_out_of_scope = ValidationError( - "Can not set a finding as out of scope without adding all mandatory notes", - code="out_of_scope_without_mandatory_notes", - ) - if context["form"]["active"].value() is False: - context["form"].add_error("active", error_inactive) - if context["form"]["false_p"].value(): - context["form"].add_error("false_p", error_false_p) - if context["form"]["out_of_scope"].value(): - context["form"].add_error("out_of_scope", error_out_of_scope) - messages.add_message( - request, - messages.ERROR, - ("Can not set a finding as inactive, " - "false positive or out of scope without adding all mandatory notes"), - extra_tags="alert-danger", - ) - - return request - - def process_mitigated_data(self, request: HttpRequest, finding: Finding, context: dict): - # If active is not checked and CAN_EDIT_MITIGATED_DATA, - # mitigate the finding and the associated endpoints status - if finding_helper.can_edit_mitigated_data(request.user) and (( - context["form"]["active"].value() is False - or context["form"]["false_p"].value() - or context["form"]["out_of_scope"].value() - ) and context["form"]["duplicate"].value() is False): - now = timezone.now() - finding.is_mitigated = True - - if settings.V3_FEATURE_LOCATIONS: - for ref in finding.locations.all(): - ref.set_status( - FindingLocationStatus.Mitigated, - context["form"].cleaned_data.get("mitigated_by") or request.user, - context["form"].cleaned_data.get("mitigated") or now, - ) - else: - # TODO: Delete this after the move to Locations - endpoint_status = finding.status_finding.all() - for status in endpoint_status: - status.mitigated_by = ( - context["form"].cleaned_data.get("mitigated_by") or request.user - ) - status.mitigated_time = ( - context["form"].cleaned_data.get("mitigated") or now - ) - status.mitigated = True - status.last_modified = timezone.now() - status.save() - - def process_false_positive_history(self, finding: Finding, *, old_false_p: bool = False): - if get_system_setting("false_positive_history", False): - # If the finding is being marked as a false positive we dont need to call the - # fp history function because it will be called by the save function. - # If finding was a false positive and is being reactivated: retroactively reactivates all equal findings. - # old_false_p must be captured before form.save(commit=False) mutates the finding in place. - if old_false_p and not finding.false_p and get_system_setting("retroactive_false_positive_history"): - logger.debug("FALSE_POSITIVE_HISTORY: Reactivating existing findings based on: %s", finding) - # QuerySet.update() bypasses Django signals, which is intentional here — it mirrors - # the previous save_no_options() calls that also disabled all post-save processing. - # match_finding_to_existing_findings returns a lazy QS with no .only() applied, - # so any field can be added here without needing a corresponding .only() change in deduplication.py#_fetch_fp_candidates_for_batch. - match_finding_to_existing_findings( - finding, product=finding.test.engagement.product, - ).filter(false_p=True).update( - false_p=False, - active=finding.active, - verified=finding.verified, - out_of_scope=finding.out_of_scope, - is_mitigated=finding.is_mitigated, - ) - - def process_burp_request_response(self, finding: Finding, context: dict): - if "request" in context["form"].cleaned_data or "response" in context["form"].cleaned_data: - try: - burp_rr, _ = BurpRawRequestResponse.objects.get_or_create(finding=finding) - except BurpRawRequestResponse.MultipleObjectsReturned: - burp_rr = BurpRawRequestResponse.objects.filter(finding=finding).first() - burp_rr.burpRequestBase64 = base64.b64encode( - context["form"].cleaned_data["request"].encode(), - ) - burp_rr.burpResponseBase64 = base64.b64encode( - context["form"].cleaned_data["response"].encode(), - ) - burp_rr.clean() - burp_rr.save() - - def process_finding_form(self, request: HttpRequest, finding: Finding, context: dict): - if context["form"].is_valid(): - # process some of the easy stuff first - # Capture false_p before form.save(commit=False) mutates the finding in place, - # so process_false_positive_history can detect a false-positive → active transition. - old_false_p = finding.false_p - new_finding = context["form"].save(commit=False) - new_finding.test = finding.test - new_finding.numerical_severity = Finding.get_numerical_severity(new_finding.severity) - new_finding.last_reviewed = timezone.now() - new_finding.last_reviewed_by = request.user - new_finding.tags = context["form"].cleaned_data["tags"] - # Handle group related things - if "group" in context["form"].cleaned_data: - finding_group = context["form"].cleaned_data["group"] - finding_helper.update_finding_group(new_finding, finding_group) - # Handle risk exception related things - if "risk_accepted" in context["form"].cleaned_data and context["form"]["risk_accepted"].value(): - if new_finding.test.engagement.product.enable_simple_risk_acceptance: - ra_helper.simple_risk_accept(request.user, new_finding, perform_save=False) - elif new_finding.risk_accepted: - ra_helper.risk_unaccept(request.user, new_finding, perform_save=False) - # Save and add new locations; replace=True so deselected endpoints are removed - associated_locations = finding_helper.add_locations(new_finding, context["form"], replace=True) - # Remove unrelated endpoints - if settings.V3_FEATURE_LOCATIONS: - for ref in new_finding.locations.all(): - if ref.location not in associated_locations: - ref.location.disassociate_from_finding(new_finding) - else: - # TODO: Delete this after the move to Locations - endpoint_status_list = Endpoint_Status.objects.filter(finding=new_finding) - for endpoint_status in endpoint_status_list: - if endpoint_status.endpoint not in new_finding.endpoints.all(): - endpoint_status.delete() - # Handle some of the other steps - self.process_mitigated_data(request, new_finding, context) - self.process_false_positive_history(new_finding, old_false_p=old_false_p) - self.process_burp_request_response(new_finding, context) - # Save the vulnerability IDs - finding_helper.save_vulnerability_ids(new_finding, context["form"].cleaned_data["vulnerability_ids"].split()) - # Add a success message - messages.add_message( - request, - messages.SUCCESS, - "Finding saved successfully.", - extra_tags="alert-success", - ) - - return finding, request, True - add_error_message_to_response("The form has errors, please correct them below.") - add_field_errors_to_response(context["form"]) - - return finding, request, False - - def process_jira_form(self, request: HttpRequest, finding: Finding, context: dict): - # Capture case if the jira not being enabled - if context["jform"] is None: - return request, True, False - - if context["jform"] and context["jform"].is_valid(): - jira_message = None - logger.debug("jform.jira_issue: %s", context["jform"].cleaned_data.get("jira_issue")) - logger.debug(JFORM_PUSH_TO_JIRA_MESSAGE, context["jform"].cleaned_data.get("push_to_jira")) - # can't use helper as when push_all_jira_issues is True, the checkbox gets disabled and is always false - push_to_jira_checkbox = context["jform"].cleaned_data.get("push_to_jira") - push_all_jira_issues = jira_services.is_push_all_issues(finding) - push_to_jira = push_all_jira_issues or push_to_jira_checkbox or jira_services.is_keep_in_sync(finding) - logger.debug("push_to_jira: %s", push_to_jira) - logger.debug("push_all_jira_issues: %s", push_all_jira_issues) - logger.debug("has_jira_group_issue: %s", finding.has_jira_group_issue) - # if the jira issue key was changed, update database - new_jira_issue_key = context["jform"].cleaned_data.get("jira_issue") - # we only support linking / changing if there is no group issue - if not finding.has_jira_group_issue: - if finding.has_jira_issue: - """ - everything in DD around JIRA integration is based on the internal id - of the issue in JIRA instead of on the public jira issue key. - I have no idea why, but it means we have to retrieve the issue from JIRA - to get the internal JIRA id. we can assume the issue exist, - which is already checked in the validation of the form - """ - if not new_jira_issue_key: - jira_services.unlink_finding(request, finding) - jira_message = "Link to JIRA issue removed successfully." - elif new_jira_issue_key != finding.jira_issue.jira_key: - jira_services.unlink_finding(request, finding) - jira_services.link_finding(request, finding, new_jira_issue_key) - jira_message = "Changed JIRA link successfully." - elif new_jira_issue_key: - jira_services.link_finding(request, finding, new_jira_issue_key) - jira_message = "Linked a JIRA issue successfully." - # any existing finding should be updated - # Determine if a message should be added - if jira_message: - messages.add_message( - request, messages.SUCCESS, jira_message, extra_tags="alert-success", - ) - - return request, True, push_to_jira - add_field_errors_to_response(context["jform"]) - - return request, False, False - - def process_github_form(self, request: HttpRequest, finding: Finding, context: dict, old_status: str): - if "githubform-push_to_github" not in request.POST: - return request, True - - if context["gform"].is_valid(): - if GITHUB_Issue.objects.filter(finding=finding).exists(): - update_external_issue(finding.id, old_status, "github") - else: - add_external_issue(finding.id, "github") - - return request, True - add_field_errors_to_response(context["gform"]) - - return request, False - - def process_forms(self, request: HttpRequest, finding: Finding, context: dict): - form_success_list = [] - # Set vars for the completed forms - old_status = finding.status() - old_finding = copy.copy(finding) - # Validate finding mitigation - request = self.validate_status_change(request, finding, context) - # Check the validity of the form overall - new_finding, request, success = self.process_finding_form(request, finding, context) - form_success_list.append(success) - request, success, push_to_jira = self.process_jira_form(request, new_finding, context) - form_success_list.append(success) - request, success = self.process_github_form(request, new_finding, context, old_status) - form_success_list.append(success) - # Determine if all forms were successful - all_forms_valid = all(form_success_list) - # Check the validity of all the forms - if all_forms_valid: - # if we're removing the "duplicate" in the edit finding screen - # do not relaunch deduplication, otherwise, it's never taken into account - if old_finding.duplicate and not new_finding.duplicate: - new_finding.duplicate_finding = None - new_finding.save(push_to_jira=push_to_jira, dedupe_option=False) - else: - new_finding.save(push_to_jira=push_to_jira) - # we only push the group after storing the finding to make sure - # the updated data of the finding is pushed as part of the group - if push_to_jira and finding.finding_group: - jira_services.push(finding.finding_group) - - return request, all_forms_valid - - def get_template(self): - return "dojo/edit_finding.html" - - def get(self, request: HttpRequest, finding_id: int): - # Get the initial objects - finding = self.get_finding(finding_id) - # Make sure the user is authorized - user_has_permission_or_403(request.user, finding, "edit") - # Set up the initial context - context = self.get_initial_context(request, finding) - # Render the form - return render(request, self.get_template(), context) - - def post(self, request: HttpRequest, finding_id: int): - # Get the initial objects - finding = self.get_finding(finding_id) - # Make sure the user is authorized - user_has_permission_or_403(request.user, finding, "edit") - # Set up the initial context - context = self.get_initial_context(request, finding) - # Process the form - request, success = self.process_forms(request, finding, context) - # Handle the case of a successful form - if success: - return redirect_to_return_url_or_else(request, reverse("view_finding", args=(finding_id,))) - # Render the form - return render(request, self.get_template(), context) - - -class DeleteFinding(View): - def get_finding(self, finding_id: int): - return get_object_or_404(Finding, id=finding_id) - - def process_form(self, request: HttpRequest, finding: Finding, context: dict): - if context["form"].is_valid(): - product = finding.test.engagement.product - finding.delete() - # Update the grade of the product async - dojo_dispatch_task(calculate_grade, product.id) - # Add a message to the request that the finding was successfully deleted - messages.add_message( - request, - messages.SUCCESS, - "Finding deleted successfully.", - extra_tags="alert-success", - ) - - # Note: this notification has not be moved to "@receiver(post_delete, sender=Finding)" method as many other notifications - # Because it could generate too much noise, we keep it here only for findings created by hand in WebUI - # TODO: but same should be implemented for API endpoint - - # Send a notification that the finding had been deleted - create_notification( - event="finding_deleted", - title=f"Deletion of {finding.title}", - description=f'The finding "{finding.title}" was deleted by {request.user}', - product=product, - url=request.build_absolute_uri(reverse("all_findings")), - recipients=[finding.test.engagement.lead], - icon="exclamation-triangle", - ) - # return the request - return request, True - - # Add a failure message - messages.add_message( - request, - messages.ERROR, - "Unable to delete finding, please try again.", - extra_tags="alert-danger", - ) - - return request, False - - def post(self, request: HttpRequest, finding_id): - # Get the initial objects - finding = self.get_finding(finding_id) - # Make sure the user is authorized - user_has_permission_or_403(request.user, finding, "delete") - # Get the finding form - context = { - "form": DeleteFindingForm(request.POST, instance=finding), - } - # Process the form - request, success = self.process_form(request, finding, context) - # Handle the case of a successful form - if success: - return redirect_to_return_url_or_else(request, reverse("view_test", args=(finding.test.id,))) - raise PermissionDenied - - -def close_finding(request, fid): - finding = get_object_or_404(Finding, id=fid) - # in order to close a finding, we need to capture why it was closed - # we can do this with a Note - note_type_activation = Note_Type.objects.filter(is_active=True) - missing_note_types = get_missing_mandatory_notetypes(finding) if len(note_type_activation) else note_type_activation - form = CloseFindingForm( - instance=finding, - missing_note_types=missing_note_types, - can_edit_mitigated_data=finding_helper.can_edit_mitigated_data(request.user), - ) - if request.method == "POST": - form = CloseFindingForm( - request.POST, - instance=finding, - missing_note_types=missing_note_types, - can_edit_mitigated_data=finding_helper.can_edit_mitigated_data(request.user), - ) - - if form.is_valid(): - messages.add_message(request, messages.SUCCESS, "Note Saved.", extra_tags="alert-success") - - if len(missing_note_types) <= 1: - finding_helper.close_finding( - finding=finding, - user=request.user, - is_mitigated=True, - mitigated=form.cleaned_data.get("mitigated"), - mitigated_by=form.cleaned_data.get("mitigated_by") or request.user, - false_p=form.cleaned_data.get("false_p", False), - out_of_scope=form.cleaned_data.get("out_of_scope", False), - duplicate=form.cleaned_data.get("duplicate", False), - note_entry=form.cleaned_data.get("entry"), - note_type=form.cleaned_data.get("note_type"), - ) - - messages.add_message( - request, - messages.SUCCESS, - "Finding closed.", - extra_tags="alert-success", - ) - - # Notification sent by helper - return HttpResponseRedirect( - reverse("view_test", args=(finding.test.id,)), - ) - return HttpResponseRedirect( - reverse("close_finding", args=(finding.id,)), - ) - - product_tab = Product_Tab( - finding.test.engagement.product, title="Close", tab="findings", - ) - - return render( - request, - "dojo/close_finding.html", - { - "finding": finding, - "product_tab": product_tab, - "active_tab": "findings", - "user": request.user, - "form": form, - "note_types": missing_note_types, - }, - ) - - -def verify_finding(request, fid): - finding = get_object_or_404(Finding, id=fid) - - if finding.verified: - messages.add_message( - request, - messages.INFO, - "Finding already verified.", - extra_tags="alert-info", - ) - return redirect_to_return_url_or_else( - request, - reverse("view_finding", args=(finding.id,)), - ) - - form = NoteForm(data=request.POST or None) - form.fields["entry"].required = False - form.fields["entry"].label = _("Comment (optional)") - - if request.method == "POST" and form.is_valid(): - entry = form.cleaned_data.get("entry", "") - finding_helper.verify_finding( - finding=finding, - user=request.user, - note_entry=entry, - ) - - messages.add_message( - request, - messages.SUCCESS, - "Finding verified.", - extra_tags="alert-success", - ) - - return redirect_to_return_url_or_else( - request, - reverse("view_finding", args=(finding.id,)), - ) - - product_tab = Product_Tab( - finding.test.engagement.product, - title="Verify Finding", - tab="findings", - ) - - return render( - request, - "dojo/verify_finding.html", - { - "finding": finding, - "product_tab": product_tab, - "user": request.user, - "form": form, - "active_tab": "findings", - }, - ) - - -def defect_finding_review(request, fid): - finding = get_object_or_404(Finding, id=fid) - # in order to close a finding, we need to capture why it was closed - # we can do this with a Note - if request.method == "POST": - form = DefectFindingForm(request.POST) - if form.is_valid(): - now = timezone.now() - new_note = form.save(commit=False) - new_note.author = request.user - new_note.date = now - new_note.save() - finding.notes.add(new_note) - finding.under_review = False - defect_choice = form.cleaned_data["defect_choice"] - - if defect_choice == "Close Finding": - finding.active = False - finding.verified = True - finding.mitigated = now - finding.mitigated_by = request.user - finding.is_mitigated = True - finding.last_reviewed = finding.mitigated - finding.last_reviewed_by = request.user - finding.endpoints.clear() - else: - finding.active = True - finding.verified = True - finding.mitigated = None - finding.mitigated_by = None - finding.is_mitigated = False - finding.last_reviewed = now - finding.last_reviewed_by = request.user - - # Manage the jira status changes - push_to_jira = False - # Determine if the finding is in a group. if so, not push to jira - finding_in_group = finding.has_finding_group - # Check if there is a jira issue that needs to be updated - jira_issue_exists = finding.has_jira_issue or (finding.finding_group and finding.finding_group.has_jira_issue) - # Only push if the finding is not in a group - if jira_issue_exists: - # Determine if any automatic sync should occur - jira_instance = jira_services.get_instance(finding) - push_to_jira = jira_services.is_push_all_issues(finding) \ - or (jira_instance and jira_instance.finding_jira_sync) - # Add the closing note - if push_to_jira and not finding_in_group: - if defect_choice == "Close Finding": - new_note.entry += "\nJira issue set to resolved." - else: - new_note.entry += "\nJira issue re-opened." - jira_services.add_comment(finding, new_note, force_push=True) - # Save the finding - finding.save(push_to_jira=(push_to_jira and not finding_in_group)) - - # we only push the group after saving the finding to make sure - # the updated data of the finding is pushed as part of the group - if push_to_jira and finding_in_group: - jira_services.push(finding.finding_group) - - messages.add_message( - request, messages.SUCCESS, "Defect Reviewed", extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_test", args=(finding.test.id,))) - - else: - form = DefectFindingForm() - - product_tab = Product_Tab( - finding.test.engagement.product, title="Jira Status Review", tab="findings", - ) - - return render( - request, - "dojo/defect_finding_review.html", - { - "finding": finding, - "product_tab": product_tab, - "user": request.user, - "form": form, - }, - ) - - -def reopen_finding(request, fid): - finding = get_object_or_404(Finding, id=fid) - finding.active = True - finding.mitigated = None - finding.mitigated_by = request.user - finding.is_mitigated = False - finding.last_reviewed = finding.mitigated - finding.last_reviewed_by = request.user - finding.under_review = False - if settings.V3_FEATURE_LOCATIONS: - for ref in finding.locations.all(): - ref.set_status(FindingLocationStatus.Active, request.user, timezone.now()) - else: - endpoint_status = finding.status_finding.all() - for status in endpoint_status: - status.mitigated_by = None - status.mitigated_time = None - status.mitigated = False - status.last_modified = timezone.now() - status.save() - # Clear the risk acceptance, if present - ra_helper.risk_unaccept(request.user, finding) - finding.save(dedupe_option=False, push_to_jira=False) - if jira_services.is_push_all_issues(finding) or jira_services.is_keep_in_sync(finding): - jira_services.push(finding) - - reopen_external_issue(finding.id, "re-opened by defectdojo", "github") - - messages.add_message( - request, messages.SUCCESS, "Finding Reopened.", extra_tags="alert-success", - ) - - # Note: this notification has not be moved to "@receiver(pre_save, sender=Finding)" method as many other notifications - # Because it could generate too much noise, we keep it here only for findings created by hand in WebUI - # TODO: but same should be implemented for API endpoint - - create_notification( - event="finding_reopened", - title=_("Reopening of %s") % finding.title, - finding=finding, - description=f'The finding "{finding.title}" was reopened by {request.user}', - url=reverse("view_finding", args=(finding.id,)), - ) - return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) - - -def copy_finding(request, fid): - finding = get_object_or_404(Finding, id=fid) - product = finding.test.engagement.product - tests = get_authorized_tests("edit").filter( - engagement=finding.test.engagement, - ) - form = CopyFindingForm(tests=tests) - - if request.method == "POST": - form = CopyFindingForm(request.POST, tests=tests) - if form.is_valid(): - test = form.cleaned_data.get("test") - product = finding.test.engagement.product - finding_copy = finding.copy(test=test) - dojo_dispatch_task(calculate_grade, product.id) - messages.add_message( - request, - messages.SUCCESS, - "Finding Copied successfully.", - extra_tags="alert-success", - ) - create_notification( - event="finding_copied", # TODO: - if 'copy' functionality will be supported by API as well, 'create_notification' needs to be migrated to place where it will be able to cover actions from both interfaces - title=_("Copying of %s") % finding.title, - description=f'The finding "{finding.title}" was copied by {request.user} to {test.title}', - product=product, - url=request.build_absolute_uri( - reverse("copy_finding", args=(finding_copy.id,)), - ), - recipients=[finding.test.engagement.lead], - icon="exclamation-triangle", - ) - return redirect_to_return_url_or_else( - request, reverse("view_test", args=(test.id,)), - ) - messages.add_message( - request, - messages.ERROR, - "Unable to copy finding, please try again.", - extra_tags="alert-danger", - ) - - product_tab = Product_Tab(product, title="Copy Finding", tab="findings") - return render( - request, - "dojo/copy_object.html", - { - "source": finding, - "source_label": "Finding", - "destination_label": "Test", - "product_tab": product_tab, - "form": form, - }, - ) - - -def remediation_date(request, fid): - finding = get_object_or_404(Finding, id=fid) - user = get_object_or_404(Dojo_User, id=request.user.id) - - if request.method == "POST": - form = EditPlannedRemediationDateFindingForm(request.POST) - - if form.is_valid(): - finding.planned_remediation_date = request.POST.get( - "planned_remediation_date", "", - ) - finding.save() - messages.add_message( - request, - messages.SUCCESS, - "Finding Planned Remediation Date saved.", - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) - - else: - form = EditPlannedRemediationDateFindingForm(finding=finding) - - product_tab = Product_Tab( - finding.test.engagement.product, - title="Planned Remediation Date", - tab="findings", - ) - - return render( - request, - "dojo/remediation_date.html", - {"finding": finding, "product_tab": product_tab, "user": user, "form": form}, - ) - - -def touch_finding(request, fid): - finding = get_object_or_404(Finding, id=fid) - finding.last_reviewed = timezone.now() - finding.last_reviewed_by = request.user - finding.save() - return redirect_to_return_url_or_else( - request, reverse("view_finding", args=(finding.id,)), - ) - - -def simple_risk_accept(request, fid): - finding = get_object_or_404(Finding, id=fid) - - if not finding.test.engagement.product.enable_simple_risk_acceptance: - raise PermissionDenied - - ra_helper.simple_risk_accept(request.user, finding) - - messages.add_message( - request, messages.WARNING, "Finding risk accepted.", extra_tags="alert-success", - ) - - return redirect_to_return_url_or_else( - request, reverse("view_finding", args=(finding.id,)), - ) - - -def risk_unaccept(request, fid): - finding = get_object_or_404(Finding, id=fid) - ra_helper.risk_unaccept(request.user, finding) - - messages.add_message( - request, - messages.WARNING, - "Finding risk unaccepted.", - extra_tags="alert-success", - ) - - return redirect_to_return_url_or_else( - request, reverse("view_finding", args=(finding.id,)), - ) - - -def request_finding_review(request, fid): - finding = get_object_or_404(Finding, id=fid) - user = get_object_or_404(Dojo_User, id=request.user.id) - form = ReviewFindingForm(finding=finding, user=user) - # in order to review a finding, we need to capture why a review is needed - # we can do this with a Note - if request.method == "POST": - form = ReviewFindingForm(request.POST, finding=finding, user=user) - - if form.is_valid(): - now = timezone.now() - new_note = Notes() - new_note.entry = "Review Request: " + form.cleaned_data["entry"] - new_note.private = True - new_note.author = request.user - new_note.date = now - new_note.save() - finding.notes.add(new_note) - finding.active = True - finding.verified = False - finding.is_mitigated = False - finding.under_review = True - finding.review_requested_by = user - finding.last_reviewed = now - finding.last_reviewed_by = request.user - - reviewers = form.cleaned_data["reviewers"] - finding.reviewers.set(reviewers) - - # Manage the jira status changes - push_to_jira = False - # Determine if the finding is in a group. if so, not push to jira - finding_in_group = finding.has_finding_group - # Check if there is a jira issue that needs to be updated - jira_issue_exists = finding.has_jira_issue or (finding.finding_group and finding.finding_group.has_jira_issue) - # Only push if the finding is not in a group - if jira_issue_exists: - # Determine if any automatic sync should occur - jira_instance = jira_services.get_instance(finding) - push_to_jira = jira_services.is_push_all_issues(finding) \ - or (jira_instance and jira_instance.finding_jira_sync) - # Add the closing note - if push_to_jira and not finding_in_group: - jira_services.add_comment(finding, new_note, force_push=True) - # Save the finding - finding.save(push_to_jira=(push_to_jira and not finding_in_group)) - - # we only push the group after saving the finding to make sure - # the updated data of the finding is pushed as part of the group - if push_to_jira and finding_in_group: - jira_services.push(finding.finding_group) - - reviewers = Dojo_User.objects.filter(id__in=form.cleaned_data["reviewers"]) - reviewers_string = ", ".join([f"{user} ({user.id})" for user in reviewers]) - reviewers_usernames = [user.username for user in reviewers] - logger.debug("Asking %s for review", reviewers_string) - - create_notification( - event="review_requested", # TODO: - if 'review_requested' functionality will be supported by API as well, 'create_notification' needs to be migrated to place where it will be able to cover actions from both interfaces - title=f"Finding review requested for Test created for {finding.test.engagement.product}: {finding.test.engagement.name}: {finding.test} - {finding.title}", - requested_by=user, - note=new_note, - finding=finding, - reviewers=reviewers, - recipients=reviewers_usernames, - description=f'User {user.get_full_name()}({user.id}) has requested that user(s) {reviewers_string} review the finding "{finding.title}" for accuracy:\n\n{new_note}', - icon="check", - url=reverse("view_finding", args=(finding.id,)), - ) - - messages.add_message( - request, - messages.SUCCESS, - "Finding marked for review and reviewers notified.", - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) - - product_tab = Product_Tab( - finding.test.engagement.product, title="Review Finding", tab="findings", - ) - - return render( - request, - "dojo/review_finding.html", - {"finding": finding, "product_tab": product_tab, "user": user, "form": form, "enable_table_filtering": get_system_setting("enable_ui_table_based_searching")}, - ) - - -def clear_finding_review(request, fid): - finding = get_object_or_404(Finding, id=fid) - user = get_object_or_404(Dojo_User, id=request.user.id) - # If the user wanting to clear the review is not the user who requested - # the review or one of the users requested to provide the review, then - # do not allow the user to clear the review. - if user != finding.review_requested_by and user not in finding.reviewers.all(): - raise PermissionDenied - - # in order to clear a review for a finding, we need to capture why and how it was reviewed - # we can do this with a Note - if request.method == "POST": - form = ClearFindingReviewForm(request.POST, instance=finding) - - if form.is_valid(): - now = timezone.now() - new_note = Notes() - new_note.entry = "Review Cleared: " + form.cleaned_data["entry"] - new_note.author = request.user - new_note.date = now - new_note.save() - - finding = form.save(commit=False) - - if finding.is_mitigated: - finding.mitigated = now - finding.mitigated_by = request.user - finding.under_review = False - finding.last_reviewed = now - finding.last_reviewed_by = request.user - - finding.reviewers.set([]) - finding.notes.add(new_note) - - # Manage the jira status changes - push_to_jira = False - # Determine if the finding is in a group. if so, not push to jira - finding_in_group = finding.has_finding_group - # Check if there is a jira issue that needs to be updated - jira_issue_exists = finding.has_jira_issue or (finding.finding_group and finding.finding_group.has_jira_issue) - # Only push if the finding is not in a group - if jira_issue_exists: - # Determine if any automatic sync should occur - jira_instance = jira_services.get_instance(finding) - push_to_jira = jira_services.is_push_all_issues(finding) \ - or (jira_instance and jira_instance.finding_jira_sync) - # Add the closing note - if push_to_jira and not finding_in_group: - jira_services.add_comment(finding, new_note, force_push=True) - # Save the finding - finding.save(push_to_jira=(push_to_jira and not finding_in_group)) - - # we only push the group after saving the finding to make sure - # the updated data of the finding is pushed as part of the group - if push_to_jira and finding_in_group: - jira_services.push(finding.finding_group) - - messages.add_message( - request, - messages.SUCCESS, - "Finding review has been updated successfully.", - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) - - else: - form = ClearFindingReviewForm(instance=finding) - - product_tab = Product_Tab( - finding.test.engagement.product, title="Clear Finding Review", tab="findings", - ) - - return render( - request, - "dojo/clear_finding_review.html", - {"finding": finding, "product_tab": product_tab, "user": user, "form": form}, - ) - - -def mktemplate(request, fid): - user_has_global_permission_or_403(request.user, "add") - finding = get_object_or_404(Finding, id=fid) - templates = Finding_Template.objects.filter(title=finding.title) - if len(templates) > 0: - messages.add_message( - request, - messages.ERROR, - "A finding template with that title already exists.", - extra_tags="alert-danger", - ) - else: - template = Finding_Template( - title=finding.title, - cwe=finding.cwe, - cvssv3=finding.cvssv3, - cvssv3_score=finding.cvssv3_score, - cvssv4=finding.cvssv4, - cvssv4_score=finding.cvssv4_score, - severity=finding.severity, - description=finding.description, - mitigation=finding.mitigation, - impact=finding.impact, - references=finding.references, - numerical_severity=finding.numerical_severity, - fix_available=finding.fix_available, - fix_version=finding.fix_version, - planned_remediation_version=finding.planned_remediation_version, - effort_for_fixing=finding.effort_for_fixing, - steps_to_reproduce=finding.steps_to_reproduce, - severity_justification=finding.severity_justification, - component_name=finding.component_name, - component_version=finding.component_version, - tags=finding.tags.all(), - ) - template.save() - template.tags = finding.tags.all() - # Ensure template tags exist in Finding's tag model - # (They should already exist since they come from a finding, but ensure for consistency) - ensure_template_tags_in_finding_model(template) - - # Save vulnerability IDs using helper (handles both old and new format) - finding_helper.save_vulnerability_ids_template(template, finding.vulnerability_ids) - - # Copy endpoints if they exist - if finding.endpoints.exists(): - endpoint_urls = [str(ep) for ep in finding.endpoints.all()] - finding_helper.save_endpoints_template(template, endpoint_urls) - - messages.add_message( - request, - messages.SUCCESS, - mark_safe( - 'Finding template added successfully. You may edit it here.'.format(reverse("edit_template", args=(template.id,))), - ), - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) - - -def find_template_to_apply(request, fid): - # Templates may contain sensitive data from any product; require global permission - # to match the authorization level of the /template list view - user_has_global_permission_or_403(request.user, "edit") - finding = get_object_or_404(Finding, id=fid) - test = get_object_or_404(Test, id=finding.test.id) - templates_by_cve = ( - Finding_Template.objects.annotate( - cve_len=Length("cve"), order=models.Value(1, models.IntegerField()), - ) - .filter(cve=finding.cve, cve_len__gt=0) - .order_by("-last_used") - ) - if templates_by_cve.count() == 0: - templates_by_last_used = ( - Finding_Template.objects.all() - .order_by("-last_used") - .annotate( - cve_len=Length("cve"), order=models.Value(2, models.IntegerField()), - ) - ) - templates = templates_by_last_used - else: - templates_by_last_used = ( - Finding_Template.objects.all() - .exclude(cve=finding.cve) - .order_by("-last_used") - .annotate( - cve_len=Length("cve"), order=models.Value(2, models.IntegerField()), - ) - ) - union_queryset = templates_by_last_used.union(templates_by_cve).order_by( - "order", "-last_used", - ) - # Convert union queryset to regular queryset to avoid issues with distinct() in filters - # Get IDs from union queryset and create a new queryset filtered by those IDs - template_ids = list(union_queryset.values_list("id", flat=True)) - templates = Finding_Template.objects.filter(id__in=template_ids).annotate( - cve_len=Length("cve"), - order=Case( - *[When(id=template_id, then=models.Value(i + 1)) for i, template_id in enumerate(template_ids)], - default=models.Value(len(template_ids) + 1), - output_field=models.IntegerField(), - ), - ).order_by("order", "-last_used") - - templates = TemplateFindingFilter(request.GET, queryset=templates) - paged_templates = get_page_items(request, templates.qs, 25) - - # just query all templates as this weird ordering above otherwise breaks Django ORM - title_words = get_words_for_field(Finding_Template, "title") - product_tab = Product_Tab( - test.engagement.product, title="Apply Template to Finding", tab="findings", - ) - return render( - request, - "dojo/templates.html", - { - "templates": paged_templates, - "product_tab": product_tab, - "filtered": templates, - "title_words": title_words, - "tid": test.id, - "fid": fid, - "add_from_template": False, - "apply_template": True, - }, - ) - - -def choose_finding_template_options(request, tid, fid): - finding = get_object_or_404(Finding, id=fid) - user_has_permission_or_403(request.user, finding, "edit") - template = get_object_or_404(Finding_Template, id=tid) - data = finding.__dict__.copy() - # Remove tags and other non-serializable fields - data.pop("tags", None) - data.pop("_state", None) - data.pop("_tags_tagulous", None) - - # Populate from template for fields that exist on template - template_fields = ["cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", - "fix_available", "fix_version", "planned_remediation_version", - "effort_for_fixing", "steps_to_reproduce", "severity_justification", - "component_name", "component_version", "notes"] - for field in template_fields: - if hasattr(template, field): - value = getattr(template, field) - if value is not None: - data[field] = value - - # Handle vulnerability_ids and endpoints (convert lists to strings) - data["vulnerability_ids"] = "\n".join(finding.vulnerability_ids) - if hasattr(template, "endpoints") and template.endpoints: - endpoints_value = template.endpoints - if isinstance(endpoints_value, list): - data["endpoints"] = "\n".join(endpoints_value) - else: - data["endpoints"] = endpoints_value - - template_tag_names = [tag.name for tag in template.tags.all()] - # Add tags as comma-separated string for TagField - if template_tag_names: - data["tags"] = ", ".join(template_tag_names) - - form = ApplyFindingTemplateForm(data=data, template=template) - # Combine tags from both Finding_Template and Finding tag models - # This ensures we don't lose tags that exist on templates but may have been removed from findings - if "tags" in form.fields: - finding_tag_model = Finding.tags.tag_model - template_tag_model = Finding_Template.tags.tag_model - - # Get all tags from Finding_Template model - template_tags = set(template_tag_model.objects.values_list("name", flat=True)) - # Get all tags from Finding model - finding_tags = set(finding_tag_model.objects.values_list("name", flat=True)) - # Combine both sets to get all unique tag names - all_tag_names = template_tags | finding_tags - - # Ensure all tags from both models exist in Finding's tag model (where they'll be applied) - # Strictly speaking, creating tags here isn't necessary since TagField can create them on save, - # but it's the safest option to avoid tags getting lost or not getting rendered properly. - # This prevents tagulous from removing tags that only exist on templates and ensures - # TagField can display them correctly during form rendering. - # Store tag objects in a dict for reuse - tag_objects = {} - for tag_name in all_tag_names: - tag, _ = finding_tag_model.objects.get_or_create( - name=tag_name, - defaults={"name": tag_name, "protected": False}, - ) - tag_objects[tag_name] = tag - - # Update autocomplete_tags to include tags from both models - form.fields["tags"].autocomplete_tags = finding_tag_model.objects.all().order_by("name") - - # Set initial value using template tags (already created above) - if template_tag_names: - template_finding_tags = [tag_objects[tag_name] for tag_name in template_tag_names] - form.fields["tags"].initial = template_finding_tags - product_tab = Product_Tab( - finding.test.engagement.product, - title="Finding Template Options", - tab="findings", - ) - return render( - request, - "dojo/apply_finding_template.html", - { - "finding": finding, - "product_tab": product_tab, - "template": template, - "form": form, - "finding_tags": [tag.name for tag in finding.tags.all()], - }, - ) - - -def apply_template_to_finding(request, fid, tid): - finding = get_object_or_404(Finding, id=fid) - user_has_permission_or_403(request.user, finding, "edit") - template = get_object_or_404(Finding_Template, id=tid) - - if request.method == "POST": - form = ApplyFindingTemplateForm(data=request.POST) - - if form.is_valid(): - template.last_used = timezone.now() - template.save() - - # Apply basic fields (existing) - finding.title = form.cleaned_data["title"] - finding.cwe = form.cleaned_data["cwe"] - finding.severity = form.cleaned_data["severity"] - finding.description = form.cleaned_data["description"] - finding.mitigation = form.cleaned_data["mitigation"] - finding.impact = form.cleaned_data["impact"] - finding.references = form.cleaned_data["references"] - finding.tags = form.cleaned_data["tags"] - - # Copy template fields (using centralized helper) - finding_helper.copy_template_fields_to_finding( - finding=finding, - template=template, - form_data=form.cleaned_data, - user=request.user, - copy_vulnerability_ids=True, - copy_endpoints=True, - copy_notes=True, - ) - - # Update review fields - finding.last_reviewed = timezone.now() - finding.last_reviewed_by = request.user - - # Save finding (this will trigger CVSS score computation if vectors are set) - finding.save() - else: - messages.add_message( - request, - messages.ERROR, - "There appears to be errors on the form, please correct below.", - extra_tags="alert-danger", - ) - product_tab = Product_Tab( - finding.test.engagement.product, - title="Apply Finding Template", - tab="findings", - ) - return render( - request, - "dojo/apply_finding_template.html", - { - "finding": finding, - "product_tab": product_tab, - "template": template, - "form": form, - }, - ) - - return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) - return HttpResponseRedirect(reverse("view_finding", args=(finding.id,))) - - -def templates(request): - templates = Finding_Template.objects.all().order_by("cwe") - templates = TemplateFindingFilter(request.GET, queryset=templates) - paged_templates = get_page_items(request, templates.qs, 25) - - title_words = get_words_for_field(templates.qs, "title") - - add_breadcrumb(title="Template Listing", top_level=True, request=request) - return render( - request, - "dojo/templates.html", - { - "templates": paged_templates, - "filtered": templates, - "title_words": title_words, - }, - ) - - -def export_templates_to_json(request): - leads_as_json = serializers.serialize("json", Finding_Template.objects.all()) - return HttpResponse(leads_as_json, content_type="application/json") - - -def ensure_template_tags_in_finding_model(template): - """ - Ensure all tags on a Finding_Template also exist in Finding's tag model. - This prevents tags from being lost when tagulous cleans up unused tags and ensures - tags can be properly applied when templates are used. - """ - if not template or not template.pk: - return - - finding_tag_model = Finding.tags.tag_model - - # Get all tag names from the template - template_tag_names = [tag.name for tag in template.tags.all()] - - # Ensure each tag exists in Finding's tag model - for tag_name in template_tag_names: - finding_tag_model.objects.get_or_create( - name=tag_name, - defaults={"name": tag_name, "protected": False}, - ) - - -def apply_cwe_mitigation(apply_to_findings, template, *, update=True): - count = 0 - if apply_to_findings and template.template_match and template.cwe is not None: - # Update active, verified findings with the CWE template - # If CWE only match only update issues where there isn't a CWE + Title match - if template.template_match_title: - count = Finding.objects.filter( - active=True, - verified=True, - cwe=template.cwe, - title__icontains=template.title, - ).update( - mitigation=template.mitigation, - impact=template.impact, - references=template.references, - ) - else: - finding_templates = Finding_Template.objects.filter( - cwe=template.cwe, template_match=True, template_match_title=True, - ) - - finding_ids = None - result_list = None - # Exclusion list - for title_template in finding_templates: - finding_ids = Finding.objects.filter( - active=True, - verified=True, - cwe=title_template.cwe, - title__icontains=title_template.title, - ).values_list("id", flat=True) - result_list = finding_ids if result_list is None else list(chain(result_list, finding_ids)) - - # If result_list is None the filter exclude won't work - if result_list: - count = Finding.objects.filter( - active=True, verified=True, cwe=template.cwe, - ).exclude(id__in=result_list) - else: - count = Finding.objects.filter( - active=True, verified=True, cwe=template.cwe, - ) - - if update: - # MySQL won't allow an 'update in statement' so loop will have to do - for finding in count: - finding.mitigation = template.mitigation - finding.impact = template.impact - finding.references = template.references - template.last_used = timezone.now() - template.save() - new_note = Notes() - new_note.entry = ( - f"CWE remediation text applied to finding for CWE: {template.cwe} using template: {template.title}." - ) - new_note.author, _created = User.objects.get_or_create( - username="System", - ) - new_note.save() - finding.notes.add(new_note) - finding.save() - - count = count.count() - return count - - -def add_template(request): - form = FindingTemplateForm() - if request.method == "POST": - form = FindingTemplateForm(request.POST) - if form.is_valid(): - template = form.save(commit=False) - template.numerical_severity = Finding.get_numerical_severity( - template.severity, - ) - # Save vulnerability IDs using helper - finding_helper.save_vulnerability_ids_template( - template, form.cleaned_data["vulnerability_ids"].split(), - ) - # Save endpoints using helper - if form.cleaned_data.get("endpoints"): - endpoint_urls = [url.strip() for url in form.cleaned_data["endpoints"].split("\n") if url.strip()] - finding_helper.save_endpoints_template(template, endpoint_urls) - template.save() - form.save_m2m() - # Ensure template tags exist in Finding's tag model - ensure_template_tags_in_finding_model(template) - - messages.add_message( - request, - messages.SUCCESS, - "Template created successfully.", - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("templates")) - messages.add_message( - request, - messages.ERROR, - "Template form has error, please revise and try again.", - extra_tags="alert-danger", - ) - add_breadcrumb(title="Add Template", top_level=False, request=request) - return render( - request, "dojo/add_template.html", {"form": form, "name": "Add Template"}, - ) - - -def edit_template(request, tid): - template = get_object_or_404(Finding_Template, id=tid) - initial_data = {"vulnerability_ids": "\n".join(template.vulnerability_ids)} - # Add endpoints to initial data if they exist - if hasattr(template, "endpoints") and template.endpoints: - endpoints_value = template.endpoints - if isinstance(endpoints_value, list): - initial_data["endpoints"] = "\n".join(endpoints_value) - else: - initial_data["endpoints"] = endpoints_value - form = FindingTemplateForm( - instance=template, - initial=initial_data, - ) - - if request.method == "POST": - form = FindingTemplateForm(request.POST, instance=template) - if form.is_valid(): - template = form.save(commit=False) - template.numerical_severity = Finding.get_numerical_severity( - template.severity, - ) - # Save vulnerability IDs using helper - finding_helper.save_vulnerability_ids_template( - template, form.cleaned_data["vulnerability_ids"].split(), - ) - # Save endpoints using helper - if form.cleaned_data.get("endpoints"): - endpoint_urls = [url.strip() for url in form.cleaned_data["endpoints"].split("\n") if url.strip()] - finding_helper.save_endpoints_template(template, endpoint_urls) - template.save() - form.save_m2m() - # Ensure template tags exist in Finding's tag model - ensure_template_tags_in_finding_model(template) - - messages.add_message( - request, - messages.SUCCESS, - "Template updated successfully.", - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("templates")) - messages.add_message( - request, - messages.ERROR, - "Template form has error, please revise and try again.", - extra_tags="alert-danger", - ) - - add_breadcrumb(title="Edit Template", top_level=False, request=request) - return render( - request, - "dojo/add_template.html", - { - "form": form, - "name": "Edit Template", - "template": template, - }, - ) - - -def delete_template(request, tid): - template = get_object_or_404(Finding_Template, id=tid) - if request.method == "POST": - form = DeleteFindingTemplateForm(request.POST, instance=template) - if form.is_valid(): - template.delete() - messages.add_message( - request, - messages.SUCCESS, - "Finding Template deleted successfully.", - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("templates")) - messages.add_message( - request, - messages.ERROR, - "Unable to delete Template, please revise and try again.", - extra_tags="alert-danger", - ) - return None - raise PermissionDenied - - -def download_finding_pic(request, token): - class Thumbnail(ImageSpec): - processors = [ResizeToFill(100, 100)] - format = "JPEG" - options = {"quality": 70} - - class Small(ImageSpec): - processors = [ResizeToFill(640, 480)] - format = "JPEG" - options = {"quality": 100} - - class Medium(ImageSpec): - processors = [ResizeToFill(800, 600)] - format = "JPEG" - options = {"quality": 100} - - class Large(ImageSpec): - processors = [ResizeToFill(1024, 768)] - format = "JPEG" - options = {"quality": 100} - - class Original(ImageSpec): - format = "JPEG" - options = {"quality": 100} - - mimetypes.init() - - size_map = { - "thumbnail": Thumbnail, - "small": Small, - "medium": Medium, - "large": Large, - "original": Original, - } - - try: - access_token = FileAccessToken.objects.get(token=token) - if access_token.size not in list(size_map.keys()): - raise Http404 - size = access_token.size - access_token.delete() - except Exception: - raise PermissionDenied - - with Path(access_token.file.file.file.name).open("rb") as file: - file_name = file.name - image = size_map[size](source=file).generate() - response = StreamingHttpResponse(FileIterWrapper(image)) - response["Content-Disposition"] = "inline" - mimetype, _encoding = mimetypes.guess_type(file_name) - response["Content-Type"] = mimetype or "application/octet-stream" - return response - - -def merge_finding_product(request, pid): - product = get_object_or_404(Product, pk=pid) - finding_to_update = request.GET.getlist("finding_to_update") - findings = None - - if ( - request.GET.get("merge_findings") or request.method == "POST" - ) and finding_to_update: - finding = Finding.objects.get( - id=finding_to_update[0], test__engagement__product=product, - ) - findings = Finding.objects.filter( - id__in=finding_to_update, test__engagement__product=product, - ) - form = MergeFindings( - finding=finding, - findings=findings, - initial={"finding_to_merge_into": finding_to_update[0]}, - ) - - if request.method == "POST": - form = MergeFindings(request.POST, finding=finding, findings=findings) - if form.is_valid(): - finding_to_merge_into = form.cleaned_data["finding_to_merge_into"] - findings_to_merge = form.cleaned_data["findings_to_merge"] - finding_descriptions = "" - finding_references = "" - notes_entry = "" - static = False - dynamic = False - - if finding_to_merge_into not in findings_to_merge: - for finding in findings_to_merge.exclude( - pk=finding_to_merge_into.pk, - ): - notes_entry = f"{notes_entry}\n- {finding.title} ({finding.id})," - if finding.static_finding: - static = finding.static_finding - - if finding.dynamic_finding: - dynamic = finding.dynamic_finding - - if form.cleaned_data["append_description"]: - finding_descriptions = f"{finding_descriptions}\n{finding.description}" - # Workaround until file path is one to many - if finding.file_path: - finding_descriptions = f"{finding_descriptions}\n**File Path:** {finding.file_path}\n" - - # If checked merge the Reference - if ( - form.cleaned_data["append_reference"] - and finding.references is not None - ): - finding_references = f"{finding_references}\n{finding.references}" - - # if checked merge the endpoints - if form.cleaned_data["add_endpoints"]: - finding_to_merge_into.endpoints.add( - *finding.endpoints.all(), - ) - - # if checked merge the tags - if form.cleaned_data["tag_finding"]: - for tag in finding.tags.all(): - finding_to_merge_into.tags.add(tag) - - # if checked re-assign the burp requests to the merged finding - if form.cleaned_data["dynamic_raw"]: - BurpRawRequestResponse.objects.filter( - finding=finding, - ).update(finding=finding_to_merge_into) - - # Add merge finding information to the note if set to inactive - if form.cleaned_data["finding_action"] == "inactive": - single_finding_notes_entry = ("Finding has been set to inactive " - f"and merged with the finding: {finding_to_merge_into.title}.") - note = Notes( - entry=single_finding_notes_entry, author=request.user, - ) - note.save() - finding.notes.add(note) - - # If the merged finding should be tagged as merged-into - if form.cleaned_data["mark_tag_finding"]: - finding.tags.add("merged-inactive") - - # Update the finding to merge into - if finding_descriptions: - finding_to_merge_into.description = f"{finding_to_merge_into.description}\n\n{finding_descriptions}" - - if finding_to_merge_into.static_finding: - static = finding.static_finding - - if finding_to_merge_into.dynamic_finding: - dynamic = finding.dynamic_finding - - if finding_references: - finding_to_merge_into.references = f"{finding_to_merge_into.references}\n{finding_references}" - - finding_to_merge_into.static_finding = static - finding_to_merge_into.dynamic_finding = dynamic - - # Update the timestamp - finding_to_merge_into.last_reviewed = timezone.now() - finding_to_merge_into.last_reviewed_by = request.user - - # Save the data to the merged finding - finding_to_merge_into.save() - - # If the finding merged into should be tagged as merged - if form.cleaned_data["mark_tag_finding"]: - finding_to_merge_into.tags.add("merged") - - finding_action = "" - # Take action on the findings - if form.cleaned_data["finding_action"] == "inactive": - finding_action = "inactivated" - findings_to_merge.exclude(pk=finding_to_merge_into.pk).update( - active=False, - last_reviewed=timezone.now(), - last_reviewed_by=request.user, - ) - elif form.cleaned_data["finding_action"] == "delete": - finding_action = "deleted" - findings_to_merge.delete() - - notes_entry = ("Finding consists of merged findings from the following " - f"findings which have been {finding_action}: {notes_entry[:-1]}") - note = Notes(entry=notes_entry, author=request.user) - note.save() - finding_to_merge_into.notes.add(note) - - messages.add_message( - request, - messages.SUCCESS, - "Findings merged", - extra_tags="alert-success", - ) - return HttpResponseRedirect( - reverse("edit_finding", args=(finding_to_merge_into.id,)), - ) - messages.add_message( - request, - messages.ERROR, - "Unable to merge findings. Findings to merge contained in finding to merge into.", - extra_tags="alert-danger", - ) - else: - messages.add_message( - request, - messages.ERROR, - "Unable to merge findings. Required fields were not selected.", - extra_tags="alert-danger", - ) - - product_tab = Product_Tab( - finding.test.engagement.product, title="Merge Findings", tab="findings", - ) - custom_breadcrumb = { - "Open Findings": reverse( - "product_open_findings", args=(finding.test.engagement.product.id,), - ) - + "?test__engagement__product=" - + str(finding.test.engagement.product.id), - } - - return render( - request, - "dojo/merge_findings.html", - { - "form": form, - "name": "Merge Findings", - "finding": finding, - "product_tab": product_tab, - "title": product_tab.title, - "custom_breadcrumb": custom_breadcrumb, - }, - ) - -# bulk update and delete are combined, so we can't have the nice user_is_authorized decorator - - -def _bulk_delete_findings(request, pid, form, finding_to_update, finds, total_find_count): - """Helper function to handle bulk deletion of findings.""" - if form.is_valid() and finding_to_update: - if pid is not None: - product = get_object_or_404(Product, id=pid) - user_has_permission_or_403( - request.user, product, "delete", - ) - - finds = get_authorized_findings_for_queryset( - "delete", finds, - ).distinct() - - skipped_find_count = total_find_count - finds.count() - deleted_find_count = finds.count() - - for find in finds: - find.delete() - - if skipped_find_count > 0: - add_error_message_to_response( - f"Skipped deletion of {skipped_find_count} findings because you are not authorized.", - ) - - if deleted_find_count > 0: - messages.add_message( - request, - messages.SUCCESS, - f"Bulk delete of {deleted_find_count} findings was successful.", - extra_tags="alert-success", - ) - - -def _bulk_update_finding_status_and_severity(finds, form, request, system_settings, prods, now): - """Helper function to handle status and severity updates for findings.""" - skipped_duplicate_count = 0 - actually_updated_count = 0 - - if form.cleaned_data["severity"] or form.cleaned_data["status"]: - # Accumulate findings for batched FP-history processing after the per-finding loop - fp_findings = [] # findings being marked as FP - reactivation_findings = [] # findings being un-FP'd (retroactive reactivation) - - for find in finds: - old_find = copy.deepcopy(find) - - if form.cleaned_data["severity"]: - find.severity = form.cleaned_data["severity"] - find.numerical_severity = Finding.get_numerical_severity( - form.cleaned_data["severity"], - ) - find.last_reviewed = now - find.last_reviewed_by = request.user - - if form.cleaned_data["status"]: - # logger.debug('setting status from bulk edit form: %s', form) - # Check if finding is duplicate and user wants to set active/verified - if find.duplicate and (form.cleaned_data["active"] or form.cleaned_data["verified"]): - # Skip active/verified but allow other status changes - skipped_duplicate_count += 1 - # Set other fields but not active/verified - find.false_p = form.cleaned_data["false_p"] - find.out_of_scope = form.cleaned_data["out_of_scope"] - find.is_mitigated = form.cleaned_data["is_mitigated"] - find.under_review = form.cleaned_data["under_review"] - else: - # Apply all status changes normally - find.active = form.cleaned_data["active"] - find.verified = form.cleaned_data["verified"] - find.false_p = form.cleaned_data["false_p"] - find.out_of_scope = form.cleaned_data["out_of_scope"] - find.is_mitigated = form.cleaned_data["is_mitigated"] - find.under_review = form.cleaned_data["under_review"] - find.last_reviewed = timezone.now() - find.last_reviewed_by = request.user - - # use super to avoid all custom logic in our overriden save method - # it will trigger the pre_save signal - find.save_no_options() - actually_updated_count += 1 - - if system_settings.false_positive_history: - if find.false_p: - fp_findings.append(find) - elif old_find.false_p and not find.false_p: - reactivation_findings.append(find) - - # --- Batch FP history: one DB query per (product, algorithm) group instead of one per finding --- - if system_settings.false_positive_history and fp_findings: - groups: dict = defaultdict(list) - for find in fp_findings: - groups[find.test.engagement.product_id, find.test.deduplication_algorithm].append(find) - for group_findings in groups.values(): - do_false_positive_history_batch(group_findings) - - # --- Batch retroactive reactivation --- - if ( - system_settings.false_positive_history - and system_settings.retroactive_false_positive_history - and reactivation_findings - ): - all_fp_ids_to_reactivate: set = set() - groups = defaultdict(list) - for find in reactivation_findings: - groups[find.test.engagement.product_id, find.test.deduplication_algorithm].append(find) - for (_, dedup_alg), group_findings in groups.items(): - product = group_findings[0].test.engagement.product - candidates = _fetch_fp_candidates_for_batch(group_findings, product, dedup_alg) - for find in group_findings: - if dedup_alg == "unique_id_from_tool_or_hash_code": - by_uid, by_hash = candidates - uid_matches = by_uid.get(find.unique_id_from_tool, []) if find.unique_id_from_tool else [] - hash_matches = by_hash.get(find.hash_code, []) if find.hash_code else [] - seen: dict = {} - for ef in uid_matches + hash_matches: - seen.setdefault(ef.id, ef) - existing = list(seen.values()) - elif dedup_alg == "hash_code": - existing = candidates.get(find.hash_code, []) if find.hash_code else [] - elif dedup_alg == "unique_id_from_tool": - existing = candidates.get(find.unique_id_from_tool, []) if find.unique_id_from_tool else [] - elif dedup_alg == "legacy": - lookup_key = (find.title.lower(), find.severity) if find.title else None - existing = candidates.get(lookup_key, []) if lookup_key else [] - else: - existing = [] - for ef in existing: - if ef.false_p: - all_fp_ids_to_reactivate.add(ef.id) - - if all_fp_ids_to_reactivate: - logger.debug( - "FALSE_POSITIVE_HISTORY: Reactivating %i finding(s): %s", - len(all_fp_ids_to_reactivate), - sorted(all_fp_ids_to_reactivate), - ) - # All reactivation findings received the same form values, so a single bulk update covers all. - # QuerySet.update() bypasses Django signals, which is intentional here — it mirrors - # the previous save_no_options() calls that also disabled all post-save processing. - Finding.objects.filter(id__in=all_fp_ids_to_reactivate).update( - false_p=False, - active=form.cleaned_data["active"], - verified=form.cleaned_data["verified"], - out_of_scope=form.cleaned_data["out_of_scope"], - is_mitigated=form.cleaned_data["is_mitigated"], - ) - - for prod in prods: - calculate_grade(prod.id) - - if skipped_duplicate_count > 0: - messages.add_message( - request, - messages.WARNING, - f"Skipped status update of {skipped_duplicate_count} duplicate findings. Duplicate findings cannot be active or verified.", - extra_tags="alert-warning", - ) - - return skipped_duplicate_count, actually_updated_count - - -def _bulk_update_simple_fields(finds, form): - """Helper function to handle simple field updates (date, planned_remediation_date, etc.).""" - if form.cleaned_data["date"]: - for finding in finds: - finding.date = form.cleaned_data["date"] - finding.save_no_options() - - if form.cleaned_data["planned_remediation_date"]: - for finding in finds: - finding.planned_remediation_date = form.cleaned_data[ - "planned_remediation_date" - ] - finding.save_no_options() - - if form.cleaned_data["planned_remediation_version"]: - for finding in finds: - finding.planned_remediation_version = form.cleaned_data[ - "planned_remediation_version" - ] - finding.save_no_options() - - -def _bulk_update_risk_acceptance(finds, form, request, prods): - """Helper function to handle risk acceptance updates.""" - skipped_risk_accept_count = 0 - - if form.cleaned_data["risk_acceptance"]: - for finding in finds: - if form.cleaned_data["risk_accept"]: - if ( - not finding.test.engagement.product.enable_simple_risk_acceptance - ): - skipped_risk_accept_count += 1 - else: - ra_helper.simple_risk_accept(request.user, finding) - elif form.cleaned_data["risk_unaccept"]: - ra_helper.risk_unaccept(request.user, finding) - - for prod in prods: - calculate_grade(prod.id) - - if skipped_risk_accept_count > 0: - messages.add_message( - request, - messages.WARNING, - (f"Skipped simple risk acceptance of {skipped_risk_accept_count} findings, " - "simple risk acceptance is disabled on the related products"), - extra_tags="alert-warning", - ) - - return skipped_risk_accept_count - - -def _bulk_update_finding_groups(finds, form): - """Helper function to handle finding group operations.""" - return_url = None - - if form.cleaned_data["finding_group_create"]: - logger.debug("finding_group_create checked!") - finding_group_name = form.cleaned_data["finding_group_create_name"] - logger.debug("finding_group_create_name: %s", finding_group_name) - finding_group, added, skipped = finding_helper.create_finding_group( - finds, finding_group_name, - ) - - if added: - add_success_message_to_response( - f"Created finding group with {added} findings", - ) - return_url = reverse( - "view_finding_group", args=(finding_group.id,), - ) - - if skipped: - add_success_message_to_response( - f"Skipped {skipped} findings in group creation, findings already part of another group", - ) - - # refresh findings from db - finds = finds.all() - - if form.cleaned_data["finding_group_add"]: - logger.debug("finding_group_add checked!") - fgid = form.cleaned_data["add_to_finding_group_id"] - finding_group = Finding_Group.objects.get(id=fgid) - finding_group, added, skipped = finding_helper.add_to_finding_group( - finding_group, finds, - ) - - if added: - add_success_message_to_response( - f"Added {added} findings to finding group {finding_group.name}", - ) - return_url = reverse( - "view_finding_group", args=(finding_group.id,), - ) - - if skipped: - add_success_message_to_response( - f"Skipped {skipped} findings when adding to finding group {finding_group.name}, " - "findings already part of another group", - ) - - # refresh findings from db - finds = finds.all() - - if form.cleaned_data["finding_group_remove"]: - logger.debug("finding_group_remove checked!") - ( - finding_groups, - removed, - skipped, - ) = finding_helper.remove_from_finding_group(finds) - - if removed: - add_success_message_to_response( - "Removed {} findings from finding groups {}".format( - removed, - ",".join( - [ - finding_group.name - for finding_group in finding_groups - ], - ), - ), - ) - - if skipped: - add_success_message_to_response( - f"Skipped {skipped} findings when removing from any finding group, findings not part of any group", - ) - - # refresh findings from db - finds = finds.all() - - if form.cleaned_data["finding_group_by"]: - logger.debug("finding_group_by checked!") - logger.debug(form.cleaned_data) - finding_group_by_option = form.cleaned_data[ - "finding_group_by_option" - ] - logger.debug("finding_group_by_option: %s", finding_group_by_option) - - ( - finding_groups, - grouped, - skipped, - groups_created, - ) = finding_helper.group_findings_by(finds, finding_group_by_option) - - if grouped: - add_success_message_to_response( - f"Grouped {grouped} findings into {len(finding_groups)} ({groups_created} newly created) finding groups", - ) - - if skipped: - add_success_message_to_response( - f"Skipped {skipped} findings when grouping by {finding_group_by_option} as these findings " - "were already in an existing group", - ) - - # refresh findings from db - finds = finds.all() - - return return_url, finds - - -def _bulk_push_to_jira(finds, form, note): - """Helper function to handle JIRA push operations.""" - error_counts = defaultdict(lambda: 0) - success_count = 0 - finding_groups = set( # noqa: C401 - finding.finding_group - for finding in finds - if finding.has_finding_group - and ( - jira_services.is_push_all_issues(finding) - or jira_services.is_keep_in_sync(finding) - or form.cleaned_data.get("push_to_jira") - ) - ) - logger.debug("finding_groups: %s", finding_groups) - for group in finding_groups: - if ( - form.cleaned_data.get("push_to_jira") - or jira_services.is_push_all_issues(group) - or jira_services.is_keep_in_sync(group) - ): - ( - can_be_pushed_to_jira, - error_message, - _error_code, - ) = jira_services.can_be_pushed(group) - if not can_be_pushed_to_jira: - error_counts[error_message] += 1 - jira_services.log_cannot_be_pushed_reason(error_message, group) - else: - logger.debug( - "pushing to jira from finding.finding_bulk_update_all()", - ) - jira_services.push(group) - success_count += 1 - - for error_message, error_count in error_counts.items(): - add_error_message_to_response(f"{error_count} finding groups could not be pushed to JIRA: {error_message}") - - if success_count > 0: - add_success_message_to_response(f"{success_count} finding groups pushed to JIRA successfully") - - # refresh from db - finds = finds.all() - - error_counts = defaultdict(lambda: 0) - success_count = 0 - for finding in finds: - tool_issue_updater.async_tool_issue_update(finding) - - # not sure yet if we want to support bulk unlink, so leave as commented out for now - # if form.cleaned_data['unlink_from_jira']: - # if finding.has_jira_issue: - # jira_services.unlink_finding(request, finding) - - # Because we never call finding.save() in a bulk update, we need to actually - # push the JIRA stuff here, rather than in finding.save() - # can't use helper as when push_all_jira_issues is True, - # the checkbox gets disabled and is always false - # push_to_jira = jira_services.is_push_to_jira(new_finding, - # form.cleaned_data.get('push_to_jira')) - if ( - form.cleaned_data.get("push_to_jira") - or jira_services.is_push_all_issues(finding) - or jira_services.is_keep_in_sync(finding) - ) and not finding.has_finding_group: - ( - can_be_pushed_to_jira, - error_message, - _error_code, - ) = jira_services.can_be_pushed(finding) - if finding.has_jira_group_issue and not finding.has_jira_issue: - error_message = ( - "finding already pushed as part of Finding Group" - ) - error_counts[error_message] += 1 - jira_services.log_cannot_be_pushed_reason(error_message, finding) - elif not can_be_pushed_to_jira: - error_counts[error_message] += 1 - jira_services.log_cannot_be_pushed_reason(error_message, finding) - else: - logger.debug( - "pushing to jira from finding.finding_bulk_update_all()", - ) - jira_services.push(finding) - if note is not None and isinstance(note, Notes): - jira_services.add_comment(finding, note) - success_count += 1 - - for error_message, error_count in error_counts.items(): - add_error_message_to_response(f"{error_count} findings could not be pushed to JIRA: {error_message}") - - if success_count > 0: - add_success_message_to_response(f"{success_count} findings pushed to JIRA successfully") - - -def finding_bulk_update_all(request, pid=None): - system_settings = System_Settings.objects.get() - - logger.debug("bulk 10") - form = FindingBulkUpdateForm(request.POST) - now = timezone.now() - return_url = None - - if request.method == "POST": - logger.debug("bulk 20") - - finding_to_update = request.POST.getlist("finding_to_update") - # Add pghistory context for audit trail (adds to existing middleware context) - pghistory.context( - source="bulk_edit", - finding_count=len(finding_to_update), - ) - finds = Finding.objects.filter(id__in=finding_to_update).order_by("id") - total_find_count = finds.count() - prods = set(find.test.engagement.product for find in finds) # noqa: C401 - if request.POST.get("delete_bulk_findings"): - _bulk_delete_findings(request, pid, form, finding_to_update, finds, total_find_count) - elif form.is_valid() and finding_to_update: - if pid is not None: - product = get_object_or_404(Product, id=pid) - user_has_permission_or_403( - request.user, product, "edit", - ) - - # make sure users are not editing stuff they are not authorized for - finds = get_authorized_findings_for_queryset( - "edit", finds, - ).distinct() - - skipped_find_count = total_find_count - finds.count() - updated_find_count = finds.count() - - if skipped_find_count > 0: - add_error_message_to_response( - f"Skipped update of {skipped_find_count} findings because you are not authorized.", - ) - - finds = prefetch_for_findings(finds) - note = None - actually_updated_count = 0 - - _skipped_duplicate_count, actually_updated_count = _bulk_update_finding_status_and_severity( - finds, form, request, system_settings, prods, now, - ) - - _bulk_update_simple_fields(finds, form) - - _skipped_risk_accept_count = _bulk_update_risk_acceptance( - finds, form, request, prods, - ) - - group_return_url, finds = _bulk_update_finding_groups(finds, form) - if group_return_url: - return_url = group_return_url - - if form.cleaned_data["push_to_github"]: - logger.debug("push selected findings to github") - for finding in finds: - logger.debug("will push to GitHub finding: " + str(finding)) - old_status = finding.status() - if form.cleaned_data["push_to_github"]: - if GITHUB_Issue.objects.filter(finding=finding).exists(): - update_external_issue(finding.id, old_status, "github") - else: - add_external_issue(finding.id, "github") - - if form.cleaned_data["notes"]: - logger.debug("Setting bulk notes") - note = Notes( - entry=form.cleaned_data["notes"], - author=request.user, - date=timezone.now(), - ) - note.save() - history = NoteHistory( - data=note.entry, time=note.date, current_editor=note.author, - ) - history.save() - note.history.add(history) - for finding in finds: - finding.notes.add(note) - finding.save() - - if form.cleaned_data["tags"]: - tags = form.cleaned_data["tags"] - logger.debug("bulk_edit: adding tags to %d findings: %s", finds.count(), tags) - # Delegate parsing and handling of strings/iterables to helper - bulk_add_tags_to_instances(tag_or_tags=tags, instances=finds, tag_field_name="tags") - - _bulk_push_to_jira(finds, form, note) - - # Show success message if status/severity updates were made (using actually_updated_count) - # or if other updates were made (using updated_find_count) - if (form.cleaned_data["severity"] or form.cleaned_data["status"]) and actually_updated_count > 0: - messages.add_message( - request, - messages.SUCCESS, - f"Bulk update of {actually_updated_count} findings was successful.", - extra_tags="alert-success", - ) - elif updated_find_count > 0 and ( - form.cleaned_data["date"] or form.cleaned_data["planned_remediation_date"] - or form.cleaned_data["planned_remediation_version"] or form.cleaned_data["tags"] - or form.cleaned_data["notes"] or form.cleaned_data["risk_acceptance"] - or form.cleaned_data["finding_group_create"] or form.cleaned_data["finding_group_add"] - or form.cleaned_data["finding_group_remove"] or form.cleaned_data["finding_group_by"] - or form.cleaned_data["push_to_jira"] or form.cleaned_data["push_to_github"] - ): - messages.add_message( - request, - messages.SUCCESS, - f"Bulk update of {updated_find_count} findings was successful.", - extra_tags="alert-success", - ) - else: - messages.add_message( - request, - messages.ERROR, - "Unable to process bulk update. Required fields were not selected.", - extra_tags="alert-danger", - ) - - if return_url: - redirect(request, return_url) - - return redirect_to_return_url_or_else(request, None) - - -def find_available_notetypes(notes): - single_note_types = Note_Type.objects.filter( - is_single=True, is_active=True, - ).values_list("id", flat=True) - multiple_note_types = Note_Type.objects.filter( - is_single=False, is_active=True, - ).values_list("id", flat=True) - available_note_types = [] - for note_type_id in multiple_note_types: - available_note_types.append(note_type_id) - for note_type_id in single_note_types: - for note in notes: - if note_type_id == note.note_type_id: - break - else: - available_note_types.append(note_type_id) - return Note_Type.objects.filter(id__in=available_note_types).order_by("-id") - - -def get_missing_mandatory_notetypes(finding): - notes = finding.notes.all() - mandatory_note_types = Note_Type.objects.filter( - is_mandatory=True, is_active=True, - ).values_list("id", flat=True) - notes_to_be_added = [] - for note_type_id in mandatory_note_types: - for note in notes: - if note_type_id == note.note_type_id: - break - else: - notes_to_be_added.append(note_type_id) - return Note_Type.objects.filter(id__in=notes_to_be_added) - - -@require_POST -def mark_finding_duplicate(request, original_id, duplicate_id): - - original = get_object_or_404(Finding, id=original_id) - duplicate = get_object_or_404( - Finding.objects.filter(test__engagement__product=original.test.engagement.product), - id=duplicate_id, - ) - - if original.test.engagement != duplicate.test.engagement: - if (original.test.engagement.deduplication_on_engagement - or duplicate.test.engagement.deduplication_on_engagement): - messages.add_message( - request, - messages.ERROR, - ("Marking finding as duplicate/original failed as they are not in the same engagement " - "and deduplication_on_engagement is enabled for at least one of them"), - extra_tags="alert-danger", - ) - return redirect_to_return_url_or_else( - request, reverse("view_finding", args=(duplicate.id,)), - ) - - duplicate.duplicate = True - duplicate.active = False - duplicate.verified = False - # make sure we don't create circular or transitive duplicates - if original.duplicate: - duplicate.duplicate_finding = original.duplicate_finding - else: - duplicate.duplicate_finding = original - - logger.debug( - "marking finding %i as duplicate of %i", - duplicate.id, - duplicate.duplicate_finding.id, - ) - - duplicate.last_reviewed = timezone.now() - duplicate.last_reviewed_by = request.user - duplicate.save(dedupe_option=False) - original.found_by.add(duplicate.test.test_type) - original.save(dedupe_option=False) - - return redirect_to_return_url_or_else( - request, reverse("view_finding", args=(duplicate.id,)), - ) - - -def reset_finding_duplicate_status_internal(user, duplicate_id): - duplicate = get_object_or_404(Finding, id=duplicate_id) - - if not duplicate.duplicate: - return None - - logger.debug("resetting duplicate status of %i", duplicate.id) - duplicate.duplicate = False - duplicate.active = True - if duplicate.duplicate_finding: - # duplicate.duplicate_finding.original_finding.remove(duplicate) # shouldn't be needed - duplicate.duplicate_finding = None - duplicate.last_reviewed = timezone.now() - duplicate.last_reviewed_by = user - duplicate.save(dedupe_option=False) - - return duplicate.id - - -@require_POST -def reset_finding_duplicate_status(request, duplicate_id): - checked_duplicate_id = reset_finding_duplicate_status_internal( - request.user, duplicate_id, - ) - if checked_duplicate_id is None: - messages.add_message( - request, - messages.ERROR, - "Can't reset duplicate status of a finding that is not a duplicate", - extra_tags="alert-danger", - ) - return redirect_to_return_url_or_else( - request, reverse("view_finding", args=(duplicate_id,)), - ) - - return redirect_to_return_url_or_else( - request, reverse("view_finding", args=(checked_duplicate_id,)), - ) - - -def set_finding_as_original_internal(user, finding_id, new_original_id): - finding = get_object_or_404(Finding, id=finding_id) - new_original = get_object_or_404( - Finding.objects.filter(test__engagement__product=finding.test.engagement.product), - id=new_original_id, - ) - - if finding.test.engagement != new_original.test.engagement: - if (finding.test.engagement.deduplication_on_engagement - or new_original.test.engagement.deduplication_on_engagement): - return False - - if finding.duplicate or finding.original_finding.all(): - # existing cluster, so update all cluster members - - if finding.duplicate and finding.duplicate_finding: - logger.debug( - "setting old original %i as duplicate of %i", - finding.duplicate_finding.id, - new_original.id, - ) - finding.duplicate_finding.duplicate_finding = new_original - finding.duplicate_finding.duplicate = True - finding.duplicate_finding.save(dedupe_option=False) - - for cluster_member in finding.duplicate_finding_set(): - if cluster_member != new_original: - logger.debug( - "setting new original for %i to %i", - cluster_member.id, - new_original.id, - ) - cluster_member.duplicate_finding = new_original - cluster_member.save(dedupe_option=False) - - logger.debug( - "setting new original for old root %i to %i", finding.id, new_original.id, - ) - finding.duplicate = True - finding.duplicate_finding = new_original - finding.save(dedupe_option=False) - - else: - # creating a new cluster, so mark finding as duplicate - logger.debug("marking %i as duplicate of %i", finding.id, new_original.id) - finding.duplicate = True - finding.active = False - finding.duplicate_finding = new_original - finding.last_reviewed = timezone.now() - finding.last_reviewed_by = user - finding.save(dedupe_option=False) - - logger.debug("marking new original %i as not duplicate", new_original.id) - new_original.duplicate = False - new_original.duplicate_finding = None - new_original.save(dedupe_option=False) - - return True - - -@require_POST -def set_finding_as_original(request, finding_id, new_original_id): - success = set_finding_as_original_internal( - request.user, finding_id, new_original_id, - ) - if not success: - messages.add_message( - request, - messages.ERROR, - ("Marking finding as duplicate/original failed as they are not in the same engagement " - "and deduplication_on_engagement is enabled for at least one of them"), - extra_tags="alert-danger", - ) - - return redirect_to_return_url_or_else( - request, reverse("view_finding", args=(finding_id,)), - ) - - -@require_POST -def unlink_jira(request, fid): - finding = get_object_or_404(Finding, id=fid) - logger.info( - "trying to unlink a linked jira issue from %d:%s", finding.id, finding.title, - ) - if finding.has_jira_issue: - try: - jira_services.unlink_finding(request, finding) - - messages.add_message( - request, - messages.SUCCESS, - "Link to JIRA issue succesfully deleted", - extra_tags="alert-success", - ) - - return JsonResponse({"result": "OK"}) - except Exception: - logger.exception("Link to JIRA could not be deleted") - messages.add_message( - request, - messages.ERROR, - "Link to JIRA could not be deleted, see alerts for details", - extra_tags="alert-danger", - ) - - return HttpResponse(status=500) - else: - messages.add_message( - request, messages.ERROR, "Link to JIRA not found", extra_tags="alert-danger", - ) - return HttpResponse(status=400) - - -@require_POST -def push_to_jira(request, fid): - finding = get_object_or_404(Finding, id=fid) - try: - logger.info( - "trying to push %d:%s to JIRA to create or update JIRA issue", - finding.id, - finding.title, - ) - logger.debug("pushing to jira from finding.push_to-jira()") - - # it may look like succes here, but the push_to_jira are swallowing exceptions - # but cant't change too much now without having a test suite, - # so leave as is for now with the addition warning message - # to check alerts for background errors. - if jira_services.push(finding): - messages.add_message( - request, - messages.SUCCESS, - message="Action queued to create or update linked JIRA issue, check alerts for background errors.", - extra_tags="alert-success", - ) - else: - messages.add_message( - request, - messages.SUCCESS, - "Push to JIRA failed, check alerts on the top right for errors", - extra_tags="alert-danger", - ) - - return JsonResponse({"result": "OK"}) - except Exception: - logger.exception("Error pushing to JIRA") - messages.add_message( - request, messages.ERROR, "Error pushing to JIRA", extra_tags="alert-danger", - ) - return HttpResponse(status=500) - - -# precalculate because we need related_actions to be set -def duplicate_cluster(request, finding): - duplicate_cluster = finding.duplicate_finding_set() - - duplicate_cluster = prefetch_for_findings(duplicate_cluster) - - # populate actions for findings in duplicate cluster - for duplicate_member in duplicate_cluster: - duplicate_member.related_actions = ( - calculate_possible_related_actions_for_similar_finding( - request, finding, duplicate_member, - ) - ) - - return duplicate_cluster - - -# django doesn't allow much logic or even method calls with parameters in templates. -# so we have to use a function in this view to calculate the possible actions on a similar (or duplicate) finding. -# and we assign this dictionary to the finding so it can be accessed in the template. -# these actions are always calculated in the context of the finding the user is viewing -# because this determines which actions are possible -def calculate_possible_related_actions_for_similar_finding( - request, finding, similar_finding, -): - actions = [] - if similar_finding.test.engagement != finding.test.engagement and ( - similar_finding.test.engagement.deduplication_on_engagement - or finding.test.engagement.deduplication_on_engagement - ): - actions.append( - { - "action": "None", - "reason": ("This finding is in a different engagement and deduplication_inside_engagment " - "is enabled here or in that finding"), - }, - ) - elif finding.duplicate_finding == similar_finding: - actions.append( - { - "action": "None", - "reason": ("This finding is the root of the cluster, use an action on another row, " - "or the finding on top of the page to change the root of the cluser"), - }, - ) - elif similar_finding.original_finding.all(): - actions.append( - { - "action": "None", - "reason": ("This finding is similar, but is already an original in a different cluster. " - "Remove it from that cluster before you connect it to this cluster."), - }, - ) - elif similar_finding.duplicate_finding: - # reset duplicate status is always possible - actions.append( - { - "action": "reset_finding_duplicate_status", - "reason": ("This will remove the finding from the cluster, " - "effectively marking it no longer as duplicate. " - "Will not trigger deduplication logic after saving."), - }, - ) - - if similar_finding.duplicate_finding in {finding, finding.duplicate_finding}: - # duplicate inside the same cluster - actions.append( - { - "action": "set_finding_as_original", - "reason": ("Sets this finding as the Original for the whole cluster. " - "The existing Original will be downgraded to become a member of the cluster and, " - "together with the other members, will be marked as duplicate of the new Original."), - }, - ) - else: - # duplicate inside different cluster - actions.append( - { - "action": "mark_finding_duplicate", - "reason": ("Will mark this finding as duplicate of the root finding in this cluster, " - "effectively adding it to the cluster and removing it from the other cluster."), - }, - ) - # similar is not a duplicate yet - elif finding.duplicate or finding.original_finding.all(): - actions.extend(( - { - "action": "mark_finding_duplicate", - "reason": "Will mark this finding as duplicate of the root finding in this cluster", - }, { - "action": "set_finding_as_original", - "reason": ( - "Sets this finding as the Original for the whole cluster. " - "The existing Original will be downgraded to become a member of the cluster and, " - "together with the other members, will be marked as duplicate of the new Original." - ), - }, - )) - else: - # similar_finding is not an original/root of a cluster as per earlier if clause - actions.extend(( - { - "action": "mark_finding_duplicate", - "reason": "Will mark this finding as duplicate of the finding on this page.", - }, { - "action": "set_finding_as_original", - "reason": ( - "Sets this finding as the Original marking the finding " - "on this page as duplicate of this original." - ), - }, - )) - - return actions +# Backward-compat shim: the view logic moved to dojo.finding.ui.views during the +# module reorg. External consumers (e.g. dojo-pro) still import from +# dojo.finding.views, so re-export the public names from their new location. +from dojo.finding.ui.views import * # noqa: F403 -- backward compat re-export diff --git a/dojo/finding_group/views.py b/dojo/finding_group/views.py index c77ddefccd5..6c6bb2907e9 100644 --- a/dojo/finding_group/views.py +++ b/dojo/finding_group/views.py @@ -13,12 +13,12 @@ from django.views.decorators.http import require_POST from dojo.authorization.authorization import user_has_permission_or_403 -from dojo.filters import ( +from dojo.finding.queries import prefetch_for_findings +from dojo.finding.ui.filters import ( FindingFilter, FindingFilterWithoutObjectLookups, FindingGroupsFilter, ) -from dojo.finding.queries import prefetch_for_findings from dojo.forms import DeleteFindingGroupForm, EditFindingGroupForm, FindingBulkUpdateForm from dojo.jira import services as jira_services from dojo.models import Engagement, Finding, Finding_Group, GITHUB_PKey, Product diff --git a/dojo/forms.py b/dojo/forms.py index cb90d0f57de..c9075e07b6a 100644 --- a/dojo/forms.py +++ b/dojo/forms.py @@ -1,14 +1,8 @@ -import json import logging import re -import warnings from datetime import date, datetime from pathlib import Path -import tagulous -from crispy_forms.bootstrap import InlineCheckboxes, InlineRadios -from crispy_forms.helper import FormHelper -from crispy_forms.layout import Layout from crum import get_current_user from dateutil.relativedelta import relativedelta from django import forms @@ -16,21 +10,15 @@ from django.contrib.auth.models import Permission from django.contrib.auth.password_validation import validate_password from django.core.exceptions import ValidationError -from django.core.validators import URLValidator -from django.db.models import Count from django.forms import modelformset_factory from django.forms.widgets import Select, Widget from django.utils import timezone from django.utils.dates import MONTHS from django.utils.safestring import mark_safe from django.utils.translation import gettext_lazy as _ -from polymorphic.base import ManagerInheritanceWarning from tagulous.forms import TagField -from dojo.authorization.authorization import user_has_configuration_permission, user_is_superuser_or_global_owner -from dojo.endpoint.utils import endpoint_filter, endpoint_get_or_create, validate_endpoints_to_add -from dojo.engagement.queries import get_authorized_engagements -from dojo.finding.queries import get_authorized_findings +from dojo.endpoint.utils import validate_endpoints_to_add from dojo.github.ui.forms import ( # noqa: F401 -- backward compat DeleteGITHUBConfForm, ExpressGITHUBForm, @@ -39,7 +27,6 @@ GITHUBFindingForm, GITHUBForm, ) -from dojo.jira import services as jira_services from dojo.jira.forms import ( # noqa: F401 backward compat JIRA_TEMPLATE_CHOICES, AdvancedJIRAForm, @@ -57,65 +44,30 @@ from dojo.location.models import Location from dojo.location.utils import validate_locations_to_add from dojo.models import ( - EFFORT_FOR_FIXING_CHOICES, SEVERITY_CHOICES, - Announcement, - Answered_Survey, App_Analysis, - Benchmark_Product, - Benchmark_Product_Summary, - Benchmark_Requirement, Check_List, - Choice, - ChoiceAnswer, - ChoiceQuestion, Development_Environment, Dojo_User, DojoMeta, Endpoint, - Engagement, - Engagement_Presets, - Engagement_Survey, FileUpload, - Finding, Finding_Group, - Finding_Template, - General_Survey, - Note_Type, - Notes, - Objects_Product, - Product, Product_API_Scan_Configuration, Product_Type, - Question, - Regulation, - Risk_Acceptance, SLA_Configuration, - System_Settings, - Test, Test_Type, - TextAnswer, - TextQuestion, - Tool_Configuration, - Tool_Product_Settings, - Tool_Type, User, - UserContactInfo, ) -from dojo.product.queries import get_authorized_products from dojo.product_type.queries import get_authorized_product_types from dojo.tools.factory import get_choices_sorted, requires_file, requires_tool_type -from dojo.user.queries import get_authorized_users, get_authorized_users_for_product_and_product_type from dojo.user.utils import get_configuration_permissions_fields from dojo.utils import ( get_password_requirements_string, - get_product, - get_system_setting, is_finding_groups_enabled, is_scan_file_too_large, ) -from dojo.validators import ImporterFileExtensionValidator, cvss3_validator, cvss4_validator, tag_validator -from dojo.widgets import TableCheckboxWidget +from dojo.validators import ImporterFileExtensionValidator, tag_validator logger = logging.getLogger(__name__) @@ -128,46 +80,6 @@ ("duplicate", "Duplicate"), ("out_of_scope", "Out of Scope")) -CVSS_CALCULATOR_URLS = { - "https://www.first.org/cvss/calculator/3-0": "CVSS3 Calculator by FIRST", - "https://www.first.org/cvss/calculator/4-0": "CVSS4 Calculator by FIRST", - "https://www.metaeffekt.com/security/cvss/calculator/": "CVSS2/3/4 Calculator by Metaeffekt", - } - - -vulnerability_ids_field = forms.CharField(max_length=5000, - required=False, - label="Vulnerability Ids", - help_text="Ids of vulnerabilities in security advisories associated with this finding. Can be Common Vulnerabilities and Exposures (CVE) or from other sources." - "You may enter one vulnerability id per line.", - widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) - -EFFORT_FOR_FIXING_INVALID_CHOICE = _("Select valid choice: Low,Medium,High") - - -class BulletListDisplayWidget(forms.Widget): - def __init__(self, urls_dict=None, *args, **kwargs): - self.urls_dict = urls_dict or {} - super().__init__(*args, **kwargs) - - def render(self, name, value, attrs=None, renderer=None): - if not self.urls_dict: - return "" - - html = '
    ' - for url, text in self.urls_dict.items(): - html += f'
  • {text}
    • ' - html += "
" - return mark_safe(html) - - -class MultipleSelectWithPop(forms.SelectMultiple): - def render(self, name, *args, **kwargs): - html = super().render(name, *args, **kwargs) - popup_plus = '
' + html + '
' - - return mark_safe(popup_plus) - class MonthYearWidget(Widget): @@ -243,44 +155,7 @@ def value_from_datadict(self, data, files, name): return data.get(name, None) -class Product_TypeForm(forms.ModelForm): - description = forms.CharField(widget=forms.Textarea(attrs={}), - required=False) - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.fields["critical_product"].label = labels.ORG_CRITICAL_PRODUCT_LABEL - self.fields["key_product"].label = labels.ORG_KEY_PRODUCT_LABEL - - class Meta: - model = Product_Type - fields = ["name", "description", "critical_product", "key_product"] - - -class Delete_Product_TypeForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Product_Type - fields = ["id"] - - -class Add_Product_Type_AuthorizedUsersForm(forms.Form): - users = forms.ModelMultipleChoiceField( - queryset=Dojo_User.objects.none(), required=True, label="Users", - ) - - def __init__(self, *args, product_type=None, **kwargs): - super().__init__(*args, **kwargs) - self.product_type = product_type - current = product_type.authorized_users.values_list("pk", flat=True) - self.fields["users"].queryset = ( - Dojo_User.objects.filter(is_active=True) - .exclude(is_superuser=True) - .exclude(pk__in=current) - .order_by("first_name", "last_name") - ) +from dojo.product_type.ui.forms import Add_Product_Type_AuthorizedUsersForm, Delete_Product_TypeForm, Product_TypeForm # noqa: E402, F401, I001 class Test_TypeForm(forms.ModelForm): @@ -300,134 +175,29 @@ def clean_name(self): return self.cleaned_data["name"] -class Development_EnvironmentForm(forms.ModelForm): - class Meta: - model = Development_Environment - fields = ["name"] - - -class Delete_Dev_EnvironmentForm(forms.ModelForm): - class Meta: - model = Development_Environment - fields = ["id"] - - -class ProductForm(forms.ModelForm): - name = forms.CharField(max_length=255, required=True) - description = forms.CharField(widget=forms.Textarea(attrs={}), - required=True) - - prod_type = forms.ModelChoiceField(label=labels.ORG_LABEL, - queryset=Product_Type.objects.none(), - required=True) - - sla_configuration = forms.ModelChoiceField(label="SLA Configuration", - queryset=SLA_Configuration.objects.all(), - required=True, - initial="Default") - - product_manager = forms.ModelChoiceField(label=labels.ASSET_MANAGER_LABEL, - queryset=Dojo_User.objects.exclude(is_active=False).order_by("first_name", "last_name"), required=False) - technical_contact = forms.ModelChoiceField(queryset=Dojo_User.objects.exclude(is_active=False).order_by("first_name", "last_name"), required=False) - team_manager = forms.ModelChoiceField(queryset=Dojo_User.objects.exclude(is_active=False).order_by("first_name", "last_name"), required=False) - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.fields["prod_type"].queryset = get_authorized_product_types("add") - self.fields["enable_product_tag_inheritance"].label = labels.ASSET_TAG_INHERITANCE_ENABLE_LABEL - self.fields["enable_product_tag_inheritance"].help_text = labels.ASSET_TAG_INHERITANCE_ENABLE_HELP - if prod_type_id := kwargs.get("instance", Product()).prod_type_id: # we are editing existing instance - self.fields["prod_type"].queryset |= Product_Type.objects.filter(pk=prod_type_id) # even if user does not have permission for any other ProdType we need to add at least assign ProdType to make form submittable (otherwise empty list was here which generated invalid form) - - # if this product has findings being asynchronously updated, disable the sla config field - if self.instance.async_updating: - self.fields["sla_configuration"].disabled = True - self.fields["sla_configuration"].widget.attrs["message"] = ( - "Finding SLA expiration dates are currently being recalculated. " - "This field cannot be changed until the calculation is complete." - ) - - class Meta: - model = Product - fields = ["name", "description", "tags", "product_manager", "technical_contact", "team_manager", "prod_type", "sla_configuration", "regulations", - "business_criticality", "platform", "lifecycle", "origin", "user_records", "revenue", "external_audience", "enable_product_tag_inheritance", - "internet_accessible", "enable_simple_risk_acceptance", "enable_full_risk_acceptance", "disable_sla_breach_notifications"] - - def clean_tags(self): - tag_validator(self.cleaned_data.get("tags")) - return self.cleaned_data.get("tags") - - -class DeleteProductForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Product - fields = ["id"] - - -class EditFindingGroupForm(forms.ModelForm): - name = forms.CharField(max_length=255, required=True, label="Finding Group Name") - jira_issue = forms.CharField(max_length=255, required=False, label="Linked JIRA Issue", - help_text="Leave empty and check push to jira to create a new JIRA issue for this finding group.") - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.fields["push_to_jira"] = forms.BooleanField() - self.fields["push_to_jira"].required = False - self.fields["push_to_jira"].help_text = "Checking this will overwrite content of your JIRA issue, or create one." - - self.fields["push_to_jira"].label = "Push to JIRA" - - if hasattr(self.instance, "has_jira_issue") and self.instance.has_jira_issue: - jira_url = jira_services.get_url(self.instance) - self.fields["jira_issue"].initial = jira_url - self.fields["push_to_jira"].widget.attrs["checked"] = "checked" - - class Meta: - model = Finding_Group - fields = ["name"] - - -class DeleteFindingGroupForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Finding_Group - fields = ["id"] - - -class Add_Product_AuthorizedUsersForm(forms.Form): - users = forms.ModelMultipleChoiceField( - queryset=Dojo_User.objects.none(), required=True, label="Users", - ) - - def __init__(self, *args, product=None, **kwargs): - super().__init__(*args, **kwargs) - self.product = product - current = product.authorized_users.values_list("pk", flat=True) - self.fields["users"].queryset = ( - Dojo_User.objects.filter(is_active=True) - .exclude(is_superuser=True) - .exclude(pk__in=current) - .order_by("first_name", "last_name") - ) - - -class Authorize_User_For_ProductsForm(forms.Form): - products = forms.ModelMultipleChoiceField( - queryset=Product.objects.none(), required=True, label=labels.ASSET_PLURAL_LABEL, - ) +from dojo.development_environment.ui.forms import ( # noqa: E402, F401 -- re-export + Delete_Dev_EnvironmentForm, + Development_EnvironmentForm, +) - def __init__(self, *args, user=None, **kwargs): - super().__init__(*args, **kwargs) - self.user = user - # Show products the user is not already directly authorized for. - self.fields["products"].queryset = ( - Product.objects.exclude(authorized_users=user).order_by("name") - ) +# Re-exported for external consumers (finding_group/test/engagement/product views + unittests). +# The remaining finding forms live only in dojo.finding.ui.forms and are imported there by finding's own views. +from dojo.finding.ui.forms import ( # noqa: E402, F401 -- backward compat + AddFindingForm, + AddFindingsRiskAcceptanceForm, + AdHocFindingForm, + DeleteFindingGroupForm, + EditFindingGroupForm, + FindingBulkUpdateForm, + FindingForm, +) +from dojo.user.ui.forms import ( # noqa: E402, F401 -- backward compat re-export + AddDojoUserForm, + DeleteUserForm, + DojoUserForm, + EditDojoUserForm, + UserContactInfoForm, +) class Authorize_User_For_ProductTypesForm(forms.Form): @@ -443,36 +213,11 @@ def __init__(self, *args, user=None, **kwargs): ) -class NoteTypeForm(forms.ModelForm): - description = forms.CharField(widget=forms.Textarea(attrs={}), - required=True) - - class Meta: - model = Note_Type - fields = ["name", "description", "is_single", "is_mandatory"] - - -class EditNoteTypeForm(NoteTypeForm): - - def __init__(self, *args, **kwargs): - is_single = kwargs.pop("is_single") - super().__init__(*args, **kwargs) - if is_single is False: - self.fields["is_single"].widget = forms.HiddenInput() - - -class DisableOrEnableNoteTypeForm(NoteTypeForm): - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.fields["name"].disabled = True - self.fields["description"].disabled = True - self.fields["is_single"].disabled = True - self.fields["is_mandatory"].disabled = True - self.fields["is_active"].disabled = True - - class Meta: - model = Note_Type - fields = "__all__" +from dojo.note_type.ui.forms import ( # noqa: E402, F401 -- backward compat + DisableOrEnableNoteTypeForm, + EditNoteTypeForm, + NoteTypeForm, +) class DojoMetaDataForm(forms.ModelForm): @@ -764,29 +509,12 @@ def clean_scan_date(self): return date -class ImportEndpointMetaForm(forms.Form): - file = forms.FileField(widget=forms.widgets.FileInput( - attrs={"accept": ".csv"}), - label="Choose meta file", - required=True) # Could not get required=True to actually accept the file as present - create_endpoints = forms.BooleanField( - label="Create nonexisting Endpoint", - initial=True, - required=False, - help_text="Create endpoints that do not already exist") - create_tags = forms.BooleanField( - label="Add Tags", - initial=True, - required=False, - help_text="Add meta from file as tags in the format key:value") - create_dojo_meta = forms.BooleanField( - label="Add Meta", - initial=False, - required=False, - help_text="Add data from file as Metadata. Metadata is used for displaying custom fields") - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) +from dojo.endpoint.ui.forms import ( # noqa: E402, F401 -- backward compat re-export + AddEndpointForm, + DeleteEndpointForm, + EditEndpointForm, + ImportEndpointMetaForm, +) class DoneForm(forms.Form): @@ -817,111 +545,6 @@ def clean(self): raise ValidationError(msg) -class MergeFindings(forms.ModelForm): - FINDING_ACTION = (("", "Select an Action"), ("inactive", "Inactive"), ("delete", "Delete")) - - append_description = forms.BooleanField(label="Append Description", initial=True, required=False, - help_text="Description in all findings will be appended into the merged finding.") - - add_endpoints = forms.BooleanField(label="Add Endpoints", initial=True, required=False, - help_text="Endpoints in all findings will be merged into the merged finding.") - - dynamic_raw = forms.BooleanField(label="Dynamic Scanner Raw Requests", initial=True, required=False, - help_text="Dynamic scanner raw requests in all findings will be merged into the merged finding.") - - tag_finding = forms.BooleanField(label="Add Tags", initial=True, required=False, - help_text="Tags in all findings will be merged into the merged finding.") - - mark_tag_finding = forms.BooleanField(label="Tag Merged Finding", initial=True, required=False, - help_text="Creates a tag titled 'merged' for the finding that will be merged. If the 'Finding Action' is set to 'inactive' the inactive findings will be tagged with 'merged-inactive'.") - - append_reference = forms.BooleanField(label="Append Reference", initial=True, required=False, - help_text="Reference in all findings will be appended into the merged finding.") - - finding_action = forms.ChoiceField( - required=True, - choices=FINDING_ACTION, - label="Finding Action", - help_text="The action to take on the merged finding. Set the findings to inactive or delete the findings.") - - def __init__(self, *args, **kwargs): - _ = kwargs.pop("finding") - findings = kwargs.pop("findings") - super().__init__(*args, **kwargs) - - self.fields["finding_to_merge_into"] = forms.ModelChoiceField( - queryset=findings, initial=0, required="False", label="Finding to Merge Into", help_text="Findings selected below will be merged into this finding.") - - # Exclude the finding to merge into from the findings to merge into - self.fields["findings_to_merge"] = forms.ModelMultipleChoiceField( - queryset=findings, required=True, label="Findings to Merge", - widget=forms.widgets.SelectMultiple(attrs={"size": 10}), - help_text=("Select the findings to merge.")) - self.field_order = ["finding_to_merge_into", "findings_to_merge", "append_description", "add_endpoints", "append_reference"] - - class Meta: - model = Finding - fields = ["append_description", "add_endpoints", "append_reference"] - - -class EditRiskAcceptanceForm(forms.ModelForm): - # unfortunately django forces us to repeat many things here. choices, default, required etc. - recommendation = forms.ChoiceField(choices=Risk_Acceptance.TREATMENT_CHOICES, initial=Risk_Acceptance.TREATMENT_ACCEPT, widget=forms.RadioSelect, label="Security Recommendation") - decision = forms.ChoiceField(choices=Risk_Acceptance.TREATMENT_CHOICES, initial=Risk_Acceptance.TREATMENT_ACCEPT, widget=forms.RadioSelect) - - path = forms.FileField(label="Proof", required=False, widget=forms.widgets.FileInput(attrs={"accept": ", ".join(settings.FILE_IMPORT_TYPES)})) - expiration_date = forms.DateTimeField(required=False, widget=forms.TextInput(attrs={"class": "datepicker"})) - - class Meta: - model = Risk_Acceptance - exclude = ["accepted_findings", "notes"] - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.fields["path"].help_text = f"Existing proof uploaded: {self.instance.filename()}" if self.instance.filename() else "None" - self.fields["expiration_date_warned"].disabled = True - self.fields["expiration_date_handled"].disabled = True - - def clean_path(self): - if (data := self.cleaned_data.get("path")) is not None: - ext = Path(data.name).suffix # [0] returns path+filename - valid_extensions = settings.FILE_UPLOAD_TYPES - if ext.lower() not in valid_extensions: - if accepted_extensions := f"{', '.join(valid_extensions)}": - msg = f"Unsupported extension. Supported extensions are as follows: {accepted_extensions}" - else: - msg = "File uploads are prohibited due to the list of acceptable file extensions being empty" - raise ValidationError(msg) - return data - - -class RiskAcceptanceForm(EditRiskAcceptanceForm): - accepted_findings = forms.ModelMultipleChoiceField( - queryset=Finding.objects.none(), required=True, - widget=forms.widgets.SelectMultiple(attrs={"size": 10}), - help_text=("Active, verified findings listed, please select to add findings.")) - notes = forms.CharField(required=False, max_length=2400, - widget=forms.Textarea, - label="Notes") - - class Meta: - model = Risk_Acceptance - fields = "__all__" - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - expiration_delta_days = get_system_setting("risk_acceptance_form_default_days") - logger.debug("expiration_delta_days: %i", expiration_delta_days) - if expiration_delta_days > 0: - expiration_date = timezone.now().date() + relativedelta(days=expiration_delta_days) - # logger.debug('setting default expiration_date: %s', expiration_date) - self.fields["expiration_date"].initial = expiration_date - # self.fields['path'].help_text = 'Existing proof uploaded: %s' % self.instance.filename() if self.instance.filename() else 'None' - self.fields["accepted_findings"].queryset = get_authorized_findings("edit") - if disclaimer := get_system_setting("disclaimer_notes"): - self.disclaimer = disclaimer.strip() - - class BaseManageFileFormSet(forms.BaseModelFormSet): def clean(self): """Validate the IP/Mask combo is in CIDR format""" @@ -951,30 +574,13 @@ def clean(self): ManageFileFormSet = modelformset_factory(FileUpload, extra=3, max_num=10, fields=["title", "file"], can_delete=True, formset=BaseManageFileFormSet) -class ReplaceRiskAcceptanceProofForm(forms.ModelForm): - path = forms.FileField(label="Proof", required=True, widget=forms.widgets.FileInput(attrs={"accept": ".jpg,.png,.pdf"})) - - class Meta: - model = Risk_Acceptance - fields = ["path"] - - -class AddFindingsRiskAcceptanceForm(forms.ModelForm): - - accepted_findings = forms.ModelMultipleChoiceField( - queryset=Finding.objects.none(), - required=True, - label="", - widget=TableCheckboxWidget(attrs={"size": 25}), - ) - - class Meta: - model = Risk_Acceptance - fields = ["accepted_findings"] - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.fields["accepted_findings"].queryset = get_authorized_findings("edit") +# Risk acceptance forms live in dojo/risk_acceptance/ui/forms.py. Re-exported here for +# backward compat — engagement's UI views import them from dojo.forms. +from dojo.risk_acceptance.ui.forms import ( # noqa: E402, F401 -- backward compat + EditRiskAcceptanceForm, + ReplaceRiskAcceptanceProofForm, + RiskAcceptanceForm, +) class CheckForm(forms.ModelForm): @@ -1009,1289 +615,95 @@ class Meta: "sensitive_data", "sensitive_issues", "other", "other_issues"] -class EngForm(forms.ModelForm): - name = forms.CharField( - max_length=300, required=False, - help_text=( - "Add a descriptive name to identify this engagement. " - "Without a name the target start date will be set." - )) - description = forms.CharField(widget=forms.Textarea(attrs={}), - required=False, help_text="Description of the engagement and details regarding the engagement.") - product = forms.ModelChoiceField(label=labels.ASSET_LABEL, - queryset=Product.objects.none(), - required=True) - target_start = forms.DateField(widget=forms.TextInput( - attrs={"class": "datepicker", "autocomplete": "off"})) - target_end = forms.DateField(widget=forms.TextInput( - attrs={"class": "datepicker", "autocomplete": "off"})) - lead = forms.ModelChoiceField( - queryset=None, - required=True, label="Testing Lead") - test_strategy = forms.URLField(required=False, label="Test Strategy URL") - - def __init__(self, *args, **kwargs): - cicd = False - product = None - if "cicd" in kwargs: - cicd = kwargs.pop("cicd") +# Engagement forms live in dojo/engagement/ui/forms.py. Re-exported here for +# backward compat. DeleteEngagementForm has no external consumers, so it is not +# re-exported (imported directly from dojo.engagement.ui.forms by its only user). +from dojo.engagement.ui.forms import ( # noqa: E402, F401 -- backward compat + AddEngagementForm, + DeleteEngagementPresetsForm, + EngagementPresetsForm, + EngForm, + ExistingEngagementForm, +) +from dojo.notes.ui.forms import ( # noqa: E402, F401 -- backward compat + DeleteNoteForm, + NoteForm, + TypedNoteForm, +) +from dojo.test.ui.forms import TestForm # noqa: E402, F401 -- backward compat - if "product" in kwargs: - product = kwargs.pop("product") - self.user = None - if "user" in kwargs: - self.user = kwargs.pop("user") +class WeeklyMetricsForm(forms.Form): + dates = forms.ChoiceField() + def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) + wmf_options = [] - if product: - self.fields["preset"] = forms.ModelChoiceField(help_text="Settings and notes for performing this engagement.", required=False, queryset=Engagement_Presets.objects.filter(product=product)) - self.fields["lead"].queryset = get_authorized_users_for_product_and_product_type(None, product, "view").filter(is_active=True) - else: - self.fields["lead"].queryset = get_authorized_users("view").filter(is_active=True) - - self.fields["product"].queryset = get_authorized_products("add") - - # Don't show CICD fields on a interactive engagement - if cicd is False: - del self.fields["build_id"] - del self.fields["commit_hash"] - del self.fields["branch_tag"] - del self.fields["build_server"] - del self.fields["source_code_management_server"] - # del self.fields['source_code_management_uri'] - del self.fields["orchestration_engine"] - else: - del self.fields["test_strategy"] - del self.fields["status"] - - def is_valid(self): - valid = super().is_valid() - - # we're done now if not valid - if not valid: - return valid - if self.cleaned_data["target_start"] > self.cleaned_data["target_end"]: - self.add_error("target_start", "Your target start date exceeds your target end date") - self.add_error("target_end", "Your target start date exceeds your target end date") - return False - return True - - def clean_tags(self): - tag_validator(self.cleaned_data.get("tags")) - return self.cleaned_data.get("tags") + for i in range(6): + # Weeks start on Monday + curr = datetime.now() - relativedelta(weeks=i) + start_of_period = curr - relativedelta(weeks=1, weekday=0, + hour=0, minute=0, second=0) + end_of_period = curr + relativedelta(weeks=0, weekday=0, + hour=0, minute=0, second=0) - class Meta: - model = Engagement - exclude = ("first_contacted", "real_start", "engagement_type", "inherited_tags", - "real_end", "requester", "reason", "updated", "report_type", - "product", "threat_model", "api_test", "pen_test", "check_list") + wmf_options.append((end_of_period.strftime("%b %d %Y %H %M %S %Z"), + start_of_period.strftime("%b %d") + + " - " + end_of_period.strftime("%b %d"))) + wmf_options = tuple(wmf_options) -class DeleteEngagementForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) + self.fields["dates"].choices = wmf_options - class Meta: - model = Engagement - fields = ["id"] +class SimpleMetricsForm(forms.Form): + date = forms.DateField( + label="", + widget=MonthYearWidget()) -class TestForm(forms.ModelForm): - title = forms.CharField(max_length=255, required=False) - description = forms.CharField(widget=forms.Textarea(attrs={"rows": "3"}), required=False) - test_type = forms.ModelChoiceField(queryset=Test_Type.objects.all().order_by("name")) - environment = forms.ModelChoiceField( - queryset=Development_Environment.objects.all().order_by("name")) - target_start = forms.DateTimeField(widget=forms.TextInput( - attrs={"class": "datepicker", "autocomplete": "off"})) - target_end = forms.DateTimeField(widget=forms.TextInput( - attrs={"class": "datepicker", "autocomplete": "off"})) - lead = forms.ModelChoiceField( - queryset=None, - required=False, label="Testing Lead") - def __init__(self, *args, **kwargs): - obj = None +class SimpleSearchForm(forms.Form): + query = forms.CharField(required=False) - if "engagement" in kwargs: - obj = kwargs.pop("engagement") - if "instance" in kwargs: - obj = kwargs.get("instance") +class DateRangeMetrics(forms.Form): + start_date = forms.DateField(required=True, label="To", + widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) + end_date = forms.DateField(required=True, + label="From", + widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) - super().__init__(*args, **kwargs) - if obj: - product = get_product(obj) - self.fields["lead"].queryset = get_authorized_users_for_product_and_product_type(None, product, "view").filter(is_active=True) - self.fields["api_scan_configuration"].queryset = Product_API_Scan_Configuration.objects.filter(product=product) - else: - self.fields["lead"].queryset = get_authorized_users("view").filter(is_active=True) - - def is_valid(self): - valid = super().is_valid() - - # we're done now if not valid - if not valid: - return valid - if self.cleaned_data["target_start"] > self.cleaned_data["target_end"]: - self.add_error("target_start", "Your target start date exceeds your target end date") - self.add_error("target_end", "Your target start date exceeds your target end date") - return False - return True +class MetricsFilterForm(forms.Form): + start_date = forms.DateField(required=False, + label="To", + widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) + end_date = forms.DateField(required=False, + label="From", + widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) + finding_status = forms.MultipleChoiceField( + required=False, + widget=forms.CheckboxSelectMultiple, + choices=FINDING_STATUS, + label="Status") + severity = forms.MultipleChoiceField(required=False, + choices=(("Low", "Low"), + ("Medium", "Medium"), + ("High", "High"), + ("Critical", "Critical")), + help_text=('Hold down "Control", or ' + '"Command" on a Mac, to ' + 'select more than one.')) + exclude_product_types = forms.ModelMultipleChoiceField( + required=False, queryset=Product_Type.objects.all().order_by("name")) - class Meta: - model = Test - fields = ["title", "test_type", "target_start", "target_end", "description", - "environment", "percent_complete", "tags", "lead", "version", "branch_tag", "build_id", "commit_hash", - "api_scan_configuration"] - - def clean_tags(self): - tag_validator(self.cleaned_data.get("tags")) - return self.cleaned_data.get("tags") - - -class DeleteTestForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Test - fields = ["id"] - - -class CopyTestForm(forms.Form): - engagement = forms.ModelChoiceField( - required=True, - queryset=Engagement.objects.none(), - error_messages={"required": "*"}) - - def __init__(self, *args, **kwargs): - authorized_lists = kwargs.pop("engagements", None) - super().__init__(*args, **kwargs) - self.fields["engagement"].queryset = authorized_lists - - -class AddFindingForm(forms.ModelForm): - title = forms.CharField(max_length=1000) - date = forms.DateField(required=True, - widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) - cwe = forms.IntegerField(required=False) - vulnerability_ids = vulnerability_ids_field - cvssv3 = forms.CharField(label="CVSS3 Vector", max_length=117, required=False, widget=forms.TextInput(attrs={"class": "cvsscalculator", "data-toggle": "dropdown", "aria-haspopup": "true", "aria-expanded": "false"})) - cvssv3_score = forms.FloatField(label="CVSS3 Score", required=False, max_value=10.0, min_value=0.0) - cvssv4 = forms.CharField(label="CVSS4 Vector", max_length=255, required=False) - cvssv4_score = forms.FloatField(label="CVSS4 Score", required=False, max_value=10.0, min_value=0.0) - description = forms.CharField(widget=forms.Textarea) - severity = forms.ChoiceField( - choices=SEVERITY_CHOICES, - error_messages={ - "required": "Select valid choice: In Progress, On Hold, Completed", - "invalid_choice": EFFORT_FOR_FIXING_INVALID_CHOICE}) - mitigation = forms.CharField(widget=forms.Textarea, required=False) - impact = forms.CharField(widget=forms.Textarea, required=False) - request = forms.CharField(widget=forms.Textarea, required=False) - response = forms.CharField(widget=forms.Textarea, required=False) - endpoints = forms.ModelMultipleChoiceField(Location.objects.none(), required=False, label="Systems / Endpoints") - endpoints_to_add = forms.CharField(max_length=5000, required=False, label="Endpoints to add", - help_text="The IP address, host name or full URL. You may enter one endpoint per line. " - "Each must be valid.", - widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) - references = forms.CharField(widget=forms.Textarea, required=False) - publish_date = forms.DateField(widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"}), required=False) - planned_remediation_date = forms.DateField(widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"}), required=False) - planned_remediation_version = forms.CharField(max_length=99, required=False) - effort_for_fixing = forms.ChoiceField( - required=False, - choices=EFFORT_FOR_FIXING_CHOICES, - error_messages={ - "invalid_choice": EFFORT_FOR_FIXING_INVALID_CHOICE}) - - # the only reliable way without hacking internal fields to get predicatble ordering is to make it explicit - field_order = ("title", "date", "cwe", "vulnerability_ids", "severity", "cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", "description", "mitigation", "impact", "request", "response", "steps_to_reproduce", - "severity_justification", "endpoints", "endpoints_to_add", "references", "active", "verified", "false_p", "duplicate", "out_of_scope", - "risk_accepted", "under_defect_review") - - def __init__(self, *args, **kwargs): - req_resp = kwargs.pop("req_resp") - - product = None - if "product" in kwargs: - product = kwargs.pop("product") - - super().__init__(*args, **kwargs) - - if settings.V3_FEATURE_LOCATIONS and product: - self.fields["endpoints"].queryset = Location.objects.filter(products__product=product) - # TODO: Delete this after the move to Locations - elif product: - self.fields["endpoints"].queryset = Endpoint.objects.filter(product=product) - else: - self.fields["endpoints"].queryset = Endpoint.objects.none() - - if req_resp: - self.fields["request"].initial = req_resp[0] - self.fields["response"].initial = req_resp[1] - - self.endpoints_to_add_list = [] - - # Hide CVSS fields based on system settings - hide_cvss_fields_if_disabled(self) - - def clean(self): - cleaned_data = super().clean() - if ((cleaned_data["active"] or cleaned_data["verified"]) and cleaned_data["duplicate"]): - msg = "Duplicate findings cannot be verified or active" - raise forms.ValidationError(msg) - if cleaned_data["false_p"] and cleaned_data["verified"]: - msg = "False positive findings cannot be verified." - raise forms.ValidationError(msg) - if cleaned_data["active"] and "risk_accepted" in cleaned_data and cleaned_data["risk_accepted"]: - msg = "Active findings cannot be risk accepted." - raise forms.ValidationError(msg) - - if settings.V3_FEATURE_LOCATIONS: - endpoints_to_add_list, errors = validate_locations_to_add(cleaned_data["endpoints_to_add"]) - else: - # TODO: Delete this after the move to Locations - endpoints_to_add_list, errors = validate_endpoints_to_add(cleaned_data["endpoints_to_add"]) - - if errors: - raise forms.ValidationError(errors) - self.endpoints_to_add_list = endpoints_to_add_list - - return cleaned_data - - def clean_tags(self): - tag_validator(self.cleaned_data.get("tags")) - return self.cleaned_data.get("tags") - - class Meta: - model = Finding - exclude = ("reporter", "url", "numerical_severity", "under_review", "reviewers", "cve", "inherited_tags", - "review_requested_by", "is_mitigated", "jira_creation", "jira_change", "endpoints", "sla_start_date") - - -class AdHocFindingForm(forms.ModelForm): - title = forms.CharField(max_length=1000) - date = forms.DateField(required=True, - widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) - cwe = forms.IntegerField(required=False) - vulnerability_ids = vulnerability_ids_field - - cvss_info = forms.CharField( - label="CVSS", - widget=BulletListDisplayWidget(CVSS_CALCULATOR_URLS), - required=False, - disabled=True) - - cvssv3 = forms.CharField(label="CVSS3 Vector", max_length=117, required=False, widget=forms.TextInput(attrs={"class": "cvsscalculator", "data-toggle": "dropdown", "aria-haspopup": "true", "aria-expanded": "false"})) - cvssv3_score = forms.FloatField(label="CVSS3 Score", required=False, max_value=10.0, min_value=0.0) - cvssv4 = forms.CharField(label="CVSS4 Vector", max_length=255, required=False) - cvssv4_score = forms.FloatField(label="CVSS4 Score", required=False, max_value=10.0, min_value=0.0) - description = forms.CharField(widget=forms.Textarea) - severity = forms.ChoiceField( - choices=SEVERITY_CHOICES, - error_messages={ - "required": "Select valid choice: In Progress, On Hold, Completed", - "invalid_choice": EFFORT_FOR_FIXING_INVALID_CHOICE}) - mitigation = forms.CharField(widget=forms.Textarea, required=False) - impact = forms.CharField(widget=forms.Textarea, required=False) - request = forms.CharField(widget=forms.Textarea, required=False) - response = forms.CharField(widget=forms.Textarea, required=False) - endpoints = forms.ModelMultipleChoiceField(queryset=Location.objects.all(), required=False, - label="Systems / Endpoints") - endpoints_to_add = forms.CharField(max_length=5000, required=False, label="Endpoints to add", - help_text="The IP address, host name or full URL. You may enter one endpoint per line. " - "Each must be valid.", - widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) - references = forms.CharField(widget=forms.Textarea, required=False) - publish_date = forms.DateField(widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"}), required=False) - planned_remediation_date = forms.DateField(widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"}), required=False) - planned_remediation_version = forms.CharField(max_length=99, required=False) - effort_for_fixing = forms.ChoiceField( - required=False, - choices=EFFORT_FOR_FIXING_CHOICES, - error_messages={ - "invalid_choice": EFFORT_FOR_FIXING_INVALID_CHOICE}) - - # the only reliable way without hacking internal fields to get predicatble ordering is to make it explicit - field_order = ("title", "date", "cwe", "vulnerability_ids", "severity", "cvss_info", "cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", "description", "mitigation", - "impact", "request", "response", "steps_to_reproduce", "severity_justification", "endpoints", "endpoints_to_add", "references", - "active", "verified", "false_p", "duplicate", "out_of_scope", "risk_accepted", "under_defect_review", "sla_start_date", "sla_expiration_date") - - def __init__(self, *args, **kwargs): - req_resp = kwargs.pop("req_resp") - - product = None - if "product" in kwargs: - product = kwargs.pop("product") - - super().__init__(*args, **kwargs) - - if settings.V3_FEATURE_LOCATIONS and product: - self.fields["endpoints"].queryset = Location.objects.filter(products__product=product) - # TODO: Delete this after the move to Locations - elif product: - self.fields["endpoints"].queryset = Endpoint.objects.filter(product=product) - else: - self.fields["endpoints"].queryset = Endpoint.objects.none() - - if req_resp: - self.fields["request"].initial = req_resp[0] - self.fields["response"].initial = req_resp[1] - - self.endpoints_to_add_list = [] - - # Hide CVSS fields based on system settings - hide_cvss_fields_if_disabled(self) - - def clean(self): - cleaned_data = super().clean() - if ((cleaned_data["active"] or cleaned_data["verified"]) and cleaned_data["duplicate"]): - msg = "Duplicate findings cannot be verified or active" - raise forms.ValidationError(msg) - if cleaned_data["false_p"] and cleaned_data["verified"]: - msg = "False positive findings cannot be verified." - raise forms.ValidationError(msg) - - if settings.V3_FEATURE_LOCATIONS: - endpoints_to_add_list, errors = validate_locations_to_add(cleaned_data["endpoints_to_add"]) - else: - # TODO: Delete this after the move to Locations - endpoints_to_add_list, errors = validate_endpoints_to_add(cleaned_data["endpoints_to_add"]) - - self.endpoints_to_add_list = endpoints_to_add_list - - if errors: - raise forms.ValidationError(errors) - - return cleaned_data - - def clean_tags(self): - tag_validator(self.cleaned_data.get("tags")) - return self.cleaned_data.get("tags") - - class Meta: - model = Finding - exclude = ("reporter", "url", "numerical_severity", "under_review", "reviewers", "cve", "inherited_tags", - "review_requested_by", "is_mitigated", "jira_creation", "jira_change", "endpoints", "sla_start_date", - "sla_expiration_date") - - -class PromoteFindingForm(forms.ModelForm): - title = forms.CharField(max_length=1000) - date = forms.DateField(required=True, - widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) - cwe = forms.IntegerField(required=False) - vulnerability_ids = vulnerability_ids_field - - cvss_info = forms.CharField( - label="CVSS", - widget=BulletListDisplayWidget(CVSS_CALCULATOR_URLS), - required=False, - disabled=True) - - cvssv3 = forms.CharField(label="CVSS3 Vector", max_length=117, required=False, widget=forms.TextInput(attrs={"class": "cvsscalculator", "data-toggle": "dropdown", "aria-haspopup": "true", "aria-expanded": "false"})) - cvssv3_score = forms.FloatField(label="CVSS3 Score", required=False, max_value=10.0, min_value=0.0) - cvssv4 = forms.CharField(label="CVSS4 Vector", max_length=255, required=False) - cvssv4_score = forms.FloatField(label="CVSS4 Score", required=False, max_value=10.0, min_value=0.0) - description = forms.CharField(widget=forms.Textarea) - severity = forms.ChoiceField( - choices=SEVERITY_CHOICES, - error_messages={ - "required": "Select valid choice: In Progress, On Hold, Completed", - "invalid_choice": "Select valid choice: Critical,High,Medium,Low"}) - mitigation = forms.CharField(widget=forms.Textarea, required=False) - impact = forms.CharField(widget=forms.Textarea, required=False) - endpoints = forms.ModelMultipleChoiceField(Location.objects.none(), required=False, label="Systems / Endpoints") - endpoints_to_add = forms.CharField(max_length=5000, required=False, label="Endpoints to add", - help_text="The IP address, host name or full URL. You may enter one endpoint per line. " - "Each must be valid.", - widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) - references = forms.CharField(widget=forms.Textarea, required=False) - - # the onyl reliable way without hacking internal fields to get predicatble ordering is to make it explicit - field_order = ("title", "group", "date", "sla_start_date", "sla_expiration_date", "cwe", "vulnerability_ids", "severity", "cvss_info", "cvssv3", - "cvssv3_score", "cvssv4", "cvssv4_score", "description", "mitigation", "impact", "request", "response", "steps_to_reproduce", - "severity_justification", "endpoints", "endpoints_to_add", "references", "active", "mitigated", "mitigated_by", "verified", - "false_p", "duplicate", "out_of_scope", "risk_accept", "under_defect_review") - - def __init__(self, *args, **kwargs): - product = None - if "product" in kwargs: - product = kwargs.pop("product") - - super().__init__(*args, **kwargs) - - if settings.V3_FEATURE_LOCATIONS and product: - self.fields["endpoints"].queryset = Location.objects.filter(products__product=product) - # TODO: Delete this after the move to Locations - elif product: - self.fields["endpoints"].queryset = Endpoint.objects.filter(product=product) - else: - self.fields["endpoints"].queryset = Endpoint.objects.none() - - self.endpoints_to_add_list = [] - - # Hide CVSS fields based on system settings - hide_cvss_fields_if_disabled(self) - - def clean(self): - cleaned_data = super().clean() - - if settings.V3_FEATURE_LOCATIONS: - endpoints_to_add_list, errors = validate_locations_to_add(cleaned_data["endpoints_to_add"]) - else: - # TODO: Delete this after the move to Locations - endpoints_to_add_list, errors = validate_endpoints_to_add(cleaned_data["endpoints_to_add"]) - - if errors: - raise forms.ValidationError(errors) - self.endpoints_to_add_list = endpoints_to_add_list - - return cleaned_data - - def clean_tags(self): - tag_validator(self.cleaned_data.get("tags")) - return self.cleaned_data.get("tags") - - class Meta: - model = Finding - exclude = ("reporter", "url", "numerical_severity", "active", "false_p", "verified", "endpoint_status", "cve", "inherited_tags", - "duplicate", "out_of_scope", "under_review", "reviewers", "review_requested_by", "is_mitigated", "jira_creation", "jira_change", "planned_remediation_date", "planned_remediation_version", "effort_for_fixing") - - -class FindingForm(forms.ModelForm): - title = forms.CharField(max_length=1000) - group = forms.ModelChoiceField(required=False, queryset=Finding_Group.objects.none(), help_text="The Finding Group to which this finding belongs, leave empty to remove the finding from the group. Groups can only be created via Bulk Edit for now.") - date = forms.DateField(required=True, - widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) - cwe = forms.IntegerField(required=False) - vulnerability_ids = vulnerability_ids_field - - cvss_info = forms.CharField( - label="CVSS", - widget=BulletListDisplayWidget(CVSS_CALCULATOR_URLS), - required=False, - disabled=True) - - cvssv3 = forms.CharField(label="CVSS3 Vector", max_length=117, required=False, widget=forms.TextInput(attrs={"class": "cvsscalculator", "data-toggle": "dropdown", "aria-haspopup": "true", "aria-expanded": "false"})) - cvssv3_score = forms.FloatField(label="CVSS3 Score", required=False, max_value=10.0, min_value=0.0) - cvssv4 = forms.CharField(label="CVSS4 Vector", max_length=255, required=False) - cvssv4_score = forms.FloatField(label="CVSS4 Score", required=False, max_value=10.0, min_value=0.0) - - description = forms.CharField(widget=forms.Textarea) - severity = forms.ChoiceField( - choices=SEVERITY_CHOICES, - error_messages={ - "required": "Select valid choice: In Progress, On Hold, Completed", - "invalid_choice": "Select valid choice: Critical,High,Medium,Low"}) - mitigation = forms.CharField(widget=forms.Textarea, required=False) - impact = forms.CharField(widget=forms.Textarea, required=False) - request = forms.CharField(widget=forms.Textarea, required=False) - response = forms.CharField(widget=forms.Textarea, required=False) - endpoints = forms.ModelMultipleChoiceField(queryset=Location.objects.none(), required=False, label="Systems / Endpoints") - endpoints_to_add = forms.CharField(max_length=5000, required=False, label="Endpoints to add", - help_text="The IP address, host name or full URL. You may enter one endpoint per line. " - "Each must be valid.", - widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) - references = forms.CharField(widget=forms.Textarea, required=False) - - mitigated = forms.DateField(required=False, help_text="Date and time when the flaw has been fixed", widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) - mitigated_by = forms.ModelChoiceField(required=False, queryset=Dojo_User.objects.none()) - - publish_date = forms.DateField(widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"}), required=False) - planned_remediation_date = forms.DateField(widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"}), required=False) - planned_remediation_version = forms.CharField(max_length=99, required=False) - effort_for_fixing = forms.ChoiceField( - required=False, - choices=EFFORT_FOR_FIXING_CHOICES, - error_messages={ - "invalid_choice": EFFORT_FOR_FIXING_INVALID_CHOICE}) - - # the only reliable way without hacking internal fields to get predicatble ordering is to make it explicit - field_order = ("title", "group", "date", "sla_start_date", "sla_expiration_date", "cwe", "vulnerability_ids", "severity", "cvss_info", "cvssv3", - "cvssv3_score", "cvssv4", "cvssv4_score", "description", "mitigation", "impact", "request", "response", "steps_to_reproduce", "severity_justification", - "endpoints", "endpoints_to_add", "references", "active", "mitigated", "mitigated_by", "verified", "false_p", "duplicate", - "out_of_scope", "risk_accept", "under_defect_review") - - def __init__(self, *args, **kwargs): - req_resp = None - if "req_resp" in kwargs: - req_resp = kwargs.pop("req_resp") - - self.can_edit_mitigated_data = kwargs.pop("can_edit_mitigated_data") if "can_edit_mitigated_data" in kwargs \ - else False - - super().__init__(*args, **kwargs) - - if settings.V3_FEATURE_LOCATIONS: - self.fields["endpoints"].queryset = Location.objects.filter(products__product=self.instance.test.engagement.product) - if self.instance and self.instance.pk: - self.fields["endpoints"].initial = Location.objects.filter(findings__finding=self.instance) - else: - # TODO: Delete this after the move to Locations - self.fields["endpoints"].queryset = Endpoint.objects.filter(product=self.instance.test.engagement.product) - if self.instance and self.instance.pk: - self.fields["endpoints"].initial = self.instance.endpoints.all() - - self.fields["mitigated_by"].queryset = get_authorized_users("edit") - - # do not show checkbox if finding is not accepted and simple risk acceptance is disabled - # if checked, always show to allow unaccept also with full risk acceptance enabled - # when adding from template, we don't have access to the test. quickfix for now to just hide simple risk acceptance - if not hasattr(self.instance, "test") or (not self.instance.risk_accepted and not self.instance.test.engagement.product.enable_simple_risk_acceptance): - del self.fields["risk_accepted"] - elif self.instance.risk_accepted: - self.fields["risk_accepted"].help_text = "Uncheck to unaccept the risk. Use full risk acceptance from the dropdown menu if you need advanced settings such as an expiry date." - elif self.instance.test.engagement.product.enable_simple_risk_acceptance: - self.fields["risk_accepted"].help_text = "Check to accept the risk. Use full risk acceptance from the dropdown menu if you need advanced settings such as an expiry date." - - # self.fields['tags'].widget.choices = t - if req_resp: - self.fields["request"].initial = req_resp[0] - self.fields["response"].initial = req_resp[1] - - if self.instance.duplicate: - self.fields["duplicate"].help_text = "Original finding that is being duplicated here (readonly). Use view finding page to manage duplicate relationships. Unchecking duplicate here will reset this findings duplicate status, but will trigger deduplication logic." - else: - self.fields["duplicate"].help_text = "You can mark findings as duplicate only from the view finding page." - - self.fields["sla_start_date"].disabled = True - self.fields["sla_expiration_date"].disabled = True - - if self.can_edit_mitigated_data: - if hasattr(self, "instance"): - self.fields["mitigated"].initial = self.instance.mitigated - self.fields["mitigated_by"].initial = self.instance.mitigated_by - else: - del self.fields["mitigated"] - del self.fields["mitigated_by"] - - if not is_finding_groups_enabled() or not hasattr(self.instance, "test"): - del self.fields["group"] - else: - self.fields["group"].queryset = self.instance.test.finding_group_set.all() - self.fields["group"].initial = self.instance.finding_group - - self.endpoints_to_add_list = [] - - # Hide CVSS fields based on system settings - hide_cvss_fields_if_disabled(self) - - def clean(self): - cleaned_data = super().clean() - - if (cleaned_data["active"] or cleaned_data["verified"]) and cleaned_data["duplicate"]: - msg = "Duplicate findings cannot be verified or active" - raise forms.ValidationError(msg) - if cleaned_data["false_p"] and cleaned_data["verified"]: - msg = "False positive findings cannot be verified." - raise forms.ValidationError(msg) - if cleaned_data["active"] and "risk_accepted" in cleaned_data and cleaned_data["risk_accepted"]: - msg = "Active findings cannot be risk accepted." - raise forms.ValidationError(msg) - - if settings.V3_FEATURE_LOCATIONS: - endpoints_to_add_list, errors = validate_locations_to_add(cleaned_data["endpoints_to_add"]) - else: - # TODO: Delete this after the move to Locations - endpoints_to_add_list, errors = validate_endpoints_to_add(cleaned_data["endpoints_to_add"]) - - self.endpoints_to_add_list = endpoints_to_add_list - - if errors: - raise forms.ValidationError(errors) - - return cleaned_data - - def clean_tags(self): - tag_validator(self.cleaned_data.get("tags")) - return self.cleaned_data.get("tags") - - def _post_clean(self): - super()._post_clean() - - if self.can_edit_mitigated_data: - opts = self.instance._meta - try: - opts.get_field("mitigated").save_form_data(self.instance, self.cleaned_data.get("mitigated")) - opts.get_field("mitigated_by").save_form_data(self.instance, self.cleaned_data.get("mitigated_by")) - except forms.ValidationError as e: - self._update_errors(e) - - class Meta: - model = Finding - exclude = ("reporter", "url", "numerical_severity", "under_review", "reviewers", "cve", "inherited_tags", - "review_requested_by", "is_mitigated", "jira_creation", "jira_change", "sonarqube_issue", - "endpoints", "endpoint_status") - - -class ApplyFindingTemplateForm(forms.Form): - - title = forms.CharField(max_length=1000, required=True) - - cwe = forms.IntegerField(label="CWE", required=False) - vulnerability_ids = vulnerability_ids_field - cvssv3 = forms.CharField(label="CVSSv3", max_length=117, required=False, widget=forms.TextInput(attrs={"class": "btn btn-secondary dropdown-toggle", "data-toggle": "dropdown", "aria-haspopup": "true", "aria-expanded": "false"})) - cvssv3_score = forms.FloatField(required=False, label="CVSSv3 Score") - cvssv4 = forms.CharField(label="CVSSv4", max_length=255, required=False) - cvssv4_score = forms.FloatField(required=False, label="CVSSv4 Score") - - severity = forms.ChoiceField(required=False, choices=SEVERITY_CHOICES, error_messages={"required": "Select valid choice: In Progress, On Hold, Completed", "invalid_choice": "Select valid choice: Critical,High,Medium,Low"}) - - description = forms.CharField(widget=forms.Textarea) - mitigation = forms.CharField(widget=forms.Textarea, required=False) - impact = forms.CharField(widget=forms.Textarea, required=False) - references = forms.CharField(widget=forms.Textarea, required=False) - - # Remediation planning fields - fix_available = forms.BooleanField(required=False) - fix_version = forms.CharField(max_length=100, required=False) - planned_remediation_version = forms.CharField(max_length=99, required=False) - effort_for_fixing = forms.CharField(max_length=99, required=False) - - # Technical details fields - steps_to_reproduce = forms.CharField(widget=forms.Textarea, required=False) - severity_justification = forms.CharField(widget=forms.Textarea, required=False) - component_name = forms.CharField(max_length=500, required=False) - component_version = forms.CharField(max_length=100, required=False) - - # Notes field - notes = forms.CharField(widget=forms.Textarea, required=False, help_text="Note content to add when applying template") - - # Endpoints field - endpoints = forms.CharField(max_length=5000, required=False, - help_text="Endpoint URLs (one per line)", - widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) - - tags = TagField(required=False, help_text="Add tags that help describe this finding template. Choose from the list or add new tags. Press Enter key to add.", initial=Finding.tags.tag_model.objects.all().order_by("name")) - - def __init__(self, template=None, *args, **kwargs): - super().__init__(*args, **kwargs) - self.fields["tags"].autocomplete_tags = Finding.tags.tag_model.objects.all().order_by("name") - self.template = template - if template: - # Populate vulnerability_ids field initial value - self.fields["vulnerability_ids"].initial = "\n".join(template.vulnerability_ids) - - # Populate CVSS fields from template - if hasattr(template, "cvssv3"): - self.fields["cvssv3"].initial = template.cvssv3 - if hasattr(template, "cvssv4"): - self.fields["cvssv4"].initial = template.cvssv4 - if hasattr(template, "cvssv3_score"): - self.fields["cvssv3_score"].initial = template.cvssv3_score - if hasattr(template, "cvssv4_score"): - self.fields["cvssv4_score"].initial = template.cvssv4_score - - # Populate all other new fields from template - for field_name in ["fix_available", "fix_version", "planned_remediation_version", - "effort_for_fixing", "steps_to_reproduce", "severity_justification", - "component_name", "component_version", "notes"]: - if hasattr(template, field_name): - value = getattr(template, field_name) - if value is not None: - self.fields[field_name].initial = value - - # Populate endpoints - if hasattr(template, "endpoints"): - endpoints_value = template.endpoints - if endpoints_value: - if isinstance(endpoints_value, list): - self.fields["endpoints"].initial = "\n".join(endpoints_value) - else: - self.fields["endpoints"].initial = endpoints_value - - # Hide CVSS fields based on system settings - hide_cvss_fields_if_disabled(self) - - def clean(self): - cleaned_data = super().clean() - - if "title" in cleaned_data: - if len(cleaned_data["title"]) <= 0: - msg = "The title is required." - raise forms.ValidationError(msg) - else: - msg = "The title is required." - raise forms.ValidationError(msg) - - return cleaned_data - - def clean_tags(self): - tag_validator(self.cleaned_data.get("tags")) - return self.cleaned_data.get("tags") - - class Meta: - fields = ["title", "cwe", "vulnerability_ids", "cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", - "severity", "description", "mitigation", "impact", "references", "tags", - "fix_available", "fix_version", "planned_remediation_version", "effort_for_fixing", - "steps_to_reproduce", "severity_justification", "component_name", "component_version", - "notes", "endpoints"] - order = ("title", "cwe", "vulnerability_ids", "cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", - "severity", "description", "impact", "steps_to_reproduce", "severity_justification", - "mitigation", "fix_available", "fix_version", "planned_remediation_version", - "effort_for_fixing", "component_name", "component_version", "references", "notes", - "endpoints", "tags") - - -class FindingTemplateForm(forms.ModelForm): - title = forms.CharField(max_length=1000, required=True) - - cwe = forms.IntegerField(label="CWE", required=False) - vulnerability_ids = vulnerability_ids_field - cvssv3 = forms.CharField(label="CVSS3 Vector", max_length=117, required=False, widget=forms.TextInput(attrs={"class": "btn btn-secondary dropdown-toggle", "data-toggle": "dropdown", "aria-haspopup": "true", "aria-expanded": "false"})) - cvssv3_score = forms.FloatField(required=False, label="CVSSv3 Score") - cvssv4 = forms.CharField(label="CVSS4 Vector", max_length=255, required=False) - cvssv4_score = forms.FloatField(required=False, label="CVSSv4 Score") - severity = forms.ChoiceField( - required=False, - choices=SEVERITY_CHOICES, - error_messages={ - "required": "Select valid choice: In Progress, On Hold, Completed", - "invalid_choice": "Select valid choice: Critical,High,Medium,Low"}) - - # Remediation planning fields - fix_available = forms.BooleanField(required=False) - fix_version = forms.CharField(max_length=100, required=False) - planned_remediation_version = forms.CharField(max_length=99, required=False) - effort_for_fixing = forms.CharField(max_length=99, required=False) - - # Technical details fields - steps_to_reproduce = forms.CharField(widget=forms.Textarea, required=False) - severity_justification = forms.CharField(widget=forms.Textarea, required=False) - component_name = forms.CharField(max_length=500, required=False) - component_version = forms.CharField(max_length=100, required=False) - - # Notes field - notes = forms.CharField(widget=forms.Textarea, required=False, help_text="Note content to add when applying template") - - # Endpoints field - endpoints = forms.CharField(max_length=5000, required=False, - help_text="Endpoint URLs (one per line)", - widget=forms.widgets.Textarea(attrs={"rows": "3", "cols": "400"})) - - field_order = ["title", "cwe", "vulnerability_ids", "severity", "cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", - "description", "impact", "steps_to_reproduce", "severity_justification", "mitigation", - "fix_available", "fix_version", "planned_remediation_version", "effort_for_fixing", - "component_name", "component_version", "references", "notes", "endpoints", "tags"] - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.fields["tags"].autocomplete_tags = Finding.tags.tag_model.objects.all().order_by("name") - - # Hide CVSS fields based on system settings - hide_cvss_fields_if_disabled(self) - - class Meta: - model = Finding_Template - order = ("title", "cwe", "vulnerability_ids", "cvssv3", "cvssv3_score", "cvssv4", "cvssv4_score", "severity", "description", "impact", - "steps_to_reproduce", "severity_justification", "mitigation", "fix_available", "fix_version", - "planned_remediation_version", "effort_for_fixing", "component_name", "component_version", - "references", "notes", "endpoints", "tags") - exclude = ("numerical_severity", "is_mitigated", "last_used", "endpoint_status", "cve", "vulnerability_ids_text") - - def clean_cvssv3(self): - value = self.cleaned_data.get("cvssv3") - if value: - try: - cvss3_validator(value) - except ValidationError as e: - raise forms.ValidationError(e.messages) - return value - - def clean_cvssv4(self): - value = self.cleaned_data.get("cvssv4") - if value: - try: - cvss4_validator(value) - except ValidationError as e: - raise forms.ValidationError(e.messages) - return value - - def clean_tags(self): - tag_validator(self.cleaned_data.get("tags")) - return self.cleaned_data.get("tags") - - -class DeleteFindingTemplateForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Finding_Template - fields = ["id"] - - -class FindingBulkUpdateForm(forms.ModelForm): - status = forms.BooleanField(required=False) - risk_acceptance = forms.BooleanField(required=False) - risk_accept = forms.BooleanField(required=False) - risk_unaccept = forms.BooleanField(required=False) - - date = forms.DateField(required=False, widget=forms.DateInput(attrs={"class": "datepicker"})) - planned_remediation_date = forms.DateField(required=False, widget=forms.DateInput(attrs={"class": "datepicker"})) - planned_remediation_version = forms.CharField(required=False, max_length=99, widget=forms.TextInput(attrs={"class": "form-control"})) - finding_group = forms.BooleanField(required=False) - finding_group_create = forms.BooleanField(required=False) - finding_group_create_name = forms.CharField(required=False) - finding_group_add = forms.BooleanField(required=False) - add_to_finding_group_id = forms.CharField(required=False) - finding_group_remove = forms.BooleanField(required=False) - finding_group_by = forms.BooleanField(required=False) - finding_group_by_option = forms.CharField(required=False) - - push_to_jira = forms.BooleanField(required=False) - # unlink_from_jira = forms.BooleanField(required=False) - push_to_github = forms.BooleanField(required=False) - tags = TagField(required=False, autocomplete_tags=Finding.tags.tag_model.objects.all().order_by("name")) - notes = forms.CharField(required=False, max_length=1024, widget=forms.TextInput(attrs={"class": "form-control"})) - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.fields["severity"].required = False - # we need to defer initialization to prevent multiple initializations if other forms are shown - self.fields["tags"].widget.tag_options = tagulous.models.options.TagOptions(autocomplete_settings={"width": "200px", "defer": True}) - if disclaimer := get_system_setting("disclaimer_notes"): - self.disclaimer = disclaimer.strip() - - def clean(self): - cleaned_data = super().clean() - - if (cleaned_data["active"] or cleaned_data["verified"]) and cleaned_data["duplicate"]: - msg = "Duplicate findings cannot be verified or active" - raise forms.ValidationError(msg) - if cleaned_data["false_p"] and cleaned_data["verified"]: - msg = "False positive findings cannot be verified." - raise forms.ValidationError(msg) - if cleaned_data["active"] and cleaned_data.get("risk_acceptance") and cleaned_data.get("risk_accept"): - msg = "Active findings cannot be risk accepted." - raise forms.ValidationError(msg) - return cleaned_data - - def clean_tags(self): - tag_validator(self.cleaned_data.get("tags")) - return self.cleaned_data.get("tags") - - class Meta: - model = Finding - fields = ("severity", "date", "planned_remediation_date", "active", "verified", "false_p", "duplicate", "out_of_scope", - "under_review", "is_mitigated") - - -class EditEndpointForm(forms.ModelForm): - class Meta: - model = Endpoint - exclude = ["product", "inherited_tags"] - - def __init__(self, *args, **kwargs): - self.product = None - self.endpoint_instance = None - super().__init__(*args, **kwargs) - if "instance" in kwargs: - self.endpoint_instance = kwargs.pop("instance") - self.product = self.endpoint_instance.product - product_id = self.endpoint_instance.product.pk - findings = Finding.objects.filter(test__engagement__product__id=product_id) - self.fields["findings"].queryset = findings - - def clean(self): - - cleaned_data = super().clean() - - protocol = cleaned_data["protocol"] - userinfo = cleaned_data["userinfo"] - host = cleaned_data["host"] - port = cleaned_data["port"] - path = cleaned_data["path"] - query = cleaned_data["query"] - fragment = cleaned_data["fragment"] - - endpoint = endpoint_filter( - protocol=protocol, - userinfo=userinfo, - host=host, - port=port, - path=path, - query=query, - fragment=fragment, - product=self.product, - ) - if endpoint.count() > 1 or (endpoint.count() == 1 and endpoint.first().pk != self.endpoint_instance.pk): - msg = "It appears as though an endpoint with this data already exists for this product." - raise forms.ValidationError(msg, code="invalid") - - return cleaned_data - - def clean_tags(self): - tag_validator(self.cleaned_data.get("tags")) - return self.cleaned_data.get("tags") - - -class AddEndpointForm(forms.Form): - endpoint = forms.CharField(max_length=5000, required=True, label="Endpoint(s)", - help_text="The IP address, host name or full URL. You may enter one endpoint per line. " - "Each must be valid.", - widget=forms.widgets.Textarea(attrs={"rows": "15", "cols": "400"})) - product = forms.CharField(required=True, - label=labels.ASSET_LABEL, help_text=labels.ASSET_ENDPOINT_HELP, - widget=forms.widgets.HiddenInput()) - tags = TagField(required=False, - help_text="Add tags that help describe this endpoint. " - "Choose from the list or add new tags. Press Enter key to add.") - - def __init__(self, *args, **kwargs): - product = None - if "product" in kwargs: - product = kwargs.pop("product") - super().__init__(*args, **kwargs) - self.fields["product"] = forms.ModelChoiceField( - queryset=get_authorized_products("add"), - label=labels.ASSET_LABEL, - help_text=labels.ASSET_ENDPOINT_HELP) - if product is not None: - self.fields["product"].initial = product.id - - self.product = product - self.endpoints_to_process = [] - - def save(self): - processed_endpoints = [] - for e in self.endpoints_to_process: - endpoint, _created = endpoint_get_or_create( - protocol=e[0], - userinfo=e[1], - host=e[2], - port=e[3], - path=e[4], - query=e[5], - fragment=e[6], - product=self.product, - ) - processed_endpoints.append(endpoint) - return processed_endpoints - - def clean(self): - - cleaned_data = super().clean() - - if "endpoint" in cleaned_data and "product" in cleaned_data: - endpoint = cleaned_data["endpoint"] - product = cleaned_data["product"] - if isinstance(product, Product): - self.product = product - else: - self.product = Product.objects.get(id=int(product)) - else: - msg = "Please enter a valid URL or IP address." - raise forms.ValidationError(msg, code="invalid") - - endpoints_to_add_list, errors = validate_endpoints_to_add(endpoint) - if errors: - raise forms.ValidationError(errors) - self.endpoints_to_process = endpoints_to_add_list - - return cleaned_data - - def clean_tags(self): - tag_validator(self.cleaned_data.get("tags")) - return self.cleaned_data.get("tags") - - -class DeleteEndpointForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Endpoint - fields = ["id"] - - -class NoteForm(forms.ModelForm): - entry = forms.CharField(max_length=2400, widget=forms.Textarea(attrs={"rows": 4, "cols": 15}), - label="Notes:") - - class Meta: - model = Notes - fields = ["entry", "private"] - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - if disclaimer := get_system_setting("disclaimer_notes"): - self.disclaimer = disclaimer.strip() - - -class TypedNoteForm(NoteForm): - - def __init__(self, *args, **kwargs): - queryset = kwargs.pop("available_note_types") - super().__init__(*args, **kwargs) - self.fields["note_type"] = forms.ModelChoiceField(queryset=queryset, label="Note Type", required=True) - - class Meta: - model = Notes - fields = ["note_type", "entry", "private"] - - -class DeleteNoteForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Notes - fields = ["id"] - - -class CloseFindingForm(forms.ModelForm): - entry = forms.CharField( - required=True, max_length=2400, - widget=forms.Textarea, label="Notes:", - error_messages={"required": ("The reason for closing a finding is " - "required, please use the text area " - "below to provide documentation.")}) - - mitigated = forms.DateField(required=False, help_text="Date and time when the flaw has been fixed", widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) - mitigated_by = forms.ModelChoiceField(required=False, queryset=Dojo_User.objects.none()) - false_p = forms.BooleanField(initial=False, required=False, label="False Positive") - out_of_scope = forms.BooleanField(initial=False, required=False, label="Out of Scope") - duplicate = forms.BooleanField(initial=False, required=False, label="Duplicate") - - def __init__(self, *args, **kwargs): - queryset = kwargs.pop("missing_note_types") - # must pop custom kwargs before calling parent __init__ to avoid unexpected kwarg errors - self.can_edit_mitigated_data = kwargs.pop("can_edit_mitigated_data") if "can_edit_mitigated_data" in kwargs \ - else False - super().__init__(*args, **kwargs) - if len(queryset) == 0: - self.fields["note_type"].widget = forms.HiddenInput() - else: - self.fields["note_type"] = forms.ModelChoiceField(queryset=queryset, label="Note Type", required=True) - - if self.can_edit_mitigated_data: - self.fields["mitigated_by"].queryset = get_authorized_users("edit") - self.fields["mitigated"].initial = self.instance.mitigated - self.fields["mitigated_by"].initial = self.instance.mitigated_by - if disclaimer := get_system_setting("disclaimer_notes"): - self.disclaimer = disclaimer.strip() - - def _post_clean(self): - super()._post_clean() - - if self.can_edit_mitigated_data: - opts = self.instance._meta - if not self.cleaned_data.get("active"): - try: - opts.get_field("mitigated").save_form_data(self.instance, self.cleaned_data.get("mitigated")) - opts.get_field("mitigated_by").save_form_data(self.instance, self.cleaned_data.get("mitigated_by")) - except forms.ValidationError as e: - self._update_errors(e) - - class Meta: - model = Notes - fields = ["note_type", "entry", "mitigated", "mitigated_by", "false_p", "out_of_scope", "duplicate"] - - -class EditPlannedRemediationDateFindingForm(forms.ModelForm): - def __init__(self, *args, **kwargs): - finding = None - if "finding" in kwargs: - finding = kwargs.pop("finding") - - super().__init__(*args, **kwargs) - - self.fields["planned_remediation_date"].required = True - self.fields["planned_remediation_date"].widget = forms.DateInput(attrs={"class": "datepicker"}) - - if finding is not None: - self.fields["planned_remediation_date"].initial = finding.planned_remediation_date - - class Meta: - model = Finding - fields = ["planned_remediation_date"] - - -class DefectFindingForm(forms.ModelForm): - CLOSE_CHOICES = (("Close Finding", "Close Finding"), ("Not Fixed", "Not Fixed")) - defect_choice = forms.ChoiceField(required=True, choices=CLOSE_CHOICES) - - entry = forms.CharField( - required=True, max_length=2400, - widget=forms.Textarea, label="Notes:", - error_messages={"required": ("The reason for closing a finding is " - "required, please use the text area " - "below to provide documentation.")}) - - class Meta: - model = Notes - fields = ["entry"] - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - if disclaimer := get_system_setting("disclaimer_notes"): - self.disclaimer = disclaimer.strip() - - -class ClearFindingReviewForm(forms.ModelForm): - entry = forms.CharField( - required=True, max_length=2400, - help_text="Please provide a message.", - widget=forms.Textarea, label="Notes:", - error_messages={"required": ("The reason for clearing a review is " - "required, please use the text area " - "below to provide documentation.")}) - - class Meta: - model = Finding - fields = ["active", "verified", "false_p", "out_of_scope", "duplicate", "is_mitigated"] - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - if disclaimer := get_system_setting("disclaimer_notes"): - self.disclaimer = disclaimer.strip() - - -class ReviewFindingForm(forms.Form): - reviewers = forms.MultipleChoiceField( - help_text=( - "Select all users who can review Finding. Only users with " - "at least write permission to this finding can be selected"), - required=False, - ) - entry = forms.CharField( - required=True, max_length=2400, - help_text="Please provide a message for reviewers.", - widget=forms.Textarea, label="Notes:", - error_messages={"required": ("The reason for requesting a review is " - "required, please use the text area " - "below to provide documentation.")}) - allow_all_reviewers = forms.BooleanField( - required=False, - label="Allow All Eligible Reviewers", - help_text=("Checking this box will allow any user in the drop down " - "above to provide a review for this finding")) - - def __init__(self, *args, **kwargs): - finding = kwargs.pop("finding", None) - kwargs.pop("user", None) - super().__init__(*args, **kwargs) - # Get the list of users - if finding is not None: - users = get_authorized_users_for_product_and_product_type(None, finding.test.engagement.product, "edit") - else: - users = get_authorized_users("edit").filter(is_active=True) - # Save a copy of the original query to be used in the validator - self.reviewer_queryset = users - # Set the users in the form - self.fields["reviewers"].choices = self._get_choices(self.reviewer_queryset) - if disclaimer := get_system_setting("disclaimer_notes"): - self.disclaimer = disclaimer.strip() - - @staticmethod - def _get_choices(queryset): - return [(item.pk, item.get_full_name()) for item in queryset] - - def clean(self): - cleaned_data = super().clean() - if cleaned_data.get("allow_all_reviewers", False): - cleaned_data["reviewers"] = [user.id for user in self.reviewer_queryset] - if len(cleaned_data.get("reviewers", [])) == 0: - msg = "Please select at least one user from the reviewers list" - raise ValidationError(msg) - return cleaned_data - - class Meta: - fields = ["reviewers", "entry", "allow_all_reviewers"] - - -class WeeklyMetricsForm(forms.Form): - dates = forms.ChoiceField() - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - wmf_options = [] - - for i in range(6): - # Weeks start on Monday - curr = datetime.now() - relativedelta(weeks=i) - start_of_period = curr - relativedelta(weeks=1, weekday=0, - hour=0, minute=0, second=0) - end_of_period = curr + relativedelta(weeks=0, weekday=0, - hour=0, minute=0, second=0) - - wmf_options.append((end_of_period.strftime("%b %d %Y %H %M %S %Z"), - start_of_period.strftime("%b %d") - + " - " + end_of_period.strftime("%b %d"))) - - wmf_options = tuple(wmf_options) - - self.fields["dates"].choices = wmf_options - - -class SimpleMetricsForm(forms.Form): - date = forms.DateField( - label="", - widget=MonthYearWidget()) - - -class SimpleSearchForm(forms.Form): - query = forms.CharField(required=False) - - -class DateRangeMetrics(forms.Form): - start_date = forms.DateField(required=True, label="To", - widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) - end_date = forms.DateField(required=True, - label="From", - widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) - - -class MetricsFilterForm(forms.Form): - start_date = forms.DateField(required=False, - label="To", - widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) - end_date = forms.DateField(required=False, - label="From", - widget=forms.TextInput(attrs={"class": "datepicker", "autocomplete": "off"})) - finding_status = forms.MultipleChoiceField( - required=False, - widget=forms.CheckboxSelectMultiple, - choices=FINDING_STATUS, - label="Status") - severity = forms.MultipleChoiceField(required=False, - choices=(("Low", "Low"), - ("Medium", "Medium"), - ("High", "High"), - ("Critical", "Critical")), - help_text=('Hold down "Control", or ' - '"Command" on a Mac, to ' - 'select more than one.')) - exclude_product_types = forms.ModelMultipleChoiceField( - required=False, queryset=Product_Type.objects.all().order_by("name")) - - # add the ability to exclude the exclude_product_types field - def __init__(self, *args, **kwargs): - exclude_product_types = kwargs.pop("exclude_product_types", False) - super().__init__(*args, **kwargs) - if exclude_product_types: - del self.fields["exclude_product_types"] - - -class DojoUserForm(forms.ModelForm): - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - if not get_current_user().is_superuser and not get_system_setting("enable_user_profile_editable"): - for field in self.fields: - self.fields[field].disabled = True - - class Meta: - model = Dojo_User - exclude = ["password", "last_login", "is_superuser", "groups", - "username", "is_staff", "is_active", "date_joined", - "user_permissions"] + # add the ability to exclude the exclude_product_types field + def __init__(self, *args, **kwargs): + exclude_product_types = kwargs.pop("exclude_product_types", False) + super().__init__(*args, **kwargs) + if exclude_product_types: + del self.fields["exclude_product_types"] class ChangePasswordForm(forms.Form): @@ -2333,107 +745,16 @@ def clean(self): return cleaned_data -class AddDojoUserForm(forms.ModelForm): - email = forms.EmailField(required=True) - password = forms.CharField(widget=forms.PasswordInput, - required=settings.REQUIRE_PASSWORD_ON_USER, - validators=[validate_password], - help_text="") - - class Meta: - model = Dojo_User - fields = ["username", "password", "first_name", "last_name", "email", "is_active", "is_staff", "is_superuser"] - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - current_user = get_current_user() - if not current_user.is_superuser: - self.fields["is_staff"].disabled = True - self.fields["is_superuser"].disabled = True - self.fields["password"].help_text = get_password_requirements_string() - - -class EditDojoUserForm(forms.ModelForm): - email = forms.EmailField(required=True) - - class Meta: - model = Dojo_User - fields = ["username", "first_name", "last_name", "email", "is_active", "is_staff", "is_superuser"] - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - current_user = get_current_user() - if not current_user.is_superuser: - self.fields["is_staff"].disabled = True - self.fields["is_superuser"].disabled = True - - -class DeleteUserForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = User - fields = ["id"] - - -class UserContactInfoForm(forms.ModelForm): - reset_api_token = forms.BooleanField( - required=False, - label=_("Reset API token"), - help_text=_("Upon saving, a new token will be generated and a notification of category 'Other' is triggered."), - ) - - class Meta: - model = UserContactInfo - exclude = ["user", "slack_user_id"] - # Swap order: password_last_reset before token_last_reset - field_order = [ - "title", "phone_number", "cell_number", "twitter_username", "github_username", - "slack_username", "ui_use_tailwind", "block_execution", "force_password_reset", "reset_api_token", - "password_last_reset", "token_last_reset", - ] - - def __init__(self, *args, **kwargs): - user = kwargs.pop("user", None) - super().__init__(*args, **kwargs) - # Make timestamp fields readonly. - # NOTE: `disabled=True` is enforced server-side by Django forms: posted values for disabled fields - # are ignored during binding/cleaning, so these timestamps cannot be modified via this form. - if "password_last_reset" in self.fields: - self.fields["password_last_reset"].disabled = True - if "token_last_reset" in self.fields: - self.fields["token_last_reset"].disabled = True - # Do not expose force password reset if the current user does not have a password to reset - if user is not None: - if not user.has_usable_password(): - self.fields["force_password_reset"].disabled = True - self.fields["force_password_reset"].help_text = "This user is authorized through SSO, and does not have a password to reset" - # Determine some other settings based on the current user - current_user = get_current_user() - if not current_user.is_superuser: - if not user_has_configuration_permission(current_user, "auth.change_user") and \ - not user_has_configuration_permission(current_user, "auth.add_user"): - self.fields.pop("force_password_reset", None) - if not get_system_setting("enable_user_profile_editable"): - for field in self.fields: - self.fields[field].disabled = True - - # Only show reset_api_token to superusers or global owners, and only if API tokens are enabled - if not settings.API_TOKENS_ENABLED or not user_is_superuser_or_global_owner(current_user): - self.fields.pop("reset_api_token", None) - - -def get_years(): - now = timezone.now() - return [(now.year, now.year), (now.year - 1, now.year - 1), (now.year - 2, now.year - 2)] - - -class ProductCountsFormBase(forms.Form): - month = forms.ChoiceField(choices=list(MONTHS.items()), required=True, error_messages={ - "required": "*"}) - year = forms.ChoiceField(choices=get_years, required=True, error_messages={ - "required": "*"}) +# Product forms live in dojo/product/ui/forms.py. Re-exported here for backward +# compat: ProductCountsFormBase is subclassed by ProductTypeCountsForm below, +# Authorize_User_For_ProductsForm by dojo/user/views.py, ProductTagCountsForm by +# dojo/metrics/views.py. The other product forms are imported directly from +# dojo.product.ui.forms by the product module's own views. +from dojo.product.ui.forms import ( # noqa: E402, F401 -- backward compat + Authorize_User_For_ProductsForm, + ProductCountsFormBase, + ProductTagCountsForm, +) class ProductTypeCountsForm(ProductCountsFormBase): @@ -2448,20 +769,6 @@ def __init__(self, *args, **kwargs): self.fields["product_type"].queryset = get_authorized_product_types("view") -class ProductTagCountsForm(ProductCountsFormBase): - product_tag = forms.ModelChoiceField(required=True, - queryset=Product.tags.tag_model.objects.none().order_by("name"), - label=labels.ASSET_TAG_LABEL, - error_messages={ - "required": "*"}) - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - prods = get_authorized_products("view") - tags_available_to_user = Product.tags.tag_model.objects.filter(product__in=prods) - self.fields["product_tag"].queryset = tags_available_to_user - - class APIKeyForm(forms.ModelForm): id = forms.IntegerField(required=True, widget=forms.widgets.HiddenInput()) @@ -2473,130 +780,14 @@ class Meta: "date_joined", "user_permissions"] -class ReportOptionsForm(forms.Form): - yes_no = (("0", "No"), ("1", "Yes")) - include_finding_notes = forms.ChoiceField(choices=yes_no, label="Finding Notes") - include_finding_images = forms.ChoiceField(choices=yes_no, label="Finding Images") - include_executive_summary = forms.ChoiceField(choices=yes_no, label="Executive Summary") - include_table_of_contents = forms.ChoiceField(choices=yes_no, label="Table of Contents") - include_disclaimer = forms.ChoiceField(choices=yes_no, label="Disclaimer") - report_type = forms.ChoiceField(choices=(("HTML", "HTML"),)) - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - if get_system_setting("disclaimer_reports_forced"): - self.fields["include_disclaimer"].disabled = True - self.fields["include_disclaimer"].initial = "1" # represents yes - self.fields["include_disclaimer"].help_text = "Administrator of the system enforced placement of disclaimer in all reports. You are not able exclude disclaimer from this report." - - -class CustomReportOptionsForm(forms.Form): - yes_no = (("0", "No"), ("1", "Yes")) - report_name = forms.CharField(required=False, max_length=100) - include_finding_notes = forms.ChoiceField(required=False, choices=yes_no) - include_finding_images = forms.ChoiceField(choices=yes_no, label="Finding Images") - report_type = forms.ChoiceField(choices=(("HTML", "HTML"),)) - - -class DeleteFindingForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Finding - fields = ["id"] - - -class CopyFindingForm(forms.Form): - test = forms.ModelChoiceField( - required=True, - queryset=Test.objects.none(), - error_messages={"required": "*"}) - - def __init__(self, *args, **kwargs): - authorized_lists = kwargs.pop("tests", None) - super().__init__(*args, **kwargs) - self.fields["test"].queryset = authorized_lists - - -class FindingFormID(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Finding - fields = ("id",) - - -class Benchmark_Product_SummaryForm(forms.ModelForm): - - class Meta: - model = Benchmark_Product_Summary - exclude = ["product", "current_level", "benchmark_type", "asvs_level_1_benchmark", "asvs_level_1_score", "asvs_level_2_benchmark", "asvs_level_2_score", "asvs_level_3_benchmark", "asvs_level_3_score"] - - -class DeleteBenchmarkForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Benchmark_Product_Summary - fields = ["id"] - - -class Product_API_Scan_ConfigurationForm(forms.ModelForm): - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - - tool_configuration = forms.ModelChoiceField( - label="Tool Configuration", - queryset=Tool_Configuration.objects.all().order_by("name"), - required=True, - ) - - class Meta: - model = Product_API_Scan_Configuration - exclude = ["product"] - - -class DeleteProduct_API_Scan_ConfigurationForm(forms.ModelForm): - id = forms.IntegerField(required=True, widget=forms.widgets.HiddenInput()) - - class Meta: - model = Product_API_Scan_Configuration - fields = ["id"] - - -class ToolTypeForm(forms.ModelForm): - class Meta: - model = Tool_Type - exclude = ["product"] - - def __init__(self, *args, **kwargs): - instance = kwargs.get("instance") - self.newly_created = True - if instance is not None: - self.newly_created = instance.pk is None - super().__init__(*args, **kwargs) - - def clean(self): - form_data = self.cleaned_data - if self.newly_created: - name = form_data.get("name") - # Make sure this will not create a duplicate test type - if Tool_Type.objects.filter(name=name).count() > 0: - msg = "A Tool Type with the name already exists" - raise forms.ValidationError(msg) - - return form_data - - -class RegulationForm(forms.ModelForm): - class Meta: - model = Regulation - exclude = ["product"] - +from dojo.benchmark.ui.forms import ( # noqa: E402, F401 -- backward compat + Benchmark_Product_SummaryForm, + Benchmark_RequirementForm, + BenchmarkForm, + DeleteBenchmarkForm, +) +from dojo.regulations.ui.forms import RegulationForm # noqa: E402, F401 -- re-export + class AppAnalysisForm(forms.ModelForm): user = forms.ModelChoiceField(queryset=Dojo_User.objects.exclude(is_active=False).order_by("first_name", "last_name"), required=True) @@ -2622,28 +813,6 @@ def __init__(self, *args, **kwargs): self.fields["website_found"].disabled = True -class ToolConfigForm(forms.ModelForm): - tool_type = forms.ModelChoiceField(queryset=Tool_Type.objects.all(), label="Tool Type") - ssh = forms.CharField(widget=forms.Textarea(attrs={}), required=False, label="SSH Key") - - class Meta: - model = Tool_Configuration - exclude = ["product"] - - def clean(self): - form_data = self.cleaned_data - - try: - if form_data["url"] is not None: - url_validator = URLValidator(schemes=["ssh", "http", "https"]) - url_validator(form_data["url"]) - except forms.ValidationError: - msg = "It does not appear as though this endpoint is a valid URL/SSH or IP address." - raise forms.ValidationError(msg, code="invalid") - - return form_data - - class SLAConfigForm(forms.ModelForm): def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) @@ -2683,148 +852,16 @@ class Meta: fields = ["id"] -class DeleteObjectsSettingsForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Objects_Product - fields = ["id"] - - -class DeleteToolProductSettingsForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Tool_Product_Settings - fields = ["id"] - - -class ToolProductSettingsForm(forms.ModelForm): - tool_configuration = forms.ModelChoiceField(queryset=Tool_Configuration.objects.all(), label="Tool Configuration") - - class Meta: - model = Tool_Product_Settings - fields = ["name", "description", "url", "tool_configuration", "tool_project_id"] - exclude = ["tool_type"] - order = ["name"] - - def clean(self): - form_data = self.cleaned_data - - try: - if form_data["url"] is not None: - url_validator = URLValidator(schemes=["ssh", "http", "https"]) - url_validator(form_data["url"]) - except forms.ValidationError: - msg = "It does not appear as though this endpoint is a valid URL/SSH or IP address." - raise forms.ValidationError(msg, code="invalid") - - return form_data - - -class ObjectSettingsForm(forms.ModelForm): - - # tags = forms.CharField(widget=forms.SelectMultiple(choices=[]), - # required=False, - # help_text="Add tags that help describe this object. " - # "Choose from the list or add new tags. Press TAB key to add.") - - class Meta: - model = Objects_Product - fields = ["path", "folder", "artifact", "name", "review_status", "tags"] - exclude = ["product"] - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - - def clean(self): - return self.cleaned_data - - -class EngagementPresetsForm(forms.ModelForm): - - notes = forms.CharField(widget=forms.Textarea(attrs={}), - required=False, help_text="Description of what needs to be tested or setting up environment for testing") - - scope = forms.CharField(widget=forms.Textarea(attrs={}), - required=False, help_text="Scope of Engagement testing, IP's/Resources/URL's)") - - class Meta: - model = Engagement_Presets - exclude = ["product"] - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - if disclaimer := get_system_setting("disclaimer_notes"): - self.disclaimer = disclaimer.strip() - - -class DeleteEngagementPresetsForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Engagement_Presets - fields = ["id"] - - -class SystemSettingsForm(forms.ModelForm): - jira_webhook_secret = forms.CharField(required=False) - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - - self.fields["enable_product_tracking_files"].label = labels.SETTINGS_TRACKED_FILES_ENABLE_LABEL - self.fields["enable_product_tracking_files"].help_text = labels.SETTINGS_TRACKED_FILES_ENABLE_HELP - - self.fields[ - "enforce_verified_status_product_grading"].label = labels.SETTINGS_ASSET_GRADING_ENFORCE_VERIFIED_LABEL - self.fields[ - "enforce_verified_status_product_grading"].help_text = labels.SETTINGS_ASSET_GRADING_ENFORCE_VERIFIED_HELP - - self.fields["enable_product_grade"].label = labels.SETTINGS_ASSET_GRADING_ENABLE_LABEL - self.fields["enable_product_grade"].help_text = labels.SETTINGS_ASSET_GRADING_ENABLE_HELP - - self.fields["enable_product_tag_inheritance"].label = labels.SETTINGS_ASSET_TAG_INHERITANCE_ENABLE_LABEL - self.fields["enable_product_tag_inheritance"].help_text = labels.SETTINGS_ASSET_TAG_INHERITANCE_ENABLE_HELP - - def clean(self): - cleaned_data = super().clean() - enable_jira_value = cleaned_data.get("enable_jira") - jira_webhook_secret_value = cleaned_data.get("jira_webhook_secret").strip() - - if enable_jira_value and not jira_webhook_secret_value: - self.add_error("jira_webhook_secret", "This field is required when enable Jira Integration is True") - - return cleaned_data - - class Meta: - model = System_Settings - exclude = () - - -class BenchmarkForm(forms.ModelForm): - - class Meta: - model = Benchmark_Product - exclude = ["product", "control"] - - -class Benchmark_RequirementForm(forms.ModelForm): - - class Meta: - model = Benchmark_Requirement - exclude = [""] - - from dojo.notifications.ui.forms import ( # noqa: E402, F401 -- backward compat DeleteNotificationsWebhookForm, NotificationsForm, NotificationsWebhookForm, ProductNotificationsForm, ) +from dojo.object.ui.forms import ( # noqa: E402, F401 -- re-export + DeleteObjectsSettingsForm, + ObjectSettingsForm, +) class AjaxChoiceField(forms.ChoiceField): @@ -2832,444 +869,11 @@ def valid_value(self, value): return True -class LoginBanner(forms.Form): - banner_enable = forms.BooleanField( - label="Enable login banner", - initial=False, - required=False, - help_text="Tick this box to enable a text banner on the login page", - ) - - banner_message = forms.CharField( - required=False, - label="Message to display on the login page", - ) - - def clean(self): - return super().clean() - - -class AnnouncementCreateForm(forms.ModelForm): - class Meta: - model = Announcement - fields = "__all__" - - -class AnnouncementRemoveForm(AnnouncementCreateForm): - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.fields["dismissable"].disabled = True - self.fields["message"].disabled = True - self.fields["style"].disabled = True - - -# ============================== -# Defect Dojo Engaegment Surveys -# ============================== - -# List of validator_name:func_name -# Show in admin a multichoice list of validator names -# pass this to form using field_name='validator_name' ? -class QuestionForm(forms.Form): - - """Base class for a Question""" - - def __init__(self, *args, **kwargs): - self.helper = FormHelper() - self.helper.form_method = "post" - - # If true crispy-forms will render a
..
tags - self.helper.form_tag = kwargs.pop("form_tag", True) - - self.engagement_survey = kwargs.get("engagement_survey") - - self.answered_survey = kwargs.get("answered_survey") - if not self.answered_survey: - del kwargs["engagement_survey"] - else: - del kwargs["answered_survey"] - - self.helper.form_class = kwargs.get("form_class", "") - - self.question = kwargs.pop("question", None) - - if not self.question: - msg = "Need a question to render" - raise ValueError(msg) - - super().__init__(*args, **kwargs) - - -class TextQuestionForm(QuestionForm): - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - - # work out initial data - - initial_answer = TextAnswer.objects.filter( - answered_survey=self.answered_survey, - question=self.question, - ) - - initial_answer = initial_answer[0].answer if initial_answer.exists() else "" - - self.fields["answer"] = forms.CharField( - label=self.question.text, - widget=forms.Textarea(attrs={"rows": 3, "cols": 10}), - required=not self.question.optional, - initial=initial_answer, - ) - - def save(self): - if not self.is_valid(): - msg = "form is not valid" - raise forms.ValidationError(msg) - - answer = self.cleaned_data.get("answer") - - if not answer: - if self.fields["answer"].required: - msg = "Required" - raise forms.ValidationError(msg) - return - - text_answer, created = TextAnswer.objects.get_or_create( - answered_survey=self.answered_survey, - question=self.question, - ) - - if created: - text_answer.answered_survey = self.answered_survey - text_answer.answer = answer - text_answer.save() - - -class ChoiceQuestionForm(QuestionForm): - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - - choices = [(c.id, c.label) for c in self.question.choices.all()] - - # initial values - - initial_choices = [] - choice_answer = ChoiceAnswer.objects.filter( - answered_survey=self.answered_survey, - question=self.question, - ).annotate(a=Count("answer")).filter(a__gt=0) - - # we have ChoiceAnswer instance - if choice_answer: - choice_answer = choice_answer[0] - initial_choices = list(choice_answer.answer.all().values_list("id", flat=True)) - if self.question.multichoice is False: - initial_choices = initial_choices[0] - - # default classes - widget = forms.RadioSelect - field_type = forms.ChoiceField - inline_type = InlineRadios - - if self.question.multichoice: - field_type = forms.MultipleChoiceField - widget = forms.CheckboxSelectMultiple - inline_type = InlineCheckboxes - - field = field_type( - label=self.question.text, - required=not self.question.optional, - choices=choices, - initial=initial_choices, - widget=widget, - ) - - self.fields["answer"] = field - - # Render choice buttons inline - self.helper.layout = Layout( - inline_type("answer"), - ) - - def clean_answer(self): - real_answer = self.cleaned_data.get("answer") - - # for single choice questions, the selected answer is a single string - if not isinstance(real_answer, list): - real_answer = [real_answer] - return real_answer - - def save(self): - if not self.is_valid(): - msg = "Form is not valid" - raise forms.ValidationError(msg) - - real_answer = self.cleaned_data.get("answer") - - if not real_answer: - if self.fields["answer"].required: - msg = "Required" - raise forms.ValidationError(msg) - return - - choices = Choice.objects.filter(id__in=real_answer) - - # find ChoiceAnswer and filter in answer ! - choice_answer = ChoiceAnswer.objects.filter( - answered_survey=self.answered_survey, - question=self.question, - ) - - # we have ChoiceAnswer instance - if choice_answer: - choice_answer = choice_answer[0] - - if not choice_answer: - # create a ChoiceAnswer - choice_answer = ChoiceAnswer.objects.create( - answered_survey=self.answered_survey, - question=self.question, - ) - - # re save out the choices - choice_answer.answered_survey = self.answered_survey - choice_answer.answer.set(choices) - choice_answer.save() - - -class Add_Questionnaire_Form(forms.ModelForm): - survey = forms.ModelChoiceField( - queryset=Engagement_Survey.objects.all(), - required=True, - widget=forms.widgets.Select(), - help_text="Select the Questionnaire to add.") - - class Meta: - model = Answered_Survey - exclude = ("responder", - "completed", - "engagement", - "answered_on", - "assignee") - - -class AddGeneralQuestionnaireForm(forms.ModelForm): - survey = forms.ModelChoiceField( - queryset=Engagement_Survey.objects.all(), - required=True, - widget=forms.widgets.Select(), - help_text="Select the Questionnaire to add.") - expiration = forms.DateField(widget=forms.TextInput( - attrs={"class": "datepicker", "autocomplete": "off"})) - - class Meta: - model = General_Survey - exclude = ("num_responses", "generated") - - # date can only be today or in the past, not the future - def clean_expiration(self): - expiration = self.cleaned_data.get("expiration", None) - if expiration: - today = datetime.today().date() - if expiration < today: - msg = "The expiration cannot be in the past" - raise forms.ValidationError(msg) - if expiration == today: - msg = "The expiration cannot be today" - raise forms.ValidationError(msg) - return timezone.make_aware( - datetime.combine(expiration, datetime.min.time()), - ) - msg = "An expiration for the survey must be supplied" - raise forms.ValidationError(msg) - - -class Delete_Questionnaire_Form(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Answered_Survey - fields = ["id"] - - -class DeleteGeneralQuestionnaireForm(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = General_Survey - fields = ["id"] - - -class Delete_Eng_Survey_Form(forms.ModelForm): - id = forms.IntegerField(required=True, - widget=forms.widgets.HiddenInput()) - - class Meta: - model = Engagement_Survey - fields = ["id"] - - -class CreateQuestionnaireForm(forms.ModelForm): - class Meta: - model = Engagement_Survey - exclude = ["questions"] - - -with warnings.catch_warnings(action="ignore", category=ManagerInheritanceWarning): - class EditQuestionnaireQuestionsForm(forms.ModelForm): - questions = forms.ModelMultipleChoiceField( - Question.polymorphic.all(), - required=True, - help_text="Select questions to include on this questionnaire. Field can be used to search available questions.", - widget=MultipleSelectWithPop(attrs={"size": "11"})) - - class Meta: - model = Engagement_Survey - exclude = ["name", "description", "active"] - - -class CreateQuestionForm(forms.Form): - type = forms.ChoiceField( - choices=(("---", "-----"), ("text", "Text"), ("choice", "Choice"))) - order = forms.IntegerField( - min_value=1, - widget=forms.TextInput(attrs={"data-type": "both"}), - help_text="The order the question will appear on the questionnaire") - optional = forms.BooleanField(help_text="If selected, user doesn't have to answer this question", - initial=False, - required=False, - widget=forms.CheckboxInput(attrs={"data-type": "both"})) - text = forms.CharField(widget=forms.Textarea(attrs={"data-type": "text"}), - label="Question Text", - help_text="The actual question.") - - -class CreateTextQuestionForm(forms.Form): - class Meta: - model = TextQuestion - exclude = ["order", "optional"] - - -class MultiWidgetBasic(forms.widgets.MultiWidget): - def __init__(self, attrs=None): - widgets = [forms.TextInput(attrs={"data-type": "choice"}), - forms.TextInput(attrs={"data-type": "choice"}), - forms.TextInput(attrs={"data-type": "choice"}), - forms.TextInput(attrs={"data-type": "choice"}), - forms.TextInput(attrs={"data-type": "choice"}), - forms.TextInput(attrs={"data-type": "choice"})] - super().__init__(widgets, attrs) - - def decompress(self, value): - if value: - return json.loads(value) - return [None, None, None, None, None, None] - - def format_output(self, rendered_widgets): - return "
".join(rendered_widgets) - - -class MultiExampleField(forms.fields.MultiValueField): - widget = MultiWidgetBasic - - def __init__(self, *args, **kwargs): - list_fields = [forms.fields.CharField(required=True), - forms.fields.CharField(required=True), - forms.fields.CharField(required=False), - forms.fields.CharField(required=False), - forms.fields.CharField(required=False), - forms.fields.CharField(required=False)] - super().__init__(list_fields, *args, **kwargs) - - def compress(self, values): - return json.dumps(values) - - -class CreateChoiceQuestionForm(forms.Form): - multichoice = forms.BooleanField(required=False, - initial=False, - widget=forms.CheckboxInput(attrs={"data-type": "choice"}), - help_text="Can more than one choice can be selected?") - - answer_choices = MultiExampleField(required=False, widget=MultiWidgetBasic(attrs={"data-type": "choice"})) - - class Meta: - model = ChoiceQuestion - exclude = ["order", "optional", "choices"] - - -class EditQuestionForm(forms.ModelForm): - class Meta: - model = Question - exclude = [] - - -class EditTextQuestionForm(EditQuestionForm): - class Meta: - model = TextQuestion - exclude = [] - - -class EditChoiceQuestionForm(EditQuestionForm): - choices = forms.ModelMultipleChoiceField( - Choice.objects.all(), - required=True, - help_text="Select choices to include on this question. Field can be used to search available choices.", - widget=MultipleSelectWithPop(attrs={"size": "11"})) - - class Meta: - model = ChoiceQuestion - exclude = [] - - -class AddChoicesForm(forms.ModelForm): - class Meta: - model = Choice - exclude = [] - - -class AssignUserForm(forms.ModelForm): - assignee = forms.CharField(required=False, - widget=forms.widgets.HiddenInput()) - - def __init__(self, *args, **kwargs): - assignee = None - if "assignee" in kwargs: - assignee = kwargs.pop("asignees") - super().__init__(*args, **kwargs) - if assignee is None: - self.fields["assignee"] = forms.ModelChoiceField(queryset=get_authorized_users("view"), empty_label="Not Assigned", required=False) - else: - self.fields["assignee"].initial = assignee - - class Meta: - model = Answered_Survey - exclude = ["engagement", "survey", "responder", "completed", "answered_on"] - - -class AddEngagementForm(forms.Form): - product = forms.ModelChoiceField( - queryset=Product.objects.none(), - required=True, - widget=forms.widgets.Select(), - help_text="Select which product to attach Engagement") - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.fields["product"].queryset = get_authorized_products("add") - - -class ExistingEngagementForm(forms.Form): - engagement = forms.ModelChoiceField( - queryset=Engagement.objects.none(), - required=True, - widget=forms.widgets.Select(), - help_text="Select which Engagement to link the Questionnaire to") - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.fields["engagement"].queryset = get_authorized_engagements("edit").order_by("-target_start") +from dojo.announcement.ui.forms import ( # noqa: E402, F401 -- re-export + AnnouncementCreateForm, + AnnouncementRemoveForm, +) +from dojo.banner.ui.forms import LoginBanner # noqa: E402, F401 -- re-export class ConfigurationPermissionsForm(forms.Form): @@ -3316,28 +920,3 @@ def set_permission(self, codename): else: msg = "Neither user or group are set" raise Exception(msg) - - -def hide_cvss_fields_if_disabled(form_instance): - """Hide CVSS fields based on system settings.""" - enable_cvss3 = get_system_setting("enable_cvss3_display", True) - enable_cvss4 = get_system_setting("enable_cvss4_display", True) - - # Hide CVSS3 fields if disabled - if not enable_cvss3: - if "cvssv3" in form_instance.fields: - del form_instance.fields["cvssv3"] - if "cvssv3_score" in form_instance.fields: - del form_instance.fields["cvssv3_score"] - - # Hide CVSS4 fields if disabled - if not enable_cvss4: - if "cvssv4" in form_instance.fields: - del form_instance.fields["cvssv4"] - if "cvssv4_score" in form_instance.fields: - del form_instance.fields["cvssv4_score"] - - # If both are disabled, hide all CVSS related fields - if not enable_cvss3 and not enable_cvss4: - if "cvss_info" in form_instance.fields: - del form_instance.fields["cvss_info"] diff --git a/dojo/metrics/utils.py b/dojo/metrics/utils.py index f72e5d71063..ff12b94f83d 100644 --- a/dojo/metrics/utils.py +++ b/dojo/metrics/utils.py @@ -20,11 +20,13 @@ from dojo.filters import ( MetricsEndpointFilter, MetricsEndpointFilterWithoutObjectLookups, - MetricsFindingFilter, - MetricsFindingFilterWithoutObjectLookups, ) from dojo.finding.helper import ACCEPTED_FINDINGS_QUERY, CLOSED_FINDINGS_QUERY, OPEN_FINDINGS_QUERY from dojo.finding.queries import get_authorized_findings +from dojo.finding.ui.filters import ( + MetricsFindingFilter, + MetricsFindingFilterWithoutObjectLookups, +) from dojo.models import Endpoint_Status, Finding, Product_Type from dojo.utils import ( get_system_setting, diff --git a/dojo/metrics/views.py b/dojo/metrics/views.py index 28141bc95de..c2321d5a450 100644 --- a/dojo/metrics/views.py +++ b/dojo/metrics/views.py @@ -19,7 +19,6 @@ from django.views.decorators.vary import vary_on_cookie from dojo.authorization.authorization import user_has_permission_or_403 -from dojo.filters import UserFilter from dojo.forms import ProductTagCountsForm, ProductTypeCountsForm, SimpleMetricsForm from dojo.labels import get_labels from dojo.metrics.utils import ( @@ -35,6 +34,7 @@ from dojo.models import Dojo_User, Finding, Product_Type, Risk_Acceptance from dojo.product.queries import get_authorized_products from dojo.product_type.queries import get_authorized_product_types +from dojo.user.ui.filters import UserFilter from dojo.utils import ( add_breadcrumb, count_findings, diff --git a/dojo/models.py b/dojo/models.py index a41f5640889..c4d3d0aaa68 100644 --- a/dojo/models.py +++ b/dojo/models.py @@ -1,55 +1,24 @@ -import base64 -import contextlib import copy -import hashlib import logging -import re -import warnings -from contextlib import suppress -from datetime import datetime, timedelta -from decimal import Decimal +from datetime import timedelta from pathlib import Path -from typing import TYPE_CHECKING -from urllib.parse import urlparse from uuid import uuid4 -import dateutil -import hyperlink import tagulous.admin -from dateutil.parser import parse as datetutilsparse -from dateutil.relativedelta import relativedelta -from django import forms -from django.conf import settings from django.contrib import admin from django.contrib.auth import get_user_model from django.core.exceptions import ValidationError -from django.core.files.base import ContentFile -from django.core.validators import MaxValueValidator, MinValueValidator, RegexValidator, validate_ipv46_address -from django.db import connection, models -from django.db.models import Count, F, JSONField, Q +from django.db import models +from django.db.models import Count from django.db.models.expressions import Case, When from django.db.models.functions import Lower from django.urls import reverse from django.utils import timezone from django.utils.deconstruct import deconstructible -from django.utils.functional import cached_property -from django.utils.html import escape from django.utils.timezone import now from django.utils.translation import gettext as _ -from django_extensions.db.models import TimeStampedModel -from polymorphic.base import ManagerInheritanceWarning -from polymorphic.managers import PolymorphicManager -from polymorphic.models import PolymorphicModel from tagulous.models import TagField from tagulous.models.managers import FakeTagRelatedManager # noqa: F401 -- backward compat re-export -from titlecase import titlecase - -from dojo.base_models.base import BaseModel -from dojo.validators import cvss3_validator, cvss4_validator - -if TYPE_CHECKING: - from dojo.importers.location_manager import UnsavedLocation - logger = logging.getLogger(__name__) deduplicationLogger = logging.getLogger("dojo.specific-loggers.deduplication") @@ -66,18 +35,6 @@ # default template with all values set to 0 DEFAULT_STATS = {sev.lower(): dict.fromkeys(STATS_FIELDS, 0) for sev in SEVERITIES} -IMPORT_CREATED_FINDING = "N" -IMPORT_CLOSED_FINDING = "C" -IMPORT_REACTIVATED_FINDING = "R" -IMPORT_UNTOUCHED_FINDING = "U" - -IMPORT_ACTIONS = [ - (IMPORT_CREATED_FINDING, "created"), - (IMPORT_CLOSED_FINDING, "closed"), - (IMPORT_REACTIVATED_FINDING, "reactivated"), - (IMPORT_UNTOUCHED_FINDING, "untouched"), -] - def _get_annotations_for_statistics(): annotations = {stats_field.lower(): Count(Case(When(**{stats_field: True}, then=1))) for stats_field in STATS_FIELDS if stats_field != "total"} @@ -162,465 +119,12 @@ def __call__(self, model_instance, filename): return Path(now().strftime(self.directory)) / filename -class Regulation(models.Model): - PRIVACY_CATEGORY = "privacy" - FINANCE_CATEGORY = "finance" - EDUCATION_CATEGORY = "education" - MEDICAL_CATEGORY = "medical" - CORPORATE_CATEGORY = "corporate" - SECURITY_CATEGORY = "security" - GOVERNMENT_CATEGORY = "government" - OTHER_CATEGORY = "other" - CATEGORY_CHOICES = ( - (PRIVACY_CATEGORY, _("Privacy")), - (FINANCE_CATEGORY, _("Finance")), - (EDUCATION_CATEGORY, _("Education")), - (MEDICAL_CATEGORY, _("Medical")), - (CORPORATE_CATEGORY, _("Corporate")), - (SECURITY_CATEGORY, _("Security")), - (GOVERNMENT_CATEGORY, _("Government")), - (OTHER_CATEGORY, _("Other")), - ) - - name = models.CharField(max_length=128, unique=True, help_text=_("The name of the regulation.")) - acronym = models.CharField(max_length=20, unique=True, help_text=_("A shortened representation of the name.")) - category = models.CharField(max_length=16, choices=CATEGORY_CHOICES, help_text=_("The subject of the regulation.")) - jurisdiction = models.CharField(max_length=64, help_text=_("The territory over which the regulation applies.")) - description = models.TextField(blank=True, help_text=_("Information about the regulation's purpose.")) - reference = models.URLField(blank=True, help_text=_("An external URL for more information.")) - - class Meta: - ordering = ["name"] - - def __str__(self): - return self.acronym + " (" + self.jurisdiction + ")" - - User = get_user_model() -# proxy class for convenience and UI -class Dojo_User(User): - class Meta: - proxy = True - ordering = ["first_name"] - - def get_full_name(self): - return Dojo_User.generate_full_name(self) - - def __str__(self): - return self.get_full_name() - - @staticmethod - def wants_block_execution(user): - # this return False if there is no user, i.e. in celery processes, unittests, etc. - return hasattr(user, "usercontactinfo") and user.usercontactinfo.block_execution - - @staticmethod - def force_password_reset(user): - return hasattr(user, "usercontactinfo") and user.usercontactinfo.force_password_reset - - def disable_force_password_reset(self): - if hasattr(self, "usercontactinfo"): - self.usercontactinfo.force_password_reset = False - self.usercontactinfo.save() - - def enable_force_password_reset(self): - if hasattr(self, "usercontactinfo"): - self.usercontactinfo.force_password_reset = True - self.usercontactinfo.save() - - @staticmethod - def generate_full_name(user): - """Returns the first_name plus the last_name, with a space in between.""" - full_name = f"{user.first_name} {user.last_name} ({user.username})" - return full_name.strip() - - -class UserContactInfo(models.Model): - user = models.OneToOneField(Dojo_User, on_delete=models.CASCADE) - title = models.CharField(blank=True, null=True, max_length=150) - phone_regex = RegexValidator(regex=r"^\+?1?\d{9,15}$", - message=_("Phone number must be entered in the format: '+999999999'. " - "Up to 15 digits allowed.")) - phone_number = models.CharField(validators=[phone_regex], blank=True, - max_length=15, - help_text=_("Phone number must be entered in the format: '+999999999'. " - "Up to 15 digits allowed.")) - cell_number = models.CharField(validators=[phone_regex], blank=True, - max_length=15, - help_text=_("Phone number must be entered in the format: '+999999999'. " - "Up to 15 digits allowed.")) - twitter_username = models.CharField(blank=True, null=True, max_length=150) - github_username = models.CharField(blank=True, null=True, max_length=150) - slack_username = models.CharField(blank=True, null=True, max_length=150, help_text=_("Email address associated with your slack account"), verbose_name=_("Slack Email Address")) - slack_user_id = models.CharField(blank=True, null=True, max_length=25) - block_execution = models.BooleanField(default=False, help_text=_("Instead of async deduping a finding the findings will be deduped synchronously and will 'block' the user until completion.")) - force_password_reset = models.BooleanField(default=False, help_text=_("Forces this user to reset their password on next login.")) - ui_use_tailwind = models.BooleanField(default=False, verbose_name=_("Use new UI (beta)"), help_text=_("Opt in to the new Tailwind-based UI. Leave off for the classic UI.")) - token_last_reset = models.DateTimeField(null=True, blank=True, help_text=_("Timestamp of the most recent API token reset for this user.")) - password_last_reset = models.DateTimeField(null=True, blank=True, help_text=_("Timestamp of the most recent password reset for this user.")) - - -class System_Settings(models.Model): - enable_deduplication = models.BooleanField( - default=False, - blank=False, - verbose_name=_("Deduplicate findings"), - help_text=_("With this setting turned on, DefectDojo deduplicates findings by " - "comparing endpoints, cwe fields, and titles. " - "If two findings share a URL and have the same CWE or " - "title, DefectDojo marks the recent finding as a duplicate. " - "When deduplication is enabled, a list of " - "deduplicated findings is added to the engagement view.")) - delete_duplicates = models.BooleanField(default=False, blank=False, help_text=_("Requires next setting: maximum number of duplicates to retain.")) - max_dupes = models.IntegerField(blank=True, null=True, default=10, - verbose_name=_("Max Duplicates"), - help_text=_("When enabled, if a single " - "issue reaches the maximum " - "number of duplicates, the " - "oldest will be deleted. Duplicate will not be deleted when left empty. A value of 0 will remove all duplicates.")) - - email_from = models.CharField(max_length=200, default="no-reply@example.com", blank=True) - - enable_jira = models.BooleanField(default=False, - verbose_name=_("Enable JIRA integration"), - blank=False) - - enable_jira_web_hook = models.BooleanField(default=False, - verbose_name=_("Enable JIRA web hook"), - help_text=_("Please note: It is strongly recommended to use a secret below and / or IP whitelist the JIRA server using a proxy such as Nginx."), - blank=False) - - disable_jira_webhook_secret = models.BooleanField(default=False, - verbose_name=_("Disable web hook secret"), - help_text=_("Allows incoming requests without a secret (discouraged legacy behaviour)"), - blank=False) - - # will be set to random / uuid by initializer so null needs to be True - jira_webhook_secret = models.CharField(max_length=64, blank=False, null=True, verbose_name=_("JIRA Webhook URL"), - help_text=_("Secret needed in URL for incoming JIRA Webhook")) - - jira_choices = (("Critical", "Critical"), - ("High", "High"), - ("Medium", "Medium"), - ("Low", "Low"), - ("Info", "Info")) - jira_minimum_severity = models.CharField(max_length=20, blank=True, - null=True, choices=jira_choices, - default="Low") - jira_labels = models.CharField(max_length=200, blank=True, null=True, - help_text=_("JIRA issue labels space seperated")) - - add_vulnerability_id_to_jira_label = models.BooleanField(default=False, - verbose_name=_("Add vulnerability Id as a JIRA label"), - blank=False) - - enable_github = models.BooleanField(default=False, - verbose_name=_("Enable GITHUB integration"), - blank=False) - - enable_slack_notifications = \ - models.BooleanField(default=False, - verbose_name=_("Enable Slack notifications"), - blank=False) - slack_channel = models.CharField(max_length=100, default="", blank=True, - help_text=_("Optional. Needed if you want to send global notifications.")) - slack_token = models.CharField(max_length=100, default="", blank=True, - help_text=_("Token required for interacting " - "with Slack. Get one at " - "https://api.slack.com/tokens")) - slack_username = models.CharField(max_length=100, default="", blank=True, - help_text=_("Optional. Will take your bot name otherwise.")) - enable_msteams_notifications = \ - models.BooleanField(default=False, - verbose_name=_("Enable Microsoft Teams notifications"), - blank=False) - msteams_url = models.CharField(max_length=400, default="", blank=True, - help_text=_("The full URL of the " - "incoming webhook")) - enable_mail_notifications = models.BooleanField(default=False, blank=False) - mail_notifications_to = models.CharField(max_length=200, default="", - blank=True) - - enable_webhooks_notifications = \ - models.BooleanField(default=False, - verbose_name=_("Enable Webhook notifications"), - blank=False) - webhooks_notifications_timeout = models.IntegerField(default=10, - help_text=_("How many seconds will DefectDojo waits for response from webhook endpoint")) - - enforce_verified_status = models.BooleanField( - default=True, - verbose_name=_("Enforce Verified Status - Globally"), - help_text=_( - "When enabled, features such as product grading, jira " - "integration, metrics, and reports will only interact " - "with verified findings. This setting will override " - "individually scoped verified toggles.", - ), - ) - enforce_verified_status_jira = models.BooleanField( - default=True, - verbose_name=_("Enforce Verified Status - Jira"), - help_text=_("When enabled, findings must have a verified status to be pushed to jira."), - ) - enforce_verified_status_product_grading = models.BooleanField( - default=True, - verbose_name=_("Enforce Verified Status - Product Grading"), - help_text=_( - "When enabled, findings must have a verified status to be considered as part of a product's grading.", - ), - ) - enforce_verified_status_metrics = models.BooleanField( - default=True, - verbose_name=_("Enforce Verified Status - Metrics"), - help_text=_( - "When enabled, findings must have a verified status to be counted in metric calculations, " - "be included in reports, and filters.", - ), - ) - - false_positive_history = models.BooleanField( - default=False, help_text=_( - "(EXPERIMENTAL) DefectDojo will automatically mark the finding as a " - "false positive if an equal finding (according to its dedupe algorithm) " - "has been previously marked as a false positive on the same product. " - "ATTENTION: Although the deduplication algorithm is used to determine " - "if a finding should be marked as a false positive, this feature will " - "not work if deduplication is enabled since it doesn't make sense to use both.", - ), - ) - - retroactive_false_positive_history = models.BooleanField( - default=False, help_text=_( - "(EXPERIMENTAL) FP History will also retroactively mark/unmark all " - "existing equal findings in the same product as a false positives. " - "Only works if the False Positive History feature is also enabled.", - ), - ) - - url_prefix = models.CharField(max_length=300, default="", blank=True, help_text=_("URL prefix if DefectDojo is installed in it's own virtual subdirectory.")) - team_name = models.CharField(max_length=100, default="", blank=True) - enable_product_grade = models.BooleanField(default=False, verbose_name=_("Enable Product Grading"), help_text=_("Displays a grade letter next to a product to show the overall health.")) - product_grade_a = models.IntegerField(default=90, - verbose_name=_("Grade A"), - help_text=_("Percentage score for an " - "'A' >=")) - product_grade_b = models.IntegerField(default=80, - verbose_name=_("Grade B"), - help_text=_("Percentage score for a " - "'B' >=")) - product_grade_c = models.IntegerField(default=70, - verbose_name=_("Grade C"), - help_text=_("Percentage score for a " - "'C' >=")) - product_grade_d = models.IntegerField(default=60, - verbose_name=_("Grade D"), - help_text=_("Percentage score for a " - "'D' >=")) - product_grade_f = models.IntegerField(default=59, - verbose_name=_("Grade F"), - help_text=_("Percentage score for an " - "'F' <=")) - enable_product_tag_inheritance = models.BooleanField( - default=False, - blank=False, - verbose_name=_("Enable Product Tag Inheritance"), - help_text=_("Enables product tag inheritance globally for all products. Any tags added on a product will automatically be added to all Engagements, Tests, and Findings")) - - enable_benchmark = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable Benchmarks"), - help_text=_("Enables Benchmarks such as the OWASP ASVS " - "(Application Security Verification Standard)")) - - enable_similar_findings = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable Similar Findings"), - help_text=_("Enable the query of similar findings on the view finding page. This feature can involve potentially large queries and negatively impact performance")) - - engagement_auto_close = models.BooleanField( - default=False, - blank=False, - verbose_name=_("Enable Engagement Auto-Close"), - help_text=_("Closes an engagement after 3 days (default) past due date including last update.")) - - engagement_auto_close_days = models.IntegerField( - default=3, - blank=False, - verbose_name=_("Engagement Auto-Close Days"), - help_text=_("Closes an engagement after the specified number of days past due date including last update.")) - - enable_finding_sla = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable Finding SLA's"), - help_text=_("Enables Finding SLA's for time to remediate.")) - - enable_notify_sla_active = models.BooleanField( - default=False, - blank=False, - verbose_name=_("Enable Notify SLA's Breach for active Findings"), - help_text=_("Enables Notify when time to remediate according to Finding SLA's is breached for active Findings.")) - - enable_notify_sla_active_verified = models.BooleanField( - default=False, - blank=False, - verbose_name=_("Enable Notify SLA's Breach for active, verified Findings"), - help_text=_("Enables Notify when time to remediate according to Finding SLA's is breached for active, verified Findings.")) - - enable_notify_sla_jira_only = models.BooleanField( - default=False, - blank=False, - verbose_name=_("Enable Notify SLA's Breach only for Findings linked to JIRA"), - help_text=_("Enables Notify when time to remediate according to Finding SLA's is breached for Findings that are linked to JIRA issues. Notification is disabled for Findings not linked to JIRA issues")) - - enable_notify_sla_exponential_backoff = models.BooleanField( - default=False, - blank=False, - verbose_name=_("Enable an exponential backoff strategy for SLA breach notifications."), - help_text=_("Enable an exponential backoff strategy for SLA breach notifications, e.g. 1, 2, 4, 8, etc. Otherwise it alerts every day")) - - allow_anonymous_survey_repsonse = models.BooleanField( - default=False, - blank=False, - verbose_name=_("Allow Anonymous Survey Responses"), - help_text=_("Enable anyone with a link to the survey to answer a survey"), - ) - disclaimer_notifications = models.TextField(max_length=3000, default="", blank=True, - verbose_name=_("Custom Disclaimer for Notifications"), - help_text=_("Include this custom disclaimer on all notifications")) - disclaimer_reports = models.TextField(max_length=5000, default="", blank=True, - verbose_name=_("Custom Disclaimer for Reports"), - help_text=_("Include this custom disclaimer on generated reports")) - disclaimer_reports_forced = models.BooleanField( - default=False, - blank=False, - verbose_name=_("Force to add disclaimer reports"), - help_text=_("Disclaimer will be added to all reports even if user didn't selected 'Include disclaimer'.")) - disclaimer_notes = models.TextField(max_length=3000, default="", blank=True, - verbose_name=_("Custom Disclaimer for Notes"), - help_text=_("Include this custom disclaimer next to input form for notes")) - risk_acceptance_form_default_days = models.IntegerField(null=True, blank=True, default=180, help_text=_("Default expiry period for risk acceptance form.")) - risk_acceptance_notify_before_expiration = models.IntegerField(null=True, blank=True, default=10, - verbose_name=_("Risk acceptance expiration heads up days"), help_text=_("Notify X days before risk acceptance expires. Leave empty to disable.")) - enable_questionnaires = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable questionnaires"), - help_text=_("With this setting turned off, questionnaires will be disabled in the user interface.")) - enable_checklists = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable checklists"), - help_text=_("With this setting turned off, checklists will be disabled in the user interface.")) - enable_endpoint_metadata_import = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable Endpoint Metadata Import"), - help_text=_("With this setting turned off, endpoint metadata import will be disabled in the user interface.")) - enable_user_profile_editable = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable user profile for writing"), - help_text=_("When turned on users can edit their profiles")) - enable_product_tracking_files = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable Product Tracking Files"), - help_text=_("With this setting turned off, the product tracking files will be disabled in the user interface.")) - enable_finding_groups = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable Finding Groups"), - help_text=_("With this setting turned off, the Finding Groups will be disabled.")) - enable_ui_table_based_searching = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable UI Table Based Filtering/Sorting"), - help_text=_("With this setting enabled, table headings will contain sort buttons for the current page of data in addition to sorting buttons that consider data from all pages.")) - enable_calendar = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable Calendar"), - help_text=_("With this setting turned off, the Calendar will be disabled in the user interface.")) - enable_cvss3_display = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable CVSS3 Display"), - help_text=_("With this setting turned off, CVSS3 fields will be hidden in the user interface.")) - enable_cvss4_display = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Enable CVSS4 Display"), - help_text=_("With this setting turned off, CVSS4 fields will be hidden in the user interface.")) - minimum_password_length = models.IntegerField( - default=9, - verbose_name=_("Minimum password length"), - help_text=_("Requires user to set passwords greater than minimum length."), - validators=[MinValueValidator(9), MaxValueValidator(48)]) - maximum_password_length = models.IntegerField( - default=48, - verbose_name=_("Maximum password length"), - help_text=_("Requires user to set passwords less than maximum length."), - validators=[MinValueValidator(9), MaxValueValidator(48)]) - number_character_required = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Password must contain one digit"), - help_text=_("Requires user passwords to contain at least one digit (0-9).")) - special_character_required = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Password must contain one special character"), - help_text=_("Requires user passwords to contain at least one special character (()[]{}|\\`~!@#$%^&*_-+=;:'\",<>./?).")) - lowercase_character_required = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Password must contain one lowercase letter"), - help_text=_("Requires user passwords to contain at least one lowercase letter (a-z).")) - uppercase_character_required = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Password must contain one uppercase letter"), - help_text=_("Requires user passwords to contain at least one uppercase letter (A-Z).")) - non_common_password_required = models.BooleanField( - default=True, - blank=False, - verbose_name=_("Password must not be common"), - help_text=_("Requires user passwords to not be part of list of common passwords.")) - api_expose_error_details = models.BooleanField( - default=False, - blank=False, - verbose_name=_("API expose error details"), - help_text=_("When turned on, the API will expose error details in the response.")) - filter_string_matching = models.BooleanField( - default=False, - blank=False, - verbose_name=_("Filter String Matching Optimization"), - help_text=_( - "When turned on, all filter operations in the UI will require string matches rather than ID. " - "This is a performance enhancement to avoid fetching objects unnecessarily.", - )) - - from dojo.middleware import System_Settings_Manager # noqa: PLC0415 circular import - objects = System_Settings_Manager() - - def clean(self): - super().clean() - - if ( - self.minimum_password_length is not None - and self.maximum_password_length is not None - ): - if self.minimum_password_length > self.maximum_password_length: - msg = "Minimum required password length must be larger than the maximum required password length." - raise ValidationError({ - "minimum_password_length": msg, - }) +from dojo.regulations.models import Regulation # noqa: E402, F401, I001 -- re-export; user/system_settings block intentionally out-of-order (load-order) +from dojo.user.models import Contact, Dojo_User, UserContactInfo # noqa: E402, F401 -- must precede system_settings (middleware load-order) +from dojo.system_settings.models import System_Settings # noqa: E402, F401 -- re-export def get_current_date(): @@ -631,234 +135,30 @@ def get_current_datetime(): return timezone.now() -class Contact(models.Model): - name = models.CharField(max_length=100) - email = models.EmailField() - team = models.CharField(max_length=100) - is_admin = models.BooleanField(default=False) - is_globally_read_only = models.BooleanField(default=False) - updated = models.DateTimeField(auto_now=True) - - -class Note_Type(models.Model): - name = models.CharField(max_length=100, unique=True) - description = models.CharField(max_length=200) - is_single = models.BooleanField(default=False, null=False) - is_active = models.BooleanField(default=True, null=False) - is_mandatory = models.BooleanField(default=True, null=False) - - def __str__(self): - return self.name - - -class NoteHistory(models.Model): - note_type = models.ForeignKey(Note_Type, null=True, blank=True, on_delete=models.CASCADE) - data = models.TextField() - time = models.DateTimeField(null=True, editable=False, - default=get_current_datetime) - current_editor = models.ForeignKey(Dojo_User, editable=False, null=True, on_delete=models.CASCADE) - - def copy(self): - copy = copy_model_util(self) - copy.save() - return copy - - -class Notes(models.Model): - note_type = models.ForeignKey(Note_Type, related_name="note_type", null=True, blank=True, on_delete=models.CASCADE) - entry = models.TextField() - date = models.DateTimeField(null=False, editable=False, - default=get_current_datetime) - author = models.ForeignKey(Dojo_User, related_name="editor_notes_set", editable=False, on_delete=models.CASCADE) - private = models.BooleanField(default=False) - edited = models.BooleanField(default=False) - editor = models.ForeignKey(Dojo_User, related_name="author_notes_set", editable=False, null=True, on_delete=models.CASCADE) - edit_time = models.DateTimeField(null=True, editable=False, - default=get_current_datetime) - history = models.ManyToManyField(NoteHistory, blank=True, - editable=False) - - class Meta: - ordering = ["-date"] - - def __str__(self): - return self.entry - - def copy(self): - copy = copy_model_util(self) - # Save the necessary ManyToMany relationships - old_history = list(self.history.all()) - # Save the object before setting any ManyToMany relationships - copy.save() - # Copy the history - for history in old_history: - copy.history.add(history.copy()) - - return copy - - -class FileUpload(models.Model): - title = models.CharField(max_length=100, unique=True) - file = models.FileField(upload_to=UniqueUploadNameProvider("uploaded_files")) - - def delete(self, *args, **kwargs): - """Delete the model and remove the file from storage.""" - storage = self.file.storage - path = self.file.path - super().delete(*args, **kwargs) - if path and storage.exists(path): - storage.delete(path) - - def copy(self): - copy = copy_model_util(self) - # Add unique modifier to file name - # Truncate title to ensure it doesn't exceed max_length (100) when appending suffix - # Suffix " - clone-{8 chars}" is 17 characters, so truncate to 83 chars - clone_suffix = f" - clone-{str(uuid4())[:8]}" - max_title_length = 100 - len(clone_suffix) - truncated_title = self.title[:max_title_length] if len(self.title) > max_title_length else self.title - copy.title = f"{truncated_title}{clone_suffix}" - # Create new unique file name - current_url = self.file.url - _, current_full_filename = current_url.rsplit("/", 1) - _, extension = current_full_filename.split(".", 1) - new_file = ContentFile(self.file.read(), name=f"{uuid4()}.{extension}") - copy.file = new_file - copy.save() - - return copy - - def get_accessible_url(self, obj, obj_id): - if isinstance(obj, Engagement): - obj_type = "Engagement" - elif isinstance(obj, Test): - obj_type = "Test" - elif isinstance(obj, Finding): - obj_type = "Finding" - - return f"access_file/{self.id}/{obj_id}/{obj_type}" - - def clean(self): - if not self.title: - self.title = "" - - valid_extensions = settings.FILE_UPLOAD_TYPES - - # why does this not work with self.file.... - file_name = self.file.url if self.file else self.title - if Path(file_name).suffix.lower() not in valid_extensions: - if accepted_extensions := f"{', '.join(valid_extensions)}": - msg = ( - _("Unsupported extension. Supported extensions are as follows: %s") % accepted_extensions - ) - else: - msg = ( - _("File uploads are prohibited due to the list of acceptable file extensions being empty") - ) - raise ValidationError(msg) - - -class Product_Type(BaseModel): - - """ - Product types represent the top level model, these can be business unit divisions, different offices or locations, development teams, or any other logical way of distinguishing "types" of products. - ` - Examples: - * IAM Team - * Internal / 3rd Party - * Main company / Acquisition - * San Francisco / New York offices - """ - - name = models.CharField(max_length=255, unique=True) - description = models.CharField(max_length=4000, null=True, blank=True) - critical_product = models.BooleanField(default=False) - key_product = models.BooleanField(default=False) - authorized_users = models.ManyToManyField(Dojo_User, related_name="authorized_product_types", blank=True) - - class Meta: - ordering = ("name",) - - def __str__(self): - return self.name - - def get_absolute_url(self): - return reverse("product_type", args=[str(self.id)]) - - def get_breadcrumbs(self): - return [{"title": str(self), - "url": reverse("edit_product_type", args=(self.id,))}] - - @cached_property - def critical_present(self): - c_findings = Finding.objects.filter( - test__engagement__product__prod_type=self, severity="Critical") - if c_findings.count() > 0: - return True - return None - - @cached_property - def high_present(self): - c_findings = Finding.objects.filter( - test__engagement__product__prod_type=self, severity="High") - if c_findings.count() > 0: - return True - return None - - @cached_property - def calc_health(self): - h_findings = Finding.objects.filter( - test__engagement__product__prod_type=self, severity="High") - c_findings = Finding.objects.filter( - test__engagement__product__prod_type=self, severity="Critical") - health = 100 - if c_findings.count() > 0: - health = 40 - health -= ((c_findings.count() - 1) * 5) - if h_findings.count() > 0: - if health == 100: - health = 60 - health -= ((h_findings.count() - 1) * 2) - if health < 5: - return 5 - return health - - # only used by bulk risk acceptance api - @property - def unaccepted_open_findings(self): - return Finding.objects.filter(risk_accepted=False, active=True, duplicate=False, test__engagement__product__prod_type=self) - - -class Product_Line(models.Model): - name = models.CharField(max_length=300) - description = models.CharField(max_length=2000) - - def __str__(self): - return self.name - - -class Report_Type(models.Model): - name = models.CharField(max_length=255) - - -class Test_Type(models.Model): - name = models.CharField(max_length=200, unique=True) - static_tool = models.BooleanField(default=False) - dynamic_tool = models.BooleanField(default=False) - active = models.BooleanField(default=True) - dynamically_generated = models.BooleanField( - default=False, - help_text=_("Set to True for test types that are created at import time")) - - class Meta: - ordering = ("name",) - - def __str__(self): - return self.name - - def get_breadcrumbs(self): - return [{"title": str(self), - "url": None}] +from dojo.file_uploads.models import FileAccessToken, FileUpload # noqa: E402, F401 -- re-export +from dojo.note_type.models import Note_Type # noqa: E402, F401 -- re-export +from dojo.notes.models import ( # noqa: E402, F401 -- re-export; Notes used by Risk_Acceptance.notes M2M below + NoteHistory, + Notes, +) +from dojo.product.models import ( # noqa: E402 -- re-export; class-body FKs below reference these + Product, + Product_API_Scan_Configuration, # noqa: F401 -- re-export + Product_Line, # noqa: F401 -- re-export +) +from dojo.product_type.models import Product_Type # noqa: E402, F401 -- re-export +from dojo.reports.models import Report_Type # noqa: E402, F401 -- re-export +from dojo.test.models import ( # noqa: E402 -- re-export; class-body FKs below reference these + IMPORT_ACTIONS, # noqa: F401 -- re-export + IMPORT_CLOSED_FINDING, # noqa: F401 -- re-export + IMPORT_CREATED_FINDING, # noqa: F401 -- re-export + IMPORT_REACTIVATED_FINDING, # noqa: F401 -- re-export + IMPORT_UNTOUCHED_FINDING, # noqa: F401 -- re-export + Test, + Test_Import, # noqa: F401 -- re-export + Test_Import_Finding_Action, # noqa: F401 -- re-export + Test_Type, # noqa: F401 -- re-export +) class DojoMeta(models.Model): @@ -1046,2914 +346,114 @@ def get_summary(self): return f"{self.name} - Critical: {self.critical}, High: {self.high}, Medium: {self.medium}, Low: {self.low}" -class Product(BaseModel): - WEB_PLATFORM = "web" - IOT = "iot" - DESKTOP_PLATFORM = "desktop" - MOBILE_PLATFORM = "mobile" - WEB_SERVICE_PLATFORM = "web service" - PLATFORM_CHOICES = ( - (WEB_SERVICE_PLATFORM, _("API")), - (DESKTOP_PLATFORM, _("Desktop")), - (IOT, _("Internet of Things")), - (MOBILE_PLATFORM, _("Mobile")), - (WEB_PLATFORM, _("Web")), - ) - - CONSTRUCTION = "construction" - PRODUCTION = "production" - RETIREMENT = "retirement" - LIFECYCLE_CHOICES = ( - (CONSTRUCTION, _("Construction")), - (PRODUCTION, _("Production")), - (RETIREMENT, _("Retirement")), - ) - - THIRD_PARTY_LIBRARY_ORIGIN = "third party library" - PURCHASED_ORIGIN = "purchased" - CONTRACTOR_ORIGIN = "contractor" - INTERNALLY_DEVELOPED_ORIGIN = "internal" - OPEN_SOURCE_ORIGIN = "open source" - OUTSOURCED_ORIGIN = "outsourced" - ORIGIN_CHOICES = ( - (THIRD_PARTY_LIBRARY_ORIGIN, _("Third Party Library")), - (PURCHASED_ORIGIN, _("Purchased")), - (CONTRACTOR_ORIGIN, _("Contractor Developed")), - (INTERNALLY_DEVELOPED_ORIGIN, _("Internally Developed")), - (OPEN_SOURCE_ORIGIN, _("Open Source")), - (OUTSOURCED_ORIGIN, _("Outsourced")), - ) - - VERY_HIGH_CRITICALITY = "very high" - HIGH_CRITICALITY = "high" - MEDIUM_CRITICALITY = "medium" - LOW_CRITICALITY = "low" - VERY_LOW_CRITICALITY = "very low" - NONE_CRITICALITY = "none" - BUSINESS_CRITICALITY_CHOICES = ( - (VERY_HIGH_CRITICALITY, _("Very High")), - (HIGH_CRITICALITY, _("High")), - (MEDIUM_CRITICALITY, _("Medium")), - (LOW_CRITICALITY, _("Low")), - (VERY_LOW_CRITICALITY, _("Very Low")), - (NONE_CRITICALITY, _("None")), - ) +from dojo.tool_config.models import Tool_Configuration # noqa: E402, F401 -- re-export +from dojo.tool_type.models import Tool_Type # noqa: E402, F401 -- re-export - name = models.CharField(max_length=255, unique=True) - description = models.CharField(max_length=4000) - product_manager = models.ForeignKey(Dojo_User, null=True, blank=True, - related_name="product_manager", on_delete=models.RESTRICT) - technical_contact = models.ForeignKey(Dojo_User, null=True, blank=True, - related_name="technical_contact", on_delete=models.RESTRICT) - team_manager = models.ForeignKey(Dojo_User, null=True, blank=True, - related_name="team_manager", on_delete=models.RESTRICT) +class Network_Locations(models.Model): + location = models.CharField(max_length=500, help_text=_("Location of network testing: Examples: VPN, Internet or Internal.")) - prod_type = models.ForeignKey(Product_Type, related_name="prod_type", - null=False, blank=False, on_delete=models.CASCADE) - sla_configuration = models.ForeignKey(SLA_Configuration, - related_name="sla_config", - null=False, - blank=False, - default=1, - on_delete=models.RESTRICT) - tid = models.IntegerField(default=0, editable=False) - authorized_users = models.ManyToManyField(Dojo_User, related_name="authorized_products", blank=True) - prod_numeric_grade = models.IntegerField(null=True, blank=True) + def __str__(self): + return self.location - # Metadata - business_criticality = models.CharField(max_length=9, choices=BUSINESS_CRITICALITY_CHOICES, blank=True, null=True) - platform = models.CharField(max_length=11, choices=PLATFORM_CHOICES, blank=True, null=True) - lifecycle = models.CharField(max_length=12, choices=LIFECYCLE_CHOICES, blank=True, null=True) - origin = models.CharField(max_length=19, choices=ORIGIN_CHOICES, blank=True, null=True) - user_records = models.PositiveIntegerField(blank=True, null=True, help_text=_("Estimate the number of user records within the application.")) - revenue = models.DecimalField(max_digits=15, decimal_places=2, blank=True, null=True, validators=[MinValueValidator(Decimal("0.00"))], help_text=_("Estimate the application's revenue.")) - external_audience = models.BooleanField(default=False, help_text=_("Specify if the application is used by people outside the organization.")) - internet_accessible = models.BooleanField(default=False, help_text=_("Specify if the application is accessible from the public internet.")) - regulations = models.ManyToManyField(Regulation, blank=True) - tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this product. Choose from the list or add new tags. Press Enter key to add.")) - enable_product_tag_inheritance = models.BooleanField( - default=False, - blank=False, - verbose_name=_("Enable Product Tag Inheritance"), - help_text=_("Enables product tag inheritance. Any tags added on a product will automatically be added to all Engagements, Tests, and Findings")) - enable_simple_risk_acceptance = models.BooleanField(default=False, help_text=_("Allows simple risk acceptance by checking/unchecking a checkbox.")) - enable_full_risk_acceptance = models.BooleanField(default=True, help_text=_("Allows full risk acceptance using a risk acceptance form, expiration date, uploaded proof, etc.")) +from dojo.development_environment.models import Development_Environment # noqa: E402, F401 -- re-export +from dojo.endpoint.models import Endpoint, Endpoint_Params, Endpoint_Status # noqa: E402, F401 -- re-export +from dojo.engagement.models import ( # noqa: E402 -- re-export; class-body FKs below reference these + ENGAGEMENT_STATUS_CHOICES, # noqa: F401 -- re-export + Engagement, + Engagement_Presets, # noqa: F401 -- re-export +) - disable_sla_breach_notifications = models.BooleanField( - default=False, - blank=False, - verbose_name=_("Disable SLA breach notifications"), - help_text=_("Disable SLA breach notifications if configured in the global settings")) - async_updating = models.BooleanField(default=False, - help_text=_("Findings under this Product or SLA configuration are asynchronously being updated")) - class Meta: - ordering = ("name",) +class Sonarqube_Issue(models.Model): + key = models.CharField(max_length=60, unique=True, help_text=_("SonarQube issue key")) + status = models.CharField(max_length=20, help_text=_("SonarQube issue status")) + type = models.CharField(max_length=20, help_text=_("SonarQube issue type")) def __str__(self): - return self.name - - def save(self, *args, **kwargs): - # get the product's sla config before saving (if this is an existing product) - initial_sla_config = None - if self.pk is not None: - initial_sla_config = getattr(Product.objects.get(pk=self.pk), "sla_configuration", None) - # if initial sla config exists and async finding update is already running, revert sla config before saving - if initial_sla_config and self.async_updating: - self.sla_configuration = initial_sla_config - - super().save(*args, **kwargs) - - # if the initial sla config exists and async finding update is not running - if initial_sla_config is not None and not self.async_updating: - # get the new sla config from the saved product - new_sla_config = getattr(self, "sla_configuration", None) - # if the sla config has changed, update finding sla expiration dates within this product - if new_sla_config and (initial_sla_config != new_sla_config): - # set the async updating flag to true for this product - self.async_updating = True - super().save(*args, **kwargs) - # set the async updating flag to true for the sla config assigned to this product - sla_config = getattr(self, "sla_configuration", None) - if sla_config: - sla_config.async_updating = True - super(SLA_Configuration, sla_config).save() - # launch the async task to update all finding sla expiration dates - from dojo.sla_config.helpers import async_update_sla_expiration_dates_sla_config_sync # noqa: I001, PLC0415 circular import - from dojo.celery_dispatch import dojo_dispatch_task # noqa: PLC0415 circular import - - dojo_dispatch_task( - async_update_sla_expiration_dates_sla_config_sync, - sla_config.id, - [self.id], - ) - # The async task refetches and resets async_updating on its own copies. - # Mirror that on this in-memory product and the in-memory sla_config so a - # subsequent save() on either does not trigger their lock-revert paths. - self.async_updating = False - if sla_config: - sla_config.async_updating = False + return self.key - def get_absolute_url(self): - return reverse("view_product", args=[str(self.id)]) - @cached_property - def findings_count(self): - try: - # if prefetched, it's already there - return self.active_finding_count - except AttributeError: - # ideally it's always prefetched and we can remove this code in the future - self.active_finding_count = Finding.objects.filter(active=True, - test__engagement__product=self).count() - return self.active_finding_count +class Sonarqube_Issue_Transition(models.Model): + sonarqube_issue = models.ForeignKey(Sonarqube_Issue, on_delete=models.CASCADE, db_index=True) + created = models.DateTimeField(auto_now_add=True, null=False) + finding_status = models.CharField(max_length=100) + sonarqube_status = models.CharField(max_length=50) + transitions = models.CharField(max_length=100) - @cached_property - def findings_active_verified_count(self): - try: - # if prefetched, it's already there - return self.active_verified_finding_count - except AttributeError: - # ideally it's always prefetched and we can remove this code in the future - self.active_verified_finding_count = Finding.objects.filter(active=True, - verified=True, - test__engagement__product=self).count() - return self.active_verified_finding_count + class Meta: + ordering = ("-created", ) - # TODO: Delete this after the move to Locations - @cached_property - def endpoint_host_count(self): - # active_endpoints is (should be) prefetched - endpoints = getattr(self, "active_endpoints", None) - hosts = [] - for e in endpoints: - if e.host in hosts: - continue - hosts.append(e.host) +from dojo.finding.models import ( # noqa: E402 -- re-export; class-body FKs below reference these + CWE, # noqa: F401 -- re-export + BurpRawRequestResponse, # noqa: F401 -- re-export + Finding, + Finding_Group, # noqa: F401 -- re-export + Finding_Template, + Vulnerability_Id, # noqa: F401 -- re-export +) - return len(hosts) - # TODO: Delete this after the move to Locations - @cached_property - def endpoint_count(self): - # active_endpoints is (should be) prefetched - endpoints = getattr(self, "active_endpoints", None) - if endpoints: - return len(self.active_endpoints) - return 0 +class Check_List(models.Model): + session_management = models.CharField(max_length=50, default="none") + session_issues = models.ManyToManyField(Finding, + related_name="session_issues", + blank=True) + encryption_crypto = models.CharField(max_length=50, default="none") + crypto_issues = models.ManyToManyField(Finding, + related_name="crypto_issues", + blank=True) + configuration_management = models.CharField(max_length=50, default="") + config_issues = models.ManyToManyField(Finding, + related_name="config_issues", + blank=True) + authentication = models.CharField(max_length=50, default="none") + auth_issues = models.ManyToManyField(Finding, + related_name="auth_issues", + blank=True) + authorization_and_access_control = models.CharField(max_length=50, + default="none") + author_issues = models.ManyToManyField(Finding, + related_name="author_issues", + blank=True) + data_input_sanitization_validation = models.CharField(max_length=50, + default="none") + data_issues = models.ManyToManyField(Finding, related_name="data_issues", + blank=True) + sensitive_data = models.CharField(max_length=50, default="none") + sensitive_issues = models.ManyToManyField(Finding, + related_name="sensitive_issues", + blank=True) + other = models.CharField(max_length=50, default="none") + other_issues = models.ManyToManyField(Finding, related_name="other_issues", + blank=True) + engagement = models.ForeignKey(Engagement, editable=False, + related_name="eng_for_check", on_delete=models.CASCADE) - def open_findings(self, start_date=None, end_date=None): - if start_date is None or end_date is None: - return {} + @staticmethod + def get_status(pass_fail): + if pass_fail == "Pass": # noqa: S105 + return "success" + if pass_fail == "Fail": # noqa: S105 + return "danger" + return "warning" - from dojo.utils import get_system_setting # noqa: PLC0415 circular import - findings = Finding.objects.filter(test__engagement__product=self, - mitigated__isnull=True, - false_p=False, - duplicate=False, - out_of_scope=False, - date__range=[start_date, - end_date]) + def get_breadcrumb(self): + bc = self.engagement.get_breadcrumb() + bc += [{"title": "Check List", + "url": reverse("complete_checklist", + args=(self.engagement.id,))}] + return bc - if get_system_setting("enforce_verified_status", True) or get_system_setting("enforce_verified_status_metrics", True): - findings = findings.filter(verified=True) - critical = findings.filter(severity="Critical").count() - high = findings.filter(severity="High").count() - medium = findings.filter(severity="Medium").count() - low = findings.filter(severity="Low").count() - - return {"Critical": critical, - "High": high, - "Medium": medium, - "Low": low, - "Total": (critical + high + medium + low)} - - def get_breadcrumbs(self): - return [{"title": str(self), - "url": reverse("view_product", args=(self.id,))}] - - @property - def get_product_type(self): - return self.prod_type if self.prod_type is not None else "unknown" - - # only used in APIv2 serializers.py, should be deprecated or at least prefetched - def open_findings_list(self): - findings = Finding.objects.filter(test__engagement__product=self, active=True).values_list("id", flat=True) - return list(findings) - - @property - def has_jira_configured(self): - from dojo.jira import services as jira_services # noqa: PLC0415 circular import - return jira_services.has_configured(self) - - def violates_sla(self): - findings = Finding.objects.filter(test__engagement__product=self, - active=True, - sla_expiration_date__lt=timezone.now().date()) - return findings.count() > 0 - - -class Tool_Type(models.Model): - name = models.CharField(max_length=200) - description = models.CharField(max_length=2000, null=True, blank=True) - - class Meta: - ordering = ["name"] - - def __str__(self): - return self.name - - -class Tool_Configuration(models.Model): - name = models.CharField(max_length=200, null=False) - description = models.CharField(max_length=2000, null=True, blank=True) - url = models.CharField(max_length=2000, null=True, blank=True) - tool_type = models.ForeignKey(Tool_Type, related_name="tool_type", on_delete=models.CASCADE) - authentication_type = models.CharField(max_length=15, - choices=( - ("API", "API Key"), - ("Password", - "Username/Password"), - ("SSH", "SSH")), - null=True, blank=True) - extras = models.CharField(max_length=255, null=True, blank=True, help_text=_("Additional definitions that will be " - "consumed by scanner")) - username = models.CharField(max_length=200, null=True, blank=True) - password = models.CharField(max_length=600, null=True, blank=True) - auth_title = models.CharField(max_length=200, null=True, blank=True, - verbose_name=_("Title for SSH/API Key")) - ssh = models.CharField(max_length=6000, null=True, blank=True) - api_key = models.CharField(max_length=600, null=True, blank=True, - verbose_name=_("API Key")) - - class Meta: - ordering = ["name"] - - def __str__(self): - return self.name - - -class Product_API_Scan_Configuration(models.Model): - product = models.ForeignKey(Product, null=False, blank=False, on_delete=models.CASCADE) - tool_configuration = models.ForeignKey(Tool_Configuration, null=False, blank=False, on_delete=models.CASCADE) - service_key_1 = models.CharField(max_length=200, null=True, blank=True) - service_key_2 = models.CharField(max_length=200, null=True, blank=True) - service_key_3 = models.CharField(max_length=200, null=True, blank=True) - - def __str__(self): - name = self.tool_configuration.name - if self.service_key_1 or self.service_key_2 or self.service_key_3: - name += f" ({self.details})" - return name - - @property - def details(self): - details = "" - if self.service_key_1: - details += f"{self.service_key_1}" - if self.service_key_2: - details += f" | {self.service_key_2}" - if self.service_key_3: - details += f" | {self.service_key_3}" - return details - - -# declare form here as we can't import forms.py due to circular imports not even locally -class ToolConfigForm_Admin(forms.ModelForm): - password = forms.CharField(widget=forms.PasswordInput, required=False) - api_key = forms.CharField(widget=forms.PasswordInput, required=False) - ssh = forms.CharField(widget=forms.PasswordInput, required=False) - - # django doesn't seem to have an easy way to handle password fields as PasswordInput requires reentry of passwords - password_from_db = None - ssh_from_db = None - api_key_from_db = None - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - if self.instance: - # keep password from db to use if the user entered no password - self.password_from_db = self.instance.password - self.ssh_from_db = self.instance.ssh - self.api_key = self.instance.api_key - - def clean(self): - cleaned_data = super().clean() - if not cleaned_data["password"] and not cleaned_data["ssh"] and not cleaned_data["api_key"]: - cleaned_data["password"] = self.password_from_db - cleaned_data["ssh"] = self.ssh_from_db - cleaned_data["api_key"] = self.api_key_from_db - - return cleaned_data - - -class Tool_Configuration_Admin(admin.ModelAdmin): - form = ToolConfigForm_Admin - - -class Network_Locations(models.Model): - location = models.CharField(max_length=500, help_text=_("Location of network testing: Examples: VPN, Internet or Internal.")) - - def __str__(self): - return self.location - - -class Engagement_Presets(models.Model): - title = models.CharField(max_length=500, default=None, help_text=_("Brief description of preset.")) - test_type = models.ManyToManyField(Test_Type, default=None, blank=True) - network_locations = models.ManyToManyField(Network_Locations, default=None, blank=True) - notes = models.CharField(max_length=2000, help_text=_("Description of what needs to be tested or setting up environment for testing"), null=True, blank=True) - scope = models.CharField(max_length=800, help_text=_("Scope of Engagement testing, IP's/Resources/URL's)"), default=None, blank=True) - product = models.ForeignKey(Product, on_delete=models.CASCADE) - created = models.DateTimeField(auto_now_add=True, null=False) - - class Meta: - ordering = ["title"] - - def __str__(self): - return self.title - - -ENGAGEMENT_STATUS_CHOICES = (("Not Started", "Not Started"), - ("Blocked", "Blocked"), - ("Cancelled", "Cancelled"), - ("Completed", "Completed"), - ("In Progress", "In Progress"), - ("On Hold", "On Hold"), - ("Scheduled", "Scheduled"), - ("Waiting for Resource", "Waiting for Resource")) - - -class Engagement(BaseModel): - name = models.CharField(max_length=300, null=True, blank=True) - description = models.CharField(max_length=2000, null=True, blank=True) - version = models.CharField(max_length=100, null=True, blank=True, help_text=_("Version of the product the engagement tested.")) - first_contacted = models.DateField(null=True, blank=True) - target_start = models.DateField(null=False, blank=False) - target_end = models.DateField(null=False, blank=False) - lead = models.ForeignKey(Dojo_User, editable=True, null=True, blank=True, on_delete=models.RESTRICT) - requester = models.ForeignKey(Contact, null=True, blank=True, on_delete=models.CASCADE) - preset = models.ForeignKey(Engagement_Presets, null=True, blank=True, help_text=_("Settings and notes for performing this engagement."), on_delete=models.CASCADE) - reason = models.CharField(max_length=2000, null=True, blank=True) - report_type = models.ForeignKey(Report_Type, null=True, blank=True, on_delete=models.CASCADE) - product = models.ForeignKey(Product, on_delete=models.CASCADE) - active = models.BooleanField(default=True, editable=False) - tracker = models.URLField(max_length=200, help_text=_("Link to epic or ticket system with changes to version."), editable=True, blank=True, null=True) - test_strategy = models.URLField(editable=True, blank=True, null=True) - threat_model = models.BooleanField(default=True) - api_test = models.BooleanField(default=True) - pen_test = models.BooleanField(default=True) - check_list = models.BooleanField(default=True) - notes = models.ManyToManyField(Notes, blank=True, editable=False) - files = models.ManyToManyField(FileUpload, blank=True, editable=False) - status = models.CharField(editable=True, max_length=2000, default="Not Started", - null=True, - choices=ENGAGEMENT_STATUS_CHOICES) - progress = models.CharField(max_length=100, - default="threat_model", editable=False) - tmodel_path = models.CharField(max_length=1000, default="none", - editable=False, blank=True, null=True) - risk_acceptance = models.ManyToManyField("Risk_Acceptance", - default=None, - editable=False, - blank=True) - done_testing = models.BooleanField(default=False, editable=False) - engagement_type = models.CharField(editable=True, max_length=30, default="Interactive", - null=True, - choices=(("Interactive", "Interactive"), - ("CI/CD", "CI/CD"))) - build_id = models.CharField(editable=True, max_length=150, - null=True, blank=True, help_text=_("Build ID of the product the engagement tested."), verbose_name=_("Build ID")) - commit_hash = models.CharField(editable=True, max_length=150, - null=True, blank=True, help_text=_("Commit hash from repo"), verbose_name=_("Commit Hash")) - branch_tag = models.CharField(editable=True, max_length=150, - null=True, blank=True, help_text=_("Tag or branch of the product the engagement tested."), verbose_name=_("Branch/Tag")) - build_server = models.ForeignKey(Tool_Configuration, verbose_name=_("Build Server"), help_text=_("Build server responsible for CI/CD test"), null=True, blank=True, related_name="build_server", on_delete=models.CASCADE) - source_code_management_server = models.ForeignKey(Tool_Configuration, null=True, blank=True, verbose_name=_("SCM Server"), help_text=_("Source code server for CI/CD test"), related_name="source_code_management_server", on_delete=models.CASCADE) - source_code_management_uri = models.URLField(max_length=600, null=True, blank=True, editable=True, verbose_name=_("Repo"), help_text=_("Resource link to source code")) - orchestration_engine = models.ForeignKey(Tool_Configuration, verbose_name=_("Orchestration Engine"), help_text=_("Orchestration service responsible for CI/CD test"), null=True, blank=True, related_name="orchestration", on_delete=models.CASCADE) - deduplication_on_engagement = models.BooleanField(default=False, verbose_name=_("Deduplication within this engagement only"), help_text=_("If enabled deduplication will only mark a finding in this engagement as duplicate of another finding if both findings are in this engagement. If disabled, deduplication is on the product level.")) - - tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this engagement. Choose from the list or add new tags. Press Enter key to add.")) - inherited_tags = TagField(blank=True, force_lowercase=True, help_text=_("Internal use tags sepcifically for maintaining parity with product. This field will be present as a subset in the tags field")) - - class Meta: - ordering = ["-target_start"] - indexes = [ - models.Index(fields=["product", "active"]), - ] - - def __str__(self): - return "Engagement {}: {} ({})".format(self.id if id else 0, self.name or "", - self.target_start.strftime( - "%b %d, %Y")) - - def get_absolute_url(self): - return reverse("view_engagement", args=[str(self.id)]) - - def copy(self): - copy = copy_model_util(self) - # Save the necessary ManyToMany relationships - old_notes = list(self.notes.all()) - old_files = list(self.files.all()) - old_tags = list(self.tags.all()) - old_risk_acceptances = list(self.risk_acceptance.all()) - old_tests = list(Test.objects.filter(engagement=self)) - # Save the object before setting any ManyToMany relationships - copy.save() - # Copy the notes - for notes in old_notes: - copy.notes.add(notes.copy()) - # Copy the files - for files in old_files: - copy.files.add(files.copy()) - # Copy the tests - for test in old_tests: - test.copy(engagement=copy) - # Copy the risk_acceptances - for risk_acceptance in old_risk_acceptances: - copy.risk_acceptance.add(risk_acceptance.copy(engagement=copy)) - # Assign any tags - copy.tags.set(old_tags) - - return copy - - def is_overdue(self): - overdue_grace_days = 10 if self.engagement_type == "CI/CD" else 0 - - max_end_date = timezone.now() - relativedelta(days=overdue_grace_days) - - return self.target_end < max_end_date.date() - - def get_breadcrumbs(self): - bc = self.product.get_breadcrumbs() - bc += [{"title": str(self), - "url": reverse("view_engagement", args=(self.id,))}] - return bc - - # only used by bulk risk acceptance api - @property - def unaccepted_open_findings(self): - from dojo.utils import get_system_setting # noqa: PLC0415 circular import - - findings = Finding.objects.filter(risk_accepted=False, active=True, duplicate=False, test__engagement=self) - if get_system_setting("enforce_verified_status", True) or get_system_setting("enforce_verified_status_metrics", True): - findings = findings.filter(verified=True) - - return findings - - def accept_risks(self, accepted_risks): - self.risk_acceptance.add(*accepted_risks) - - @property - def has_jira_issue(self): - from dojo.jira import services as jira_services # noqa: PLC0415 circular import - return jira_services.has_issue(self) - - @property - def is_ci_cd(self): - return self.engagement_type == "CI/CD" - - def delete(self, *args, **kwargs): - logger.debug("%d engagement delete", self.id) - from dojo.finding import helper as finding_helper # noqa: PLC0415 circular import - finding_helper.prepare_duplicates_for_delete(self) - super().delete(*args, **kwargs) - with suppress(Engagement.DoesNotExist, Product.DoesNotExist): - # Suppressing a potential issue created from async delete removing - # related objects in a separate task - from dojo.utils import perform_product_grading # noqa: PLC0415 circular import - perform_product_grading(self.product) - - -class CWE(models.Model): - url = models.CharField(max_length=1000) - description = models.CharField(max_length=2000) - number = models.IntegerField() - - -class Endpoint_Params(models.Model): - param = models.CharField(max_length=150) - value = models.CharField(max_length=150) - method_type = (("GET", "GET"), - ("POST", "POST")) - method = models.CharField(max_length=20, blank=False, null=True, choices=method_type) - - -class Endpoint_Status(models.Model): - date = models.DateField(default=get_current_date) - last_modified = models.DateTimeField(null=True, editable=False, default=get_current_datetime) - mitigated = models.BooleanField(default=False, blank=True) - mitigated_time = models.DateTimeField(editable=False, null=True, blank=True) - mitigated_by = models.ForeignKey(Dojo_User, editable=True, null=True, on_delete=models.RESTRICT) - false_positive = models.BooleanField(default=False, blank=True) - out_of_scope = models.BooleanField(default=False, blank=True) - risk_accepted = models.BooleanField(default=False, blank=True) - endpoint = models.ForeignKey("Endpoint", null=False, blank=False, on_delete=models.CASCADE, related_name="status_endpoint") - finding = models.ForeignKey("Finding", null=False, blank=False, on_delete=models.CASCADE, related_name="status_finding") - - class Meta: - indexes = [ - models.Index(fields=["finding", "mitigated"]), - models.Index(fields=["endpoint", "mitigated"]), - # Optimize frequent lookups of "active" statuses (mitigated/flags all False) - models.Index( - name="idx_eps_active_by_endpoint", - fields=["endpoint"], - condition=Q(mitigated=False, false_positive=False, out_of_scope=False, risk_accepted=False), - ), - models.Index( - name="idx_eps_active_by_finding", - fields=["finding"], - condition=Q(mitigated=False, false_positive=False, out_of_scope=False, risk_accepted=False), - ), - ] - constraints = [ - models.UniqueConstraint(fields=["finding", "endpoint"], name="endpoint-finding relation"), - ] - - def __str__(self): - with Endpoint.allow_endpoint_init(): # TODO: Delete this after the move to Locations - return f"'{self.finding}' on '{self.endpoint}'" - - def copy(self, finding=None): - copy = copy_model_util(self) - current_endpoint = self.endpoint - if finding: - copy.finding = finding - copy.endpoint = current_endpoint - copy.save() - - return copy - - @property - def age(self): - - diff = self.mitigated_time.date() - self.date if self.mitigated else get_current_date() - self.date - days = diff.days - return max(0, days) - - -class Endpoint(models.Model): - protocol = models.CharField(null=True, blank=True, max_length=20, - help_text=_("The communication protocol/scheme such as 'http', 'ftp', 'dns', etc.")) - userinfo = models.CharField(null=True, blank=True, max_length=500, - help_text=_("User info as 'alice', 'bob', etc.")) - host = models.CharField(null=True, blank=True, max_length=500, - help_text=_("The host name or IP address. It must not include the port number. " - "For example '127.0.0.1', 'localhost', 'yourdomain.com'.")) - port = models.IntegerField(null=True, blank=True, - help_text=_("The network port associated with the endpoint.")) - path = models.CharField(null=True, blank=True, max_length=500, - help_text=_("The location of the resource, it must not start with a '/'. For example " - "endpoint/420/edit")) - query = models.CharField(null=True, blank=True, max_length=1000, - help_text=_("The query string, the question mark should be omitted." - "For example 'group=4&team=8'")) - fragment = models.CharField(null=True, blank=True, max_length=500, - help_text=_("The fragment identifier which follows the hash mark. The hash mark should " - "be omitted. For example 'section-13', 'paragraph-2'.")) - product = models.ForeignKey(Product, null=True, blank=True, on_delete=models.CASCADE) - endpoint_params = models.ManyToManyField(Endpoint_Params, blank=True, editable=False) - findings = models.ManyToManyField("Finding", - blank=True, - verbose_name=_("Findings"), - through=Endpoint_Status) - - tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this endpoint. Choose from the list or add new tags. Press Enter key to add.")) - inherited_tags = TagField(blank=True, force_lowercase=True, help_text=_("Internal use tags sepcifically for maintaining parity with product. This field will be present as a subset in the tags field")) - - class Meta: - ordering = ["product", "host", "protocol", "port", "userinfo", "path", "query", "fragment"] - indexes = [ - models.Index(fields=["product"]), - # Fast case-insensitive equality on host within product scope - models.Index( - F("product"), - Lower("host"), - name="idx_ep_product_lower_host", - ), - ] - - def __init__(self, *args, **kwargs): - if settings.V3_FEATURE_LOCATIONS and not getattr(self, "_allow_v3_init", False): - msg = "Endpoint model is deprecated when V3_FEATURE_LOCATIONS is enabled" - raise NotImplementedError(msg) - super().__init__(*args, **kwargs) - - def __hash__(self): - return self.__str__().__hash__() - - def __eq__(self, other): - if isinstance(other, Endpoint): - contents_match = str(self) == str(other) - # Use product_id (cached integer) instead of self.product to avoid - # triggering a FK lookup on every comparison inside NestedObjects.add_edge. - if self.product_id is not None and other.product_id is not None: - return self.product_id == other.product_id and contents_match - return contents_match - - return NotImplemented - - def __str__(self): - try: - if self.host: - dummy_scheme = "dummy-scheme" # workaround for https://github.com/python-hyper/hyperlink/blob/b8c9152cd826bbe8e6cc125648f3738235019705/src/hyperlink/_url.py#L988 - url = hyperlink.EncodedURL( - scheme=self.protocol or dummy_scheme, - userinfo=self.userinfo or "", - host=self.host, - port=self.port, - path=tuple(self.path.split("/")) if self.path else (), - query=tuple( - ( - qe.split("=", 1) - if "=" in qe - else (qe, None) - ) - for qe in self.query.split("&") - ) if self.query else (), # inspired by https://github.com/python-hyper/hyperlink/blob/b8c9152cd826bbe8e6cc125648f3738235019705/src/hyperlink/_url.py#L1427 - fragment=self.fragment or "", - ) - # Return a normalized version of the URL to avoid differences where there shouldn't be any difference. - # Example: https://google.com and https://google.com:443 - normalize_path = self.path # it used to add '/' at the end of host - clean_url = url.normalize(scheme=True, host=True, path=normalize_path, query=True, fragment=True, userinfo=True, percents=True).to_uri().to_text() - if not self.protocol: - if clean_url[:len(dummy_scheme) + 3] == (dummy_scheme + "://"): - clean_url = clean_url[len(dummy_scheme) + 3:] - else: - msg = "hyperlink lib did not create URL as was expected" - raise ValueError(msg) - return clean_url - msg = "Missing host" - raise ValueError(msg) - except: - url = "" - if self.protocol: - url += f"{self.protocol}://" - if self.userinfo: - url += f"{self.userinfo}@" - if self.host: - url += self.host - if self.port: - url += f":{self.port}" - if self.path: - url += "{}{}".format("/" if self.path[0] != "/" else "", self.path) - if self.query: - url += f"?{self.query}" - if self.fragment: - url += f"#{self.fragment}" - return url - - def get_absolute_url(self): - return reverse("view_endpoint", args=[str(self.id)]) - - @classmethod - @contextlib.contextmanager - def allow_endpoint_init(cls): - # When migrating to Locations, Endpoints are not deleted (hooray backup!). Disallowing the initialization of - # Endpoints is a good way to catch where they might still be used (oops!). However, there are some circumstances - # -- object deletes -- where Django itself attempts to instantiate an Endpoint object. This, we need to allow: - # if a user wants to delete an object, including whatever Endpoints are attached to it, they should be able to. - # This context manager allows code to initialize Endpoints at our discretion. - old = getattr(cls, "_allow_v3_init", None) - cls._allow_v3_init = True - try: - yield - finally: - cls._allow_v3_init = old - - def clean(self): - errors = [] - null_char_list = ["0x00", "\x00"] - db_type = connection.vendor - if self.protocol is not None: - if not re.match(r"^[A-Za-z][A-Za-z0-9\.\-\+]+$", self.protocol): # https://tools.ietf.org/html/rfc3986#section-3.1 - errors.append(ValidationError(f'Protocol "{self.protocol}" has invalid format')) - if not self.protocol: - self.protocol = None - - if self.userinfo is not None: - if not re.match(r"^[A-Za-z0-9\.\-_~%\!\$&\'\(\)\*\+,;=:]+$", self.userinfo): # https://tools.ietf.org/html/rfc3986#section-3.2.1 - errors.append(ValidationError(f'Userinfo "{self.userinfo}" has invalid format')) - if not self.userinfo: - self.userinfo = None - - if self.host: - if not re.match(r"^[A-Za-z0-9_\-\+][A-Za-z0-9_\.\-\+]+$", self.host): - try: - validate_ipv46_address(self.host) - except ValidationError: - errors.append(ValidationError(f'Host "{self.host}" has invalid format')) - else: - errors.append(ValidationError("Host must not be empty")) - - if self.port is not None: - try: - int_port = int(self.port) - if not (0 <= int_port < 65536): - errors.append(ValidationError(f'Port "{self.port}" has invalid format - out of range')) - self.port = int_port - except ValueError: - errors.append(ValidationError(f'Port "{self.port}" has invalid format - it is not a number')) - - if self.path is not None: - while len(self.path) > 0 and self.path[0] == "/": # Endpoint store "root-less" path - self.path = self.path[1:] - if any(null_char in self.path for null_char in null_char_list): - old_value = self.path - if "postgres" in db_type: - action_string = "Postgres does not accept NULL character. Attempting to replace with %00..." - for remove_str in null_char_list: - self.path = self.path.replace(remove_str, "%00") - logger.error('Path "%s" has invalid format - It contains the NULL character. The following action was taken: %s', old_value, action_string) - if not self.path: - self.path = None - - if self.query is not None: - if len(self.query) > 0 and self.query[0] == "?": - self.query = self.query[1:] - if any(null_char in self.query for null_char in null_char_list): - old_value = self.query - if "postgres" in db_type: - action_string = "Postgres does not accept NULL character. Attempting to replace with %00..." - for remove_str in null_char_list: - self.query = self.query.replace(remove_str, "%00") - logger.error('Query "%s" has invalid format - It contains the NULL character. The following action was taken: %s', old_value, action_string) - if not self.query: - self.query = None - - if self.fragment is not None: - if len(self.fragment) > 0 and self.fragment[0] == "#": - self.fragment = self.fragment[1:] - if any(null_char in self.fragment for null_char in null_char_list): - old_value = self.fragment - if "postgres" in db_type: - action_string = "Postgres does not accept NULL character. Attempting to replace with %00..." - for remove_str in null_char_list: - self.fragment = self.fragment.replace(remove_str, "%00") - logger.error('Fragment "%s" has invalid format - It contains the NULL character. The following action was taken: %s', old_value, action_string) - if not self.fragment: - self.fragment = None - - if errors: - raise ValidationError(errors) - - @property - def is_broken(self): - try: - self.clean() - except: - return True - else: - return not self.product - - @property - def mitigated(self): - return not self.vulnerable - - @property - def vulnerable(self): - return Endpoint_Status.objects.filter( - endpoint=self, - mitigated=False, - false_positive=False, - out_of_scope=False, - risk_accepted=False, - ).count() > 0 - - @property - def findings_count(self): - return self.findings.all().count() - - def active_findings(self): - return self.findings.filter( - active=True, - out_of_scope=False, - mitigated__isnull=True, - false_p=False, - duplicate=False, - status_finding__false_positive=False, - status_finding__out_of_scope=False, - status_finding__risk_accepted=False, - ).order_by("numerical_severity") - - def active_verified_findings(self): - return self.findings.filter( - active=True, - verified=True, - out_of_scope=False, - mitigated__isnull=True, - false_p=False, - duplicate=False, - status_finding__false_positive=False, - status_finding__out_of_scope=False, - status_finding__risk_accepted=False, - ).order_by("numerical_severity") - - @property - def active_findings_count(self): - return self.active_findings().count() - - @property - def active_verified_findings_count(self): - return self.active_verified_findings().count() - - def host_endpoints(self): - return Endpoint.objects.filter(host=self.host, - product=self.product).distinct() - - @property - def host_endpoints_count(self): - return self.host_endpoints().count() - - def host_mitigated_endpoints(self): - meps = Endpoint_Status.objects \ - .filter(endpoint__in=self.host_endpoints()) \ - .filter(Q(mitigated=True) - | Q(false_positive=True) - | Q(out_of_scope=True) - | Q(risk_accepted=True) - | Q(finding__out_of_scope=True) - | Q(finding__mitigated__isnull=False) - | Q(finding__false_p=True) - | Q(finding__duplicate=True) - | Q(finding__active=False)) - return Endpoint.objects.filter(status_endpoint__in=meps).distinct() - - @property - def host_mitigated_endpoints_count(self): - return self.host_mitigated_endpoints().count() - - def host_findings(self): - return Finding.objects.filter(endpoints__in=self.host_endpoints()).distinct() - - @property - def host_findings_count(self): - return self.host_findings().count() - - def host_active_findings(self): - return Finding.objects.filter( - active=True, - out_of_scope=False, - mitigated__isnull=True, - false_p=False, - duplicate=False, - status_finding__false_positive=False, - status_finding__out_of_scope=False, - status_finding__risk_accepted=False, - endpoints__in=self.host_endpoints(), - ).order_by("numerical_severity") - - def host_active_verified_findings(self): - return Finding.objects.filter( - active=True, - verified=True, - out_of_scope=False, - mitigated__isnull=True, - false_p=False, - duplicate=False, - status_finding__false_positive=False, - status_finding__out_of_scope=False, - status_finding__risk_accepted=False, - endpoints__in=self.host_endpoints(), - ).order_by("numerical_severity") - - @property - def host_active_findings_count(self): - return self.host_active_findings().count() - - @property - def host_active_verified_findings_count(self): - return self.host_active_verified_findings().count() - - def get_breadcrumbs(self): - bc = self.product.get_breadcrumbs() - bc += [{"title": self.host, - "url": reverse("view_endpoint", args=(self.id,))}] - return bc - - @staticmethod - def from_uri(uri): - try: - url = hyperlink.parse(url=uri) - except UnicodeDecodeError: - url = hyperlink.parse(url="//" + urlparse(uri).netloc) - except hyperlink.URLParseError as e: - msg = f"Invalid URL format: {e}" - raise ValidationError(msg) - - query_parts = [] # inspired by https://github.com/python-hyper/hyperlink/blob/b8c9152cd826bbe8e6cc125648f3738235019705/src/hyperlink/_url.py#L1768 - for k, v in url.query: - if v is None: - query_parts.append(k) - else: - query_parts.append(f"{k}={v}") - query_string = "&".join(query_parts) - - protocol = url.scheme or None - userinfo = ":".join(url.userinfo) if url.userinfo not in {(), ("",)} else None - host = url.host or None - port = url.port - path = "/".join(url.path)[:500] if url.path not in {None, (), ("",)} else None - query = query_string[:1000] if query_string is not None and query_string else None - fragment = url.fragment[:500] if url.fragment is not None and url.fragment else None - - return Endpoint( - protocol=protocol, - userinfo=userinfo, - host=host, - port=port, - path=path, - query=query, - fragment=fragment, - ) - - -class Development_Environment(models.Model): - name = models.CharField(max_length=200) - - def __str__(self): - return self.name - - def get_breadcrumbs(self): - return [{"title": str(self), - "url": reverse("edit_dev_env", args=(self.id,))}] - - -class Sonarqube_Issue(models.Model): - key = models.CharField(max_length=60, unique=True, help_text=_("SonarQube issue key")) - status = models.CharField(max_length=20, help_text=_("SonarQube issue status")) - type = models.CharField(max_length=20, help_text=_("SonarQube issue type")) - - def __str__(self): - return self.key - - -class Sonarqube_Issue_Transition(models.Model): - sonarqube_issue = models.ForeignKey(Sonarqube_Issue, on_delete=models.CASCADE, db_index=True) - created = models.DateTimeField(auto_now_add=True, null=False) - finding_status = models.CharField(max_length=100) - sonarqube_status = models.CharField(max_length=50) - transitions = models.CharField(max_length=100) - - class Meta: - ordering = ("-created", ) - - -class Test(models.Model): - engagement = models.ForeignKey(Engagement, editable=False, on_delete=models.CASCADE) - lead = models.ForeignKey(Dojo_User, editable=True, null=True, blank=True, on_delete=models.RESTRICT) - test_type = models.ForeignKey(Test_Type, on_delete=models.CASCADE) - scan_type = models.TextField(null=True) - title = models.CharField(max_length=255, null=True, blank=True) - description = models.TextField(null=True, blank=True) - target_start = models.DateTimeField() - target_end = models.DateTimeField() - percent_complete = models.IntegerField(null=True, blank=True, - editable=True) - notes = models.ManyToManyField(Notes, blank=True, - editable=False) - files = models.ManyToManyField(FileUpload, blank=True, editable=False) - environment = models.ForeignKey(Development_Environment, null=True, - blank=False, on_delete=models.RESTRICT) - - updated = models.DateTimeField(auto_now=True, null=True) - created = models.DateTimeField(auto_now_add=True, null=True) - - tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this test. Choose from the list or add new tags. Press Enter key to add.")) - inherited_tags = TagField(blank=True, force_lowercase=True, help_text=_("Internal use tags sepcifically for maintaining parity with product. This field will be present as a subset in the tags field")) - - version = models.CharField(max_length=100, null=True, blank=True) - - build_id = models.CharField(editable=True, max_length=150, - null=True, blank=True, help_text=_("Build ID that was tested, a reimport may update this field."), verbose_name=_("Build ID")) - commit_hash = models.CharField(editable=True, max_length=150, - null=True, blank=True, help_text=_("Commit hash tested, a reimport may update this field."), verbose_name=_("Commit Hash")) - branch_tag = models.CharField(editable=True, max_length=150, - null=True, blank=True, help_text=_("Tag or branch that was tested, a reimport may update this field."), verbose_name=_("Branch/Tag")) - api_scan_configuration = models.ForeignKey(Product_API_Scan_Configuration, null=True, editable=True, blank=True, on_delete=models.CASCADE, verbose_name=_("API Scan Configuration")) - - class Meta: - indexes = [ - models.Index(fields=["engagement", "test_type"]), - ] - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.unsaved_metadata: list = [] - - def __str__(self): - if self.title: - return f"{self.title} ({self.test_type})" - return str(self.test_type) - - def get_absolute_url(self): - return reverse("view_test", args=[str(self.id)]) - - def test_type_name(self) -> str: - return self.test_type.name - - def get_breadcrumbs(self): - bc = self.engagement.get_breadcrumbs() - bc += [{"title": str(self), - "url": reverse("view_test", args=(self.id,))}] - return bc - - def copy(self, engagement=None): - copy = copy_model_util(self) - # Save the necessary ManyToMany relationships - old_notes = list(self.notes.all()) - old_files = list(self.files.all()) - old_tags = list(self.tags.all()) - old_findings = list(Finding.objects.filter(test=self)) - if engagement: - copy.engagement = engagement - # Save the object before setting any ManyToMany relationships - copy.save() - # Copy the notes - for notes in old_notes: - copy.notes.add(notes.copy()) - # Copy the files - for files in old_files: - copy.files.add(files.copy()) - # Copy the Findings - for finding in old_findings: - finding.copy(test=copy) - # Assign any tags - copy.tags.set(old_tags) - - return copy - - # only used by bulk risk acceptance api - @property - def unaccepted_open_findings(self): - from dojo.utils import get_system_setting # noqa: PLC0415 circular import - findings = Finding.objects.filter(risk_accepted=False, active=True, duplicate=False, test=self) - if get_system_setting("enforce_verified_status", True) or get_system_setting("enforce_verified_status_metrics", True): - findings = findings.filter(verified=True) - - return findings - - def accept_risks(self, accepted_risks): - self.engagement.risk_acceptance.add(*accepted_risks) - - @property - def deduplication_algorithm(self): - deduplicationAlgorithm = settings.DEDUPE_ALGO_LEGACY - - if hasattr(settings, "DEDUPLICATION_ALGORITHM_PER_PARSER"): - if (self.test_type.name in settings.DEDUPLICATION_ALGORITHM_PER_PARSER): - deduplicationLogger.debug(f"using DEDUPLICATION_ALGORITHM_PER_PARSER for test_type.name: {self.test_type.name}") - deduplicationAlgorithm = settings.DEDUPLICATION_ALGORITHM_PER_PARSER[self.test_type.name] - elif (self.scan_type in settings.DEDUPLICATION_ALGORITHM_PER_PARSER): - deduplicationLogger.debug(f"using DEDUPLICATION_ALGORITHM_PER_PARSER for scan_type: {self.scan_type}") - deduplicationAlgorithm = settings.DEDUPLICATION_ALGORITHM_PER_PARSER[self.scan_type] - else: - deduplicationLogger.debug("Section DEDUPLICATION_ALGORITHM_PER_PARSER not found in settings.dist.py") - - deduplicationLogger.debug(f"DEDUPLICATION_ALGORITHM_PER_PARSER is: {deduplicationAlgorithm}") - return deduplicationAlgorithm - - @property - def hash_code_fields(self): - """Retrieve OS HASH_CODE_FIELDS_PER_SCANNER settings. Be aware when calling this to make sure Pro doesn't use these OS seetings""" - hashCodeFields = None - - if hasattr(settings, "HASHCODE_FIELDS_PER_SCANNER"): - if (self.test_type.name in settings.HASHCODE_FIELDS_PER_SCANNER): - deduplicationLogger.debug(f"using HASHCODE_FIELDS_PER_SCANNER for test_type.name: {self.test_type.name}") - hashCodeFields = settings.HASHCODE_FIELDS_PER_SCANNER[self.test_type.name] - elif (self.scan_type in settings.HASHCODE_FIELDS_PER_SCANNER): - deduplicationLogger.debug(f"using HASHCODE_FIELDS_PER_SCANNER for scan_type: {self.scan_type}") - hashCodeFields = settings.HASHCODE_FIELDS_PER_SCANNER[self.scan_type] - else: - deduplicationLogger.warning(f"test_type name {self.test_type.name} and scan_type {self.scan_type} not found in HASHCODE_FIELDS_PER_SCANNER") - else: - deduplicationLogger.debug("Section HASHCODE_FIELDS_PER_SCANNER not found in settings.dist.py") - - hash_code_fields_always = getattr(settings, "HASH_CODE_FIELDS_ALWAYS", []) - deduplicationLogger.debug(f"HASHCODE_FIELDS_PER_SCANNER is: {hashCodeFields} + HASH_CODE_FIELDS_ALWAYS: {hash_code_fields_always}") - - return hashCodeFields - - @property - def hash_code_allows_null_cwe(self): - hashCodeAllowsNullCwe = True - - if hasattr(settings, "HASHCODE_ALLOWS_NULL_CWE"): - if (self.test_type.name in settings.HASHCODE_ALLOWS_NULL_CWE): - deduplicationLogger.debug(f"using HASHCODE_ALLOWS_NULL_CWE for test_type.name: {self.test_type.name}") - hashCodeAllowsNullCwe = settings.HASHCODE_ALLOWS_NULL_CWE[self.test_type.name] - elif (self.scan_type in settings.HASHCODE_ALLOWS_NULL_CWE): - deduplicationLogger.debug(f"using HASHCODE_ALLOWS_NULL_CWE for scan_type: {self.scan_type}") - hashCodeAllowsNullCwe = settings.HASHCODE_ALLOWS_NULL_CWE[self.scan_type] - else: - deduplicationLogger.debug("Section HASHCODE_ALLOWS_NULL_CWE not found in settings.dist.py") - - deduplicationLogger.debug(f"HASHCODE_ALLOWS_NULL_CWE is: {hashCodeAllowsNullCwe}") - return hashCodeAllowsNullCwe - - def delete(self, *args, product_grading_option=True, **kwargs): - logger.debug("%d test delete", self.id) - super().delete(*args, **kwargs) - if product_grading_option: - with suppress(Test.DoesNotExist, Engagement.DoesNotExist, Product.DoesNotExist): - # Suppressing a potential issue created from async delete removing - # related objects in a separate task - from dojo.utils import perform_product_grading # noqa: PLC0415 circular import - perform_product_grading(self.engagement.product) - - @property - def statistics(self): - """Queries the database, no prefetching, so could be slow for lists of model instances""" - return _get_statistics_for_queryset(Finding.objects.filter(test=self), _get_annotations_for_statistics) - - -class Test_Import(TimeStampedModel): - - IMPORT_TYPE = "import" - REIMPORT_TYPE = "reimport" - - test = models.ForeignKey(Test, editable=False, null=False, blank=False, on_delete=models.CASCADE) - findings_affected = models.ManyToManyField("Finding", through="Test_Import_Finding_Action") - import_settings = JSONField(null=True) - type = models.CharField(max_length=64, null=False, blank=False, default="unknown") - - version = models.CharField(max_length=100, null=True, blank=True) - build_id = models.CharField(editable=True, max_length=150, - null=True, blank=True, help_text=_("Build ID that was tested, a reimport may update this field."), verbose_name=_("Build ID")) - commit_hash = models.CharField(editable=True, max_length=150, - null=True, blank=True, help_text=_("Commit hash tested, a reimport may update this field."), verbose_name=_("Commit Hash")) - branch_tag = models.CharField(editable=True, max_length=150, - null=True, blank=True, help_text=_("Tag or branch that was tested, a reimport may update this field."), verbose_name=_("Branch/Tag")) - - def get_queryset(self): - logger.debug("prefetch test_import counts") - super_query = super().get_queryset() - super_query = super_query.annotate(created_findings_count=Count("findings", filter=Q(test_import_finding_action__action=IMPORT_CREATED_FINDING))) - super_query = super_query.annotate(closed_findings_count=Count("findings", filter=Q(test_import_finding_action__action=IMPORT_CLOSED_FINDING))) - super_query = super_query.annotate(reactivated_findings_count=Count("findings", filter=Q(test_import_finding_action__action=IMPORT_REACTIVATED_FINDING))) - return super_query.annotate(untouched_findings_count=Count("findings", filter=Q(test_import_finding_action__action=IMPORT_UNTOUCHED_FINDING))) - - class Meta: - ordering = ("-id",) - indexes = [ - models.Index(fields=["created", "test", "type"]), - ] - - def __str__(self): - return self.created.strftime("%Y-%m-%d %H:%M:%S") - - @property - def statistics(self): - """Queries the database, no prefetching, so could be slow for lists of model instances""" - stats = {} - for action in IMPORT_ACTIONS: - stats[action[1].lower()] = _get_statistics_for_queryset(Finding.objects.filter(test_import_finding_action__test_import=self, test_import_finding_action__action=action[0]), _get_annotations_for_statistics) - return stats - - -class Test_Import_Finding_Action(TimeStampedModel): - test_import = models.ForeignKey(Test_Import, editable=False, null=False, blank=False, on_delete=models.CASCADE) - finding = models.ForeignKey("Finding", editable=False, null=False, blank=False, on_delete=models.CASCADE) - action = models.CharField(max_length=100, null=True, blank=True, choices=IMPORT_ACTIONS) - - class Meta: - indexes = [ - models.Index(fields=["finding", "action", "test_import"]), - ] - unique_together = (("test_import", "finding")) - ordering = ("test_import", "action", "finding") - - def __str__(self): - return f"{self.finding.id}: {self.action}" - - -class Finding(BaseModel): - # Fields loaded when performing deduplication (used by get_finding_models_for_deduplication - # and build_candidate_scope_queryset to restrict the SELECT to only what is needed). - # Covers the union of all deduplication algorithms so that a single queryset works - # regardless of which algorithm is in use. Large text fields (description, mitigation, - # impact, references, …) are intentionally excluded. - DEDUPLICATION_FIELDS = [ - "id", - # FK required for select_related("test") — must not be deferred - "test", - # Fields written by set_duplicate - "duplicate", - "active", - "verified", - "duplicate_finding", - # Guard checks in set_duplicate - "is_mitigated", - "mitigated", - "out_of_scope", - "false_p", - # Accessed by status() (debug logging only) - "under_review", - "risk_accepted", - # Used by hash-code and legacy algorithms for endpoint/location matching - "dynamic_finding", - "static_finding", - # Algorithm-specific matching fields - "hash_code", # hash_code, uid_or_hash, legacy - "unique_id_from_tool", # unique_id, uid_or_hash - "title", # legacy - "cwe", # legacy - "file_path", # legacy - "line", # legacy - ] - - # Large text fields deferred in build_candidate_scope_queryset. These are - # never accessed during deduplication or reimport candidate matching, so - # excluding them reduces the data loaded for every candidate finding. - DEDUPLICATION_DEFERRED_FIELDS = [ - "description", - "mitigation", - "impact", - "steps_to_reproduce", - "severity_justification", - "references", - "url", - "cvssv3", - "cvssv4", - ] - - title = models.CharField(max_length=511, - verbose_name=_("Title"), - help_text=_("A short description of the flaw.")) - date = models.DateField(default=get_current_date, - verbose_name=_("Date"), - help_text=_("The date the flaw was discovered.")) - sla_start_date = models.DateField( - blank=True, - null=True, - verbose_name=_("SLA Start Date"), - help_text=_("(readonly)The date used as start date for SLA calculation. Set by expiring risk acceptances. Empty by default, causing a fallback to 'date'.")) - sla_expiration_date = models.DateField( - blank=True, - null=True, - verbose_name=_("SLA Expiration Date"), - help_text=_("(readonly)The date SLA expires for this finding. Empty by default, causing a fallback to 'date'.")) - cwe = models.IntegerField(default=0, null=True, blank=True, - verbose_name=_("CWE"), - help_text=_("The CWE number associated with this flaw.")) - cve = models.CharField(max_length=50, - null=True, - blank=False, - verbose_name=_("Vulnerability Id"), - help_text=_("An id of a vulnerability in a security advisory associated with this finding. Can be a Common Vulnerabilities and Exposures (CVE) or from other sources.")) - epss_score = models.FloatField(default=None, null=True, blank=True, - verbose_name=_("EPSS Score"), - help_text=_("EPSS score for the CVE. Describes how likely it is the vulnerability will be exploited in the next 30 days."), - validators=[MinValueValidator(0.0), MaxValueValidator(1.0)]) - epss_percentile = models.FloatField(default=None, null=True, blank=True, - verbose_name=_("EPSS percentile"), - help_text=_("EPSS percentile for the CVE. Describes how many CVEs are scored at or below this one."), - validators=[MinValueValidator(0.0), MaxValueValidator(1.0)]) - known_exploited = models.BooleanField(default=False, - verbose_name=_("Known Exploited"), - help_text=_("Whether this vulnerability is known to have been exploited in the wild.")) - ransomware_used = models.BooleanField(default=False, - verbose_name=_("Used in Ransomware"), - help_text=_("Whether this vulnerability is known to have been leveraged as part of a ransomware campaign.")) - kev_date = models.DateField(null=True, blank=True, - verbose_name=_("KEV Date Added"), - help_text=_("The date the vulnerability was added to the KEV catalog."), - validators=[MaxValueValidator(tomorrow)]) - cvssv3 = models.TextField(validators=[cvss3_validator], - max_length=117, - null=True, - verbose_name=_("CVSS3 Vector"), - help_text=_("Common Vulnerability Scoring System version 3 (CVSS3) score associated with this finding.")) - cvssv3_score = models.FloatField(null=True, - blank=True, - verbose_name=_("CVSS3 Score"), - help_text=_("Numerical CVSSv3 score for the vulnerability. If the vector is given, the score is updated while saving the finding. The value must be between 0-10."), - validators=[MinValueValidator(0.0), MaxValueValidator(10.0)]) - - cvssv4 = models.TextField(validators=[cvss4_validator], - max_length=255, - null=True, - verbose_name=_("CVSS4 vector"), - help_text=_("Common Vulnerability Scoring System version 4 (CVSS4) score associated with this finding.")) - cvssv4_score = models.FloatField(null=True, - blank=True, - verbose_name=_("CVSSv4 Score"), - help_text=_("Numerical CVSSv4 score for the vulnerability. If the vector is given, the score is updated while saving the finding. The value must be between 0-10."), - validators=[MinValueValidator(0.0), MaxValueValidator(10.0)]) - - url = models.TextField(null=True, - blank=True, - editable=False, - verbose_name=_("URL"), - help_text=_("External reference that provides more information about this flaw.")) # not displayed and pretty much the same as references. To remove? - severity = models.CharField(max_length=200, - verbose_name=_("Severity"), - help_text=_("The severity level of this flaw (Critical, High, Medium, Low, Info).")) - description = models.TextField(verbose_name=_("Description"), - help_text=_("Longer more descriptive information about the flaw.")) - mitigation = models.TextField(verbose_name=_("Mitigation"), - null=True, - blank=True, - help_text=_("Text describing how to best fix the flaw.")) - fix_available = models.BooleanField(null=True, - default=None, - verbose_name=_("Fix Available"), - help_text=_("Denotes if there is a fix available for this flaw.")) - fix_version = models.CharField(null=True, - blank=True, - max_length=100, - verbose_name=_("Fix version"), - help_text=_("Version of the affected component in which the flaw is fixed.")) - impact = models.TextField(verbose_name=_("Impact"), - null=True, - blank=True, - help_text=_("Text describing the impact this flaw has on systems, products, enterprise, etc.")) - steps_to_reproduce = models.TextField(null=True, - blank=True, - verbose_name=_("Steps to Reproduce"), - help_text=_("Text describing the steps that must be followed in order to reproduce the flaw / bug.")) - severity_justification = models.TextField(null=True, - blank=True, - verbose_name=_("Severity Justification"), - help_text=_("Text describing why a certain severity was associated with this flaw.")) - # TODO: Delete this after the move to Locations - endpoints = models.ManyToManyField(Endpoint, - blank=True, - verbose_name=_("Endpoints"), - help_text=_("The hosts within the product that are susceptible to this flaw. + The status of the endpoint associated with this flaw (Vulnerable, Mitigated, ...)."), - through=Endpoint_Status) - references = models.TextField(null=True, - blank=True, - db_column="refs", - verbose_name=_("References"), - help_text=_("The external documentation available for this flaw.")) - test = models.ForeignKey(Test, - editable=False, - on_delete=models.CASCADE, - verbose_name=_("Test"), - help_text=_("The test that is associated with this flaw.")) - active = models.BooleanField(default=True, - verbose_name=_("Active"), - help_text=_("Denotes if this flaw is active or not.")) - # note that false positive findings cannot be verified - # in defectdojo verified means: "we have verified the finding and it turns out that it's not a false positive" - verified = models.BooleanField(default=False, - verbose_name=_("Verified"), - help_text=_("Denotes if this flaw has been manually verified by the tester.")) - false_p = models.BooleanField(default=False, - verbose_name=_("False Positive"), - help_text=_("Denotes if this flaw has been deemed a false positive by the tester.")) - duplicate = models.BooleanField(default=False, - verbose_name=_("Duplicate"), - help_text=_("Denotes if this flaw is a duplicate of other flaws reported.")) - duplicate_finding = models.ForeignKey("self", - editable=False, - null=True, - related_name="original_finding", - blank=True, on_delete=models.DO_NOTHING, - verbose_name=_("Duplicate Finding"), - help_text=_("Link to the original finding if this finding is a duplicate.")) - out_of_scope = models.BooleanField(default=False, - verbose_name=_("Out Of Scope"), - help_text=_("Denotes if this flaw falls outside the scope of the test and/or engagement.")) - risk_accepted = models.BooleanField(default=False, - verbose_name=_("Risk Accepted"), - help_text=_("Denotes if this finding has been marked as an accepted risk.")) - under_review = models.BooleanField(default=False, - verbose_name=_("Under Review"), - help_text=_("Denotes is this flaw is currently being reviewed.")) - - last_status_update = models.DateTimeField(editable=False, - null=True, - blank=True, - auto_now_add=True, - verbose_name=_("Last Status Update"), - help_text=_("Timestamp of latest status update (change in status related fields).")) - - review_requested_by = models.ForeignKey(Dojo_User, - null=True, - blank=True, - related_name="review_requested_by", - on_delete=models.RESTRICT, - verbose_name=_("Review Requested By"), - help_text=_("Documents who requested a review for this finding.")) - reviewers = models.ManyToManyField(Dojo_User, - blank=True, - verbose_name=_("Reviewers"), - help_text=_("Documents who reviewed the flaw.")) - - # Defect Tracking Review - under_defect_review = models.BooleanField(default=False, - verbose_name=_("Under Defect Review"), - help_text=_("Denotes if this finding is under defect review.")) - defect_review_requested_by = models.ForeignKey(Dojo_User, - null=True, - blank=True, - related_name="defect_review_requested_by", - on_delete=models.RESTRICT, - verbose_name=_("Defect Review Requested By"), - help_text=_("Documents who requested a defect review for this flaw.")) - is_mitigated = models.BooleanField(default=False, - verbose_name=_("Is Mitigated"), - help_text=_("Denotes if this flaw has been fixed.")) - thread_id = models.IntegerField(default=0, - editable=False, - verbose_name=_("Thread ID")) - mitigated = models.DateTimeField(editable=False, - null=True, - blank=True, - verbose_name=_("Mitigated"), - help_text=_("Denotes if this flaw has been fixed by storing the date it was fixed.")) - mitigated_by = models.ForeignKey(Dojo_User, - null=True, - editable=False, - related_name="mitigated_by", - on_delete=models.RESTRICT, - verbose_name=_("Mitigated By"), - help_text=_("Documents who has marked this flaw as fixed.")) - reporter = models.ForeignKey(Dojo_User, - editable=False, - default=1, - related_name="reporter", - on_delete=models.RESTRICT, - verbose_name=_("Reporter"), - help_text=_("Documents who reported the flaw.")) - notes = models.ManyToManyField(Notes, - blank=True, - editable=False, - verbose_name=_("Notes"), - help_text=_("Stores information pertinent to the flaw or the mitigation.")) - numerical_severity = models.CharField(max_length=4, - verbose_name=_("Numerical Severity"), - help_text=_("The numerical representation of the severity (S0, S1, S2, S3, S4).")) - last_reviewed = models.DateTimeField(null=True, - editable=False, - verbose_name=_("Last Reviewed"), - help_text=_("Provides the date the flaw was last 'touched' by a tester.")) - last_reviewed_by = models.ForeignKey(Dojo_User, - null=True, - editable=False, - related_name="last_reviewed_by", - on_delete=models.RESTRICT, - verbose_name=_("Last Reviewed By"), - help_text=_("Provides the person who last reviewed the flaw.")) - files = models.ManyToManyField(FileUpload, - blank=True, - editable=False, - verbose_name=_("Files"), - help_text=_("Files(s) related to the flaw.")) - param = models.TextField(null=True, - blank=True, - editable=False, - verbose_name=_("Parameter"), - help_text=_("Parameter used to trigger the issue (DAST).")) - payload = models.TextField(null=True, - blank=True, - editable=False, - verbose_name=_("Payload"), - help_text=_("Payload used to attack the service / application and trigger the bug / problem.")) - hash_code = models.CharField(null=True, - blank=True, - editable=False, - max_length=64, - verbose_name=_("Hash Code"), - help_text=_("A hash over a configurable set of fields that is used for findings deduplication.")) - line = models.IntegerField(null=True, - blank=True, - verbose_name=_("Line number"), - help_text=_("Source line number of the attack vector.")) - file_path = models.CharField(null=True, - blank=True, - max_length=4000, - verbose_name=_("File path"), - help_text=_("Identified file(s) containing the flaw.")) - component_name = models.CharField(null=True, - blank=True, - max_length=500, - verbose_name=_("Component name"), - help_text=_("Name of the affected component (library name, part of a system, ...).")) - component_version = models.CharField(null=True, - blank=True, - max_length=100, - verbose_name=_("Component version"), - help_text=_("Version of the affected component.")) - found_by = models.ManyToManyField(Test_Type, - editable=False, - verbose_name=_("Found by"), - help_text=_("The name of the scanner that identified the flaw.")) - static_finding = models.BooleanField(default=False, - verbose_name=_("Static finding (SAST)"), - help_text=_("Flaw has been detected from a Static Application Security Testing tool (SAST).")) - dynamic_finding = models.BooleanField(default=True, - verbose_name=_("Dynamic finding (DAST)"), - help_text=_("Flaw has been detected from a Dynamic Application Security Testing tool (DAST).")) - scanner_confidence = models.IntegerField(null=True, - blank=True, - default=None, - editable=False, - verbose_name=_("Scanner confidence"), - help_text=_("Confidence level of vulnerability which is supplied by the scanner.")) - sonarqube_issue = models.ForeignKey(Sonarqube_Issue, - null=True, - blank=True, - help_text=_("The SonarQube issue associated with this finding."), - verbose_name=_("SonarQube issue"), - on_delete=models.CASCADE) - unique_id_from_tool = models.CharField(null=True, - blank=True, - max_length=500, - verbose_name=_("Unique ID from tool"), - help_text=_("Vulnerability technical id from the source tool. Allows to track unique vulnerabilities over time across subsequent scans.")) - vuln_id_from_tool = models.CharField(null=True, - blank=True, - max_length=500, - verbose_name=_("Vulnerability ID from tool"), - help_text=_("Non-unique technical id from the source tool associated with the vulnerability type.")) - sast_source_object = models.CharField(null=True, - blank=True, - max_length=500, - verbose_name=_("SAST Source Object"), - help_text=_("Source object (variable, function...) of the attack vector.")) - sast_sink_object = models.CharField(null=True, - blank=True, - max_length=500, - verbose_name=_("SAST Sink Object"), - help_text=_("Sink object (variable, function...) of the attack vector.")) - sast_source_line = models.IntegerField(null=True, - blank=True, - verbose_name=_("SAST Source Line number"), - help_text=_("Source line number of the attack vector.")) - sast_source_file_path = models.CharField(null=True, - blank=True, - max_length=4000, - verbose_name=_("SAST Source File Path"), - help_text=_("Source file path of the attack vector.")) - nb_occurences = models.IntegerField(null=True, - blank=True, - verbose_name=_("Number of occurences"), - help_text=_("Number of occurences in the source tool when several vulnerabilites were found and aggregated by the scanner.")) - - # this is useful for vulnerabilities on dependencies : helps answer the question "Did I add this vulnerability or was it discovered recently?" - publish_date = models.DateField(null=True, - blank=True, - verbose_name=_("Publish date"), - help_text=_("Date when this vulnerability was made publicly available.")) - - # The service is used to generate the hash_code, so that it gets part of the deduplication of findings. - service = models.CharField(null=True, - blank=True, - max_length=200, - verbose_name=_("Service"), - help_text=_("A service is a self-contained piece of functionality within a Product. This is an optional field which is used in deduplication of findings when set.")) - - planned_remediation_date = models.DateField(null=True, - editable=True, - verbose_name=_("Planned Remediation Date"), - help_text=_("The date the flaw is expected to be remediated.")) - - planned_remediation_version = models.CharField(null=True, - blank=True, - max_length=99, - verbose_name=_("Planned remediation version"), - help_text=_("The target version when the vulnerability should be fixed / remediated")) - - effort_for_fixing = models.CharField(null=True, - blank=True, - max_length=99, - verbose_name=_("Effort for fixing"), - help_text=_("Effort for fixing / remediating the vulnerability (Low, Medium, High)")) - - tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this finding. Choose from the list or add new tags. Press Enter key to add.")) - inherited_tags = TagField(blank=True, force_lowercase=True, help_text=_("Internal use tags sepcifically for maintaining parity with product. This field will be present as a subset in the tags field")) - - SEVERITIES = {"Info": 4, "Low": 3, "Medium": 2, - "High": 1, "Critical": 0} - - class Meta: - ordering = ("numerical_severity", "-date", "title", "epss_score", "epss_percentile") - indexes = [ - models.Index(fields=["test", "active", "verified"]), - - models.Index(fields=["test", "is_mitigated"]), - models.Index(fields=["test", "duplicate"]), - models.Index(fields=["test", "out_of_scope"]), - models.Index(fields=["test", "false_p"]), - - models.Index(fields=["test", "unique_id_from_tool", "duplicate"]), - models.Index(fields=["test", "hash_code", "duplicate"]), - - models.Index(fields=["test", "component_name"]), - - models.Index(fields=["cve"]), - models.Index(fields=["epss_score"]), - models.Index(fields=["epss_percentile"]), - models.Index(fields=["cwe"]), - models.Index(fields=["out_of_scope"]), - models.Index(fields=["false_p"]), - models.Index(fields=["verified"]), - models.Index(fields=["mitigated"]), - models.Index(fields=["active"]), - models.Index(fields=["numerical_severity"]), - models.Index(fields=["date"]), - models.Index(fields=["title"]), - models.Index(fields=["hash_code"]), - models.Index(fields=["unique_id_from_tool"]), - # models.Index(fields=['file_path']), # can't add index because the field has max length 4000. - models.Index(fields=["line"]), - models.Index(fields=["component_name"]), - models.Index(fields=["duplicate"]), - models.Index(fields=["is_mitigated"]), - models.Index(fields=["duplicate_finding", "id"]), - models.Index(fields=["known_exploited"]), - models.Index(fields=["ransomware_used"]), - models.Index(fields=["kev_date"]), - ] - - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - if settings.V3_FEATURE_LOCATIONS: - self.unsaved_locations: list[UnsavedLocation] = [] - else: - # TODO: Delete this after the move to Locations - self.unsaved_endpoints = [] - self.unsaved_request = None - self.unsaved_response = None - self.unsaved_tags = None - self.unsaved_files = None - self.unsaved_vulnerability_ids = None - - def __str__(self): - return self.title - - def save(self, dedupe_option=True, rules_option=True, product_grading_option=True, # noqa: FBT002 - issue_updater_option=True, push_to_jira=False, user=None, *args, **kwargs): # noqa: FBT002 - this is bit hard to fix nice have this universally fixed - logger.debug("Start saving finding of id " + str(self.id) + " dedupe_option:" + str(dedupe_option) + " (self.pk is %s)", "None" if self.pk is None else "not None") - from dojo.finding import helper as finding_helper # noqa: PLC0415 circular import - - is_new_finding = self.pk is None - - # if not isinstance(self.date, (datetime, date)): - # raise ValidationError(_("The 'date' field must be a valid date or datetime object.")) - - if not user: - from dojo.utils import get_current_user # noqa: PLC0415 circular import - user = get_current_user() - # Title Casing - self.title = titlecase(self.title[:511]) - # Set the date of the finding if nothing is supplied - if self.date is None: - self.date = timezone.now() - # Assign the numerical severity for correct sorting order - self.numerical_severity = Finding.get_numerical_severity(self.severity) - - # Synchronize cvssv3 score using cvssv3 vector - - if self.cvssv3: - try: - cvss_data = parse_cvss_data(self.cvssv3) - if cvss_data: - self.cvssv3 = cvss_data.get("cvssv3") - self.cvssv3_score = cvss_data.get("cvssv3_score") - - except Exception as ex: - logger.warning("Can't compute cvssv3 score for finding id %i. Invalid cvssv3 vector found: '%s'. Exception: %s.", self.id, self.cvssv3, ex) - # remove invalid cvssv3 vector for new findings, or should we just throw a ValidationError? - if self.pk is None: - self.cvssv3 = None - - # behaviour for CVVS4 is slightly different. Extracting this into a method would lead to probably hard to read code - if self.cvssv4: - try: - cvss_data = parse_cvss_data(self.cvssv4) - if cvss_data: - self.cvssv4 = cvss_data.get("cvssv4") - self.cvssv4_score = cvss_data.get("cvssv4_score") - - except Exception as ex: - logger.warning("Can't compute cvssv4 score for finding id %i. Invalid cvssv4 vector found: '%s'. Exception: %s.", self.id, self.cvssv4, ex) - self.cvssv4 = None - - self.set_hash_code(dedupe_option) - - if is_new_finding: - if settings.V3_FEATURE_LOCATIONS: - if (self.file_path is not None) and (len(self.unsaved_locations) == 0): - self.static_finding = True - self.dynamic_finding = False - elif (self.file_path is not None): - self.static_finding = True - # TODO: Delete this after the move to Locations - elif (self.file_path is not None) and (len(self.unsaved_endpoints) == 0): - self.static_finding = True - self.dynamic_finding = False - elif (self.file_path is not None): - self.static_finding = True - - # because we have reduced the number of (super()).save() calls, the helper is no longer called for new findings - # so we call it manually - finding_helper.update_finding_status(self, user, changed_fields={"id": (None, None)}) - - # logger.debug('setting static / dynamic in save') - # need to have an id/pk before we can access locations/endpoints - elif self.file_path is not None: - if settings.V3_FEATURE_LOCATIONS: - if not self.locations.exists(): - self.static_finding = True - self.dynamic_finding = False - else: - self.static_finding = True - # TODO: Delete this after the move to Locations - elif not self.endpoints.exists(): - self.static_finding = True - self.dynamic_finding = False - else: - self.static_finding = True - - # update the SLA expiration date last, after all other finding fields have been updated - self.set_sla_expiration_date() - - logger.debug("Saving finding of id " + str(self.id) + " dedupe_option:" + str(dedupe_option) + " (self.pk is %s)", "None" if self.pk is None else "not None") - # We cannot run the full_clean method here without issue, so we specify skip_validation - super().save(*args, **kwargs, skip_validation=True) - - # Only add to found_by for newly-created findings (avoid doing this on every update) - if is_new_finding: - self.found_by.add(self.test.test_type) - - # only perform post processing (in celery task) if needed. this check avoids submitting 1000s of tasks to celery that will do nothing - system_settings = System_Settings.objects.get() - if dedupe_option or issue_updater_option or (product_grading_option and system_settings.enable_product_grade) or push_to_jira: - finding_helper.post_process_finding_save(self.id, dedupe_option=dedupe_option, rules_option=rules_option, product_grading_option=product_grading_option, - issue_updater_option=issue_updater_option, push_to_jira=push_to_jira, user=user, *args, **kwargs) - else: - logger.debug("no options selected that require finding post processing") - - def get_absolute_url(self): - return reverse("view_finding", args=[str(self.id)]) - - def copy(self, test=None): - copy = copy_model_util(self) - # Save the necessary ManyToMany relationships - old_notes = list(self.notes.all()) - old_files = list(self.files.all()) - old_reviewers = list(self.reviewers.all()) - old_found_by = list(self.found_by.all()) - old_tags = list(self.tags.all()) - # Wipe the IDs of the new object - if test: - copy.test = test - # Save the object before setting any ManyToMany relationships - copy.save() - # Copy the notes - for notes in old_notes: - copy.notes.add(notes.copy()) - # Copy the files - for files in old_files: - copy.files.add(files.copy()) - if settings.V3_FEATURE_LOCATIONS: - old_location_refs = self.locations.all() - for location_ref in old_location_refs: - location_ref.copy(copy) - else: - # TODO: Delete this after the move to Locations - # Copy the endpoint_status - old_status_findings = list(self.status_finding.all()) - for endpoint_status in old_status_findings: - endpoint_status.copy(finding=copy) # adding or setting is not necessary, link is created by Endpoint_Status.copy() - # Assign any reviewers - copy.reviewers.set(old_reviewers) - # Assign any found_by - copy.found_by.set(old_found_by) - # Assign any tags - copy.tags.set(old_tags) - - return copy - - def delete(self, *args, product_grading_option=True, **kwargs): - logger.debug("%d finding delete", self.id) - from dojo.finding import helper as finding_helper # noqa: PLC0415 circular import - finding_helper.finding_delete(self) - super().delete(*args, **kwargs) - if product_grading_option: - with suppress(Finding.DoesNotExist, Test.DoesNotExist, Engagement.DoesNotExist, Product.DoesNotExist): - # Suppressing a potential issue created from async delete removing - # related objects in a separate task - from dojo.utils import perform_product_grading # noqa: PLC0415 circular import - perform_product_grading(self.test.engagement.product) - - # only used by bulk risk acceptance api - @classmethod - def unaccepted_open_findings(cls): - from dojo.utils import get_system_setting # noqa: PLC0415 circular import - results = cls.objects.filter(active=True, duplicate=False, risk_accepted=False) - if get_system_setting("enforce_verified_status", True) or get_system_setting("enforce_verified_status_metrics", True): - results = results.filter(verified=True) - - return results - - @property - def risk_acceptance(self): - ras = self.risk_acceptance_set.all() - if ras: - return ras[0] - - return None - - def compute_hash_code(self): - # Allow Pro to overwrite compute hash_code which gets dedupe settings from a database instead of django.settings - from dojo.utils import get_custom_method # noqa: PLC0415 circular import - if compute_hash_code_method := get_custom_method("FINDING_COMPUTE_HASH_METHOD"): - deduplicationLogger.debug("using custom FINDING_COMPUTE_HASH_METHOD method") - return compute_hash_code_method(self) - - # Check if all needed settings are defined - if not hasattr(settings, "HASHCODE_FIELDS_PER_SCANNER") or not hasattr(settings, "HASHCODE_ALLOWS_NULL_CWE") or not hasattr(settings, "HASHCODE_ALLOWED_FIELDS"): - deduplicationLogger.debug("no or incomplete configuration per hash_code found; using legacy algorithm") - return self.compute_hash_code_legacy() - - hash_code_fields = self.test.hash_code_fields - - # Check if hash_code fields are found in the settings - if not hash_code_fields: - deduplicationLogger.debug( - "No configuration for hash_code computation found; using default fields for " + ("dynamic" if self.dynamic_finding else "static") + " scanners") - return self.compute_hash_code_legacy() - - # Check if all elements of HASHCODE_FIELDS_PER_SCANNER are in HASHCODE_ALLOWED_FIELDS - if not (all(elem in settings.HASHCODE_ALLOWED_FIELDS for elem in hash_code_fields)): - deduplicationLogger.debug( - "compute_hash_code - configuration error: some elements of HASHCODE_FIELDS_PER_SCANNER are not in the allowed list HASHCODE_ALLOWED_FIELDS. " - "Using default fields") - return self.compute_hash_code_legacy() - - # Make sure that we have a cwe if we need one - if self.cwe == 0 and not self.test.hash_code_allows_null_cwe: - deduplicationLogger.debug( - "Cannot compute hash_code based on configured fields because cwe is 0 for finding of title '" + self.title + "' found in file '" + str(self.file_path) - + "'. Fallback to legacy mode for this finding.") - return self.compute_hash_code_legacy() - - deduplicationLogger.debug("computing hash_code for finding id " + str(self.id) + " based on: " + ", ".join(hash_code_fields)) - - fields_to_hash = "" - for hashcodeField in hash_code_fields: - # Note: preserve this field label ("endpoints") for settings purposes through the Locations migration - if hashcodeField == "endpoints": - # For locations/endpoints, need to compute the field - locations = self.get_locations() - fields_to_hash += locations - deduplicationLogger.debug(hashcodeField + " : " + locations) - elif hashcodeField == "vulnerability_ids": - # For vulnerability_ids, need to compute the field - my_vulnerability_ids = self.get_vulnerability_ids() - fields_to_hash += my_vulnerability_ids - deduplicationLogger.debug(hashcodeField + " : " + my_vulnerability_ids) - else: - # Generically use the finding attribute having the same name, converts to str in case it's integer - fields_to_hash += str(getattr(self, hashcodeField)) - deduplicationLogger.debug(hashcodeField + " : " + str(getattr(self, hashcodeField))) - - # Log the hash_code fields that are always included (but are not part of the hash_code_fields list as they are inserted downtstream in self.hash_fields) - hash_code_fields_always = getattr(settings, "HASH_CODE_FIELDS_ALWAYS", []) - for hashcodeField in hash_code_fields_always: - if getattr(self, hashcodeField): - deduplicationLogger.debug(hashcodeField + " : " + str(getattr(self, hashcodeField))) - - deduplicationLogger.debug("compute_hash_code - fields_to_hash = " + fields_to_hash) - return self.hash_fields(fields_to_hash) - - def compute_hash_code_legacy(self): - fields_to_hash = self.title + str(self.cwe) + str(self.line) + str(self.file_path) + self.description - deduplicationLogger.debug("compute_hash_code_legacy - fields_to_hash = " + fields_to_hash) - return self.hash_fields(fields_to_hash) - - # Get vulnerability_ids to use for hash_code computation - def get_vulnerability_ids(self): - - def _get_unsaved_vulnerability_ids(finding) -> str: - if finding.unsaved_vulnerability_ids: - deduplicationLogger.debug("get_vulnerability_ids before the finding was saved") - # convert list of unsaved vulnerability_ids to the list of their canonical representation - vulnerability_id_str_list = [str(vulnerability_id) for vulnerability_id in finding.unsaved_vulnerability_ids] - # deduplicate (usually done upon saving finding) and sort endpoints - return "".join(sorted(dict.fromkeys(vulnerability_id_str_list))) - deduplicationLogger.debug("finding has no unsaved vulnerability references") - return "" - - def _get_saved_vulnerability_ids(finding) -> str: - if finding.id is not None: - vulnerability_ids = Vulnerability_Id.objects.filter(finding=finding) - deduplicationLogger.debug("get_vulnerability_ids after the finding was saved. Vulnerability references count: " + str(vulnerability_ids.count())) - # convert list of vulnerability_ids to the list of their canonical representation - vulnerability_id_str_list = [str(vulnerability_id) for vulnerability_id in vulnerability_ids.all()] - # sort vulnerability_ids strings - return "".join(sorted(vulnerability_id_str_list)) - return "" - - return _get_saved_vulnerability_ids(self) or _get_unsaved_vulnerability_ids(self) - - # Get locations/endpoints to use for hash_code computation - def get_locations(self): - # TODO: Delete this after the move to Locations - if not settings.V3_FEATURE_LOCATIONS: - # Get endpoints to use for hash_code computation - # (This sometimes reports "None") - def _get_unsaved_endpoints(finding) -> str: - if len(finding.unsaved_endpoints) > 0: - deduplicationLogger.debug("get_endpoints before the finding was saved") - # convert list of unsaved endpoints to the list of their canonical representation - endpoint_str_list = [str(endpoint) for endpoint in finding.unsaved_endpoints] - # deduplicate (usually done upon saving finding) and sort endpoints - return "".join(dict.fromkeys(endpoint_str_list)) - # we can get here when the parser defines static_finding=True but leaves dynamic_finding defaulted - # In this case, before saving the finding, both static_finding and dynamic_finding are True - # After saving dynamic_finding may be set to False probably during the saving process (observed on Bandit scan before forcing dynamic_finding=False at parser level) - deduplicationLogger.debug("trying to get endpoints on a finding before it was saved but no endpoints found (static parser wrongly identified as dynamic?") - return "" - - def _get_saved_endpoints(finding) -> str: - if finding.id is not None: - deduplicationLogger.debug("get_endpoints: after the finding was saved. Endpoints count: " + str(finding.endpoints.count())) - # convert list of endpoints to the list of their canonical representation - endpoint_str_list = [str(endpoint) for endpoint in finding.endpoints.all()] - # sort endpoints strings - return "".join(sorted(endpoint_str_list)) - return "" - - return _get_saved_endpoints(self) or _get_unsaved_endpoints(self) - - def _get_unsaved_locations(finding) -> str: - if len(finding.unsaved_locations) > 0: - deduplicationLogger.debug("get_locations before the finding was saved") - # convert list of unsaved locations to the list of their canonical representation - from dojo.importers.location_manager import LocationManager # noqa: PLC0415 - unsaved_locations = LocationManager.clean_unsaved_locations(finding.unsaved_locations) - # deduplicate (usually done upon saving finding) and sort locations - locations = sorted({location.get_location_value() for location in unsaved_locations}) - return "".join(locations) - # we can get here when the parser defines static_finding=True but leaves dynamic_finding defaulted - # In this case, before saving the finding, both static_finding and dynamic_finding are True - # After saving dynamic_finding may be set to False probably during the saving process (observed on Bandit scan before forcing dynamic_finding=False at parser level) - deduplicationLogger.debug("trying to get locations on a finding before it was saved but no locations found (static parser wrongly identified as dynamic?") - return "" - - def _get_saved_locations(finding) -> str: - if finding.id is not None: - from dojo.url.models import URL # noqa: PLC0415 - url_locations = finding.locations.filter(location__location_type=URL.get_location_type()) - deduplicationLogger.debug("get_locations: after the finding was saved. Locations count: " + str(url_locations.count())) - # convert list of locations to the list of their canonical representation - locations = sorted({location_ref.location.get_location_value() for location_ref in url_locations.all()}) - # sort locations strings - return "".join(sorted(locations)) - return "" - - return _get_saved_locations(self) or _get_unsaved_locations(self) - - # Compute the hash_code from the fields to hash - def hash_fields(self, fields_to_hash): - if hasattr(settings, "HASH_CODE_FIELDS_ALWAYS"): - for field in settings.HASH_CODE_FIELDS_ALWAYS: - if getattr(self, field): - deduplicationLogger.debug("adding HASH_CODE_FIELDS_ALWAYSfield %s to hash_fields: %s", field, getattr(self, field)) - fields_to_hash += str(getattr(self, field)) - - logger.debug("fields_to_hash : %s", fields_to_hash) - logger.debug("fields_to_hash lower: %s", fields_to_hash.lower()) - return hashlib.sha256(fields_to_hash.casefold().encode("utf-8").strip()).hexdigest() - - def duplicate_finding_set(self): - if self.duplicate: - if self.duplicate_finding is not None: - return Finding.objects.get( - id=self.duplicate_finding.id).original_finding.all().order_by("title") - return [] - return self.original_finding.all().order_by("title") - - def get_scanner_confidence_text(self): - if self.scanner_confidence and isinstance(self.scanner_confidence, int): - if self.scanner_confidence <= 2: - return "Certain" - if self.scanner_confidence >= 3 and self.scanner_confidence <= 5: - return "Firm" - return "Tentative" - return "" - - @staticmethod - def get_numerical_severity(severity): - if severity == "Critical": - return "S0" - if severity == "High": - return "S1" - if severity == "Medium": - return "S2" - if severity == "Low": - return "S3" - if severity == "Info": - return "S4" - return "S5" - - @staticmethod - def get_number_severity(severity): - if severity == "Critical": - return 4 - if severity == "High": - return 3 - if severity == "Medium": - return 2 - if severity == "Low": - return 1 - if severity == "Info": - return 0 - return 5 - - @staticmethod - def get_severity(num_severity): - severities = {0: "Info", 1: "Low", 2: "Medium", 3: "High", 4: "Critical"} - if num_severity in severities: - return severities[num_severity] - - return None - - def status(self): - status = [] - if self.under_review: - status += ["Under Review"] - if self.active: - status += ["Active"] - else: - status += ["Inactive"] - if self.verified: - status += ["Verified"] - if self.mitigated or self.is_mitigated: - status += ["Mitigated"] - if self.false_p: - status += ["False Positive"] - if self.out_of_scope: - status += ["Out Of Scope"] - if self.duplicate: - status += ["Duplicate"] - if self.risk_accepted: - status += ["Risk Accepted"] - if not len(status): - status += ["Initial"] - - return ", ".join([str(s) for s in status]) - - def _age(self, start_date): - if start_date and isinstance(start_date, str): - start_date = datetutilsparse(start_date).date() - - if isinstance(start_date, datetime): - start_date = start_date.date() - - if self.mitigated: - mitigated_date = self.mitigated - if isinstance(mitigated_date, datetime): - mitigated_date = self.mitigated.date() - diff = mitigated_date - start_date - else: - diff = get_current_date() - start_date - days = diff.days - return max(0, days) - - @property - def age(self): - return self._age(self.date) - - @property - def sla_age(self): - return self._age(self.get_sla_start_date()) - - def get_sla_start_date(self): - if self.sla_start_date: - return self.sla_start_date - return self.date - - def get_sla_configuration(self): - return self.test.engagement.product.sla_configuration - - def get_sla_period(self): - # Determine which method to use to calculate the SLA - from dojo.utils import get_custom_method # noqa: PLC0415 circular import - if method := get_custom_method("FINDING_SLA_PERIOD_METHOD"): - return method(self) - # Run the default method - sla_configuration = self.get_sla_configuration() - sla_period = getattr(sla_configuration, self.severity.lower(), None) - enforce_period = getattr(sla_configuration, str("enforce_" + self.severity.lower()), None) - return sla_period, enforce_period - - def set_sla_expiration_date(self): - # First check if SLA is enabled globally - system_settings = System_Settings.objects.get() - if not system_settings.enable_finding_sla: - return - # Call the internal method to set the sla expiration date - self._set_sla_expiration_date() - - def _set_sla_expiration_date(self): - # some parsers provide date as a `str` instead of a `date` in which case we need to parse it #12299 on GitHub - sla_start_date = self.get_sla_start_date() - if sla_start_date and isinstance(sla_start_date, str): - sla_start_date = dateutil.parser.parse(sla_start_date).date() - - sla_period, enforce_period = self.get_sla_period() - if sla_period is not None and enforce_period: - self.sla_expiration_date = sla_start_date + relativedelta(days=sla_period) - else: - self.sla_expiration_date = None - - def sla_days_remaining(self): - if self.sla_expiration_date: - if self.mitigated: - mitigated_date = self.mitigated - if isinstance(mitigated_date, datetime): - mitigated_date = self.mitigated.date() - return (self.sla_expiration_date - mitigated_date).days - return (self.sla_expiration_date - get_current_date()).days - return None - - def sla_deadline(self): - return self.sla_expiration_date - - def github(self): - try: - return self.github_issue - except GITHUB_Issue.DoesNotExist: - return None - - def has_github_issue(self): - try: - # Attempt to access the github issue if it exists. If not, an exception will be caught - _ = self.github_issue - except GITHUB_Issue.DoesNotExist: - return False - return True - - def github_conf(self): - try: - github_product_key = GITHUB_PKey.objects.get(product=self.test.engagement.product) - github_conf = github_product_key.conf - except: - github_conf = None - return github_conf - - # newer version that can work with prefetching - def github_conf_new(self): - try: - return self.test.engagement.product.github_pkey_set.all()[0].git_conf - except: - return None - - @property - def has_jira_issue(self): - from dojo.jira import services as jira_services # noqa: PLC0415 circular import - return jira_services.has_issue(self) - - @cached_property - def finding_group(self): - return self.finding_group_set.all().first() - # logger.debug('finding.finding_group: %s', group) - - @cached_property - def has_jira_group_issue(self): - if not self.has_finding_group: - return False - - from dojo.jira import services as jira_services # noqa: PLC0415 circular import - return jira_services.has_issue(self.finding_group) - - @property - def has_jira_configured(self): - from dojo.jira import services as jira_services # noqa: PLC0415 circular import - return jira_services.has_configured(self) - - @cached_property - def has_finding_group(self): - return self.finding_group is not None - - def save_no_options(self, *args, **kwargs): - logger.debug("save_no_options") - return self.save(dedupe_option=False, rules_option=False, product_grading_option=False, - issue_updater_option=False, push_to_jira=False, user=None, *args, **kwargs) - - # Check if a mandatory field is empty. If it's the case, fill it with "no given" - def clean(self): - no_check = ["test", "reporter"] - bigfields = ["description"] - for field_obj in self._meta.fields: - field = field_obj.name - if field not in no_check: - val = getattr(self, field) - if not val and field == "title": - setattr(self, field, "No title given") - if not val and field in bigfields: - setattr(self, field, f"No {field} given") - - def severity_display(self): - return self.severity - - def get_breadcrumbs(self): - bc = self.test.get_breadcrumbs() - bc += [{"title": str(self), - "url": reverse("view_finding", args=(self.id,))}] - return bc - - def get_valid_request_response_pairs(self): - empty_value = base64.b64encode(b"") - # Get a list of all req/resp pairs - all_req_resps = self.burprawrequestresponse_set.all() - # Filter away those that do not have any contents - return all_req_resps.exclude( - burpRequestBase64__exact=empty_value, - burpResponseBase64__exact=empty_value, - ) - - def get_report_requests(self): - # Get the list of request response pairs that are non empty - request_response_pairs = self.get_valid_request_response_pairs() - # Determine how many to return - if request_response_pairs.count() >= 3: - return request_response_pairs[0:3] - if request_response_pairs.count() > 0: - return request_response_pairs - return None - - def get_request(self): - # Get the list of request response pairs that are non empty - request_response_pairs = self.get_valid_request_response_pairs() - # Determine what to return - if request_response_pairs.count() > 0: - reqres = request_response_pairs.first() - return base64.b64decode(reqres.burpRequestBase64) - - def get_response(self): - # Get the list of request response pairs that are non empty - request_response_pairs = self.get_valid_request_response_pairs() - # Determine what to return - if request_response_pairs.count() > 0: - reqres = request_response_pairs.first() - res = base64.b64decode(reqres.burpResponseBase64) - # Removes all blank lines - return re.sub(r"\n\s*\n", "\n", res) - - def latest_note(self): - if self.notes.all(): - note = self.notes.all()[0] - return note.date.strftime("%Y-%m-%d %H:%M:%S") + ": " + note.author.get_full_name() + " : " + note.entry - - return "" - - def get_sast_source_file_path_with_link(self): - from dojo.utils import create_bleached_link # noqa: PLC0415 circular import - if self.sast_source_file_path is None: - return None - if self.test.engagement.source_code_management_uri is None: - return escape(self.sast_source_file_path) - link = self.test.engagement.source_code_management_uri + "/" + self.sast_source_file_path - if self.sast_source_line: - link = link + "#L" + str(self.sast_source_line) - return create_bleached_link(link, self.sast_source_file_path) - - def get_file_path_with_link(self): - from dojo.utils import create_bleached_link # noqa: PLC0415 circular import - if self.file_path is None: - return None - if self.test.engagement.source_code_management_uri is None: - return escape(self.file_path) - link = self.get_file_path_with_raw_link() - return create_bleached_link(link, self.file_path) - - def get_scm_type(self): - # extract scm type from product custom field 'scm-type' - - if hasattr(self.test.engagement, "product"): - dojo_meta = DojoMeta.objects.filter(product=self.test.engagement.product, name="scm-type").first() - if dojo_meta: - st = dojo_meta.value.strip() - if st: - return st.lower() - return "" - - def scm_public_prepare_base_link(self, uri): - # scm public (https://scm-domain.org) url template for browse is: - # https://scm-domain.org// - # but when you get repo url for git, its template is: - # https://scm-domain.org//.git - # so to create browser url - git url should be recomposed like below: - - parts_uri = uri.split(".git") - return parts_uri[0] - - def git_public_prepare_scm_link(self, uri, scm_type): - # if commit hash or branch/tag is set for engagement/test - - # hash or branch/tag should be appended to base browser link - intermediate_path = "/blob/" if scm_type in {"github", "gitlab"} else "/src/" - - link = self.scm_public_prepare_base_link(uri) - if self.test.commit_hash: - link += intermediate_path + self.test.commit_hash + "/" + self.file_path - elif self.test.engagement.commit_hash: - link += intermediate_path + self.test.engagement.commit_hash + "/" + self.file_path - elif self.test.branch_tag: - link += intermediate_path + self.test.branch_tag + "/" + self.file_path - elif self.test.engagement.branch_tag: - link += intermediate_path + self.test.engagement.branch_tag + "/" + self.file_path - else: - link += intermediate_path + "master/" + self.file_path - - return link - - def bitbucket_standalone_prepare_scm_base_link(self, uri): - # bitbucket onpremise/standalone url template for browse is: - # https://bb.example.com/projects//repos/ - # but when you get repo url for git, its template is: - # https://bb.example.com/scm//.git - # or for user public repo^ - # https://bb.example.com/users//repos/ - # but when you get repo url for git, its template is: - # https://bb.example.com/scm//.git (username often could be prefixed with ~) - # so to create borwser url - git url should be recomposed like below: - - parts_uri = uri.split(".git") - parts_scm = parts_uri[0].split("/scm/") - parts_project = parts_scm[1].split("/") - project = parts_project[0] - if project.startswith("~"): - return parts_scm[0] + "/users/" + parts_project[0][1:] + "/repos/" + parts_project[1] + "/browse" - return parts_scm[0] + "/projects/" + parts_project[0] + "/repos/" + parts_project[1] + "/browse" - - def bitbucket_standalone_prepare_scm_link(self, uri): - # if commit hash or branch/tag is set for engagement/test - - # hash or barnch/tag should be appended to base browser link - - link = self.bitbucket_standalone_prepare_scm_base_link(uri) - if self.test.commit_hash: - link += "/" + self.file_path + "?at=" + self.test.commit_hash - elif self.test.engagement.commit_hash: - link += "/" + self.file_path + "?at=" + self.test.engagement.commit_hash - elif self.test.branch_tag: - link += "/" + self.file_path + "?at=" + self.test.branch_tag - elif self.test.engagement.branch_tag: - link += "/" + self.file_path + "?at=" + self.test.engagement.branch_tag - else: - link += "/" + self.file_path - - return link - - def get_file_path_with_raw_link(self): - if self.file_path is None: - return None - - link = self.test.engagement.source_code_management_uri - scm_type = self.get_scm_type() - if (self.test.engagement.source_code_management_uri is not None): - if scm_type == "bitbucket-standalone": - link = self.bitbucket_standalone_prepare_scm_link(link) - elif scm_type in {"github", "gitlab", "gitea", "codeberg", "bitbucket"}: - link = self.git_public_prepare_scm_link(link, scm_type) - elif "https://github.com/" in self.test.engagement.source_code_management_uri: - link = self.git_public_prepare_scm_link(link, "github") - else: - link += "/" + self.file_path - else: - link += "/" + self.file_path - - # than - add line part to browser url - if self.line: - if scm_type in {"github", "gitlab", "gitea", "codeberg"} or "https://github.com/" in self.test.engagement.source_code_management_uri: - link = link + "#L" + str(self.line) - elif scm_type == "bitbucket-standalone": - link = link + "#" + str(self.line) - elif scm_type == "bitbucket": - link = link + "#lines-" + str(self.line) - return link - - def get_references_with_links(self): - from dojo.utils import create_bleached_link # noqa: PLC0415 circular import - if self.references is None: - return None - matches = re.findall(r"([\(|\[]?(https?):((//)|(\\\\))+([\w\d:#@%/;$~_?\+-=\\\.&](#!)?)*[\)|\]]?)", self.references) - - processed_matches = [] - for match in matches: - # Check if match isn't already a markdown link - # Only replace the same matches one time, otherwise the links will be corrupted - if not (match[0].startswith("[") or match[0].startswith("(")) and match[0] not in processed_matches: - self.references = self.references.replace(match[0], create_bleached_link(match[0], match[0]), 1) - processed_matches.append(match[0]) - - return self.references - - @cached_property - def vulnerability_ids(self): - # Get vulnerability ids from database and convert to list of strings - vulnerability_ids_model = self.vulnerability_id_set.all() - vulnerability_ids = [vulnerability_id.vulnerability_id for vulnerability_id in vulnerability_ids_model] - - # Synchronize the cve field with the unsaved_vulnerability_ids - # We do this to be as flexible as possible to handle the fields until - # the cve field is not needed anymore and can be removed. - if vulnerability_ids and self.cve: - # Make sure the first entry of the list is the value of the cve field - vulnerability_ids.insert(0, self.cve) - elif not vulnerability_ids and self.cve: - # If there is no list, make one with the value of the cve field - vulnerability_ids = [self.cve] - - # Remove duplicates - return list(dict.fromkeys(vulnerability_ids)) - - @property - def violates_sla(self): - return (self.sla_expiration_date and self.sla_expiration_date < timezone.now().date()) - - def set_hash_code(self, dedupe_option): - from dojo.utils import get_custom_method # noqa: PLC0415 circular import - if hash_method := get_custom_method("FINDING_HASH_METHOD"): - deduplicationLogger.debug("Using custom hash method") - hash_method(self, dedupe_option) - # Finding.save is called once from serializers.py with dedupe_option=False because the finding is not ready yet, for example the endpoints are not built - # It is then called a second time with dedupe_option defaulted to true; now we can compute the hash_code and run the deduplication - elif dedupe_option: - finding_id = self.id if self.id is not None else "unsaved" - if self.hash_code is not None: - deduplicationLogger.debug("Hash_code already computed for finding: %s", finding_id) - else: - self.hash_code = self.compute_hash_code() - deduplicationLogger.debug("Hash_code computed for finding: %s: %s", finding_id, self.hash_code) - - -class FindingAdmin(admin.ModelAdmin): - # TODO: Delete this after the move to Locations - # For efficiency with large databases, display many-to-many fields with raw - # IDs rather than multi-select - raw_id_fields = ( - "endpoints", - ) - - -class Vulnerability_Id(models.Model): - finding = models.ForeignKey(Finding, editable=False, on_delete=models.CASCADE) - vulnerability_id = models.TextField(max_length=50, blank=False, null=False) - - def __str__(self): - return self.vulnerability_id - - def get_absolute_url(self): - return reverse("view_finding", args=[str(self.finding.id)]) - - -class Finding_Group(TimeStampedModel): - - GROUP_BY_OPTIONS = [("component_name", "Component Name"), - ("component_name+component_version", "Component Name + Version"), - ("file_path", "File path"), - ("finding_title", "Finding Title"), - ("vuln_id_from_tool", "Vulnerability ID from Tool")] - - name = models.CharField(max_length=255, blank=False, null=False) - test = models.ForeignKey(Test, on_delete=models.CASCADE) - findings = models.ManyToManyField(Finding) - creator = models.ForeignKey(Dojo_User, on_delete=models.RESTRICT) - - def __str__(self): - return self.name - - @property - def has_jira_issue(self): - from dojo.jira import services as jira_services # noqa: PLC0415 circular import - return jira_services.has_issue(self) - - @cached_property - def severity(self): - if not self.findings.all(): - return None - max_number_severity = max(Finding.get_number_severity(find.severity) for find in self.findings.all()) - return Finding.get_severity(max_number_severity) - - @cached_property - def components(self): - components: dict[str, set[str | None]] = {} - for finding in self.findings.all(): - if finding.component_name is not None: - components.setdefault(finding.component_name, set()).add(finding.component_version) - return "; ".join(f"""{name}: {", ".join(map(str, versions))}""" for name, versions in components.items()) - - @property - def age(self): - if not self.findings.all(): - return None - - return max(find.age for find in self.findings.all()) - - @cached_property - def sla_days_remaining_internal(self): - if not self.findings.all(): - return None - - return min([find.sla_days_remaining() for find in self.findings.all() if find.sla_days_remaining()], default=None) - - def sla_days_remaining(self): - return self.sla_days_remaining_internal - - def sla_deadline(self): - if not self.findings.all(): - return None - - return min([find.sla_deadline() for find in self.findings.all() if find.sla_deadline()], default=None) - - def status(self): - if not self.findings.all(): - return None - - if any(find.active for find in self.findings.all()): - return "Active" - - if all(find.is_mitigated for find in self.findings.all()): - return "Mitigated" - - return "Inactive" - - @cached_property - def mitigated(self): - return all(find.mitigated is not None for find in self.findings.all()) - - def get_sla_start_date(self): - return min(find.get_sla_start_date() for find in self.findings.all()) - - def get_absolute_url(self): - return reverse("view_test", args=[str(self.test.id)]) - - class Meta: - ordering = ["id"] - - -class Finding_Template(models.Model): - title = models.TextField(max_length=1000) - cwe = models.IntegerField(default=None, null=True, blank=True) - cve = models.CharField(max_length=50, - null=True, - blank=False, - verbose_name="Vulnerability Id", - help_text="An id of a vulnerability in a security advisory associated with this finding. Can be a Common Vulnerabilities and Exposures (CVE) or from other sources.") - cvssv3 = models.TextField(help_text=_("Common Vulnerability Scoring System version 3 (CVSSv3) score associated with this finding."), validators=[cvss3_validator], max_length=117, null=True, verbose_name=_("CVSS v3 vector")) - cvssv3_score = models.FloatField(null=True, blank=True, help_text=_("CVSSv3 score")) - cvssv4 = models.TextField(help_text=_("Common Vulnerability Scoring System version 4 (CVSS4) score associated with this finding."), validators=[cvss4_validator], max_length=255, null=True, verbose_name=_("CVSS4 vector")) - cvssv4_score = models.FloatField(null=True, blank=True, help_text=_("CVSSv4 score")) - - severity = models.CharField(max_length=200, null=True, blank=True) - description = models.TextField(null=True, blank=True) - mitigation = models.TextField(null=True, blank=True) - impact = models.TextField(null=True, blank=True) - references = models.TextField(null=True, blank=True, db_column="refs") - last_used = models.DateTimeField(null=True, editable=False) - numerical_severity = models.CharField(max_length=4, null=True, blank=True, editable=False) - - # Remediation planning fields - fix_available = models.BooleanField(null=True, blank=True, help_text=_("Indicates if a fix is available for this vulnerability type")) - fix_version = models.CharField(max_length=100, null=True, blank=True, help_text=_("Version where fix is available")) - planned_remediation_version = models.CharField(max_length=99, null=True, blank=True, help_text=_("Target version for remediation")) - effort_for_fixing = models.CharField(max_length=99, null=True, blank=True, help_text=_("Effort estimate for fixing (e.g., Low/Medium/High)")) - - # Technical details fields - steps_to_reproduce = models.TextField(null=True, blank=True, help_text=_("Standard reproduction steps for this vulnerability type")) - severity_justification = models.TextField(null=True, blank=True, help_text=_("Explanation of why this severity level is appropriate")) - component_name = models.CharField(max_length=500, null=True, blank=True, help_text=_("Affected component name (e.g., library name)")) - component_version = models.CharField(max_length=100, null=True, blank=True, help_text=_("Affected component version")) - - # Notes field (single note content, not a list) - notes = models.TextField(null=True, blank=True, help_text=_("Note content to add when applying this template")) - - # String-based list fields (newline-separated) - vulnerability_ids_text = models.TextField(null=True, blank=True, help_text=_("Vulnerability IDs (one per line)")) - endpoints_text = models.TextField(null=True, blank=True, help_text=_("Endpoint URLs (one per line)")) - - tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this finding template. Choose from the list or add new tags. Press Enter key to add.")) - - SEVERITIES = {"Info": 4, "Low": 3, "Medium": 2, - "High": 1, "Critical": 0} - - class Meta: - ordering = ["-cwe"] - - def __str__(self): - return self.title - - def get_absolute_url(self): - return reverse("edit_template", args=[str(self.id)]) - - def get_breadcrumbs(self): - return [{"title": str(self), - "url": reverse("view_template", args=(self.id,))}] - - @property - def vulnerability_ids(self): - """Parse vulnerability IDs from TextField string (newline-separated).""" - vulnerability_ids = [] - - # Get from the TextField - if self.vulnerability_ids_text: - # Parse newline-separated string, remove empty lines - vulnerability_ids = [line.strip() for line in self.vulnerability_ids_text.split("\n") if line.strip()] - - # Synchronize the cve field with the vulnerability_ids - # We do this to be as flexible as possible to handle the fields until - # the cve field is not needed anymore and can be removed. - if vulnerability_ids and self.cve and self.cve not in vulnerability_ids: - # Make sure the first entry of the list is the value of the cve field - vulnerability_ids.insert(0, self.cve) - elif not vulnerability_ids and self.cve: - # If there is no list, make one with the value of the cve field - vulnerability_ids = [self.cve] - - # Remove duplicates - return list(dict.fromkeys(vulnerability_ids)) - - @property - def endpoints(self): - """Parse endpoint URLs from TextField string (newline-separated).""" - if not self.endpoints_text: - return [] - # Parse newline-separated string, remove empty lines - return [line.strip() for line in self.endpoints_text.split("\n") if line.strip()] - - -class Check_List(models.Model): - session_management = models.CharField(max_length=50, default="none") - session_issues = models.ManyToManyField(Finding, - related_name="session_issues", - blank=True) - encryption_crypto = models.CharField(max_length=50, default="none") - crypto_issues = models.ManyToManyField(Finding, - related_name="crypto_issues", - blank=True) - configuration_management = models.CharField(max_length=50, default="") - config_issues = models.ManyToManyField(Finding, - related_name="config_issues", - blank=True) - authentication = models.CharField(max_length=50, default="none") - auth_issues = models.ManyToManyField(Finding, - related_name="auth_issues", - blank=True) - authorization_and_access_control = models.CharField(max_length=50, - default="none") - author_issues = models.ManyToManyField(Finding, - related_name="author_issues", - blank=True) - data_input_sanitization_validation = models.CharField(max_length=50, - default="none") - data_issues = models.ManyToManyField(Finding, related_name="data_issues", - blank=True) - sensitive_data = models.CharField(max_length=50, default="none") - sensitive_issues = models.ManyToManyField(Finding, - related_name="sensitive_issues", - blank=True) - other = models.CharField(max_length=50, default="none") - other_issues = models.ManyToManyField(Finding, related_name="other_issues", - blank=True) - engagement = models.ForeignKey(Engagement, editable=False, - related_name="eng_for_check", on_delete=models.CASCADE) - - @staticmethod - def get_status(pass_fail): - if pass_fail == "Pass": # noqa: S105 - return "success" - if pass_fail == "Fail": # noqa: S105 - return "danger" - return "warning" - - def get_breadcrumb(self): - bc = self.engagement.get_breadcrumb() - bc += [{"title": "Check List", - "url": reverse("complete_checklist", - args=(self.engagement.id,))}] - return bc - - -class BurpRawRequestResponse(models.Model): - finding = models.ForeignKey(Finding, blank=True, null=True, on_delete=models.CASCADE) - burpRequestBase64 = models.BinaryField() - burpResponseBase64 = models.BinaryField() - - def get_request(self): - return str(base64.b64decode(self.burpRequestBase64), errors="ignore") - - def get_response(self): - res = str(base64.b64decode(self.burpResponseBase64), errors="ignore") - # Removes all blank lines - return re.sub(r"\n\s*\n", "\n", res) - - -class Risk_Acceptance(models.Model): - TREATMENT_ACCEPT = "A" - TREATMENT_AVOID = "V" - TREATMENT_MITIGATE = "M" - TREATMENT_FIX = "F" - TREATMENT_TRANSFER = "T" - - TREATMENT_TRANSLATIONS = { - TREATMENT_ACCEPT: _("Accept (The risk is acknowledged, yet remains)"), - TREATMENT_AVOID: _("Avoid (Do not engage with whatever creates the risk)"), - TREATMENT_MITIGATE: _("Mitigate (The risk still exists, yet compensating controls make it less of a threat)"), - TREATMENT_FIX: _("Fix (The risk is eradicated)"), - TREATMENT_TRANSFER: _("Transfer (The risk is transferred to a 3rd party)"), - } - - TREATMENT_CHOICES = [ - (TREATMENT_ACCEPT, TREATMENT_TRANSLATIONS[TREATMENT_ACCEPT]), - (TREATMENT_AVOID, TREATMENT_TRANSLATIONS[TREATMENT_AVOID]), - (TREATMENT_MITIGATE, TREATMENT_TRANSLATIONS[TREATMENT_MITIGATE]), - (TREATMENT_FIX, TREATMENT_TRANSLATIONS[TREATMENT_FIX]), - (TREATMENT_TRANSFER, TREATMENT_TRANSLATIONS[TREATMENT_TRANSFER]), - ] - - name = models.CharField(max_length=300, null=False, blank=False, help_text=_("Descriptive name which in the future may also be used to group risk acceptances together across engagements and products")) - - accepted_findings = models.ManyToManyField(Finding) - - recommendation = models.CharField(choices=TREATMENT_CHOICES, max_length=2, null=False, default=TREATMENT_FIX, help_text=_("Recommendation from the security team."), verbose_name=_("Security Recommendation")) - - recommendation_details = models.TextField(null=True, - blank=True, - help_text=_("Explanation of security recommendation"), verbose_name=_("Security Recommendation Details")) - - decision = models.CharField(choices=TREATMENT_CHOICES, max_length=2, null=False, default=TREATMENT_ACCEPT, help_text=_("Risk treatment decision by risk owner")) - decision_details = models.TextField(default=None, blank=True, null=True, help_text=_("If a compensating control exists to mitigate the finding or reduce risk, then list the compensating control(s).")) - - accepted_by = models.CharField(max_length=200, default=None, null=True, blank=True, verbose_name=_("Accepted By"), help_text=_("The person that accepts the risk, can be outside of DefectDojo.")) - path = models.FileField(upload_to="risk/%Y/%m/%d", - editable=True, null=True, - blank=True, verbose_name=_("Proof")) - owner = models.ForeignKey(Dojo_User, editable=True, on_delete=models.RESTRICT, help_text=_("User in DefectDojo owning this acceptance. Only the owner and staff users can edit the risk acceptance.")) - - expiration_date = models.DateTimeField(default=None, null=True, blank=True, help_text=_("When the risk acceptance expires, the findings will be reactivated (unless disabled below).")) - expiration_date_warned = models.DateTimeField(default=None, null=True, blank=True, help_text=_("(readonly) Date at which notice about the risk acceptance expiration was sent.")) - expiration_date_handled = models.DateTimeField(default=None, null=True, blank=True, help_text=_("(readonly) When the risk acceptance expiration was handled (manually or by the daily job).")) - reactivate_expired = models.BooleanField(null=False, blank=False, default=True, verbose_name=_("Reactivate findings on expiration"), help_text=_("Reactivate findings when risk acceptance expires?")) - restart_sla_expired = models.BooleanField(default=False, null=False, verbose_name=_("Restart SLA on expiration"), help_text=_("When enabled, the SLA for findings is restarted when the risk acceptance expires.")) - - notes = models.ManyToManyField(Notes, editable=False) - created = models.DateTimeField(auto_now_add=True, null=False) - updated = models.DateTimeField(auto_now=True, editable=False) - - def __str__(self): - return str(self.name) - - def filename(self): - # logger.debug('path: "%s"', self.path) - if not self.path: - return None - return Path(self.path.name).name - - @property - def name_and_expiration_info(self): - return str(self.name) + (" (expired " if self.is_expired else " (expires ") + (timezone.localtime(self.expiration_date).strftime("%b %d, %Y") if self.expiration_date else "Never") + ")" - - def get_breadcrumbs(self): - bc = self.engagement_set.first().get_breadcrumbs() - bc += [{"title": str(self), - "url": reverse("view_risk_acceptance", args=( - self.engagement_set.first().product.id, self.id))}] - return bc - - @property - def is_expired(self): - return self.expiration_date_handled is not None - - # relationship is many to many, but we use it as one-to-many - @property - def engagement(self): - engs = self.engagement_set.all() - if engs: - return engs[0] - - return None - - def copy(self, engagement=None): - copy = copy_model_util(self) - # Save the necessary ManyToMany relationships - old_notes = list(self.notes.all()) - old_accepted_findings_hash_codes = [finding.hash_code for finding in self.accepted_findings.all()] - # Save the object before setting any ManyToMany relationships - copy.save() - # Copy the notes - for notes in old_notes: - copy.notes.add(notes.copy()) - # Assign any accepted findings - if engagement: - new_accepted_findings = Finding.objects.filter(test__engagement=engagement, hash_code__in=old_accepted_findings_hash_codes, risk_accepted=True).distinct() - copy.accepted_findings.set(new_accepted_findings) - return copy - - -class FileAccessToken(models.Model): - - """ - This will allow reports to request the images without exposing the - media root to the world without - authentication - """ - - user = models.ForeignKey(Dojo_User, null=False, blank=False, on_delete=models.CASCADE) - file = models.ForeignKey(FileUpload, null=False, blank=False, on_delete=models.CASCADE) - token = models.CharField(max_length=255) - size = models.CharField(max_length=9, - choices=( - ("small", "Small"), - ("medium", "Medium"), - ("large", "Large"), - ("thumbnail", "Thumbnail"), - ("original", "Original")), - default="medium") - - def save(self, *args, **kwargs): - if not self.token: - self.token = uuid4() - return super().save(*args, **kwargs) - - -ANNOUNCEMENT_STYLE_CHOICES = ( - ("info", "Info"), - ("success", "Success"), - ("warning", "Warning"), - ("danger", "Danger"), +from dojo.announcement.models import ( # noqa: E402 -- re-export + ANNOUNCEMENT_STYLE_CHOICES, # noqa: F401 -- re-export + Announcement, # noqa: F401 -- re-export + UserAnnouncement, # noqa: F401 -- re-export ) - - -class Announcement(models.Model): - message = models.CharField(max_length=500, - help_text=_("This dismissable message will be displayed on all pages for authenticated users. It can contain basic html tags, for example https://example.com"), - default="") - style = models.CharField(max_length=64, choices=ANNOUNCEMENT_STYLE_CHOICES, default="info", - help_text=_("The style of banner to display. (info, success, warning, danger)")) - dismissable = models.BooleanField(default=False, - null=False, - blank=True, - verbose_name=_("Dismissable?"), - help_text=_("Ticking this box allows users to dismiss the current announcement"), - ) - - -class UserAnnouncement(models.Model): - announcement = models.ForeignKey(Announcement, null=True, editable=False, on_delete=models.CASCADE, related_name="user_announcement") - user = models.ForeignKey(Dojo_User, null=True, editable=False, on_delete=models.CASCADE) - - -class BannerConf(models.Model): - banner_enable = models.BooleanField(default=False, null=True, blank=True) - banner_message = models.CharField(max_length=500, help_text=_("This message will be displayed on the login page. It can contain basic html tags, for example https://example.com"), default="") - - +from dojo.banner.models import BannerConf # noqa: E402, F401 -- re-export from dojo.github.models import ( # noqa: E402, F401 -- backward compat GITHUB_Clone, GITHUB_Conf, @@ -3980,28 +480,8 @@ class BannerConf(models.Model): Notification_Webhooks, Notifications, ) - - -class Tool_Product_Settings(models.Model): - name = models.CharField(max_length=200, null=False) - description = models.CharField(max_length=2000, null=True, blank=True) - url = models.CharField(max_length=2000, null=True, blank=True) - product = models.ForeignKey(Product, default=1, editable=False, on_delete=models.CASCADE) - tool_configuration = models.ForeignKey(Tool_Configuration, null=False, - related_name="tool_configuration", on_delete=models.CASCADE) - tool_project_id = models.CharField(max_length=200, null=True, blank=True) - notes = models.ManyToManyField(Notes, blank=True, editable=False) - - class Meta: - ordering = ["name"] - - -class Tool_Product_History(models.Model): - product = models.ForeignKey(Tool_Product_Settings, editable=False, on_delete=models.CASCADE) - last_scan = models.DateTimeField(null=False, editable=False, default=now) - succesfull = models.BooleanField(default=True, verbose_name=_("Succesfully")) - configuration_details = models.CharField(max_length=2000, null=True, - blank=True) +from dojo.risk_acceptance.models import Risk_Acceptance # noqa: E402, F401 -- re-export +from dojo.tool_product.models import Tool_Product_History, Tool_Product_Settings # noqa: E402, F401 -- re-export class Language_Type(models.Model): @@ -4046,38 +526,7 @@ def __str__(self): return self.name + " | " + self.product.name -class Objects_Review(models.Model): - name = models.CharField(max_length=100, null=True, blank=True) - created = models.DateTimeField(auto_now_add=True, null=False) - - def __str__(self): - return self.name - - -class Objects_Product(models.Model): - product = models.ForeignKey(Product, on_delete=models.CASCADE) - name = models.CharField(max_length=100, null=True, blank=True) - path = models.CharField(max_length=600, verbose_name=_("Full file path"), - null=True, blank=True) - folder = models.CharField(max_length=400, verbose_name=_("Folder"), - null=True, blank=True) - artifact = models.CharField(max_length=400, verbose_name=_("Artifact"), - null=True, blank=True) - review_status = models.ForeignKey(Objects_Review, on_delete=models.CASCADE) - created = models.DateTimeField(auto_now_add=True, null=False) - - tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this object. Choose from the list or add new tags. Press Enter key to add.")) - - def __str__(self): - name = None - if self.path is not None: - name = self.path - elif self.folder is not None: - name = self.folder - elif self.artifact is not None: - name = self.artifact - - return name +from dojo.object.models import Objects_Product, Objects_Review # noqa: E402, F401 -- re-export class Testing_Guide_Category(models.Model): @@ -4107,286 +556,34 @@ def __str__(self): return self.testing_guide_category.name + ": " + self.name -class Benchmark_Type(models.Model): - name = models.CharField(max_length=300) - version = models.CharField(max_length=15) - source = (("PCI", "PCI"), - ("OWASP ASVS", "OWASP ASVS"), - ("OWASP Mobile ASVS", "OWASP Mobile ASVS")) - benchmark_source = models.CharField(max_length=20, blank=False, - null=True, choices=source, - default="OWASP ASVS") - created = models.DateTimeField(auto_now_add=True, null=False) - updated = models.DateTimeField(auto_now=True) - enabled = models.BooleanField(default=True) - - def __str__(self): - return self.name + " " + self.version - - -class Benchmark_Category(models.Model): - type = models.ForeignKey(Benchmark_Type, verbose_name=_("Benchmark Type"), on_delete=models.CASCADE) - name = models.CharField(max_length=300) - objective = models.TextField() - references = models.TextField(blank=True, null=True) - enabled = models.BooleanField(default=True) - created = models.DateTimeField(auto_now_add=True, null=False) - updated = models.DateTimeField(auto_now=True) - - class Meta: - ordering = ("name",) - - def __str__(self): - return self.name + ": " + self.type.name - - -class Benchmark_Requirement(models.Model): - category = models.ForeignKey(Benchmark_Category, on_delete=models.CASCADE) - objective_number = models.CharField(max_length=15, null=True, blank=True) - objective = models.TextField() - references = models.TextField(blank=True, null=True) - level_1 = models.BooleanField(default=False) - level_2 = models.BooleanField(default=False) - level_3 = models.BooleanField(default=False) - enabled = models.BooleanField(default=True) - cwe_mapping = models.ManyToManyField(CWE, blank=True) - testing_guide = models.ManyToManyField(Testing_Guide, blank=True) - created = models.DateTimeField(auto_now_add=True, null=False) - updated = models.DateTimeField(auto_now=True) - - def __str__(self): - return str(self.objective_number) + ": " + self.category.name - - -class Benchmark_Product(models.Model): - product = models.ForeignKey(Product, on_delete=models.CASCADE) - control = models.ForeignKey(Benchmark_Requirement, on_delete=models.CASCADE) - pass_fail = models.BooleanField(default=False, verbose_name=_("Pass"), - help_text=_("Does the product meet the requirement?")) - enabled = models.BooleanField(default=True, - help_text=_("Applicable for this specific product.")) - notes = models.ManyToManyField(Notes, blank=True, editable=False) - created = models.DateTimeField(auto_now_add=True, null=False) - updated = models.DateTimeField(auto_now=True) - - class Meta: - unique_together = [("product", "control")] - - def __str__(self): - return self.product.name + ": " + self.control.objective_number + ": " + self.control.category.name - - -class Benchmark_Product_Summary(models.Model): - product = models.ForeignKey(Product, on_delete=models.CASCADE) - benchmark_type = models.ForeignKey(Benchmark_Type, on_delete=models.CASCADE) - asvs_level = (("Level 1", "Level 1"), - ("Level 2", "Level 2"), - ("Level 3", "Level 3")) - desired_level = models.CharField(max_length=15, - null=False, choices=asvs_level, - default="Level 1") - current_level = models.CharField(max_length=15, blank=True, - null=True, choices=asvs_level, - default="None") - asvs_level_1_benchmark = models.IntegerField(null=False, default=0, help_text=_("Total number of active benchmarks for this application.")) - asvs_level_1_score = models.IntegerField(null=False, default=0, help_text=_("ASVS Level 1 Score")) - asvs_level_2_benchmark = models.IntegerField(null=False, default=0, help_text=_("Total number of active benchmarks for this application.")) - asvs_level_2_score = models.IntegerField(null=False, default=0, help_text=_("ASVS Level 2 Score")) - asvs_level_3_benchmark = models.IntegerField(null=False, default=0, help_text=_("Total number of active benchmarks for this application.")) - asvs_level_3_score = models.IntegerField(null=False, default=0, help_text=_("ASVS Level 3 Score")) - publish = models.BooleanField(default=False, help_text=_("Publish score to Product.")) - created = models.DateTimeField(auto_now_add=True, null=False) - updated = models.DateTimeField(auto_now=True) - - class Meta: - unique_together = [("product", "benchmark_type")] - - def __str__(self): - return self.product.name + ": " + self.benchmark_type.name - - -# ========================== -# Defect Dojo Engaegment Surveys -# ============================== -with warnings.catch_warnings(action="ignore", category=ManagerInheritanceWarning): - class Question(PolymorphicModel, TimeStampedModel): - - """Represents a question.""" - - class Meta: - ordering = ["order"] - - order = models.PositiveIntegerField(default=1, - help_text=_("The render order")) - - optional = models.BooleanField( - default=False, - help_text=_("If selected, user doesn't have to answer this question")) - - text = models.TextField(blank=False, help_text=_("The question text"), default="") - objects = models.Manager() - polymorphic = PolymorphicManager() - - def __str__(self): - return self.text - - -class TextQuestion(Question): - - """Question with a text answer""" - - objects = PolymorphicManager() - - def get_form(self): - """Returns the form for this model""" - from .forms import TextQuestionForm # noqa: PLC0415 - return TextQuestionForm - - -class Choice(TimeStampedModel): - - """Model to store the choices for multi choice questions""" - - order = models.PositiveIntegerField(default=1) - - label = models.TextField(default="") - - class Meta: - ordering = ["order"] - - def __str__(self): - return self.label - - -class ChoiceQuestion(Question): - - """ - Question with answers that are chosen from a list of choices defined - by the user. - """ - - multichoice = models.BooleanField(default=False, - help_text=_("Select one or more")) - choices = models.ManyToManyField(Choice) - objects = PolymorphicManager() - - def get_form(self): - """Returns the form for this model""" - from .forms import ChoiceQuestionForm # noqa: PLC0415 - return ChoiceQuestionForm - - -# meant to be a abstract survey, identified by name for purpose -class Engagement_Survey(models.Model): - name = models.CharField(max_length=200, null=False, blank=False, - editable=True, default="") - description = models.TextField(editable=True, default="") - questions = models.ManyToManyField(Question) - active = models.BooleanField(default=True) - - class Meta: - verbose_name = _("Engagement Survey") - verbose_name_plural = "Engagement Surveys" - ordering = ("-active", "name") - - def __str__(self): - return self.name - - -# meant to be an answered survey tied to an engagement - -class Answered_Survey(models.Model): - # tie this to a specific engagement - engagement = models.ForeignKey(Engagement, related_name="engagement+", - null=True, blank=False, editable=True, - on_delete=models.CASCADE) - # what surveys have been answered - survey = models.ForeignKey(Engagement_Survey, on_delete=models.CASCADE) - assignee = models.ForeignKey(Dojo_User, related_name="assignee", - null=True, blank=True, editable=True, - default=None, on_delete=models.RESTRICT) - # who answered it - responder = models.ForeignKey(Dojo_User, related_name="responder", - null=True, blank=True, editable=True, - default=None, on_delete=models.RESTRICT) - completed = models.BooleanField(default=False) - answered_on = models.DateField(null=True) - - class Meta: - verbose_name = _("Answered Engagement Survey") - verbose_name_plural = _("Answered Engagement Surveys") - - def __str__(self): - return self.survey.name - - -def default_expiration(): - return timezone.now() + timedelta(days=7) - - -class General_Survey(models.Model): - survey = models.ForeignKey(Engagement_Survey, on_delete=models.CASCADE) - num_responses = models.IntegerField(default=0) - generated = models.DateTimeField(auto_now_add=True, null=True) - expiration = models.DateTimeField(default=default_expiration) - - class Meta: - verbose_name = _("General Engagement Survey") - verbose_name_plural = _("General Engagement Surveys") - - def __str__(self): - return self.survey.name - - def clean(self): - if self.expiration and timezone.is_naive(self.expiration): - self.expiration = timezone.make_aware(self.expiration) - - -with warnings.catch_warnings(action="ignore", category=ManagerInheritanceWarning): - class Answer(PolymorphicModel, TimeStampedModel): - - """Base Answer model""" - - question = models.ForeignKey(Question, on_delete=models.CASCADE) - - answered_survey = models.ForeignKey(Answered_Survey, - null=False, - blank=False, - on_delete=models.CASCADE) - objects = models.Manager() - polymorphic = PolymorphicManager() - - -class TextAnswer(Answer): - answer = models.TextField( - blank=False, - help_text=_("The answer text"), - default="") - objects = PolymorphicManager() - - def __str__(self): - return self.answer - - -class ChoiceAnswer(Answer): - answer = models.ManyToManyField( - Choice, - help_text=_("The selected choices as the answer")) - objects = PolymorphicManager() - - def __str__(self): - if len(self.answer.all()): - return str(self.answer.all()[0]) - return "No Response" - +from dojo.benchmark.models import ( # noqa: E402, I001 -- re-export; backward compat + Benchmark_Category, # noqa: F401 + Benchmark_Product, # noqa: F401 + Benchmark_Product_Summary, # noqa: F401 + Benchmark_Requirement, # noqa: F401 + Benchmark_Type, # noqa: F401 +) +from dojo.survey.models import ( # noqa: E402 -- re-export; backward compat + Answer, # noqa: F401 + Answered_Survey, # noqa: F401 + Choice, # noqa: F401 + ChoiceAnswer, # noqa: F401 + ChoiceQuestion, # noqa: F401 + Engagement_Survey, # noqa: F401 + General_Survey, # noqa: F401 + Question, # noqa: F401 + TextAnswer, # noqa: F401 + TextQuestion, # noqa: F401 + default_expiration, # noqa: F401 +) # Audit logging registration is now handled in auditlog.py and configured in apps.py # This allows for conditional registration of either django-auditlog or django-pghistory # The audit system is configured in DojoAppConfig.ready() to ensure all models are loaded -from dojo.utils import ( # noqa: E402 # there is issue due to a circular import - parse_cvss_data, +from dojo.utils import ( # noqa: E402 + parse_cvss_data, # noqa: F401 -- backward compat re-export; side-effect loads dojo.utils → dojo.location models ) tagulous.admin.register(Product.tags) @@ -4396,53 +593,25 @@ def __str__(self): tagulous.admin.register(Finding.inherited_tags) tagulous.admin.register(Engagement.tags) tagulous.admin.register(Engagement.inherited_tags) -tagulous.admin.register(Endpoint.tags) -tagulous.admin.register(Endpoint.inherited_tags) tagulous.admin.register(Finding_Template.tags) tagulous.admin.register(App_Analysis.tags) -tagulous.admin.register(Objects_Product.tags) - -# Benchmarks -admin.site.register(Benchmark_Type) -admin.site.register(Benchmark_Requirement) -admin.site.register(Benchmark_Category) -admin.site.register(Benchmark_Product) -admin.site.register(Benchmark_Product_Summary) +# Objects_Product.tags registered in dojo/object/admin.py # Testing admin.site.register(Testing_Guide_Category) admin.site.register(Testing_Guide) -admin.site.register(Engagement_Presets) admin.site.register(Network_Locations) -admin.site.register(Objects_Product) -admin.site.register(Objects_Review) +# Objects_Product + Objects_Review admin registered in dojo/object/admin.py admin.site.register(Languages) admin.site.register(Language_Type) admin.site.register(App_Analysis) -admin.site.register(Test) -admin.site.register(Finding, FindingAdmin) -admin.site.register(FileUpload) -admin.site.register(FileAccessToken) -admin.site.register(Engagement) -admin.site.register(Risk_Acceptance) +# FileUpload + FileAccessToken admin registered in dojo/file_uploads/admin.py admin.site.register(Check_List) -admin.site.register(Test_Type) -admin.site.register(Endpoint_Params) -admin.site.register(Endpoint_Status) -admin.site.register(Endpoint) -admin.site.register(Product) -admin.site.register(Product_Type) -admin.site.register(UserContactInfo) -admin.site.register(Notes) -admin.site.register(Note_Type) -admin.site.register(Tool_Configuration, Tool_Configuration_Admin) -admin.site.register(Tool_Product_Settings) -admin.site.register(Tool_Type) -admin.site.register(System_Settings) +# Notes + NoteHistory admin registered in dojo/notes/admin.py +# Note_Type admin registered in dojo/note_type/admin.py admin.site.register(SLA_Configuration) -admin.site.register(CWE) -admin.site.register(Regulation) +# Regulation admin registered in dojo/regulations/admin.py from dojo.authorization.models import ( # noqa: E402 Dojo_Group, Dojo_Group_Member, @@ -4468,21 +637,9 @@ def __str__(self): admin.site.register(Product_Type_Member) admin.site.register(Product_Type_Group) -admin.site.register(Contact) -admin.site.register(NoteHistory) -admin.site.register(Product_Line) -admin.site.register(Report_Type) +# NoteHistory admin registered in dojo/notes/admin.py +# Report_Type admin registered in dojo/reports/admin.py admin.site.register(DojoMeta) -admin.site.register(Product_API_Scan_Configuration) -admin.site.register(Development_Environment) -admin.site.register(Finding_Template) -admin.site.register(Vulnerability_Id) -admin.site.register(BurpRawRequestResponse) -admin.site.register(Announcement) -admin.site.register(UserAnnouncement) -admin.site.register(BannerConf) -admin.site.register(Tool_Product_History) -admin.site.register(General_Survey) -admin.site.register(Test_Import) -admin.site.register(Test_Import_Finding_Action) -admin.site.register(Finding_Group) +# Development_Environment admin registered in dojo/development_environment/admin.py +# Announcement + UserAnnouncement admin registered in dojo/announcement/admin.py +# BannerConf admin registered in dojo/banner/admin.py diff --git a/dojo/note_type/__init__.py b/dojo/note_type/__init__.py index e69de29bb2d..2c7095d7b93 100644 --- a/dojo/note_type/__init__.py +++ b/dojo/note_type/__init__.py @@ -0,0 +1 @@ +import dojo.note_type.admin # noqa: F401 diff --git a/dojo/note_type/admin.py b/dojo/note_type/admin.py new file mode 100644 index 00000000000..6b1baab68bc --- /dev/null +++ b/dojo/note_type/admin.py @@ -0,0 +1,5 @@ +from django.contrib import admin + +from dojo.note_type.models import Note_Type + +admin.site.register(Note_Type) diff --git a/dojo/note_type/api/__init__.py b/dojo/note_type/api/__init__.py new file mode 100644 index 00000000000..e39da828cac --- /dev/null +++ b/dojo/note_type/api/__init__.py @@ -0,0 +1 @@ +path = "note_type" # noqa: RUF067 diff --git a/dojo/note_type/api/serializer.py b/dojo/note_type/api/serializer.py new file mode 100644 index 00000000000..459773cc38d --- /dev/null +++ b/dojo/note_type/api/serializer.py @@ -0,0 +1,9 @@ +from rest_framework import serializers + +from dojo.note_type.models import Note_Type + + +class NoteTypeSerializer(serializers.ModelSerializer): + class Meta: + model = Note_Type + fields = "__all__" diff --git a/dojo/note_type/api/urls.py b/dojo/note_type/api/urls.py new file mode 100644 index 00000000000..f7c5a878568 --- /dev/null +++ b/dojo/note_type/api/urls.py @@ -0,0 +1,7 @@ +from dojo.note_type.api import path +from dojo.note_type.api.views import NoteTypeViewSet + + +def add_note_type_urls(router): + router.register(path, NoteTypeViewSet, basename="note_type") + return router diff --git a/dojo/note_type/api/views.py b/dojo/note_type/api/views.py new file mode 100644 index 00000000000..cde7c4d862f --- /dev/null +++ b/dojo/note_type/api/views.py @@ -0,0 +1,27 @@ +from django_filters.rest_framework import DjangoFilterBackend + +from dojo.api_v2.views import DojoModelViewSet +from dojo.authorization import api_permissions as permissions +from dojo.note_type.api.serializer import NoteTypeSerializer +from dojo.note_type.models import Note_Type + + +# Authorization: configuration +class NoteTypeViewSet( + DojoModelViewSet, +): + serializer_class = NoteTypeSerializer + queryset = Note_Type.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = [ + "id", + "name", + "description", + "is_single", + "is_active", + "is_mandatory", + ] + permission_classes = (permissions.UserHasConfigurationPermissionSuperuser,) + + def get_queryset(self): + return Note_Type.objects.all().order_by("id") diff --git a/dojo/note_type/models.py b/dojo/note_type/models.py new file mode 100644 index 00000000000..6ba489a8ca4 --- /dev/null +++ b/dojo/note_type/models.py @@ -0,0 +1,12 @@ +from django.db import models + + +class Note_Type(models.Model): + name = models.CharField(max_length=100, unique=True) + description = models.CharField(max_length=200) + is_single = models.BooleanField(default=False, null=False) + is_active = models.BooleanField(default=True, null=False) + is_mandatory = models.BooleanField(default=True, null=False) + + def __str__(self): + return self.name diff --git a/dojo/note_type/ui/__init__.py b/dojo/note_type/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/note_type/ui/forms.py b/dojo/note_type/ui/forms.py new file mode 100644 index 00000000000..8cbbef09020 --- /dev/null +++ b/dojo/note_type/ui/forms.py @@ -0,0 +1,35 @@ +from django import forms + +from dojo.note_type.models import Note_Type + + +class NoteTypeForm(forms.ModelForm): + description = forms.CharField(widget=forms.Textarea(attrs={}), + required=True) + + class Meta: + model = Note_Type + fields = ["name", "description", "is_single", "is_mandatory"] + + +class EditNoteTypeForm(NoteTypeForm): + + def __init__(self, *args, **kwargs): + is_single = kwargs.pop("is_single") + super().__init__(*args, **kwargs) + if is_single is False: + self.fields["is_single"].widget = forms.HiddenInput() + + +class DisableOrEnableNoteTypeForm(NoteTypeForm): + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.fields["name"].disabled = True + self.fields["description"].disabled = True + self.fields["is_single"].disabled = True + self.fields["is_mandatory"].disabled = True + self.fields["is_active"].disabled = True + + class Meta: + model = Note_Type + fields = "__all__" diff --git a/dojo/note_type/urls.py b/dojo/note_type/ui/urls.py similarity index 93% rename from dojo/note_type/urls.py rename to dojo/note_type/ui/urls.py index 76e3c3a6a2c..4f422a5d502 100644 --- a/dojo/note_type/urls.py +++ b/dojo/note_type/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from dojo.note_type import views +from dojo.note_type.ui import views urlpatterns = [ re_path(r"^note_type$", diff --git a/dojo/note_type/views.py b/dojo/note_type/ui/views.py similarity index 96% rename from dojo/note_type/views.py rename to dojo/note_type/ui/views.py index 65c908c740e..27aa93f6958 100644 --- a/dojo/note_type/views.py +++ b/dojo/note_type/ui/views.py @@ -6,8 +6,8 @@ from django.urls import reverse from dojo.filters import NoteTypesFilter -from dojo.forms import DisableOrEnableNoteTypeForm, EditNoteTypeForm, NoteTypeForm -from dojo.models import Note_Type +from dojo.note_type.models import Note_Type +from dojo.note_type.ui.forms import DisableOrEnableNoteTypeForm, EditNoteTypeForm, NoteTypeForm from dojo.utils import add_breadcrumb, get_page_items logger = logging.getLogger(__name__) diff --git a/dojo/notes/__init__.py b/dojo/notes/__init__.py index e69de29bb2d..6871614a351 100644 --- a/dojo/notes/__init__.py +++ b/dojo/notes/__init__.py @@ -0,0 +1 @@ +import dojo.notes.admin # noqa: F401 diff --git a/dojo/notes/admin.py b/dojo/notes/admin.py new file mode 100644 index 00000000000..2c3ccf06f9c --- /dev/null +++ b/dojo/notes/admin.py @@ -0,0 +1,6 @@ +from django.contrib import admin + +from dojo.notes.models import NoteHistory, Notes + +admin.site.register(Notes) +admin.site.register(NoteHistory) diff --git a/dojo/notes/api/__init__.py b/dojo/notes/api/__init__.py new file mode 100644 index 00000000000..c042966fa4f --- /dev/null +++ b/dojo/notes/api/__init__.py @@ -0,0 +1 @@ +path = "notes" # noqa: RUF067 diff --git a/dojo/notes/api/serializer.py b/dojo/notes/api/serializer.py new file mode 100644 index 00000000000..e4bd3f83e5d --- /dev/null +++ b/dojo/notes/api/serializer.py @@ -0,0 +1,41 @@ +from django.utils import timezone +from rest_framework import serializers + +from dojo.note_type.api.serializer import NoteTypeSerializer +from dojo.notes.models import NoteHistory, Notes +from dojo.user.api.serializer import UserStubSerializer + + +class NoteHistorySerializer(serializers.ModelSerializer): + current_editor = UserStubSerializer(read_only=True) + note_type = NoteTypeSerializer(read_only=True, many=False) + + class Meta: + model = NoteHistory + fields = "__all__" + + +class NoteSerializer(serializers.ModelSerializer): + author = UserStubSerializer(many=False, read_only=True) + editor = UserStubSerializer(read_only=True, many=False, allow_null=True) + history = NoteHistorySerializer(read_only=True, many=True) + note_type = NoteTypeSerializer(read_only=True, many=False) + + def update(self, instance, validated_data): + instance.entry = validated_data.get("entry") + instance.edited = True + instance.editor = self.context["request"].user + instance.edit_time = timezone.now() + history = NoteHistory( + data=instance.entry, + time=instance.edit_time, + current_editor=instance.editor, + ) + history.save() + instance.history.add(history) + instance.save() + return instance + + class Meta: + model = Notes + fields = "__all__" diff --git a/dojo/notes/api/urls.py b/dojo/notes/api/urls.py new file mode 100644 index 00000000000..2d7d582551a --- /dev/null +++ b/dojo/notes/api/urls.py @@ -0,0 +1,7 @@ +from dojo.notes.api import path +from dojo.notes.api.views import NotesViewSet + + +def add_notes_urls(router): + router.register(path, NotesViewSet, basename="notes") + return router diff --git a/dojo/notes/api/views.py b/dojo/notes/api/views.py new file mode 100644 index 00000000000..29fe0e74030 --- /dev/null +++ b/dojo/notes/api/views.py @@ -0,0 +1,31 @@ +from django_filters.rest_framework import DjangoFilterBackend +from rest_framework import mixins, viewsets +from rest_framework.permissions import DjangoModelPermissions + +from dojo.authorization import api_permissions as permissions +from dojo.notes.api.serializer import NoteSerializer +from dojo.notes.models import Notes + + +# Authorization: superuser +class NotesViewSet( + mixins.UpdateModelMixin, + viewsets.ReadOnlyModelViewSet, +): + serializer_class = NoteSerializer + queryset = Notes.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = [ + "id", + "entry", + "author", + "private", + "date", + "edited", + "edit_time", + "editor", + ] + permission_classes = (permissions.IsSuperUser, DjangoModelPermissions) + + def get_queryset(self): + return Notes.objects.all().order_by("id") diff --git a/dojo/notes/models.py b/dojo/notes/models.py new file mode 100644 index 00000000000..26df9ab7a7a --- /dev/null +++ b/dojo/notes/models.py @@ -0,0 +1,49 @@ +from django.db import models + +from dojo.models import copy_model_util, get_current_datetime + + +class NoteHistory(models.Model): + note_type = models.ForeignKey("dojo.Note_Type", null=True, blank=True, on_delete=models.CASCADE) + data = models.TextField() + time = models.DateTimeField(null=True, editable=False, + default=get_current_datetime) + current_editor = models.ForeignKey("dojo.Dojo_User", editable=False, null=True, on_delete=models.CASCADE) + + def copy(self): + copy = copy_model_util(self) + copy.save() + return copy + + +class Notes(models.Model): + note_type = models.ForeignKey("dojo.Note_Type", related_name="note_type", null=True, blank=True, on_delete=models.CASCADE) + entry = models.TextField() + date = models.DateTimeField(null=False, editable=False, + default=get_current_datetime) + author = models.ForeignKey("dojo.Dojo_User", related_name="editor_notes_set", editable=False, on_delete=models.CASCADE) + private = models.BooleanField(default=False) + edited = models.BooleanField(default=False) + editor = models.ForeignKey("dojo.Dojo_User", related_name="author_notes_set", editable=False, null=True, on_delete=models.CASCADE) + edit_time = models.DateTimeField(null=True, editable=False, + default=get_current_datetime) + history = models.ManyToManyField("dojo.NoteHistory", blank=True, + editable=False) + + class Meta: + ordering = ["-date"] + + def __str__(self): + return self.entry + + def copy(self): + copy = copy_model_util(self) + # Save the necessary ManyToMany relationships + old_history = list(self.history.all()) + # Save the object before setting any ManyToMany relationships + copy.save() + # Copy the history + for history in old_history: + copy.history.add(history.copy()) + + return copy diff --git a/dojo/notes/ui/__init__.py b/dojo/notes/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/notes/ui/forms.py b/dojo/notes/ui/forms.py new file mode 100644 index 00000000000..85e9423bb26 --- /dev/null +++ b/dojo/notes/ui/forms.py @@ -0,0 +1,39 @@ +from django import forms + +from dojo.notes.models import Notes +from dojo.utils import get_system_setting + + +class NoteForm(forms.ModelForm): + entry = forms.CharField(max_length=2400, widget=forms.Textarea(attrs={"rows": 4, "cols": 15}), + label="Notes:") + + class Meta: + model = Notes + fields = ["entry", "private"] + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + if disclaimer := get_system_setting("disclaimer_notes"): + self.disclaimer = disclaimer.strip() + + +class TypedNoteForm(NoteForm): + + def __init__(self, *args, **kwargs): + queryset = kwargs.pop("available_note_types") + super().__init__(*args, **kwargs) + self.fields["note_type"] = forms.ModelChoiceField(queryset=queryset, label="Note Type", required=True) + + class Meta: + model = Notes + fields = ["note_type", "entry", "private"] + + +class DeleteNoteForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Notes + fields = ["id"] diff --git a/dojo/notes/urls.py b/dojo/notes/ui/urls.py similarity index 92% rename from dojo/notes/urls.py rename to dojo/notes/ui/urls.py index 00a9f17a83a..cf95618e5eb 100644 --- a/dojo/notes/urls.py +++ b/dojo/notes/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from . import views +from dojo.notes.ui import views urlpatterns = [ re_path(r"^notes/(?P\d+)/delete/(?P[\w-]+)/(?P\d+)$", views.delete_note, name="delete_note"), diff --git a/dojo/notes/views.py b/dojo/notes/ui/views.py similarity index 96% rename from dojo/notes/views.py rename to dojo/notes/ui/views.py index 66c4d0aecda..4b4f7c27457 100644 --- a/dojo/notes/views.py +++ b/dojo/notes/ui/views.py @@ -16,8 +16,10 @@ from dojo.finding.queries import get_authorized_findings # Local application/library imports -from dojo.forms import DeleteNoteForm, NoteForm, TypedNoteForm -from dojo.models import Engagement, Finding, Note_Type, NoteHistory, Notes, Test +from dojo.models import Engagement, Finding, Test +from dojo.note_type.models import Note_Type +from dojo.notes.models import NoteHistory, Notes +from dojo.notes.ui.forms import DeleteNoteForm, NoteForm, TypedNoteForm from dojo.test.queries import get_authorized_tests logger = logging.getLogger(__name__) diff --git a/dojo/object/__init__.py b/dojo/object/__init__.py index e69de29bb2d..2a0769e5c0e 100644 --- a/dojo/object/__init__.py +++ b/dojo/object/__init__.py @@ -0,0 +1 @@ +import dojo.object.admin # noqa: F401 diff --git a/dojo/object/admin.py b/dojo/object/admin.py new file mode 100644 index 00000000000..5782d037789 --- /dev/null +++ b/dojo/object/admin.py @@ -0,0 +1,8 @@ +import tagulous.admin +from django.contrib import admin + +from dojo.object.models import Objects_Product, Objects_Review + +admin.site.register(Objects_Product) +admin.site.register(Objects_Review) +tagulous.admin.register(Objects_Product.tags) diff --git a/dojo/object/models.py b/dojo/object/models.py new file mode 100644 index 00000000000..37f99568d40 --- /dev/null +++ b/dojo/object/models.py @@ -0,0 +1,37 @@ +from django.db import models +from django.utils.translation import gettext as _ +from tagulous.models import TagField + + +class Objects_Review(models.Model): + name = models.CharField(max_length=100, null=True, blank=True) + created = models.DateTimeField(auto_now_add=True, null=False) + + def __str__(self): + return self.name + + +class Objects_Product(models.Model): + product = models.ForeignKey("dojo.Product", on_delete=models.CASCADE) + name = models.CharField(max_length=100, null=True, blank=True) + path = models.CharField(max_length=600, verbose_name=_("Full file path"), + null=True, blank=True) + folder = models.CharField(max_length=400, verbose_name=_("Folder"), + null=True, blank=True) + artifact = models.CharField(max_length=400, verbose_name=_("Artifact"), + null=True, blank=True) + review_status = models.ForeignKey("dojo.Objects_Review", on_delete=models.CASCADE) + created = models.DateTimeField(auto_now_add=True, null=False) + + tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this object. Choose from the list or add new tags. Press Enter key to add.")) + + def __str__(self): + name = None + if self.path is not None: + name = self.path + elif self.folder is not None: + name = self.folder + elif self.artifact is not None: + name = self.artifact + + return name diff --git a/dojo/object/ui/__init__.py b/dojo/object/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/object/ui/forms.py b/dojo/object/ui/forms.py new file mode 100644 index 00000000000..1c94c6bd9e8 --- /dev/null +++ b/dojo/object/ui/forms.py @@ -0,0 +1,26 @@ +from django import forms + +from dojo.object.models import Objects_Product + + +class DeleteObjectsSettingsForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Objects_Product + fields = ["id"] + + +class ObjectSettingsForm(forms.ModelForm): + + class Meta: + model = Objects_Product + fields = ["path", "folder", "artifact", "name", "review_status", "tags"] + exclude = ["product"] + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + + def clean(self): + return self.cleaned_data diff --git a/dojo/object/urls.py b/dojo/object/ui/urls.py similarity index 93% rename from dojo/object/urls.py rename to dojo/object/ui/urls.py index b31e9350648..9aa1d7cbbdc 100644 --- a/dojo/object/urls.py +++ b/dojo/object/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from . import views +from dojo.object.ui import views urlpatterns = [ re_path(r"^product/(?P\d+)/object/add$", views.new_object, name="new_object"), diff --git a/dojo/object/views.py b/dojo/object/ui/views.py similarity index 88% rename from dojo/object/views.py rename to dojo/object/ui/views.py index 40cc57a45b2..c0f9342c6c1 100644 --- a/dojo/object/views.py +++ b/dojo/object/ui/views.py @@ -6,9 +6,9 @@ from django.shortcuts import get_object_or_404, render from django.urls import reverse -from dojo.forms import DeleteObjectsSettingsForm, ObjectSettingsForm from dojo.labels import get_labels -from dojo.models import Objects_Product, Product +from dojo.object.models import Objects_Product +from dojo.object.ui.forms import DeleteObjectsSettingsForm, ObjectSettingsForm from dojo.utils import Product_Tab logger = logging.getLogger(__name__) @@ -18,6 +18,7 @@ def new_object(request, pid): page_name = labels.ASSET_TRACKED_FILES_ADD_LABEL + from dojo.models import Product # noqa: PLC0415 -- lazy import, avoids circular dependency prod = get_object_or_404(Product, id=pid) if request.method == "POST": tform = ObjectSettingsForm(request.POST) @@ -44,6 +45,7 @@ def new_object(request, pid): def view_objects(request, pid): + from dojo.models import Product # noqa: PLC0415 -- lazy import, avoids circular dependency product = get_object_or_404(Product, id=pid) object_queryset = Objects_Product.objects.filter(product=pid).order_by("path", "folder", "artifact") @@ -59,6 +61,7 @@ def view_objects(request, pid): def edit_object(request, pid, ttid): object_prod = get_object_or_404(Objects_Product, pk=ttid) + from dojo.models import Product # noqa: PLC0415 -- lazy import, avoids circular dependency product = get_object_or_404(Product, id=pid) if object_prod.product != product: raise PermissionDenied @@ -87,6 +90,7 @@ def edit_object(request, pid, ttid): def delete_object(request, pid, ttid): object_prod = get_object_or_404(Objects_Product, pk=ttid) + from dojo.models import Product # noqa: PLC0415 -- lazy import, avoids circular dependency product = get_object_or_404(Product, id=pid) if object_prod.product != product: raise PermissionDenied diff --git a/dojo/organization/urls.py b/dojo/organization/urls.py index 0555a654b20..e3801572bd8 100644 --- a/dojo/organization/urls.py +++ b/dojo/organization/urls.py @@ -1,8 +1,8 @@ from django.conf import settings from django.urls import re_path -from dojo.product import views as product_views -from dojo.product_type import views +from dojo.product.ui import views as product_views +from dojo.product_type.ui import views from dojo.utils import redirect_view # TODO: remove the else: branch once v3 migration is complete diff --git a/dojo/product/__init__.py b/dojo/product/__init__.py index e69de29bb2d..df5e047d856 100644 --- a/dojo/product/__init__.py +++ b/dojo/product/__init__.py @@ -0,0 +1 @@ +import dojo.product.admin # noqa: F401 diff --git a/dojo/product/admin.py b/dojo/product/admin.py new file mode 100644 index 00000000000..e6a32855567 --- /dev/null +++ b/dojo/product/admin.py @@ -0,0 +1,21 @@ +from django.contrib import admin + +from dojo.product.models import Product, Product_API_Scan_Configuration, Product_Line + + +@admin.register(Product_Line) +class ProductLineAdmin(admin.ModelAdmin): + + """Admin support for the Product_Line model.""" + + +@admin.register(Product) +class ProductAdmin(admin.ModelAdmin): + + """Admin support for the Product model.""" + + +@admin.register(Product_API_Scan_Configuration) +class ProductAPIScanConfigurationAdmin(admin.ModelAdmin): + + """Admin support for the Product_API_Scan_Configuration model.""" diff --git a/dojo/product/api/__init__.py b/dojo/product/api/__init__.py new file mode 100644 index 00000000000..36b10db0174 --- /dev/null +++ b/dojo/product/api/__init__.py @@ -0,0 +1 @@ +path = "products" # noqa: RUF067 diff --git a/dojo/product/api/filters.py b/dojo/product/api/filters.py new file mode 100644 index 00000000000..e75b95684ad --- /dev/null +++ b/dojo/product/api/filters.py @@ -0,0 +1,97 @@ +from django_filters import ( + BooleanFilter, + CharFilter, + MultipleChoiceFilter, + NumberFilter, + OrderingFilter, +) +from drf_spectacular.types import OpenApiTypes +from drf_spectacular.utils import extend_schema_field + +from dojo.filters import ( + CharFieldFilterANDExpression, + CharFieldInFilter, + DateRangeFilter, + DojoFilter, + NumberInFilter, + ProductSLAFilter, +) +from dojo.labels import get_labels +from dojo.models import Product + +labels = get_labels() + + +class ApiProductFilter(DojoFilter): + # BooleanFilter + external_audience = BooleanFilter(field_name="external_audience") + internet_accessible = BooleanFilter(field_name="internet_accessible") + # CharFilter + name = CharFilter(lookup_expr="icontains") + name_exact = CharFilter(field_name="name", lookup_expr="iexact") + description = CharFilter(lookup_expr="icontains") + business_criticality = MultipleChoiceFilter(choices=Product.BUSINESS_CRITICALITY_CHOICES) + platform = MultipleChoiceFilter(choices=Product.PLATFORM_CHOICES) + lifecycle = MultipleChoiceFilter(choices=Product.LIFECYCLE_CHOICES) + origin = MultipleChoiceFilter(choices=Product.ORIGIN_CHOICES) + # NumberInFilter + id = NumberInFilter(field_name="id", lookup_expr="in") + product_manager = NumberInFilter(field_name="product_manager", lookup_expr="in") + technical_contact = NumberInFilter(field_name="technical_contact", lookup_expr="in") + team_manager = NumberInFilter(field_name="team_manager", lookup_expr="in") + prod_type = NumberInFilter(field_name="prod_type", lookup_expr="in") + tid = NumberInFilter(field_name="tid", lookup_expr="in") + prod_numeric_grade = NumberInFilter(field_name="prod_numeric_grade", lookup_expr="in") + user_records = NumberInFilter(field_name="user_records", lookup_expr="in") + regulations = NumberInFilter(field_name="regulations", lookup_expr="in") + + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag name contains") + tags = CharFieldInFilter( + field_name="tags__name", + lookup_expr="in", + help_text="Comma separated list of exact tags (uses OR for multiple values)") + tags__and = CharFieldFilterANDExpression( + field_name="tags__name", + help_text="Comma separated list of exact tags to match with an AND expression") + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Not Tag name contains", exclude="True") + not_tags = CharFieldInFilter(field_name="tags__name", lookup_expr="in", + help_text=labels.ASSET_FILTERS_CSV_TAGS_NOT_HELP, exclude="True") + has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") + outside_of_sla = extend_schema_field(OpenApiTypes.NUMBER)(ProductSLAFilter()) + + # DateRangeFilter + created = DateRangeFilter() + updated = DateRangeFilter() + # NumberFilter + revenue = NumberFilter() + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("id", "id"), + ("tid", "tid"), + ("name", "name"), + ("created", "created"), + ("prod_numeric_grade", "prod_numeric_grade"), + ("business_criticality", "business_criticality"), + ("platform", "platform"), + ("lifecycle", "lifecycle"), + ("origin", "origin"), + ("revenue", "revenue"), + ("external_audience", "external_audience"), + ("internet_accessible", "internet_accessible"), + ("product_manager", "product_manager"), + ("product_manager__first_name", "product_manager__first_name"), + ("product_manager__last_name", "product_manager__last_name"), + ("technical_contact", "technical_contact"), + ("technical_contact__first_name", "technical_contact__first_name"), + ("technical_contact__last_name", "technical_contact__last_name"), + ("team_manager", "team_manager"), + ("team_manager__first_name", "team_manager__first_name"), + ("team_manager__last_name", "team_manager__last_name"), + ("prod_type", "prod_type"), + ("prod_type__name", "prod_type__name"), + ("updated", "updated"), + ("user_records", "user_records"), + ), + ) diff --git a/dojo/product/api/serializer.py b/dojo/product/api/serializer.py new file mode 100644 index 00000000000..53b89033a28 --- /dev/null +++ b/dojo/product/api/serializer.py @@ -0,0 +1,60 @@ +from rest_framework import serializers + +from dojo.models import DojoMeta, Product, Product_API_Scan_Configuration + + +class ProductMetaSerializer(serializers.ModelSerializer): + class Meta: + model = DojoMeta + fields = ("name", "value") + + +class ProductAPIScanConfigurationSerializer(serializers.ModelSerializer): + class Meta: + model = Product_API_Scan_Configuration + fields = "__all__" + + +class ProductSerializer(serializers.ModelSerializer): + findings_count = serializers.SerializerMethodField() + findings_list = serializers.SerializerMethodField() + + business_criticality = serializers.ChoiceField(choices=Product.BUSINESS_CRITICALITY_CHOICES, allow_blank=True, allow_null=True, required=False) + platform = serializers.ChoiceField(choices=Product.PLATFORM_CHOICES, allow_blank=True, allow_null=True, required=False) + lifecycle = serializers.ChoiceField(choices=Product.LIFECYCLE_CHOICES, allow_blank=True, allow_null=True, required=False) + origin = serializers.ChoiceField(choices=Product.ORIGIN_CHOICES, allow_blank=True, allow_null=True, required=False) + + product_meta = ProductMetaSerializer(read_only=True, many=True) + + class Meta: + model = Product + exclude = ( + "tid", + "updated", + "async_updating", + ) + + def get_fields(self): + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + TagListSerializerField, + ) + fields = super().get_fields() + fields["tags"] = TagListSerializerField(required=False) + return fields + + def validate(self, data): + async_updating = getattr(self.instance, "async_updating", None) + if async_updating: + new_sla_config = data.get("sla_configuration", None) + old_sla_config = getattr(self.instance, "sla_configuration", None) + if new_sla_config and old_sla_config and new_sla_config != old_sla_config: + msg = "Finding SLA expiration dates are currently being recalculated. The SLA configuration for this product cannot be changed until the calculation is complete." + raise serializers.ValidationError(msg) + return data + + def get_findings_count(self, obj) -> int: + return obj.findings_count + + # TODO: maybe extend_schema_field is needed here? + def get_findings_list(self, obj) -> list[int]: + return obj.open_findings_list() diff --git a/dojo/product/api/urls.py b/dojo/product/api/urls.py new file mode 100644 index 00000000000..0e7e34974c0 --- /dev/null +++ b/dojo/product/api/urls.py @@ -0,0 +1,7 @@ +from dojo.product.api.views import ProductAPIScanConfigurationViewSet, ProductViewSet + + +def add_product_urls(router): + router.register("products", ProductViewSet, basename="product") + router.register("product_api_scan_configurations", ProductAPIScanConfigurationViewSet, basename="product_api_scan_configuration") + return router diff --git a/dojo/product/api/views.py b/dojo/product/api/views.py new file mode 100644 index 00000000000..a9ebd2dca0e --- /dev/null +++ b/dojo/product/api/views.py @@ -0,0 +1,129 @@ +from django_filters.rest_framework import DjangoFilterBackend +from drf_spectacular.utils import extend_schema, extend_schema_view +from rest_framework import mixins, status, viewsets +from rest_framework.decorators import action +from rest_framework.permissions import IsAuthenticated +from rest_framework.response import Response + +import dojo.api_v2.mixins as dojo_mixins +from dojo.api_v2 import prefetch +from dojo.api_v2 import serializers as api_v2_serializers +from dojo.api_v2.views import PrefetchDojoModelViewSet, report_generate, schema_with_prefetch +from dojo.authorization import api_permissions as permissions +from dojo.models import Endpoint, Product, Product_API_Scan_Configuration +from dojo.product.api.filters import ApiProductFilter +from dojo.product.api.serializer import ( + ProductAPIScanConfigurationSerializer, + ProductSerializer, +) +from dojo.product.queries import ( + get_authorized_product_api_scan_configurations, + get_authorized_products, +) +from dojo.utils import async_delete, get_setting + + +# Authorization: object-based +class ProductAPIScanConfigurationViewSet( + PrefetchDojoModelViewSet, +): + serializer_class = ProductAPIScanConfigurationSerializer + queryset = Product_API_Scan_Configuration.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = [ + "id", + "product", + "tool_configuration", + "service_key_1", + "service_key_2", + "service_key_3", + ] + permission_classes = ( + IsAuthenticated, + permissions.UserHasProductAPIScanConfigurationPermission, + ) + + def get_queryset(self): + return get_authorized_product_api_scan_configurations( + "view", + ) + + +@extend_schema_view(**schema_with_prefetch()) +class ProductViewSet( + prefetch.PrefetchListMixin, + prefetch.PrefetchRetrieveMixin, + mixins.CreateModelMixin, + mixins.DestroyModelMixin, + mixins.UpdateModelMixin, + viewsets.GenericViewSet, + dojo_mixins.DeletePreviewModelMixin, +): + serializer_class = ProductSerializer + queryset = Product.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_class = ApiProductFilter + permission_classes = ( + IsAuthenticated, + permissions.UserHasProductPermission, + ) + + def get_queryset(self): + return get_authorized_products("view").distinct() + + def destroy(self, request, *args, **kwargs): + instance = self.get_object() + if get_setting("ASYNC_OBJECT_DELETE"): + async_del = async_delete() + async_del.delete(instance) + else: + with Endpoint.allow_endpoint_init(): # TODO: Delete this after the move to Locations + instance.delete() + return Response(status=status.HTTP_204_NO_CONTENT) + + # def list(self, request): + # # Note the use of `get_queryset()` instead of `self.queryset` + # queryset = self.get_queryset() + # serializer = self.serializer_class(queryset, many=True) + # return Response(serializer.data) + + @extend_schema( + request=api_v2_serializers.ReportGenerateOptionSerializer, + responses={status.HTTP_200_OK: api_v2_serializers.ReportGenerateSerializer}, + ) + @action( + detail=True, methods=["post"], + # IsAuthenticated only: report generation requires View permission, + # enforced by the permission-filtered get_queryset(). The viewset's + # permission_classes would check Edit (POST), which is too restrictive. + permission_classes=[IsAuthenticated], + ) + def generate_report(self, request, pk=None): + product = self.get_object() + + options = {} + # prepare post data + report_options = api_v2_serializers.ReportGenerateOptionSerializer( + data=request.data, + ) + if report_options.is_valid(): + options["include_finding_notes"] = report_options.validated_data[ + "include_finding_notes" + ] + options["include_finding_images"] = report_options.validated_data[ + "include_finding_images" + ] + options[ + "include_executive_summary" + ] = report_options.validated_data["include_executive_summary"] + options[ + "include_table_of_contents" + ] = report_options.validated_data["include_table_of_contents"] + else: + return Response( + report_options.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + data = report_generate(request, product, options) + report = api_v2_serializers.ReportGenerateSerializer(data) + return Response(report.data) diff --git a/dojo/product/models.py b/dojo/product/models.py new file mode 100644 index 00000000000..cd49e157ac4 --- /dev/null +++ b/dojo/product/models.py @@ -0,0 +1,303 @@ +from decimal import Decimal + +from django.core.validators import MinValueValidator +from django.db import models +from django.urls import reverse +from django.utils import timezone +from django.utils.functional import cached_property +from django.utils.translation import gettext as _ +from tagulous.models import TagField + +from dojo.base_models.base import BaseModel + + +class Product_Line(models.Model): + name = models.CharField(max_length=300) + description = models.CharField(max_length=2000) + + def __str__(self): + return self.name + + +class Product(BaseModel): + WEB_PLATFORM = "web" + IOT = "iot" + DESKTOP_PLATFORM = "desktop" + MOBILE_PLATFORM = "mobile" + WEB_SERVICE_PLATFORM = "web service" + PLATFORM_CHOICES = ( + (WEB_SERVICE_PLATFORM, _("API")), + (DESKTOP_PLATFORM, _("Desktop")), + (IOT, _("Internet of Things")), + (MOBILE_PLATFORM, _("Mobile")), + (WEB_PLATFORM, _("Web")), + ) + + CONSTRUCTION = "construction" + PRODUCTION = "production" + RETIREMENT = "retirement" + LIFECYCLE_CHOICES = ( + (CONSTRUCTION, _("Construction")), + (PRODUCTION, _("Production")), + (RETIREMENT, _("Retirement")), + ) + + THIRD_PARTY_LIBRARY_ORIGIN = "third party library" + PURCHASED_ORIGIN = "purchased" + CONTRACTOR_ORIGIN = "contractor" + INTERNALLY_DEVELOPED_ORIGIN = "internal" + OPEN_SOURCE_ORIGIN = "open source" + OUTSOURCED_ORIGIN = "outsourced" + ORIGIN_CHOICES = ( + (THIRD_PARTY_LIBRARY_ORIGIN, _("Third Party Library")), + (PURCHASED_ORIGIN, _("Purchased")), + (CONTRACTOR_ORIGIN, _("Contractor Developed")), + (INTERNALLY_DEVELOPED_ORIGIN, _("Internally Developed")), + (OPEN_SOURCE_ORIGIN, _("Open Source")), + (OUTSOURCED_ORIGIN, _("Outsourced")), + ) + + VERY_HIGH_CRITICALITY = "very high" + HIGH_CRITICALITY = "high" + MEDIUM_CRITICALITY = "medium" + LOW_CRITICALITY = "low" + VERY_LOW_CRITICALITY = "very low" + NONE_CRITICALITY = "none" + BUSINESS_CRITICALITY_CHOICES = ( + (VERY_HIGH_CRITICALITY, _("Very High")), + (HIGH_CRITICALITY, _("High")), + (MEDIUM_CRITICALITY, _("Medium")), + (LOW_CRITICALITY, _("Low")), + (VERY_LOW_CRITICALITY, _("Very Low")), + (NONE_CRITICALITY, _("None")), + ) + + name = models.CharField(max_length=255, unique=True) + description = models.CharField(max_length=4000) + + product_manager = models.ForeignKey("dojo.Dojo_User", null=True, blank=True, + related_name="product_manager", on_delete=models.RESTRICT) + technical_contact = models.ForeignKey("dojo.Dojo_User", null=True, blank=True, + related_name="technical_contact", on_delete=models.RESTRICT) + team_manager = models.ForeignKey("dojo.Dojo_User", null=True, blank=True, + related_name="team_manager", on_delete=models.RESTRICT) + + prod_type = models.ForeignKey("dojo.Product_Type", related_name="prod_type", + null=False, blank=False, on_delete=models.CASCADE) + sla_configuration = models.ForeignKey("dojo.SLA_Configuration", + related_name="sla_config", + null=False, + blank=False, + default=1, + on_delete=models.RESTRICT) + tid = models.IntegerField(default=0, editable=False) + authorized_users = models.ManyToManyField("dojo.Dojo_User", related_name="authorized_products", blank=True) + prod_numeric_grade = models.IntegerField(null=True, blank=True) + + # Metadata + business_criticality = models.CharField(max_length=9, choices=BUSINESS_CRITICALITY_CHOICES, blank=True, null=True) + platform = models.CharField(max_length=11, choices=PLATFORM_CHOICES, blank=True, null=True) + lifecycle = models.CharField(max_length=12, choices=LIFECYCLE_CHOICES, blank=True, null=True) + origin = models.CharField(max_length=19, choices=ORIGIN_CHOICES, blank=True, null=True) + user_records = models.PositiveIntegerField(blank=True, null=True, help_text=_("Estimate the number of user records within the application.")) + revenue = models.DecimalField(max_digits=15, decimal_places=2, blank=True, null=True, validators=[MinValueValidator(Decimal("0.00"))], help_text=_("Estimate the application's revenue.")) + external_audience = models.BooleanField(default=False, help_text=_("Specify if the application is used by people outside the organization.")) + internet_accessible = models.BooleanField(default=False, help_text=_("Specify if the application is accessible from the public internet.")) + regulations = models.ManyToManyField("dojo.Regulation", blank=True) + + tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this product. Choose from the list or add new tags. Press Enter key to add.")) + enable_product_tag_inheritance = models.BooleanField( + default=False, + blank=False, + verbose_name=_("Enable Product Tag Inheritance"), + help_text=_("Enables product tag inheritance. Any tags added on a product will automatically be added to all Engagements, Tests, and Findings")) + enable_simple_risk_acceptance = models.BooleanField(default=False, help_text=_("Allows simple risk acceptance by checking/unchecking a checkbox.")) + enable_full_risk_acceptance = models.BooleanField(default=True, help_text=_("Allows full risk acceptance using a risk acceptance form, expiration date, uploaded proof, etc.")) + + disable_sla_breach_notifications = models.BooleanField( + default=False, + blank=False, + verbose_name=_("Disable SLA breach notifications"), + help_text=_("Disable SLA breach notifications if configured in the global settings")) + async_updating = models.BooleanField(default=False, + help_text=_("Findings under this Product or SLA configuration are asynchronously being updated")) + + class Meta: + ordering = ("name",) + + def __str__(self): + return self.name + + def save(self, *args, **kwargs): + # get the product's sla config before saving (if this is an existing product) + initial_sla_config = None + if self.pk is not None: + initial_sla_config = getattr(Product.objects.get(pk=self.pk), "sla_configuration", None) + # if initial sla config exists and async finding update is already running, revert sla config before saving + if initial_sla_config and self.async_updating: + self.sla_configuration = initial_sla_config + + super().save(*args, **kwargs) + + # if the initial sla config exists and async finding update is not running + if initial_sla_config is not None and not self.async_updating: + # get the new sla config from the saved product + new_sla_config = getattr(self, "sla_configuration", None) + # if the sla config has changed, update finding sla expiration dates within this product + if new_sla_config and (initial_sla_config != new_sla_config): + # set the async updating flag to true for this product + self.async_updating = True + super().save(*args, **kwargs) + # set the async updating flag to true for the sla config assigned to this product + sla_config = getattr(self, "sla_configuration", None) + if sla_config: + from dojo.models import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + SLA_Configuration, + ) + sla_config.async_updating = True + super(SLA_Configuration, sla_config).save() + # launch the async task to update all finding sla expiration dates + from dojo.sla_config.helpers import async_update_sla_expiration_dates_sla_config_sync # noqa: I001, PLC0415 circular import + from dojo.celery_dispatch import dojo_dispatch_task # noqa: PLC0415 circular import + + dojo_dispatch_task( + async_update_sla_expiration_dates_sla_config_sync, + sla_config.id, + [self.id], + ) + # The async task refetches and resets async_updating on its own copies. + # Mirror that on this in-memory product and the in-memory sla_config so a + # subsequent save() on either does not trigger their lock-revert paths. + self.async_updating = False + if sla_config: + sla_config.async_updating = False + + def get_absolute_url(self): + return reverse("view_product", args=[str(self.id)]) + + @cached_property + def findings_count(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + try: + # if prefetched, it's already there + return self.active_finding_count + except AttributeError: + # ideally it's always prefetched and we can remove this code in the future + self.active_finding_count = Finding.objects.filter(active=True, + test__engagement__product=self).count() + return self.active_finding_count + + @cached_property + def findings_active_verified_count(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + try: + # if prefetched, it's already there + return self.active_verified_finding_count + except AttributeError: + # ideally it's always prefetched and we can remove this code in the future + self.active_verified_finding_count = Finding.objects.filter(active=True, + verified=True, + test__engagement__product=self).count() + return self.active_verified_finding_count + + # TODO: Delete this after the move to Locations + @cached_property + def endpoint_host_count(self): + # active_endpoints is (should be) prefetched + endpoints = getattr(self, "active_endpoints", None) + + hosts = [] + for e in endpoints: + if e.host in hosts: + continue + hosts.append(e.host) + + return len(hosts) + + # TODO: Delete this after the move to Locations + @cached_property + def endpoint_count(self): + # active_endpoints is (should be) prefetched + endpoints = getattr(self, "active_endpoints", None) + if endpoints: + return len(self.active_endpoints) + return 0 + + def open_findings(self, start_date=None, end_date=None): + if start_date is None or end_date is None: + return {} + + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + from dojo.utils import get_system_setting # noqa: PLC0415 circular import + findings = Finding.objects.filter(test__engagement__product=self, + mitigated__isnull=True, + false_p=False, + duplicate=False, + out_of_scope=False, + date__range=[start_date, + end_date]) + + if get_system_setting("enforce_verified_status", True) or get_system_setting("enforce_verified_status_metrics", True): + findings = findings.filter(verified=True) + + critical = findings.filter(severity="Critical").count() + high = findings.filter(severity="High").count() + medium = findings.filter(severity="Medium").count() + low = findings.filter(severity="Low").count() + + return {"Critical": critical, + "High": high, + "Medium": medium, + "Low": low, + "Total": (critical + high + medium + low)} + + def get_breadcrumbs(self): + return [{"title": str(self), + "url": reverse("view_product", args=(self.id,))}] + + @property + def get_product_type(self): + return self.prod_type if self.prod_type is not None else "unknown" + + # only used in APIv2 serializers.py, should be deprecated or at least prefetched + def open_findings_list(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + findings = Finding.objects.filter(test__engagement__product=self, active=True).values_list("id", flat=True) + return list(findings) + + @property + def has_jira_configured(self): + from dojo.jira import services as jira_services # noqa: PLC0415 circular import + return jira_services.has_configured(self) + + def violates_sla(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + findings = Finding.objects.filter(test__engagement__product=self, + active=True, + sla_expiration_date__lt=timezone.now().date()) + return findings.count() > 0 + + +class Product_API_Scan_Configuration(models.Model): + product = models.ForeignKey("dojo.Product", null=False, blank=False, on_delete=models.CASCADE) + tool_configuration = models.ForeignKey("dojo.Tool_Configuration", null=False, blank=False, on_delete=models.CASCADE) + service_key_1 = models.CharField(max_length=200, null=True, blank=True) + service_key_2 = models.CharField(max_length=200, null=True, blank=True) + service_key_3 = models.CharField(max_length=200, null=True, blank=True) + + def __str__(self): + name = self.tool_configuration.name + if self.service_key_1 or self.service_key_2 or self.service_key_3: + name += f" ({self.details})" + return name + + @property + def details(self): + details = "" + if self.service_key_1: + details += f"{self.service_key_1}" + if self.service_key_2: + details += f" | {self.service_key_2}" + if self.service_key_3: + details += f" | {self.service_key_3}" + return details diff --git a/dojo/product/ui/__init__.py b/dojo/product/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/product/ui/filters.py b/dojo/product/ui/filters.py new file mode 100644 index 00000000000..f5deccf9a02 --- /dev/null +++ b/dojo/product/ui/filters.py @@ -0,0 +1,213 @@ +from django.conf import settings +from django.forms import HiddenInput +from django_filters import ( + BooleanFilter, + CharFilter, + FilterSet, + ModelMultipleChoiceFilter, + MultipleChoiceFilter, + NumberFilter, + OrderingFilter, +) + +from dojo.filters import ( + DojoFilter, + ProductSLAFilter, + filter_endpoints_base, + filter_endpoints_host_base, +) +from dojo.labels import get_labels +from dojo.location.status import ProductLocationStatus +from dojo.models import Product, Product_Type +from dojo.product.queries import get_authorized_products +from dojo.product_type.queries import get_authorized_product_types + +labels = get_labels() + + +class ProductComponentFilter(DojoFilter): + component_name = CharFilter(lookup_expr="icontains", label="Module Name") + component_version = CharFilter(lookup_expr="icontains", label="Module Version") + + o = OrderingFilter( + fields=( + ("component_name", "component_name"), + ("component_version", "component_version"), + ("active", "active"), + ("duplicate", "duplicate"), + ("total", "total"), + ), + field_labels={ + "component_name": "Component Name", + "component_version": "Component Version", + "active": "Active", + "duplicate": "Duplicate", + "total": "Total", + }, + ) + + +class ComponentFilterWithoutObjectLookups(ProductComponentFilter): + test__engagement__product__prod_type__name = CharFilter( + field_name="test__engagement__product__prod_type__name", + lookup_expr="iexact", + label=labels.ORG_FILTERS_NAME_LABEL, + help_text=labels.ORG_FILTERS_NAME_HELP) + test__engagement__product__prod_type__name_contains = CharFilter( + field_name="test__engagement__product__prod_type__name", + lookup_expr="icontains", + label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, + help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) + test__engagement__product__name = CharFilter( + field_name="test__engagement__product__name", + lookup_expr="iexact", + label=labels.ASSET_FILTERS_NAME_LABEL, + help_text=labels.ASSET_FILTERS_NAME_HELP) + test__engagement__product__name_contains = CharFilter( + field_name="test__engagement__product__name", + lookup_expr="icontains", + label=labels.ASSET_FILTERS_NAME_CONTAINS_LABEL, + help_text=labels.ASSET_FILTERS_NAME_CONTAINS_HELP) + + +class ComponentFilter(ProductComponentFilter): + test__engagement__product__prod_type = ModelMultipleChoiceFilter( + queryset=Product_Type.objects.none(), + label=labels.ORG_FILTERS_LABEL) + test__engagement__product = ModelMultipleChoiceFilter( + queryset=Product.objects.none(), + label=labels.ASSET_FILTERS_LABEL) + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.form.fields[ + "test__engagement__product__prod_type"].queryset = get_authorized_product_types("view") + self.form.fields[ + "test__engagement__product"].queryset = get_authorized_products("view") + + +class ProductFilterHelper(FilterSet): + name = CharFilter(lookup_expr="icontains", label=labels.ASSET_FILTERS_NAME_LABEL) + name_exact = CharFilter(field_name="name", lookup_expr="iexact", label=labels.ASSET_FILTERS_NAME_EXACT_LABEL) + business_criticality = MultipleChoiceFilter(choices=Product.BUSINESS_CRITICALITY_CHOICES, null_label="Empty") + platform = MultipleChoiceFilter(choices=Product.PLATFORM_CHOICES, null_label="Empty") + lifecycle = MultipleChoiceFilter(choices=Product.LIFECYCLE_CHOICES, null_label="Empty") + origin = MultipleChoiceFilter(choices=Product.ORIGIN_CHOICES, null_label="Empty") + external_audience = BooleanFilter(field_name="external_audience") + internet_accessible = BooleanFilter(field_name="internet_accessible") + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Tag contains") + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", label="Not tag name contains", exclude=True) + outside_of_sla = ProductSLAFilter(label="Outside of SLA") + has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") + if settings.V3_FEATURE_LOCATIONS: + location_status = MultipleChoiceFilter( + field_name="locations__status", + choices=ProductLocationStatus.choices, + help_text="Status of the Location from the Products relationship", + ) + endpoints__host = CharFilter( + field_name="locations__location__url__host", method="filter_endpoints_host", label="Endpoint Host", + ) + endpoints = NumberFilter(field_name="locations__location", method="filter_endpoints", widget=HiddenInput()) + + def filter_endpoints_host(self, queryset, name, value): + return filter_endpoints_host_base( + queryset, + name, + value, + endpoint_id=self.data.get("endpoints"), + statuses=self.data.getlist("location_status"), + ) + + def filter_endpoints(self, queryset, name, value): + return filter_endpoints_base( + queryset, + name, + value, + statuses=self.data.getlist("location_status"), + host=self.data.get("endpoints__host"), + ) + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("name", "name"), + ("name_exact", "name_exact"), + ("prod_type__name", "prod_type__name"), + ("business_criticality", "business_criticality"), + ("platform", "platform"), + ("lifecycle", "lifecycle"), + ("origin", "origin"), + ("external_audience", "external_audience"), + ("internet_accessible", "internet_accessible"), + ("findings_count", "findings_count"), + ), + field_labels={ + "name": labels.ASSET_FILTERS_NAME_LABEL, + "name_exact": labels.ASSET_FILTERS_NAME_EXACT_LABEL, + "prod_type__name": labels.ORG_FILTERS_LABEL, + "business_criticality": "Business Criticality", + "platform": "Platform ", + "lifecycle": "Lifecycle ", + "origin": "Origin ", + "external_audience": "External Audience ", + "internet_accessible": "Internet Accessible ", + "findings_count": "Findings Count ", + }, + ) + + +class ProductFilter(ProductFilterHelper, DojoFilter): + prod_type = ModelMultipleChoiceFilter( + queryset=Product_Type.objects.none(), + label=labels.ORG_FILTERS_LABEL) + tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + queryset=Product.tags.tag_model.objects.all().order_by("name")) + not_tags = ModelMultipleChoiceFilter( + field_name="tags__name", + to_field_name="name", + exclude=True, + queryset=Product.tags.tag_model.objects.all().order_by("name")) + + def __init__(self, *args, **kwargs): + self.user = None + if "user" in kwargs: + self.user = kwargs.pop("user") + super().__init__(*args, **kwargs) + self.form.fields["prod_type"].queryset = get_authorized_product_types("view") + self.form.fields["tags"].help_text = labels.ASSET_FILTERS_TAGS_HELP + self.form.fields["not_tags"].help_text = labels.ASSET_FILTERS_NOT_TAGS_HELP + + class Meta: + model = Product + fields = [ + "name", "name_exact", "prod_type", "business_criticality", + "platform", "lifecycle", "origin", "external_audience", + "internet_accessible", "tags", + ] + + +class ProductFilterWithoutObjectLookups(ProductFilterHelper): + prod_type__name = CharFilter( + field_name="prod_type__name", + lookup_expr="iexact", + label=labels.ORG_FILTERS_NAME_LABEL, + help_text=labels.ORG_FILTERS_NAME_HELP) + prod_type__name_contains = CharFilter( + field_name="prod_type__name", + lookup_expr="icontains", + label=labels.ORG_FILTERS_NAME_CONTAINS_LABEL, + help_text=labels.ORG_FILTERS_NAME_CONTAINS_HELP) + + def __init__(self, *args, **kwargs): + kwargs.pop("user", None) + super().__init__(*args, **kwargs) + + class Meta: + model = Product + fields = [ + "name", "name_exact", "business_criticality", "platform", + "lifecycle", "origin", "external_audience", "internet_accessible", + ] diff --git a/dojo/product/ui/forms.py b/dojo/product/ui/forms.py new file mode 100644 index 00000000000..9f86884bfea --- /dev/null +++ b/dojo/product/ui/forms.py @@ -0,0 +1,154 @@ +from django import forms +from django.utils import timezone +from django.utils.dates import MONTHS + +from dojo.labels import get_labels +from dojo.models import ( + Dojo_User, + Product, + Product_API_Scan_Configuration, + Product_Type, + SLA_Configuration, + Tool_Configuration, +) +from dojo.product.queries import get_authorized_products +from dojo.product_type.queries import get_authorized_product_types +from dojo.validators import tag_validator + +labels = get_labels() + + +class ProductForm(forms.ModelForm): + name = forms.CharField(max_length=255, required=True) + description = forms.CharField(widget=forms.Textarea(attrs={}), + required=True) + + prod_type = forms.ModelChoiceField(label=labels.ORG_LABEL, + queryset=Product_Type.objects.none(), + required=True) + + sla_configuration = forms.ModelChoiceField(label="SLA Configuration", + queryset=SLA_Configuration.objects.all(), + required=True, + initial="Default") + + product_manager = forms.ModelChoiceField(label=labels.ASSET_MANAGER_LABEL, + queryset=Dojo_User.objects.exclude(is_active=False).order_by("first_name", "last_name"), required=False) + technical_contact = forms.ModelChoiceField(queryset=Dojo_User.objects.exclude(is_active=False).order_by("first_name", "last_name"), required=False) + team_manager = forms.ModelChoiceField(queryset=Dojo_User.objects.exclude(is_active=False).order_by("first_name", "last_name"), required=False) + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.fields["prod_type"].queryset = get_authorized_product_types("add") + self.fields["enable_product_tag_inheritance"].label = labels.ASSET_TAG_INHERITANCE_ENABLE_LABEL + self.fields["enable_product_tag_inheritance"].help_text = labels.ASSET_TAG_INHERITANCE_ENABLE_HELP + if prod_type_id := kwargs.get("instance", Product()).prod_type_id: # we are editing existing instance + self.fields["prod_type"].queryset |= Product_Type.objects.filter(pk=prod_type_id) # even if user does not have permission for any other ProdType we need to add at least assign ProdType to make form submittable (otherwise empty list was here which generated invalid form) + + # if this product has findings being asynchronously updated, disable the sla config field + if self.instance.async_updating: + self.fields["sla_configuration"].disabled = True + self.fields["sla_configuration"].widget.attrs["message"] = ( + "Finding SLA expiration dates are currently being recalculated. " + "This field cannot be changed until the calculation is complete." + ) + + class Meta: + model = Product + fields = ["name", "description", "tags", "product_manager", "technical_contact", "team_manager", "prod_type", "sla_configuration", "regulations", + "business_criticality", "platform", "lifecycle", "origin", "user_records", "revenue", "external_audience", "enable_product_tag_inheritance", + "internet_accessible", "enable_simple_risk_acceptance", "enable_full_risk_acceptance", "disable_sla_breach_notifications"] + + def clean_tags(self): + tag_validator(self.cleaned_data.get("tags")) + return self.cleaned_data.get("tags") + + +class DeleteProductForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Product + fields = ["id"] + + +class Add_Product_AuthorizedUsersForm(forms.Form): + users = forms.ModelMultipleChoiceField( + queryset=Dojo_User.objects.none(), required=True, label="Users", + ) + + def __init__(self, *args, product=None, **kwargs): + super().__init__(*args, **kwargs) + self.product = product + current = product.authorized_users.values_list("pk", flat=True) + self.fields["users"].queryset = ( + Dojo_User.objects.filter(is_active=True) + .exclude(is_superuser=True) + .exclude(pk__in=current) + .order_by("first_name", "last_name") + ) + + +class Authorize_User_For_ProductsForm(forms.Form): + products = forms.ModelMultipleChoiceField( + queryset=Product.objects.none(), required=True, label=labels.ASSET_PLURAL_LABEL, + ) + + def __init__(self, *args, user=None, **kwargs): + super().__init__(*args, **kwargs) + self.user = user + # Show products the user is not already directly authorized for. + self.fields["products"].queryset = ( + Product.objects.exclude(authorized_users=user).order_by("name") + ) + + +def get_years(): + now = timezone.now() + return [(now.year, now.year), (now.year - 1, now.year - 1), (now.year - 2, now.year - 2)] + + +class ProductCountsFormBase(forms.Form): + month = forms.ChoiceField(choices=list(MONTHS.items()), required=True, error_messages={ + "required": "*"}) + year = forms.ChoiceField(choices=get_years, required=True, error_messages={ + "required": "*"}) + + +class ProductTagCountsForm(ProductCountsFormBase): + product_tag = forms.ModelChoiceField(required=True, + queryset=Product.tags.tag_model.objects.none().order_by("name"), + label=labels.ASSET_TAG_LABEL, + error_messages={ + "required": "*"}) + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + prods = get_authorized_products("view") + tags_available_to_user = Product.tags.tag_model.objects.filter(product__in=prods) + self.fields["product_tag"].queryset = tags_available_to_user + + +class Product_API_Scan_ConfigurationForm(forms.ModelForm): + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + + tool_configuration = forms.ModelChoiceField( + label="Tool Configuration", + queryset=Tool_Configuration.objects.all().order_by("name"), + required=True, + ) + + class Meta: + model = Product_API_Scan_Configuration + exclude = ["product"] + + +class DeleteProduct_API_Scan_ConfigurationForm(forms.ModelForm): + id = forms.IntegerField(required=True, widget=forms.widgets.HiddenInput()) + + class Meta: + model = Product_API_Scan_Configuration + fields = ["id"] diff --git a/dojo/product/ui/views.py b/dojo/product/ui/views.py new file mode 100644 index 00000000000..5d6924ae033 --- /dev/null +++ b/dojo/product/ui/views.py @@ -0,0 +1,1830 @@ +# # product +import base64 +import calendar as tcalendar +import logging +from collections import OrderedDict +from datetime import date, datetime, timedelta +from functools import partial +from math import ceil + +from dateutil.relativedelta import relativedelta +from django.conf import settings +from django.contrib import messages +from django.contrib.admin.utils import NestedObjects +from django.contrib.postgres.aggregates import StringAgg +from django.core.exceptions import PermissionDenied, ValidationError +from django.db import DEFAULT_DB_ALIAS, connection +from django.db.models import Count, DateField, F, OuterRef, Prefetch, Q, Subquery, Sum, Value +from django.db.models.functions import Coalesce +from django.db.models.query import QuerySet +from django.http import Http404, HttpRequest, HttpResponseRedirect, JsonResponse +from django.shortcuts import get_object_or_404, render +from django.urls import reverse +from django.utils import timezone +from django.utils.translation import gettext as _ +from django.views import View +from github import Github + +import dojo.finding.helper as finding_helper +from dojo.authorization.authorization import user_has_permission_or_403 +from dojo.authorization.roles_permissions import Permissions +from dojo.components.sql_group_concat import Sql_GroupConcat +from dojo.engagement.ui.filters import ( + EngagementFilter, + EngagementFilterWithoutObjectLookups, + ProductEngagementFilter, + ProductEngagementFilterWithoutObjectLookups, +) +from dojo.filters import ( + MetricsEndpointFilter, + MetricsEndpointFilterWithoutObjectLookups, +) +from dojo.finding.ui.filters import ( + MetricsFindingFilter, + MetricsFindingFilterWithoutObjectLookups, +) +from dojo.forms import ( + AdHocFindingForm, + AppAnalysisForm, + DeleteAppAnalysisForm, + DeleteEngagementPresetsForm, + DojoMetaFormSet, + EngagementPresetsForm, + EngForm, + GITHUB_Product_Form, + GITHUBFindingForm, + JIRAEngagementForm, + JIRAFindingForm, + JIRAProjectForm, + ProductNotificationsForm, + SLA_Configuration, +) +from dojo.jira import services as jira_services +from dojo.labels import get_labels +from dojo.models import ( + App_Analysis, + Benchmark_Product_Summary, + Benchmark_Type, + BurpRawRequestResponse, + Dojo_User, + DojoMeta, + Endpoint, + Endpoint_Status, + Engagement, + Engagement_Presets, + Finding, + GITHUB_PKey, + Languages, + Note_Type, + Notifications, + Product, + Product_API_Scan_Configuration, + Product_Type, + System_Settings, + Test, + Test_Import, + Test_Type, +) +from dojo.product.queries import ( + get_authorized_products, +) +from dojo.product.ui.filters import ( + ProductComponentFilter, + ProductFilter, + ProductFilterWithoutObjectLookups, +) +from dojo.product.ui.forms import ( + Add_Product_AuthorizedUsersForm, + DeleteProduct_API_Scan_ConfigurationForm, + DeleteProductForm, + Product_API_Scan_ConfigurationForm, + ProductForm, +) +from dojo.product_type.queries import ( + get_authorized_product_types, +) +from dojo.query_utils import build_count_subquery +from dojo.templatetags.display_tags import asvs_calc_level +from dojo.tool_config.factory import create_API +from dojo.tools.factory import get_api_scan_configuration_hints +from dojo.utils import ( + Product_Tab, + add_breadcrumb, + add_error_message_to_response, + add_external_issue, + add_field_errors_to_response, + async_delete, + calculate_finding_age, + get_enabled_notifications_list, + get_open_findings_burndown, + get_page_items, + get_punchcard_data, + get_setting, + get_system_setting, + get_zero_severity_level, + queryset_check, + sum_by_severity_level, +) + +logger = logging.getLogger(__name__) + +labels = get_labels() + + +def product(request): + prods = get_authorized_products("view") + # perform all stuff for filtering and pagination first, before annotation/prefetching + # otherwise the paginator will perform all the annotations/prefetching already only to count the total number of records + # see https://code.djangoproject.com/ticket/23771 and https://code.djangoproject.com/ticket/25375 + + name_words = prods.values_list("name", flat=True) + base_findings = Finding.objects.filter(test__engagement__product_id=OuterRef("pk"), active=True) + prods = prods.annotate( + findings_count=Coalesce( + build_count_subquery(base_findings, group_field="test__engagement__product_id"), Value(0), + ), + ) + if settings.V3_FEATURE_LOCATIONS: + prods = prods.annotate( + location_host_count=Count("locations__location__url__host", distinct=True), + location_count=Count("locations", distinct=True), + ) + + filter_string_matching = get_system_setting("filter_string_matching", False) + filter_class = ProductFilterWithoutObjectLookups if filter_string_matching else ProductFilter + prod_filter = filter_class(request.GET, queryset=prods, user=request.user) + prod_list = get_page_items(request, prod_filter.qs, 25) + + # perform annotation/prefetching by replacing the queryset in the page with an annotated/prefetched queryset. + prod_list.object_list = prefetch_for_product(prod_list.object_list) + + # Get benchmark types for the template + benchmark_types = Benchmark_Type.objects.filter(enabled=True).order_by("name") + + add_breadcrumb(title=str(labels.ASSET_READ_LIST_LABEL), top_level=not len(request.GET), request=request) + + return render(request, "dojo/product.html", { + "prod_list": prod_list, + "prod_filter": prod_filter, + "name_words": sorted(set(name_words)), + "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), + "benchmark_types": benchmark_types, + "user": request.user}) + + +def prefetch_for_product(prods): + # old code can arrive here with prods being a list because the query was already executed + if not isinstance(prods, QuerySet): + logger.debug("unable to prefetch because query was already executed") + return prods + + prefetched_prods = prods.select_related("team_manager", "product_manager", "technical_contact").prefetch_related( + "tags", + "jira_project_set__jira_instance", + ) + + engagements = Engagement.objects.filter(product_id=OuterRef("pk")) + count_subquery = partial(build_count_subquery, group_field="product_id") + prefetched_prods = prefetched_prods.annotate( + active_engagement_count=Coalesce(count_subquery(engagements.filter(active=True)), Value(0)), + closed_engagement_count=Coalesce(count_subquery(engagements.filter(active=False)), Value(0)), + last_engagement_date=Subquery( + engagements.order_by("-target_start").values("target_start")[:1], output_field=DateField(), + ), + ) + + base_findings = Finding.objects.filter(test__engagement__product_id=OuterRef("pk")) + count_subquery = partial(build_count_subquery, group_field="test__engagement__product_id") + prefetched_prods = prefetched_prods.annotate( + active_finding_count=Coalesce(count_subquery(base_findings.filter(active=True)), Value(0)), + active_verified_finding_count=Coalesce( + count_subquery(base_findings.filter(active=True, verified=True)), + Value(0), + ), + ) + prefetched_prods = prefetched_prods.annotate( + total_reimport_count=Coalesce( + count_subquery( + Test_Import.objects.filter(test__engagement__product_id=OuterRef("pk"), type=Test_Import.REIMPORT_TYPE), + ), + Value(0), + ), + ) + + # TODO: Delete this after the move to Locations + if not settings.V3_FEATURE_LOCATIONS: + active_endpoint_qs = Endpoint.objects.filter( + status_endpoint__mitigated=False, + status_endpoint__false_positive=False, + status_endpoint__out_of_scope=False, + status_endpoint__risk_accepted=False, + ).distinct() + + prefetched_prods = prefetched_prods.prefetch_related( + Prefetch("endpoint_set", queryset=active_endpoint_qs, to_attr="active_endpoints"), + ) + + if get_system_setting("enable_github"): + prefetched_prods = prefetched_prods.prefetch_related( + Prefetch( + "github_pkey_set", queryset=GITHUB_PKey.objects.all().select_related("git_conf"), to_attr="github_confs", + ), + ) + return prefetched_prods + + +def iso_to_gregorian(iso_year, iso_week, iso_day): + jan4 = date(iso_year, 1, 4) + start = jan4 - timedelta(days=jan4.isoweekday() - 1) + return start + timedelta(weeks=iso_week - 1, days=iso_day - 1) + + +def view_product(request, pid): + prod_query = Product.objects.all().select_related("product_manager", "technical_contact", "team_manager", "sla_configuration") + prod = get_object_or_404(prod_query, id=pid) + authorized_users = prod.authorized_users.order_by("first_name", "last_name", "username") + personal_notifications_form = ProductNotificationsForm( + instance=Notifications.objects.filter(user=request.user).filter(product=prod).first()) + langSummary = Languages.objects.filter(product=prod).aggregate(Sum("files"), Sum("code"), Count("files")) + languages = Languages.objects.filter(product=prod).order_by("-code").select_related("language") + app_analysis = App_Analysis.objects.filter(product=prod).order_by("name") + benchmarks = Benchmark_Product_Summary.objects.filter(product=prod, publish=True, + benchmark_type__enabled=True).order_by("benchmark_type__name") + sla = SLA_Configuration.objects.filter(id=prod.sla_configuration_id).first() + benchAndPercent = [] + for i in range(len(benchmarks)): + desired_level, total, total_pass, total_wait, total_fail, _total_viewed = asvs_calc_level(benchmarks[i]) + + success_percent = round((float(total_pass) / float(total)) * 100, 2) + waiting_percent = round((float(total_wait) / float(total)) * 100, 2) + fail_percent = round(100 - success_percent - waiting_percent, 2) + benchAndPercent.append({ + "id": benchmarks[i].benchmark_type.id, + "name": benchmarks[i].benchmark_type, + "level": desired_level, + "success": {"count": total_pass, "percent": success_percent}, + "waiting": {"count": total_wait, "percent": waiting_percent}, + "fail": {"count": total_fail, "percent": fail_percent}, + "pass": total_pass + total_fail, + "total": total, + }) + system_settings = System_Settings.objects.get() + + product_metadata = dict(prod.product_meta.order_by("name").values_list("name", "value")) + + open_findings = Finding.objects.filter(test__engagement__product=prod, + false_p=False, + active=True, + duplicate=False, + out_of_scope=False).order_by("numerical_severity").values( + "severity").annotate(count=Count("severity")) + + critical = 0 + high = 0 + medium = 0 + low = 0 + info = 0 + + for v in open_findings: + if v["severity"] == "Critical": + critical = v["count"] + elif v["severity"] == "High": + high = v["count"] + elif v["severity"] == "Medium": + medium = v["count"] + elif v["severity"] == "Low": + low = v["count"] + elif v["severity"] == "Info": + info = v["count"] + + total = critical + high + medium + low + info + + product_tab = Product_Tab(prod, title=str(labels.ASSET_LABEL), tab="overview") + return render(request, "dojo/view_product_details.html", { + "prod": prod, + "product_tab": product_tab, + "product_metadata": product_metadata, + "critical": critical, + "high": high, + "medium": medium, + "low": low, + "info": info, + "total": total, + "user": request.user, + "languages": languages, + "langSummary": langSummary, + "app_analysis": app_analysis, + "system_settings": system_settings, + "benchmarks_percents": benchAndPercent, + "benchmarks": benchmarks, + "benchmark_type": product_tab.benchmark_type, + "authorized_users": authorized_users, + "personal_notifications_form": personal_notifications_form, + "enabled_notifications": get_enabled_notifications_list(), + "sla": sla}) + + +def view_product_components(request, pid): + prod = get_object_or_404(Product, id=pid) + product_tab = Product_Tab(prod, title=str(labels.ASSET_LABEL), tab="components") + separator = ", " + + # Get components ordered by component_name and concat component versions to the same row + if connection.vendor == "postgresql": + component_query = Finding.objects.filter(test__engagement__product__id=pid).values("component_name").order_by( + "component_name").annotate( + component_version=StringAgg("component_version", delimiter=separator, distinct=True, default=Value(""))) + else: + component_query = Finding.objects.filter(test__engagement__product__id=pid).values("component_name") + component_query = component_query.annotate( + component_version=Sql_GroupConcat("component_version", separator=separator, distinct=True)) + + # Append finding counts + component_query = component_query.annotate(total=Count("id")).order_by("component_name", "component_version") + component_query = component_query.annotate(active=Count("id", filter=Q(active=True))) + component_query = component_query.annotate(duplicate=(Count("id", filter=Q(duplicate=True)))) + + # Default sort by total descending + component_query = component_query.order_by("-total") + + comp_filter = ProductComponentFilter(request.GET, queryset=component_query) + result = get_page_items(request, comp_filter.qs, 25) + + # Filter out None values for auto-complete + component_words = component_query.exclude(component_name__isnull=True).values_list("component_name", flat=True) + + return render(request, "dojo/product_components.html", { + "prod": prod, + "filter": comp_filter, + "product_tab": product_tab, + "result": result, + "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), + "component_words": sorted(set(component_words)), + }) + + +def identify_view(request): + get_data = request.GET + view = get_data.get("type", None) + if view: + # value of view is reflected in the template, make sure it's valid + # although any XSS should be catch by django autoescape, we see people sometimes using '|safe'... + if view in {"Endpoint", "Finding"}: + return view + msg = 'invalid view, view must be "Endpoint" or "Finding"' + raise ValueError(msg) + if get_data.get("finding__severity", None) or get_data.get("false_positive", None): + return "Endpoint" + referer = request.META.get("HTTP_REFERER", None) + if referer: + if referer.find("type=Endpoint") > -1: + return "Endpoint" + return "Finding" + + +def finding_queries(request, prod): + filters = {} + findings_query = Finding.objects.filter(test__engagement__product=prod) + # prefetch only what's needed to avoid lots of repeated queries + findings_query = findings_query.prefetch_related( + # 'test__engagement', + # 'test__engagement__risk_acceptance', + # 'found_by', + # 'test', + # 'test__test_type', + # 'risk_acceptance_set', + "reporter") + filter_string_matching = get_system_setting("filter_string_matching", False) + finding_filter_class = MetricsFindingFilterWithoutObjectLookups if filter_string_matching else MetricsFindingFilter + findings = finding_filter_class(request.GET, queryset=findings_query, pid=prod) + findings_qs = queryset_check(findings) + filters["form"] = findings.form + + try: + # logger.debug(findings_qs.query) + start_date = findings_qs.earliest("date").date + start_date = datetime( + start_date.year, + start_date.month, start_date.day, + tzinfo=timezone.get_current_timezone()) + end_date = findings_qs.latest("date").date + end_date = datetime( + end_date.year, + end_date.month, end_date.day, + tzinfo=timezone.get_current_timezone()) + except Exception as e: + logger.debug(e) + start_date = timezone.now() + end_date = timezone.now() + week = end_date - timedelta(days=7) # seven days and /newer are considered "new" + + filters["accepted"] = findings_qs.filter(finding_helper.ACCEPTED_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") + filters["verified"] = findings_qs.filter(finding_helper.VERIFIED_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") + filters["new_verified"] = findings_qs.filter(finding_helper.VERIFIED_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") + filters["open"] = findings_qs.filter(finding_helper.OPEN_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") + filters["inactive"] = findings_qs.filter(finding_helper.INACTIVE_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") + # Filter closed findings by mitigated date (not discovery date). + # Use timezone.now() as the upper bound so findings closed after the latest + # discovery date are not excluded from the counter. + filters["closed"] = findings_qs.filter(finding_helper.CLOSED_FINDINGS_QUERY).filter(mitigated__range=[start_date, timezone.now()], mitigated__isnull=False).order_by("mitigated") + filters["false_positive"] = findings_qs.filter(finding_helper.FALSE_POSITIVE_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") + filters["out_of_scope"] = findings_qs.filter(finding_helper.OUT_OF_SCOPE_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") + filters["all"] = findings_qs.order_by("date") + filters["open_vulns"] = findings_qs.filter(finding_helper.OPEN_FINDINGS_QUERY).filter( + cwe__isnull=False, + ).order_by("cwe").values( + "cwe", + ).annotate( + count=Count("cwe"), + ) + + filters["all_vulns"] = findings_qs.filter( + duplicate=False, + cwe__isnull=False, + ).order_by("cwe").values( + "cwe", + ).annotate( + count=Count("cwe"), + ) + + filters["start_date"] = start_date + filters["end_date"] = end_date + filters["week"] = week + + return filters + + +# TODO: Delete this after the move to Locations +def endpoint_queries(request, prod): + filters = {} + endpoints_query = Endpoint_Status.objects.filter(finding__test__engagement__product=prod, + finding__severity__in=( + "Critical", "High", "Medium", "Low", "Info")).prefetch_related( + "finding__test__engagement", + "finding__test__engagement__risk_acceptance", + "finding__risk_acceptance_set", + "finding__reporter").annotate(severity=F("finding__severity")) + filter_string_matching = get_system_setting("filter_string_matching", False) + filter_class = MetricsEndpointFilterWithoutObjectLookups if filter_string_matching else MetricsEndpointFilter + endpoints = filter_class(request.GET, queryset=endpoints_query) + endpoints_qs = queryset_check(endpoints) + filters["form"] = endpoints.form + + if not endpoints_qs and not endpoints_query: + messages.add_message( + request, + messages.WARNING, + _("No Endpoints match the current filters."), + extra_tags="alert-danger") + try: + start_date = endpoints_qs.earliest("date").date + start_date = datetime(start_date.year, + start_date.month, start_date.day, + tzinfo=timezone.get_current_timezone()) + end_date = endpoints_qs.latest("date").date + end_date = datetime(end_date.year, + end_date.month, end_date.day, + tzinfo=timezone.get_current_timezone()) + except: + start_date = timezone.now() + end_date = timezone.now() + week = end_date - timedelta(days=7) # seven days and /newnewer are considered "new" + + filters["accepted"] = endpoints_qs.filter(date__range=[start_date, end_date], + risk_accepted=True).order_by("date") + filters["verified"] = endpoints_qs.filter(date__range=[start_date, end_date], + false_positive=False, + mitigated=True, + out_of_scope=False).order_by("date") + filters["new_verified"] = endpoints_qs.filter(date__range=[week, end_date], + false_positive=False, + mitigated=True, + out_of_scope=False).order_by("date") + filters["open"] = endpoints_qs.filter(date__range=[start_date, end_date], + mitigated=False, + finding__active=True) + filters["inactive"] = endpoints_qs.filter(date__range=[start_date, end_date], + mitigated=True) + filters["closed"] = endpoints_qs.filter(date__range=[start_date, end_date], + mitigated=True) + filters["false_positive"] = endpoints_qs.filter(date__range=[start_date, end_date], + false_positive=True) + filters["out_of_scope"] = endpoints_qs.filter(date__range=[start_date, end_date], + out_of_scope=True) + filters["all"] = endpoints_qs + filters["open_vulns"] = endpoints_qs.filter( + false_positive=False, + out_of_scope=False, + mitigated=True, + finding__cwe__isnull=False, + ).order_by("finding__cwe").values( + "finding__cwe", + ).annotate( + count=Count("finding__cwe"), + ).annotate( + cwe=F("finding__cwe"), + ) + + filters["all_vulns"] = endpoints_qs.filter( + finding__cwe__isnull=False, + ).order_by("finding__cwe").values( + "finding__cwe", + ).annotate( + count=Count("finding__cwe"), + ).annotate( + cwe=F("finding__cwe"), + ) + + filters["start_date"] = start_date + filters["end_date"] = end_date + filters["week"] = week + + return filters + + +def view_product_metrics(request, pid): + prod = get_object_or_404(Product, id=pid) + engs = Engagement.objects.filter(product=prod, active=True) + view = identify_view(request) + + filter_string_matching = get_system_setting("filter_string_matching", False) + filter_class = EngagementFilterWithoutObjectLookups if filter_string_matching else EngagementFilter + result = filter_class( + request.GET, + queryset=Engagement.objects.filter(product=prod, active=False).order_by("-target_end")) + + inactive_engs_page = get_page_items(request, result.qs, 10) + + filters = {} + if view == "Finding": + filters = finding_queries(request, prod) + # TODO: Delete this after the move to Locations + elif view == "Endpoint": + filters = endpoint_queries(request, prod) + + start_date = timezone.make_aware(datetime.combine(filters["start_date"], datetime.min.time())) + end_date = filters["end_date"] + + r = relativedelta(end_date, start_date) + weeks_between = ceil((((r.years * 12) + r.months) * 4.33) + (r.days / 7)) + if weeks_between <= 0: + weeks_between += 2 + + punchcard, ticks = get_punchcard_data(filters.get("open", None), start_date, weeks_between, view) + + add_breadcrumb(parent=prod, top_level=False, request=request) + + # An ordered dict does not make sense here. + open_close_weekly = OrderedDict() + severity_weekly = OrderedDict() + critical_weekly = OrderedDict() + high_weekly = OrderedDict() + medium_weekly = OrderedDict() + open_objs_by_age = {} + + open_objs_by_severity = get_zero_severity_level() + closed_objs_by_severity = get_zero_severity_level() + accepted_objs_by_severity = get_zero_severity_level() + + # Optimization: Make all queries lists, and only pull values of fields for metrics based calculations + open_vulnerabilities = list(filters["open_vulns"].values("cwe", "count")) + all_vulnerabilities = list(filters["all_vulns"].values("cwe", "count")) + + verified_objs_by_severity = list(filters.get("verified").values("severity")) + inactive_objs_by_severity = list(filters.get("inactive").values("severity")) + false_positive_objs_by_severity = list(filters.get("false_positive").values("severity")) + out_of_scope_objs_by_severity = list(filters.get("out_of_scope").values("severity")) + new_objs_by_severity = list(filters.get("new_verified").values("severity")) + all_objs_by_severity = list(filters.get("all").values("severity")) + + all_findings = list(filters.get("all", []).values("id", "date", "severity")) + open_findings = list(filters.get("open", []).values("id", "date", "mitigated", "severity")) + # Include mitigated date for closed findings to group by when they were closed, not discovered + closed_findings = list(filters.get("closed", []).values("id", "date", "mitigated", "severity")) + accepted_findings = list(filters.get("accepted", []).values("id", "date", "severity")) + + """ + Optimization: Create dictionaries in the structure of { finding_id: True } for index based search + Previously the for-loop below used "if finding in open_findings" -- an average O(n^2) time complexity + This allows for "if open_findings.get(finding_id, None)" -- an average O(n) time complexity + """ + open_findings_dict = {f.get("id"): True for f in open_findings} + closed_findings_dict = {f.get("id"): True for f in closed_findings} + accepted_findings_dict = {f.get("id"): True for f in accepted_findings} + + for finding in all_findings: + iso_cal = finding.get("date").isocalendar() + date = iso_to_gregorian(iso_cal[0], iso_cal[1], 1) + html_date = date.strftime("%m/%d
%Y") + unix_timestamp = (tcalendar.timegm(date.timetuple()) * 1000) + + # Open findings + if open_findings_dict.get(finding.get("id", None)): + if unix_timestamp not in critical_weekly: + critical_weekly[unix_timestamp] = {"count": 0, "week": html_date} + if unix_timestamp not in high_weekly: + high_weekly[unix_timestamp] = {"count": 0, "week": html_date} + if unix_timestamp not in medium_weekly: + medium_weekly[unix_timestamp] = {"count": 0, "week": html_date} + + if unix_timestamp in open_close_weekly: + open_close_weekly[unix_timestamp]["open"] += 1 + else: + open_close_weekly[unix_timestamp] = {"closed": 0, "open": 1, "accepted": 0} + open_close_weekly[unix_timestamp]["week"] = html_date + + if view in {"Finding", "Endpoint"}: + severity = finding.get("severity") + + finding_age = calculate_finding_age(finding) + if open_objs_by_age.get(finding_age): + open_objs_by_age[finding_age] += 1 + else: + open_objs_by_age[finding_age] = 1 + + if unix_timestamp in severity_weekly: + if severity in severity_weekly[unix_timestamp]: + severity_weekly[unix_timestamp][severity] += 1 + else: + severity_weekly[unix_timestamp][severity] = 1 + else: + severity_weekly[unix_timestamp] = get_zero_severity_level() + severity_weekly[unix_timestamp][severity] = 1 + severity_weekly[unix_timestamp]["week"] = html_date + + if severity == "Critical": + if unix_timestamp in critical_weekly: + critical_weekly[unix_timestamp]["count"] += 1 + else: + critical_weekly[unix_timestamp] = {"count": 1, "week": html_date} + elif severity == "High": + if unix_timestamp in high_weekly: + high_weekly[unix_timestamp]["count"] += 1 + else: + high_weekly[unix_timestamp] = {"count": 1, "week": html_date} + elif severity == "Medium": + if unix_timestamp in medium_weekly: + medium_weekly[unix_timestamp]["count"] += 1 + else: + medium_weekly[unix_timestamp] = {"count": 1, "week": html_date} + # Optimization: count severity level on server side + if open_objs_by_severity.get(finding.get("severity")) is not None: + open_objs_by_severity[finding.get("severity")] += 1 + + # Close findings - group by mitigated date, not discovery date + elif closed_findings_dict.get(finding.get("id", None)): + # Find the closed finding to get its mitigated date + closed_finding = next((f for f in closed_findings if f.get("id") == finding.get("id")), None) + if closed_finding and closed_finding.get("mitigated"): + # Use mitigated date for grouping closed findings + mitigated_date = closed_finding.get("mitigated") + mitigated_date_only = mitigated_date.date() if isinstance(mitigated_date, datetime) else mitigated_date + iso_cal = mitigated_date_only.isocalendar() + mitigated_week_start = iso_to_gregorian(iso_cal[0], iso_cal[1], 1) + mitigated_html_date = mitigated_week_start.strftime("%m/%d
%Y") + mitigated_unix_timestamp = (tcalendar.timegm(mitigated_week_start.timetuple()) * 1000) + + if mitigated_unix_timestamp in open_close_weekly: + open_close_weekly[mitigated_unix_timestamp]["closed"] += 1 + else: + open_close_weekly[mitigated_unix_timestamp] = {"closed": 1, "open": 0, "accepted": 0} + open_close_weekly[mitigated_unix_timestamp]["week"] = mitigated_html_date + elif unix_timestamp in open_close_weekly: + # Fallback to discovery date if mitigated date is not available + open_close_weekly[unix_timestamp]["closed"] += 1 + else: + # Fallback to discovery date if mitigated date is not available + open_close_weekly[unix_timestamp] = {"closed": 1, "open": 0, "accepted": 0} + open_close_weekly[unix_timestamp]["week"] = html_date + # Optimization: count severity level on server side + if closed_objs_by_severity.get(finding.get("severity")) is not None: + closed_objs_by_severity[finding.get("severity")] += 1 + + # Risk Accepted findings + if accepted_findings_dict.get(finding.get("id", None)): + if unix_timestamp in open_close_weekly: + open_close_weekly[unix_timestamp]["accepted"] += 1 + else: + open_close_weekly[unix_timestamp] = {"closed": 0, "open": 0, "accepted": 1} + open_close_weekly[unix_timestamp]["week"] = html_date + # Optimization: count severity level on server side + if accepted_objs_by_severity.get(finding.get("severity")) is not None: + accepted_objs_by_severity[finding.get("severity")] += 1 + + tests = Test.objects.filter(engagement__product=prod).prefetch_related("finding_set", "test_type") + verified_finding_subquery = build_count_subquery( + Finding.objects.filter(test=OuterRef("pk"), verified=True), + group_field="test_id", + ) + tests = tests.annotate(verified_finding_count=Coalesce(verified_finding_subquery, Value(0))) + + test_data = {} + for t in tests: + if t.test_type.name in test_data: + test_data[t.test_type.name] += t.verified_finding_count + else: + test_data[t.test_type.name] = t.verified_finding_count + + # Optimization: Format Open/Total CWE vulnerabilities graph data here, instead of template + open_vulnerabilities = [["CWE-" + str(f.get("cwe")), f.get("count")] for f in open_vulnerabilities] + all_vulnerabilities = [["CWE-" + str(f.get("cwe")), f.get("count")] for f in all_vulnerabilities] + + product_tab = Product_Tab(prod, title=str(labels.ASSET_LABEL), tab="metrics") + + return render(request, "dojo/product_metrics.html", { + "prod": prod, + "product_tab": product_tab, + "engs": engs, + "inactive_engs": inactive_engs_page, + "view": view, + "verified_objs": len(verified_objs_by_severity), + "verified_objs_by_severity": sum_by_severity_level(verified_objs_by_severity), + "open_objs": len(open_findings), + "open_objs_by_severity": open_objs_by_severity, + "open_objs_by_age": open_objs_by_age, + "inactive_objs": len(inactive_objs_by_severity), + "inactive_objs_by_severity": sum_by_severity_level(inactive_objs_by_severity), + "closed_objs": len(closed_findings), + "closed_objs_by_severity": closed_objs_by_severity, + "false_positive_objs": len(false_positive_objs_by_severity), + "false_positive_objs_by_severity": sum_by_severity_level(false_positive_objs_by_severity), + "out_of_scope_objs": len(out_of_scope_objs_by_severity), + "out_of_scope_objs_by_severity": sum_by_severity_level(out_of_scope_objs_by_severity), + "accepted_objs": len(accepted_findings), + "accepted_objs_by_severity": accepted_objs_by_severity, + "new_objs": len(new_objs_by_severity), + "new_objs_by_severity": sum_by_severity_level(new_objs_by_severity), + "all_objs": len(all_objs_by_severity), + "all_objs_by_severity": sum_by_severity_level(all_objs_by_severity), + "form": filters.get("form", None), + "reset_link": reverse("view_product_metrics", args=(prod.id,)) + "?type=" + view, + "open_vulnerabilities_count": len(open_vulnerabilities), + "open_vulnerabilities": open_vulnerabilities, + "all_vulnerabilities_count": len(all_vulnerabilities), + "all_vulnerabilities": all_vulnerabilities, + "start_date": start_date, + "punchcard": punchcard, + "ticks": ticks, + "open_close_weekly": open_close_weekly, + "severity_weekly": severity_weekly, + "critical_weekly": critical_weekly, + "high_weekly": high_weekly, + "medium_weekly": medium_weekly, + "test_data": test_data, + "user": request.user}) + + +def async_burndown_metrics(request, pid): + prod = get_object_or_404(Product, id=pid) + open_findings_burndown = get_open_findings_burndown(prod) + + return JsonResponse({ + "critical": open_findings_burndown.get("Critical", []), + "high": open_findings_burndown.get("High", []), + "medium": open_findings_burndown.get("Medium", []), + "low": open_findings_burndown.get("Low", []), + "info": open_findings_burndown.get("Info", []), + "max": open_findings_burndown.get("y_max", 0), + "min": open_findings_burndown.get("y_min", 0), + }) + + +def view_engagements(request, pid): + prod = get_object_or_404(Product, id=pid) + default_page_num = 10 + recent_test_day_count = 7 + filter_string_matching = get_system_setting("filter_string_matching", False) + filter_class = ProductEngagementFilterWithoutObjectLookups if filter_string_matching else ProductEngagementFilter + # In Progress Engagements + engs = Engagement.objects.filter(product=prod, active=True, status="In Progress").order_by("-updated") + active_engs_filter = filter_class(request.GET, queryset=engs, prefix="active") + result_active_engs = get_page_items(request, active_engs_filter.qs, default_page_num, prefix="engs") + # prefetch only after creating the filters to avoid https://code.djangoproject.com/ticket/23771 + # and https://code.djangoproject.com/ticket/25375 + result_active_engs.object_list = prefetch_for_view_engagements( + result_active_engs.object_list, + recent_test_day_count, + ) + # Engagements that are queued because they haven't started or paused + engs = Engagement.objects.filter(~Q(status="In Progress"), product=prod, active=True).order_by("-updated") + queued_engs_filter = filter_class(request.GET, queryset=engs, prefix="queued") + result_queued_engs = get_page_items(request, queued_engs_filter.qs, default_page_num, prefix="queued_engs") + result_queued_engs.object_list = prefetch_for_view_engagements( + result_queued_engs.object_list, + recent_test_day_count, + ) + # Cancelled or Completed Engagements + engs = Engagement.objects.filter(product=prod, active=False).order_by("-target_end") + inactive_engs_filter = filter_class(request.GET, queryset=engs, prefix="closed") + result_inactive_engs = get_page_items(request, inactive_engs_filter.qs, default_page_num, prefix="inactive_engs") + result_inactive_engs.object_list = prefetch_for_view_engagements( + result_inactive_engs.object_list, + recent_test_day_count, + ) + + product_tab = Product_Tab(prod, title=_("All Engagements"), tab="engagements") + return render(request, "dojo/view_engagements.html", { + "prod": prod, + "product_tab": product_tab, + "engs": result_active_engs, + "engs_count": result_active_engs.paginator.count, + "engs_filter": active_engs_filter, + "queued_engs": result_queued_engs, + "queued_engs_count": result_queued_engs.paginator.count, + "queued_engs_filter": queued_engs_filter, + "inactive_engs": result_inactive_engs, + "inactive_engs_count": result_inactive_engs.paginator.count, + "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), + "inactive_engs_filter": inactive_engs_filter, + "recent_test_day_count": recent_test_day_count, + "user": request.user}) + + +def prefetch_for_view_engagements(engagements, recent_test_day_count): + engagements = engagements.prefetch_related( + Prefetch("test_set", queryset=Test.objects.filter( + id__in=Subquery( + Test.objects.filter( + engagement_id=OuterRef("engagement_id"), + updated__gte=timezone.now() - timedelta(days=recent_test_day_count), + ).values_list("id", flat=True), + )), + ), + "test_set__test_type", + ).select_related( + "lead", + ) + + # Use subqueries to avoid GROUP BY issues + test_subquery = build_count_subquery( + Test.objects.filter(engagement=OuterRef("pk")), group_field="engagement_id", + ) + finding_subquery = build_count_subquery( + Finding.objects.filter(test__engagement=OuterRef("pk")), group_field="test__engagement_id", + ) + finding_open_subquery = build_count_subquery( + Finding.objects.filter(test__engagement=OuterRef("pk"), active=True), group_field="test__engagement_id", + ) + finding_open_verified_subquery = build_count_subquery( + Finding.objects.filter(test__engagement=OuterRef("pk"), active=True, verified=True), group_field="test__engagement_id", + ) + finding_open_fix_available_subquery = build_count_subquery( + Finding.objects.filter(test__engagement=OuterRef("pk"), active=True, fix_available=True), group_field="test__engagement_id", + ) + finding_close_subquery = build_count_subquery( + Finding.objects.filter(test__engagement=OuterRef("pk"), is_mitigated=True), group_field="test__engagement_id", + ) + finding_duplicate_subquery = build_count_subquery( + Finding.objects.filter(test__engagement=OuterRef("pk"), duplicate=True), group_field="test__engagement_id", + ) + finding_accepted_subquery = build_count_subquery( + Finding.objects.filter(test__engagement=OuterRef("pk"), risk_accepted=True), group_field="test__engagement_id", + ) + + engagements = engagements.annotate( + count_tests=Coalesce(test_subquery, Value(0)), + count_findings_all=Coalesce(finding_subquery, Value(0)), + count_findings_open=Coalesce(finding_open_subquery, Value(0)), + count_findings_open_verified=Coalesce(finding_open_verified_subquery, Value(0)), + count_findings_fix_available=Coalesce(finding_open_fix_available_subquery, Value(0)), + count_findings_close=Coalesce(finding_close_subquery, Value(0)), + count_findings_duplicate=Coalesce(finding_duplicate_subquery, Value(0)), + count_findings_accepted=Coalesce(finding_accepted_subquery, Value(0)), + ) + + if System_Settings.objects.get().enable_jira: + engagements = engagements.prefetch_related( + "jira_project__jira_instance", + "product__jira_project_set__jira_instance", + ) + + return engagements + + +def new_product(request, ptid=None): + if get_authorized_product_types("add").count() == 0: + raise PermissionDenied + + jira_project_form = None + error = False + initial = None + if ptid is not None: + prod_type = get_object_or_404(Product_Type, pk=ptid) + initial = {"prod_type": prod_type} + + form = ProductForm(initial=initial) + + if request.method == "POST": + form = ProductForm(request.POST, instance=Product()) + + if get_system_setting("enable_github"): + gform = GITHUB_Product_Form(request.POST, instance=GITHUB_PKey()) + else: + gform = None + + if form.is_valid(): + product_type = form.instance.prod_type + user_has_permission_or_403(request.user, product_type, "add") + + product = form.save() + messages.add_message(request, + messages.SUCCESS, + labels.ASSET_CREATE_SUCCESS_MESSAGE, + extra_tags="alert-success") + success, jira_project_form = jira_services.process_project_form(request, product=product) + error = not success + + if get_system_setting("enable_github"): + if gform.is_valid(): + github_pkey = gform.save(commit=False) + if github_pkey.git_conf is not None and github_pkey.git_project: + github_pkey.product = product + github_pkey.save() + messages.add_message(request, + messages.SUCCESS, + _("GitHub information added successfully."), + extra_tags="alert-success") + # Create appropriate labels in the repo + logger.info("Create label in repo: " + github_pkey.git_project) + + description = _("This label is automatically applied to all issues created by DefectDojo") + try: + g = Github(github_pkey.git_conf.api_key) + repo = g.get_repo(github_pkey.git_project) + repo.create_label(name="security", color="FF0000", + description=description) + repo.create_label(name="security / info", color="00FEFC", + description=description) + repo.create_label(name="security / low", color="B7FE00", + description=description) + repo.create_label(name="security / medium", color="FEFE00", + description=description) + repo.create_label(name="security / high", color="FE9A00", + description=description) + repo.create_label(name="security / critical", color="FE2200", + description=description) + except: + logger.info("Labels cannot be created - they may already exists") + + if not error: + return HttpResponseRedirect(reverse("view_product", args=(product.id,))) + # engagement was saved, but JIRA errors, so goto edit_product + return HttpResponseRedirect(reverse("edit_product", args=(product.id,))) + else: + if get_system_setting("enable_jira"): + jira_project_form = JIRAProjectForm() + + gform = GITHUB_Product_Form() if get_system_setting("enable_github") else None + + add_breadcrumb(title=str(labels.ASSET_CREATE_LABEL), top_level=False, request=request) + return render(request, "dojo/new_product.html", + {"form": form, + "jform": jira_project_form, + "gform": gform}) + + +def edit_product(request, pid): + product = Product.objects.get(pk=pid) + system_settings = System_Settings.objects.get() + jira_enabled = system_settings.enable_jira + jira_project = None + jform = None + github_enabled = system_settings.enable_github + github_inst = None + gform = None + error = False + + try: + github_inst = GITHUB_PKey.objects.get(product=product) + except: + github_inst = None + + if request.method == "POST": + form = ProductForm(request.POST, instance=product) + jira_project = jira_services.get_project(product) + if form.is_valid(): + initial_sla_config = Product.objects.get(pk=form.instance.id).sla_configuration + form.save() + msg = labels.ASSET_UPDATE_SUCCESS_MESSAGE + # check if the SLA config was changed, append additional context to message + if initial_sla_config != form.instance.sla_configuration: + msg += " " + labels.ASSET_UPDATE_SLA_CHANGED_MESSAGE + messages.add_message(request, + messages.SUCCESS, + msg, + extra_tags="alert-success") + + success, jform = jira_services.process_project_form(request, instance=jira_project, product=product) + error = not success + + if get_system_setting("enable_github") and github_inst: + gform = GITHUB_Product_Form(request.POST, instance=github_inst) + if gform.is_valid(): + gform.save() + elif get_system_setting("enable_github"): + gform = GITHUB_Product_Form(request.POST) + if gform.is_valid(): + new_conf = gform.save(commit=False) + new_conf.product_id = pid + new_conf.save() + messages.add_message(request, + messages.SUCCESS, + _("GITHUB information updated successfully."), + extra_tags="alert-success") + + if not error: + return HttpResponseRedirect(reverse("view_product", args=(pid,))) + else: + form = ProductForm(instance=product) + + if jira_enabled: + jira_project = jira_services.get_project(product) + jform = JIRAProjectForm(instance=jira_project) + else: + jform = None + + if github_enabled: + gform = GITHUB_Product_Form(instance=github_inst) if github_inst is not None else GITHUB_Product_Form() + else: + gform = None + + product_tab = Product_Tab(product, title=str(labels.ASSET_UPDATE_LABEL), tab="settings") + return render(request, + "dojo/edit_product.html", + {"form": form, + "product_tab": product_tab, + "jform": jform, + "gform": gform, + "product": product, + }) + + +def delete_product(request, pid): + product = get_object_or_404(Product, pk=pid) + form = DeleteProductForm(instance=product) + + if request.method == "POST": + logger.debug("delete_product: POST") + if "id" in request.POST and str(product.id) == request.POST["id"]: + form = DeleteProductForm(request.POST, instance=product) + if form.is_valid(): + if get_setting("ASYNC_OBJECT_DELETE"): + async_del = async_delete() + async_del.delete(product) + message = labels.ASSET_DELETE_SUCCESS_ASYNC_MESSAGE + else: + message = labels.ASSET_DELETE_SUCCESS_MESSAGE + with Endpoint.allow_endpoint_init(): # TODO: Delete this after the move to Locations + product.delete() + messages.add_message(request, + messages.SUCCESS, + message, + extra_tags="alert-success") + logger.debug("delete_product: POST RETURN") + return HttpResponseRedirect(reverse("product")) + logger.debug("delete_product: POST INVALID FORM") + logger.error(form.errors) + + logger.debug("delete_product: GET") + + rels = ["Previewing the relationships has been disabled.", ""] + display_preview = get_setting("DELETE_PREVIEW") + if display_preview: + with Endpoint.allow_endpoint_init(): # TODO: Delete this after the move to Locations + collector = NestedObjects(using=DEFAULT_DB_ALIAS) + collector.collect([product]) + rels = collector.nested() + + product_tab = Product_Tab(product, title=str(labels.ASSET_LABEL), tab="settings") + + logger.debug("delete_product: GET RENDER") + + return render(request, "dojo/delete_product.html", { + "label_delete_with_name": labels.ASSET_DELETE_WITH_NAME_LABEL % {"name": product}, + "product": product, + "form": form, + "product_tab": product_tab, + "rels": rels}) + + +def new_eng_for_app(request, pid, *, cicd=False): + jira_project_form = None + jira_epic_form = None + + product = Product.objects.get(id=pid) + + if request.method == "POST": + form = EngForm(request.POST, cicd=cicd, product=product, user=request.user) + logger.debug("new_eng_for_app") + + if form.is_valid(): + # first create the new engagement + engagement = form.save(commit=False) + engagement.threat_model = False + engagement.api_test = False + engagement.pen_test = False + engagement.check_list = False + engagement.product = form.cleaned_data.get("product") + if engagement.threat_model: + engagement.progress = "threat_model" + else: + engagement.progress = "other" + if cicd: + engagement.engagement_type = "CI/CD" + engagement.status = "In Progress" + engagement.active = True + + engagement.save() + form.save_m2m() + + logger.debug("new_eng_for_app: process jira coming") + + # new engagement, so do not provide jira_project + success, jira_project_form = jira_services.process_project_form(request, instance=None, + engagement=engagement) + error = not success + + logger.debug("new_eng_for_app: process jira epic coming") + + success, jira_epic_form = jira_services.process_epic_form(request, engagement=engagement) + error = error or not success + + messages.add_message(request, + messages.SUCCESS, + _("Engagement added successfully."), + extra_tags="alert-success") + + if not error: + if "_Add Tests" in request.POST: + return HttpResponseRedirect(reverse("add_tests", args=(engagement.id,))) + if "_Import Scan Results" in request.POST: + return HttpResponseRedirect(reverse("import_scan_results", args=(engagement.id,))) + return HttpResponseRedirect(reverse("view_engagement", args=(engagement.id,))) + # engagement was saved, but JIRA errors, so goto edit_engagement + logger.debug("new_eng_for_app: jira errors") + return HttpResponseRedirect(reverse("edit_engagement", args=(engagement.id,))) + logger.debug(form.errors) + else: + form = EngForm(initial={"lead": request.user, "target_start": timezone.now().date(), + "target_end": timezone.now().date() + timedelta(days=7), "product": product}, cicd=cicd, + product=product, user=request.user) + + if get_system_setting("enable_jira"): + logger.debug("showing jira-project-form") + jira_project_form = JIRAProjectForm(target="engagement", product=product) + logger.debug("showing jira-epic-form") + jira_epic_form = JIRAEngagementForm() + + title = _("New CI/CD Engagement") if cicd else _("New Interactive Engagement") + + product_tab = Product_Tab(product, title=title, tab="engagements") + return render(request, "dojo/new_eng.html", { + "form": form, + "title": title, + "product_tab": product_tab, + "jira_epic_form": jira_epic_form, + "jira_project_form": jira_project_form}) + + +def new_tech_for_prod(request, pid): + if request.method == "POST": + form = AppAnalysisForm(request.POST) + if form.is_valid(): + tech = form.save(commit=False) + tech.product_id = pid + tech.save() + messages.add_message(request, + messages.SUCCESS, + _("Technology added successfully."), + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_product", args=(pid,))) + + form = AppAnalysisForm(initial={"user": request.user}) + product_tab = Product_Tab(get_object_or_404(Product, id=pid), title=_("Add Technology"), tab="settings") + return render(request, "dojo/new_tech.html", + {"form": form, + "product_tab": product_tab, + "pid": pid}) + + +def edit_technology(request, tid): + technology = get_object_or_404(App_Analysis, id=tid) + form = AppAnalysisForm(instance=technology) + if request.method == "POST": + form = AppAnalysisForm(request.POST, instance=technology) + if form.is_valid(): + form.save() + messages.add_message(request, + messages.SUCCESS, + _("Technology changed successfully."), + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_product", args=(technology.product.id,))) + + product_tab = Product_Tab(technology.product, title=_("Edit Technology"), tab="settings") + return render(request, "dojo/edit_technology.html", + {"form": form, + "product_tab": product_tab, + "technology": technology}) + + +def delete_technology(request, tid): + technology = get_object_or_404(App_Analysis, id=tid) + form = DeleteAppAnalysisForm(instance=technology) + if request.method == "POST": + form = DeleteAppAnalysisForm(request.POST, instance=technology) + technology = form.instance + technology.delete() + messages.add_message(request, + messages.SUCCESS, + _("Technology deleted successfully."), + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_product", args=(technology.product.id,))) + + product_tab = Product_Tab(technology.product, title=_("Delete Technology"), tab="settings") + return render(request, "dojo/delete_technology.html", { + "technology": technology, + "form": form, + "product_tab": product_tab, + }) + + +def new_eng_for_app_cicd(request, pid): + # we have to use pid=pid here as new_eng_for_app expects kwargs, because that is how django calls the function based on urls.py named groups + return new_eng_for_app(request, pid=pid, cicd=True) + + +def manage_meta_data(request, pid): + product = Product.objects.get(id=pid) + meta_data_query = DojoMeta.objects.filter(product=product) + form_mapping = {"product": product} + formset = DojoMetaFormSet(queryset=meta_data_query, form_kwargs={"fk_map": form_mapping}) + + if request.method == "POST": + formset = DojoMetaFormSet(request.POST, queryset=meta_data_query, form_kwargs={"fk_map": form_mapping}) + if formset.is_valid(): + formset.save() + messages.add_message( + request, messages.SUCCESS, "Metadata updated successfully.", extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_product", args=(pid,))) + + add_breadcrumb(parent=product, title="Manage Metadata", top_level=False, request=request) + product_tab = Product_Tab(product, "Edit Metadata", tab="products") + return render( + request, + "dojo/edit_metadata.html", + {"formset": formset, "product_tab": product_tab}, + ) + + +class AdHocFindingView(View): + def get_product(self, product_id: int): + return get_object_or_404(Product, id=product_id) + + def get_test_type(self): + test_type, _nil = Test_Type.objects.get_or_create(name=_("Pen Test")) + return test_type + + def get_engagement(self, product: Product): + try: + return Engagement.objects.get(product=product, name=_("Ad Hoc Engagement")) + except Engagement.DoesNotExist: + return Engagement.objects.create( + name=_("Ad Hoc Engagement"), + target_start=timezone.now(), + target_end=timezone.now(), + active=False, product=product) + + def get_test(self, engagement: Engagement, test_type: Test_Type): + if test := Test.objects.filter(engagement=engagement).first(): + return test + return Test.objects.create( + engagement=engagement, + test_type=test_type, + target_start=timezone.now(), + target_end=timezone.now()) + + def create_nested_objects(self, product: Product): + engagement = self.get_engagement(product) + test_type = self.get_test_type() + return self.get_test(engagement, test_type) + + def get_initial_context(self, request: HttpRequest, test: Test): + # Get the finding form first since it is used in another place + finding_form = self.get_finding_form(request, test.engagement.product) + product_tab = Product_Tab(test.engagement.product, title=_("Add Finding"), tab="engagements") + product_tab.setEngagement(test.engagement) + return { + "form": finding_form, + "product_tab": product_tab, + "temp": False, + "tid": test.id, + "pid": test.engagement.product.id, + "form_error": False, + "jform": self.get_jira_form(request, test, finding_form=finding_form), + "gform": self.get_github_form(request, test), + } + + def get_finding_form(self, request: HttpRequest, product: Product): + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = { + "initial": {"date": timezone.now().date()}, + "req_resp": None, + "product": product, + } + # Remove the initial state on post + if request.method == "POST": + kwargs.pop("initial") + + return AdHocFindingForm(*args, **kwargs) + + def get_jira_form(self, request: HttpRequest, test: Test, finding_form: AdHocFindingForm = None): + # Determine if jira should be used + if (jira_project := jira_services.get_project(test)) is not None: + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = { + "push_all": jira_services.is_push_all_issues(test), + "prefix": "jiraform", + "jira_project": jira_project, + "finding_form": finding_form, + } + + return JIRAFindingForm(*args, **kwargs) + return None + + def get_github_form(self, request: HttpRequest, test: Test): + # Determine if github should be used + if get_system_setting("enable_github"): + # Ensure there is a github conf correctly configured for the product + config_present = GITHUB_PKey.objects.filter(product=test.engagement.product) + if config_present := config_present.exclude(git_conf_id=None): + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = { + "enabled": jira_services.is_push_all_issues(test), + "prefix": "githubform", + } + + return GITHUBFindingForm(*args, **kwargs) + return None + + def validate_status_change(self, request: HttpRequest, context: dict): + if ((context["form"]["active"].value() is False + or context["form"]["false_p"].value()) + and context["form"]["duplicate"].value() is False): + + closing_disabled = Note_Type.objects.filter(is_mandatory=True, is_active=True).count() + if closing_disabled != 0: + error_inactive = ValidationError( + _("Can not set a finding as inactive without adding all mandatory notes"), + code="inactive_without_mandatory_notes", + ) + error_false_p = ValidationError( + _("Can not set a finding as false positive without adding all mandatory notes"), + code="false_p_without_mandatory_notes", + ) + if context["form"]["active"].value() is False: + context["form"].add_error("active", error_inactive) + if context["form"]["false_p"].value(): + context["form"].add_error("false_p", error_false_p) + messages.add_message( + request, + messages.ERROR, + _("Can not set a finding as inactive or false positive without adding all mandatory notes"), + extra_tags="alert-danger") + + return request + + def process_finding_form(self, request: HttpRequest, test: Test, context: dict): + finding = None + if context["form"].is_valid(): + finding = context["form"].save(commit=False) + finding.test = test + finding.reporter = request.user + finding.numerical_severity = Finding.get_numerical_severity(finding.severity) + finding.tags = context["form"].cleaned_data["tags"] + finding.unsaved_vulnerability_ids = context["form"].cleaned_data["vulnerability_ids"].split() + finding.save() + # Save and add new endpoints + finding_helper.add_locations(finding, context["form"]) + # Save the finding at the end and return + finding.save() + + return finding, request, True + add_error_message_to_response("The form has errors, please correct them below.") + add_field_errors_to_response(context["form"]) + + return finding, request, False + + def process_jira_form(self, request: HttpRequest, finding: Finding, context: dict): + # Capture case if the jira not being enabled + if context["jform"] is None: + return request, True, False + + if context["jform"] and context["jform"].is_valid(): + # Push to Jira? + logger.debug("jira form valid") + push_to_jira = jira_services.is_push_all_issues(finding) or context["jform"].cleaned_data.get("push_to_jira") + jira_message = None + # if the jira issue key was changed, update database + new_jira_issue_key = context["jform"].cleaned_data.get("jira_issue") + if finding.has_jira_issue: + # everything in DD around JIRA integration is based on the internal id of the issue in JIRA + # instead of on the public jira issue key. + # I have no idea why, but it means we have to retrieve the issue from JIRA to get the internal JIRA id. + # we can assume the issue exist, which is already checked in the validation of the jform + if not new_jira_issue_key: + jira_services.unlink_finding(request, finding) + jira_message = "Link to JIRA issue removed successfully." + + elif new_jira_issue_key != finding.jira_issue.jira_key: + jira_services.unlink_finding(request, finding) + jira_services.link_finding(request, finding, new_jira_issue_key) + jira_message = "Changed JIRA link successfully." + else: + logger.debug("finding has no jira issue yet") + if new_jira_issue_key: + logger.debug( + "finding has no jira issue yet, but jira issue specified in request. trying to link.") + jira_services.link_finding(request, finding, new_jira_issue_key) + jira_message = "Linked a JIRA issue successfully." + # Determine if a message should be added + if jira_message: + messages.add_message( + request, messages.SUCCESS, jira_message, extra_tags="alert-success", + ) + + return request, True, push_to_jira + add_field_errors_to_response(context["jform"]) + + return request, False, False + + def process_github_form(self, request: HttpRequest, finding: Finding, context: dict): + if "githubform-push_to_github" not in request.POST: + return request, True + + if context["gform"].is_valid(): + add_external_issue(finding.id, "github") + + return request, True + add_field_errors_to_response(context["gform"]) + + return request, False + + def process_forms(self, request: HttpRequest, test: Test, context: dict): + form_success_list = [] + # Set vars for the completed forms + # Validate finding mitigation + request = self.validate_status_change(request, context) + # Check the validity of the form overall + finding, request, success = self.process_finding_form(request, test, context) + form_success_list.append(success) + request, success, push_to_jira = self.process_jira_form(request, finding, context) + form_success_list.append(success) + request, success = self.process_github_form(request, finding, context) + form_success_list.append(success) + # Determine if all forms were successful + all_forms_valid = all(form_success_list) + # Check the validity of all the forms + if all_forms_valid: + # if we're removing the "duplicate" in the edit finding screen + finding_helper.save_vulnerability_ids(finding, context["form"].cleaned_data["vulnerability_ids"].split()) + # Push things to jira if needed + finding.save(push_to_jira=push_to_jira) + # Save the burp req resp + if "request" in context["form"].cleaned_data or "response" in context["form"].cleaned_data: + burp_rr = BurpRawRequestResponse( + finding=finding, + burpRequestBase64=base64.b64encode(context["form"].cleaned_data["request"].encode()), + burpResponseBase64=base64.b64encode(context["form"].cleaned_data["response"].encode()), + ) + burp_rr.clean() + burp_rr.save() + # Add a success message + messages.add_message( + request, + messages.SUCCESS, + _("Finding added successfully."), + extra_tags="alert-success") + + return finding, request, all_forms_valid + + def get_template(self): + return "dojo/ad_hoc_findings.html" + + def get(self, request: HttpRequest, product_id: int): + # Get the initial objects + product = self.get_product(product_id) + # Make sure the user is authorized + user_has_permission_or_403(request.user, product, "add") + # Create the necessary nested objects + test = self.create_nested_objects(product) + # Set up the initial context + context = self.get_initial_context(request, test) + # Render the form + return render(request, self.get_template(), context) + + def post(self, request: HttpRequest, product_id: int): + # Get the initial objects + product = self.get_product(product_id) + # Make sure the user is authorized + user_has_permission_or_403(request.user, product, "add") + # Create the necessary nested objects + test = self.create_nested_objects(product) + # Set up the initial context + context = self.get_initial_context(request, test) + # Process the form + _, request, success = self.process_forms(request, test, context) + # Handle the case of a successful form + if success: + if "_Finished" in request.POST: + return HttpResponseRedirect(reverse("view_test", args=(test.id,))) + return HttpResponseRedirect(reverse("add_findings", args=(test.id,))) + context["form_error"] = True + # Render the form + return render(request, self.get_template(), context) + + +def engagement_presets(request, pid): + prod = get_object_or_404(Product, id=pid) + presets = Engagement_Presets.objects.filter(product=prod).all() + + product_tab = Product_Tab(prod, title=_("Engagement Presets"), tab="settings") + + return render(request, "dojo/view_presets.html", + {"product_tab": product_tab, + "presets": presets, + "prod": prod}) + + +def edit_engagement_presets(request, pid, eid): + prod = get_object_or_404(Product, id=pid) + preset = get_object_or_404(Engagement_Presets.objects.filter(product=prod), id=eid) + + product_tab = Product_Tab(prod, title=_("Edit Engagement Preset"), tab="settings") + + if request.method == "POST": + tform = EngagementPresetsForm(request.POST, instance=preset) + if tform.is_valid(): + tform.save() + messages.add_message( + request, + messages.SUCCESS, + _("Engagement Preset Successfully Updated."), + extra_tags="alert-success") + return HttpResponseRedirect(reverse("engagement_presets", args=(pid,))) + else: + tform = EngagementPresetsForm(instance=preset) + + return render(request, "dojo/edit_presets.html", + {"product_tab": product_tab, + "tform": tform, + "prod": prod}) + + +def add_engagement_presets(request, pid): + prod = get_object_or_404(Product, id=pid) + if request.method == "POST": + tform = EngagementPresetsForm(request.POST) + if tform.is_valid(): + form_copy = tform.save(commit=False) + form_copy.product = prod + form_copy.save() + tform.save_m2m() + messages.add_message( + request, + messages.SUCCESS, + _("Engagement Preset Successfully Created."), + extra_tags="alert-success") + return HttpResponseRedirect(reverse("engagement_presets", args=(pid,))) + else: + tform = EngagementPresetsForm() + + product_tab = Product_Tab(prod, title=_("New Engagement Preset"), tab="settings") + return render(request, "dojo/new_params.html", {"tform": tform, "pid": pid, "product_tab": product_tab}) + + +def delete_engagement_presets(request, pid, eid): + prod = get_object_or_404(Product, id=pid) + preset = get_object_or_404(Engagement_Presets.objects.filter(product=prod), id=eid) + form = DeleteEngagementPresetsForm(instance=preset) + + if request.method == "POST": + if "id" in request.POST: + form = DeleteEngagementPresetsForm(request.POST, instance=preset) + if form.is_valid(): + preset.delete() + messages.add_message(request, + messages.SUCCESS, + _("Engagement presets and engagement relationships removed."), + extra_tags="alert-success") + return HttpResponseRedirect(reverse("engagement_presets", args=(pid,))) + + rels = ["Previewing the relationships has been disabled.", ""] + display_preview = get_setting("DELETE_PREVIEW") + if display_preview: + collector = NestedObjects(using=DEFAULT_DB_ALIAS) + collector.collect([preset]) + rels = collector.nested() + + product_tab = Product_Tab(prod, title=_("Delete Engagement Preset"), tab="settings") + return render(request, "dojo/delete_presets.html", + {"product": product, + "form": form, + "product_tab": product_tab, + "rels": rels, + }) + + +def edit_notifications(request, pid): + prod = get_object_or_404(Product, id=pid) + if request.method == "POST": + product_notifications = Notifications.objects.filter(user=request.user).filter(product=prod).first() + if not product_notifications: + product_notifications = Notifications(user=request.user, product=prod) + logger.debug("no existing product notifications found") + else: + logger.debug("existing product notifications found") + + form = ProductNotificationsForm(request.POST, instance=product_notifications) + + if form.is_valid(): + form.save() + messages.add_message(request, + messages.SUCCESS, + _("Notification settings updated."), + extra_tags="alert-success") + + return HttpResponseRedirect(reverse("view_product", args=(pid,))) + + +def add_product_authorized_users(request, pid): + product = get_object_or_404(Product, pk=pid) + user_has_permission_or_403(request.user, product, Permissions.Product_Manage_Members) + page_name = _("Add Authorized Users") + form = Add_Product_AuthorizedUsersForm(product=product) + if request.method == "POST": + form = Add_Product_AuthorizedUsersForm(request.POST, product=product) + if form.is_valid(): + users = form.cleaned_data["users"] + product.authorized_users.add(*users) + messages.add_message( + request, messages.SUCCESS, + _("Added %(count)d user(s) to authorized users.") % {"count": len(users)}, + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_product", args=(pid,))) + product_tab = Product_Tab(product, title=page_name, tab="settings") + return render(request, "dojo/new_product_authorized_users.html", { + "name": page_name, + "product": product, + "form": form, + "product_tab": product_tab, + }) + + +def delete_product_authorized_user(request, pid, user_id): + product = get_object_or_404(Product, pk=pid) + user_has_permission_or_403(request.user, product, Permissions.Product_Manage_Members) + if request.method != "POST": + raise PermissionDenied + user = get_object_or_404(Dojo_User, pk=user_id) + product.authorized_users.remove(user) + messages.add_message( + request, messages.SUCCESS, + _("Removed %(username)s from authorized users.") % {"username": user.username}, + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_product", args=(pid,))) + + +def add_api_scan_configuration(request, pid): + product = get_object_or_404(Product, id=pid) + if request.method == "POST": + form = Product_API_Scan_ConfigurationForm(request.POST) + if form.is_valid(): + product_api_scan_configuration = form.save(commit=False) + product_api_scan_configuration.product = product + try: + api = create_API(product_api_scan_configuration.tool_configuration) + if api and hasattr(api, "test_product_connection"): + result = api.test_product_connection(product_api_scan_configuration) + messages.add_message(request, + messages.SUCCESS, + _("API connection successful with message: %(result)s.") % {"result": result}, + extra_tags="alert-success") + product_api_scan_configuration.save() + messages.add_message(request, + messages.SUCCESS, + _("API Scan Configuration added successfully."), + extra_tags="alert-success") + if "add_another" in request.POST: + return HttpResponseRedirect(reverse("add_api_scan_configuration", args=(pid,))) + return HttpResponseRedirect(reverse("view_api_scan_configurations", args=(pid,))) + except Exception as e: + logger.exception("Unable to add API Scan Configuration") + messages.add_message(request, + messages.ERROR, + str(e), + extra_tags="alert-danger") + else: + form = Product_API_Scan_ConfigurationForm() + + product_tab = Product_Tab(product, title=_("Add API Scan Configuration"), tab="settings") + + return render(request, + "dojo/add_product_api_scan_configuration.html", + {"form": form, + "product_tab": product_tab, + "product": product, + "api_scan_configuration_hints": get_api_scan_configuration_hints(), + }) + + +def view_api_scan_configurations(request, pid): + product_api_scan_configurations = Product_API_Scan_Configuration.objects.filter(product=pid) + + product_tab = Product_Tab(get_object_or_404(Product, id=pid), title=_("API Scan Configurations"), tab="settings") + return render(request, + "dojo/view_product_api_scan_configurations.html", + { + "product_api_scan_configurations": product_api_scan_configurations, + "product_tab": product_tab, + "pid": pid, + }) + + +def edit_api_scan_configuration(request, pid, pascid): + product_api_scan_configuration = get_object_or_404(Product_API_Scan_Configuration, id=pascid) + + if product_api_scan_configuration.product.pk != int( + pid): # user is trying to edit Tool Configuration from another product (trying to by-pass auth) + raise Http404 + + if request.method == "POST": + form = Product_API_Scan_ConfigurationForm(request.POST, instance=product_api_scan_configuration) + if form.is_valid(): + try: + form_copy = form.save(commit=False) + api = create_API(form_copy.tool_configuration) + if api and hasattr(api, "test_product_connection"): + result = api.test_product_connection(form_copy) + messages.add_message(request, + messages.SUCCESS, + _("API connection successful with message: %(result)s.") % {"result": result}, + extra_tags="alert-success") + form.save() + + messages.add_message(request, + messages.SUCCESS, + _("API Scan Configuration successfully updated."), + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_api_scan_configurations", args=(pid,))) + except Exception as e: + logger.info(e) + messages.add_message(request, + messages.ERROR, + str(e), + extra_tags="alert-danger") + else: + form = Product_API_Scan_ConfigurationForm(instance=product_api_scan_configuration) + + product_tab = Product_Tab(get_object_or_404(Product, id=pid), title=_("Edit API Scan Configuration"), tab="settings") + return render(request, + "dojo/edit_product_api_scan_configuration.html", + { + "form": form, + "product_tab": product_tab, + "api_scan_configuration_hints": get_api_scan_configuration_hints(), + }) + + +def delete_api_scan_configuration(request, pid, pascid): + product_api_scan_configuration = get_object_or_404(Product_API_Scan_Configuration, id=pascid) + + if product_api_scan_configuration.product.pk != int( + pid): # user is trying to delete Tool Configuration from another product (trying to by-pass auth) + raise Http404 + + if request.method == "POST": + form = Product_API_Scan_ConfigurationForm(request.POST) + product_api_scan_configuration.delete() + messages.add_message(request, + messages.SUCCESS, + _("API Scan Configuration deleted."), + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_api_scan_configurations", args=(pid,))) + form = DeleteProduct_API_Scan_ConfigurationForm(instance=product_api_scan_configuration) + + product_tab = Product_Tab(get_object_or_404(Product, id=pid), title=_("Delete Tool Configuration"), tab="settings") + return render(request, + "dojo/delete_product_api_scan_configuration.html", + { + "form": form, + "product_tab": product_tab, + }) diff --git a/dojo/product/views.py b/dojo/product/views.py index 6f5afb4e9fa..70ad404054a 100644 --- a/dojo/product/views.py +++ b/dojo/product/views.py @@ -1,1822 +1,4 @@ -# # product -import base64 -import calendar as tcalendar -import logging -from collections import OrderedDict -from datetime import date, datetime, timedelta -from functools import partial -from math import ceil - -from dateutil.relativedelta import relativedelta -from django.conf import settings -from django.contrib import messages -from django.contrib.admin.utils import NestedObjects -from django.contrib.postgres.aggregates import StringAgg -from django.core.exceptions import PermissionDenied, ValidationError -from django.db import DEFAULT_DB_ALIAS, connection -from django.db.models import Count, DateField, F, OuterRef, Prefetch, Q, Subquery, Sum, Value -from django.db.models.functions import Coalesce -from django.db.models.query import QuerySet -from django.http import Http404, HttpRequest, HttpResponseRedirect, JsonResponse -from django.shortcuts import get_object_or_404, render -from django.urls import reverse -from django.utils import timezone -from django.utils.translation import gettext as _ -from django.views import View -from github import Github - -import dojo.finding.helper as finding_helper -from dojo.authorization.authorization import user_has_permission_or_403 -from dojo.authorization.roles_permissions import Permissions -from dojo.components.sql_group_concat import Sql_GroupConcat -from dojo.filters import ( - EngagementFilter, - EngagementFilterWithoutObjectLookups, - MetricsEndpointFilter, - MetricsEndpointFilterWithoutObjectLookups, - MetricsFindingFilter, - MetricsFindingFilterWithoutObjectLookups, - ProductComponentFilter, - ProductEngagementFilter, - ProductEngagementFilterWithoutObjectLookups, - ProductFilter, - ProductFilterWithoutObjectLookups, -) -from dojo.forms import ( - Add_Product_AuthorizedUsersForm, - AdHocFindingForm, - AppAnalysisForm, - DeleteAppAnalysisForm, - DeleteEngagementPresetsForm, - DeleteProduct_API_Scan_ConfigurationForm, - DeleteProductForm, - DojoMetaFormSet, - EngagementPresetsForm, - EngForm, - GITHUB_Product_Form, - GITHUBFindingForm, - JIRAEngagementForm, - JIRAFindingForm, - JIRAProjectForm, - Product_API_Scan_ConfigurationForm, - ProductForm, - ProductNotificationsForm, - SLA_Configuration, -) -from dojo.jira import services as jira_services -from dojo.labels import get_labels -from dojo.models import ( - App_Analysis, - Benchmark_Product_Summary, - Benchmark_Type, - BurpRawRequestResponse, - Dojo_User, - DojoMeta, - Endpoint, - Endpoint_Status, - Engagement, - Engagement_Presets, - Finding, - GITHUB_PKey, - Languages, - Note_Type, - Notifications, - Product, - Product_API_Scan_Configuration, - Product_Type, - System_Settings, - Test, - Test_Import, - Test_Type, -) -from dojo.product.queries import ( - get_authorized_products, -) -from dojo.product_type.queries import ( - get_authorized_product_types, -) -from dojo.query_utils import build_count_subquery -from dojo.templatetags.display_tags import asvs_calc_level -from dojo.tool_config.factory import create_API -from dojo.tools.factory import get_api_scan_configuration_hints -from dojo.utils import ( - Product_Tab, - add_breadcrumb, - add_error_message_to_response, - add_external_issue, - add_field_errors_to_response, - async_delete, - calculate_finding_age, - get_enabled_notifications_list, - get_open_findings_burndown, - get_page_items, - get_punchcard_data, - get_setting, - get_system_setting, - get_zero_severity_level, - queryset_check, - sum_by_severity_level, -) - -logger = logging.getLogger(__name__) - -labels = get_labels() - - -def product(request): - prods = get_authorized_products("view") - # perform all stuff for filtering and pagination first, before annotation/prefetching - # otherwise the paginator will perform all the annotations/prefetching already only to count the total number of records - # see https://code.djangoproject.com/ticket/23771 and https://code.djangoproject.com/ticket/25375 - - name_words = prods.values_list("name", flat=True) - base_findings = Finding.objects.filter(test__engagement__product_id=OuterRef("pk"), active=True) - prods = prods.annotate( - findings_count=Coalesce( - build_count_subquery(base_findings, group_field="test__engagement__product_id"), Value(0), - ), - ) - if settings.V3_FEATURE_LOCATIONS: - prods = prods.annotate( - location_host_count=Count("locations__location__url__host", distinct=True), - location_count=Count("locations", distinct=True), - ) - - filter_string_matching = get_system_setting("filter_string_matching", False) - filter_class = ProductFilterWithoutObjectLookups if filter_string_matching else ProductFilter - prod_filter = filter_class(request.GET, queryset=prods, user=request.user) - prod_list = get_page_items(request, prod_filter.qs, 25) - - # perform annotation/prefetching by replacing the queryset in the page with an annotated/prefetched queryset. - prod_list.object_list = prefetch_for_product(prod_list.object_list) - - # Get benchmark types for the template - benchmark_types = Benchmark_Type.objects.filter(enabled=True).order_by("name") - - add_breadcrumb(title=str(labels.ASSET_READ_LIST_LABEL), top_level=not len(request.GET), request=request) - - return render(request, "dojo/product.html", { - "prod_list": prod_list, - "prod_filter": prod_filter, - "name_words": sorted(set(name_words)), - "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), - "benchmark_types": benchmark_types, - "user": request.user}) - - -def prefetch_for_product(prods): - # old code can arrive here with prods being a list because the query was already executed - if not isinstance(prods, QuerySet): - logger.debug("unable to prefetch because query was already executed") - return prods - - prefetched_prods = prods.select_related("team_manager", "product_manager", "technical_contact").prefetch_related( - "tags", - "jira_project_set__jira_instance", - ) - - engagements = Engagement.objects.filter(product_id=OuterRef("pk")) - count_subquery = partial(build_count_subquery, group_field="product_id") - prefetched_prods = prefetched_prods.annotate( - active_engagement_count=Coalesce(count_subquery(engagements.filter(active=True)), Value(0)), - closed_engagement_count=Coalesce(count_subquery(engagements.filter(active=False)), Value(0)), - last_engagement_date=Subquery( - engagements.order_by("-target_start").values("target_start")[:1], output_field=DateField(), - ), - ) - - base_findings = Finding.objects.filter(test__engagement__product_id=OuterRef("pk")) - count_subquery = partial(build_count_subquery, group_field="test__engagement__product_id") - prefetched_prods = prefetched_prods.annotate( - active_finding_count=Coalesce(count_subquery(base_findings.filter(active=True)), Value(0)), - active_verified_finding_count=Coalesce( - count_subquery(base_findings.filter(active=True, verified=True)), - Value(0), - ), - ) - prefetched_prods = prefetched_prods.annotate( - total_reimport_count=Coalesce( - count_subquery( - Test_Import.objects.filter(test__engagement__product_id=OuterRef("pk"), type=Test_Import.REIMPORT_TYPE), - ), - Value(0), - ), - ) - - # TODO: Delete this after the move to Locations - if not settings.V3_FEATURE_LOCATIONS: - active_endpoint_qs = Endpoint.objects.filter( - status_endpoint__mitigated=False, - status_endpoint__false_positive=False, - status_endpoint__out_of_scope=False, - status_endpoint__risk_accepted=False, - ).distinct() - - prefetched_prods = prefetched_prods.prefetch_related( - Prefetch("endpoint_set", queryset=active_endpoint_qs, to_attr="active_endpoints"), - ) - - if get_system_setting("enable_github"): - prefetched_prods = prefetched_prods.prefetch_related( - Prefetch( - "github_pkey_set", queryset=GITHUB_PKey.objects.all().select_related("git_conf"), to_attr="github_confs", - ), - ) - return prefetched_prods - - -def iso_to_gregorian(iso_year, iso_week, iso_day): - jan4 = date(iso_year, 1, 4) - start = jan4 - timedelta(days=jan4.isoweekday() - 1) - return start + timedelta(weeks=iso_week - 1, days=iso_day - 1) - - -def view_product(request, pid): - prod_query = Product.objects.all().select_related("product_manager", "technical_contact", "team_manager", "sla_configuration") - prod = get_object_or_404(prod_query, id=pid) - authorized_users = prod.authorized_users.order_by("first_name", "last_name", "username") - personal_notifications_form = ProductNotificationsForm( - instance=Notifications.objects.filter(user=request.user).filter(product=prod).first()) - langSummary = Languages.objects.filter(product=prod).aggregate(Sum("files"), Sum("code"), Count("files")) - languages = Languages.objects.filter(product=prod).order_by("-code").select_related("language") - app_analysis = App_Analysis.objects.filter(product=prod).order_by("name") - benchmarks = Benchmark_Product_Summary.objects.filter(product=prod, publish=True, - benchmark_type__enabled=True).order_by("benchmark_type__name") - sla = SLA_Configuration.objects.filter(id=prod.sla_configuration_id).first() - benchAndPercent = [] - for i in range(len(benchmarks)): - desired_level, total, total_pass, total_wait, total_fail, _total_viewed = asvs_calc_level(benchmarks[i]) - - success_percent = round((float(total_pass) / float(total)) * 100, 2) - waiting_percent = round((float(total_wait) / float(total)) * 100, 2) - fail_percent = round(100 - success_percent - waiting_percent, 2) - benchAndPercent.append({ - "id": benchmarks[i].benchmark_type.id, - "name": benchmarks[i].benchmark_type, - "level": desired_level, - "success": {"count": total_pass, "percent": success_percent}, - "waiting": {"count": total_wait, "percent": waiting_percent}, - "fail": {"count": total_fail, "percent": fail_percent}, - "pass": total_pass + total_fail, - "total": total, - }) - system_settings = System_Settings.objects.get() - - product_metadata = dict(prod.product_meta.order_by("name").values_list("name", "value")) - - open_findings = Finding.objects.filter(test__engagement__product=prod, - false_p=False, - active=True, - duplicate=False, - out_of_scope=False).order_by("numerical_severity").values( - "severity").annotate(count=Count("severity")) - - critical = 0 - high = 0 - medium = 0 - low = 0 - info = 0 - - for v in open_findings: - if v["severity"] == "Critical": - critical = v["count"] - elif v["severity"] == "High": - high = v["count"] - elif v["severity"] == "Medium": - medium = v["count"] - elif v["severity"] == "Low": - low = v["count"] - elif v["severity"] == "Info": - info = v["count"] - - total = critical + high + medium + low + info - - product_tab = Product_Tab(prod, title=str(labels.ASSET_LABEL), tab="overview") - return render(request, "dojo/view_product_details.html", { - "prod": prod, - "product_tab": product_tab, - "product_metadata": product_metadata, - "critical": critical, - "high": high, - "medium": medium, - "low": low, - "info": info, - "total": total, - "user": request.user, - "languages": languages, - "langSummary": langSummary, - "app_analysis": app_analysis, - "system_settings": system_settings, - "benchmarks_percents": benchAndPercent, - "benchmarks": benchmarks, - "benchmark_type": product_tab.benchmark_type, - "authorized_users": authorized_users, - "personal_notifications_form": personal_notifications_form, - "enabled_notifications": get_enabled_notifications_list(), - "sla": sla}) - - -def view_product_components(request, pid): - prod = get_object_or_404(Product, id=pid) - product_tab = Product_Tab(prod, title=str(labels.ASSET_LABEL), tab="components") - separator = ", " - - # Get components ordered by component_name and concat component versions to the same row - if connection.vendor == "postgresql": - component_query = Finding.objects.filter(test__engagement__product__id=pid).values("component_name").order_by( - "component_name").annotate( - component_version=StringAgg("component_version", delimiter=separator, distinct=True, default=Value(""))) - else: - component_query = Finding.objects.filter(test__engagement__product__id=pid).values("component_name") - component_query = component_query.annotate( - component_version=Sql_GroupConcat("component_version", separator=separator, distinct=True)) - - # Append finding counts - component_query = component_query.annotate(total=Count("id")).order_by("component_name", "component_version") - component_query = component_query.annotate(active=Count("id", filter=Q(active=True))) - component_query = component_query.annotate(duplicate=(Count("id", filter=Q(duplicate=True)))) - - # Default sort by total descending - component_query = component_query.order_by("-total") - - comp_filter = ProductComponentFilter(request.GET, queryset=component_query) - result = get_page_items(request, comp_filter.qs, 25) - - # Filter out None values for auto-complete - component_words = component_query.exclude(component_name__isnull=True).values_list("component_name", flat=True) - - return render(request, "dojo/product_components.html", { - "prod": prod, - "filter": comp_filter, - "product_tab": product_tab, - "result": result, - "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), - "component_words": sorted(set(component_words)), - }) - - -def identify_view(request): - get_data = request.GET - view = get_data.get("type", None) - if view: - # value of view is reflected in the template, make sure it's valid - # although any XSS should be catch by django autoescape, we see people sometimes using '|safe'... - if view in {"Endpoint", "Finding"}: - return view - msg = 'invalid view, view must be "Endpoint" or "Finding"' - raise ValueError(msg) - if get_data.get("finding__severity", None) or get_data.get("false_positive", None): - return "Endpoint" - referer = request.META.get("HTTP_REFERER", None) - if referer: - if referer.find("type=Endpoint") > -1: - return "Endpoint" - return "Finding" - - -def finding_queries(request, prod): - filters = {} - findings_query = Finding.objects.filter(test__engagement__product=prod) - # prefetch only what's needed to avoid lots of repeated queries - findings_query = findings_query.prefetch_related( - # 'test__engagement', - # 'test__engagement__risk_acceptance', - # 'found_by', - # 'test', - # 'test__test_type', - # 'risk_acceptance_set', - "reporter") - filter_string_matching = get_system_setting("filter_string_matching", False) - finding_filter_class = MetricsFindingFilterWithoutObjectLookups if filter_string_matching else MetricsFindingFilter - findings = finding_filter_class(request.GET, queryset=findings_query, pid=prod) - findings_qs = queryset_check(findings) - filters["form"] = findings.form - - try: - # logger.debug(findings_qs.query) - start_date = findings_qs.earliest("date").date - start_date = datetime( - start_date.year, - start_date.month, start_date.day, - tzinfo=timezone.get_current_timezone()) - end_date = findings_qs.latest("date").date - end_date = datetime( - end_date.year, - end_date.month, end_date.day, - tzinfo=timezone.get_current_timezone()) - except Exception as e: - logger.debug(e) - start_date = timezone.now() - end_date = timezone.now() - week = end_date - timedelta(days=7) # seven days and /newer are considered "new" - - filters["accepted"] = findings_qs.filter(finding_helper.ACCEPTED_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") - filters["verified"] = findings_qs.filter(finding_helper.VERIFIED_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") - filters["new_verified"] = findings_qs.filter(finding_helper.VERIFIED_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") - filters["open"] = findings_qs.filter(finding_helper.OPEN_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") - filters["inactive"] = findings_qs.filter(finding_helper.INACTIVE_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") - # Filter closed findings by mitigated date (not discovery date). - # Use timezone.now() as the upper bound so findings closed after the latest - # discovery date are not excluded from the counter. - filters["closed"] = findings_qs.filter(finding_helper.CLOSED_FINDINGS_QUERY).filter(mitigated__range=[start_date, timezone.now()], mitigated__isnull=False).order_by("mitigated") - filters["false_positive"] = findings_qs.filter(finding_helper.FALSE_POSITIVE_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") - filters["out_of_scope"] = findings_qs.filter(finding_helper.OUT_OF_SCOPE_FINDINGS_QUERY).filter(date__range=[start_date, end_date]).order_by("date") - filters["all"] = findings_qs.order_by("date") - filters["open_vulns"] = findings_qs.filter(finding_helper.OPEN_FINDINGS_QUERY).filter( - cwe__isnull=False, - ).order_by("cwe").values( - "cwe", - ).annotate( - count=Count("cwe"), - ) - - filters["all_vulns"] = findings_qs.filter( - duplicate=False, - cwe__isnull=False, - ).order_by("cwe").values( - "cwe", - ).annotate( - count=Count("cwe"), - ) - - filters["start_date"] = start_date - filters["end_date"] = end_date - filters["week"] = week - - return filters - - -# TODO: Delete this after the move to Locations -def endpoint_queries(request, prod): - filters = {} - endpoints_query = Endpoint_Status.objects.filter(finding__test__engagement__product=prod, - finding__severity__in=( - "Critical", "High", "Medium", "Low", "Info")).prefetch_related( - "finding__test__engagement", - "finding__test__engagement__risk_acceptance", - "finding__risk_acceptance_set", - "finding__reporter").annotate(severity=F("finding__severity")) - filter_string_matching = get_system_setting("filter_string_matching", False) - filter_class = MetricsEndpointFilterWithoutObjectLookups if filter_string_matching else MetricsEndpointFilter - endpoints = filter_class(request.GET, queryset=endpoints_query) - endpoints_qs = queryset_check(endpoints) - filters["form"] = endpoints.form - - if not endpoints_qs and not endpoints_query: - messages.add_message( - request, - messages.WARNING, - _("No Endpoints match the current filters."), - extra_tags="alert-danger") - try: - start_date = endpoints_qs.earliest("date").date - start_date = datetime(start_date.year, - start_date.month, start_date.day, - tzinfo=timezone.get_current_timezone()) - end_date = endpoints_qs.latest("date").date - end_date = datetime(end_date.year, - end_date.month, end_date.day, - tzinfo=timezone.get_current_timezone()) - except: - start_date = timezone.now() - end_date = timezone.now() - week = end_date - timedelta(days=7) # seven days and /newnewer are considered "new" - - filters["accepted"] = endpoints_qs.filter(date__range=[start_date, end_date], - risk_accepted=True).order_by("date") - filters["verified"] = endpoints_qs.filter(date__range=[start_date, end_date], - false_positive=False, - mitigated=True, - out_of_scope=False).order_by("date") - filters["new_verified"] = endpoints_qs.filter(date__range=[week, end_date], - false_positive=False, - mitigated=True, - out_of_scope=False).order_by("date") - filters["open"] = endpoints_qs.filter(date__range=[start_date, end_date], - mitigated=False, - finding__active=True) - filters["inactive"] = endpoints_qs.filter(date__range=[start_date, end_date], - mitigated=True) - filters["closed"] = endpoints_qs.filter(date__range=[start_date, end_date], - mitigated=True) - filters["false_positive"] = endpoints_qs.filter(date__range=[start_date, end_date], - false_positive=True) - filters["out_of_scope"] = endpoints_qs.filter(date__range=[start_date, end_date], - out_of_scope=True) - filters["all"] = endpoints_qs - filters["open_vulns"] = endpoints_qs.filter( - false_positive=False, - out_of_scope=False, - mitigated=True, - finding__cwe__isnull=False, - ).order_by("finding__cwe").values( - "finding__cwe", - ).annotate( - count=Count("finding__cwe"), - ).annotate( - cwe=F("finding__cwe"), - ) - - filters["all_vulns"] = endpoints_qs.filter( - finding__cwe__isnull=False, - ).order_by("finding__cwe").values( - "finding__cwe", - ).annotate( - count=Count("finding__cwe"), - ).annotate( - cwe=F("finding__cwe"), - ) - - filters["start_date"] = start_date - filters["end_date"] = end_date - filters["week"] = week - - return filters - - -def view_product_metrics(request, pid): - prod = get_object_or_404(Product, id=pid) - engs = Engagement.objects.filter(product=prod, active=True) - view = identify_view(request) - - filter_string_matching = get_system_setting("filter_string_matching", False) - filter_class = EngagementFilterWithoutObjectLookups if filter_string_matching else EngagementFilter - result = filter_class( - request.GET, - queryset=Engagement.objects.filter(product=prod, active=False).order_by("-target_end")) - - inactive_engs_page = get_page_items(request, result.qs, 10) - - filters = {} - if view == "Finding": - filters = finding_queries(request, prod) - # TODO: Delete this after the move to Locations - elif view == "Endpoint": - filters = endpoint_queries(request, prod) - - start_date = timezone.make_aware(datetime.combine(filters["start_date"], datetime.min.time())) - end_date = filters["end_date"] - - r = relativedelta(end_date, start_date) - weeks_between = ceil((((r.years * 12) + r.months) * 4.33) + (r.days / 7)) - if weeks_between <= 0: - weeks_between += 2 - - punchcard, ticks = get_punchcard_data(filters.get("open", None), start_date, weeks_between, view) - - add_breadcrumb(parent=prod, top_level=False, request=request) - - # An ordered dict does not make sense here. - open_close_weekly = OrderedDict() - severity_weekly = OrderedDict() - critical_weekly = OrderedDict() - high_weekly = OrderedDict() - medium_weekly = OrderedDict() - open_objs_by_age = {} - - open_objs_by_severity = get_zero_severity_level() - closed_objs_by_severity = get_zero_severity_level() - accepted_objs_by_severity = get_zero_severity_level() - - # Optimization: Make all queries lists, and only pull values of fields for metrics based calculations - open_vulnerabilities = list(filters["open_vulns"].values("cwe", "count")) - all_vulnerabilities = list(filters["all_vulns"].values("cwe", "count")) - - verified_objs_by_severity = list(filters.get("verified").values("severity")) - inactive_objs_by_severity = list(filters.get("inactive").values("severity")) - false_positive_objs_by_severity = list(filters.get("false_positive").values("severity")) - out_of_scope_objs_by_severity = list(filters.get("out_of_scope").values("severity")) - new_objs_by_severity = list(filters.get("new_verified").values("severity")) - all_objs_by_severity = list(filters.get("all").values("severity")) - - all_findings = list(filters.get("all", []).values("id", "date", "severity")) - open_findings = list(filters.get("open", []).values("id", "date", "mitigated", "severity")) - # Include mitigated date for closed findings to group by when they were closed, not discovered - closed_findings = list(filters.get("closed", []).values("id", "date", "mitigated", "severity")) - accepted_findings = list(filters.get("accepted", []).values("id", "date", "severity")) - - """ - Optimization: Create dictionaries in the structure of { finding_id: True } for index based search - Previously the for-loop below used "if finding in open_findings" -- an average O(n^2) time complexity - This allows for "if open_findings.get(finding_id, None)" -- an average O(n) time complexity - """ - open_findings_dict = {f.get("id"): True for f in open_findings} - closed_findings_dict = {f.get("id"): True for f in closed_findings} - accepted_findings_dict = {f.get("id"): True for f in accepted_findings} - - for finding in all_findings: - iso_cal = finding.get("date").isocalendar() - date = iso_to_gregorian(iso_cal[0], iso_cal[1], 1) - html_date = date.strftime("%m/%d
%Y") - unix_timestamp = (tcalendar.timegm(date.timetuple()) * 1000) - - # Open findings - if open_findings_dict.get(finding.get("id", None)): - if unix_timestamp not in critical_weekly: - critical_weekly[unix_timestamp] = {"count": 0, "week": html_date} - if unix_timestamp not in high_weekly: - high_weekly[unix_timestamp] = {"count": 0, "week": html_date} - if unix_timestamp not in medium_weekly: - medium_weekly[unix_timestamp] = {"count": 0, "week": html_date} - - if unix_timestamp in open_close_weekly: - open_close_weekly[unix_timestamp]["open"] += 1 - else: - open_close_weekly[unix_timestamp] = {"closed": 0, "open": 1, "accepted": 0} - open_close_weekly[unix_timestamp]["week"] = html_date - - if view in {"Finding", "Endpoint"}: - severity = finding.get("severity") - - finding_age = calculate_finding_age(finding) - if open_objs_by_age.get(finding_age): - open_objs_by_age[finding_age] += 1 - else: - open_objs_by_age[finding_age] = 1 - - if unix_timestamp in severity_weekly: - if severity in severity_weekly[unix_timestamp]: - severity_weekly[unix_timestamp][severity] += 1 - else: - severity_weekly[unix_timestamp][severity] = 1 - else: - severity_weekly[unix_timestamp] = get_zero_severity_level() - severity_weekly[unix_timestamp][severity] = 1 - severity_weekly[unix_timestamp]["week"] = html_date - - if severity == "Critical": - if unix_timestamp in critical_weekly: - critical_weekly[unix_timestamp]["count"] += 1 - else: - critical_weekly[unix_timestamp] = {"count": 1, "week": html_date} - elif severity == "High": - if unix_timestamp in high_weekly: - high_weekly[unix_timestamp]["count"] += 1 - else: - high_weekly[unix_timestamp] = {"count": 1, "week": html_date} - elif severity == "Medium": - if unix_timestamp in medium_weekly: - medium_weekly[unix_timestamp]["count"] += 1 - else: - medium_weekly[unix_timestamp] = {"count": 1, "week": html_date} - # Optimization: count severity level on server side - if open_objs_by_severity.get(finding.get("severity")) is not None: - open_objs_by_severity[finding.get("severity")] += 1 - - # Close findings - group by mitigated date, not discovery date - elif closed_findings_dict.get(finding.get("id", None)): - # Find the closed finding to get its mitigated date - closed_finding = next((f for f in closed_findings if f.get("id") == finding.get("id")), None) - if closed_finding and closed_finding.get("mitigated"): - # Use mitigated date for grouping closed findings - mitigated_date = closed_finding.get("mitigated") - mitigated_date_only = mitigated_date.date() if isinstance(mitigated_date, datetime) else mitigated_date - iso_cal = mitigated_date_only.isocalendar() - mitigated_week_start = iso_to_gregorian(iso_cal[0], iso_cal[1], 1) - mitigated_html_date = mitigated_week_start.strftime("%m/%d
%Y") - mitigated_unix_timestamp = (tcalendar.timegm(mitigated_week_start.timetuple()) * 1000) - - if mitigated_unix_timestamp in open_close_weekly: - open_close_weekly[mitigated_unix_timestamp]["closed"] += 1 - else: - open_close_weekly[mitigated_unix_timestamp] = {"closed": 1, "open": 0, "accepted": 0} - open_close_weekly[mitigated_unix_timestamp]["week"] = mitigated_html_date - elif unix_timestamp in open_close_weekly: - # Fallback to discovery date if mitigated date is not available - open_close_weekly[unix_timestamp]["closed"] += 1 - else: - # Fallback to discovery date if mitigated date is not available - open_close_weekly[unix_timestamp] = {"closed": 1, "open": 0, "accepted": 0} - open_close_weekly[unix_timestamp]["week"] = html_date - # Optimization: count severity level on server side - if closed_objs_by_severity.get(finding.get("severity")) is not None: - closed_objs_by_severity[finding.get("severity")] += 1 - - # Risk Accepted findings - if accepted_findings_dict.get(finding.get("id", None)): - if unix_timestamp in open_close_weekly: - open_close_weekly[unix_timestamp]["accepted"] += 1 - else: - open_close_weekly[unix_timestamp] = {"closed": 0, "open": 0, "accepted": 1} - open_close_weekly[unix_timestamp]["week"] = html_date - # Optimization: count severity level on server side - if accepted_objs_by_severity.get(finding.get("severity")) is not None: - accepted_objs_by_severity[finding.get("severity")] += 1 - - tests = Test.objects.filter(engagement__product=prod).prefetch_related("finding_set", "test_type") - verified_finding_subquery = build_count_subquery( - Finding.objects.filter(test=OuterRef("pk"), verified=True), - group_field="test_id", - ) - tests = tests.annotate(verified_finding_count=Coalesce(verified_finding_subquery, Value(0))) - - test_data = {} - for t in tests: - if t.test_type.name in test_data: - test_data[t.test_type.name] += t.verified_finding_count - else: - test_data[t.test_type.name] = t.verified_finding_count - - # Optimization: Format Open/Total CWE vulnerabilities graph data here, instead of template - open_vulnerabilities = [["CWE-" + str(f.get("cwe")), f.get("count")] for f in open_vulnerabilities] - all_vulnerabilities = [["CWE-" + str(f.get("cwe")), f.get("count")] for f in all_vulnerabilities] - - product_tab = Product_Tab(prod, title=str(labels.ASSET_LABEL), tab="metrics") - - return render(request, "dojo/product_metrics.html", { - "prod": prod, - "product_tab": product_tab, - "engs": engs, - "inactive_engs": inactive_engs_page, - "view": view, - "verified_objs": len(verified_objs_by_severity), - "verified_objs_by_severity": sum_by_severity_level(verified_objs_by_severity), - "open_objs": len(open_findings), - "open_objs_by_severity": open_objs_by_severity, - "open_objs_by_age": open_objs_by_age, - "inactive_objs": len(inactive_objs_by_severity), - "inactive_objs_by_severity": sum_by_severity_level(inactive_objs_by_severity), - "closed_objs": len(closed_findings), - "closed_objs_by_severity": closed_objs_by_severity, - "false_positive_objs": len(false_positive_objs_by_severity), - "false_positive_objs_by_severity": sum_by_severity_level(false_positive_objs_by_severity), - "out_of_scope_objs": len(out_of_scope_objs_by_severity), - "out_of_scope_objs_by_severity": sum_by_severity_level(out_of_scope_objs_by_severity), - "accepted_objs": len(accepted_findings), - "accepted_objs_by_severity": accepted_objs_by_severity, - "new_objs": len(new_objs_by_severity), - "new_objs_by_severity": sum_by_severity_level(new_objs_by_severity), - "all_objs": len(all_objs_by_severity), - "all_objs_by_severity": sum_by_severity_level(all_objs_by_severity), - "form": filters.get("form", None), - "reset_link": reverse("view_product_metrics", args=(prod.id,)) + "?type=" + view, - "open_vulnerabilities_count": len(open_vulnerabilities), - "open_vulnerabilities": open_vulnerabilities, - "all_vulnerabilities_count": len(all_vulnerabilities), - "all_vulnerabilities": all_vulnerabilities, - "start_date": start_date, - "punchcard": punchcard, - "ticks": ticks, - "open_close_weekly": open_close_weekly, - "severity_weekly": severity_weekly, - "critical_weekly": critical_weekly, - "high_weekly": high_weekly, - "medium_weekly": medium_weekly, - "test_data": test_data, - "user": request.user}) - - -def async_burndown_metrics(request, pid): - prod = get_object_or_404(Product, id=pid) - open_findings_burndown = get_open_findings_burndown(prod) - - return JsonResponse({ - "critical": open_findings_burndown.get("Critical", []), - "high": open_findings_burndown.get("High", []), - "medium": open_findings_burndown.get("Medium", []), - "low": open_findings_burndown.get("Low", []), - "info": open_findings_burndown.get("Info", []), - "max": open_findings_burndown.get("y_max", 0), - "min": open_findings_burndown.get("y_min", 0), - }) - - -def view_engagements(request, pid): - prod = get_object_or_404(Product, id=pid) - default_page_num = 10 - recent_test_day_count = 7 - filter_string_matching = get_system_setting("filter_string_matching", False) - filter_class = ProductEngagementFilterWithoutObjectLookups if filter_string_matching else ProductEngagementFilter - # In Progress Engagements - engs = Engagement.objects.filter(product=prod, active=True, status="In Progress").order_by("-updated") - active_engs_filter = filter_class(request.GET, queryset=engs, prefix="active") - result_active_engs = get_page_items(request, active_engs_filter.qs, default_page_num, prefix="engs") - # prefetch only after creating the filters to avoid https://code.djangoproject.com/ticket/23771 - # and https://code.djangoproject.com/ticket/25375 - result_active_engs.object_list = prefetch_for_view_engagements( - result_active_engs.object_list, - recent_test_day_count, - ) - # Engagements that are queued because they haven't started or paused - engs = Engagement.objects.filter(~Q(status="In Progress"), product=prod, active=True).order_by("-updated") - queued_engs_filter = filter_class(request.GET, queryset=engs, prefix="queued") - result_queued_engs = get_page_items(request, queued_engs_filter.qs, default_page_num, prefix="queued_engs") - result_queued_engs.object_list = prefetch_for_view_engagements( - result_queued_engs.object_list, - recent_test_day_count, - ) - # Cancelled or Completed Engagements - engs = Engagement.objects.filter(product=prod, active=False).order_by("-target_end") - inactive_engs_filter = filter_class(request.GET, queryset=engs, prefix="closed") - result_inactive_engs = get_page_items(request, inactive_engs_filter.qs, default_page_num, prefix="inactive_engs") - result_inactive_engs.object_list = prefetch_for_view_engagements( - result_inactive_engs.object_list, - recent_test_day_count, - ) - - product_tab = Product_Tab(prod, title=_("All Engagements"), tab="engagements") - return render(request, "dojo/view_engagements.html", { - "prod": prod, - "product_tab": product_tab, - "engs": result_active_engs, - "engs_count": result_active_engs.paginator.count, - "engs_filter": active_engs_filter, - "queued_engs": result_queued_engs, - "queued_engs_count": result_queued_engs.paginator.count, - "queued_engs_filter": queued_engs_filter, - "inactive_engs": result_inactive_engs, - "inactive_engs_count": result_inactive_engs.paginator.count, - "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), - "inactive_engs_filter": inactive_engs_filter, - "recent_test_day_count": recent_test_day_count, - "user": request.user}) - - -def prefetch_for_view_engagements(engagements, recent_test_day_count): - engagements = engagements.prefetch_related( - Prefetch("test_set", queryset=Test.objects.filter( - id__in=Subquery( - Test.objects.filter( - engagement_id=OuterRef("engagement_id"), - updated__gte=timezone.now() - timedelta(days=recent_test_day_count), - ).values_list("id", flat=True), - )), - ), - "test_set__test_type", - ).select_related( - "lead", - ) - - # Use subqueries to avoid GROUP BY issues - test_subquery = build_count_subquery( - Test.objects.filter(engagement=OuterRef("pk")), group_field="engagement_id", - ) - finding_subquery = build_count_subquery( - Finding.objects.filter(test__engagement=OuterRef("pk")), group_field="test__engagement_id", - ) - finding_open_subquery = build_count_subquery( - Finding.objects.filter(test__engagement=OuterRef("pk"), active=True), group_field="test__engagement_id", - ) - finding_open_verified_subquery = build_count_subquery( - Finding.objects.filter(test__engagement=OuterRef("pk"), active=True, verified=True), group_field="test__engagement_id", - ) - finding_open_fix_available_subquery = build_count_subquery( - Finding.objects.filter(test__engagement=OuterRef("pk"), active=True, fix_available=True), group_field="test__engagement_id", - ) - finding_close_subquery = build_count_subquery( - Finding.objects.filter(test__engagement=OuterRef("pk"), is_mitigated=True), group_field="test__engagement_id", - ) - finding_duplicate_subquery = build_count_subquery( - Finding.objects.filter(test__engagement=OuterRef("pk"), duplicate=True), group_field="test__engagement_id", - ) - finding_accepted_subquery = build_count_subquery( - Finding.objects.filter(test__engagement=OuterRef("pk"), risk_accepted=True), group_field="test__engagement_id", - ) - - engagements = engagements.annotate( - count_tests=Coalesce(test_subquery, Value(0)), - count_findings_all=Coalesce(finding_subquery, Value(0)), - count_findings_open=Coalesce(finding_open_subquery, Value(0)), - count_findings_open_verified=Coalesce(finding_open_verified_subquery, Value(0)), - count_findings_fix_available=Coalesce(finding_open_fix_available_subquery, Value(0)), - count_findings_close=Coalesce(finding_close_subquery, Value(0)), - count_findings_duplicate=Coalesce(finding_duplicate_subquery, Value(0)), - count_findings_accepted=Coalesce(finding_accepted_subquery, Value(0)), - ) - - if System_Settings.objects.get().enable_jira: - engagements = engagements.prefetch_related( - "jira_project__jira_instance", - "product__jira_project_set__jira_instance", - ) - - return engagements - - -def new_product(request, ptid=None): - if get_authorized_product_types("add").count() == 0: - raise PermissionDenied - - jira_project_form = None - error = False - initial = None - if ptid is not None: - prod_type = get_object_or_404(Product_Type, pk=ptid) - initial = {"prod_type": prod_type} - - form = ProductForm(initial=initial) - - if request.method == "POST": - form = ProductForm(request.POST, instance=Product()) - - if get_system_setting("enable_github"): - gform = GITHUB_Product_Form(request.POST, instance=GITHUB_PKey()) - else: - gform = None - - if form.is_valid(): - product_type = form.instance.prod_type - user_has_permission_or_403(request.user, product_type, "add") - - product = form.save() - messages.add_message(request, - messages.SUCCESS, - labels.ASSET_CREATE_SUCCESS_MESSAGE, - extra_tags="alert-success") - success, jira_project_form = jira_services.process_project_form(request, product=product) - error = not success - - if get_system_setting("enable_github"): - if gform.is_valid(): - github_pkey = gform.save(commit=False) - if github_pkey.git_conf is not None and github_pkey.git_project: - github_pkey.product = product - github_pkey.save() - messages.add_message(request, - messages.SUCCESS, - _("GitHub information added successfully."), - extra_tags="alert-success") - # Create appropriate labels in the repo - logger.info("Create label in repo: " + github_pkey.git_project) - - description = _("This label is automatically applied to all issues created by DefectDojo") - try: - g = Github(github_pkey.git_conf.api_key) - repo = g.get_repo(github_pkey.git_project) - repo.create_label(name="security", color="FF0000", - description=description) - repo.create_label(name="security / info", color="00FEFC", - description=description) - repo.create_label(name="security / low", color="B7FE00", - description=description) - repo.create_label(name="security / medium", color="FEFE00", - description=description) - repo.create_label(name="security / high", color="FE9A00", - description=description) - repo.create_label(name="security / critical", color="FE2200", - description=description) - except: - logger.info("Labels cannot be created - they may already exists") - - if not error: - return HttpResponseRedirect(reverse("view_product", args=(product.id,))) - # engagement was saved, but JIRA errors, so goto edit_product - return HttpResponseRedirect(reverse("edit_product", args=(product.id,))) - else: - if get_system_setting("enable_jira"): - jira_project_form = JIRAProjectForm() - - gform = GITHUB_Product_Form() if get_system_setting("enable_github") else None - - add_breadcrumb(title=str(labels.ASSET_CREATE_LABEL), top_level=False, request=request) - return render(request, "dojo/new_product.html", - {"form": form, - "jform": jira_project_form, - "gform": gform}) - - -def edit_product(request, pid): - product = Product.objects.get(pk=pid) - system_settings = System_Settings.objects.get() - jira_enabled = system_settings.enable_jira - jira_project = None - jform = None - github_enabled = system_settings.enable_github - github_inst = None - gform = None - error = False - - try: - github_inst = GITHUB_PKey.objects.get(product=product) - except: - github_inst = None - - if request.method == "POST": - form = ProductForm(request.POST, instance=product) - jira_project = jira_services.get_project(product) - if form.is_valid(): - initial_sla_config = Product.objects.get(pk=form.instance.id).sla_configuration - form.save() - msg = labels.ASSET_UPDATE_SUCCESS_MESSAGE - # check if the SLA config was changed, append additional context to message - if initial_sla_config != form.instance.sla_configuration: - msg += " " + labels.ASSET_UPDATE_SLA_CHANGED_MESSAGE - messages.add_message(request, - messages.SUCCESS, - msg, - extra_tags="alert-success") - - success, jform = jira_services.process_project_form(request, instance=jira_project, product=product) - error = not success - - if get_system_setting("enable_github") and github_inst: - gform = GITHUB_Product_Form(request.POST, instance=github_inst) - if gform.is_valid(): - gform.save() - elif get_system_setting("enable_github"): - gform = GITHUB_Product_Form(request.POST) - if gform.is_valid(): - new_conf = gform.save(commit=False) - new_conf.product_id = pid - new_conf.save() - messages.add_message(request, - messages.SUCCESS, - _("GITHUB information updated successfully."), - extra_tags="alert-success") - - if not error: - return HttpResponseRedirect(reverse("view_product", args=(pid,))) - else: - form = ProductForm(instance=product) - - if jira_enabled: - jira_project = jira_services.get_project(product) - jform = JIRAProjectForm(instance=jira_project) - else: - jform = None - - if github_enabled: - gform = GITHUB_Product_Form(instance=github_inst) if github_inst is not None else GITHUB_Product_Form() - else: - gform = None - - product_tab = Product_Tab(product, title=str(labels.ASSET_UPDATE_LABEL), tab="settings") - return render(request, - "dojo/edit_product.html", - {"form": form, - "product_tab": product_tab, - "jform": jform, - "gform": gform, - "product": product, - }) - - -def delete_product(request, pid): - product = get_object_or_404(Product, pk=pid) - form = DeleteProductForm(instance=product) - - if request.method == "POST": - logger.debug("delete_product: POST") - if "id" in request.POST and str(product.id) == request.POST["id"]: - form = DeleteProductForm(request.POST, instance=product) - if form.is_valid(): - if get_setting("ASYNC_OBJECT_DELETE"): - async_del = async_delete() - async_del.delete(product) - message = labels.ASSET_DELETE_SUCCESS_ASYNC_MESSAGE - else: - message = labels.ASSET_DELETE_SUCCESS_MESSAGE - with Endpoint.allow_endpoint_init(): # TODO: Delete this after the move to Locations - product.delete() - messages.add_message(request, - messages.SUCCESS, - message, - extra_tags="alert-success") - logger.debug("delete_product: POST RETURN") - return HttpResponseRedirect(reverse("product")) - logger.debug("delete_product: POST INVALID FORM") - logger.error(form.errors) - - logger.debug("delete_product: GET") - - rels = ["Previewing the relationships has been disabled.", ""] - display_preview = get_setting("DELETE_PREVIEW") - if display_preview: - with Endpoint.allow_endpoint_init(): # TODO: Delete this after the move to Locations - collector = NestedObjects(using=DEFAULT_DB_ALIAS) - collector.collect([product]) - rels = collector.nested() - - product_tab = Product_Tab(product, title=str(labels.ASSET_LABEL), tab="settings") - - logger.debug("delete_product: GET RENDER") - - return render(request, "dojo/delete_product.html", { - "label_delete_with_name": labels.ASSET_DELETE_WITH_NAME_LABEL % {"name": product}, - "product": product, - "form": form, - "product_tab": product_tab, - "rels": rels}) - - -def new_eng_for_app(request, pid, *, cicd=False): - jira_project_form = None - jira_epic_form = None - - product = Product.objects.get(id=pid) - - if request.method == "POST": - form = EngForm(request.POST, cicd=cicd, product=product, user=request.user) - logger.debug("new_eng_for_app") - - if form.is_valid(): - # first create the new engagement - engagement = form.save(commit=False) - engagement.threat_model = False - engagement.api_test = False - engagement.pen_test = False - engagement.check_list = False - engagement.product = form.cleaned_data.get("product") - if engagement.threat_model: - engagement.progress = "threat_model" - else: - engagement.progress = "other" - if cicd: - engagement.engagement_type = "CI/CD" - engagement.status = "In Progress" - engagement.active = True - - engagement.save() - form.save_m2m() - - logger.debug("new_eng_for_app: process jira coming") - - # new engagement, so do not provide jira_project - success, jira_project_form = jira_services.process_project_form(request, instance=None, - engagement=engagement) - error = not success - - logger.debug("new_eng_for_app: process jira epic coming") - - success, jira_epic_form = jira_services.process_epic_form(request, engagement=engagement) - error = error or not success - - messages.add_message(request, - messages.SUCCESS, - _("Engagement added successfully."), - extra_tags="alert-success") - - if not error: - if "_Add Tests" in request.POST: - return HttpResponseRedirect(reverse("add_tests", args=(engagement.id,))) - if "_Import Scan Results" in request.POST: - return HttpResponseRedirect(reverse("import_scan_results", args=(engagement.id,))) - return HttpResponseRedirect(reverse("view_engagement", args=(engagement.id,))) - # engagement was saved, but JIRA errors, so goto edit_engagement - logger.debug("new_eng_for_app: jira errors") - return HttpResponseRedirect(reverse("edit_engagement", args=(engagement.id,))) - logger.debug(form.errors) - else: - form = EngForm(initial={"lead": request.user, "target_start": timezone.now().date(), - "target_end": timezone.now().date() + timedelta(days=7), "product": product}, cicd=cicd, - product=product, user=request.user) - - if get_system_setting("enable_jira"): - logger.debug("showing jira-project-form") - jira_project_form = JIRAProjectForm(target="engagement", product=product) - logger.debug("showing jira-epic-form") - jira_epic_form = JIRAEngagementForm() - - title = _("New CI/CD Engagement") if cicd else _("New Interactive Engagement") - - product_tab = Product_Tab(product, title=title, tab="engagements") - return render(request, "dojo/new_eng.html", { - "form": form, - "title": title, - "product_tab": product_tab, - "jira_epic_form": jira_epic_form, - "jira_project_form": jira_project_form}) - - -def new_tech_for_prod(request, pid): - if request.method == "POST": - form = AppAnalysisForm(request.POST) - if form.is_valid(): - tech = form.save(commit=False) - tech.product_id = pid - tech.save() - messages.add_message(request, - messages.SUCCESS, - _("Technology added successfully."), - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_product", args=(pid,))) - - form = AppAnalysisForm(initial={"user": request.user}) - product_tab = Product_Tab(get_object_or_404(Product, id=pid), title=_("Add Technology"), tab="settings") - return render(request, "dojo/new_tech.html", - {"form": form, - "product_tab": product_tab, - "pid": pid}) - - -def edit_technology(request, tid): - technology = get_object_or_404(App_Analysis, id=tid) - form = AppAnalysisForm(instance=technology) - if request.method == "POST": - form = AppAnalysisForm(request.POST, instance=technology) - if form.is_valid(): - form.save() - messages.add_message(request, - messages.SUCCESS, - _("Technology changed successfully."), - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_product", args=(technology.product.id,))) - - product_tab = Product_Tab(technology.product, title=_("Edit Technology"), tab="settings") - return render(request, "dojo/edit_technology.html", - {"form": form, - "product_tab": product_tab, - "technology": technology}) - - -def delete_technology(request, tid): - technology = get_object_or_404(App_Analysis, id=tid) - form = DeleteAppAnalysisForm(instance=technology) - if request.method == "POST": - form = DeleteAppAnalysisForm(request.POST, instance=technology) - technology = form.instance - technology.delete() - messages.add_message(request, - messages.SUCCESS, - _("Technology deleted successfully."), - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_product", args=(technology.product.id,))) - - product_tab = Product_Tab(technology.product, title=_("Delete Technology"), tab="settings") - return render(request, "dojo/delete_technology.html", { - "technology": technology, - "form": form, - "product_tab": product_tab, - }) - - -def new_eng_for_app_cicd(request, pid): - # we have to use pid=pid here as new_eng_for_app expects kwargs, because that is how django calls the function based on urls.py named groups - return new_eng_for_app(request, pid=pid, cicd=True) - - -def manage_meta_data(request, pid): - product = Product.objects.get(id=pid) - meta_data_query = DojoMeta.objects.filter(product=product) - form_mapping = {"product": product} - formset = DojoMetaFormSet(queryset=meta_data_query, form_kwargs={"fk_map": form_mapping}) - - if request.method == "POST": - formset = DojoMetaFormSet(request.POST, queryset=meta_data_query, form_kwargs={"fk_map": form_mapping}) - if formset.is_valid(): - formset.save() - messages.add_message( - request, messages.SUCCESS, "Metadata updated successfully.", extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_product", args=(pid,))) - - add_breadcrumb(parent=product, title="Manage Metadata", top_level=False, request=request) - product_tab = Product_Tab(product, "Edit Metadata", tab="products") - return render( - request, - "dojo/edit_metadata.html", - {"formset": formset, "product_tab": product_tab}, - ) - - -class AdHocFindingView(View): - def get_product(self, product_id: int): - return get_object_or_404(Product, id=product_id) - - def get_test_type(self): - test_type, _nil = Test_Type.objects.get_or_create(name=_("Pen Test")) - return test_type - - def get_engagement(self, product: Product): - try: - return Engagement.objects.get(product=product, name=_("Ad Hoc Engagement")) - except Engagement.DoesNotExist: - return Engagement.objects.create( - name=_("Ad Hoc Engagement"), - target_start=timezone.now(), - target_end=timezone.now(), - active=False, product=product) - - def get_test(self, engagement: Engagement, test_type: Test_Type): - if test := Test.objects.filter(engagement=engagement).first(): - return test - return Test.objects.create( - engagement=engagement, - test_type=test_type, - target_start=timezone.now(), - target_end=timezone.now()) - - def create_nested_objects(self, product: Product): - engagement = self.get_engagement(product) - test_type = self.get_test_type() - return self.get_test(engagement, test_type) - - def get_initial_context(self, request: HttpRequest, test: Test): - # Get the finding form first since it is used in another place - finding_form = self.get_finding_form(request, test.engagement.product) - product_tab = Product_Tab(test.engagement.product, title=_("Add Finding"), tab="engagements") - product_tab.setEngagement(test.engagement) - return { - "form": finding_form, - "product_tab": product_tab, - "temp": False, - "tid": test.id, - "pid": test.engagement.product.id, - "form_error": False, - "jform": self.get_jira_form(request, test, finding_form=finding_form), - "gform": self.get_github_form(request, test), - } - - def get_finding_form(self, request: HttpRequest, product: Product): - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = { - "initial": {"date": timezone.now().date()}, - "req_resp": None, - "product": product, - } - # Remove the initial state on post - if request.method == "POST": - kwargs.pop("initial") - - return AdHocFindingForm(*args, **kwargs) - - def get_jira_form(self, request: HttpRequest, test: Test, finding_form: AdHocFindingForm = None): - # Determine if jira should be used - if (jira_project := jira_services.get_project(test)) is not None: - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = { - "push_all": jira_services.is_push_all_issues(test), - "prefix": "jiraform", - "jira_project": jira_project, - "finding_form": finding_form, - } - - return JIRAFindingForm(*args, **kwargs) - return None - - def get_github_form(self, request: HttpRequest, test: Test): - # Determine if github should be used - if get_system_setting("enable_github"): - # Ensure there is a github conf correctly configured for the product - config_present = GITHUB_PKey.objects.filter(product=test.engagement.product) - if config_present := config_present.exclude(git_conf_id=None): - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = { - "enabled": jira_services.is_push_all_issues(test), - "prefix": "githubform", - } - - return GITHUBFindingForm(*args, **kwargs) - return None - - def validate_status_change(self, request: HttpRequest, context: dict): - if ((context["form"]["active"].value() is False - or context["form"]["false_p"].value()) - and context["form"]["duplicate"].value() is False): - - closing_disabled = Note_Type.objects.filter(is_mandatory=True, is_active=True).count() - if closing_disabled != 0: - error_inactive = ValidationError( - _("Can not set a finding as inactive without adding all mandatory notes"), - code="inactive_without_mandatory_notes", - ) - error_false_p = ValidationError( - _("Can not set a finding as false positive without adding all mandatory notes"), - code="false_p_without_mandatory_notes", - ) - if context["form"]["active"].value() is False: - context["form"].add_error("active", error_inactive) - if context["form"]["false_p"].value(): - context["form"].add_error("false_p", error_false_p) - messages.add_message( - request, - messages.ERROR, - _("Can not set a finding as inactive or false positive without adding all mandatory notes"), - extra_tags="alert-danger") - - return request - - def process_finding_form(self, request: HttpRequest, test: Test, context: dict): - finding = None - if context["form"].is_valid(): - finding = context["form"].save(commit=False) - finding.test = test - finding.reporter = request.user - finding.numerical_severity = Finding.get_numerical_severity(finding.severity) - finding.tags = context["form"].cleaned_data["tags"] - finding.unsaved_vulnerability_ids = context["form"].cleaned_data["vulnerability_ids"].split() - finding.save() - # Save and add new endpoints - finding_helper.add_locations(finding, context["form"]) - # Save the finding at the end and return - finding.save() - - return finding, request, True - add_error_message_to_response("The form has errors, please correct them below.") - add_field_errors_to_response(context["form"]) - - return finding, request, False - - def process_jira_form(self, request: HttpRequest, finding: Finding, context: dict): - # Capture case if the jira not being enabled - if context["jform"] is None: - return request, True, False - - if context["jform"] and context["jform"].is_valid(): - # Push to Jira? - logger.debug("jira form valid") - push_to_jira = jira_services.is_push_all_issues(finding) or context["jform"].cleaned_data.get("push_to_jira") - jira_message = None - # if the jira issue key was changed, update database - new_jira_issue_key = context["jform"].cleaned_data.get("jira_issue") - if finding.has_jira_issue: - # everything in DD around JIRA integration is based on the internal id of the issue in JIRA - # instead of on the public jira issue key. - # I have no idea why, but it means we have to retrieve the issue from JIRA to get the internal JIRA id. - # we can assume the issue exist, which is already checked in the validation of the jform - if not new_jira_issue_key: - jira_services.unlink_finding(request, finding) - jira_message = "Link to JIRA issue removed successfully." - - elif new_jira_issue_key != finding.jira_issue.jira_key: - jira_services.unlink_finding(request, finding) - jira_services.link_finding(request, finding, new_jira_issue_key) - jira_message = "Changed JIRA link successfully." - else: - logger.debug("finding has no jira issue yet") - if new_jira_issue_key: - logger.debug( - "finding has no jira issue yet, but jira issue specified in request. trying to link.") - jira_services.link_finding(request, finding, new_jira_issue_key) - jira_message = "Linked a JIRA issue successfully." - # Determine if a message should be added - if jira_message: - messages.add_message( - request, messages.SUCCESS, jira_message, extra_tags="alert-success", - ) - - return request, True, push_to_jira - add_field_errors_to_response(context["jform"]) - - return request, False, False - - def process_github_form(self, request: HttpRequest, finding: Finding, context: dict): - if "githubform-push_to_github" not in request.POST: - return request, True - - if context["gform"].is_valid(): - add_external_issue(finding.id, "github") - - return request, True - add_field_errors_to_response(context["gform"]) - - return request, False - - def process_forms(self, request: HttpRequest, test: Test, context: dict): - form_success_list = [] - # Set vars for the completed forms - # Validate finding mitigation - request = self.validate_status_change(request, context) - # Check the validity of the form overall - finding, request, success = self.process_finding_form(request, test, context) - form_success_list.append(success) - request, success, push_to_jira = self.process_jira_form(request, finding, context) - form_success_list.append(success) - request, success = self.process_github_form(request, finding, context) - form_success_list.append(success) - # Determine if all forms were successful - all_forms_valid = all(form_success_list) - # Check the validity of all the forms - if all_forms_valid: - # if we're removing the "duplicate" in the edit finding screen - finding_helper.save_vulnerability_ids(finding, context["form"].cleaned_data["vulnerability_ids"].split()) - # Push things to jira if needed - finding.save(push_to_jira=push_to_jira) - # Save the burp req resp - if "request" in context["form"].cleaned_data or "response" in context["form"].cleaned_data: - burp_rr = BurpRawRequestResponse( - finding=finding, - burpRequestBase64=base64.b64encode(context["form"].cleaned_data["request"].encode()), - burpResponseBase64=base64.b64encode(context["form"].cleaned_data["response"].encode()), - ) - burp_rr.clean() - burp_rr.save() - # Add a success message - messages.add_message( - request, - messages.SUCCESS, - _("Finding added successfully."), - extra_tags="alert-success") - - return finding, request, all_forms_valid - - def get_template(self): - return "dojo/ad_hoc_findings.html" - - def get(self, request: HttpRequest, product_id: int): - # Get the initial objects - product = self.get_product(product_id) - # Make sure the user is authorized - user_has_permission_or_403(request.user, product, "add") - # Create the necessary nested objects - test = self.create_nested_objects(product) - # Set up the initial context - context = self.get_initial_context(request, test) - # Render the form - return render(request, self.get_template(), context) - - def post(self, request: HttpRequest, product_id: int): - # Get the initial objects - product = self.get_product(product_id) - # Make sure the user is authorized - user_has_permission_or_403(request.user, product, "add") - # Create the necessary nested objects - test = self.create_nested_objects(product) - # Set up the initial context - context = self.get_initial_context(request, test) - # Process the form - _, request, success = self.process_forms(request, test, context) - # Handle the case of a successful form - if success: - if "_Finished" in request.POST: - return HttpResponseRedirect(reverse("view_test", args=(test.id,))) - return HttpResponseRedirect(reverse("add_findings", args=(test.id,))) - context["form_error"] = True - # Render the form - return render(request, self.get_template(), context) - - -def engagement_presets(request, pid): - prod = get_object_or_404(Product, id=pid) - presets = Engagement_Presets.objects.filter(product=prod).all() - - product_tab = Product_Tab(prod, title=_("Engagement Presets"), tab="settings") - - return render(request, "dojo/view_presets.html", - {"product_tab": product_tab, - "presets": presets, - "prod": prod}) - - -def edit_engagement_presets(request, pid, eid): - prod = get_object_or_404(Product, id=pid) - preset = get_object_or_404(Engagement_Presets.objects.filter(product=prod), id=eid) - - product_tab = Product_Tab(prod, title=_("Edit Engagement Preset"), tab="settings") - - if request.method == "POST": - tform = EngagementPresetsForm(request.POST, instance=preset) - if tform.is_valid(): - tform.save() - messages.add_message( - request, - messages.SUCCESS, - _("Engagement Preset Successfully Updated."), - extra_tags="alert-success") - return HttpResponseRedirect(reverse("engagement_presets", args=(pid,))) - else: - tform = EngagementPresetsForm(instance=preset) - - return render(request, "dojo/edit_presets.html", - {"product_tab": product_tab, - "tform": tform, - "prod": prod}) - - -def add_engagement_presets(request, pid): - prod = get_object_or_404(Product, id=pid) - if request.method == "POST": - tform = EngagementPresetsForm(request.POST) - if tform.is_valid(): - form_copy = tform.save(commit=False) - form_copy.product = prod - form_copy.save() - tform.save_m2m() - messages.add_message( - request, - messages.SUCCESS, - _("Engagement Preset Successfully Created."), - extra_tags="alert-success") - return HttpResponseRedirect(reverse("engagement_presets", args=(pid,))) - else: - tform = EngagementPresetsForm() - - product_tab = Product_Tab(prod, title=_("New Engagement Preset"), tab="settings") - return render(request, "dojo/new_params.html", {"tform": tform, "pid": pid, "product_tab": product_tab}) - - -def delete_engagement_presets(request, pid, eid): - prod = get_object_or_404(Product, id=pid) - preset = get_object_or_404(Engagement_Presets.objects.filter(product=prod), id=eid) - form = DeleteEngagementPresetsForm(instance=preset) - - if request.method == "POST": - if "id" in request.POST: - form = DeleteEngagementPresetsForm(request.POST, instance=preset) - if form.is_valid(): - preset.delete() - messages.add_message(request, - messages.SUCCESS, - _("Engagement presets and engagement relationships removed."), - extra_tags="alert-success") - return HttpResponseRedirect(reverse("engagement_presets", args=(pid,))) - - rels = ["Previewing the relationships has been disabled.", ""] - display_preview = get_setting("DELETE_PREVIEW") - if display_preview: - collector = NestedObjects(using=DEFAULT_DB_ALIAS) - collector.collect([preset]) - rels = collector.nested() - - product_tab = Product_Tab(prod, title=_("Delete Engagement Preset"), tab="settings") - return render(request, "dojo/delete_presets.html", - {"product": product, - "form": form, - "product_tab": product_tab, - "rels": rels, - }) - - -def edit_notifications(request, pid): - prod = get_object_or_404(Product, id=pid) - if request.method == "POST": - product_notifications = Notifications.objects.filter(user=request.user).filter(product=prod).first() - if not product_notifications: - product_notifications = Notifications(user=request.user, product=prod) - logger.debug("no existing product notifications found") - else: - logger.debug("existing product notifications found") - - form = ProductNotificationsForm(request.POST, instance=product_notifications) - - if form.is_valid(): - form.save() - messages.add_message(request, - messages.SUCCESS, - _("Notification settings updated."), - extra_tags="alert-success") - - return HttpResponseRedirect(reverse("view_product", args=(pid,))) - - -def add_product_authorized_users(request, pid): - product = get_object_or_404(Product, pk=pid) - user_has_permission_or_403(request.user, product, Permissions.Product_Manage_Members) - page_name = _("Add Authorized Users") - form = Add_Product_AuthorizedUsersForm(product=product) - if request.method == "POST": - form = Add_Product_AuthorizedUsersForm(request.POST, product=product) - if form.is_valid(): - users = form.cleaned_data["users"] - product.authorized_users.add(*users) - messages.add_message( - request, messages.SUCCESS, - _("Added %(count)d user(s) to authorized users.") % {"count": len(users)}, - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_product", args=(pid,))) - product_tab = Product_Tab(product, title=page_name, tab="settings") - return render(request, "dojo/new_product_authorized_users.html", { - "name": page_name, - "product": product, - "form": form, - "product_tab": product_tab, - }) - - -def delete_product_authorized_user(request, pid, user_id): - product = get_object_or_404(Product, pk=pid) - user_has_permission_or_403(request.user, product, Permissions.Product_Manage_Members) - if request.method != "POST": - raise PermissionDenied - user = get_object_or_404(Dojo_User, pk=user_id) - product.authorized_users.remove(user) - messages.add_message( - request, messages.SUCCESS, - _("Removed %(username)s from authorized users.") % {"username": user.username}, - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_product", args=(pid,))) - - -def add_api_scan_configuration(request, pid): - product = get_object_or_404(Product, id=pid) - if request.method == "POST": - form = Product_API_Scan_ConfigurationForm(request.POST) - if form.is_valid(): - product_api_scan_configuration = form.save(commit=False) - product_api_scan_configuration.product = product - try: - api = create_API(product_api_scan_configuration.tool_configuration) - if api and hasattr(api, "test_product_connection"): - result = api.test_product_connection(product_api_scan_configuration) - messages.add_message(request, - messages.SUCCESS, - _("API connection successful with message: %(result)s.") % {"result": result}, - extra_tags="alert-success") - product_api_scan_configuration.save() - messages.add_message(request, - messages.SUCCESS, - _("API Scan Configuration added successfully."), - extra_tags="alert-success") - if "add_another" in request.POST: - return HttpResponseRedirect(reverse("add_api_scan_configuration", args=(pid,))) - return HttpResponseRedirect(reverse("view_api_scan_configurations", args=(pid,))) - except Exception as e: - logger.exception("Unable to add API Scan Configuration") - messages.add_message(request, - messages.ERROR, - str(e), - extra_tags="alert-danger") - else: - form = Product_API_Scan_ConfigurationForm() - - product_tab = Product_Tab(product, title=_("Add API Scan Configuration"), tab="settings") - - return render(request, - "dojo/add_product_api_scan_configuration.html", - {"form": form, - "product_tab": product_tab, - "product": product, - "api_scan_configuration_hints": get_api_scan_configuration_hints(), - }) - - -def view_api_scan_configurations(request, pid): - product_api_scan_configurations = Product_API_Scan_Configuration.objects.filter(product=pid) - - product_tab = Product_Tab(get_object_or_404(Product, id=pid), title=_("API Scan Configurations"), tab="settings") - return render(request, - "dojo/view_product_api_scan_configurations.html", - { - "product_api_scan_configurations": product_api_scan_configurations, - "product_tab": product_tab, - "pid": pid, - }) - - -def edit_api_scan_configuration(request, pid, pascid): - product_api_scan_configuration = get_object_or_404(Product_API_Scan_Configuration, id=pascid) - - if product_api_scan_configuration.product.pk != int( - pid): # user is trying to edit Tool Configuration from another product (trying to by-pass auth) - raise Http404 - - if request.method == "POST": - form = Product_API_Scan_ConfigurationForm(request.POST, instance=product_api_scan_configuration) - if form.is_valid(): - try: - form_copy = form.save(commit=False) - api = create_API(form_copy.tool_configuration) - if api and hasattr(api, "test_product_connection"): - result = api.test_product_connection(form_copy) - messages.add_message(request, - messages.SUCCESS, - _("API connection successful with message: %(result)s.") % {"result": result}, - extra_tags="alert-success") - form.save() - - messages.add_message(request, - messages.SUCCESS, - _("API Scan Configuration successfully updated."), - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_api_scan_configurations", args=(pid,))) - except Exception as e: - logger.info(e) - messages.add_message(request, - messages.ERROR, - str(e), - extra_tags="alert-danger") - else: - form = Product_API_Scan_ConfigurationForm(instance=product_api_scan_configuration) - - product_tab = Product_Tab(get_object_or_404(Product, id=pid), title=_("Edit API Scan Configuration"), tab="settings") - return render(request, - "dojo/edit_product_api_scan_configuration.html", - { - "form": form, - "product_tab": product_tab, - "api_scan_configuration_hints": get_api_scan_configuration_hints(), - }) - - -def delete_api_scan_configuration(request, pid, pascid): - product_api_scan_configuration = get_object_or_404(Product_API_Scan_Configuration, id=pascid) - - if product_api_scan_configuration.product.pk != int( - pid): # user is trying to delete Tool Configuration from another product (trying to by-pass auth) - raise Http404 - - if request.method == "POST": - form = Product_API_Scan_ConfigurationForm(request.POST) - product_api_scan_configuration.delete() - messages.add_message(request, - messages.SUCCESS, - _("API Scan Configuration deleted."), - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_api_scan_configurations", args=(pid,))) - form = DeleteProduct_API_Scan_ConfigurationForm(instance=product_api_scan_configuration) - - product_tab = Product_Tab(get_object_or_404(Product, id=pid), title=_("Delete Tool Configuration"), tab="settings") - return render(request, - "dojo/delete_product_api_scan_configuration.html", - { - "form": form, - "product_tab": product_tab, - }) +# Backward-compat shim: the view logic moved to dojo.product.ui.views during the +# module reorg. External consumers (e.g. dojo-pro) still import from +# dojo.product.views, so re-export the public names from their new location. +from dojo.product.ui.views import * # noqa: F403 -- backward compat re-export diff --git a/dojo/product_type/__init__.py b/dojo/product_type/__init__.py index e69de29bb2d..83aa70f8a17 100644 --- a/dojo/product_type/__init__.py +++ b/dojo/product_type/__init__.py @@ -0,0 +1 @@ +import dojo.product_type.admin # noqa: F401 diff --git a/dojo/product_type/admin.py b/dojo/product_type/admin.py new file mode 100644 index 00000000000..86cf1fd6bc3 --- /dev/null +++ b/dojo/product_type/admin.py @@ -0,0 +1,9 @@ +from django.contrib import admin + +from dojo.product_type.models import Product_Type + + +@admin.register(Product_Type) +class Product_TypeAdmin(admin.ModelAdmin): + + """Admin support for the Product_Type model.""" diff --git a/dojo/product_type/api/__init__.py b/dojo/product_type/api/__init__.py new file mode 100644 index 00000000000..9ac0eff9870 --- /dev/null +++ b/dojo/product_type/api/__init__.py @@ -0,0 +1 @@ +path = "product_types" # noqa: RUF067 diff --git a/dojo/product_type/api/serializer.py b/dojo/product_type/api/serializer.py new file mode 100644 index 00000000000..fc09e6e7100 --- /dev/null +++ b/dojo/product_type/api/serializer.py @@ -0,0 +1,9 @@ +from rest_framework import serializers + +from dojo.product_type.models import Product_Type + + +class ProductTypeSerializer(serializers.ModelSerializer): + class Meta: + model = Product_Type + fields = "__all__" diff --git a/dojo/product_type/api/urls.py b/dojo/product_type/api/urls.py new file mode 100644 index 00000000000..419dc829307 --- /dev/null +++ b/dojo/product_type/api/urls.py @@ -0,0 +1,6 @@ +from dojo.product_type.api.views import ProductTypeViewSet + + +def add_product_type_urls(router): + router.register("product_types", ProductTypeViewSet, basename="product_type") + return router diff --git a/dojo/product_type/api/views.py b/dojo/product_type/api/views.py new file mode 100644 index 00000000000..d67abb83014 --- /dev/null +++ b/dojo/product_type/api/views.py @@ -0,0 +1,94 @@ +from django_filters.rest_framework import DjangoFilterBackend +from drf_spectacular.utils import extend_schema, extend_schema_view +from rest_framework import status +from rest_framework.decorators import action +from rest_framework.permissions import IsAuthenticated +from rest_framework.response import Response + +from dojo.api_v2.serializers import ReportGenerateOptionSerializer, ReportGenerateSerializer +from dojo.api_v2.views import PrefetchDojoModelViewSet, report_generate, schema_with_prefetch +from dojo.authorization import api_permissions as permissions +from dojo.models import Endpoint, Product_Type +from dojo.product_type.api.serializer import ProductTypeSerializer +from dojo.product_type.queries import get_authorized_product_types +from dojo.utils import async_delete, get_setting + + +# Authorization: object-based +@extend_schema_view(**schema_with_prefetch()) +# Authorization: object-based +@extend_schema_view(**schema_with_prefetch()) +class ProductTypeViewSet( + PrefetchDojoModelViewSet, +): + serializer_class = ProductTypeSerializer + queryset = Product_Type.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = [ + "id", + "name", + "critical_product", + "key_product", + "created", + "updated", + ] + permission_classes = ( + IsAuthenticated, + permissions.UserHasProductTypePermission, + ) + + def get_queryset(self): + return get_authorized_product_types( + "view", + ).distinct() + + def destroy(self, request, *args, **kwargs): + instance = self.get_object() + if get_setting("ASYNC_OBJECT_DELETE"): + async_del = async_delete() + async_del.delete(instance) + else: + with Endpoint.allow_endpoint_init(): # TODO: Delete this after the move to Locations + instance.delete() + return Response(status=status.HTTP_204_NO_CONTENT) + + @extend_schema( + request=ReportGenerateOptionSerializer, + responses={status.HTTP_200_OK: ReportGenerateSerializer}, + ) + @action( + detail=True, methods=["post"], + # IsAuthenticated only: report generation requires View permission, + # enforced by the permission-filtered get_queryset(). The viewset's + # permission_classes would check Edit (POST), which is too restrictive. + permission_classes=[IsAuthenticated], + ) + def generate_report(self, request, pk=None): + product_type = self.get_object() + + options = {} + # prepare post data + report_options = ReportGenerateOptionSerializer( + data=request.data, + ) + if report_options.is_valid(): + options["include_finding_notes"] = report_options.validated_data[ + "include_finding_notes" + ] + options["include_finding_images"] = report_options.validated_data[ + "include_finding_images" + ] + options[ + "include_executive_summary" + ] = report_options.validated_data["include_executive_summary"] + options[ + "include_table_of_contents" + ] = report_options.validated_data["include_table_of_contents"] + else: + return Response( + report_options.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + data = report_generate(request, product_type, options) + report = ReportGenerateSerializer(data) + return Response(report.data) diff --git a/dojo/product_type/models.py b/dojo/product_type/models.py new file mode 100644 index 00000000000..50bfcc722b4 --- /dev/null +++ b/dojo/product_type/models.py @@ -0,0 +1,80 @@ +from django.db import models +from django.urls import reverse +from django.utils.functional import cached_property + +from dojo.base_models.base import BaseModel + + +class Product_Type(BaseModel): + + """ + Product types represent the top level model, these can be business unit divisions, different offices or locations, development teams, or any other logical way of distinguishing "types" of products. + ` + Examples: + * IAM Team + * Internal / 3rd Party + * Main company / Acquisition + * San Francisco / New York offices + """ + + name = models.CharField(max_length=255, unique=True) + description = models.CharField(max_length=4000, null=True, blank=True) + critical_product = models.BooleanField(default=False) + key_product = models.BooleanField(default=False) + authorized_users = models.ManyToManyField("dojo.Dojo_User", related_name="authorized_product_types", blank=True) + + class Meta: + ordering = ("name",) + + def __str__(self): + return self.name + + def get_absolute_url(self): + return reverse("product_type", args=[str(self.id)]) + + def get_breadcrumbs(self): + return [{"title": str(self), + "url": reverse("edit_product_type", args=(self.id,))}] + + @cached_property + def critical_present(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + c_findings = Finding.objects.filter( + test__engagement__product__prod_type=self, severity="Critical") + if c_findings.count() > 0: + return True + return None + + @cached_property + def high_present(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + c_findings = Finding.objects.filter( + test__engagement__product__prod_type=self, severity="High") + if c_findings.count() > 0: + return True + return None + + @cached_property + def calc_health(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + h_findings = Finding.objects.filter( + test__engagement__product__prod_type=self, severity="High") + c_findings = Finding.objects.filter( + test__engagement__product__prod_type=self, severity="Critical") + health = 100 + if c_findings.count() > 0: + health = 40 + health -= ((c_findings.count() - 1) * 5) + if h_findings.count() > 0: + if health == 100: + health = 60 + health -= ((h_findings.count() - 1) * 2) + if health < 5: + return 5 + return health + + # only used by bulk risk acceptance api + @property + def unaccepted_open_findings(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + return Finding.objects.filter(risk_accepted=False, active=True, duplicate=False, test__engagement__product__prod_type=self) diff --git a/dojo/product_type/ui/__init__.py b/dojo/product_type/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/product_type/ui/filters.py b/dojo/product_type/ui/filters.py new file mode 100644 index 00000000000..bc4880b73bb --- /dev/null +++ b/dojo/product_type/ui/filters.py @@ -0,0 +1,24 @@ +import logging + +from django_filters import CharFilter, OrderingFilter + +from dojo.filters import DojoFilter +from dojo.product_type.models import Product_Type + +logger = logging.getLogger(__name__) + + +class ProductTypeFilter(DojoFilter): + name = CharFilter(lookup_expr="icontains") + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("name", "name"), + ), + ) + + class Meta: + model = Product_Type + exclude = [] + include = ("name",) diff --git a/dojo/product_type/ui/forms.py b/dojo/product_type/ui/forms.py new file mode 100644 index 00000000000..68405a9ba19 --- /dev/null +++ b/dojo/product_type/ui/forms.py @@ -0,0 +1,51 @@ +import logging + +from django import forms + +from dojo.labels import get_labels +from dojo.models import Dojo_User +from dojo.product_type.models import Product_Type + +logger = logging.getLogger(__name__) + +labels = get_labels() + + +class Product_TypeForm(forms.ModelForm): + description = forms.CharField(widget=forms.Textarea(attrs={}), + required=False) + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.fields["critical_product"].label = labels.ORG_CRITICAL_PRODUCT_LABEL + self.fields["key_product"].label = labels.ORG_KEY_PRODUCT_LABEL + + class Meta: + model = Product_Type + fields = ["name", "description", "critical_product", "key_product"] + + +class Delete_Product_TypeForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Product_Type + fields = ["id"] + + +class Add_Product_Type_AuthorizedUsersForm(forms.Form): + users = forms.ModelMultipleChoiceField( + queryset=Dojo_User.objects.none(), required=True, label="Users", + ) + + def __init__(self, *args, product_type=None, **kwargs): + super().__init__(*args, **kwargs) + self.product_type = product_type + current = product_type.authorized_users.values_list("pk", flat=True) + self.fields["users"].queryset = ( + Dojo_User.objects.filter(is_active=True) + .exclude(is_superuser=True) + .exclude(pk__in=current) + .order_by("first_name", "last_name") + ) diff --git a/dojo/product_type/views.py b/dojo/product_type/ui/views.py similarity index 98% rename from dojo/product_type/views.py rename to dojo/product_type/ui/views.py index f16a06c2e17..562646e0392 100644 --- a/dojo/product_type/views.py +++ b/dojo/product_type/ui/views.py @@ -15,7 +15,6 @@ from dojo.authorization.authorization import user_has_permission_or_403 from dojo.authorization.roles_permissions import Permissions -from dojo.filters import ProductFilter, ProductFilterWithoutObjectLookups, ProductTypeFilter from dojo.forms import ( Add_Product_Type_AuthorizedUsersForm, Delete_Product_TypeForm, @@ -24,9 +23,11 @@ from dojo.labels import get_labels from dojo.models import Dojo_User, Endpoint, Finding, Product, Product_Type from dojo.product.queries import get_authorized_products +from dojo.product.ui.filters import ProductFilter, ProductFilterWithoutObjectLookups from dojo.product_type.queries import ( get_authorized_product_types, ) +from dojo.product_type.ui.filters import ProductTypeFilter from dojo.query_utils import build_count_subquery from dojo.utils import ( add_breadcrumb, diff --git a/dojo/regulations/__init__.py b/dojo/regulations/__init__.py index e69de29bb2d..a6ad8a993aa 100644 --- a/dojo/regulations/__init__.py +++ b/dojo/regulations/__init__.py @@ -0,0 +1 @@ +import dojo.regulations.admin # noqa: F401 diff --git a/dojo/regulations/admin.py b/dojo/regulations/admin.py new file mode 100644 index 00000000000..6d5961769f5 --- /dev/null +++ b/dojo/regulations/admin.py @@ -0,0 +1,5 @@ +from django.contrib import admin + +from dojo.regulations.models import Regulation + +admin.site.register(Regulation) diff --git a/dojo/regulations/api/__init__.py b/dojo/regulations/api/__init__.py new file mode 100644 index 00000000000..de5e580ef42 --- /dev/null +++ b/dojo/regulations/api/__init__.py @@ -0,0 +1 @@ +path = "regulations" # noqa: RUF067 diff --git a/dojo/regulations/api/serializer.py b/dojo/regulations/api/serializer.py new file mode 100644 index 00000000000..519d5c0ef10 --- /dev/null +++ b/dojo/regulations/api/serializer.py @@ -0,0 +1,9 @@ +from rest_framework import serializers + +from dojo.regulations.models import Regulation + + +class RegulationSerializer(serializers.ModelSerializer): + class Meta: + model = Regulation + fields = "__all__" diff --git a/dojo/regulations/api/urls.py b/dojo/regulations/api/urls.py new file mode 100644 index 00000000000..fa88fd0086f --- /dev/null +++ b/dojo/regulations/api/urls.py @@ -0,0 +1,7 @@ +from dojo.regulations.api import path +from dojo.regulations.api.views import RegulationsViewSet + + +def add_regulations_urls(router): + router.register(path, RegulationsViewSet, basename="regulations") + return router diff --git a/dojo/regulations/api/views.py b/dojo/regulations/api/views.py new file mode 100644 index 00000000000..8d0574afc89 --- /dev/null +++ b/dojo/regulations/api/views.py @@ -0,0 +1,21 @@ +from django_filters.rest_framework import DjangoFilterBackend +from rest_framework.permissions import IsAuthenticated + +from dojo.api_v2.views import DojoModelViewSet +from dojo.authorization import api_permissions as permissions +from dojo.regulations.api.serializer import RegulationSerializer +from dojo.regulations.models import Regulation + + +# Authorization: authenticated, configuration +class RegulationsViewSet( + DojoModelViewSet, +): + serializer_class = RegulationSerializer + queryset = Regulation.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = ["id", "name", "description"] + permission_classes = (IsAuthenticated, permissions.UserHasRegulationPermission) + + def get_queryset(self): + return Regulation.objects.all().order_by("id") diff --git a/dojo/regulations/models.py b/dojo/regulations/models.py new file mode 100644 index 00000000000..4910f32f481 --- /dev/null +++ b/dojo/regulations/models.py @@ -0,0 +1,36 @@ +from django.db import models +from django.utils.translation import gettext as _ + + +class Regulation(models.Model): + PRIVACY_CATEGORY = "privacy" + FINANCE_CATEGORY = "finance" + EDUCATION_CATEGORY = "education" + MEDICAL_CATEGORY = "medical" + CORPORATE_CATEGORY = "corporate" + SECURITY_CATEGORY = "security" + GOVERNMENT_CATEGORY = "government" + OTHER_CATEGORY = "other" + CATEGORY_CHOICES = ( + (PRIVACY_CATEGORY, _("Privacy")), + (FINANCE_CATEGORY, _("Finance")), + (EDUCATION_CATEGORY, _("Education")), + (MEDICAL_CATEGORY, _("Medical")), + (CORPORATE_CATEGORY, _("Corporate")), + (SECURITY_CATEGORY, _("Security")), + (GOVERNMENT_CATEGORY, _("Government")), + (OTHER_CATEGORY, _("Other")), + ) + + name = models.CharField(max_length=128, unique=True, help_text=_("The name of the regulation.")) + acronym = models.CharField(max_length=20, unique=True, help_text=_("A shortened representation of the name.")) + category = models.CharField(max_length=16, choices=CATEGORY_CHOICES, help_text=_("The subject of the regulation.")) + jurisdiction = models.CharField(max_length=64, help_text=_("The territory over which the regulation applies.")) + description = models.TextField(blank=True, help_text=_("Information about the regulation's purpose.")) + reference = models.URLField(blank=True, help_text=_("An external URL for more information.")) + + class Meta: + ordering = ["name"] + + def __str__(self): + return self.acronym + " (" + self.jurisdiction + ")" diff --git a/dojo/regulations/ui/__init__.py b/dojo/regulations/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/regulations/ui/forms.py b/dojo/regulations/ui/forms.py new file mode 100644 index 00000000000..8e3a7c5f89b --- /dev/null +++ b/dojo/regulations/ui/forms.py @@ -0,0 +1,9 @@ +from django import forms + +from dojo.regulations.models import Regulation + + +class RegulationForm(forms.ModelForm): + class Meta: + model = Regulation + exclude = ["product"] diff --git a/dojo/regulations/urls.py b/dojo/regulations/ui/urls.py similarity index 88% rename from dojo/regulations/urls.py rename to dojo/regulations/ui/urls.py index 324669f6759..21acf979edf 100644 --- a/dojo/regulations/urls.py +++ b/dojo/regulations/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from . import views +from dojo.regulations.ui import views urlpatterns = [ re_path(r"^regulations/add", views.new_regulation, name="new_regulation"), diff --git a/dojo/regulations/views.py b/dojo/regulations/ui/views.py similarity index 96% rename from dojo/regulations/views.py rename to dojo/regulations/ui/views.py index 9bbb3296190..6fd127921a0 100644 --- a/dojo/regulations/views.py +++ b/dojo/regulations/ui/views.py @@ -8,8 +8,8 @@ from django.urls import reverse from dojo.authorization.authorization import user_has_configuration_permission_or_403 -from dojo.forms import RegulationForm -from dojo.models import Regulation +from dojo.regulations.models import Regulation +from dojo.regulations.ui.forms import RegulationForm from dojo.utils import add_breadcrumb logger = logging.getLogger(__name__) diff --git a/dojo/reports/__init__.py b/dojo/reports/__init__.py index e69de29bb2d..54faba7100c 100644 --- a/dojo/reports/__init__.py +++ b/dojo/reports/__init__.py @@ -0,0 +1 @@ +import dojo.reports.admin # noqa: F401 diff --git a/dojo/reports/admin.py b/dojo/reports/admin.py new file mode 100644 index 00000000000..f0c7f236146 --- /dev/null +++ b/dojo/reports/admin.py @@ -0,0 +1,5 @@ +from django.contrib import admin + +from dojo.reports.models import Report_Type + +admin.site.register(Report_Type) diff --git a/dojo/reports/models.py b/dojo/reports/models.py new file mode 100644 index 00000000000..c1201126ff1 --- /dev/null +++ b/dojo/reports/models.py @@ -0,0 +1,5 @@ +from django.db import models + + +class Report_Type(models.Model): + name = models.CharField(max_length=255) diff --git a/dojo/reports/ui/__init__.py b/dojo/reports/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/reports/ui/forms.py b/dojo/reports/ui/forms.py new file mode 100644 index 00000000000..8c324781195 --- /dev/null +++ b/dojo/reports/ui/forms.py @@ -0,0 +1,28 @@ +from django import forms + +from dojo.utils import get_system_setting + + +class ReportOptionsForm(forms.Form): + yes_no = (("0", "No"), ("1", "Yes")) + include_finding_notes = forms.ChoiceField(choices=yes_no, label="Finding Notes") + include_finding_images = forms.ChoiceField(choices=yes_no, label="Finding Images") + include_executive_summary = forms.ChoiceField(choices=yes_no, label="Executive Summary") + include_table_of_contents = forms.ChoiceField(choices=yes_no, label="Table of Contents") + include_disclaimer = forms.ChoiceField(choices=yes_no, label="Disclaimer") + report_type = forms.ChoiceField(choices=(("HTML", "HTML"),)) + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + if get_system_setting("disclaimer_reports_forced"): + self.fields["include_disclaimer"].disabled = True + self.fields["include_disclaimer"].initial = "1" # represents yes + self.fields["include_disclaimer"].help_text = "Administrator of the system enforced placement of disclaimer in all reports. You are not able exclude disclaimer from this report." + + +class CustomReportOptionsForm(forms.Form): + yes_no = (("0", "No"), ("1", "Yes")) + report_name = forms.CharField(required=False, max_length=100) + include_finding_notes = forms.ChoiceField(required=False, choices=yes_no) + include_finding_images = forms.ChoiceField(choices=yes_no, label="Finding Images") + report_type = forms.ChoiceField(choices=(("HTML", "HTML"),)) diff --git a/dojo/reports/urls.py b/dojo/reports/ui/urls.py similarity index 99% rename from dojo/reports/urls.py rename to dojo/reports/ui/urls.py index 19d4348478f..b7361b95b6f 100644 --- a/dojo/reports/urls.py +++ b/dojo/reports/ui/urls.py @@ -1,7 +1,7 @@ from django.conf import settings from django.urls import re_path -from dojo.reports import views +from dojo.reports.ui import views from dojo.utils import redirect_view # TODO: remove the else: branch once v3 migration is complete diff --git a/dojo/reports/ui/views.py b/dojo/reports/ui/views.py new file mode 100644 index 00000000000..47e67c36f50 --- /dev/null +++ b/dojo/reports/ui/views.py @@ -0,0 +1,1144 @@ +import csv +import logging +import re +from datetime import datetime +from tempfile import NamedTemporaryFile + +from dateutil.relativedelta import relativedelta +from django.conf import settings +from django.core.exceptions import PermissionDenied +from django.http import Http404, HttpRequest, HttpResponse, QueryDict +from django.shortcuts import get_object_or_404, render +from django.utils import timezone +from django.views import View +from openpyxl import Workbook +from openpyxl.styles import Font + +from dojo.authorization.authorization import user_has_permission_or_403 +from dojo.authorization.roles_permissions import Permissions +from dojo.endpoint.queries import get_authorized_endpoints +from dojo.endpoint.ui.filters import EndpointFilter, EndpointFilterWithoutObjectLookups +from dojo.filters import ( + EndpointReportFilter, +) +from dojo.finding.queries import get_authorized_findings +from dojo.finding.ui.filters import ( + ReportFindingFilter, + ReportFindingFilterWithoutObjectLookups, +) +from dojo.finding.ui.views import BaseListFindings +from dojo.labels import get_labels +from dojo.location.models import Location +from dojo.location.queries import get_authorized_locations +from dojo.location.status import FindingLocationStatus +from dojo.models import Dojo_User, Endpoint, Engagement, Finding, Product, Product_Type, Test +from dojo.reports.queries import prefetch_related_endpoints_for_report, prefetch_related_findings_for_report +from dojo.reports.ui.forms import ReportOptionsForm +from dojo.reports.widgets import ( + CoverPage, + CustomReportJsonForm, + EndpointList, + FindingList, + PageBreak, + ReportOptions, + TableOfContents, + Widget, + WYSIWYGContent, + report_widget_factory, +) +from dojo.url.filters import URLFilter +from dojo.utils import ( + Product_Tab, + add_breadcrumb, + get_page_items, + get_period_counts_legacy, + get_system_setting, + get_words_for_field, +) + +logger = logging.getLogger(__name__) + +labels = get_labels() + +EXCEL_CHAR_LIMIT = 32767 + + +def down(request): + return render(request, "disabled.html") + + +def report_url_resolver(request): + try: + url_resolver = request.META["HTTP_X_FORWARDED_PROTO"] + "://" + request.META["HTTP_X_FORWARDED_FOR"] + except: + hostname = request.META["HTTP_HOST"] + port_index = hostname.find(":") + if port_index != -1: + url_resolver = request.scheme + "://" + hostname[:port_index] + else: + url_resolver = request.scheme + "://" + hostname + return url_resolver + ":" + request.META["SERVER_PORT"] + + +class ReportBuilder(View): + def get(self, request: HttpRequest) -> HttpResponse: + add_breadcrumb(title="Report Builder", top_level=True, request=request) + return render(request, self.get_template(), self.get_context(request)) + + def get_findings(self, request: HttpRequest): + findings = get_authorized_findings("view") + filter_string_matching = get_system_setting("filter_string_matching", False) + filter_class = ReportFindingFilterWithoutObjectLookups if filter_string_matching else ReportFindingFilter + return filter_class(self.request.GET, queryset=findings) + + def get_endpoints(self, request: HttpRequest): + if settings.V3_FEATURE_LOCATIONS: + endpoints = Location.objects.filter(findings__status=FindingLocationStatus.Active).distinct() + filter_class = URLFilter + else: + endpoints = Endpoint.objects.filter( + finding__active=True, + finding__false_p=False, + finding__duplicate=False, + finding__out_of_scope=False, + ) + if get_system_setting("enforce_verified_status", True) or get_system_setting( + "enforce_verified_status_metrics", True, + ): + endpoints = endpoints.filter(finding__active=True) + endpoints = endpoints.distinct() + filter_string_matching = get_system_setting("filter_string_matching", False) + filter_class = EndpointFilterWithoutObjectLookups if filter_string_matching else EndpointFilter + return filter_class(request.GET, queryset=endpoints, user=request.user) + + def get_available_widgets(self, request: HttpRequest) -> list[Widget]: + return [ + CoverPage(request=request), + TableOfContents(request=request), + WYSIWYGContent(request=request), + FindingList(request=request, findings=self.get_findings(request)), + EndpointList(request=request, endpoints=self.get_endpoints(request)), + PageBreak()] + + def get_in_use_widgets(self, request): + return [ReportOptions(request=request)] + + def get_template(self): + return "dojo/report_builder.html" + + def get_context(self, request: HttpRequest) -> dict: + return { + "available_widgets": self.get_available_widgets(request), + "in_use_widgets": self.get_in_use_widgets(request)} + + +class CustomReport(View): + def post(self, request: HttpRequest) -> HttpResponse: + # saving the report + form = self.get_form(request) + if form.is_valid(): + self._set_state(request) + return render(request, self.get_template(), self.get_context()) + raise PermissionDenied + + def _set_state(self, request: HttpRequest): + self.request = request + self.host = report_url_resolver(request) + self.selected_widgets = self.get_selected_widgets(request) + self.widgets = list(self.selected_widgets.values()) + self.include_disclaimer = get_system_setting("disclaimer_reports_forced", 0) + self.disclaimer = get_system_setting("disclaimer_reports") + if self.include_disclaimer and len(self.disclaimer) == 0: + self.disclaimer = "Please configure in System Settings." + + def get_selected_widgets(self, request): + selected_widgets = report_widget_factory(json_data=request.POST["json"], request=request, host=self.host, + user=self.request.user, finding_notes=False, finding_images=False) + + if options := selected_widgets.get("report-options", None): + self.report_format = options.report_type + self.finding_notes = (options.include_finding_notes == "1") + self.finding_images = (options.include_finding_images == "1") + else: + self.report_format = "HTML" + self.finding_notes = True + self.finding_images = True + + return report_widget_factory(json_data=request.POST["json"], request=request, host=self.host, + user=request.user, finding_notes=self.finding_notes, + finding_images=self.finding_images) + + def get_form(self, request): + return CustomReportJsonForm(request.POST) + + def get_template(self): + if self.report_format == "HTML": + return "dojo/custom_html_report.html" + raise PermissionDenied + + def get_context(self): + return { + "widgets": self.widgets, + "host": self.host, + "finding_notes": self.finding_notes, + "finding_images": self.finding_images, + "user_id": self.request.user.id, + "include_disclaimer": self.include_disclaimer, + "disclaimer": self.disclaimer, + } + + +def report_findings(request): + findings = get_authorized_findings(Permissions.Finding_View) + filter_string_matching = get_system_setting("filter_string_matching", False) + filter_class = ReportFindingFilterWithoutObjectLookups if filter_string_matching else ReportFindingFilter + findings = filter_class(request.GET, queryset=findings) + + title_words = get_words_for_field(Finding, "title") + component_words = get_words_for_field(Finding, "component_name") + + paged_findings = get_page_items(request, findings.qs.distinct(), 25) + + return render(request, + "dojo/report_findings.html", + {"findings": paged_findings, + "filtered": findings, + "title_words": title_words, + "component_words": component_words, + "title": "finding-list", + "asset_label": labels.ASSET_LABEL, + }) + + +def report_endpoints(request): + if settings.V3_FEATURE_LOCATIONS: + endpoints = get_authorized_locations(Permissions.Location_View) + endpoints = endpoints.filter(findings__status=FindingLocationStatus.Active).distinct() + endpoints = URLFilter(request.GET, queryset=endpoints) + else: + # TODO: Delete this after the move to Locations + endpoints = get_authorized_endpoints(Permissions.Location_View).filter( + finding__active=True, + finding__false_p=False, + finding__duplicate=False, + finding__out_of_scope=False, + ) + if get_system_setting("enforce_verified_status", True) or get_system_setting( + "enforce_verified_status_metrics", True, + ): + endpoints = endpoints.filter(finding__active=True) + endpoints = endpoints.distinct() + endpoints = EndpointFilter(request.GET, queryset=endpoints, user=request.user) + + paged_endpoints = get_page_items(request, endpoints.qs, 25) + + return render(request, + "dojo/report_endpoints.html", + {"endpoints": paged_endpoints, + "filtered": endpoints, + "title": "endpoint-list", + }) + + +def report_cover_page(request): + report_title = request.GET.get("title", "Report") + report_subtitle = request.GET.get("subtitle", "") + report_info = request.GET.get("info", "") + + return render(request, + "dojo/report_cover_page.html", + {"report_title": report_title, + "report_subtitle": report_subtitle, + "report_info": report_info}) + + +def product_type_report(request, ptid): + product_type = get_object_or_404(Product_Type, id=ptid) + return generate_report(request, product_type) + + +def product_report(request, pid): + product = get_object_or_404(Product, id=pid) + return generate_report(request, product) + + +def product_findings_report(request): + findings = get_authorized_findings("view") + return generate_report(request, findings) + + +def engagement_report(request, eid): + engagement = get_object_or_404(Engagement, id=eid) + return generate_report(request, engagement) + + +def test_report(request, tid): + test = get_object_or_404(Test, id=tid) + return generate_report(request, test) + + +def product_endpoint_report(request, pid): + product = get_object_or_404(Product.objects.all().prefetch_related("engagement_set__test_set__test_type", "engagement_set__test_set__environment"), id=pid) + if settings.V3_FEATURE_LOCATIONS: + endpoints = Location.objects.filter(findings__status=FindingLocationStatus.Active) + endpoints = prefetch_related_endpoints_for_report(endpoints.distinct()) + endpoints = URLFilter(request.GET, queryset=endpoints) + else: + # TODO: Delete this after the move to Locations + endpoints = Endpoint.objects.filter(product=product, + finding__active=True, + finding__false_p=False, + finding__duplicate=False, + finding__out_of_scope=False) + if get_system_setting("enforce_verified_status", True) or get_system_setting("enforce_verified_status_metrics", True): + endpoints = endpoints.filter(finding__active=True) + endpoints = prefetch_related_endpoints_for_report(endpoints.distinct(), product=product) + endpoints = EndpointReportFilter(request.GET, queryset=endpoints) + + paged_endpoints = get_page_items(request, endpoints.qs, 25) + report_format = request.GET.get("report_type", "HTML") + include_finding_notes = int(request.GET.get("include_finding_notes", 0)) + include_finding_images = int(request.GET.get("include_finding_images", 0)) + include_executive_summary = int(request.GET.get("include_executive_summary", 0)) + include_table_of_contents = int(request.GET.get("include_table_of_contents", 0)) + include_disclaimer = int(request.GET.get("include_disclaimer", 0)) or (get_system_setting("disclaimer_reports_forced", 0)) + disclaimer = get_system_setting("disclaimer_reports") + if include_disclaimer and len(disclaimer) == 0: + disclaimer = "Please configure in System Settings." + generate = "_generate" in request.GET + add_breadcrumb(parent=product, title="Vulnerable Product Endpoints Report", top_level=False, request=request) + report_form = ReportOptionsForm() + template = "dojo/product_endpoint_pdf_report.html" + + if generate: + report_form = ReportOptionsForm(request.GET) + if report_format == "HTML": + return render(request, + template, + {"product_type": None, + "product": product, + "engagement": None, + "test": None, + "endpoint": None, + "endpoints": endpoints.qs, + "findings": None, + "include_finding_notes": include_finding_notes, + "include_finding_images": include_finding_images, + "include_executive_summary": include_executive_summary, + "include_table_of_contents": include_table_of_contents, + "include_disclaimer": include_disclaimer, + "disclaimer": disclaimer, + "user": request.user, + "title": "Generate Report", + }) + raise Http404 + + product_tab = Product_Tab(product, "Product Endpoint Report", tab="endpoints") + return render(request, + "dojo/request_endpoint_report.html", + {"endpoints": paged_endpoints, + "filtered": endpoints, + "product_tab": product_tab, + "report_form": report_form, + "name": "Vulnerable Product Endpoints", + }) + + +def generate_report(request, obj, *, host_view=False): + user = Dojo_User.objects.get(id=request.user.id) + product_type = None + product = None + engagement = None + test = None + endpoint = None + endpoints = None + report_title = None + + if isinstance(obj, (Product_Type, Product, Engagement, Test, Endpoint, Location)): + user_has_permission_or_403(request.user, obj, "view") + elif type(obj).__name__ in {"QuerySet", "CastTaggedQuerySet", "TagulousCastTaggedQuerySet"}: + # authorization taken care of by only selecting findings from product user is authed to see + pass + else: + if obj is None: + msg = "No object is given to generate report for" + raise Exception(msg) + msg = f"Report cannot be generated for object of type {type(obj).__name__}" + raise Exception(msg) + + report_format = request.GET.get("report_type", "HTML") + include_finding_notes = int(request.GET.get("include_finding_notes", 0)) + include_finding_images = int(request.GET.get("include_finding_images", 0)) + include_executive_summary = int(request.GET.get("include_executive_summary", 0)) + include_table_of_contents = int(request.GET.get("include_table_of_contents", 0)) + include_disclaimer = int(request.GET.get("include_disclaimer", 0)) or (get_system_setting("disclaimer_reports_forced", 0)) + disclaimer = get_system_setting("disclaimer_reports") + + if include_disclaimer and len(disclaimer) == 0: + disclaimer = "Please configure in System Settings." + generate = "_generate" in request.GET + report_name = str(obj) + filter_string_matching = get_system_setting("filter_string_matching", False) + report_finding_filter_class = ReportFindingFilterWithoutObjectLookups if filter_string_matching else ReportFindingFilter + add_breadcrumb(title="Generate Report", top_level=False, request=request) + + if isinstance(obj, Product_Type): + product_type = obj + template = "dojo/product_type_pdf_report.html" + report_name = labels.ORG_REPORT_WITH_NAME_TITLE % {"name": str(product_type)} + report_title = labels.ORG_REPORT_LABEL + findings = report_finding_filter_class(request.GET, prod_type=product_type, queryset=prefetch_related_findings_for_report(Finding.objects.filter( + test__engagement__product__prod_type=product_type))) + products = Product.objects.filter(prod_type=product_type, + engagement__test__finding__in=findings.qs).distinct() + engagements = Engagement.objects.filter(product__prod_type=product_type, + test__finding__in=findings.qs).distinct() + tests = Test.objects.filter(engagement__product__prod_type=product_type, + finding__in=findings.qs).distinct() + if len(findings.qs) > 0: + start_date = timezone.make_aware(datetime.combine(findings.qs.last().date, datetime.min.time())) + else: + start_date = timezone.now() + + end_date = timezone.now() + + r = relativedelta(end_date, start_date) + months_between = (r.years * 12) + r.months + # include current month + months_between += 1 + + endpoint_monthly_counts = get_period_counts_legacy(findings.qs.order_by("numerical_severity"), findings.qs.order_by("numerical_severity"), None, + months_between, start_date, + relative_delta="months") + + context = {"product_type": product_type, + "products": products, + "engagements": engagements, + "tests": tests, + "report_name": report_name, + "endpoint_opened_per_month": endpoint_monthly_counts[ + "opened_per_period"] if endpoint_monthly_counts is not None else [], + "endpoint_active_findings": findings.qs.distinct().order_by("numerical_severity"), + "findings": findings.qs.distinct().order_by("numerical_severity"), + "include_finding_notes": include_finding_notes, + "include_finding_images": include_finding_images, + "include_executive_summary": include_executive_summary, + "include_table_of_contents": include_table_of_contents, + "include_disclaimer": include_disclaimer, + "disclaimer": disclaimer, + "user": user, + "team_name": settings.TEAM_NAME, + "title": report_title, + "host": report_url_resolver(request), + "user_id": request.user.id} + + elif isinstance(obj, Product): + product = obj + template = "dojo/product_pdf_report.html" + report_name = labels.ASSET_REPORT_WITH_NAME_TITLE % {"name": str(product)} + report_title = labels.ASSET_REPORT_LABEL + findings = report_finding_filter_class(request.GET, product=product, queryset=prefetch_related_findings_for_report(Finding.objects.filter( + test__engagement__product=product))) + ids = findings.qs.values_list("id", flat=True) + engagements = Engagement.objects.filter(test__finding__id__in=ids).distinct() + tests = Test.objects.filter(finding__id__in=ids).distinct() + if settings.V3_FEATURE_LOCATIONS: + endpoints = Location.objects.prefetch_related("products__product").filter(products__product=product).distinct() + else: + # TODO: Delete this after the move to Locations + endpoints = Endpoint.objects.filter(product=product).distinct() + context = {"product": product, + "engagements": engagements, + "tests": tests, + "report_name": report_name, + "findings": findings.qs.distinct().order_by("numerical_severity"), + "include_finding_notes": include_finding_notes, + "include_finding_images": include_finding_images, + "include_executive_summary": include_executive_summary, + "include_table_of_contents": include_table_of_contents, + "include_disclaimer": include_disclaimer, + "disclaimer": disclaimer, + "user": user, + "team_name": settings.TEAM_NAME, + "title": report_title, + "endpoints": endpoints, + "host": report_url_resolver(request), + "user_id": request.user.id} + + elif isinstance(obj, Engagement): + logger.debug("generating report for Engagement") + engagement = obj + findings = report_finding_filter_class(request.GET, engagement=engagement, + queryset=prefetch_related_findings_for_report(Finding.objects.filter(test__engagement=engagement))) + report_name = "Engagement Report: " + str(engagement) + template = "dojo/engagement_pdf_report.html" + report_title = "Engagement Report" + + ids = findings.qs.values_list("id", flat=True) + tests = Test.objects.filter(finding__id__in=ids).distinct() + if settings.V3_FEATURE_LOCATIONS: + endpoints = Location.objects.prefetch_related("products__product").filter(products__product=engagement.product).distinct() + else: + # TODO: Delete this after the move to Locations + endpoints = Endpoint.objects.filter(product=engagement.product).distinct() + + context = {"engagement": engagement, + "tests": tests, + "report_name": report_name, + "findings": findings.qs.distinct().order_by("numerical_severity"), + "include_finding_notes": include_finding_notes, + "include_finding_images": include_finding_images, + "include_executive_summary": include_executive_summary, + "include_table_of_contents": include_table_of_contents, + "include_disclaimer": include_disclaimer, + "disclaimer": disclaimer, + "user": user, + "team_name": settings.TEAM_NAME, + "title": report_title, + "host": report_url_resolver(request), + "user_id": request.user.id, + "endpoints": endpoints} + + elif isinstance(obj, Test): + test = obj + findings = report_finding_filter_class(request.GET, engagement=test.engagement, + queryset=prefetch_related_findings_for_report(Finding.objects.filter(test=test))) + template = "dojo/test_pdf_report.html" + report_name = "Test Report: " + str(test) + report_title = "Test Report" + + context = {"test": test, + "report_name": report_name, + "findings": findings.qs.distinct().order_by("numerical_severity"), + "include_finding_notes": include_finding_notes, + "include_finding_images": include_finding_images, + "include_executive_summary": include_executive_summary, + "include_table_of_contents": include_table_of_contents, + "include_disclaimer": include_disclaimer, + "disclaimer": disclaimer, + "user": user, + "team_name": settings.TEAM_NAME, + "title": report_title, + "host": report_url_resolver(request), + "user_id": request.user.id} + + # TODO: Delete this after the move to Locations + elif isinstance(obj, Endpoint): + endpoint = obj + if host_view: + report_name = "Endpoint Host Report: " + endpoint.host + endpoints = Endpoint.objects.filter(host=endpoint.host, + product=endpoint.product).distinct() + report_title = "Endpoint Host Report" + else: + report_name = "Endpoint Report: " + str(endpoint) + endpoints = Endpoint.objects.filter(pk=endpoint.id).distinct() + report_title = "Endpoint Report" + template = "dojo/endpoint_pdf_report.html" + findings = report_finding_filter_class(request.GET, + queryset=prefetch_related_findings_for_report(Finding.objects.filter(endpoints__in=endpoints))) + + context = {"endpoint": endpoint, + "endpoints": endpoints, + "report_name": report_name, + "findings": findings.qs.distinct().order_by("numerical_severity"), + "include_finding_notes": include_finding_notes, + "include_finding_images": include_finding_images, + "include_executive_summary": include_executive_summary, + "include_table_of_contents": include_table_of_contents, + "include_disclaimer": include_disclaimer, + "disclaimer": disclaimer, + "user": user, + "team_name": get_system_setting("team_name"), + "title": report_title, + "host": report_url_resolver(request), + "user_id": request.user.id} + + elif isinstance(obj, Location): + endpoint = obj + if host_view: + report_name = "Endpoint Host Report: " + endpoint.url.host + endpoints = get_authorized_locations( + Permissions.Location_View, + queryset=Location.objects.prefetch_related("url").filter(url__host=endpoint.url.host), + user=request.user, + ).distinct() + report_title = "Endpoint Host Report" + else: + report_name = "Endpoint Report: " + str(endpoint) + endpoints = Location.objects.filter(id=endpoint.id).distinct() + report_title = "Endpoint Report" + template = "dojo/endpoint_pdf_report.html" + # Reduce the finding queryset to the requesting user's product scope -- + # a shared Location's auth check passes for any associated product the + # user can see, but rendered findings must still be limited to that scope. + findings_for_locations = get_authorized_findings( + Permissions.Finding_View, user=request.user, + ).filter(locations__location__in=endpoints) + findings = report_finding_filter_class( + request.GET, + queryset=prefetch_related_findings_for_report(findings_for_locations), + ) + + context = { + "endpoint": endpoint, + "endpoints": endpoints, + "report_name": report_name, + "findings": findings.qs.distinct().order_by("numerical_severity"), + "include_finding_notes": include_finding_notes, + "include_finding_images": include_finding_images, + "include_executive_summary": include_executive_summary, + "include_table_of_contents": include_table_of_contents, + "include_disclaimer": include_disclaimer, + "disclaimer": disclaimer, + "user": user, + "team_name": get_system_setting("team_name"), + "title": report_title, + "host": report_url_resolver(request), + "user_id": request.user.id, + } + + elif type(obj).__name__ in {"QuerySet", "CastTaggedQuerySet", "TagulousCastTaggedQuerySet"}: + findings = report_finding_filter_class(request.GET, queryset=prefetch_related_findings_for_report(obj).distinct()) + report_name = "Finding" + template = "dojo/finding_pdf_report.html" + report_title = "Finding Report" + + context = {"findings": findings.qs.distinct().order_by("numerical_severity"), + "report_name": report_name, + "include_finding_notes": include_finding_notes, + "include_finding_images": include_finding_images, + "include_executive_summary": include_executive_summary, + "include_table_of_contents": include_table_of_contents, + "include_disclaimer": include_disclaimer, + "disclaimer": disclaimer, + "user": user, + "team_name": settings.TEAM_NAME, + "title": report_title, + "host": report_url_resolver(request), + "user_id": request.user.id} + + else: + raise Http404 + + report_form = ReportOptionsForm() + + if generate: + report_form = ReportOptionsForm(request.GET) + if report_format == "HTML": + return render(request, + template, + {"product_type": product_type, + "product": product, + "engagement": engagement, + "report_name": report_name, + "test": test, + "endpoint": endpoint, + "endpoints": endpoints, + "findings": findings.qs.distinct().order_by("numerical_severity"), + "include_finding_notes": include_finding_notes, + "include_finding_images": include_finding_images, + "include_executive_summary": include_executive_summary, + "include_table_of_contents": include_table_of_contents, + "include_disclaimer": include_disclaimer, + "disclaimer": disclaimer, + "user": user, + "team_name": settings.TEAM_NAME, + "title": report_title, + "user_id": request.user.id, + "host": "", + "host_view": host_view, + "context": context, + }) + + raise Http404 + paged_findings = get_page_items(request, findings.qs.distinct().order_by("numerical_severity"), 25) + + product_tab = None + if engagement: + product_tab = Product_Tab(engagement.product, title="Engagement Report", tab="engagements") + product_tab.setEngagement(engagement) + elif test: + product_tab = Product_Tab(test.engagement.product, title="Test Report", tab="engagements") + product_tab.setEngagement(test.engagement) + elif product: + product_tab = Product_Tab(product, title=str(labels.ASSET_REPORT_LABEL), tab="findings") + elif endpoints: + # TODO: Delete this after the move to Locations + if not settings.V3_FEATURE_LOCATIONS: + if host_view: + product_tab = Product_Tab(endpoint.product, title="Endpoint Host Report", tab="endpoints") + else: + product_tab = Product_Tab(endpoint.product, title="Endpoint Report", tab="endpoints") + + return render(request, "dojo/request_report.html", + {"product_type": product_type, + "product": product, + "product_tab": product_tab, + "engagement": engagement, + "test": test, + "endpoint": endpoint, + "findings": findings, + "paged_findings": paged_findings, + "report_form": report_form, + "host_view": host_view, + "context": context, + }) + + +def get_list_index(full_list, index): + try: + element = full_list[index] + except Exception: + element = None + return element + + +def get_findings(request): + url = request.META.get("QUERY_STRING") + if not url: + msg = "Please use the report button when viewing findings" + raise Http404(msg) + url = url.removeprefix("url=") + + views = ["all", "open", "inactive", "verified", + "closed", "accepted", "out_of_scope", + "false_positive", "inactive"] + # request.path = url + obj_name = obj_id = view = query = None + path_items = list(filter(None, re.split(r"/|\?", url))) + + try: + finding_index = path_items.index("finding") + except ValueError: + finding_index = -1 + # There is a engagement or product here + if finding_index > 0: + # path_items ['product', '1', 'finding', 'closed', 'test__engagement__product=1'] + obj_name = get_list_index(path_items, 0) + obj_id = get_list_index(path_items, 1) + view = get_list_index(path_items, 3) + query = get_list_index(path_items, 4) + # Try to catch a mix up + query = query if view in views else view + # This is findings only. Accomodate view and query + elif finding_index == 0: + # path_items ['finding', 'closed', 'title=blah'] + obj_name = get_list_index(path_items, 0) + view = get_list_index(path_items, 1) + query = get_list_index(path_items, 2) + # Try to catch a mix up + query = query if view in views else view + # This is a test or engagement only + elif finding_index == -1: + # path_items ['test', '1', 'test__engagement__product=1'] + obj_name = get_list_index(path_items, 0) + obj_id = get_list_index(path_items, 1) + query = get_list_index(path_items, 2) + + filter_name = None + if view: + if view == "open": + filter_name = "Open" + elif view == "inactive": + filter_name = "Inactive" + elif view == "verified": + filter_name = "Verified" + elif view == "closed": + filter_name = "Closed" + elif view == "accepted": + filter_name = "Accepted" + elif view == "out_of_scope": + filter_name = "Out of Scope" + elif view == "false_positive": + filter_name = "False Positive" + + obj = pid = eid = tid = None + if obj_id: + if "product" in obj_name: + pid = obj_id + obj = get_object_or_404(Product, id=pid) + user_has_permission_or_403(request.user, obj, "view") + elif "engagement" in obj_name: + eid = obj_id + obj = get_object_or_404(Engagement, id=eid) + user_has_permission_or_403(request.user, obj, "view") + elif "test" in obj_name: + tid = obj_id + obj = get_object_or_404(Test, id=tid) + user_has_permission_or_403(request.user, obj, "view") + + request.GET = QueryDict(query) + list_findings = BaseListFindings( + filter_name=filter_name, + product_id=pid, + engagement_id=eid, + test_id=tid) + findings = list_findings.get_fully_filtered_findings(request).qs + + return findings, obj + + +class QuickReportView(View): + def add_findings_data(self): + return self.findings + + def get_template(self): + return "dojo/finding_pdf_report.html" + + def get(self, request): + findings, obj = get_findings(request) + self.findings = findings + findings = prefetch_related_findings_for_report(self.add_findings_data()) + return self.generate_quick_report(request, findings, obj) + + def generate_quick_report(self, request, findings, obj=None): + product = engagement = test = None + + if obj: + if type(obj).__name__ == "Product": + product = obj + elif type(obj).__name__ == "Engagement": + engagement = obj + elif type(obj).__name__ == "Test": + test = obj + + return render(request, self.get_template(), { + "report_name": "Finding Report", + "product": product, + "engagement": engagement, + "test": test, + "findings": findings, + "user": request.user, + "team_name": settings.TEAM_NAME, + "title": "Finding Report", + "user_id": request.user.id, + }) + + +def get_excludes(): + return ["SEVERITIES", "age", "github_issue", "jira_issue", "objects", "risk_acceptance", + "unsaved_endpoints", "unsaved_vulnerability_ids", "unsaved_files", "unsaved_request", "unsaved_response", + "unsaved_tags", "vulnerability_ids", "cve"] + + +def get_foreign_keys(): + return ["defect_review_requested_by", "duplicate_finding", "finding_group", "last_reviewed_by", + "mitigated_by", "reporter", "review_requested_by", "sonarqube_issue", "test"] + + +def get_attributes(): + return ["sla_age", "sla_deadline", "sla_days_remaining"] + + +class CSVExportView(View): + def add_findings_data(self): + return self.findings + + def add_extra_headers(self): + pass + + def add_extra_values(self): + pass + + def get(self, request): + findings, _obj = get_findings(request) + findings = prefetch_related_findings_for_report(findings) + self.findings = findings + findings = self.add_findings_data() + response = HttpResponse(content_type="text/csv") + response["Content-Disposition"] = "attachment; filename=findings.csv" + writer = csv.writer(response) + allowed_attributes = get_attributes() + excludes_list = get_excludes() + allowed_foreign_keys = get_foreign_keys() + first_row = True + + for finding in findings: + self.finding = finding + if first_row: + fields = [] + self.fields = fields + for key in dir(finding): + try: + if key not in excludes_list and (not callable(getattr(finding, key)) or key in allowed_attributes) and not key.startswith("_"): + if callable(getattr(finding, key)) and key not in allowed_attributes: + continue + fields.append(key) + except Exception as exc: + logger.error("Error in attribute: " + str(exc)) + fields.append(key) + continue + fields.extend(( + "test", + "found_by", + "engagement_id", + "engagement", + "product_id", + "product", + "endpoints", + "vulnerability_ids", + "tags", + "status", + "notes", + )) + self.fields = fields + self.add_extra_headers() + + writer.writerow(fields) + + first_row = False + if not first_row: + fields = [] + for key in dir(finding): + try: + if key not in excludes_list and (not callable(getattr(finding, key)) or key in allowed_attributes) and not key.startswith("_"): + if not callable(getattr(finding, key)): + value = finding.__dict__.get(key) + if (key in allowed_foreign_keys or key in allowed_attributes) and getattr(finding, key): + if callable(getattr(finding, key)): + func = getattr(finding, key) + result = func() + value = result + else: + value = str(getattr(finding, key)) + if value and isinstance(value, str): + value = value.replace("\n", " NEWLINE ").replace("\r", "") + fields.append(value) + except Exception as exc: + logger.error("Error in attribute: " + str(exc)) + fields.append("Value not supported") + continue + fields.append(finding.test.title) + fields.append(finding.test.test_type.name) + fields.append(finding.test.engagement.id) + fields.append(finding.test.engagement.name) + fields.append(finding.test.engagement.product.id) + fields.append(finding.test.engagement.product.name) + + endpoint_value = "" + for endpoint in finding.endpoints.all(): + endpoint_value += f"{endpoint}; " + endpoint_value = endpoint_value.removesuffix("; ") + if len(endpoint_value) > EXCEL_CHAR_LIMIT: + endpoint_value = endpoint_value[:EXCEL_CHAR_LIMIT - 3] + "..." + fields.append(endpoint_value) + + vulnerability_ids_value = "" + for num_vulnerability_ids, vulnerability_id in enumerate(finding.vulnerability_ids): + if num_vulnerability_ids > 5: + vulnerability_ids_value += "..." + break + vulnerability_ids_value += f"{vulnerability_id}; " + if finding.cve and vulnerability_ids_value.find(finding.cve) < 0: + vulnerability_ids_value += finding.cve + vulnerability_ids_value = vulnerability_ids_value.removesuffix("; ") + fields.append(vulnerability_ids_value) + # Tags + tags_value = "" + for num_tags, tag in enumerate(finding.tags.all()): + if num_tags > 5: + tags_value += "..." + break + tags_value += f"{tag}; " + tags_value = tags_value.removesuffix("; ") + fields.append(tags_value) + + # Status + status_value = finding.status() + fields.append(status_value) + + # Notes + notes_value = "" + for note in finding.notes.filter(private=False): + note_entry = note.entry.replace("\n", " NEWLINE ").replace("\r", "") + notes_value += f"{note_entry}; " + notes_value = notes_value.removesuffix("; ") + if len(notes_value) > EXCEL_CHAR_LIMIT: + notes_value = notes_value[:EXCEL_CHAR_LIMIT - 3] + "..." + fields.append(notes_value) + + self.fields = fields + self.finding = finding + self.add_extra_values() + + writer.writerow(fields) + + return response + + +class ExcelExportView(View): + + def add_findings_data(self): + return self.findings + + def add_extra_headers(self): + pass + + def add_extra_values(self): + pass + + def get(self, request): + findings, _obj = get_findings(request) + findings = prefetch_related_findings_for_report(findings) + self.findings = findings + findings = self.add_findings_data() + workbook = Workbook() + workbook.iso_dates = True + worksheet = workbook.active + worksheet.title = "Findings" + self.worksheet = worksheet + font_bold = Font(bold=True) + self.font_bold = font_bold + allowed_attributes = get_attributes() + excludes_list = get_excludes() + allowed_foreign_keys = get_foreign_keys() + + row_num = 1 + for finding in findings: + logger.debug(f"processing finding: {finding.id}") + if row_num == 1: + col_num = 1 + for key in dir(finding): + try: + if key not in excludes_list and (not callable(getattr(finding, key)) or key in allowed_attributes) and not key.startswith("_"): + if callable(getattr(finding, key)) and key not in allowed_attributes: + continue + cell = worksheet.cell(row=row_num, column=col_num, value=key) + cell.font = font_bold + col_num += 1 + except Exception as exc: + logger.warning(f"Error in attribute: {key}" + str(exc)) + cell = worksheet.cell(row=row_num, column=col_num, value=key) + col_num += 1 + continue + cell = worksheet.cell(row=row_num, column=col_num, value="found_by") + cell.font = font_bold + col_num += 1 + worksheet.cell(row=row_num, column=col_num, value="engagement_id") + cell = cell.font = font_bold + col_num += 1 + cell = worksheet.cell(row=row_num, column=col_num, value="engagement") + cell.font = font_bold + col_num += 1 + cell = worksheet.cell(row=row_num, column=col_num, value="product_id") + cell.font = font_bold + col_num += 1 + cell = worksheet.cell(row=row_num, column=col_num, value="product") + cell.font = font_bold + col_num += 1 + cell = worksheet.cell(row=row_num, column=col_num, value="endpoints") + cell.font = font_bold + col_num += 1 + cell = worksheet.cell(row=row_num, column=col_num, value="vulnerability_ids") + cell.font = font_bold + col_num += 1 + cell = worksheet.cell(row=row_num, column=col_num, value="tags") + cell.font = font_bold + col_num += 1 + cell = worksheet.cell(row=row_num, column=col_num, value="status") + cell.font = font_bold + col_num += 1 + cell = worksheet.cell(row=row_num, column=col_num, value="notes") + cell.font = font_bold + col_num += 1 + self.row_num = row_num + self.col_num = col_num + self.add_extra_headers() + + row_num = 2 + if row_num > 1: + col_num = 1 + for key in dir(finding): + try: + if key not in excludes_list and (not callable(getattr(finding, key)) or key in allowed_attributes) and not key.startswith("_"): + if not callable(getattr(finding, key)): + value = finding.__dict__.get(key) + if (key in allowed_foreign_keys or key in allowed_attributes) and getattr(finding, key): + if callable(getattr(finding, key)): + func = getattr(finding, key) + result = func() + value = result + else: + value = str(getattr(finding, key)) + if value and isinstance(value, datetime): + value = value.replace(tzinfo=None) + worksheet.cell(row=row_num, column=col_num, value=value) + col_num += 1 + except Exception as exc: + logger.warning(f"Error in attribute: {key}" + str(exc)) + worksheet.cell(row=row_num, column=col_num, value="Value not supported") + col_num += 1 + continue + worksheet.cell(row=row_num, column=col_num, value=finding.test.test_type.name) + col_num += 1 + worksheet.cell(row=row_num, column=col_num, value=finding.test.engagement.id) + col_num += 1 + worksheet.cell(row=row_num, column=col_num, value=finding.test.engagement.name) + col_num += 1 + worksheet.cell(row=row_num, column=col_num, value=finding.test.engagement.product.id) + col_num += 1 + worksheet.cell(row=row_num, column=col_num, value=finding.test.engagement.product.name) + col_num += 1 + + endpoint_value = "" + for endpoint in finding.endpoints.all(): + endpoint_value += f"{endpoint}; \n" + endpoint_value = endpoint_value.removesuffix("; \n") + if len(endpoint_value) > EXCEL_CHAR_LIMIT: + endpoint_value = endpoint_value[:EXCEL_CHAR_LIMIT - 3] + "..." + worksheet.cell(row=row_num, column=col_num, value=endpoint_value) + col_num += 1 + + vulnerability_ids_value = "" + for num_vulnerability_ids, vulnerability_id in enumerate(finding.vulnerability_ids): + if num_vulnerability_ids > 5: + vulnerability_ids_value += "..." + break + vulnerability_ids_value += f"{vulnerability_id}; \n" + if finding.cve and vulnerability_ids_value.find(finding.cve) < 0: + vulnerability_ids_value += finding.cve + vulnerability_ids_value = vulnerability_ids_value.removesuffix("; \n") + worksheet.cell(row=row_num, column=col_num, value=vulnerability_ids_value) + col_num += 1 + # tags + tags_value = "" + for tag in finding.tags.all(): + tags_value += f"{tag}; \n" + tags_value = tags_value.removesuffix("; \n") + worksheet.cell(row=row_num, column=col_num, value=tags_value) + col_num += 1 + + # Status + status_value = finding.status() + worksheet.cell(row=row_num, column=col_num, value=status_value) + col_num += 1 + + # Notes + notes_value = "" + for note in finding.notes.filter(private=False): + note_entry = note.entry.replace("\r", "") + notes_value += f"{note_entry}; \n" + notes_value = notes_value.removesuffix("; \n") + if len(notes_value) > EXCEL_CHAR_LIMIT: + notes_value = notes_value[:EXCEL_CHAR_LIMIT - 3] + "..." + worksheet.cell(row=row_num, column=col_num, value=notes_value) + col_num += 1 + + self.col_num = col_num + self.row_num = row_num + self.finding = finding + self.add_extra_values() + row_num += 1 + + with NamedTemporaryFile() as tmp: + workbook.save(tmp.name) + tmp.seek(0) + stream = tmp.read() + + response = HttpResponse( + content=stream, + content_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", + ) + response["Content-Disposition"] = "attachment; filename=findings.xlsx" + return response diff --git a/dojo/reports/views.py b/dojo/reports/views.py index f346dd89900..a4e62f95ffc 100644 --- a/dojo/reports/views.py +++ b/dojo/reports/views.py @@ -1,1143 +1,4 @@ -import csv -import logging -import re -from datetime import datetime -from tempfile import NamedTemporaryFile - -from dateutil.relativedelta import relativedelta -from django.conf import settings -from django.core.exceptions import PermissionDenied -from django.http import Http404, HttpRequest, HttpResponse, QueryDict -from django.shortcuts import get_object_or_404, render -from django.utils import timezone -from django.views import View -from openpyxl import Workbook -from openpyxl.styles import Font - -from dojo.authorization.authorization import user_has_permission_or_403 -from dojo.authorization.roles_permissions import Permissions -from dojo.endpoint.queries import get_authorized_endpoints -from dojo.filters import ( - EndpointFilter, - EndpointFilterWithoutObjectLookups, - EndpointReportFilter, - ReportFindingFilter, - ReportFindingFilterWithoutObjectLookups, -) -from dojo.finding.queries import get_authorized_findings -from dojo.finding.views import BaseListFindings -from dojo.forms import ReportOptionsForm -from dojo.labels import get_labels -from dojo.location.models import Location -from dojo.location.queries import get_authorized_locations -from dojo.location.status import FindingLocationStatus -from dojo.models import Dojo_User, Endpoint, Engagement, Finding, Product, Product_Type, Test -from dojo.reports.queries import prefetch_related_endpoints_for_report, prefetch_related_findings_for_report -from dojo.reports.widgets import ( - CoverPage, - CustomReportJsonForm, - EndpointList, - FindingList, - PageBreak, - ReportOptions, - TableOfContents, - Widget, - WYSIWYGContent, - report_widget_factory, -) -from dojo.url.filters import URLFilter -from dojo.utils import ( - Product_Tab, - add_breadcrumb, - get_page_items, - get_period_counts_legacy, - get_system_setting, - get_words_for_field, -) - -logger = logging.getLogger(__name__) - -labels = get_labels() - -EXCEL_CHAR_LIMIT = 32767 - - -def down(request): - return render(request, "disabled.html") - - -def report_url_resolver(request): - try: - url_resolver = request.META["HTTP_X_FORWARDED_PROTO"] + "://" + request.META["HTTP_X_FORWARDED_FOR"] - except: - hostname = request.META["HTTP_HOST"] - port_index = hostname.find(":") - if port_index != -1: - url_resolver = request.scheme + "://" + hostname[:port_index] - else: - url_resolver = request.scheme + "://" + hostname - return url_resolver + ":" + request.META["SERVER_PORT"] - - -class ReportBuilder(View): - def get(self, request: HttpRequest) -> HttpResponse: - add_breadcrumb(title="Report Builder", top_level=True, request=request) - return render(request, self.get_template(), self.get_context(request)) - - def get_findings(self, request: HttpRequest): - findings = get_authorized_findings("view") - filter_string_matching = get_system_setting("filter_string_matching", False) - filter_class = ReportFindingFilterWithoutObjectLookups if filter_string_matching else ReportFindingFilter - return filter_class(self.request.GET, queryset=findings) - - def get_endpoints(self, request: HttpRequest): - if settings.V3_FEATURE_LOCATIONS: - endpoints = Location.objects.filter(findings__status=FindingLocationStatus.Active).distinct() - filter_class = URLFilter - else: - endpoints = Endpoint.objects.filter( - finding__active=True, - finding__false_p=False, - finding__duplicate=False, - finding__out_of_scope=False, - ) - if get_system_setting("enforce_verified_status", True) or get_system_setting( - "enforce_verified_status_metrics", True, - ): - endpoints = endpoints.filter(finding__active=True) - endpoints = endpoints.distinct() - filter_string_matching = get_system_setting("filter_string_matching", False) - filter_class = EndpointFilterWithoutObjectLookups if filter_string_matching else EndpointFilter - return filter_class(request.GET, queryset=endpoints, user=request.user) - - def get_available_widgets(self, request: HttpRequest) -> list[Widget]: - return [ - CoverPage(request=request), - TableOfContents(request=request), - WYSIWYGContent(request=request), - FindingList(request=request, findings=self.get_findings(request)), - EndpointList(request=request, endpoints=self.get_endpoints(request)), - PageBreak()] - - def get_in_use_widgets(self, request): - return [ReportOptions(request=request)] - - def get_template(self): - return "dojo/report_builder.html" - - def get_context(self, request: HttpRequest) -> dict: - return { - "available_widgets": self.get_available_widgets(request), - "in_use_widgets": self.get_in_use_widgets(request)} - - -class CustomReport(View): - def post(self, request: HttpRequest) -> HttpResponse: - # saving the report - form = self.get_form(request) - if form.is_valid(): - self._set_state(request) - return render(request, self.get_template(), self.get_context()) - raise PermissionDenied - - def _set_state(self, request: HttpRequest): - self.request = request - self.host = report_url_resolver(request) - self.selected_widgets = self.get_selected_widgets(request) - self.widgets = list(self.selected_widgets.values()) - self.include_disclaimer = get_system_setting("disclaimer_reports_forced", 0) - self.disclaimer = get_system_setting("disclaimer_reports") - if self.include_disclaimer and len(self.disclaimer) == 0: - self.disclaimer = "Please configure in System Settings." - - def get_selected_widgets(self, request): - selected_widgets = report_widget_factory(json_data=request.POST["json"], request=request, host=self.host, - user=self.request.user, finding_notes=False, finding_images=False) - - if options := selected_widgets.get("report-options", None): - self.report_format = options.report_type - self.finding_notes = (options.include_finding_notes == "1") - self.finding_images = (options.include_finding_images == "1") - else: - self.report_format = "HTML" - self.finding_notes = True - self.finding_images = True - - return report_widget_factory(json_data=request.POST["json"], request=request, host=self.host, - user=request.user, finding_notes=self.finding_notes, - finding_images=self.finding_images) - - def get_form(self, request): - return CustomReportJsonForm(request.POST) - - def get_template(self): - if self.report_format == "HTML": - return "dojo/custom_html_report.html" - raise PermissionDenied - - def get_context(self): - return { - "widgets": self.widgets, - "host": self.host, - "finding_notes": self.finding_notes, - "finding_images": self.finding_images, - "user_id": self.request.user.id, - "include_disclaimer": self.include_disclaimer, - "disclaimer": self.disclaimer, - } - - -def report_findings(request): - findings = get_authorized_findings(Permissions.Finding_View) - filter_string_matching = get_system_setting("filter_string_matching", False) - filter_class = ReportFindingFilterWithoutObjectLookups if filter_string_matching else ReportFindingFilter - findings = filter_class(request.GET, queryset=findings) - - title_words = get_words_for_field(Finding, "title") - component_words = get_words_for_field(Finding, "component_name") - - paged_findings = get_page_items(request, findings.qs.distinct(), 25) - - return render(request, - "dojo/report_findings.html", - {"findings": paged_findings, - "filtered": findings, - "title_words": title_words, - "component_words": component_words, - "title": "finding-list", - "asset_label": labels.ASSET_LABEL, - }) - - -def report_endpoints(request): - if settings.V3_FEATURE_LOCATIONS: - endpoints = get_authorized_locations(Permissions.Location_View) - endpoints = endpoints.filter(findings__status=FindingLocationStatus.Active).distinct() - endpoints = URLFilter(request.GET, queryset=endpoints) - else: - # TODO: Delete this after the move to Locations - endpoints = get_authorized_endpoints(Permissions.Location_View).filter( - finding__active=True, - finding__false_p=False, - finding__duplicate=False, - finding__out_of_scope=False, - ) - if get_system_setting("enforce_verified_status", True) or get_system_setting( - "enforce_verified_status_metrics", True, - ): - endpoints = endpoints.filter(finding__active=True) - endpoints = endpoints.distinct() - endpoints = EndpointFilter(request.GET, queryset=endpoints, user=request.user) - - paged_endpoints = get_page_items(request, endpoints.qs, 25) - - return render(request, - "dojo/report_endpoints.html", - {"endpoints": paged_endpoints, - "filtered": endpoints, - "title": "endpoint-list", - }) - - -def report_cover_page(request): - report_title = request.GET.get("title", "Report") - report_subtitle = request.GET.get("subtitle", "") - report_info = request.GET.get("info", "") - - return render(request, - "dojo/report_cover_page.html", - {"report_title": report_title, - "report_subtitle": report_subtitle, - "report_info": report_info}) - - -def product_type_report(request, ptid): - product_type = get_object_or_404(Product_Type, id=ptid) - return generate_report(request, product_type) - - -def product_report(request, pid): - product = get_object_or_404(Product, id=pid) - return generate_report(request, product) - - -def product_findings_report(request): - findings = get_authorized_findings("view") - return generate_report(request, findings) - - -def engagement_report(request, eid): - engagement = get_object_or_404(Engagement, id=eid) - return generate_report(request, engagement) - - -def test_report(request, tid): - test = get_object_or_404(Test, id=tid) - return generate_report(request, test) - - -def product_endpoint_report(request, pid): - product = get_object_or_404(Product.objects.all().prefetch_related("engagement_set__test_set__test_type", "engagement_set__test_set__environment"), id=pid) - if settings.V3_FEATURE_LOCATIONS: - endpoints = Location.objects.filter(findings__status=FindingLocationStatus.Active) - endpoints = prefetch_related_endpoints_for_report(endpoints.distinct()) - endpoints = URLFilter(request.GET, queryset=endpoints) - else: - # TODO: Delete this after the move to Locations - endpoints = Endpoint.objects.filter(product=product, - finding__active=True, - finding__false_p=False, - finding__duplicate=False, - finding__out_of_scope=False) - if get_system_setting("enforce_verified_status", True) or get_system_setting("enforce_verified_status_metrics", True): - endpoints = endpoints.filter(finding__active=True) - endpoints = prefetch_related_endpoints_for_report(endpoints.distinct(), product=product) - endpoints = EndpointReportFilter(request.GET, queryset=endpoints) - - paged_endpoints = get_page_items(request, endpoints.qs, 25) - report_format = request.GET.get("report_type", "HTML") - include_finding_notes = int(request.GET.get("include_finding_notes", 0)) - include_finding_images = int(request.GET.get("include_finding_images", 0)) - include_executive_summary = int(request.GET.get("include_executive_summary", 0)) - include_table_of_contents = int(request.GET.get("include_table_of_contents", 0)) - include_disclaimer = int(request.GET.get("include_disclaimer", 0)) or (get_system_setting("disclaimer_reports_forced", 0)) - disclaimer = get_system_setting("disclaimer_reports") - if include_disclaimer and len(disclaimer) == 0: - disclaimer = "Please configure in System Settings." - generate = "_generate" in request.GET - add_breadcrumb(parent=product, title="Vulnerable Product Endpoints Report", top_level=False, request=request) - report_form = ReportOptionsForm() - template = "dojo/product_endpoint_pdf_report.html" - - if generate: - report_form = ReportOptionsForm(request.GET) - if report_format == "HTML": - return render(request, - template, - {"product_type": None, - "product": product, - "engagement": None, - "test": None, - "endpoint": None, - "endpoints": endpoints.qs, - "findings": None, - "include_finding_notes": include_finding_notes, - "include_finding_images": include_finding_images, - "include_executive_summary": include_executive_summary, - "include_table_of_contents": include_table_of_contents, - "include_disclaimer": include_disclaimer, - "disclaimer": disclaimer, - "user": request.user, - "title": "Generate Report", - }) - raise Http404 - - product_tab = Product_Tab(product, "Product Endpoint Report", tab="endpoints") - return render(request, - "dojo/request_endpoint_report.html", - {"endpoints": paged_endpoints, - "filtered": endpoints, - "product_tab": product_tab, - "report_form": report_form, - "name": "Vulnerable Product Endpoints", - }) - - -def generate_report(request, obj, *, host_view=False): - user = Dojo_User.objects.get(id=request.user.id) - product_type = None - product = None - engagement = None - test = None - endpoint = None - endpoints = None - report_title = None - - if isinstance(obj, (Product_Type, Product, Engagement, Test, Endpoint, Location)): - user_has_permission_or_403(request.user, obj, "view") - elif type(obj).__name__ in {"QuerySet", "CastTaggedQuerySet", "TagulousCastTaggedQuerySet"}: - # authorization taken care of by only selecting findings from product user is authed to see - pass - else: - if obj is None: - msg = "No object is given to generate report for" - raise Exception(msg) - msg = f"Report cannot be generated for object of type {type(obj).__name__}" - raise Exception(msg) - - report_format = request.GET.get("report_type", "HTML") - include_finding_notes = int(request.GET.get("include_finding_notes", 0)) - include_finding_images = int(request.GET.get("include_finding_images", 0)) - include_executive_summary = int(request.GET.get("include_executive_summary", 0)) - include_table_of_contents = int(request.GET.get("include_table_of_contents", 0)) - include_disclaimer = int(request.GET.get("include_disclaimer", 0)) or (get_system_setting("disclaimer_reports_forced", 0)) - disclaimer = get_system_setting("disclaimer_reports") - - if include_disclaimer and len(disclaimer) == 0: - disclaimer = "Please configure in System Settings." - generate = "_generate" in request.GET - report_name = str(obj) - filter_string_matching = get_system_setting("filter_string_matching", False) - report_finding_filter_class = ReportFindingFilterWithoutObjectLookups if filter_string_matching else ReportFindingFilter - add_breadcrumb(title="Generate Report", top_level=False, request=request) - - if isinstance(obj, Product_Type): - product_type = obj - template = "dojo/product_type_pdf_report.html" - report_name = labels.ORG_REPORT_WITH_NAME_TITLE % {"name": str(product_type)} - report_title = labels.ORG_REPORT_LABEL - findings = report_finding_filter_class(request.GET, prod_type=product_type, queryset=prefetch_related_findings_for_report(Finding.objects.filter( - test__engagement__product__prod_type=product_type))) - products = Product.objects.filter(prod_type=product_type, - engagement__test__finding__in=findings.qs).distinct() - engagements = Engagement.objects.filter(product__prod_type=product_type, - test__finding__in=findings.qs).distinct() - tests = Test.objects.filter(engagement__product__prod_type=product_type, - finding__in=findings.qs).distinct() - if len(findings.qs) > 0: - start_date = timezone.make_aware(datetime.combine(findings.qs.last().date, datetime.min.time())) - else: - start_date = timezone.now() - - end_date = timezone.now() - - r = relativedelta(end_date, start_date) - months_between = (r.years * 12) + r.months - # include current month - months_between += 1 - - endpoint_monthly_counts = get_period_counts_legacy(findings.qs.order_by("numerical_severity"), findings.qs.order_by("numerical_severity"), None, - months_between, start_date, - relative_delta="months") - - context = {"product_type": product_type, - "products": products, - "engagements": engagements, - "tests": tests, - "report_name": report_name, - "endpoint_opened_per_month": endpoint_monthly_counts[ - "opened_per_period"] if endpoint_monthly_counts is not None else [], - "endpoint_active_findings": findings.qs.distinct().order_by("numerical_severity"), - "findings": findings.qs.distinct().order_by("numerical_severity"), - "include_finding_notes": include_finding_notes, - "include_finding_images": include_finding_images, - "include_executive_summary": include_executive_summary, - "include_table_of_contents": include_table_of_contents, - "include_disclaimer": include_disclaimer, - "disclaimer": disclaimer, - "user": user, - "team_name": settings.TEAM_NAME, - "title": report_title, - "host": report_url_resolver(request), - "user_id": request.user.id} - - elif isinstance(obj, Product): - product = obj - template = "dojo/product_pdf_report.html" - report_name = labels.ASSET_REPORT_WITH_NAME_TITLE % {"name": str(product)} - report_title = labels.ASSET_REPORT_LABEL - findings = report_finding_filter_class(request.GET, product=product, queryset=prefetch_related_findings_for_report(Finding.objects.filter( - test__engagement__product=product))) - ids = findings.qs.values_list("id", flat=True) - engagements = Engagement.objects.filter(test__finding__id__in=ids).distinct() - tests = Test.objects.filter(finding__id__in=ids).distinct() - if settings.V3_FEATURE_LOCATIONS: - endpoints = Location.objects.prefetch_related("products__product").filter(products__product=product).distinct() - else: - # TODO: Delete this after the move to Locations - endpoints = Endpoint.objects.filter(product=product).distinct() - context = {"product": product, - "engagements": engagements, - "tests": tests, - "report_name": report_name, - "findings": findings.qs.distinct().order_by("numerical_severity"), - "include_finding_notes": include_finding_notes, - "include_finding_images": include_finding_images, - "include_executive_summary": include_executive_summary, - "include_table_of_contents": include_table_of_contents, - "include_disclaimer": include_disclaimer, - "disclaimer": disclaimer, - "user": user, - "team_name": settings.TEAM_NAME, - "title": report_title, - "endpoints": endpoints, - "host": report_url_resolver(request), - "user_id": request.user.id} - - elif isinstance(obj, Engagement): - logger.debug("generating report for Engagement") - engagement = obj - findings = report_finding_filter_class(request.GET, engagement=engagement, - queryset=prefetch_related_findings_for_report(Finding.objects.filter(test__engagement=engagement))) - report_name = "Engagement Report: " + str(engagement) - template = "dojo/engagement_pdf_report.html" - report_title = "Engagement Report" - - ids = findings.qs.values_list("id", flat=True) - tests = Test.objects.filter(finding__id__in=ids).distinct() - if settings.V3_FEATURE_LOCATIONS: - endpoints = Location.objects.prefetch_related("products__product").filter(products__product=engagement.product).distinct() - else: - # TODO: Delete this after the move to Locations - endpoints = Endpoint.objects.filter(product=engagement.product).distinct() - - context = {"engagement": engagement, - "tests": tests, - "report_name": report_name, - "findings": findings.qs.distinct().order_by("numerical_severity"), - "include_finding_notes": include_finding_notes, - "include_finding_images": include_finding_images, - "include_executive_summary": include_executive_summary, - "include_table_of_contents": include_table_of_contents, - "include_disclaimer": include_disclaimer, - "disclaimer": disclaimer, - "user": user, - "team_name": settings.TEAM_NAME, - "title": report_title, - "host": report_url_resolver(request), - "user_id": request.user.id, - "endpoints": endpoints} - - elif isinstance(obj, Test): - test = obj - findings = report_finding_filter_class(request.GET, engagement=test.engagement, - queryset=prefetch_related_findings_for_report(Finding.objects.filter(test=test))) - template = "dojo/test_pdf_report.html" - report_name = "Test Report: " + str(test) - report_title = "Test Report" - - context = {"test": test, - "report_name": report_name, - "findings": findings.qs.distinct().order_by("numerical_severity"), - "include_finding_notes": include_finding_notes, - "include_finding_images": include_finding_images, - "include_executive_summary": include_executive_summary, - "include_table_of_contents": include_table_of_contents, - "include_disclaimer": include_disclaimer, - "disclaimer": disclaimer, - "user": user, - "team_name": settings.TEAM_NAME, - "title": report_title, - "host": report_url_resolver(request), - "user_id": request.user.id} - - # TODO: Delete this after the move to Locations - elif isinstance(obj, Endpoint): - endpoint = obj - if host_view: - report_name = "Endpoint Host Report: " + endpoint.host - endpoints = Endpoint.objects.filter(host=endpoint.host, - product=endpoint.product).distinct() - report_title = "Endpoint Host Report" - else: - report_name = "Endpoint Report: " + str(endpoint) - endpoints = Endpoint.objects.filter(pk=endpoint.id).distinct() - report_title = "Endpoint Report" - template = "dojo/endpoint_pdf_report.html" - findings = report_finding_filter_class(request.GET, - queryset=prefetch_related_findings_for_report(Finding.objects.filter(endpoints__in=endpoints))) - - context = {"endpoint": endpoint, - "endpoints": endpoints, - "report_name": report_name, - "findings": findings.qs.distinct().order_by("numerical_severity"), - "include_finding_notes": include_finding_notes, - "include_finding_images": include_finding_images, - "include_executive_summary": include_executive_summary, - "include_table_of_contents": include_table_of_contents, - "include_disclaimer": include_disclaimer, - "disclaimer": disclaimer, - "user": user, - "team_name": get_system_setting("team_name"), - "title": report_title, - "host": report_url_resolver(request), - "user_id": request.user.id} - - elif isinstance(obj, Location): - endpoint = obj - if host_view: - report_name = "Endpoint Host Report: " + endpoint.url.host - endpoints = get_authorized_locations( - Permissions.Location_View, - queryset=Location.objects.prefetch_related("url").filter(url__host=endpoint.url.host), - user=request.user, - ).distinct() - report_title = "Endpoint Host Report" - else: - report_name = "Endpoint Report: " + str(endpoint) - endpoints = Location.objects.filter(id=endpoint.id).distinct() - report_title = "Endpoint Report" - template = "dojo/endpoint_pdf_report.html" - # Reduce the finding queryset to the requesting user's product scope -- - # a shared Location's auth check passes for any associated product the - # user can see, but rendered findings must still be limited to that scope. - findings_for_locations = get_authorized_findings( - Permissions.Finding_View, user=request.user, - ).filter(locations__location__in=endpoints) - findings = report_finding_filter_class( - request.GET, - queryset=prefetch_related_findings_for_report(findings_for_locations), - ) - - context = { - "endpoint": endpoint, - "endpoints": endpoints, - "report_name": report_name, - "findings": findings.qs.distinct().order_by("numerical_severity"), - "include_finding_notes": include_finding_notes, - "include_finding_images": include_finding_images, - "include_executive_summary": include_executive_summary, - "include_table_of_contents": include_table_of_contents, - "include_disclaimer": include_disclaimer, - "disclaimer": disclaimer, - "user": user, - "team_name": get_system_setting("team_name"), - "title": report_title, - "host": report_url_resolver(request), - "user_id": request.user.id, - } - - elif type(obj).__name__ in {"QuerySet", "CastTaggedQuerySet", "TagulousCastTaggedQuerySet"}: - findings = report_finding_filter_class(request.GET, queryset=prefetch_related_findings_for_report(obj).distinct()) - report_name = "Finding" - template = "dojo/finding_pdf_report.html" - report_title = "Finding Report" - - context = {"findings": findings.qs.distinct().order_by("numerical_severity"), - "report_name": report_name, - "include_finding_notes": include_finding_notes, - "include_finding_images": include_finding_images, - "include_executive_summary": include_executive_summary, - "include_table_of_contents": include_table_of_contents, - "include_disclaimer": include_disclaimer, - "disclaimer": disclaimer, - "user": user, - "team_name": settings.TEAM_NAME, - "title": report_title, - "host": report_url_resolver(request), - "user_id": request.user.id} - - else: - raise Http404 - - report_form = ReportOptionsForm() - - if generate: - report_form = ReportOptionsForm(request.GET) - if report_format == "HTML": - return render(request, - template, - {"product_type": product_type, - "product": product, - "engagement": engagement, - "report_name": report_name, - "test": test, - "endpoint": endpoint, - "endpoints": endpoints, - "findings": findings.qs.distinct().order_by("numerical_severity"), - "include_finding_notes": include_finding_notes, - "include_finding_images": include_finding_images, - "include_executive_summary": include_executive_summary, - "include_table_of_contents": include_table_of_contents, - "include_disclaimer": include_disclaimer, - "disclaimer": disclaimer, - "user": user, - "team_name": settings.TEAM_NAME, - "title": report_title, - "user_id": request.user.id, - "host": "", - "host_view": host_view, - "context": context, - }) - - raise Http404 - paged_findings = get_page_items(request, findings.qs.distinct().order_by("numerical_severity"), 25) - - product_tab = None - if engagement: - product_tab = Product_Tab(engagement.product, title="Engagement Report", tab="engagements") - product_tab.setEngagement(engagement) - elif test: - product_tab = Product_Tab(test.engagement.product, title="Test Report", tab="engagements") - product_tab.setEngagement(test.engagement) - elif product: - product_tab = Product_Tab(product, title=str(labels.ASSET_REPORT_LABEL), tab="findings") - elif endpoints: - # TODO: Delete this after the move to Locations - if not settings.V3_FEATURE_LOCATIONS: - if host_view: - product_tab = Product_Tab(endpoint.product, title="Endpoint Host Report", tab="endpoints") - else: - product_tab = Product_Tab(endpoint.product, title="Endpoint Report", tab="endpoints") - - return render(request, "dojo/request_report.html", - {"product_type": product_type, - "product": product, - "product_tab": product_tab, - "engagement": engagement, - "test": test, - "endpoint": endpoint, - "findings": findings, - "paged_findings": paged_findings, - "report_form": report_form, - "host_view": host_view, - "context": context, - }) - - -def get_list_index(full_list, index): - try: - element = full_list[index] - except Exception: - element = None - return element - - -def get_findings(request): - url = request.META.get("QUERY_STRING") - if not url: - msg = "Please use the report button when viewing findings" - raise Http404(msg) - url = url.removeprefix("url=") - - views = ["all", "open", "inactive", "verified", - "closed", "accepted", "out_of_scope", - "false_positive", "inactive"] - # request.path = url - obj_name = obj_id = view = query = None - path_items = list(filter(None, re.split(r"/|\?", url))) - - try: - finding_index = path_items.index("finding") - except ValueError: - finding_index = -1 - # There is a engagement or product here - if finding_index > 0: - # path_items ['product', '1', 'finding', 'closed', 'test__engagement__product=1'] - obj_name = get_list_index(path_items, 0) - obj_id = get_list_index(path_items, 1) - view = get_list_index(path_items, 3) - query = get_list_index(path_items, 4) - # Try to catch a mix up - query = query if view in views else view - # This is findings only. Accomodate view and query - elif finding_index == 0: - # path_items ['finding', 'closed', 'title=blah'] - obj_name = get_list_index(path_items, 0) - view = get_list_index(path_items, 1) - query = get_list_index(path_items, 2) - # Try to catch a mix up - query = query if view in views else view - # This is a test or engagement only - elif finding_index == -1: - # path_items ['test', '1', 'test__engagement__product=1'] - obj_name = get_list_index(path_items, 0) - obj_id = get_list_index(path_items, 1) - query = get_list_index(path_items, 2) - - filter_name = None - if view: - if view == "open": - filter_name = "Open" - elif view == "inactive": - filter_name = "Inactive" - elif view == "verified": - filter_name = "Verified" - elif view == "closed": - filter_name = "Closed" - elif view == "accepted": - filter_name = "Accepted" - elif view == "out_of_scope": - filter_name = "Out of Scope" - elif view == "false_positive": - filter_name = "False Positive" - - obj = pid = eid = tid = None - if obj_id: - if "product" in obj_name: - pid = obj_id - obj = get_object_or_404(Product, id=pid) - user_has_permission_or_403(request.user, obj, "view") - elif "engagement" in obj_name: - eid = obj_id - obj = get_object_or_404(Engagement, id=eid) - user_has_permission_or_403(request.user, obj, "view") - elif "test" in obj_name: - tid = obj_id - obj = get_object_or_404(Test, id=tid) - user_has_permission_or_403(request.user, obj, "view") - - request.GET = QueryDict(query) - list_findings = BaseListFindings( - filter_name=filter_name, - product_id=pid, - engagement_id=eid, - test_id=tid) - findings = list_findings.get_fully_filtered_findings(request).qs - - return findings, obj - - -class QuickReportView(View): - def add_findings_data(self): - return self.findings - - def get_template(self): - return "dojo/finding_pdf_report.html" - - def get(self, request): - findings, obj = get_findings(request) - self.findings = findings - findings = prefetch_related_findings_for_report(self.add_findings_data()) - return self.generate_quick_report(request, findings, obj) - - def generate_quick_report(self, request, findings, obj=None): - product = engagement = test = None - - if obj: - if type(obj).__name__ == "Product": - product = obj - elif type(obj).__name__ == "Engagement": - engagement = obj - elif type(obj).__name__ == "Test": - test = obj - - return render(request, self.get_template(), { - "report_name": "Finding Report", - "product": product, - "engagement": engagement, - "test": test, - "findings": findings, - "user": request.user, - "team_name": settings.TEAM_NAME, - "title": "Finding Report", - "user_id": request.user.id, - }) - - -def get_excludes(): - return ["SEVERITIES", "age", "github_issue", "jira_issue", "objects", "risk_acceptance", - "unsaved_endpoints", "unsaved_vulnerability_ids", "unsaved_files", "unsaved_request", "unsaved_response", - "unsaved_tags", "vulnerability_ids", "cve"] - - -def get_foreign_keys(): - return ["defect_review_requested_by", "duplicate_finding", "finding_group", "last_reviewed_by", - "mitigated_by", "reporter", "review_requested_by", "sonarqube_issue", "test"] - - -def get_attributes(): - return ["sla_age", "sla_deadline", "sla_days_remaining"] - - -class CSVExportView(View): - def add_findings_data(self): - return self.findings - - def add_extra_headers(self): - pass - - def add_extra_values(self): - pass - - def get(self, request): - findings, _obj = get_findings(request) - findings = prefetch_related_findings_for_report(findings) - self.findings = findings - findings = self.add_findings_data() - response = HttpResponse(content_type="text/csv") - response["Content-Disposition"] = "attachment; filename=findings.csv" - writer = csv.writer(response) - allowed_attributes = get_attributes() - excludes_list = get_excludes() - allowed_foreign_keys = get_foreign_keys() - first_row = True - - for finding in findings: - self.finding = finding - if first_row: - fields = [] - self.fields = fields - for key in dir(finding): - try: - if key not in excludes_list and (not callable(getattr(finding, key)) or key in allowed_attributes) and not key.startswith("_"): - if callable(getattr(finding, key)) and key not in allowed_attributes: - continue - fields.append(key) - except Exception as exc: - logger.error("Error in attribute: " + str(exc)) - fields.append(key) - continue - fields.extend(( - "test", - "found_by", - "engagement_id", - "engagement", - "product_id", - "product", - "endpoints", - "vulnerability_ids", - "tags", - "status", - "notes", - )) - self.fields = fields - self.add_extra_headers() - - writer.writerow(fields) - - first_row = False - if not first_row: - fields = [] - for key in dir(finding): - try: - if key not in excludes_list and (not callable(getattr(finding, key)) or key in allowed_attributes) and not key.startswith("_"): - if not callable(getattr(finding, key)): - value = finding.__dict__.get(key) - if (key in allowed_foreign_keys or key in allowed_attributes) and getattr(finding, key): - if callable(getattr(finding, key)): - func = getattr(finding, key) - result = func() - value = result - else: - value = str(getattr(finding, key)) - if value and isinstance(value, str): - value = value.replace("\n", " NEWLINE ").replace("\r", "") - fields.append(value) - except Exception as exc: - logger.error("Error in attribute: " + str(exc)) - fields.append("Value not supported") - continue - fields.append(finding.test.title) - fields.append(finding.test.test_type.name) - fields.append(finding.test.engagement.id) - fields.append(finding.test.engagement.name) - fields.append(finding.test.engagement.product.id) - fields.append(finding.test.engagement.product.name) - - endpoint_value = "" - for endpoint in finding.endpoints.all(): - endpoint_value += f"{endpoint}; " - endpoint_value = endpoint_value.removesuffix("; ") - if len(endpoint_value) > EXCEL_CHAR_LIMIT: - endpoint_value = endpoint_value[:EXCEL_CHAR_LIMIT - 3] + "..." - fields.append(endpoint_value) - - vulnerability_ids_value = "" - for num_vulnerability_ids, vulnerability_id in enumerate(finding.vulnerability_ids): - if num_vulnerability_ids > 5: - vulnerability_ids_value += "..." - break - vulnerability_ids_value += f"{vulnerability_id}; " - if finding.cve and vulnerability_ids_value.find(finding.cve) < 0: - vulnerability_ids_value += finding.cve - vulnerability_ids_value = vulnerability_ids_value.removesuffix("; ") - fields.append(vulnerability_ids_value) - # Tags - tags_value = "" - for num_tags, tag in enumerate(finding.tags.all()): - if num_tags > 5: - tags_value += "..." - break - tags_value += f"{tag}; " - tags_value = tags_value.removesuffix("; ") - fields.append(tags_value) - - # Status - status_value = finding.status() - fields.append(status_value) - - # Notes - notes_value = "" - for note in finding.notes.filter(private=False): - note_entry = note.entry.replace("\n", " NEWLINE ").replace("\r", "") - notes_value += f"{note_entry}; " - notes_value = notes_value.removesuffix("; ") - if len(notes_value) > EXCEL_CHAR_LIMIT: - notes_value = notes_value[:EXCEL_CHAR_LIMIT - 3] + "..." - fields.append(notes_value) - - self.fields = fields - self.finding = finding - self.add_extra_values() - - writer.writerow(fields) - - return response - - -class ExcelExportView(View): - - def add_findings_data(self): - return self.findings - - def add_extra_headers(self): - pass - - def add_extra_values(self): - pass - - def get(self, request): - findings, _obj = get_findings(request) - findings = prefetch_related_findings_for_report(findings) - self.findings = findings - findings = self.add_findings_data() - workbook = Workbook() - workbook.iso_dates = True - worksheet = workbook.active - worksheet.title = "Findings" - self.worksheet = worksheet - font_bold = Font(bold=True) - self.font_bold = font_bold - allowed_attributes = get_attributes() - excludes_list = get_excludes() - allowed_foreign_keys = get_foreign_keys() - - row_num = 1 - for finding in findings: - logger.debug(f"processing finding: {finding.id}") - if row_num == 1: - col_num = 1 - for key in dir(finding): - try: - if key not in excludes_list and (not callable(getattr(finding, key)) or key in allowed_attributes) and not key.startswith("_"): - if callable(getattr(finding, key)) and key not in allowed_attributes: - continue - cell = worksheet.cell(row=row_num, column=col_num, value=key) - cell.font = font_bold - col_num += 1 - except Exception as exc: - logger.warning(f"Error in attribute: {key}" + str(exc)) - cell = worksheet.cell(row=row_num, column=col_num, value=key) - col_num += 1 - continue - cell = worksheet.cell(row=row_num, column=col_num, value="found_by") - cell.font = font_bold - col_num += 1 - worksheet.cell(row=row_num, column=col_num, value="engagement_id") - cell = cell.font = font_bold - col_num += 1 - cell = worksheet.cell(row=row_num, column=col_num, value="engagement") - cell.font = font_bold - col_num += 1 - cell = worksheet.cell(row=row_num, column=col_num, value="product_id") - cell.font = font_bold - col_num += 1 - cell = worksheet.cell(row=row_num, column=col_num, value="product") - cell.font = font_bold - col_num += 1 - cell = worksheet.cell(row=row_num, column=col_num, value="endpoints") - cell.font = font_bold - col_num += 1 - cell = worksheet.cell(row=row_num, column=col_num, value="vulnerability_ids") - cell.font = font_bold - col_num += 1 - cell = worksheet.cell(row=row_num, column=col_num, value="tags") - cell.font = font_bold - col_num += 1 - cell = worksheet.cell(row=row_num, column=col_num, value="status") - cell.font = font_bold - col_num += 1 - cell = worksheet.cell(row=row_num, column=col_num, value="notes") - cell.font = font_bold - col_num += 1 - self.row_num = row_num - self.col_num = col_num - self.add_extra_headers() - - row_num = 2 - if row_num > 1: - col_num = 1 - for key in dir(finding): - try: - if key not in excludes_list and (not callable(getattr(finding, key)) or key in allowed_attributes) and not key.startswith("_"): - if not callable(getattr(finding, key)): - value = finding.__dict__.get(key) - if (key in allowed_foreign_keys or key in allowed_attributes) and getattr(finding, key): - if callable(getattr(finding, key)): - func = getattr(finding, key) - result = func() - value = result - else: - value = str(getattr(finding, key)) - if value and isinstance(value, datetime): - value = value.replace(tzinfo=None) - worksheet.cell(row=row_num, column=col_num, value=value) - col_num += 1 - except Exception as exc: - logger.warning(f"Error in attribute: {key}" + str(exc)) - worksheet.cell(row=row_num, column=col_num, value="Value not supported") - col_num += 1 - continue - worksheet.cell(row=row_num, column=col_num, value=finding.test.test_type.name) - col_num += 1 - worksheet.cell(row=row_num, column=col_num, value=finding.test.engagement.id) - col_num += 1 - worksheet.cell(row=row_num, column=col_num, value=finding.test.engagement.name) - col_num += 1 - worksheet.cell(row=row_num, column=col_num, value=finding.test.engagement.product.id) - col_num += 1 - worksheet.cell(row=row_num, column=col_num, value=finding.test.engagement.product.name) - col_num += 1 - - endpoint_value = "" - for endpoint in finding.endpoints.all(): - endpoint_value += f"{endpoint}; \n" - endpoint_value = endpoint_value.removesuffix("; \n") - if len(endpoint_value) > EXCEL_CHAR_LIMIT: - endpoint_value = endpoint_value[:EXCEL_CHAR_LIMIT - 3] + "..." - worksheet.cell(row=row_num, column=col_num, value=endpoint_value) - col_num += 1 - - vulnerability_ids_value = "" - for num_vulnerability_ids, vulnerability_id in enumerate(finding.vulnerability_ids): - if num_vulnerability_ids > 5: - vulnerability_ids_value += "..." - break - vulnerability_ids_value += f"{vulnerability_id}; \n" - if finding.cve and vulnerability_ids_value.find(finding.cve) < 0: - vulnerability_ids_value += finding.cve - vulnerability_ids_value = vulnerability_ids_value.removesuffix("; \n") - worksheet.cell(row=row_num, column=col_num, value=vulnerability_ids_value) - col_num += 1 - # tags - tags_value = "" - for tag in finding.tags.all(): - tags_value += f"{tag}; \n" - tags_value = tags_value.removesuffix("; \n") - worksheet.cell(row=row_num, column=col_num, value=tags_value) - col_num += 1 - - # Status - status_value = finding.status() - worksheet.cell(row=row_num, column=col_num, value=status_value) - col_num += 1 - - # Notes - notes_value = "" - for note in finding.notes.filter(private=False): - note_entry = note.entry.replace("\r", "") - notes_value += f"{note_entry}; \n" - notes_value = notes_value.removesuffix("; \n") - if len(notes_value) > EXCEL_CHAR_LIMIT: - notes_value = notes_value[:EXCEL_CHAR_LIMIT - 3] + "..." - worksheet.cell(row=row_num, column=col_num, value=notes_value) - col_num += 1 - - self.col_num = col_num - self.row_num = row_num - self.finding = finding - self.add_extra_values() - row_num += 1 - - with NamedTemporaryFile() as tmp: - workbook.save(tmp.name) - tmp.seek(0) - stream = tmp.read() - - response = HttpResponse( - content=stream, - content_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", - ) - response["Content-Disposition"] = "attachment; filename=findings.xlsx" - return response +# Backward-compat shim: the view logic moved to dojo.reports.ui.views during the +# module reorg. External consumers (e.g. dojo-pro) still import from +# dojo.reports.views, so re-export the public names from their new location. +from dojo.reports.ui.views import * # noqa: F403 -- backward compat re-export diff --git a/dojo/reports/widgets.py b/dojo/reports/widgets.py index aa88d9a4884..9b5375c1574 100644 --- a/dojo/reports/widgets.py +++ b/dojo/reports/widgets.py @@ -12,18 +12,17 @@ from django.utils.html import format_html from django.utils.safestring import mark_safe -from dojo.filters import ( - EndpointFilter, - EndpointFilterWithoutObjectLookups, +from dojo.endpoint.ui.filters import EndpointFilter, EndpointFilterWithoutObjectLookups +from dojo.finding.ui.filters import ( ReportFindingFilter, ReportFindingFilterWithoutObjectLookups, ) -from dojo.forms import CustomReportOptionsForm from dojo.labels import get_labels from dojo.location.models import Location from dojo.location.status import FindingLocationStatus from dojo.models import Endpoint, Finding from dojo.reports.queries import prefetch_related_endpoints_for_report, prefetch_related_findings_for_report +from dojo.reports.ui.forms import CustomReportOptionsForm from dojo.url.filters import URLFilter from dojo.utils import get_page_items, get_system_setting, get_words_for_field diff --git a/dojo/risk_acceptance/__init__.py b/dojo/risk_acceptance/__init__.py index e69de29bb2d..a5cede2cf2e 100644 --- a/dojo/risk_acceptance/__init__.py +++ b/dojo/risk_acceptance/__init__.py @@ -0,0 +1 @@ +import dojo.risk_acceptance.admin # noqa: F401 diff --git a/dojo/risk_acceptance/admin.py b/dojo/risk_acceptance/admin.py new file mode 100644 index 00000000000..cec3ae2295f --- /dev/null +++ b/dojo/risk_acceptance/admin.py @@ -0,0 +1,5 @@ +from django.contrib import admin + +from dojo.risk_acceptance.models import Risk_Acceptance + +admin.site.register(Risk_Acceptance) diff --git a/dojo/risk_acceptance/api/__init__.py b/dojo/risk_acceptance/api/__init__.py new file mode 100644 index 00000000000..97fa34f5990 --- /dev/null +++ b/dojo/risk_acceptance/api/__init__.py @@ -0,0 +1,12 @@ +path = "risk_acceptance" # noqa: RUF067 + +# Backward-compat: the AcceptedRisks/AcceptedFindings mixins + AcceptedRiskSerializer +# were historically importable as `dojo.risk_acceptance.api.` (via the old api.py). +# finding/test/engagement api viewsets consume them as `ra_api.` — keep them resolvable. +from dojo.risk_acceptance.api.mixins import ( # noqa: E402, F401 -- backward compat + AcceptedFindingsMixin, + AcceptedRisk, + AcceptedRiskSerializer, + AcceptedRisksMixin, + _accept_risks, +) diff --git a/dojo/risk_acceptance/api/filters.py b/dojo/risk_acceptance/api/filters.py new file mode 100644 index 00000000000..2ed3f4d8c15 --- /dev/null +++ b/dojo/risk_acceptance/api/filters.py @@ -0,0 +1,37 @@ +from dojo.filters import DateRangeFilter, DojoFilter, OrderingFilter +from dojo.models import Risk_Acceptance + + +class ApiRiskAcceptanceFilter(DojoFilter): + created = DateRangeFilter() + updated = DateRangeFilter() + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("name", "name"), + ("created", "created"), + ("updated", "updated"), + ), + ) + + class Meta: + model = Risk_Acceptance + fields = { + "name": ["exact", "icontains"], + "accepted_findings": ["exact"], + "recommendation": ["exact"], + "recommendation_details": ["exact", "icontains"], + "decision": ["exact"], + "decision_details": ["exact", "icontains"], + "accepted_by": ["exact", "icontains"], + "owner": ["exact"], + "expiration_date": ["exact", "gt", "lt", "gte", "lte"], + "expiration_date_warned": ["exact", "gt", "lt", "gte", "lte"], + "expiration_date_handled": ["exact", "gt", "lt", "gte", "lte"], + "reactivate_expired": ["exact"], + "restart_sla_expired": ["exact"], + "notes": ["exact"], + "created": ["exact", "gt", "lt", "gte", "lte"], + "updated": ["exact", "gt", "lt", "gte", "lte"], + } diff --git a/dojo/risk_acceptance/api.py b/dojo/risk_acceptance/api/mixins.py similarity index 98% rename from dojo/risk_acceptance/api.py rename to dojo/risk_acceptance/api/mixins.py index 78fa27062e9..cc245943d74 100644 --- a/dojo/risk_acceptance/api.py +++ b/dojo/risk_acceptance/api/mixins.py @@ -10,10 +10,10 @@ from rest_framework.permissions import IsAuthenticated from rest_framework.response import Response -from dojo.api_v2.serializers import RiskAcceptanceSerializer from dojo.authorization.api_permissions import UserHasRiskAcceptanceRelatedObjectPermission from dojo.engagement.queries import get_authorized_engagements from dojo.models import Engagement, Risk_Acceptance, User, Vulnerability_Id +from dojo.risk_acceptance.api.serializer import RiskAcceptanceSerializer AcceptedRisk = NamedTuple("AcceptedRisk", (("vulnerability_id", str), ("justification", str), ("accepted_by", str))) diff --git a/dojo/risk_acceptance/api/serializer.py b/dojo/risk_acceptance/api/serializer.py new file mode 100644 index 00000000000..39b4bdd879c --- /dev/null +++ b/dojo/risk_acceptance/api/serializer.py @@ -0,0 +1,137 @@ +from django.core.exceptions import PermissionDenied, ValidationError +from django.urls import reverse +from drf_spectacular.utils import extend_schema_field +from rest_framework import serializers + +import dojo.risk_acceptance.helper as ra_helper +from dojo.finding.queries import get_authorized_findings +from dojo.models import Engagement, Finding +from dojo.notes.api.serializer import NoteSerializer +from dojo.risk_acceptance.models import Risk_Acceptance + + +class RiskAcceptanceProofSerializer(serializers.ModelSerializer): + path = serializers.FileField(required=True) + + class Meta: + model = Risk_Acceptance + fields = ["path"] + + +class RiskAcceptanceToNotesSerializer(serializers.Serializer): + risk_acceptance_id = serializers.PrimaryKeyRelatedField( + queryset=Risk_Acceptance.objects.all(), many=False, allow_null=True, + ) + notes = NoteSerializer(many=True) + + +class RiskAcceptanceSerializer(serializers.ModelSerializer): + path = serializers.SerializerMethodField() + + def create(self, validated_data): + instance = super().create(validated_data) + user = getattr(self.context.get("request", None), "user", None) + ra_helper.add_findings_to_risk_acceptance(user, instance, instance.accepted_findings.all()) + + # Add risk acceptance to engagement + # This is fine as Pro has its own model + relationshop to track links with engagements. + if instance.accepted_findings.exists(): + engagement = instance.accepted_findings.first().test.engagement + engagement.risk_acceptance.add(instance) + + return instance + + def update(self, instance, validated_data): + # Determine findings to risk accept, and findings to unaccept risk + existing_findings = Finding.objects.filter(risk_acceptance=self.instance.id) + new_findings_ids = [x.id for x in validated_data.get("accepted_findings", [])] + new_findings = Finding.objects.filter(id__in=new_findings_ids) + findings_to_add = set(new_findings) - set(existing_findings) + findings_to_remove = set(existing_findings) - set(new_findings) + findings_to_add = Finding.objects.filter(id__in=[x.id for x in findings_to_add]) + findings_to_remove = Finding.objects.filter(id__in=[x.id for x in findings_to_remove]) + # Make the update in the database + instance = super().update(instance, validated_data) + user = getattr(self.context.get("request", None), "user", None) + # Add the new findings + ra_helper.add_findings_to_risk_acceptance(user, instance, findings_to_add) + # Remove the ones that were not present in the payload + for finding in findings_to_remove: + ra_helper.remove_finding_from_risk_acceptance(user, instance, finding) + + # Handle orphaned risk acceptances: link to engagement if it now has findings + # This is fine as Pro has its own model + relationshop to track links with engagements. + if instance.accepted_findings.exists() and not instance.engagement: + engagement = instance.accepted_findings.first().test.engagement + engagement.risk_acceptance.add(instance) + + return instance + + @extend_schema_field(serializers.CharField()) + def get_path(self, obj): + engagement = Engagement.objects.filter( + risk_acceptance__id__in=[obj.id], + ).first() + path = "No proof has been supplied" + if engagement and obj.filename() is not None: + path = reverse( + "download_risk_acceptance", args=(engagement.id, obj.id), + ) + request = self.context.get("request") + if request: + path = request.build_absolute_uri(path) + return path + + @extend_schema_field(serializers.IntegerField()) + def get_engagement(self, obj): + from dojo.engagement.api.serializer import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + EngagementSerializer, + ) + engagement = Engagement.objects.filter( + risk_acceptance__id__in=[obj.id], + ).first() + return EngagementSerializer(read_only=True).to_representation( + engagement, + ) + + def validate(self, data): + def validate_findings_have_same_engagement(finding_objects: list[Finding]): + engagements = finding_objects.values_list("test__engagement__id", flat=True).distinct().count() + if engagements > 1: + msg = "You are not permitted to add findings from multiple engagements" + raise PermissionDenied(msg) + + findings = data.get("accepted_findings", []) + findings_ids = [x.id for x in findings] + finding_objects = Finding.objects.filter(id__in=findings_ids) + authed_findings = get_authorized_findings("edit").filter(id__in=findings_ids) + if len(findings) != len(authed_findings): + msg = "You are not permitted to add one or more selected findings to this risk acceptance" + raise PermissionDenied(msg) + if self.context["request"].method == "POST": + validate_findings_have_same_engagement(finding_objects) + + # Validate product allows full risk acceptance BEFORE creating instance + if finding_objects.exists(): + engagement = finding_objects.first().test.engagement + if not engagement.product.enable_full_risk_acceptance: + msg = "Full risk acceptance is not enabled for this product" + raise PermissionDenied(msg) + elif self.context["request"].method in {"PATCH", "PUT"}: + # Use the reverse relation instead of filtering + existing_findings = self.instance.accepted_findings.all() + existing_and_new_findings = existing_findings | finding_objects + validate_findings_have_same_engagement(existing_and_new_findings) + + # Explicit check to prevent engagement switching + risk_acceptance_engagement = self.instance.engagement + if risk_acceptance_engagement and finding_objects.exists(): + new_findings_engagement = finding_objects.first().test.engagement + if risk_acceptance_engagement.id != new_findings_engagement.id: + msg = f"Risk Acceptance belongs to engagement {risk_acceptance_engagement.id}. Cannot add findings from engagement {new_findings_engagement.id}" + raise ValidationError(msg) + return data + + class Meta: + model = Risk_Acceptance + fields = "__all__" diff --git a/dojo/risk_acceptance/api/urls.py b/dojo/risk_acceptance/api/urls.py new file mode 100644 index 00000000000..5b3387ce2ce --- /dev/null +++ b/dojo/risk_acceptance/api/urls.py @@ -0,0 +1,7 @@ +from dojo.risk_acceptance.api import path +from dojo.risk_acceptance.api.views import RiskAcceptanceViewSet + + +def add_risk_acceptance_urls(router): + router.register(path, RiskAcceptanceViewSet, basename="risk_acceptance") + return router diff --git a/dojo/risk_acceptance/api/views.py b/dojo/risk_acceptance/api/views.py new file mode 100644 index 00000000000..1b6a540cf0a --- /dev/null +++ b/dojo/risk_acceptance/api/views.py @@ -0,0 +1,148 @@ +import mimetypes +from pathlib import Path + +from django.conf import settings +from django.http import FileResponse +from django.urls import reverse +from django_filters.rest_framework import DjangoFilterBackend +from drf_spectacular.utils import extend_schema +from rest_framework import status +from rest_framework.decorators import action +from rest_framework.permissions import IsAuthenticated +from rest_framework.response import Response + +from dojo.api_v2 import serializers as api_v2_serializers +from dojo.api_v2.views import PrefetchDojoModelViewSet +from dojo.authorization import api_permissions as permissions +from dojo.models import NoteHistory, Notes, Risk_Acceptance +from dojo.risk_acceptance.api.filters import ApiRiskAcceptanceFilter +from dojo.risk_acceptance.api.serializer import ( + RiskAcceptanceProofSerializer, + RiskAcceptanceSerializer, + RiskAcceptanceToNotesSerializer, +) +from dojo.risk_acceptance.helper import remove_finding_from_risk_acceptance +from dojo.risk_acceptance.queries import get_authorized_risk_acceptances +from dojo.utils import process_tag_notifications + + +class RiskAcceptanceViewSet( + PrefetchDojoModelViewSet, +): + serializer_class = RiskAcceptanceSerializer + queryset = Risk_Acceptance.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_class = ApiRiskAcceptanceFilter + + permission_classes = ( + IsAuthenticated, + permissions.UserHasRiskAcceptancePermission, + ) + + def destroy(self, request, pk=None): + instance = self.get_object() + # Remove any findings on the risk acceptance + for finding in instance.accepted_findings.all(): + remove_finding_from_risk_acceptance(request.user, instance, finding) + # return the response of the object being deleted + return super().destroy(request, pk=pk) + + def get_queryset(self): + return ( + get_authorized_risk_acceptances("edit") + .prefetch_related( + "notes", "engagement_set", "owner", "accepted_findings", + ) + .distinct() + ) + + @extend_schema( + methods=["GET"], + responses={ + status.HTTP_200_OK: RiskAcceptanceToNotesSerializer, + }, + ) + @extend_schema( + methods=["POST"], + request=api_v2_serializers.AddNewNoteOptionSerializer, + responses={status.HTTP_201_CREATED: api_v2_serializers.NoteSerializer}, + ) + @action(detail=True, methods=["get", "post"], permission_classes=(IsAuthenticated, permissions.UserHasRiskAcceptanceRelatedObjectPermission)) + def notes(self, request, pk=None): + risk_acceptance = self.get_object() + if request.method == "POST": + new_note = api_v2_serializers.AddNewNoteOptionSerializer(data=request.data) + if new_note.is_valid(): + entry = new_note.validated_data["entry"] + private = new_note.validated_data.get("private", False) + note_type = new_note.validated_data.get("note_type", None) + else: + return Response(new_note.errors, status=status.HTTP_400_BAD_REQUEST) + + notes = risk_acceptance.notes.filter(note_type=note_type).first() + if notes and note_type and note_type.is_single: + return Response("Only one instance of this note_type allowed on a risk acceptance.", status=status.HTTP_400_BAD_REQUEST) + + author = request.user + note = Notes(entry=entry, author=author, private=private, note_type=note_type) + note.save() + history = NoteHistory.objects.create(data=note.entry, time=note.date, current_editor=note.author) + note.history.add(history) + risk_acceptance.notes.add(note) + engagement = risk_acceptance.engagement + if engagement: + process_tag_notifications( + request=request, + note=note, + parent_url=request.build_absolute_uri( + reverse("view_risk_acceptance", args=(engagement.id, risk_acceptance.id)), + ), + parent_title=f"Risk Acceptance: {risk_acceptance.name}", + ) + + serialized_note = api_v2_serializers.NoteSerializer( + {"author": author, "entry": entry, "private": private}, + ) + return Response(serialized_note.data, status=status.HTTP_201_CREATED) + + notes = risk_acceptance.notes.all() + serialized_notes = RiskAcceptanceToNotesSerializer( + {"risk_acceptance_id": risk_acceptance, "notes": notes}, + ) + return Response(serialized_notes.data, status=status.HTTP_200_OK) + + @extend_schema( + methods=["GET"], + responses={ + status.HTTP_200_OK: RiskAcceptanceProofSerializer, + }, + ) + @action(detail=True, methods=["get"], permission_classes=(IsAuthenticated, permissions.UserHasRiskAcceptanceRelatedObjectPermission)) + def download_proof(self, request, pk=None): + risk_acceptance = self.get_object() + # Get the file object + file_object = risk_acceptance.path + if file_object is None or risk_acceptance.filename() is None: + return Response( + {"error": "Proof has not provided to this risk acceptance..."}, + status=status.HTTP_404_NOT_FOUND, + ) + # Get the path of the file in media root + file_path = Path(settings.MEDIA_ROOT) / file_object.name + # NOTE: FileResponse takes ownership of closing the file handle when the response is closed. + # Explicitly register the closer to avoid potential resource leaks and satisfy static analyzers. + file_handle = file_path.open("rb") + # send file + response = FileResponse( + file_handle, + content_type=mimetypes.guess_type(str(file_path))[0] or "application/octet-stream", + status=status.HTTP_200_OK, + ) + if hasattr(response, "_resource_closers"): + response._resource_closers.append(file_handle.close) + response["Content-Length"] = file_object.size + response[ + "Content-Disposition" + ] = f'attachment; filename="{risk_acceptance.filename()}"' + + return response diff --git a/dojo/risk_acceptance/models.py b/dojo/risk_acceptance/models.py new file mode 100644 index 00000000000..54354e67df2 --- /dev/null +++ b/dojo/risk_acceptance/models.py @@ -0,0 +1,113 @@ +from pathlib import Path + +from django.db import models +from django.urls import reverse +from django.utils import timezone +from django.utils.translation import gettext as _ + +# copy_model_util is defined early in dojo.models, before the re-export that loads this +# module, so this resolves despite the partial circular load. +from dojo.models import copy_model_util + + +class Risk_Acceptance(models.Model): + TREATMENT_ACCEPT = "A" + TREATMENT_AVOID = "V" + TREATMENT_MITIGATE = "M" + TREATMENT_FIX = "F" + TREATMENT_TRANSFER = "T" + + TREATMENT_TRANSLATIONS = { + TREATMENT_ACCEPT: _("Accept (The risk is acknowledged, yet remains)"), + TREATMENT_AVOID: _("Avoid (Do not engage with whatever creates the risk)"), + TREATMENT_MITIGATE: _("Mitigate (The risk still exists, yet compensating controls make it less of a threat)"), + TREATMENT_FIX: _("Fix (The risk is eradicated)"), + TREATMENT_TRANSFER: _("Transfer (The risk is transferred to a 3rd party)"), + } + + TREATMENT_CHOICES = [ + (TREATMENT_ACCEPT, TREATMENT_TRANSLATIONS[TREATMENT_ACCEPT]), + (TREATMENT_AVOID, TREATMENT_TRANSLATIONS[TREATMENT_AVOID]), + (TREATMENT_MITIGATE, TREATMENT_TRANSLATIONS[TREATMENT_MITIGATE]), + (TREATMENT_FIX, TREATMENT_TRANSLATIONS[TREATMENT_FIX]), + (TREATMENT_TRANSFER, TREATMENT_TRANSLATIONS[TREATMENT_TRANSFER]), + ] + + name = models.CharField(max_length=300, null=False, blank=False, help_text=_("Descriptive name which in the future may also be used to group risk acceptances together across engagements and products")) + + accepted_findings = models.ManyToManyField("dojo.Finding") + + recommendation = models.CharField(choices=TREATMENT_CHOICES, max_length=2, null=False, default=TREATMENT_FIX, help_text=_("Recommendation from the security team."), verbose_name=_("Security Recommendation")) + + recommendation_details = models.TextField(null=True, + blank=True, + help_text=_("Explanation of security recommendation"), verbose_name=_("Security Recommendation Details")) + + decision = models.CharField(choices=TREATMENT_CHOICES, max_length=2, null=False, default=TREATMENT_ACCEPT, help_text=_("Risk treatment decision by risk owner")) + decision_details = models.TextField(default=None, blank=True, null=True, help_text=_("If a compensating control exists to mitigate the finding or reduce risk, then list the compensating control(s).")) + + accepted_by = models.CharField(max_length=200, default=None, null=True, blank=True, verbose_name=_("Accepted By"), help_text=_("The person that accepts the risk, can be outside of DefectDojo.")) + path = models.FileField(upload_to="risk/%Y/%m/%d", + editable=True, null=True, + blank=True, verbose_name=_("Proof")) + owner = models.ForeignKey("dojo.Dojo_User", editable=True, on_delete=models.RESTRICT, help_text=_("User in DefectDojo owning this acceptance. Only the owner and staff users can edit the risk acceptance.")) + + expiration_date = models.DateTimeField(default=None, null=True, blank=True, help_text=_("When the risk acceptance expires, the findings will be reactivated (unless disabled below).")) + expiration_date_warned = models.DateTimeField(default=None, null=True, blank=True, help_text=_("(readonly) Date at which notice about the risk acceptance expiration was sent.")) + expiration_date_handled = models.DateTimeField(default=None, null=True, blank=True, help_text=_("(readonly) When the risk acceptance expiration was handled (manually or by the daily job).")) + reactivate_expired = models.BooleanField(null=False, blank=False, default=True, verbose_name=_("Reactivate findings on expiration"), help_text=_("Reactivate findings when risk acceptance expires?")) + restart_sla_expired = models.BooleanField(default=False, null=False, verbose_name=_("Restart SLA on expiration"), help_text=_("When enabled, the SLA for findings is restarted when the risk acceptance expires.")) + + notes = models.ManyToManyField("dojo.Notes", editable=False) + created = models.DateTimeField(auto_now_add=True, null=False) + updated = models.DateTimeField(auto_now=True, editable=False) + + def __str__(self): + return str(self.name) + + def filename(self): + # logger.debug('path: "%s"', self.path) + if not self.path: + return None + return Path(self.path.name).name + + @property + def name_and_expiration_info(self): + return str(self.name) + (" (expired " if self.is_expired else " (expires ") + (timezone.localtime(self.expiration_date).strftime("%b %d, %Y") if self.expiration_date else "Never") + ")" + + def get_breadcrumbs(self): + bc = self.engagement_set.first().get_breadcrumbs() + bc += [{"title": str(self), + "url": reverse("view_risk_acceptance", args=( + self.engagement_set.first().product.id, self.id))}] + return bc + + @property + def is_expired(self): + return self.expiration_date_handled is not None + + # relationship is many to many, but we use it as one-to-many + @property + def engagement(self): + engs = self.engagement_set.all() + if engs: + return engs[0] + + return None + + def copy(self, engagement=None): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + copy = copy_model_util(self) + # Save the necessary ManyToMany relationships + old_notes = list(self.notes.all()) + old_accepted_findings_hash_codes = [finding.hash_code for finding in self.accepted_findings.all()] + # Save the object before setting any ManyToMany relationships + copy.save() + # Copy the notes + for notes in old_notes: + copy.notes.add(notes.copy()) + # Assign any accepted findings + if engagement: + new_accepted_findings = Finding.objects.filter(test__engagement=engagement, hash_code__in=old_accepted_findings_hash_codes, risk_accepted=True).distinct() + copy.accepted_findings.set(new_accepted_findings) + return copy diff --git a/dojo/risk_acceptance/ui/__init__.py b/dojo/risk_acceptance/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/risk_acceptance/ui/forms.py b/dojo/risk_acceptance/ui/forms.py new file mode 100644 index 00000000000..8344bc0e769 --- /dev/null +++ b/dojo/risk_acceptance/ui/forms.py @@ -0,0 +1,80 @@ +import logging +from pathlib import Path + +from dateutil.relativedelta import relativedelta +from django import forms +from django.conf import settings +from django.core.exceptions import ValidationError +from django.utils import timezone + +from dojo.finding.queries import get_authorized_findings +from dojo.models import Finding, Risk_Acceptance +from dojo.utils import get_system_setting + +logger = logging.getLogger(__name__) + + +class EditRiskAcceptanceForm(forms.ModelForm): + # unfortunately django forces us to repeat many things here. choices, default, required etc. + recommendation = forms.ChoiceField(choices=Risk_Acceptance.TREATMENT_CHOICES, initial=Risk_Acceptance.TREATMENT_ACCEPT, widget=forms.RadioSelect, label="Security Recommendation") + decision = forms.ChoiceField(choices=Risk_Acceptance.TREATMENT_CHOICES, initial=Risk_Acceptance.TREATMENT_ACCEPT, widget=forms.RadioSelect) + + path = forms.FileField(label="Proof", required=False, widget=forms.widgets.FileInput(attrs={"accept": ", ".join(settings.FILE_IMPORT_TYPES)})) + expiration_date = forms.DateTimeField(required=False, widget=forms.TextInput(attrs={"class": "datepicker"})) + + class Meta: + model = Risk_Acceptance + exclude = ["accepted_findings", "notes"] + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.fields["path"].help_text = f"Existing proof uploaded: {self.instance.filename()}" if self.instance.filename() else "None" + self.fields["expiration_date_warned"].disabled = True + self.fields["expiration_date_handled"].disabled = True + + def clean_path(self): + if (data := self.cleaned_data.get("path")) is not None: + ext = Path(data.name).suffix # [0] returns path+filename + valid_extensions = settings.FILE_UPLOAD_TYPES + if ext.lower() not in valid_extensions: + if accepted_extensions := f"{', '.join(valid_extensions)}": + msg = f"Unsupported extension. Supported extensions are as follows: {accepted_extensions}" + else: + msg = "File uploads are prohibited due to the list of acceptable file extensions being empty" + raise ValidationError(msg) + return data + + +class RiskAcceptanceForm(EditRiskAcceptanceForm): + accepted_findings = forms.ModelMultipleChoiceField( + queryset=Finding.objects.none(), required=True, + widget=forms.widgets.SelectMultiple(attrs={"size": 10}), + help_text=("Active, verified findings listed, please select to add findings.")) + notes = forms.CharField(required=False, max_length=2400, + widget=forms.Textarea, + label="Notes") + + class Meta: + model = Risk_Acceptance + fields = "__all__" + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + expiration_delta_days = get_system_setting("risk_acceptance_form_default_days") + logger.debug("expiration_delta_days: %i", expiration_delta_days) + if expiration_delta_days > 0: + expiration_date = timezone.now().date() + relativedelta(days=expiration_delta_days) + # logger.debug('setting default expiration_date: %s', expiration_date) + self.fields["expiration_date"].initial = expiration_date + # self.fields['path'].help_text = 'Existing proof uploaded: %s' % self.instance.filename() if self.instance.filename() else 'None' + self.fields["accepted_findings"].queryset = get_authorized_findings("edit") + if disclaimer := get_system_setting("disclaimer_notes"): + self.disclaimer = disclaimer.strip() + + +class ReplaceRiskAcceptanceProofForm(forms.ModelForm): + path = forms.FileField(label="Proof", required=True, widget=forms.widgets.FileInput(attrs={"accept": ".jpg,.png,.pdf"})) + + class Meta: + model = Risk_Acceptance + fields = ["path"] diff --git a/dojo/search/views.py b/dojo/search/views.py index e7022ede68d..050286c5999 100644 --- a/dojo/search/views.py +++ b/dojo/search/views.py @@ -10,10 +10,10 @@ from watson import search as watson from dojo.endpoint.queries import get_authorized_endpoints -from dojo.endpoint.views import prefetch_for_endpoints +from dojo.endpoint.ui.views import prefetch_for_endpoints from dojo.engagement.queries import get_authorized_engagements -from dojo.filters import FindingFilter, FindingFilterWithoutObjectLookups from dojo.finding.queries import get_authorized_findings, get_authorized_vulnerability_ids, prefetch_for_findings +from dojo.finding.ui.filters import FindingFilter, FindingFilterWithoutObjectLookups from dojo.forms import FindingBulkUpdateForm, SimpleSearchForm from dojo.location.queries import get_authorized_locations, prefetch_for_locations from dojo.models import Engagement, Finding, Finding_Template, Languages, Product, Test diff --git a/dojo/survey/__init__.py b/dojo/survey/__init__.py index e69de29bb2d..dcf96374631 100644 --- a/dojo/survey/__init__.py +++ b/dojo/survey/__init__.py @@ -0,0 +1 @@ +import dojo.survey.admin # noqa: F401 diff --git a/dojo/survey/admin.py b/dojo/survey/admin.py new file mode 100644 index 00000000000..15b76ad2c8a --- /dev/null +++ b/dojo/survey/admin.py @@ -0,0 +1,5 @@ +from django.contrib import admin + +from dojo.survey.models import General_Survey + +admin.site.register(General_Survey) diff --git a/dojo/survey/models.py b/dojo/survey/models.py new file mode 100644 index 00000000000..6e8b98a9cf7 --- /dev/null +++ b/dojo/survey/models.py @@ -0,0 +1,181 @@ +import warnings +from datetime import timedelta + +from django.db import models +from django.utils import timezone +from django.utils.translation import gettext as _ +from django_extensions.db.models import TimeStampedModel +from polymorphic.base import ManagerInheritanceWarning +from polymorphic.managers import PolymorphicManager +from polymorphic.models import PolymorphicModel + +with warnings.catch_warnings(action="ignore", category=ManagerInheritanceWarning): + class Question(PolymorphicModel, TimeStampedModel): + + """Represents a question.""" + + class Meta: + ordering = ["order"] + + order = models.PositiveIntegerField(default=1, + help_text=_("The render order")) + + optional = models.BooleanField( + default=False, + help_text=_("If selected, user doesn't have to answer this question")) + + text = models.TextField(blank=False, help_text=_("The question text"), default="") + objects = models.Manager() + polymorphic = PolymorphicManager() + + def __str__(self): + return self.text + + +class TextQuestion(Question): + + """Question with a text answer""" + + objects = PolymorphicManager() + + def get_form(self): + """Returns the form for this model""" + from dojo.survey.ui.forms import TextQuestionForm # noqa: PLC0415 -- lazy import, avoids circular dependency + return TextQuestionForm + + +class Choice(TimeStampedModel): + + """Model to store the choices for multi choice questions""" + + order = models.PositiveIntegerField(default=1) + + label = models.TextField(default="") + + class Meta: + ordering = ["order"] + + def __str__(self): + return self.label + + +class ChoiceQuestion(Question): + + """ + Question with answers that are chosen from a list of choices defined + by the user. + """ + + multichoice = models.BooleanField(default=False, + help_text=_("Select one or more")) + choices = models.ManyToManyField("dojo.Choice") + objects = PolymorphicManager() + + def get_form(self): + """Returns the form for this model""" + from dojo.survey.ui.forms import ChoiceQuestionForm # noqa: PLC0415 -- lazy import, avoids circular dependency + return ChoiceQuestionForm + + +# meant to be a abstract survey, identified by name for purpose +class Engagement_Survey(models.Model): + name = models.CharField(max_length=200, null=False, blank=False, + editable=True, default="") + description = models.TextField(editable=True, default="") + questions = models.ManyToManyField("dojo.Question") + active = models.BooleanField(default=True) + + class Meta: + verbose_name = _("Engagement Survey") + verbose_name_plural = "Engagement Surveys" + ordering = ("-active", "name") + + def __str__(self): + return self.name + + +# meant to be an answered survey tied to an engagement + +class Answered_Survey(models.Model): + # tie this to a specific engagement + engagement = models.ForeignKey("dojo.Engagement", related_name="engagement+", + null=True, blank=False, editable=True, + on_delete=models.CASCADE) + # what surveys have been answered + survey = models.ForeignKey("dojo.Engagement_Survey", on_delete=models.CASCADE) + assignee = models.ForeignKey("dojo.Dojo_User", related_name="assignee", + null=True, blank=True, editable=True, + default=None, on_delete=models.RESTRICT) + # who answered it + responder = models.ForeignKey("dojo.Dojo_User", related_name="responder", + null=True, blank=True, editable=True, + default=None, on_delete=models.RESTRICT) + completed = models.BooleanField(default=False) + answered_on = models.DateField(null=True) + + class Meta: + verbose_name = _("Answered Engagement Survey") + verbose_name_plural = _("Answered Engagement Surveys") + + def __str__(self): + return self.survey.name + + +def default_expiration(): + return timezone.now() + timedelta(days=7) + + +class General_Survey(models.Model): + survey = models.ForeignKey("dojo.Engagement_Survey", on_delete=models.CASCADE) + num_responses = models.IntegerField(default=0) + generated = models.DateTimeField(auto_now_add=True, null=True) + expiration = models.DateTimeField(default=default_expiration) + + class Meta: + verbose_name = _("General Engagement Survey") + verbose_name_plural = _("General Engagement Surveys") + + def __str__(self): + return self.survey.name + + def clean(self): + if self.expiration and timezone.is_naive(self.expiration): + self.expiration = timezone.make_aware(self.expiration) + + +with warnings.catch_warnings(action="ignore", category=ManagerInheritanceWarning): + class Answer(PolymorphicModel, TimeStampedModel): + + """Base Answer model""" + + question = models.ForeignKey("dojo.Question", on_delete=models.CASCADE) + + answered_survey = models.ForeignKey("dojo.Answered_Survey", + null=False, + blank=False, + on_delete=models.CASCADE) + objects = models.Manager() + polymorphic = PolymorphicManager() + + +class TextAnswer(Answer): + answer = models.TextField( + blank=False, + help_text=_("The answer text"), + default="") + objects = PolymorphicManager() + + def __str__(self): + return self.answer + + +class ChoiceAnswer(Answer): + answer = models.ManyToManyField( + "dojo.Choice", + help_text=_("The selected choices as the answer")) + objects = PolymorphicManager() + + def __str__(self): + if len(self.answer.all()): + return str(self.answer.all()[0]) + return "No Response" diff --git a/dojo/survey/ui/__init__.py b/dojo/survey/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/survey/ui/filters.py b/dojo/survey/ui/filters.py new file mode 100644 index 00000000000..c1cac668f04 --- /dev/null +++ b/dojo/survey/ui/filters.py @@ -0,0 +1,63 @@ +import warnings + +import six +from django.contrib.contenttypes.models import ContentType +from django.utils.translation import gettext_lazy as _ +from django_filters import BooleanFilter, CharFilter, FilterSet +from django_filters.filters import ChoiceFilter +from polymorphic.base import ManagerInheritanceWarning + +from dojo.survey.models import ChoiceQuestion, Engagement_Survey, Question, TextQuestion + + +class QuestionnaireFilter(FilterSet): + name = CharFilter(lookup_expr="icontains") + description = CharFilter(lookup_expr="icontains") + active = BooleanFilter() + + class Meta: + model = Engagement_Survey + exclude = ["questions"] + + survey_set = FilterSet + + +class QuestionTypeFilter(ChoiceFilter): + def any(self, qs, name): + return qs.all() + + def text_question(self, qs, name): + return qs.filter(polymorphic_ctype=ContentType.objects.get_for_model(TextQuestion)) + + def choice_question(self, qs, name): + return qs.filter(polymorphic_ctype=ContentType.objects.get_for_model(ChoiceQuestion)) + + options = { + None: (_("Any"), any), # noqa: A003 -- shadows builtin; matches original dojo/filters.py pattern + 1: (_("Text Question"), text_question), + 2: (_("Choice Question"), choice_question), + } + + def __init__(self, *args, **kwargs): + kwargs["choices"] = [ + (key, value[0]) for key, value in six.iteritems(self.options)] + super().__init__(*args, **kwargs) + + def filter(self, qs, value): + try: + value = int(value) + except (ValueError, TypeError): + value = None + return self.options[value][1](self, qs, self.options[value][0]) + + +with warnings.catch_warnings(action="ignore", category=ManagerInheritanceWarning): + class QuestionFilter(FilterSet): + text = CharFilter(lookup_expr="icontains") + type = QuestionTypeFilter() + + class Meta: + model = Question + exclude = ["polymorphic_ctype", "created", "modified", "order"] + + question_set = FilterSet diff --git a/dojo/survey/ui/forms.py b/dojo/survey/ui/forms.py new file mode 100644 index 00000000000..72d1898f4b0 --- /dev/null +++ b/dojo/survey/ui/forms.py @@ -0,0 +1,417 @@ +import json +import warnings +from datetime import datetime + +from crispy_forms.bootstrap import InlineCheckboxes, InlineRadios +from crispy_forms.helper import FormHelper +from crispy_forms.layout import Layout +from django import forms +from django.db.models import Count +from django.utils import timezone +from polymorphic.base import ManagerInheritanceWarning + +from dojo.survey.models import ( + Answered_Survey, + Choice, + ChoiceAnswer, + ChoiceQuestion, + Engagement_Survey, + General_Survey, + Question, + TextAnswer, + TextQuestion, +) +from dojo.user.queries import get_authorized_users + + +class MultipleSelectWithPop(forms.SelectMultiple): + def render(self, name, *args, **kwargs): + from django.utils.safestring import mark_safe # noqa: PLC0415 -- lazy import, avoids circular dependency + html = super().render(name, *args, **kwargs) + popup_plus = '
' + html + '
' + return mark_safe(popup_plus) + + +# ============================== +# Defect Dojo Engaegment Surveys +# ============================== + +# List of validator_name:func_name +# Show in admin a multichoice list of validator names +# pass this to form using field_name='validator_name' ? +class QuestionForm(forms.Form): + + """Base class for a Question""" + + def __init__(self, *args, **kwargs): + self.helper = FormHelper() + self.helper.form_method = "post" + + # If true crispy-forms will render a
..
tags + self.helper.form_tag = kwargs.pop("form_tag", True) + + self.engagement_survey = kwargs.get("engagement_survey") + + self.answered_survey = kwargs.get("answered_survey") + if not self.answered_survey: + del kwargs["engagement_survey"] + else: + del kwargs["answered_survey"] + + self.helper.form_class = kwargs.get("form_class", "") + + self.question = kwargs.pop("question", None) + + if not self.question: + msg = "Need a question to render" + raise ValueError(msg) + + super().__init__(*args, **kwargs) + + +class TextQuestionForm(QuestionForm): + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + + # work out initial data + + initial_answer = TextAnswer.objects.filter( + answered_survey=self.answered_survey, + question=self.question, + ) + + initial_answer = initial_answer[0].answer if initial_answer.exists() else "" + + self.fields["answer"] = forms.CharField( + label=self.question.text, + widget=forms.Textarea(attrs={"rows": 3, "cols": 10}), + required=not self.question.optional, + initial=initial_answer, + ) + + def save(self): + if not self.is_valid(): + msg = "form is not valid" + raise forms.ValidationError(msg) + + answer = self.cleaned_data.get("answer") + + if not answer: + if self.fields["answer"].required: + msg = "Required" + raise forms.ValidationError(msg) + return + + text_answer, created = TextAnswer.objects.get_or_create( + answered_survey=self.answered_survey, + question=self.question, + ) + + if created: + text_answer.answered_survey = self.answered_survey + text_answer.answer = answer + text_answer.save() + + +class ChoiceQuestionForm(QuestionForm): + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + + choices = [(c.id, c.label) for c in self.question.choices.all()] + + # initial values + + initial_choices = [] + choice_answer = ChoiceAnswer.objects.filter( + answered_survey=self.answered_survey, + question=self.question, + ).annotate(a=Count("answer")).filter(a__gt=0) + + # we have ChoiceAnswer instance + if choice_answer: + choice_answer = choice_answer[0] + initial_choices = list(choice_answer.answer.all().values_list("id", flat=True)) + if self.question.multichoice is False: + initial_choices = initial_choices[0] + + # default classes + widget = forms.RadioSelect + field_type = forms.ChoiceField + inline_type = InlineRadios + + if self.question.multichoice: + field_type = forms.MultipleChoiceField + widget = forms.CheckboxSelectMultiple + inline_type = InlineCheckboxes + + field = field_type( + label=self.question.text, + required=not self.question.optional, + choices=choices, + initial=initial_choices, + widget=widget, + ) + + self.fields["answer"] = field + + # Render choice buttons inline + self.helper.layout = Layout( + inline_type("answer"), + ) + + def clean_answer(self): + real_answer = self.cleaned_data.get("answer") + + # for single choice questions, the selected answer is a single string + if not isinstance(real_answer, list): + real_answer = [real_answer] + return real_answer + + def save(self): + if not self.is_valid(): + msg = "Form is not valid" + raise forms.ValidationError(msg) + + real_answer = self.cleaned_data.get("answer") + + if not real_answer: + if self.fields["answer"].required: + msg = "Required" + raise forms.ValidationError(msg) + return + + choices = Choice.objects.filter(id__in=real_answer) + + # find ChoiceAnswer and filter in answer ! + choice_answer = ChoiceAnswer.objects.filter( + answered_survey=self.answered_survey, + question=self.question, + ) + + # we have ChoiceAnswer instance + if choice_answer: + choice_answer = choice_answer[0] + + if not choice_answer: + # create a ChoiceAnswer + choice_answer = ChoiceAnswer.objects.create( + answered_survey=self.answered_survey, + question=self.question, + ) + + # re save out the choices + choice_answer.answered_survey = self.answered_survey + choice_answer.answer.set(choices) + choice_answer.save() + + +class Add_Questionnaire_Form(forms.ModelForm): + survey = forms.ModelChoiceField( + queryset=Engagement_Survey.objects.all(), + required=True, + widget=forms.widgets.Select(), + help_text="Select the Questionnaire to add.") + + class Meta: + model = Answered_Survey + exclude = ("responder", + "completed", + "engagement", + "answered_on", + "assignee") + + +class AddGeneralQuestionnaireForm(forms.ModelForm): + survey = forms.ModelChoiceField( + queryset=Engagement_Survey.objects.all(), + required=True, + widget=forms.widgets.Select(), + help_text="Select the Questionnaire to add.") + expiration = forms.DateField(widget=forms.TextInput( + attrs={"class": "datepicker", "autocomplete": "off"})) + + class Meta: + model = General_Survey + exclude = ("num_responses", "generated") + + # date can only be today or in the past, not the future + def clean_expiration(self): + expiration = self.cleaned_data.get("expiration", None) + if expiration: + today = datetime.today().date() + if expiration < today: + msg = "The expiration cannot be in the past" + raise forms.ValidationError(msg) + if expiration == today: + msg = "The expiration cannot be today" + raise forms.ValidationError(msg) + return timezone.make_aware( + datetime.combine(expiration, datetime.min.time()), + ) + msg = "An expiration for the survey must be supplied" + raise forms.ValidationError(msg) + + +class Delete_Questionnaire_Form(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Answered_Survey + fields = ["id"] + + +class DeleteGeneralQuestionnaireForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = General_Survey + fields = ["id"] + + +class Delete_Eng_Survey_Form(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Engagement_Survey + fields = ["id"] + + +class CreateQuestionnaireForm(forms.ModelForm): + class Meta: + model = Engagement_Survey + exclude = ["questions"] + + +with warnings.catch_warnings(action="ignore", category=ManagerInheritanceWarning): + class EditQuestionnaireQuestionsForm(forms.ModelForm): + questions = forms.ModelMultipleChoiceField( + Question.polymorphic.all(), + required=True, + help_text="Select questions to include on this questionnaire. Field can be used to search available questions.", + widget=MultipleSelectWithPop(attrs={"size": "11"})) + + class Meta: + model = Engagement_Survey + exclude = ["name", "description", "active"] + + +class CreateQuestionForm(forms.Form): + type = forms.ChoiceField( + choices=(("---", "-----"), ("text", "Text"), ("choice", "Choice"))) + order = forms.IntegerField( + min_value=1, + widget=forms.TextInput(attrs={"data-type": "both"}), + help_text="The order the question will appear on the questionnaire") + optional = forms.BooleanField(help_text="If selected, user doesn't have to answer this question", + initial=False, + required=False, + widget=forms.CheckboxInput(attrs={"data-type": "both"})) + text = forms.CharField(widget=forms.Textarea(attrs={"data-type": "text"}), + label="Question Text", + help_text="The actual question.") + + +class CreateTextQuestionForm(forms.Form): + class Meta: + model = TextQuestion + exclude = ["order", "optional"] + + +class MultiWidgetBasic(forms.widgets.MultiWidget): + def __init__(self, attrs=None): + widgets = [forms.TextInput(attrs={"data-type": "choice"}), + forms.TextInput(attrs={"data-type": "choice"}), + forms.TextInput(attrs={"data-type": "choice"}), + forms.TextInput(attrs={"data-type": "choice"}), + forms.TextInput(attrs={"data-type": "choice"}), + forms.TextInput(attrs={"data-type": "choice"})] + super().__init__(widgets, attrs) + + def decompress(self, value): + if value: + return json.loads(value) + return [None, None, None, None, None, None] + + def format_output(self, rendered_widgets): + return "
".join(rendered_widgets) + + +class MultiExampleField(forms.fields.MultiValueField): + widget = MultiWidgetBasic + + def __init__(self, *args, **kwargs): + list_fields = [forms.fields.CharField(required=True), + forms.fields.CharField(required=True), + forms.fields.CharField(required=False), + forms.fields.CharField(required=False), + forms.fields.CharField(required=False), + forms.fields.CharField(required=False)] + super().__init__(list_fields, *args, **kwargs) + + def compress(self, values): + return json.dumps(values) + + +class CreateChoiceQuestionForm(forms.Form): + multichoice = forms.BooleanField(required=False, + initial=False, + widget=forms.CheckboxInput(attrs={"data-type": "choice"}), + help_text="Can more than one choice can be selected?") + + answer_choices = MultiExampleField(required=False, widget=MultiWidgetBasic(attrs={"data-type": "choice"})) + + class Meta: + model = ChoiceQuestion + exclude = ["order", "optional", "choices"] + + +class EditQuestionForm(forms.ModelForm): + class Meta: + model = Question + exclude = [] + + +class EditTextQuestionForm(EditQuestionForm): + class Meta: + model = TextQuestion + exclude = [] + + +class EditChoiceQuestionForm(EditQuestionForm): + choices = forms.ModelMultipleChoiceField( + Choice.objects.all(), + required=True, + help_text="Select choices to include on this question. Field can be used to search available choices.", + widget=MultipleSelectWithPop(attrs={"size": "11"})) + + class Meta: + model = ChoiceQuestion + exclude = [] + + +class AddChoicesForm(forms.ModelForm): + class Meta: + model = Choice + exclude = [] + + +class AssignUserForm(forms.ModelForm): + assignee = forms.CharField(required=False, + widget=forms.widgets.HiddenInput()) + + def __init__(self, *args, **kwargs): + assignee = None + if "assignee" in kwargs: + assignee = kwargs.pop("asignees") + super().__init__(*args, **kwargs) + if assignee is None: + self.fields["assignee"] = forms.ModelChoiceField(queryset=get_authorized_users("view"), empty_label="Not Assigned", required=False) + else: + self.fields["assignee"].initial = assignee + + class Meta: + model = Answered_Survey + exclude = ["engagement", "survey", "responder", "completed", "answered_on"] diff --git a/dojo/survey/urls.py b/dojo/survey/ui/urls.py similarity index 94% rename from dojo/survey/urls.py rename to dojo/survey/ui/urls.py index a592719aaf2..43865e055a1 100644 --- a/dojo/survey/urls.py +++ b/dojo/survey/ui/urls.py @@ -3,16 +3,9 @@ @author: jay7958 """ -from django.apps import apps -from django.contrib import admin from django.urls import re_path -from dojo.survey import views - -if not apps.ready: - apps.get_models() - -admin.autodiscover() +from dojo.survey.ui import views urlpatterns = [ re_path(r"^questionnaire$", diff --git a/dojo/survey/views.py b/dojo/survey/ui/views.py similarity index 99% rename from dojo/survey/views.py rename to dojo/survey/ui/views.py index b47e6bc3502..866184829c5 100644 --- a/dojo/survey/views.py +++ b/dojo/survey/ui/views.py @@ -18,11 +18,28 @@ user_has_permission, user_has_permission_or_403, ) -from dojo.filters import QuestionFilter, QuestionnaireFilter from dojo.forms import ( + AddEngagementForm, + ExistingEngagementForm, +) +from dojo.models import ( + Engagement, + System_Settings, +) +from dojo.survey.models import ( + Answer, + Answered_Survey, + Choice, + ChoiceQuestion, + Engagement_Survey, + General_Survey, + Question, + TextQuestion, +) +from dojo.survey.ui.filters import QuestionFilter, QuestionnaireFilter +from dojo.survey.ui.forms import ( Add_Questionnaire_Form, AddChoicesForm, - AddEngagementForm, AddGeneralQuestionnaireForm, AssignUserForm, CreateChoiceQuestionForm, @@ -35,19 +52,6 @@ EditChoiceQuestionForm, EditQuestionnaireQuestionsForm, EditTextQuestionForm, - ExistingEngagementForm, -) -from dojo.models import ( - Answer, - Answered_Survey, - Choice, - ChoiceQuestion, - Engagement, - Engagement_Survey, - General_Survey, - Question, - System_Settings, - TextQuestion, ) from dojo.utils import add_breadcrumb, get_page_items, get_setting diff --git a/dojo/system_settings/__init__.py b/dojo/system_settings/__init__.py index e69de29bb2d..50305d372ec 100644 --- a/dojo/system_settings/__init__.py +++ b/dojo/system_settings/__init__.py @@ -0,0 +1 @@ +import dojo.system_settings.admin # noqa: F401 diff --git a/dojo/system_settings/admin.py b/dojo/system_settings/admin.py new file mode 100644 index 00000000000..6a06a94bbf1 --- /dev/null +++ b/dojo/system_settings/admin.py @@ -0,0 +1,5 @@ +from django.contrib import admin + +from dojo.system_settings.models import System_Settings + +admin.site.register(System_Settings) diff --git a/dojo/system_settings/api/__init__.py b/dojo/system_settings/api/__init__.py new file mode 100644 index 00000000000..d8f9bbde95a --- /dev/null +++ b/dojo/system_settings/api/__init__.py @@ -0,0 +1 @@ +path = "system_settings" # noqa: RUF067 diff --git a/dojo/system_settings/api/serializer.py b/dojo/system_settings/api/serializer.py new file mode 100644 index 00000000000..c58fe7549b4 --- /dev/null +++ b/dojo/system_settings/api/serializer.py @@ -0,0 +1,9 @@ +from rest_framework import serializers + +from dojo.system_settings.models import System_Settings + + +class SystemSettingsSerializer(serializers.ModelSerializer): + class Meta: + model = System_Settings + fields = "__all__" diff --git a/dojo/system_settings/api/urls.py b/dojo/system_settings/api/urls.py new file mode 100644 index 00000000000..d22f65e5dbe --- /dev/null +++ b/dojo/system_settings/api/urls.py @@ -0,0 +1,7 @@ +from dojo.system_settings.api import path +from dojo.system_settings.api.views import SystemSettingsViewSet + + +def add_system_settings_urls(router): + router.register(path, SystemSettingsViewSet, basename="system_settings") + return router diff --git a/dojo/system_settings/api/views.py b/dojo/system_settings/api/views.py new file mode 100644 index 00000000000..3e3f6d90ec9 --- /dev/null +++ b/dojo/system_settings/api/views.py @@ -0,0 +1,21 @@ +from rest_framework import mixins, viewsets +from rest_framework.permissions import DjangoModelPermissions + +from dojo.authorization import api_permissions as permissions +from dojo.system_settings.api.serializer import SystemSettingsSerializer +from dojo.system_settings.models import System_Settings + + +# Authorization: superuser +class SystemSettingsViewSet( + mixins.ListModelMixin, mixins.UpdateModelMixin, viewsets.GenericViewSet, +): + + """Basic control over System Settings. Use 'id' 1 for PUT, PATCH operations""" + + permission_classes = (permissions.IsSuperUser, DjangoModelPermissions) + serializer_class = SystemSettingsSerializer + queryset = System_Settings.objects.none() + + def get_queryset(self): + return System_Settings.objects.all().order_by("id") diff --git a/dojo/system_settings/models.py b/dojo/system_settings/models.py new file mode 100644 index 00000000000..81024a58383 --- /dev/null +++ b/dojo/system_settings/models.py @@ -0,0 +1,365 @@ +from django.core.exceptions import ValidationError +from django.core.validators import MaxValueValidator, MinValueValidator +from django.db import models +from django.utils.translation import gettext as _ + + +class System_Settings(models.Model): + enable_deduplication = models.BooleanField( + default=False, + blank=False, + verbose_name=_("Deduplicate findings"), + help_text=_("With this setting turned on, DefectDojo deduplicates findings by " + "comparing endpoints, cwe fields, and titles. " + "If two findings share a URL and have the same CWE or " + "title, DefectDojo marks the recent finding as a duplicate. " + "When deduplication is enabled, a list of " + "deduplicated findings is added to the engagement view.")) + delete_duplicates = models.BooleanField(default=False, blank=False, help_text=_("Requires next setting: maximum number of duplicates to retain.")) + max_dupes = models.IntegerField(blank=True, null=True, default=10, + verbose_name=_("Max Duplicates"), + help_text=_("When enabled, if a single " + "issue reaches the maximum " + "number of duplicates, the " + "oldest will be deleted. Duplicate will not be deleted when left empty. A value of 0 will remove all duplicates.")) + + email_from = models.CharField(max_length=200, default="no-reply@example.com", blank=True) + + enable_jira = models.BooleanField(default=False, + verbose_name=_("Enable JIRA integration"), + blank=False) + + enable_jira_web_hook = models.BooleanField(default=False, + verbose_name=_("Enable JIRA web hook"), + help_text=_("Please note: It is strongly recommended to use a secret below and / or IP whitelist the JIRA server using a proxy such as Nginx."), + blank=False) + + disable_jira_webhook_secret = models.BooleanField(default=False, + verbose_name=_("Disable web hook secret"), + help_text=_("Allows incoming requests without a secret (discouraged legacy behaviour)"), + blank=False) + + # will be set to random / uuid by initializer so null needs to be True + jira_webhook_secret = models.CharField(max_length=64, blank=False, null=True, verbose_name=_("JIRA Webhook URL"), + help_text=_("Secret needed in URL for incoming JIRA Webhook")) + + jira_choices = (("Critical", "Critical"), + ("High", "High"), + ("Medium", "Medium"), + ("Low", "Low"), + ("Info", "Info")) + jira_minimum_severity = models.CharField(max_length=20, blank=True, + null=True, choices=jira_choices, + default="Low") + jira_labels = models.CharField(max_length=200, blank=True, null=True, + help_text=_("JIRA issue labels space seperated")) + + add_vulnerability_id_to_jira_label = models.BooleanField(default=False, + verbose_name=_("Add vulnerability Id as a JIRA label"), + blank=False) + + enable_github = models.BooleanField(default=False, + verbose_name=_("Enable GITHUB integration"), + blank=False) + + enable_slack_notifications = \ + models.BooleanField(default=False, + verbose_name=_("Enable Slack notifications"), + blank=False) + slack_channel = models.CharField(max_length=100, default="", blank=True, + help_text=_("Optional. Needed if you want to send global notifications.")) + slack_token = models.CharField(max_length=100, default="", blank=True, + help_text=_("Token required for interacting " + "with Slack. Get one at " + "https://api.slack.com/tokens")) + slack_username = models.CharField(max_length=100, default="", blank=True, + help_text=_("Optional. Will take your bot name otherwise.")) + enable_msteams_notifications = \ + models.BooleanField(default=False, + verbose_name=_("Enable Microsoft Teams notifications"), + blank=False) + msteams_url = models.CharField(max_length=400, default="", blank=True, + help_text=_("The full URL of the " + "incoming webhook")) + enable_mail_notifications = models.BooleanField(default=False, blank=False) + mail_notifications_to = models.CharField(max_length=200, default="", + blank=True) + + enable_webhooks_notifications = \ + models.BooleanField(default=False, + verbose_name=_("Enable Webhook notifications"), + blank=False) + webhooks_notifications_timeout = models.IntegerField(default=10, + help_text=_("How many seconds will DefectDojo waits for response from webhook endpoint")) + + enforce_verified_status = models.BooleanField( + default=True, + verbose_name=_("Enforce Verified Status - Globally"), + help_text=_( + "When enabled, features such as product grading, jira " + "integration, metrics, and reports will only interact " + "with verified findings. This setting will override " + "individually scoped verified toggles.", + ), + ) + enforce_verified_status_jira = models.BooleanField( + default=True, + verbose_name=_("Enforce Verified Status - Jira"), + help_text=_("When enabled, findings must have a verified status to be pushed to jira."), + ) + enforce_verified_status_product_grading = models.BooleanField( + default=True, + verbose_name=_("Enforce Verified Status - Product Grading"), + help_text=_( + "When enabled, findings must have a verified status to be considered as part of a product's grading.", + ), + ) + enforce_verified_status_metrics = models.BooleanField( + default=True, + verbose_name=_("Enforce Verified Status - Metrics"), + help_text=_( + "When enabled, findings must have a verified status to be counted in metric calculations, " + "be included in reports, and filters.", + ), + ) + + false_positive_history = models.BooleanField( + default=False, help_text=_( + "(EXPERIMENTAL) DefectDojo will automatically mark the finding as a " + "false positive if an equal finding (according to its dedupe algorithm) " + "has been previously marked as a false positive on the same product. " + "ATTENTION: Although the deduplication algorithm is used to determine " + "if a finding should be marked as a false positive, this feature will " + "not work if deduplication is enabled since it doesn't make sense to use both.", + ), + ) + + retroactive_false_positive_history = models.BooleanField( + default=False, help_text=_( + "(EXPERIMENTAL) FP History will also retroactively mark/unmark all " + "existing equal findings in the same product as a false positives. " + "Only works if the False Positive History feature is also enabled.", + ), + ) + + url_prefix = models.CharField(max_length=300, default="", blank=True, help_text=_("URL prefix if DefectDojo is installed in it's own virtual subdirectory.")) + team_name = models.CharField(max_length=100, default="", blank=True) + enable_product_grade = models.BooleanField(default=False, verbose_name=_("Enable Product Grading"), help_text=_("Displays a grade letter next to a product to show the overall health.")) + product_grade_a = models.IntegerField(default=90, + verbose_name=_("Grade A"), + help_text=_("Percentage score for an " + "'A' >=")) + product_grade_b = models.IntegerField(default=80, + verbose_name=_("Grade B"), + help_text=_("Percentage score for a " + "'B' >=")) + product_grade_c = models.IntegerField(default=70, + verbose_name=_("Grade C"), + help_text=_("Percentage score for a " + "'C' >=")) + product_grade_d = models.IntegerField(default=60, + verbose_name=_("Grade D"), + help_text=_("Percentage score for a " + "'D' >=")) + product_grade_f = models.IntegerField(default=59, + verbose_name=_("Grade F"), + help_text=_("Percentage score for an " + "'F' <=")) + enable_product_tag_inheritance = models.BooleanField( + default=False, + blank=False, + verbose_name=_("Enable Product Tag Inheritance"), + help_text=_("Enables product tag inheritance globally for all products. Any tags added on a product will automatically be added to all Engagements, Tests, and Findings")) + + enable_benchmark = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable Benchmarks"), + help_text=_("Enables Benchmarks such as the OWASP ASVS " + "(Application Security Verification Standard)")) + + enable_similar_findings = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable Similar Findings"), + help_text=_("Enable the query of similar findings on the view finding page. This feature can involve potentially large queries and negatively impact performance")) + + engagement_auto_close = models.BooleanField( + default=False, + blank=False, + verbose_name=_("Enable Engagement Auto-Close"), + help_text=_("Closes an engagement after 3 days (default) past due date including last update.")) + + engagement_auto_close_days = models.IntegerField( + default=3, + blank=False, + verbose_name=_("Engagement Auto-Close Days"), + help_text=_("Closes an engagement after the specified number of days past due date including last update.")) + + enable_finding_sla = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable Finding SLA's"), + help_text=_("Enables Finding SLA's for time to remediate.")) + + enable_notify_sla_active = models.BooleanField( + default=False, + blank=False, + verbose_name=_("Enable Notify SLA's Breach for active Findings"), + help_text=_("Enables Notify when time to remediate according to Finding SLA's is breached for active Findings.")) + + enable_notify_sla_active_verified = models.BooleanField( + default=False, + blank=False, + verbose_name=_("Enable Notify SLA's Breach for active, verified Findings"), + help_text=_("Enables Notify when time to remediate according to Finding SLA's is breached for active, verified Findings.")) + + enable_notify_sla_jira_only = models.BooleanField( + default=False, + blank=False, + verbose_name=_("Enable Notify SLA's Breach only for Findings linked to JIRA"), + help_text=_("Enables Notify when time to remediate according to Finding SLA's is breached for Findings that are linked to JIRA issues. Notification is disabled for Findings not linked to JIRA issues")) + + enable_notify_sla_exponential_backoff = models.BooleanField( + default=False, + blank=False, + verbose_name=_("Enable an exponential backoff strategy for SLA breach notifications."), + help_text=_("Enable an exponential backoff strategy for SLA breach notifications, e.g. 1, 2, 4, 8, etc. Otherwise it alerts every day")) + + allow_anonymous_survey_repsonse = models.BooleanField( + default=False, + blank=False, + verbose_name=_("Allow Anonymous Survey Responses"), + help_text=_("Enable anyone with a link to the survey to answer a survey"), + ) + disclaimer_notifications = models.TextField(max_length=3000, default="", blank=True, + verbose_name=_("Custom Disclaimer for Notifications"), + help_text=_("Include this custom disclaimer on all notifications")) + disclaimer_reports = models.TextField(max_length=5000, default="", blank=True, + verbose_name=_("Custom Disclaimer for Reports"), + help_text=_("Include this custom disclaimer on generated reports")) + disclaimer_reports_forced = models.BooleanField( + default=False, + blank=False, + verbose_name=_("Force to add disclaimer reports"), + help_text=_("Disclaimer will be added to all reports even if user didn't selected 'Include disclaimer'.")) + disclaimer_notes = models.TextField(max_length=3000, default="", blank=True, + verbose_name=_("Custom Disclaimer for Notes"), + help_text=_("Include this custom disclaimer next to input form for notes")) + risk_acceptance_form_default_days = models.IntegerField(null=True, blank=True, default=180, help_text=_("Default expiry period for risk acceptance form.")) + risk_acceptance_notify_before_expiration = models.IntegerField(null=True, blank=True, default=10, + verbose_name=_("Risk acceptance expiration heads up days"), help_text=_("Notify X days before risk acceptance expires. Leave empty to disable.")) + enable_questionnaires = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable questionnaires"), + help_text=_("With this setting turned off, questionnaires will be disabled in the user interface.")) + enable_checklists = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable checklists"), + help_text=_("With this setting turned off, checklists will be disabled in the user interface.")) + enable_endpoint_metadata_import = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable Endpoint Metadata Import"), + help_text=_("With this setting turned off, endpoint metadata import will be disabled in the user interface.")) + enable_user_profile_editable = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable user profile for writing"), + help_text=_("When turned on users can edit their profiles")) + enable_product_tracking_files = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable Product Tracking Files"), + help_text=_("With this setting turned off, the product tracking files will be disabled in the user interface.")) + enable_finding_groups = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable Finding Groups"), + help_text=_("With this setting turned off, the Finding Groups will be disabled.")) + enable_ui_table_based_searching = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable UI Table Based Filtering/Sorting"), + help_text=_("With this setting enabled, table headings will contain sort buttons for the current page of data in addition to sorting buttons that consider data from all pages.")) + enable_calendar = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable Calendar"), + help_text=_("With this setting turned off, the Calendar will be disabled in the user interface.")) + enable_cvss3_display = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable CVSS3 Display"), + help_text=_("With this setting turned off, CVSS3 fields will be hidden in the user interface.")) + enable_cvss4_display = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Enable CVSS4 Display"), + help_text=_("With this setting turned off, CVSS4 fields will be hidden in the user interface.")) + minimum_password_length = models.IntegerField( + default=9, + verbose_name=_("Minimum password length"), + help_text=_("Requires user to set passwords greater than minimum length."), + validators=[MinValueValidator(9), MaxValueValidator(48)]) + maximum_password_length = models.IntegerField( + default=48, + verbose_name=_("Maximum password length"), + help_text=_("Requires user to set passwords less than maximum length."), + validators=[MinValueValidator(9), MaxValueValidator(48)]) + number_character_required = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Password must contain one digit"), + help_text=_("Requires user passwords to contain at least one digit (0-9).")) + special_character_required = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Password must contain one special character"), + help_text=_("Requires user passwords to contain at least one special character (()[]{}|\\`~!@#$%^&*_-+=;:'\",<>./?).")) + lowercase_character_required = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Password must contain one lowercase letter"), + help_text=_("Requires user passwords to contain at least one lowercase letter (a-z).")) + uppercase_character_required = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Password must contain one uppercase letter"), + help_text=_("Requires user passwords to contain at least one uppercase letter (A-Z).")) + non_common_password_required = models.BooleanField( + default=True, + blank=False, + verbose_name=_("Password must not be common"), + help_text=_("Requires user passwords to not be part of list of common passwords.")) + api_expose_error_details = models.BooleanField( + default=False, + blank=False, + verbose_name=_("API expose error details"), + help_text=_("When turned on, the API will expose error details in the response.")) + filter_string_matching = models.BooleanField( + default=False, + blank=False, + verbose_name=_("Filter String Matching Optimization"), + help_text=_( + "When turned on, all filter operations in the UI will require string matches rather than ID. " + "This is a performance enhancement to avoid fetching objects unnecessarily.", + )) + + from dojo.middleware import System_Settings_Manager # noqa: PLC0415 circular import + objects = System_Settings_Manager() + + def clean(self): + super().clean() + + if ( + self.minimum_password_length is not None + and self.maximum_password_length is not None + ): + if self.minimum_password_length > self.maximum_password_length: + msg = "Minimum required password length must be larger than the maximum required password length." + raise ValidationError({ + "minimum_password_length": msg, + }) diff --git a/dojo/system_settings/ui/__init__.py b/dojo/system_settings/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/system_settings/ui/forms.py b/dojo/system_settings/ui/forms.py new file mode 100644 index 00000000000..699ed2add7f --- /dev/null +++ b/dojo/system_settings/ui/forms.py @@ -0,0 +1,41 @@ +from django import forms + +from dojo.labels import get_labels +from dojo.system_settings.models import System_Settings + +labels = get_labels() + + +class SystemSettingsForm(forms.ModelForm): + jira_webhook_secret = forms.CharField(required=False) + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + + self.fields["enable_product_tracking_files"].label = labels.SETTINGS_TRACKED_FILES_ENABLE_LABEL + self.fields["enable_product_tracking_files"].help_text = labels.SETTINGS_TRACKED_FILES_ENABLE_HELP + + self.fields[ + "enforce_verified_status_product_grading"].label = labels.SETTINGS_ASSET_GRADING_ENFORCE_VERIFIED_LABEL + self.fields[ + "enforce_verified_status_product_grading"].help_text = labels.SETTINGS_ASSET_GRADING_ENFORCE_VERIFIED_HELP + + self.fields["enable_product_grade"].label = labels.SETTINGS_ASSET_GRADING_ENABLE_LABEL + self.fields["enable_product_grade"].help_text = labels.SETTINGS_ASSET_GRADING_ENABLE_HELP + + self.fields["enable_product_tag_inheritance"].label = labels.SETTINGS_ASSET_TAG_INHERITANCE_ENABLE_LABEL + self.fields["enable_product_tag_inheritance"].help_text = labels.SETTINGS_ASSET_TAG_INHERITANCE_ENABLE_HELP + + def clean(self): + cleaned_data = super().clean() + enable_jira_value = cleaned_data.get("enable_jira") + jira_webhook_secret_value = cleaned_data.get("jira_webhook_secret").strip() + + if enable_jira_value and not jira_webhook_secret_value: + self.add_error("jira_webhook_secret", "This field is required when enable Jira Integration is True") + + return cleaned_data + + class Meta: + model = System_Settings + exclude = () diff --git a/dojo/system_settings/urls.py b/dojo/system_settings/ui/urls.py similarity index 87% rename from dojo/system_settings/urls.py rename to dojo/system_settings/ui/urls.py index 8268f6ee0ca..ff93931611d 100644 --- a/dojo/system_settings/urls.py +++ b/dojo/system_settings/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from dojo.system_settings import views +from dojo.system_settings.ui import views urlpatterns = [ re_path( diff --git a/dojo/system_settings/ui/views.py b/dojo/system_settings/ui/views.py new file mode 100644 index 00000000000..a2088f4b7ea --- /dev/null +++ b/dojo/system_settings/ui/views.py @@ -0,0 +1,124 @@ +import logging + +from django.contrib import messages +from django.core.exceptions import PermissionDenied +from django.http import HttpRequest, HttpResponse +from django.shortcuts import render +from django.views import View + +from dojo.system_settings.models import System_Settings +from dojo.system_settings.ui.forms import SystemSettingsForm +from dojo.utils import add_breadcrumb + +logger = logging.getLogger(__name__) + + +class SystemSettingsView(View): + def permission_check( + self, + request: HttpRequest, + ) -> None: + if not request.user.is_superuser: + raise PermissionDenied + + def get_settings_object(self) -> System_Settings: + return System_Settings.objects.get(no_cache=True) + + def get_context( + self, + request: HttpRequest, + ) -> dict: + system_settings_obj = self.get_settings_object() + return { + "system_settings_obj": system_settings_obj, + "form": self.get_form(request, system_settings_obj), + } + + def get_form( + self, + request: HttpRequest, + system_settings: System_Settings, + ) -> SystemSettingsForm: + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = { + "instance": system_settings, + } + + return SystemSettingsForm(*args, **kwargs) + + def validate_form( + self, + request: HttpRequest, + context: dict, + ) -> tuple[HttpRequest, bool]: + if context["form"].is_valid(): + if context["form"].cleaned_data["minimum_password_length"] >= context["form"].cleaned_data["maximum_password_length"]: + messages.add_message( + request, + messages.WARNING, + "Settings cannot be saved: Minimum required password length must be less than maximum required password length.", + extra_tags="alert-warning") + elif context["form"].cleaned_data["enable_deduplication"] is True and context["form"].cleaned_data["false_positive_history"] is True: + messages.add_message( + request, + messages.WARNING, + "Settings cannot be saved: Deduplicate findings and False positive history can not be set at the same time.", + extra_tags="alert-warning") + elif context["form"].cleaned_data["retroactive_false_positive_history"] is True and context["form"].cleaned_data["false_positive_history"] is False: + messages.add_message( + request, + messages.WARNING, + "Settings cannot be saved: Retroactive false positive history can not be set without False positive history.", + extra_tags="alert-warning") + else: + context["form"].save() + messages.add_message( + request, + messages.SUCCESS, + "Settings saved.", + extra_tags="alert-success") + return request, True + return request, False + + def get_template(self) -> str: + return "dojo/system_settings.html" + + def get( + self, + request: HttpRequest, + ) -> HttpResponse: + # permission check + self.permission_check(request) + # Set up the initial context + context = self.get_context(request) + # Add some breadcrumbs + add_breadcrumb(title="System settings", top_level=False, request=request) + # Render the page + return render(request, self.get_template(), context) + + def post( + self, + request: HttpRequest, + ) -> HttpResponse: + # permission check + self.permission_check(request) + # Set up the initial context + context = self.get_context(request) + request, _ = self.validate_form(request, context) + # Add some breadcrumbs + add_breadcrumb(title="System settings", top_level=False, request=request) + # Render the page + return render(request, self.get_template(), context) + + +class CeleryStatusView(View): + def get( + self, + request: HttpRequest, + ) -> HttpResponse: + if not request.user.is_superuser: + raise PermissionDenied + add_breadcrumb(title="Celery status", top_level=False, request=request) + return render(request, "dojo/celery_status.html") diff --git a/dojo/system_settings/views.py b/dojo/system_settings/views.py index 2b375627ae2..8fce6fa45c8 100644 --- a/dojo/system_settings/views.py +++ b/dojo/system_settings/views.py @@ -1,124 +1,4 @@ -import logging - -from django.contrib import messages -from django.core.exceptions import PermissionDenied -from django.http import HttpRequest, HttpResponse -from django.shortcuts import render -from django.views import View - -from dojo.forms import SystemSettingsForm -from dojo.models import System_Settings -from dojo.utils import add_breadcrumb - -logger = logging.getLogger(__name__) - - -class SystemSettingsView(View): - def permission_check( - self, - request: HttpRequest, - ) -> None: - if not request.user.is_superuser: - raise PermissionDenied - - def get_settings_object(self) -> System_Settings: - return System_Settings.objects.get(no_cache=True) - - def get_context( - self, - request: HttpRequest, - ) -> dict: - system_settings_obj = self.get_settings_object() - return { - "system_settings_obj": system_settings_obj, - "form": self.get_form(request, system_settings_obj), - } - - def get_form( - self, - request: HttpRequest, - system_settings: System_Settings, - ) -> SystemSettingsForm: - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = { - "instance": system_settings, - } - - return SystemSettingsForm(*args, **kwargs) - - def validate_form( - self, - request: HttpRequest, - context: dict, - ) -> tuple[HttpRequest, bool]: - if context["form"].is_valid(): - if context["form"].cleaned_data["minimum_password_length"] >= context["form"].cleaned_data["maximum_password_length"]: - messages.add_message( - request, - messages.WARNING, - "Settings cannot be saved: Minimum required password length must be less than maximum required password length.", - extra_tags="alert-warning") - elif context["form"].cleaned_data["enable_deduplication"] is True and context["form"].cleaned_data["false_positive_history"] is True: - messages.add_message( - request, - messages.WARNING, - "Settings cannot be saved: Deduplicate findings and False positive history can not be set at the same time.", - extra_tags="alert-warning") - elif context["form"].cleaned_data["retroactive_false_positive_history"] is True and context["form"].cleaned_data["false_positive_history"] is False: - messages.add_message( - request, - messages.WARNING, - "Settings cannot be saved: Retroactive false positive history can not be set without False positive history.", - extra_tags="alert-warning") - else: - context["form"].save() - messages.add_message( - request, - messages.SUCCESS, - "Settings saved.", - extra_tags="alert-success") - return request, True - return request, False - - def get_template(self) -> str: - return "dojo/system_settings.html" - - def get( - self, - request: HttpRequest, - ) -> HttpResponse: - # permission check - self.permission_check(request) - # Set up the initial context - context = self.get_context(request) - # Add some breadcrumbs - add_breadcrumb(title="System settings", top_level=False, request=request) - # Render the page - return render(request, self.get_template(), context) - - def post( - self, - request: HttpRequest, - ) -> HttpResponse: - # permission check - self.permission_check(request) - # Set up the initial context - context = self.get_context(request) - request, _ = self.validate_form(request, context) - # Add some breadcrumbs - add_breadcrumb(title="System settings", top_level=False, request=request) - # Render the page - return render(request, self.get_template(), context) - - -class CeleryStatusView(View): - def get( - self, - request: HttpRequest, - ) -> HttpResponse: - if not request.user.is_superuser: - raise PermissionDenied - add_breadcrumb(title="Celery status", top_level=False, request=request) - return render(request, "dojo/celery_status.html") +# Backward-compat shim: the view logic moved to dojo.system_settings.ui.views during the +# module reorg. External consumers (e.g. dojo-pro) still import from +# dojo.system_settings.views, so re-export the public names from their new location. +from dojo.system_settings.ui.views import * # noqa: F403 -- backward compat re-export diff --git a/dojo/test/__init__.py b/dojo/test/__init__.py index e69de29bb2d..1a931c0dba9 100644 --- a/dojo/test/__init__.py +++ b/dojo/test/__init__.py @@ -0,0 +1 @@ +import dojo.test.admin # noqa: F401 diff --git a/dojo/test/admin.py b/dojo/test/admin.py new file mode 100644 index 00000000000..a6f18c4bb82 --- /dev/null +++ b/dojo/test/admin.py @@ -0,0 +1,21 @@ +from django.contrib import admin + +from dojo.test.models import Test, Test_Import, Test_Type + + +@admin.register(Test_Type) +class Test_TypeAdmin(admin.ModelAdmin): + + """Admin support for the Test_Type model.""" + + +@admin.register(Test) +class TestAdmin(admin.ModelAdmin): + + """Admin support for the Test model.""" + + +@admin.register(Test_Import) +class Test_ImportAdmin(admin.ModelAdmin): + + """Admin support for the Test_Import model.""" diff --git a/dojo/test/api/__init__.py b/dojo/test/api/__init__.py new file mode 100644 index 00000000000..ab9d3d2e082 --- /dev/null +++ b/dojo/test/api/__init__.py @@ -0,0 +1 @@ +path = "tests" # noqa: RUF067 diff --git a/dojo/test/api/filters.py b/dojo/test/api/filters.py new file mode 100644 index 00000000000..9d5d0614653 --- /dev/null +++ b/dojo/test/api/filters.py @@ -0,0 +1,114 @@ +from django_filters import ( + BooleanFilter, + CharFilter, + OrderingFilter, +) + +from dojo.filters import ( + CharFieldFilterANDExpression, + CharFieldInFilter, + DojoFilter, +) +from dojo.labels import get_labels +from dojo.models import ( + Test, + Test_Import, +) + +labels = get_labels() + + +class ApiTestFilter(DojoFilter): + tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Tag name contains") + tags = CharFieldInFilter( + field_name="tags__name", + lookup_expr="in", + help_text="Comma separated list of exact tags (uses OR for multiple values)") + tags__and = CharFieldFilterANDExpression( + field_name="tags__name", + help_text="Comma separated list of exact tags to match with an AND expression") + engagement__tags = CharFieldInFilter( + field_name="engagement__tags__name", + lookup_expr="in", + help_text="Comma separated list of exact tags present on engagement (uses OR for multiple values)") + engagement__tags__and = CharFieldFilterANDExpression( + field_name="engagement__tags__name", + help_text="Comma separated list of exact tags to match with an AND expression present on engagement") + engagement__product__tags = CharFieldInFilter( + field_name="engagement__product__tags__name", + lookup_expr="in", + help_text=labels.ASSET_FILTERS_CSV_TAGS_OR_HELP) + engagement__product__tags__and = CharFieldFilterANDExpression( + field_name="engagement__product__tags__name", + help_text=labels.ASSET_FILTERS_CSV_TAGS_AND_HELP) + + not_tag = CharFilter(field_name="tags__name", lookup_expr="icontains", help_text="Not Tag name contains", exclude="True") + not_tags = CharFieldInFilter(field_name="tags__name", lookup_expr="in", + help_text="Comma separated list of exact tags not present on model", exclude="True") + not_engagement__tags = CharFieldInFilter(field_name="engagement__tags__name", lookup_expr="in", + help_text="Comma separated list of exact tags not present on engagement", + exclude="True") + not_engagement__product__tags = CharFieldInFilter(field_name="engagement__product__tags__name", + lookup_expr="in", + help_text=labels.ASSET_FILTERS_CSV_TAGS_NOT_HELP, + exclude="True") + has_tags = BooleanFilter(field_name="tags", lookup_expr="isnull", exclude=True, label="Has tags") + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("title", "title"), + ("version", "version"), + ("target_start", "target_start"), + ("target_end", "target_end"), + ("test_type", "test_type"), + ("lead", "lead"), + ("version", "version"), + ("branch_tag", "branch_tag"), + ("build_id", "build_id"), + ("commit_hash", "commit_hash"), + ("api_scan_configuration", "api_scan_configuration"), + ("engagement", "engagement"), + ("created", "created"), + ("updated", "updated"), + ), + field_labels={ + "name": "Test Name", + }, + ) + + class Meta: + model = Test + fields = ["id", "title", "test_type", "target_start", + "target_end", "notes", "percent_complete", + "engagement", "version", + "branch_tag", "build_id", "commit_hash", + "api_scan_configuration", "scan_type"] + + +class TestImportAPIFilter(DojoFilter): + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("id", "id"), + ("created", "created"), + ("modified", "modified"), + ("version", "version"), + ("branch_tag", "branch_tag"), + ("build_id", "build_id"), + ("commit_hash", "commit_hash"), + + ), + ) + + class Meta: + model = Test_Import + fields = ["test", + "findings_affected", + "version", + "branch_tag", + "build_id", + "commit_hash", + "test_import_finding_action__action", + "test_import_finding_action__finding", + "test_import_finding_action__created"] diff --git a/dojo/test/api/serializer.py b/dojo/test/api/serializer.py new file mode 100644 index 00000000000..c5dda0409a8 --- /dev/null +++ b/dojo/test/api/serializer.py @@ -0,0 +1,134 @@ +from django.conf import settings +from rest_framework import serializers + +from dojo.models import ( + Engagement, + Notes, + Test, + Test_Import, + Test_Import_Finding_Action, + Test_Type, +) + + +class TestSerializer(serializers.ModelSerializer): + test_type_name = serializers.ReadOnlyField() + + class Meta: + model = Test + exclude = ("inherited_tags",) + + def get_fields(self): + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + TagListSerializerField, + ) + from dojo.finding.api.serializer import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + FindingGroupSerializer, + ) + fields = super().get_fields() + fields["tags"] = TagListSerializerField(required=False) + fields["finding_groups"] = FindingGroupSerializer( + source="finding_group_set", many=True, read_only=True, + ) + return fields + + def build_relational_field(self, field_name, relation_info): + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + FileSerializer, + NoteSerializer, + ) + if field_name == "notes": + return NoteSerializer, {"many": True, "read_only": True} + if field_name == "files": + return FileSerializer, {"many": True, "read_only": True} + return super().build_relational_field(field_name, relation_info) + + +class TestCreateSerializer(serializers.ModelSerializer): + engagement = serializers.PrimaryKeyRelatedField( + queryset=Engagement.objects.all(), + ) + notes = serializers.PrimaryKeyRelatedField( + allow_null=True, + queryset=Notes.objects.all(), + many=True, + required=False, + ) + + class Meta: + model = Test + exclude = ("inherited_tags",) + + def get_fields(self): + from dojo.api_v2.serializers import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + TagListSerializerField, + ) + fields = super().get_fields() + fields["tags"] = TagListSerializerField(required=False) + return fields + + +class TestTypeCreateSerializer(serializers.ModelSerializer): + + class Meta: + model = Test_Type + exclude = ("dynamically_generated",) + + +class TestTypeSerializer(serializers.ModelSerializer): + name = serializers.ReadOnlyField() + + class Meta: + model = Test_Type + exclude = ("dynamically_generated",) + + +class TestToNotesSerializer(serializers.Serializer): + test_id = serializers.PrimaryKeyRelatedField( + queryset=Test.objects.all(), many=False, allow_null=True, + ) + + def get_fields(self): + from dojo.api_v2.serializers import NoteSerializer # noqa: PLC0415 -- lazy import, avoids circular dependency + fields = super().get_fields() + fields["notes"] = NoteSerializer(many=True) + return fields + + +class TestToFilesSerializer(serializers.Serializer): + test_id = serializers.PrimaryKeyRelatedField( + queryset=Test.objects.all(), many=False, allow_null=True, + ) + + def get_fields(self): + from dojo.api_v2.serializers import FileSerializer # noqa: PLC0415 -- lazy import, avoids circular dependency + fields = super().get_fields() + fields["files"] = FileSerializer(many=True) + return fields + + def to_representation(self, data): + test = data.get("test_id") + files = data.get("files") + new_files = [{ + "id": file.id, + "file": f"{settings.SITE_URL}/{file.get_accessible_url(test, test.id)}", + "title": file.title, + } for file in files] + return {"test_id": test.id, "files": new_files} + + +class TestImportFindingActionSerializer(serializers.ModelSerializer): + class Meta: + model = Test_Import_Finding_Action + fields = "__all__" + + +class TestImportSerializer(serializers.ModelSerializer): + # findings = TestImportFindingActionSerializer(source='test_import_finding_action', many=True, read_only=True) + test_import_finding_action_set = TestImportFindingActionSerializer( + many=True, read_only=True, + ) + + class Meta: + model = Test_Import + fields = "__all__" diff --git a/dojo/test/api/urls.py b/dojo/test/api/urls.py new file mode 100644 index 00000000000..b98f633d3bd --- /dev/null +++ b/dojo/test/api/urls.py @@ -0,0 +1,8 @@ +from dojo.test.api.views import TestImportViewSet, TestsViewSet, TestTypesViewSet + + +def add_test_urls(router): + router.register("tests", TestsViewSet, basename="test") + router.register("test_types", TestTypesViewSet, basename="test_type") + router.register("test_imports", TestImportViewSet, basename="test_imports") + return router diff --git a/dojo/test/api/views.py b/dojo/test/api/views.py new file mode 100644 index 00000000000..7c2066374c4 --- /dev/null +++ b/dojo/test/api/views.py @@ -0,0 +1,325 @@ +from django.urls import reverse +from django_filters.rest_framework import DjangoFilterBackend +from drf_spectacular.utils import extend_schema +from rest_framework import mixins, status, viewsets +from rest_framework.decorators import action +from rest_framework.parsers import MultiPartParser +from rest_framework.permissions import DjangoModelPermissions, IsAuthenticated +from rest_framework.response import Response + +from dojo.api_v2 import serializers as api_v2_serializers +from dojo.api_v2.views import PrefetchDojoModelViewSet, report_generate +from dojo.authorization import api_permissions as permissions +from dojo.models import ( + FileUpload, + NoteHistory, + Notes, + Test, + Test_Import, + Test_Type, +) +from dojo.risk_acceptance import api as ra_api +from dojo.test.api.filters import ApiTestFilter, TestImportAPIFilter +from dojo.test.api.serializer import ( + TestCreateSerializer, + TestImportSerializer, + TestSerializer, + TestToFilesSerializer, + TestToNotesSerializer, + TestTypeCreateSerializer, + TestTypeSerializer, +) +from dojo.test.queries import get_authorized_test_imports, get_authorized_tests +from dojo.utils import ( + async_delete, + generate_file_response, + get_setting, + process_tag_notifications, +) + + +# Authorization: object-based +# @extend_schema_view(**schema_with_prefetch()) +# Nested models with prefetch make the response schema too long for Swagger UI +class TestsViewSet( + PrefetchDojoModelViewSet, + ra_api.AcceptedRisksMixin, +): + serializer_class = TestSerializer + queryset = Test.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_class = ApiTestFilter + permission_classes = (IsAuthenticated, permissions.UserHasTestPermission) + + @property + def risk_application_model_class(self): + return Test + + def get_queryset(self): + return ( + get_authorized_tests("view") + .prefetch_related("notes", "files") + .distinct() + ) + + def destroy(self, request, *args, **kwargs): + instance = self.get_object() + if get_setting("ASYNC_OBJECT_DELETE"): + async_del = async_delete() + async_del.delete(instance) + else: + instance.delete() + return Response(status=status.HTTP_204_NO_CONTENT) + + def get_serializer_class(self): + if self.request and self.request.method == "POST": + if self.action == "accept_risks": + return ra_api.AcceptedRiskSerializer + return TestCreateSerializer + return TestSerializer + + @extend_schema( + request=api_v2_serializers.ReportGenerateOptionSerializer, + responses={status.HTTP_200_OK: api_v2_serializers.ReportGenerateSerializer}, + ) + @action( + detail=True, methods=["post"], + # IsAuthenticated only: report generation requires View permission, + # enforced by the permission-filtered get_queryset(). The viewset's + # permission_classes would check Edit (POST), which is too restrictive. + permission_classes=[IsAuthenticated], + ) + def generate_report(self, request, pk=None): + test = self.get_object() + + options = {} + # prepare post data + report_options = api_v2_serializers.ReportGenerateOptionSerializer( + data=request.data, + ) + if report_options.is_valid(): + options["include_finding_notes"] = report_options.validated_data[ + "include_finding_notes" + ] + options["include_finding_images"] = report_options.validated_data[ + "include_finding_images" + ] + options[ + "include_executive_summary" + ] = report_options.validated_data["include_executive_summary"] + options[ + "include_table_of_contents" + ] = report_options.validated_data["include_table_of_contents"] + else: + return Response( + report_options.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + data = report_generate(request, test, options) + report = api_v2_serializers.ReportGenerateSerializer(data) + return Response(report.data) + + @extend_schema( + methods=["GET"], + responses={status.HTTP_200_OK: TestToNotesSerializer}, + ) + @extend_schema( + methods=["POST"], + request=api_v2_serializers.AddNewNoteOptionSerializer, + responses={status.HTTP_201_CREATED: api_v2_serializers.NoteSerializer}, + ) + @action(detail=True, methods=["get", "post"], permission_classes=(IsAuthenticated, permissions.UserHasTestNotePermission)) + def notes(self, request, pk=None): + test = self.get_object() + if request.method == "POST": + new_note = api_v2_serializers.AddNewNoteOptionSerializer( + data=request.data, + ) + if new_note.is_valid(): + entry = new_note.validated_data["entry"] + private = new_note.validated_data.get("private", False) + note_type = new_note.validated_data.get("note_type", None) + else: + return Response( + new_note.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + notes = test.notes.filter(note_type=note_type).first() + if notes and note_type and note_type.is_single: + return Response("Only one instance of this note_type allowed on a test.", status=status.HTTP_400_BAD_REQUEST) + + author = request.user + note = Notes( + entry=entry, + author=author, + private=private, + note_type=note_type, + ) + note.save() + # Add an entry to the note history + history = NoteHistory.objects.create(data=note.entry, time=note.date, current_editor=note.author) + note.history.add(history) + # Now add the note to the object + test.notes.add(note) + # Determine if we need to send any notifications for user mentioned + process_tag_notifications( + request=request, + note=note, + parent_url=request.build_absolute_uri( + reverse("view_test", args=(test.id,)), + ), + parent_title=f"Test: {test.title}", + ) + + serialized_note = api_v2_serializers.NoteSerializer( + {"author": author, "entry": entry, "private": private}, + ) + return Response( + serialized_note.data, status=status.HTTP_201_CREATED, + ) + notes = test.notes.all() + + serialized_notes = TestToNotesSerializer( + {"test_id": test, "notes": notes}, + ) + return Response(serialized_notes.data, status=status.HTTP_200_OK) + + @extend_schema( + methods=["GET"], + responses={status.HTTP_200_OK: TestToFilesSerializer}, + ) + @extend_schema( + methods=["POST"], + request=api_v2_serializers.AddNewFileOptionSerializer, + responses={status.HTTP_201_CREATED: api_v2_serializers.FileSerializer}, + ) + @action( + detail=True, methods=["get", "post"], parser_classes=(MultiPartParser,), permission_classes=(IsAuthenticated, permissions.UserHasTestRelatedObjectPermission), + ) + def files(self, request, pk=None): + test = self.get_object() + if request.method == "POST": + new_file = api_v2_serializers.FileSerializer(data=request.data) + if new_file.is_valid(): + title = new_file.validated_data["title"] + file = new_file.validated_data["file"] + else: + return Response( + new_file.errors, status=status.HTTP_400_BAD_REQUEST, + ) + + file = FileUpload(title=title, file=file) + file.save() + test.files.add(file) + + serialized_file = api_v2_serializers.FileSerializer(file) + return Response( + serialized_file.data, status=status.HTTP_201_CREATED, + ) + + files = test.files.all() + serialized_files = TestToFilesSerializer( + {"test_id": test, "files": files}, + ) + return Response(serialized_files.data, status=status.HTTP_200_OK) + + @extend_schema( + methods=["GET"], + responses={ + status.HTTP_200_OK: api_v2_serializers.RawFileSerializer, + }, + ) + @action( + detail=True, + methods=["get"], + url_path=r"files/download/(?P\d+)", + permission_classes=(IsAuthenticated, permissions.UserHasTestRelatedObjectPermission), + ) + def download_file(self, request, file_id, pk=None): + test = self.get_object() + # Get the file object + file_object_qs = test.files.filter(id=file_id) + file_object = ( + file_object_qs.first() if len(file_object_qs) > 0 else None + ) + if file_object is None: + return Response( + {"error": "File ID not associated with Test"}, + status=status.HTTP_404_NOT_FOUND, + ) + # send file + return generate_file_response(file_object) + + +# Authorization: authenticated, configuration +class TestTypesViewSet( + mixins.UpdateModelMixin, + mixins.CreateModelMixin, + viewsets.ReadOnlyModelViewSet, +): + serializer_class = TestTypeSerializer + queryset = Test_Type.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = [ + "name", + ] + permission_classes = (IsAuthenticated, DjangoModelPermissions) + + def get_queryset(self): + return Test_Type.objects.all().order_by("id") + + def get_serializer_class(self): + if self.action == "create": + return TestTypeCreateSerializer + return TestTypeSerializer + + +# @extend_schema_view(**schema_with_prefetch()) +# Nested models with prefetch make the response schema too long for Swagger UI +class TestImportViewSet( + PrefetchDojoModelViewSet, +): + serializer_class = TestImportSerializer + queryset = Test_Import.objects.none() + filter_backends = (DjangoFilterBackend,) + + filterset_class = TestImportAPIFilter + + permission_classes = ( + IsAuthenticated, + permissions.UserHasTestImportPermission, + ) + + def get_queryset(self): + return get_authorized_test_imports( + "view", + ).prefetch_related( + "test_import_finding_action_set", + "findings_affected", + "findings_affected__endpoints", + "findings_affected__status_finding", + "findings_affected__finding_meta", + "findings_affected__jira_issue", + "findings_affected__burprawrequestresponse_set", + "findings_affected__jira_issue", + "findings_affected__jira_issue", + "findings_affected__jira_issue", + "findings_affected__reviewers", + "findings_affected__notes", + "findings_affected__notes__author", + "findings_affected__notes__history", + "findings_affected__files", + "findings_affected__found_by", + "findings_affected__tags", + "findings_affected__risk_acceptance_set", + "test", + "test__tags", + "test__notes", + "test__notes__author", + "test__files", + "test__test_type", + "test__engagement", + "test__environment", + "test__engagement__product", + "test__engagement__product__prod_type", + ) diff --git a/dojo/test/models.py b/dojo/test/models.py new file mode 100644 index 00000000000..31cefaf52ac --- /dev/null +++ b/dojo/test/models.py @@ -0,0 +1,287 @@ +import logging +from contextlib import suppress + +from django.conf import settings +from django.db import models +from django.db.models import Count, Q +from django.urls import reverse +from django.utils.translation import gettext as _ +from django_extensions.db.models import TimeStampedModel +from tagulous.models import TagField + +logger = logging.getLogger(__name__) +deduplicationLogger = logging.getLogger("dojo.specific-loggers.deduplication") + +IMPORT_CREATED_FINDING = "N" +IMPORT_CLOSED_FINDING = "C" +IMPORT_REACTIVATED_FINDING = "R" +IMPORT_UNTOUCHED_FINDING = "U" + +IMPORT_ACTIONS = [ + (IMPORT_CREATED_FINDING, "created"), + (IMPORT_CLOSED_FINDING, "closed"), + (IMPORT_REACTIVATED_FINDING, "reactivated"), + (IMPORT_UNTOUCHED_FINDING, "untouched"), +] + + +class Test_Type(models.Model): + name = models.CharField(max_length=200, unique=True) + static_tool = models.BooleanField(default=False) + dynamic_tool = models.BooleanField(default=False) + active = models.BooleanField(default=True) + dynamically_generated = models.BooleanField( + default=False, + help_text=_("Set to True for test types that are created at import time")) + + class Meta: + ordering = ("name",) + + def __str__(self): + return self.name + + def get_breadcrumbs(self): + return [{"title": str(self), + "url": None}] + + +class Test(models.Model): + engagement = models.ForeignKey("dojo.Engagement", editable=False, on_delete=models.CASCADE) + lead = models.ForeignKey("dojo.Dojo_User", editable=True, null=True, blank=True, on_delete=models.RESTRICT) + test_type = models.ForeignKey("dojo.Test_Type", on_delete=models.CASCADE) + scan_type = models.TextField(null=True) + title = models.CharField(max_length=255, null=True, blank=True) + description = models.TextField(null=True, blank=True) + target_start = models.DateTimeField() + target_end = models.DateTimeField() + percent_complete = models.IntegerField(null=True, blank=True, + editable=True) + notes = models.ManyToManyField("dojo.Notes", blank=True, + editable=False) + files = models.ManyToManyField("dojo.FileUpload", blank=True, editable=False) + environment = models.ForeignKey("dojo.Development_Environment", null=True, + blank=False, on_delete=models.RESTRICT) + + updated = models.DateTimeField(auto_now=True, null=True) + created = models.DateTimeField(auto_now_add=True, null=True) + + tags = TagField(blank=True, force_lowercase=True, help_text=_("Add tags that help describe this test. Choose from the list or add new tags. Press Enter key to add.")) + inherited_tags = TagField(blank=True, force_lowercase=True, help_text=_("Internal use tags sepcifically for maintaining parity with product. This field will be present as a subset in the tags field")) + + version = models.CharField(max_length=100, null=True, blank=True) + + build_id = models.CharField(editable=True, max_length=150, + null=True, blank=True, help_text=_("Build ID that was tested, a reimport may update this field."), verbose_name=_("Build ID")) + commit_hash = models.CharField(editable=True, max_length=150, + null=True, blank=True, help_text=_("Commit hash tested, a reimport may update this field."), verbose_name=_("Commit Hash")) + branch_tag = models.CharField(editable=True, max_length=150, + null=True, blank=True, help_text=_("Tag or branch that was tested, a reimport may update this field."), verbose_name=_("Branch/Tag")) + api_scan_configuration = models.ForeignKey("dojo.Product_API_Scan_Configuration", null=True, editable=True, blank=True, on_delete=models.CASCADE, verbose_name=_("API Scan Configuration")) + + class Meta: + indexes = [ + models.Index(fields=["engagement", "test_type"]), + ] + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.unsaved_metadata: list = [] + + def __str__(self): + if self.title: + return f"{self.title} ({self.test_type})" + return str(self.test_type) + + def get_absolute_url(self): + return reverse("view_test", args=[str(self.id)]) + + def test_type_name(self) -> str: + return self.test_type.name + + def get_breadcrumbs(self): + bc = self.engagement.get_breadcrumbs() + bc += [{"title": str(self), + "url": reverse("view_test", args=(self.id,))}] + return bc + + def copy(self, engagement=None): + from dojo.models import Finding, copy_model_util # noqa: PLC0415 -- lazy import, avoids circular dependency # isort: skip + copy = copy_model_util(self) + # Save the necessary ManyToMany relationships + old_notes = list(self.notes.all()) + old_files = list(self.files.all()) + old_tags = list(self.tags.all()) + old_findings = list(Finding.objects.filter(test=self)) + if engagement: + copy.engagement = engagement + # Save the object before setting any ManyToMany relationships + copy.save() + # Copy the notes + for notes in old_notes: + copy.notes.add(notes.copy()) + # Copy the files + for files in old_files: + copy.files.add(files.copy()) + # Copy the Findings + for finding in old_findings: + finding.copy(test=copy) + # Assign any tags + copy.tags.set(old_tags) + + return copy + + # only used by bulk risk acceptance api + @property + def unaccepted_open_findings(self): + from dojo.models import Finding # noqa: PLC0415 -- lazy import, avoids circular dependency + from dojo.utils import get_system_setting # noqa: PLC0415 circular import + findings = Finding.objects.filter(risk_accepted=False, active=True, duplicate=False, test=self) + if get_system_setting("enforce_verified_status", True) or get_system_setting("enforce_verified_status_metrics", True): + findings = findings.filter(verified=True) + + return findings + + def accept_risks(self, accepted_risks): + self.engagement.risk_acceptance.add(*accepted_risks) + + @property + def deduplication_algorithm(self): + deduplicationAlgorithm = settings.DEDUPE_ALGO_LEGACY + + if hasattr(settings, "DEDUPLICATION_ALGORITHM_PER_PARSER"): + if (self.test_type.name in settings.DEDUPLICATION_ALGORITHM_PER_PARSER): + deduplicationLogger.debug(f"using DEDUPLICATION_ALGORITHM_PER_PARSER for test_type.name: {self.test_type.name}") + deduplicationAlgorithm = settings.DEDUPLICATION_ALGORITHM_PER_PARSER[self.test_type.name] + elif (self.scan_type in settings.DEDUPLICATION_ALGORITHM_PER_PARSER): + deduplicationLogger.debug(f"using DEDUPLICATION_ALGORITHM_PER_PARSER for scan_type: {self.scan_type}") + deduplicationAlgorithm = settings.DEDUPLICATION_ALGORITHM_PER_PARSER[self.scan_type] + else: + deduplicationLogger.debug("Section DEDUPLICATION_ALGORITHM_PER_PARSER not found in settings.dist.py") + + deduplicationLogger.debug(f"DEDUPLICATION_ALGORITHM_PER_PARSER is: {deduplicationAlgorithm}") + return deduplicationAlgorithm + + @property + def hash_code_fields(self): + """Retrieve OS HASH_CODE_FIELDS_PER_SCANNER settings. Be aware when calling this to make sure Pro doesn't use these OS seetings""" + hashCodeFields = None + + if hasattr(settings, "HASHCODE_FIELDS_PER_SCANNER"): + if (self.test_type.name in settings.HASHCODE_FIELDS_PER_SCANNER): + deduplicationLogger.debug(f"using HASHCODE_FIELDS_PER_SCANNER for test_type.name: {self.test_type.name}") + hashCodeFields = settings.HASHCODE_FIELDS_PER_SCANNER[self.test_type.name] + elif (self.scan_type in settings.HASHCODE_FIELDS_PER_SCANNER): + deduplicationLogger.debug(f"using HASHCODE_FIELDS_PER_SCANNER for scan_type: {self.scan_type}") + hashCodeFields = settings.HASHCODE_FIELDS_PER_SCANNER[self.scan_type] + else: + deduplicationLogger.warning(f"test_type name {self.test_type.name} and scan_type {self.scan_type} not found in HASHCODE_FIELDS_PER_SCANNER") + else: + deduplicationLogger.debug("Section HASHCODE_FIELDS_PER_SCANNER not found in settings.dist.py") + + hash_code_fields_always = getattr(settings, "HASH_CODE_FIELDS_ALWAYS", []) + deduplicationLogger.debug(f"HASHCODE_FIELDS_PER_SCANNER is: {hashCodeFields} + HASH_CODE_FIELDS_ALWAYS: {hash_code_fields_always}") + + return hashCodeFields + + @property + def hash_code_allows_null_cwe(self): + hashCodeAllowsNullCwe = True + + if hasattr(settings, "HASHCODE_ALLOWS_NULL_CWE"): + if (self.test_type.name in settings.HASHCODE_ALLOWS_NULL_CWE): + deduplicationLogger.debug(f"using HASHCODE_ALLOWS_NULL_CWE for test_type.name: {self.test_type.name}") + hashCodeAllowsNullCwe = settings.HASHCODE_ALLOWS_NULL_CWE[self.test_type.name] + elif (self.scan_type in settings.HASHCODE_ALLOWS_NULL_CWE): + deduplicationLogger.debug(f"using HASHCODE_ALLOWS_NULL_CWE for scan_type: {self.scan_type}") + hashCodeAllowsNullCwe = settings.HASHCODE_ALLOWS_NULL_CWE[self.scan_type] + else: + deduplicationLogger.debug("Section HASHCODE_ALLOWS_NULL_CWE not found in settings.dist.py") + + deduplicationLogger.debug(f"HASHCODE_ALLOWS_NULL_CWE is: {hashCodeAllowsNullCwe}") + return hashCodeAllowsNullCwe + + def delete(self, *args, product_grading_option=True, **kwargs): + logger.debug("%d test delete", self.id) + super().delete(*args, **kwargs) + if product_grading_option: + from dojo.models import Engagement, Product # noqa: PLC0415 -- lazy import, avoids circular dependency + with suppress(Test.DoesNotExist, Engagement.DoesNotExist, Product.DoesNotExist): + # Suppressing a potential issue created from async delete removing + # related objects in a separate task + from dojo.utils import perform_product_grading # noqa: PLC0415 circular import + perform_product_grading(self.engagement.product) + + @property + def statistics(self): + """Queries the database, no prefetching, so could be slow for lists of model instances""" + from dojo.models import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + Finding, + _get_annotations_for_statistics, + _get_statistics_for_queryset, + ) + return _get_statistics_for_queryset(Finding.objects.filter(test=self), _get_annotations_for_statistics) + + +class Test_Import(TimeStampedModel): + + IMPORT_TYPE = "import" + REIMPORT_TYPE = "reimport" + + test = models.ForeignKey("dojo.Test", editable=False, null=False, blank=False, on_delete=models.CASCADE) + findings_affected = models.ManyToManyField("dojo.Finding", through="dojo.Test_Import_Finding_Action") + import_settings = models.JSONField(null=True) + type = models.CharField(max_length=64, null=False, blank=False, default="unknown") + + version = models.CharField(max_length=100, null=True, blank=True) + build_id = models.CharField(editable=True, max_length=150, + null=True, blank=True, help_text=_("Build ID that was tested, a reimport may update this field."), verbose_name=_("Build ID")) + commit_hash = models.CharField(editable=True, max_length=150, + null=True, blank=True, help_text=_("Commit hash tested, a reimport may update this field."), verbose_name=_("Commit Hash")) + branch_tag = models.CharField(editable=True, max_length=150, + null=True, blank=True, help_text=_("Tag or branch that was tested, a reimport may update this field."), verbose_name=_("Branch/Tag")) + + def get_queryset(self): + logger.debug("prefetch test_import counts") + super_query = super().get_queryset() + super_query = super_query.annotate(created_findings_count=Count("findings", filter=Q(test_import_finding_action__action=IMPORT_CREATED_FINDING))) + super_query = super_query.annotate(closed_findings_count=Count("findings", filter=Q(test_import_finding_action__action=IMPORT_CLOSED_FINDING))) + super_query = super_query.annotate(reactivated_findings_count=Count("findings", filter=Q(test_import_finding_action__action=IMPORT_REACTIVATED_FINDING))) + return super_query.annotate(untouched_findings_count=Count("findings", filter=Q(test_import_finding_action__action=IMPORT_UNTOUCHED_FINDING))) + + class Meta: + ordering = ("-id",) + indexes = [ + models.Index(fields=["created", "test", "type"]), + ] + + def __str__(self): + return self.created.strftime("%Y-%m-%d %H:%M:%S") + + @property + def statistics(self): + """Queries the database, no prefetching, so could be slow for lists of model instances""" + from dojo.models import ( # noqa: PLC0415 -- lazy import, avoids circular dependency + Finding, + _get_annotations_for_statistics, + _get_statistics_for_queryset, + ) + stats = {} + for action in IMPORT_ACTIONS: + stats[action[1].lower()] = _get_statistics_for_queryset(Finding.objects.filter(test_import_finding_action__test_import=self, test_import_finding_action__action=action[0]), _get_annotations_for_statistics) + return stats + + +class Test_Import_Finding_Action(TimeStampedModel): + test_import = models.ForeignKey("dojo.Test_Import", editable=False, null=False, blank=False, on_delete=models.CASCADE) + finding = models.ForeignKey("dojo.Finding", editable=False, null=False, blank=False, on_delete=models.CASCADE) + action = models.CharField(max_length=100, null=True, blank=True, choices=IMPORT_ACTIONS) + + class Meta: + indexes = [ + models.Index(fields=["finding", "action", "test_import"]), + ] + unique_together = (("test_import", "finding")) + ordering = ("test_import", "action", "finding") + + def __str__(self): + return f"{self.finding.id}: {self.action}" diff --git a/dojo/test/services.py b/dojo/test/services.py new file mode 100644 index 00000000000..fbd5dbf3b59 --- /dev/null +++ b/dojo/test/services.py @@ -0,0 +1,33 @@ +# # tests +import logging + +from django.urls import reverse +from django.utils.translation import gettext as _ + +from dojo.celery_dispatch import dojo_dispatch_task +from dojo.notifications.helper import create_notification +from dojo.utils import calculate_grade + +logger = logging.getLogger(__name__) + + +def copy_test(test, engagement, user): + """ + Copy a test (and its findings) into the given engagement, recalculate the product + grade, and notify. Returns the new test. + + HTTP-free so both the UI view and (eventually) the API can call it. + """ + product = test.engagement.product + test_copy = test.copy(engagement=engagement) + dojo_dispatch_task(calculate_grade, product.id) + create_notification( + event="test_copied", + title=_("Copying of %s") % test.title, + description=f'The test "{test.title}" was copied by {user} to {engagement.name}', + product=product, + url=reverse("view_test", args=(test_copy.id,)), + recipients=[test.engagement.lead], + icon="exclamation-triangle", + ) + return test_copy diff --git a/dojo/test/ui/__init__.py b/dojo/test/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/test/ui/filters.py b/dojo/test/ui/filters.py new file mode 100644 index 00000000000..60bc7960d54 --- /dev/null +++ b/dojo/test/ui/filters.py @@ -0,0 +1,64 @@ +import logging + +from django_filters import BooleanFilter, CharFilter, MultipleChoiceFilter, OrderingFilter + +from dojo.filters import DojoFilter +from dojo.models import IMPORT_ACTIONS, Test_Import, Test_Import_Finding_Action, Test_Type + +logger = logging.getLogger(__name__) + + +class TestImportFilter(DojoFilter): + version = CharFilter(field_name="version", lookup_expr="icontains") + version_exact = CharFilter(field_name="version", lookup_expr="iexact", label="Version Exact") + branch_tag = CharFilter(lookup_expr="icontains", label="Branch/Tag") + build_id = CharFilter(lookup_expr="icontains", label="Build ID") + commit_hash = CharFilter(lookup_expr="icontains", label="Commit hash") + + findings_affected = BooleanFilter(field_name="findings_affected", lookup_expr="isnull", exclude=True, label="Findings affected") + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("date", "date"), + ("version", "version"), + ("branch_tag", "branch_tag"), + ("build_id", "build_id"), + ("commit_hash", "commit_hash"), + + ), + ) + + class Meta: + model = Test_Import + fields = [] + + +class TestImportFindingActionFilter(DojoFilter): + action = MultipleChoiceFilter(choices=IMPORT_ACTIONS) + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("action", "action"), + ), + ) + + class Meta: + model = Test_Import_Finding_Action + fields = [] + + +class TestTypeFilter(DojoFilter): + name = CharFilter(lookup_expr="icontains") + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("name", "name"), + ), + ) + + class Meta: + model = Test_Type + exclude = [] + include = ("name",) diff --git a/dojo/test/ui/forms.py b/dojo/test/ui/forms.py new file mode 100644 index 00000000000..6114817ef00 --- /dev/null +++ b/dojo/test/ui/forms.py @@ -0,0 +1,86 @@ +import logging + +from django import forms + +from dojo.models import Development_Environment, Engagement, Product_API_Scan_Configuration, Test, Test_Type +from dojo.user.queries import get_authorized_users, get_authorized_users_for_product_and_product_type +from dojo.utils import get_product +from dojo.validators import tag_validator + +logger = logging.getLogger(__name__) + + +class TestForm(forms.ModelForm): + title = forms.CharField(max_length=255, required=False) + description = forms.CharField(widget=forms.Textarea(attrs={"rows": "3"}), required=False) + test_type = forms.ModelChoiceField(queryset=Test_Type.objects.all().order_by("name")) + environment = forms.ModelChoiceField( + queryset=Development_Environment.objects.all().order_by("name")) + target_start = forms.DateTimeField(widget=forms.TextInput( + attrs={"class": "datepicker", "autocomplete": "off"})) + target_end = forms.DateTimeField(widget=forms.TextInput( + attrs={"class": "datepicker", "autocomplete": "off"})) + lead = forms.ModelChoiceField( + queryset=None, + required=False, label="Testing Lead") + + def __init__(self, *args, **kwargs): + obj = None + + if "engagement" in kwargs: + obj = kwargs.pop("engagement") + + if "instance" in kwargs: + obj = kwargs.get("instance") + + super().__init__(*args, **kwargs) + + if obj: + product = get_product(obj) + self.fields["lead"].queryset = get_authorized_users_for_product_and_product_type(None, product, "view").filter(is_active=True) + self.fields["api_scan_configuration"].queryset = Product_API_Scan_Configuration.objects.filter(product=product) + else: + self.fields["lead"].queryset = get_authorized_users("view").filter(is_active=True) + + def is_valid(self): + valid = super().is_valid() + + # we're done now if not valid + if not valid: + return valid + if self.cleaned_data["target_start"] > self.cleaned_data["target_end"]: + self.add_error("target_start", "Your target start date exceeds your target end date") + self.add_error("target_end", "Your target start date exceeds your target end date") + return False + return True + + class Meta: + model = Test + fields = ["title", "test_type", "target_start", "target_end", "description", + "environment", "percent_complete", "tags", "lead", "version", "branch_tag", "build_id", "commit_hash", + "api_scan_configuration"] + + def clean_tags(self): + tag_validator(self.cleaned_data.get("tags")) + return self.cleaned_data.get("tags") + + +class DeleteTestForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Test + fields = ["id"] + + +class CopyTestForm(forms.Form): + engagement = forms.ModelChoiceField( + required=True, + queryset=Engagement.objects.none(), + error_messages={"required": "*"}) + + def __init__(self, *args, **kwargs): + authorized_lists = kwargs.pop("engagements", None) + super().__init__(*args, **kwargs) + self.fields["engagement"].queryset = authorized_lists diff --git a/dojo/test/urls.py b/dojo/test/ui/urls.py similarity index 97% rename from dojo/test/urls.py rename to dojo/test/ui/urls.py index 335cf260b86..403068023cd 100644 --- a/dojo/test/urls.py +++ b/dojo/test/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from dojo.test import views +from dojo.test.ui import views urlpatterns = [ # tests diff --git a/dojo/test/ui/views.py b/dojo/test/ui/views.py new file mode 100644 index 00000000000..b89c53cd4e3 --- /dev/null +++ b/dojo/test/ui/views.py @@ -0,0 +1,1120 @@ +# # tests +import base64 +import logging +import operator +import time +from datetime import datetime, timedelta +from functools import reduce + +import pghistory +from django.conf import settings +from django.contrib import messages +from django.contrib.admin.utils import NestedObjects +from django.core.exceptions import ValidationError +from django.db import DEFAULT_DB_ALIAS +from django.db.models import Count, Q +from django.http import HttpRequest, HttpResponse, HttpResponseRedirect +from django.shortcuts import get_object_or_404, render +from django.urls import Resolver404, reverse +from django.utils import timezone +from django.utils.translation import gettext as _ +from django.views import View +from django.views.decorators.cache import cache_page +from django.views.decorators.vary import vary_on_cookie + +import dojo.finding.helper as finding_helper +from dojo.authorization.authorization import user_has_permission_or_403 +from dojo.engagement.queries import get_authorized_engagements +from dojo.finding.queries import prefetch_for_findings +from dojo.finding.ui.filters import FindingFilter, FindingFilterWithoutObjectLookups, TemplateFindingFilter +from dojo.finding.ui.views import find_available_notetypes +from dojo.forms import ( + AddFindingForm, + FindingBulkUpdateForm, + JIRAFindingForm, + JIRAImportScanForm, + NoteForm, + ReImportScanForm, + TestForm, + TypedNoteForm, +) +from dojo.importers.base_importer import BaseImporter +from dojo.importers.default_reimporter import DefaultReImporter +from dojo.jira import services as jira_services +from dojo.location.models import Location +from dojo.models import ( + BurpRawRequestResponse, + Endpoint, + Finding, + Finding_Group, + Finding_Template, + Note_Type, + Product_API_Scan_Configuration, + Test, + Test_Import, +) +from dojo.notifications.helper import create_notification +from dojo.product_announcements import ( + ErrorPageProductAnnouncement, + LargeScanSizeProductAnnouncement, + ScanTypeProductAnnouncement, +) +from dojo.test.queries import get_authorized_tests +from dojo.test.services import copy_test as copy_test_service +from dojo.test.ui.filters import TestImportFilter +from dojo.test.ui.forms import CopyTestForm, DeleteTestForm +from dojo.tools.factory import get_choices_sorted, get_scan_types_sorted +from dojo.user.queries import get_authorized_users +from dojo.utils import ( + Product_Tab, + add_breadcrumb, + add_error_message_to_response, + add_field_errors_to_response, + add_success_message_to_response, + async_delete, + get_cal_event, + get_page_items, + get_page_items_and_count, + get_setting, + get_system_setting, + get_words_for_field, + process_tag_notifications, + redirect_to_return_url_or_else, +) + +logger = logging.getLogger(__name__) +parse_logger = logging.getLogger("dojo") +deduplicationLogger = logging.getLogger("dojo.specific-loggers.deduplication") + + +class ViewTest(View): + def get_test(self, test_id: int): + test_prefetched = get_authorized_tests("view") + test_prefetched = test_prefetched.annotate(total_reimport_count=Count("test_import__id", distinct=True)) + return get_object_or_404(test_prefetched, pk=test_id) + + def get_test_import_data(self, request: HttpRequest, test: Test): + test_imports = Test_Import.objects.filter(test=test) + test_import_filter = TestImportFilter(request.GET, test_imports) + + paged_test_imports = get_page_items_and_count(request, test_import_filter.qs, 5, prefix="test_imports") + paged_test_imports.object_list = paged_test_imports.object_list.prefetch_related("test_import_finding_action_set") + + return { + "paged_test_imports": paged_test_imports, + "test_import_filter": test_import_filter, + } + + def get_findings(self, request: HttpRequest, test: Test): + findings = Finding.objects.filter(test=test).order_by("numerical_severity") + filter_string_matching = get_system_setting("filter_string_matching", False) + finding_filter_class = FindingFilterWithoutObjectLookups if filter_string_matching else FindingFilter + findings = finding_filter_class(request.GET, pid=test.engagement.product.id, eid=test.engagement.id, tid=test.id, queryset=findings) + paged_findings = get_page_items_and_count(request, prefetch_for_findings(findings.qs), 25, prefix="findings") + fix_available_count = findings.qs.filter(fix_available=True).count() + + return { + "findings": paged_findings, + "filtered": findings, + "fix_available_count": fix_available_count, + } + + def get_note_form(self, request: HttpRequest): + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = {} + + return NoteForm(*args, **kwargs) + + def get_typed_note_form(self, request: HttpRequest, context: dict): + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = { + "available_note_types": context.get("available_note_types"), + } + + return TypedNoteForm(*args, **kwargs) + + def get_form(self, request: HttpRequest, context: dict): + return ( + self.get_typed_note_form(request, context) + if context.get("note_type_activation") + else self.get_note_form(request) + ) + + def get_initial_context(self, request: HttpRequest, test: Test): + # Set up the product tab + product_tab = Product_Tab(test.engagement.product, title=_("Test"), tab="engagements") + product_tab.setEngagement(test.engagement) + # Set up the notes and associated info to generate the form with + notes = test.notes.all() + note_type_activation = Note_Type.objects.filter(is_active=True).count() + available_note_types = None + if note_type_activation: + available_note_types = find_available_notetypes(notes) + # Set the current context + context = { + "test": test, + "prod": test.engagement.product, + "product_tab": product_tab, + "title_words": get_words_for_field(Finding, "title"), + "component_words": get_words_for_field(Finding, "component_name"), + "notes": notes, + "note_type_activation": note_type_activation, + "available_note_types": available_note_types, + "files": test.files.all(), + "person": request.user.username, + "request": request, + "show_re_upload": any(test.test_type.name in code for code in get_choices_sorted()), + "jira_project": jira_services.get_project(test), + "bulk_edit_form": FindingBulkUpdateForm(request.GET), + "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), + "finding_groups": test.finding_group_set.all().prefetch_related("findings", "jira_issue", "creator", "findings__vulnerability_id_set"), + "finding_group_by_options": Finding_Group.GROUP_BY_OPTIONS, + } + # Set the form using the context, and then update the context + form = self.get_form(request, context) + context["form"] = form + # Add some of the related objects + context |= self.get_findings(request, test) + context |= self.get_test_import_data(request, test) + + return context + + def process_form(self, request: HttpRequest, test: Test, context: dict): + if context["form"].is_valid(): + # Save the note + new_note = context["form"].save(commit=False) + new_note.author = request.user + new_note.date = timezone.now() + new_note.save() + test.notes.add(new_note) + # Make a notification for this actions + url = request.build_absolute_uri(reverse("view_test", args=(test.id,))) + title = f"Test: {test.test_type.name} on {test.engagement.product.name}" + process_tag_notifications(request, new_note, url, title) + messages.add_message( + request, + messages.SUCCESS, + _("Note added successfully."), + extra_tags="alert-success") + + return request, True + return request, False + + def get_template(self): + return "dojo/view_test.html" + + def get(self, request: HttpRequest, test_id: int): + # Get the initial objects + test = self.get_test(test_id) + # Make sure the user is authorized + user_has_permission_or_403(request.user, test, "view") + # Quick perms check to determine if the user has access to add a note to the test + user_has_permission_or_403(request.user, test, "add") + # Set up the initial context + context = self.get_initial_context(request, test) + # Render the form + return render(request, self.get_template(), context) + + def post(self, request: HttpRequest, test_id: int): + # Get the initial objects + test = self.get_test(test_id) + # Make sure the user is authorized + user_has_permission_or_403(request.user, test, "view") + # Quick perms check to determine if the user has access to add a note to the test + user_has_permission_or_403(request.user, test, "add") + # Set up the initial context + context = self.get_initial_context(request, test) + # Determine the validity of the form + request, success = self.process_form(request, test, context) + # Handle the case of a successful form + if success: + return redirect_to_return_url_or_else(request, reverse("view_test", args=(test_id,))) + # Render the form + return render(request, self.get_template(), context) + +# def prefetch_for_test_imports(test_imports): +# prefetched_test_imports = test_imports +# if isinstance(test_imports, QuerySet): # old code can arrive here with prods being a list because the query was already executed +# #could we make this dynamic, i.e for action_type in IMPORT_ACTIONS: prefetch +# prefetched_test_imports = prefetched_test_imports.annotate(created_findings_count=Count('findings', filter=Q(test_import_finding_action__action=IMPORT_CREATED_FINDING))) +# prefetched_test_imports = prefetched_test_imports.annotate(closed_findings_count=Count('findings', filter=Q(test_import_finding_action__action=IMPORT_CLOSED_FINDING))) +# prefetched_test_imports = prefetched_test_imports.annotate(reactivated_findings_count=Count('findings', filter=Q(test_import_finding_action__action=IMPORT_REACTIVATED_FINDING))) +# prefetched_test_imports = prefetched_test_imports.annotate(updated_findings_count=Count('findings', filter=Q(test_import_finding_action__action=IMPORT_UNTOUCHED_FINDING))) + +# return prefetch_for_test_imports + + +def edit_test(request, tid): + test = get_object_or_404(Test, pk=tid) + form = TestForm(instance=test) + if request.method == "POST": + form = TestForm(request.POST, instance=test) + if form.is_valid(): + form.save() + messages.add_message(request, + messages.SUCCESS, + _("Test saved."), + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_engagement", args=(test.engagement.id,))) + + form.initial["target_start"] = test.target_start.date() + form.initial["target_end"] = test.target_end.date() + form.initial["description"] = test.description + + product_tab = Product_Tab(test.engagement.product, title=_("Edit Test"), tab="engagements") + product_tab.setEngagement(test.engagement) + return render(request, "dojo/edit_test.html", + {"test": test, + "product_tab": product_tab, + "form": form, + }) + + +def delete_test(request, tid): + test = get_object_or_404(Test, pk=tid) + eng = test.engagement + form = DeleteTestForm(instance=test) + + if request.method == "POST": + if "id" in request.POST and str(test.id) == request.POST["id"]: + form = DeleteTestForm(request.POST, instance=test) + if form.is_valid(): + if get_setting("ASYNC_OBJECT_DELETE"): + async_del = async_delete() + async_del.delete(test) + message = _("Test and relationships will be removed in the background.") + else: + message = _("Test and relationships removed.") + test.delete() + messages.add_message(request, + messages.SUCCESS, + message, + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_engagement", args=(eng.id,))) + + rels = ["Previewing the relationships has been disabled.", ""] + display_preview = get_setting("DELETE_PREVIEW") + if display_preview: + collector = NestedObjects(using=DEFAULT_DB_ALIAS) + collector.collect([test]) + rels = collector.nested() + + product_tab = Product_Tab(test.engagement.product, title=_("Delete Test"), tab="engagements") + product_tab.setEngagement(test.engagement) + return render(request, "dojo/delete_test.html", + {"test": test, + "product_tab": product_tab, + "form": form, + "rels": rels, + "deletable_objects": rels, + }) + + +def copy_test(request, tid): + test = get_object_or_404(Test, id=tid) + product = test.engagement.product + engagement_list = get_authorized_engagements("edit").filter(product=product) + form = CopyTestForm(engagements=engagement_list) + + if request.method == "POST": + form = CopyTestForm(request.POST, engagements=engagement_list) + if form.is_valid(): + engagement = form.cleaned_data.get("engagement") + copy_test_service(test, engagement, request.user) + messages.add_message( + request, + messages.SUCCESS, + "Test Copied successfully.", + extra_tags="alert-success") + return redirect_to_return_url_or_else(request, reverse("view_engagement", args=(engagement.id, ))) + messages.add_message( + request, + messages.ERROR, + "Unable to copy test, please try again.", + extra_tags="alert-danger") + + product_tab = Product_Tab(product, title="Copy Test", tab="engagements") + return render(request, "dojo/copy_object.html", { + "source": test, + "source_label": "Test", + "destination_label": "Engagement", + "product_tab": product_tab, + "form": form, + }) + + +@cache_page(60 * 5) # cache for 5 minutes +@vary_on_cookie +def test_calendar(request): + + if not get_system_setting("enable_calendar"): + raise Resolver404 + + if "lead" not in request.GET or "0" in request.GET.getlist("lead"): + tests = get_authorized_tests("view") + else: + filters = [] + leads = request.GET.getlist("lead", "") + if "-1" in request.GET.getlist("lead"): + leads.remove("-1") + filters.append(Q(lead__isnull=True)) + filters.append(Q(lead__in=leads)) + tests = get_authorized_tests("view").filter(reduce(operator.or_, filters)) + + tests = tests.prefetch_related("test_type", "lead", "engagement__product") + + add_breadcrumb(title=_("Test Calendar"), top_level=True, request=request) + for t in tests: + if t.target_end: + t.target_end += timedelta(days=1) + return render(request, "dojo/calendar.html", { + "caltype": "tests", + "leads": request.GET.getlist("lead", ""), + "tests": tests, + "users": get_authorized_users("view")}) + + +def test_ics(request, tid): + test = get_object_or_404(Test, id=tid) + start_date = datetime.combine(test.target_start, datetime.min.time()) + end_date = datetime.combine(test.target_end, datetime.max.time()) + if timezone.is_naive(start_date): + start_date = timezone.make_aware(start_date) + if timezone.is_naive(end_date): + end_date = timezone.make_aware(end_date) + uid = f"dojo_test_{test.id}_{test.engagement.id}_{test.engagement.product.id}" + cal = get_cal_event( + start_date, + end_date, + _("Test: %s (%s)") % ( + test.test_type.name, + test.engagement.product.name, + ), + _( + "Set aside for test %s, on product %s. " + "Additional detail can be found at %s", + ) % ( + test.test_type.name, + test.engagement.product.name, + request.build_absolute_uri(reverse("view_test", args=(test.id,))), + ), + uid, + ) + output = cal.serialize() + response = HttpResponse(content=output) + response["Content-Type"] = "text/calendar" + response["Content-Disposition"] = f"attachment; filename={test.test_type.name}.ics" + return response + + +class AddFindingView(View): + def get_test(self, test_id: int): + return get_object_or_404(Test, id=test_id) + + def get_initial_context(self, request: HttpRequest, test: Test): + # Get the finding form first since it is used in another place + finding_form = self.get_finding_form(request, test) + product_tab = Product_Tab(test.engagement.product, title=_("Add Finding"), tab="engagements") + product_tab.setEngagement(test.engagement) + return { + "form": finding_form, + "product_tab": product_tab, + "temp": False, + "test": test, + "tid": test.id, + "pid": test.engagement.product.id, + "form_error": False, + "jform": self.get_jira_form(request, test, finding_form=finding_form), + } + + def get_finding_form(self, request: HttpRequest, test: Test): + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = { + "initial": {"date": timezone.now().date(), "verified": True, "dynamic_finding": False}, + "req_resp": None, + "product": test.engagement.product, + } + # Remove the initial state on post + if request.method == "POST": + kwargs.pop("initial") + + return AddFindingForm(*args, **kwargs) + + def get_jira_form(self, request: HttpRequest, test: Test, finding_form: AddFindingForm = None): + # Determine if jira should be used + if (jira_project := jira_services.get_project(test)) is not None: + # Set up the args for the form + args = [request.POST] if request.method == "POST" else [] + # Set the initial form args + kwargs = { + "push_all": jira_services.is_push_all_issues(test), + "prefix": "jiraform", + "jira_project": jira_project, + "finding_form": finding_form, + } + + return JIRAFindingForm(*args, **kwargs) + return None + + def validate_status_change(self, request: HttpRequest, context: dict): + if ((context["form"]["active"].value() is False + or context["form"]["false_p"].value()) + and context["form"]["duplicate"].value() is False): + + closing_disabled = Note_Type.objects.filter(is_mandatory=True, is_active=True).count() + if closing_disabled != 0: + error_inactive = ValidationError( + _("Can not set a finding as inactive without adding all mandatory notes"), + code="inactive_without_mandatory_notes") + error_false_p = ValidationError( + _("Can not set a finding as false positive without adding all mandatory notes"), + code="false_p_without_mandatory_notes") + if context["form"]["active"].value() is False: + context["form"].add_error("active", error_inactive) + if context["form"]["false_p"].value(): + context["form"].add_error("false_p", error_false_p) + messages.add_message( + request, + messages.ERROR, + _("Can not set a finding as inactive or false positive without adding all mandatory notes"), + extra_tags="alert-danger") + + return request + + def process_finding_form(self, request: HttpRequest, test: Test, context: dict): + finding = None + if context["form"].is_valid(): + finding = context["form"].save(commit=False) + finding.test = test + finding.reporter = request.user + finding.numerical_severity = Finding.get_numerical_severity(finding.severity) + finding.tags = context["form"].cleaned_data["tags"] + finding.unsaved_vulnerability_ids = context["form"].cleaned_data["vulnerability_ids"].split() + finding.save() + # Save and add new locations + finding_helper.add_locations(finding, context["form"]) + # Save the finding at the end and return + finding.save() + + return finding, request, True + add_error_message_to_response("The form has errors, please correct them below.") + add_field_errors_to_response(context["form"]) + + return finding, request, False + + def process_jira_form(self, request: HttpRequest, finding: Finding, context: dict): + # Capture case if the jira not being enabled + if context["jform"] is None: + return request, True, False + + if context["jform"] and context["jform"].is_valid(): + # can't use helper as when push_all_jira_issues is True, the checkbox gets disabled and is always false + # push_to_jira = jira_services.is_push_to_jira(finding, jform.cleaned_data.get('push_to_jira')) + push_to_jira = jira_services.is_push_all_issues(finding) or context["jform"].cleaned_data.get("push_to_jira") + jira_message = None + # if the jira issue key was changed, update database + new_jira_issue_key = context["jform"].cleaned_data.get("jira_issue") + if finding.has_jira_issue: + # everything in DD around JIRA integration is based on the internal id of the issue in JIRA + # instead of on the public jira issue key. + # I have no idea why, but it means we have to retrieve the issue from JIRA to get the internal JIRA id. + # we can assume the issue exist, which is already checked in the validation of the jform + if not new_jira_issue_key: + jira_services.unlink_finding(request, finding) + jira_message = "Link to JIRA issue removed successfully." + + elif new_jira_issue_key != finding.jira_issue.jira_key: + jira_services.unlink_finding(request, finding) + jira_services.link_finding(request, finding, new_jira_issue_key) + jira_message = "Changed JIRA link successfully." + else: + logger.debug("finding has no jira issue yet") + if new_jira_issue_key: + logger.debug("finding has no jira issue yet, but jira issue specified in request. trying to link.") + jira_services.link_finding(request, finding, new_jira_issue_key) + jira_message = "Linked a JIRA issue successfully." + # Determine if a message should be added + if jira_message: + messages.add_message( + request, messages.SUCCESS, jira_message, extra_tags="alert-success", + ) + + return request, True, push_to_jira + add_field_errors_to_response(context["jform"]) + + return request, False, False + + def process_forms(self, request: HttpRequest, test: Test, context: dict): + form_success_list = [] + finding = None + # Set vars for the completed forms + # Validate finding mitigation + request = self.validate_status_change(request, context) + # Check the validity of the form overall + finding, request, success = self.process_finding_form(request, test, context) + form_success_list.append(success) + request, success, push_to_jira = self.process_jira_form(request, finding, context) + form_success_list.append(success) + # Determine if all forms were successful + all_forms_valid = all(form_success_list) + # Check the validity of all the forms + if all_forms_valid: + # if we're removing the "duplicate" in the edit finding screen + finding_helper.save_vulnerability_ids(finding, context["form"].cleaned_data["vulnerability_ids"].split()) + # Push things to jira if needed + finding.save(push_to_jira=push_to_jira) + # Save the burp req resp + if "request" in context["form"].cleaned_data or "response" in context["form"].cleaned_data: + burp_rr = BurpRawRequestResponse( + finding=finding, + burpRequestBase64=base64.b64encode(context["form"].cleaned_data["request"].encode()), + burpResponseBase64=base64.b64encode(context["form"].cleaned_data["response"].encode()), + ) + burp_rr.clean() + burp_rr.save() + + # Note: this notification has not be moved to "@receiver(post_save, sender=Finding)" method as many other notifications + # Because it could generate too much noise, we keep it here only for findings created by hand in WebUI + + # Create a notification + create_notification( + event="finding_added", + title=_("Addition of %s") % finding.title, + finding=finding, + description=_('Finding "%s" was added by %s') % (finding.title, request.user), + url=reverse("view_finding", args=(finding.id,)), + icon="exclamation-triangle") + # Add a success message + messages.add_message( + request, + messages.SUCCESS, + _("Finding added successfully."), + extra_tags="alert-success") + + return finding, request, all_forms_valid + + def get_template(self): + return "dojo/add_findings.html" + + def get(self, request: HttpRequest, test_id: int): + # Get the initial objects + test = self.get_test(test_id) + # Make sure the user is authorized + user_has_permission_or_403(request.user, test, "add") + # Set up the initial context + context = self.get_initial_context(request, test) + # Render the form + return render(request, self.get_template(), context) + + def post(self, request: HttpRequest, test_id: int): + # Get the initial objects + test = self.get_test(test_id) + # Make sure the user is authorized + user_has_permission_or_403(request.user, test, "add") + # Set up the initial context + context = self.get_initial_context(request, test) + # Process the form + _, request, success = self.process_forms(request, test, context) + # Handle the case of a successful form + if success: + if "_Finished" in request.POST: + return HttpResponseRedirect(reverse("view_test", args=(test.id,))) + return HttpResponseRedirect(reverse("add_findings", args=(test.id,))) + context["form_error"] = True + # Render the form + return render(request, self.get_template(), context) + + +def add_finding_from_template(request, tid, fid): + jform = None + test = get_object_or_404(Test, id=tid) + user_has_permission_or_403(request.user, test, "add") + template = get_object_or_404(Finding_Template, id=fid) + findings = Finding_Template.objects.all() + push_all_jira_issues = jira_services.is_push_all_issues(template) + + if request.method == "POST": + + form = AddFindingForm(request.POST, req_resp=None, product=test.engagement.product) + if jira_services.get_project(test): + jform = JIRAFindingForm(push_all=jira_services.is_push_all_issues(test), prefix="jiraform", jira_project=jira_services.get_project(test), finding_form=form) + logger.debug(f"jform valid: {jform.is_valid()}") + + if (form["active"].value() is False or form["false_p"].value()) and form["duplicate"].value() is False: + closing_disabled = Note_Type.objects.filter(is_mandatory=True, is_active=True).count() + if closing_disabled != 0: + error_inactive = ValidationError( + _("Can not set a finding as inactive without adding all mandatory notes"), + code="not_active_or_false_p_true") + error_false_p = ValidationError( + _("Can not set a finding as false positive without adding all mandatory notes"), + code="not_active_or_false_p_true") + if form["active"].value() is False: + form.add_error("active", error_inactive) + if form["false_p"].value(): + form.add_error("false_p", error_false_p) + messages.add_message(request, + messages.ERROR, + _("Can not set a finding as inactive or false positive without adding all mandatory notes"), + extra_tags="alert-danger") + if form.is_valid(): + template.last_used = timezone.now() + template.save() + new_finding = form.save(commit=False) + new_finding.test = test + new_finding.reporter = request.user + new_finding.numerical_severity = Finding.get_numerical_severity( + new_finding.severity) + + new_finding.tags = form.cleaned_data["tags"] + new_finding.date = form.cleaned_data["date"] or datetime.today() + + finding_helper.update_finding_status(new_finding, request.user) + + new_finding.save(dedupe_option=False) + + # Copy all fields from template + finding_helper.copy_template_fields_to_finding( + finding=new_finding, + template=template, + user=request.user, + copy_vulnerability_ids=True, + copy_endpoints=True, + copy_notes=True, + ) + + # Save and add new locations from form (user may have added more) + finding_helper.add_locations(new_finding, form) + + new_finding.save() + if "jiraform-push_to_jira" in request.POST: + jform = JIRAFindingForm(request.POST, prefix="jiraform", instance=new_finding, push_all=push_all_jira_issues, jira_project=jira_services.get_project(test), finding_form=form) + if jform.is_valid(): + if jform.cleaned_data.get("push_to_jira"): + jira_services.push(new_finding) + else: + add_error_message_to_response(f"jira form validation failed: {jform.errors}") + if "request" in form.cleaned_data or "response" in form.cleaned_data: + burp_rr = BurpRawRequestResponse( + finding=new_finding, + burpRequestBase64=base64.b64encode(form.cleaned_data.get("request", "").encode("utf-8")), + burpResponseBase64=base64.b64encode(form.cleaned_data.get("response", "").encode("utf-8")), + ) + burp_rr.clean() + burp_rr.save() + messages.add_message(request, + messages.SUCCESS, + _("Finding from template added successfully."), + extra_tags="alert-success") + + return HttpResponseRedirect(reverse("view_test", args=(test.id,))) + messages.add_message(request, + messages.ERROR, + _("The form has errors, please correct them below."), + extra_tags="alert-danger") + + else: + # Build initial data with all template fields + initial_data = { + "active": False, + "date": timezone.now().date(), + "verified": False, + "false_p": False, + "duplicate": False, + "out_of_scope": False, + "title": template.title, + "description": template.description, + "cwe": template.cwe, + "severity": template.severity, + "mitigation": template.mitigation, + "impact": template.impact, + "references": template.references, + "numerical_severity": template.numerical_severity, + } + + # Add CVSS fields + if template.cvssv3: + initial_data["cvssv3"] = template.cvssv3 + if template.cvssv3_score is not None: + initial_data["cvssv3_score"] = template.cvssv3_score + if template.cvssv4: + initial_data["cvssv4"] = template.cvssv4 + if template.cvssv4_score is not None: + initial_data["cvssv4_score"] = template.cvssv4_score + + # Add remediation fields + if template.fix_available is not None: + initial_data["fix_available"] = template.fix_available + if template.fix_version: + initial_data["fix_version"] = template.fix_version + if template.planned_remediation_version: + initial_data["planned_remediation_version"] = template.planned_remediation_version + if template.effort_for_fixing: + initial_data["effort_for_fixing"] = template.effort_for_fixing + + # Add technical details fields + if template.steps_to_reproduce: + initial_data["steps_to_reproduce"] = template.steps_to_reproduce + if template.severity_justification: + initial_data["severity_justification"] = template.severity_justification + if template.component_name: + initial_data["component_name"] = template.component_name + if template.component_version: + initial_data["component_version"] = template.component_version + + # Add vulnerability IDs + if template.vulnerability_ids: + initial_data["vulnerability_ids"] = " ".join(template.vulnerability_ids) + + # Add endpoints to endpoints_to_add field + if template.endpoints: + endpoint_urls = template.endpoints if isinstance(template.endpoints, list) else template.endpoints.split("\n") + initial_data["endpoints_to_add"] = "\n".join([url.strip() for url in endpoint_urls if url.strip()]) + + form = AddFindingForm(req_resp=None, product=test.engagement.product, initial=initial_data) + + if jira_services.get_project(test): + jform = JIRAFindingForm(push_all=jira_services.is_push_all_issues(test), prefix="jiraform", jira_project=jira_services.get_project(test), finding_form=form) + + product_tab = Product_Tab(test.engagement.product, title=_("Add Finding"), tab="engagements") + product_tab.setEngagement(test.engagement) + return render(request, "dojo/add_findings.html", + {"form": form, + "product_tab": product_tab, + "jform": jform, + "findings": findings, + "temp": True, + "fid": template.id, + "tid": test.id, + "test": test, + }) + + +def search(request, tid): + test = get_object_or_404(Test, id=tid) + templates = Finding_Template.objects.all() + templates = TemplateFindingFilter(request.GET, queryset=templates) + paged_templates = get_page_items(request, templates.qs, 25) + + title_words = get_words_for_field(Finding_Template, "title") + + add_breadcrumb(parent=test, title=_("Add From Template"), top_level=False, request=request) + return render(request, "dojo/templates.html", + {"templates": paged_templates, + "filtered": templates, + "title_words": title_words, + "tid": tid, + "add_from_template": True, + }) + + +class ReImportScanResultsView(View): + def get_template(self) -> str: + """Returns the template that will be presented to the user""" + return "dojo/import_scan_results.html" + + def get_form( + self, + request: HttpRequest, + test: Test, + **kwargs: dict, + ) -> ReImportScanForm: + """Returns the default import form for importing findings""" + if request.method == "POST": + return ReImportScanForm(request.POST, request.FILES, test=test, **kwargs) + return ReImportScanForm(test=test, **kwargs) + + def get_jira_form( + self, + request: HttpRequest, + test: Test, + ) -> tuple[JIRAImportScanForm | None, bool]: + """Returns a JiraImportScanForm if jira is enabled""" + jira_form = None + push_all_jira_issues = False + # Decide if we need to present the Push to JIRA form + if get_system_setting("enable_jira"): + # Determine if jira issues should be pushed automatically + push_all_jira_issues = jira_services.is_push_all_issues(test) + # Only return the form if the jira is enabled on this engagement or product + if jira_services.get_project(test): + if request.method == "POST": + jira_form = JIRAImportScanForm( + request.POST, + push_all=push_all_jira_issues, + prefix="jiraform", + ) + else: + jira_form = JIRAImportScanForm( + push_all=push_all_jira_issues, + prefix="jiraform", + ) + return jira_form, push_all_jira_issues + + def handle_request( + self, + request: HttpRequest, + test_id: int, + ) -> tuple[HttpRequest, dict]: + """ + Process the common behaviors between request types, and then return + the request and context dict back to be rendered + """ + # Get the test object + test = get_object_or_404(Test, id=test_id) + # Ensure the supplied user has access to import to the engagement or product + user_has_permission_or_403(request.user, test, "import") + # by default we keep a trace of the scan_type used to create the test + # if it's not here, we use the "name" of the test type + # this feature exists to provide custom label for tests for some parsers + scan_type = test.scan_type or test.test_type.name + # Set the product tab + product_tab = Product_Tab(test.engagement.product, title=_("Re-upload a %s") % scan_type, tab="engagements") + product_tab.setEngagement(test.engagement) + # Get the import form with some initial data in place + if settings.V3_FEATURE_LOCATIONS: + endpoints = Location.objects.filter(products__product__id=product_tab.product.id) + else: + # TODO: Delete this after the move to Locations + endpoints = Endpoint.objects.filter(product__id=product_tab.product.id) + form = self.get_form( + request, + test, + endpoints=endpoints, + api_scan_configuration=test.api_scan_configuration, + api_scan_configuration_queryset=Product_API_Scan_Configuration.objects.filter(product__id=product_tab.product.id), + ) + # Get the jira form + jira_form, push_all_jira_issues = self.get_jira_form(request, test) + # Return the request and the context + return request, { + "test": test, + "form": form, + "product_tab": product_tab, + "eid": test.engagement.id, + "jform": jira_form, + "scan_type": scan_type, + "scan_types": get_scan_types_sorted(), + "push_all_jira_issues": push_all_jira_issues, + "additional_message": ( + "When re-uploading a scan, any findings not found in original scan will be updated as " + "mitigated. The process attempts to identify the differences, however manual verification " + "is highly recommended." + ), + } + + def validate_forms( + self, + context: dict, + ) -> bool: + """ + Validates each of the forms to ensure all errors from the form + level are bubbled up to the user first before we process too much + """ + form_validation_list = [] + for form_name in ["form", "jform"]: + if (form := context.get(form_name)) is not None: + if errors := form.errors: + form_validation_list.append(errors) + return form_validation_list + + def process_form( + self, + request: HttpRequest, + form: ReImportScanForm, + context: dict, + ) -> str | None: + """Process the form and manipulate the input in any way that is appropriate""" + # Update the running context dict with cleaned form input + context.update({ + "scan": request.FILES.get("file", None), + "scan_date": form.cleaned_data.get("scan_date"), + "minimum_severity": form.cleaned_data.get("minimum_severity"), + "do_not_reactivate": form.cleaned_data.get("do_not_reactivate"), + "tags": form.cleaned_data.get("tags"), + "version": form.cleaned_data.get("version") or None, + "branch_tag": form.cleaned_data.get("branch_tag") or None, + "build_id": form.cleaned_data.get("build_id") or None, + "commit_hash": form.cleaned_data.get("commit_hash") or None, + "api_scan_configuration": form.cleaned_data.get("api_scan_configuration") or None, + "service": form.cleaned_data.get("service") or None, + "apply_tags_to_findings": form.cleaned_data.get("apply_tags_to_findings", False), + "apply_tags_to_endpoints": form.cleaned_data.get("apply_tags_to_endpoints", False), + "group_by": form.cleaned_data.get("group_by") or None, + "close_old_findings": form.cleaned_data.get("close_old_findings", None), + "create_finding_groups_for_all_findings": form.cleaned_data.get("create_finding_groups_for_all_findings", None), + }) + # Override the form values of active and verified + if activeChoice := form.cleaned_data.get("active", None): + if activeChoice == "force_to_true": + context["active"] = True + elif activeChoice == "force_to_false": + context["active"] = False + if verifiedChoice := form.cleaned_data.get("verified", None): + if verifiedChoice == "force_to_true": + context["verified"] = True + elif verifiedChoice == "force_to_false": + context["verified"] = False + # Override the tags and version + context.get("test").tags = context.get("tags") + context.get("test").version = context.get("version") + return None + + def process_jira_form( + self, + request: HttpRequest, + form: JIRAImportScanForm, + context: dict, + ) -> str | None: + """ + Process the jira form by first making sure one was supplied + and then setting any values supplied by the user. An error + may be returned and will be bubbled up in the form of a message + """ + # Determine if push all issues is enabled + push_all_jira_issues = context.get("push_all_jira_issues", False) + context["push_to_jira"] = push_all_jira_issues or (form and form.cleaned_data.get("push_to_jira")) + return None + + def get_reimporter( + self, + context: dict, + ) -> BaseImporter: + """Gets the reimporter to use""" + return DefaultReImporter(**context) + + def reimport_findings( + self, + context: dict, + ) -> str | None: + """Attempt to import with all the supplied information""" + try: + # Log only user-entered form values, excluding internal objects + user_values = { + "scan_type": context.get("scan_type"), + "scan_date": context.get("scan_date"), + "minimum_severity": context.get("minimum_severity"), + "active": context.get("active"), + "verified": context.get("verified"), + "tags": context.get("tags"), + "version": context.get("version"), + "branch_tag": context.get("branch_tag"), + "build_id": context.get("build_id"), + "commit_hash": context.get("commit_hash"), + "service": context.get("service"), + "close_old_findings": context.get("close_old_findings"), + "apply_tags_to_findings": context.get("apply_tags_to_findings"), + "apply_tags_to_endpoints": context.get("apply_tags_to_endpoints"), + "close_old_findings_product_scope": context.get("close_old_findings_product_scope"), + "group_by": context.get("group_by"), + "create_finding_groups_for_all_findings": context.get("create_finding_groups_for_all_findings"), + "push_to_jira": context.get("push_to_jira"), + "push_all_jira_issues": context.get("push_all_jira_issues"), + "do_not_reactivate": context.get("do_not_reactivate"), + } + logger.debug(f"reimport_findings called with user values: {user_values}") + importer_client = self.get_reimporter(context) + ( + context["test"], + finding_count, + new_finding_count, + closed_finding_count, + reactivated_finding_count, + untouched_finding_count, + _, + ) = importer_client.process_scan( + context.pop("scan", None), + ) + # Add a message to the view for the user to see the results + add_success_message_to_response(importer_client.construct_imported_message( + finding_count=finding_count, + new_finding_count=new_finding_count, + closed_finding_count=closed_finding_count, + reactivated_finding_count=reactivated_finding_count, + untouched_finding_count=untouched_finding_count, + )) + except Exception as e: + logger.exception("An exception error occurred during the report import") + return f"An exception error occurred during the report import: {e}" + return None + + def success_redirect( + self, + request: HttpRequest, + context: dict, + ) -> HttpResponseRedirect: + """Redirect the user to a place that indicates a successful import""" + duration = time.perf_counter() - request._start_time + LargeScanSizeProductAnnouncement(request=request, duration=duration) + ScanTypeProductAnnouncement(request=request, scan_type=context.get("scan_type")) + return HttpResponseRedirect(reverse("view_test", args=(context.get("test").id, ))) + + def failure_redirect( + self, + request: HttpRequest, + context: dict, + ) -> HttpResponseRedirect: + """Redirect the user to a place that indicates a failed import""" + ErrorPageProductAnnouncement(request=request) + return HttpResponseRedirect(reverse( + "re_import_scan_results", + args=(context.get("test").id, ), + )) + + def get( + self, + request: HttpRequest, + test_id: int, + ) -> HttpResponse: + """Process GET requests for the ReImport View""" + # process the request and path parameters + request, context = self.handle_request( + request, + test_id=test_id, + ) + # Render the form + return render(request, self.get_template(), context) + + def post( + self, + request: HttpRequest, + test_id: int, + ) -> HttpResponse: + """Process POST requests for the ReImport View""" + # process the request and path parameters + request, context = self.handle_request( + request, + test_id=test_id, + ) + request._start_time = time.perf_counter() + # ensure all three forms are valid first before moving forward + if form_errors := self.validate_forms(context): + for form_error in form_errors: + add_error_message_to_response(form_error) + return self.failure_redirect(request, context) + # Process the jira form if it is present + if form_error := self.process_jira_form(request, context.get("jform"), context): + add_error_message_to_response(form_error) + return self.failure_redirect(request, context) + # Process the import form + if form_error := self.process_form(request, context.get("form"), context): + add_error_message_to_response(form_error) + return self.failure_redirect(request, context) + # Add pghistory context for audit trail (adds to existing middleware context) + pghistory.context( + source="reimport", + test_id=context.get("test").id, + scan_type=context.get("scan_type"), + ) + # Kick off the import process + if import_error := self.reimport_findings(context): + add_error_message_to_response(import_error) + return self.failure_redirect(request, context) + # Otherwise return the user back to the engagement (if present) or the product + return self.success_redirect(request, context) diff --git a/dojo/test/views.py b/dojo/test/views.py index 4e7f9c54dba..c8a504b106a 100644 --- a/dojo/test/views.py +++ b/dojo/test/views.py @@ -1,1130 +1,4 @@ -# # tests -import base64 -import logging -import operator -import time -from datetime import datetime, timedelta -from functools import reduce - -import pghistory -from django.conf import settings -from django.contrib import messages -from django.contrib.admin.utils import NestedObjects -from django.core.exceptions import ValidationError -from django.db import DEFAULT_DB_ALIAS -from django.db.models import Count, Q -from django.http import HttpRequest, HttpResponse, HttpResponseRedirect -from django.shortcuts import get_object_or_404, render -from django.urls import Resolver404, reverse -from django.utils import timezone -from django.utils.translation import gettext as _ -from django.views import View -from django.views.decorators.cache import cache_page -from django.views.decorators.vary import vary_on_cookie - -import dojo.finding.helper as finding_helper -from dojo.authorization.authorization import user_has_permission_or_403 -from dojo.celery_dispatch import dojo_dispatch_task -from dojo.engagement.queries import get_authorized_engagements -from dojo.filters import FindingFilter, FindingFilterWithoutObjectLookups, TemplateFindingFilter, TestImportFilter -from dojo.finding.queries import prefetch_for_findings -from dojo.finding.views import find_available_notetypes -from dojo.forms import ( - AddFindingForm, - CopyTestForm, - DeleteTestForm, - FindingBulkUpdateForm, - JIRAFindingForm, - JIRAImportScanForm, - NoteForm, - ReImportScanForm, - TestForm, - TypedNoteForm, -) -from dojo.importers.base_importer import BaseImporter -from dojo.importers.default_reimporter import DefaultReImporter -from dojo.jira import services as jira_services -from dojo.location.models import Location -from dojo.models import ( - BurpRawRequestResponse, - Endpoint, - Finding, - Finding_Group, - Finding_Template, - Note_Type, - Product_API_Scan_Configuration, - Test, - Test_Import, -) -from dojo.notifications.helper import create_notification -from dojo.product_announcements import ( - ErrorPageProductAnnouncement, - LargeScanSizeProductAnnouncement, - ScanTypeProductAnnouncement, -) -from dojo.test.queries import get_authorized_tests -from dojo.tools.factory import get_choices_sorted, get_scan_types_sorted -from dojo.user.queries import get_authorized_users -from dojo.utils import ( - Product_Tab, - add_breadcrumb, - add_error_message_to_response, - add_field_errors_to_response, - add_success_message_to_response, - async_delete, - calculate_grade, - get_cal_event, - get_page_items, - get_page_items_and_count, - get_setting, - get_system_setting, - get_words_for_field, - process_tag_notifications, - redirect_to_return_url_or_else, -) - -logger = logging.getLogger(__name__) -parse_logger = logging.getLogger("dojo") -deduplicationLogger = logging.getLogger("dojo.specific-loggers.deduplication") - - -class ViewTest(View): - def get_test(self, test_id: int): - test_prefetched = get_authorized_tests("view") - test_prefetched = test_prefetched.annotate(total_reimport_count=Count("test_import__id", distinct=True)) - return get_object_or_404(test_prefetched, pk=test_id) - - def get_test_import_data(self, request: HttpRequest, test: Test): - test_imports = Test_Import.objects.filter(test=test) - test_import_filter = TestImportFilter(request.GET, test_imports) - - paged_test_imports = get_page_items_and_count(request, test_import_filter.qs, 5, prefix="test_imports") - paged_test_imports.object_list = paged_test_imports.object_list.prefetch_related("test_import_finding_action_set") - - return { - "paged_test_imports": paged_test_imports, - "test_import_filter": test_import_filter, - } - - def get_findings(self, request: HttpRequest, test: Test): - findings = Finding.objects.filter(test=test).order_by("numerical_severity") - filter_string_matching = get_system_setting("filter_string_matching", False) - finding_filter_class = FindingFilterWithoutObjectLookups if filter_string_matching else FindingFilter - findings = finding_filter_class(request.GET, pid=test.engagement.product.id, eid=test.engagement.id, tid=test.id, queryset=findings) - paged_findings = get_page_items_and_count(request, prefetch_for_findings(findings.qs), 25, prefix="findings") - fix_available_count = findings.qs.filter(fix_available=True).count() - - return { - "findings": paged_findings, - "filtered": findings, - "fix_available_count": fix_available_count, - } - - def get_note_form(self, request: HttpRequest): - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = {} - - return NoteForm(*args, **kwargs) - - def get_typed_note_form(self, request: HttpRequest, context: dict): - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = { - "available_note_types": context.get("available_note_types"), - } - - return TypedNoteForm(*args, **kwargs) - - def get_form(self, request: HttpRequest, context: dict): - return ( - self.get_typed_note_form(request, context) - if context.get("note_type_activation") - else self.get_note_form(request) - ) - - def get_initial_context(self, request: HttpRequest, test: Test): - # Set up the product tab - product_tab = Product_Tab(test.engagement.product, title=_("Test"), tab="engagements") - product_tab.setEngagement(test.engagement) - # Set up the notes and associated info to generate the form with - notes = test.notes.all() - note_type_activation = Note_Type.objects.filter(is_active=True).count() - available_note_types = None - if note_type_activation: - available_note_types = find_available_notetypes(notes) - # Set the current context - context = { - "test": test, - "prod": test.engagement.product, - "product_tab": product_tab, - "title_words": get_words_for_field(Finding, "title"), - "component_words": get_words_for_field(Finding, "component_name"), - "notes": notes, - "note_type_activation": note_type_activation, - "available_note_types": available_note_types, - "files": test.files.all(), - "person": request.user.username, - "request": request, - "show_re_upload": any(test.test_type.name in code for code in get_choices_sorted()), - "jira_project": jira_services.get_project(test), - "bulk_edit_form": FindingBulkUpdateForm(request.GET), - "enable_table_filtering": get_system_setting("enable_ui_table_based_searching"), - "finding_groups": test.finding_group_set.all().prefetch_related("findings", "jira_issue", "creator", "findings__vulnerability_id_set"), - "finding_group_by_options": Finding_Group.GROUP_BY_OPTIONS, - } - # Set the form using the context, and then update the context - form = self.get_form(request, context) - context["form"] = form - # Add some of the related objects - context |= self.get_findings(request, test) - context |= self.get_test_import_data(request, test) - - return context - - def process_form(self, request: HttpRequest, test: Test, context: dict): - if context["form"].is_valid(): - # Save the note - new_note = context["form"].save(commit=False) - new_note.author = request.user - new_note.date = timezone.now() - new_note.save() - test.notes.add(new_note) - # Make a notification for this actions - url = request.build_absolute_uri(reverse("view_test", args=(test.id,))) - title = f"Test: {test.test_type.name} on {test.engagement.product.name}" - process_tag_notifications(request, new_note, url, title) - messages.add_message( - request, - messages.SUCCESS, - _("Note added successfully."), - extra_tags="alert-success") - - return request, True - return request, False - - def get_template(self): - return "dojo/view_test.html" - - def get(self, request: HttpRequest, test_id: int): - # Get the initial objects - test = self.get_test(test_id) - # Make sure the user is authorized - user_has_permission_or_403(request.user, test, "view") - # Quick perms check to determine if the user has access to add a note to the test - user_has_permission_or_403(request.user, test, "add") - # Set up the initial context - context = self.get_initial_context(request, test) - # Render the form - return render(request, self.get_template(), context) - - def post(self, request: HttpRequest, test_id: int): - # Get the initial objects - test = self.get_test(test_id) - # Make sure the user is authorized - user_has_permission_or_403(request.user, test, "view") - # Quick perms check to determine if the user has access to add a note to the test - user_has_permission_or_403(request.user, test, "add") - # Set up the initial context - context = self.get_initial_context(request, test) - # Determine the validity of the form - request, success = self.process_form(request, test, context) - # Handle the case of a successful form - if success: - return redirect_to_return_url_or_else(request, reverse("view_test", args=(test_id,))) - # Render the form - return render(request, self.get_template(), context) - -# def prefetch_for_test_imports(test_imports): -# prefetched_test_imports = test_imports -# if isinstance(test_imports, QuerySet): # old code can arrive here with prods being a list because the query was already executed -# #could we make this dynamic, i.e for action_type in IMPORT_ACTIONS: prefetch -# prefetched_test_imports = prefetched_test_imports.annotate(created_findings_count=Count('findings', filter=Q(test_import_finding_action__action=IMPORT_CREATED_FINDING))) -# prefetched_test_imports = prefetched_test_imports.annotate(closed_findings_count=Count('findings', filter=Q(test_import_finding_action__action=IMPORT_CLOSED_FINDING))) -# prefetched_test_imports = prefetched_test_imports.annotate(reactivated_findings_count=Count('findings', filter=Q(test_import_finding_action__action=IMPORT_REACTIVATED_FINDING))) -# prefetched_test_imports = prefetched_test_imports.annotate(updated_findings_count=Count('findings', filter=Q(test_import_finding_action__action=IMPORT_UNTOUCHED_FINDING))) - -# return prefetch_for_test_imports - - -def edit_test(request, tid): - test = get_object_or_404(Test, pk=tid) - form = TestForm(instance=test) - if request.method == "POST": - form = TestForm(request.POST, instance=test) - if form.is_valid(): - form.save() - messages.add_message(request, - messages.SUCCESS, - _("Test saved."), - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_engagement", args=(test.engagement.id,))) - - form.initial["target_start"] = test.target_start.date() - form.initial["target_end"] = test.target_end.date() - form.initial["description"] = test.description - - product_tab = Product_Tab(test.engagement.product, title=_("Edit Test"), tab="engagements") - product_tab.setEngagement(test.engagement) - return render(request, "dojo/edit_test.html", - {"test": test, - "product_tab": product_tab, - "form": form, - }) - - -def delete_test(request, tid): - test = get_object_or_404(Test, pk=tid) - eng = test.engagement - form = DeleteTestForm(instance=test) - - if request.method == "POST": - if "id" in request.POST and str(test.id) == request.POST["id"]: - form = DeleteTestForm(request.POST, instance=test) - if form.is_valid(): - if get_setting("ASYNC_OBJECT_DELETE"): - async_del = async_delete() - async_del.delete(test) - message = _("Test and relationships will be removed in the background.") - else: - message = _("Test and relationships removed.") - test.delete() - messages.add_message(request, - messages.SUCCESS, - message, - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_engagement", args=(eng.id,))) - - rels = ["Previewing the relationships has been disabled.", ""] - display_preview = get_setting("DELETE_PREVIEW") - if display_preview: - collector = NestedObjects(using=DEFAULT_DB_ALIAS) - collector.collect([test]) - rels = collector.nested() - - product_tab = Product_Tab(test.engagement.product, title=_("Delete Test"), tab="engagements") - product_tab.setEngagement(test.engagement) - return render(request, "dojo/delete_test.html", - {"test": test, - "product_tab": product_tab, - "form": form, - "rels": rels, - "deletable_objects": rels, - }) - - -def copy_test(request, tid): - test = get_object_or_404(Test, id=tid) - product = test.engagement.product - engagement_list = get_authorized_engagements("edit").filter(product=product) - form = CopyTestForm(engagements=engagement_list) - - if request.method == "POST": - form = CopyTestForm(request.POST, engagements=engagement_list) - if form.is_valid(): - engagement = form.cleaned_data.get("engagement") - product = test.engagement.product - test_copy = test.copy(engagement=engagement) - dojo_dispatch_task(calculate_grade, product.id) - messages.add_message( - request, - messages.SUCCESS, - "Test Copied successfully.", - extra_tags="alert-success") - create_notification(event="test_copied", # TODO: - if 'copy' functionality will be supported by API as well, 'create_notification' needs to be migrated to place where it will be able to cover actions from both interfaces - title=f"Copying of {test.title}", - description=f'The test "{test.title}" was copied by {request.user} to {engagement.name}', - product=product, - url=request.build_absolute_uri(reverse("view_test", args=(test_copy.id,))), - recipients=[test.engagement.lead], - icon="exclamation-triangle") - return redirect_to_return_url_or_else(request, reverse("view_engagement", args=(engagement.id, ))) - messages.add_message( - request, - messages.ERROR, - "Unable to copy test, please try again.", - extra_tags="alert-danger") - - product_tab = Product_Tab(product, title="Copy Test", tab="engagements") - return render(request, "dojo/copy_object.html", { - "source": test, - "source_label": "Test", - "destination_label": "Engagement", - "product_tab": product_tab, - "form": form, - }) - - -@cache_page(60 * 5) # cache for 5 minutes -@vary_on_cookie -def test_calendar(request): - - if not get_system_setting("enable_calendar"): - raise Resolver404 - - if "lead" not in request.GET or "0" in request.GET.getlist("lead"): - tests = get_authorized_tests("view") - else: - filters = [] - leads = request.GET.getlist("lead", "") - if "-1" in request.GET.getlist("lead"): - leads.remove("-1") - filters.append(Q(lead__isnull=True)) - filters.append(Q(lead__in=leads)) - tests = get_authorized_tests("view").filter(reduce(operator.or_, filters)) - - tests = tests.prefetch_related("test_type", "lead", "engagement__product") - - add_breadcrumb(title=_("Test Calendar"), top_level=True, request=request) - for t in tests: - if t.target_end: - t.target_end += timedelta(days=1) - return render(request, "dojo/calendar.html", { - "caltype": "tests", - "leads": request.GET.getlist("lead", ""), - "tests": tests, - "users": get_authorized_users("view")}) - - -def test_ics(request, tid): - test = get_object_or_404(Test, id=tid) - start_date = datetime.combine(test.target_start, datetime.min.time()) - end_date = datetime.combine(test.target_end, datetime.max.time()) - if timezone.is_naive(start_date): - start_date = timezone.make_aware(start_date) - if timezone.is_naive(end_date): - end_date = timezone.make_aware(end_date) - uid = f"dojo_test_{test.id}_{test.engagement.id}_{test.engagement.product.id}" - cal = get_cal_event( - start_date, - end_date, - _("Test: %s (%s)") % ( - test.test_type.name, - test.engagement.product.name, - ), - _( - "Set aside for test %s, on product %s. " - "Additional detail can be found at %s", - ) % ( - test.test_type.name, - test.engagement.product.name, - request.build_absolute_uri(reverse("view_test", args=(test.id,))), - ), - uid, - ) - output = cal.serialize() - response = HttpResponse(content=output) - response["Content-Type"] = "text/calendar" - response["Content-Disposition"] = f"attachment; filename={test.test_type.name}.ics" - return response - - -class AddFindingView(View): - def get_test(self, test_id: int): - return get_object_or_404(Test, id=test_id) - - def get_initial_context(self, request: HttpRequest, test: Test): - # Get the finding form first since it is used in another place - finding_form = self.get_finding_form(request, test) - product_tab = Product_Tab(test.engagement.product, title=_("Add Finding"), tab="engagements") - product_tab.setEngagement(test.engagement) - return { - "form": finding_form, - "product_tab": product_tab, - "temp": False, - "test": test, - "tid": test.id, - "pid": test.engagement.product.id, - "form_error": False, - "jform": self.get_jira_form(request, test, finding_form=finding_form), - } - - def get_finding_form(self, request: HttpRequest, test: Test): - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = { - "initial": {"date": timezone.now().date(), "verified": True, "dynamic_finding": False}, - "req_resp": None, - "product": test.engagement.product, - } - # Remove the initial state on post - if request.method == "POST": - kwargs.pop("initial") - - return AddFindingForm(*args, **kwargs) - - def get_jira_form(self, request: HttpRequest, test: Test, finding_form: AddFindingForm = None): - # Determine if jira should be used - if (jira_project := jira_services.get_project(test)) is not None: - # Set up the args for the form - args = [request.POST] if request.method == "POST" else [] - # Set the initial form args - kwargs = { - "push_all": jira_services.is_push_all_issues(test), - "prefix": "jiraform", - "jira_project": jira_project, - "finding_form": finding_form, - } - - return JIRAFindingForm(*args, **kwargs) - return None - - def validate_status_change(self, request: HttpRequest, context: dict): - if ((context["form"]["active"].value() is False - or context["form"]["false_p"].value()) - and context["form"]["duplicate"].value() is False): - - closing_disabled = Note_Type.objects.filter(is_mandatory=True, is_active=True).count() - if closing_disabled != 0: - error_inactive = ValidationError( - _("Can not set a finding as inactive without adding all mandatory notes"), - code="inactive_without_mandatory_notes") - error_false_p = ValidationError( - _("Can not set a finding as false positive without adding all mandatory notes"), - code="false_p_without_mandatory_notes") - if context["form"]["active"].value() is False: - context["form"].add_error("active", error_inactive) - if context["form"]["false_p"].value(): - context["form"].add_error("false_p", error_false_p) - messages.add_message( - request, - messages.ERROR, - _("Can not set a finding as inactive or false positive without adding all mandatory notes"), - extra_tags="alert-danger") - - return request - - def process_finding_form(self, request: HttpRequest, test: Test, context: dict): - finding = None - if context["form"].is_valid(): - finding = context["form"].save(commit=False) - finding.test = test - finding.reporter = request.user - finding.numerical_severity = Finding.get_numerical_severity(finding.severity) - finding.tags = context["form"].cleaned_data["tags"] - finding.unsaved_vulnerability_ids = context["form"].cleaned_data["vulnerability_ids"].split() - finding.save() - # Save and add new locations - finding_helper.add_locations(finding, context["form"]) - # Save the finding at the end and return - finding.save() - - return finding, request, True - add_error_message_to_response("The form has errors, please correct them below.") - add_field_errors_to_response(context["form"]) - - return finding, request, False - - def process_jira_form(self, request: HttpRequest, finding: Finding, context: dict): - # Capture case if the jira not being enabled - if context["jform"] is None: - return request, True, False - - if context["jform"] and context["jform"].is_valid(): - # can't use helper as when push_all_jira_issues is True, the checkbox gets disabled and is always false - # push_to_jira = jira_services.is_push_to_jira(finding, jform.cleaned_data.get('push_to_jira')) - push_to_jira = jira_services.is_push_all_issues(finding) or context["jform"].cleaned_data.get("push_to_jira") - jira_message = None - # if the jira issue key was changed, update database - new_jira_issue_key = context["jform"].cleaned_data.get("jira_issue") - if finding.has_jira_issue: - # everything in DD around JIRA integration is based on the internal id of the issue in JIRA - # instead of on the public jira issue key. - # I have no idea why, but it means we have to retrieve the issue from JIRA to get the internal JIRA id. - # we can assume the issue exist, which is already checked in the validation of the jform - if not new_jira_issue_key: - jira_services.unlink_finding(request, finding) - jira_message = "Link to JIRA issue removed successfully." - - elif new_jira_issue_key != finding.jira_issue.jira_key: - jira_services.unlink_finding(request, finding) - jira_services.link_finding(request, finding, new_jira_issue_key) - jira_message = "Changed JIRA link successfully." - else: - logger.debug("finding has no jira issue yet") - if new_jira_issue_key: - logger.debug("finding has no jira issue yet, but jira issue specified in request. trying to link.") - jira_services.link_finding(request, finding, new_jira_issue_key) - jira_message = "Linked a JIRA issue successfully." - # Determine if a message should be added - if jira_message: - messages.add_message( - request, messages.SUCCESS, jira_message, extra_tags="alert-success", - ) - - return request, True, push_to_jira - add_field_errors_to_response(context["jform"]) - - return request, False, False - - def process_forms(self, request: HttpRequest, test: Test, context: dict): - form_success_list = [] - finding = None - # Set vars for the completed forms - # Validate finding mitigation - request = self.validate_status_change(request, context) - # Check the validity of the form overall - finding, request, success = self.process_finding_form(request, test, context) - form_success_list.append(success) - request, success, push_to_jira = self.process_jira_form(request, finding, context) - form_success_list.append(success) - # Determine if all forms were successful - all_forms_valid = all(form_success_list) - # Check the validity of all the forms - if all_forms_valid: - # if we're removing the "duplicate" in the edit finding screen - finding_helper.save_vulnerability_ids(finding, context["form"].cleaned_data["vulnerability_ids"].split()) - # Push things to jira if needed - finding.save(push_to_jira=push_to_jira) - # Save the burp req resp - if "request" in context["form"].cleaned_data or "response" in context["form"].cleaned_data: - burp_rr = BurpRawRequestResponse( - finding=finding, - burpRequestBase64=base64.b64encode(context["form"].cleaned_data["request"].encode()), - burpResponseBase64=base64.b64encode(context["form"].cleaned_data["response"].encode()), - ) - burp_rr.clean() - burp_rr.save() - - # Note: this notification has not be moved to "@receiver(post_save, sender=Finding)" method as many other notifications - # Because it could generate too much noise, we keep it here only for findings created by hand in WebUI - - # Create a notification - create_notification( - event="finding_added", - title=_("Addition of %s") % finding.title, - finding=finding, - description=_('Finding "%s" was added by %s') % (finding.title, request.user), - url=reverse("view_finding", args=(finding.id,)), - icon="exclamation-triangle") - # Add a success message - messages.add_message( - request, - messages.SUCCESS, - _("Finding added successfully."), - extra_tags="alert-success") - - return finding, request, all_forms_valid - - def get_template(self): - return "dojo/add_findings.html" - - def get(self, request: HttpRequest, test_id: int): - # Get the initial objects - test = self.get_test(test_id) - # Make sure the user is authorized - user_has_permission_or_403(request.user, test, "add") - # Set up the initial context - context = self.get_initial_context(request, test) - # Render the form - return render(request, self.get_template(), context) - - def post(self, request: HttpRequest, test_id: int): - # Get the initial objects - test = self.get_test(test_id) - # Make sure the user is authorized - user_has_permission_or_403(request.user, test, "add") - # Set up the initial context - context = self.get_initial_context(request, test) - # Process the form - _, request, success = self.process_forms(request, test, context) - # Handle the case of a successful form - if success: - if "_Finished" in request.POST: - return HttpResponseRedirect(reverse("view_test", args=(test.id,))) - return HttpResponseRedirect(reverse("add_findings", args=(test.id,))) - context["form_error"] = True - # Render the form - return render(request, self.get_template(), context) - - -def add_finding_from_template(request, tid, fid): - jform = None - test = get_object_or_404(Test, id=tid) - user_has_permission_or_403(request.user, test, "add") - template = get_object_or_404(Finding_Template, id=fid) - findings = Finding_Template.objects.all() - push_all_jira_issues = jira_services.is_push_all_issues(template) - - if request.method == "POST": - - form = AddFindingForm(request.POST, req_resp=None, product=test.engagement.product) - if jira_services.get_project(test): - jform = JIRAFindingForm(push_all=jira_services.is_push_all_issues(test), prefix="jiraform", jira_project=jira_services.get_project(test), finding_form=form) - logger.debug(f"jform valid: {jform.is_valid()}") - - if (form["active"].value() is False or form["false_p"].value()) and form["duplicate"].value() is False: - closing_disabled = Note_Type.objects.filter(is_mandatory=True, is_active=True).count() - if closing_disabled != 0: - error_inactive = ValidationError( - _("Can not set a finding as inactive without adding all mandatory notes"), - code="not_active_or_false_p_true") - error_false_p = ValidationError( - _("Can not set a finding as false positive without adding all mandatory notes"), - code="not_active_or_false_p_true") - if form["active"].value() is False: - form.add_error("active", error_inactive) - if form["false_p"].value(): - form.add_error("false_p", error_false_p) - messages.add_message(request, - messages.ERROR, - _("Can not set a finding as inactive or false positive without adding all mandatory notes"), - extra_tags="alert-danger") - if form.is_valid(): - template.last_used = timezone.now() - template.save() - new_finding = form.save(commit=False) - new_finding.test = test - new_finding.reporter = request.user - new_finding.numerical_severity = Finding.get_numerical_severity( - new_finding.severity) - - new_finding.tags = form.cleaned_data["tags"] - new_finding.date = form.cleaned_data["date"] or datetime.today() - - finding_helper.update_finding_status(new_finding, request.user) - - new_finding.save(dedupe_option=False) - - # Copy all fields from template - finding_helper.copy_template_fields_to_finding( - finding=new_finding, - template=template, - user=request.user, - copy_vulnerability_ids=True, - copy_endpoints=True, - copy_notes=True, - ) - - # Save and add new locations from form (user may have added more) - finding_helper.add_locations(new_finding, form) - - new_finding.save() - if "jiraform-push_to_jira" in request.POST: - jform = JIRAFindingForm(request.POST, prefix="jiraform", instance=new_finding, push_all=push_all_jira_issues, jira_project=jira_services.get_project(test), finding_form=form) - if jform.is_valid(): - if jform.cleaned_data.get("push_to_jira"): - jira_services.push(new_finding) - else: - add_error_message_to_response(f"jira form validation failed: {jform.errors}") - if "request" in form.cleaned_data or "response" in form.cleaned_data: - burp_rr = BurpRawRequestResponse( - finding=new_finding, - burpRequestBase64=base64.b64encode(form.cleaned_data.get("request", "").encode("utf-8")), - burpResponseBase64=base64.b64encode(form.cleaned_data.get("response", "").encode("utf-8")), - ) - burp_rr.clean() - burp_rr.save() - messages.add_message(request, - messages.SUCCESS, - _("Finding from template added successfully."), - extra_tags="alert-success") - - return HttpResponseRedirect(reverse("view_test", args=(test.id,))) - messages.add_message(request, - messages.ERROR, - _("The form has errors, please correct them below."), - extra_tags="alert-danger") - - else: - # Build initial data with all template fields - initial_data = { - "active": False, - "date": timezone.now().date(), - "verified": False, - "false_p": False, - "duplicate": False, - "out_of_scope": False, - "title": template.title, - "description": template.description, - "cwe": template.cwe, - "severity": template.severity, - "mitigation": template.mitigation, - "impact": template.impact, - "references": template.references, - "numerical_severity": template.numerical_severity, - } - - # Add CVSS fields - if template.cvssv3: - initial_data["cvssv3"] = template.cvssv3 - if template.cvssv3_score is not None: - initial_data["cvssv3_score"] = template.cvssv3_score - if template.cvssv4: - initial_data["cvssv4"] = template.cvssv4 - if template.cvssv4_score is not None: - initial_data["cvssv4_score"] = template.cvssv4_score - - # Add remediation fields - if template.fix_available is not None: - initial_data["fix_available"] = template.fix_available - if template.fix_version: - initial_data["fix_version"] = template.fix_version - if template.planned_remediation_version: - initial_data["planned_remediation_version"] = template.planned_remediation_version - if template.effort_for_fixing: - initial_data["effort_for_fixing"] = template.effort_for_fixing - - # Add technical details fields - if template.steps_to_reproduce: - initial_data["steps_to_reproduce"] = template.steps_to_reproduce - if template.severity_justification: - initial_data["severity_justification"] = template.severity_justification - if template.component_name: - initial_data["component_name"] = template.component_name - if template.component_version: - initial_data["component_version"] = template.component_version - - # Add vulnerability IDs - if template.vulnerability_ids: - initial_data["vulnerability_ids"] = " ".join(template.vulnerability_ids) - - # Add endpoints to endpoints_to_add field - if template.endpoints: - endpoint_urls = template.endpoints if isinstance(template.endpoints, list) else template.endpoints.split("\n") - initial_data["endpoints_to_add"] = "\n".join([url.strip() for url in endpoint_urls if url.strip()]) - - form = AddFindingForm(req_resp=None, product=test.engagement.product, initial=initial_data) - - if jira_services.get_project(test): - jform = JIRAFindingForm(push_all=jira_services.is_push_all_issues(test), prefix="jiraform", jira_project=jira_services.get_project(test), finding_form=form) - - product_tab = Product_Tab(test.engagement.product, title=_("Add Finding"), tab="engagements") - product_tab.setEngagement(test.engagement) - return render(request, "dojo/add_findings.html", - {"form": form, - "product_tab": product_tab, - "jform": jform, - "findings": findings, - "temp": True, - "fid": template.id, - "tid": test.id, - "test": test, - }) - - -def search(request, tid): - test = get_object_or_404(Test, id=tid) - templates = Finding_Template.objects.all() - templates = TemplateFindingFilter(request.GET, queryset=templates) - paged_templates = get_page_items(request, templates.qs, 25) - - title_words = get_words_for_field(Finding_Template, "title") - - add_breadcrumb(parent=test, title=_("Add From Template"), top_level=False, request=request) - return render(request, "dojo/templates.html", - {"templates": paged_templates, - "filtered": templates, - "title_words": title_words, - "tid": tid, - "add_from_template": True, - }) - - -class ReImportScanResultsView(View): - def get_template(self) -> str: - """Returns the template that will be presented to the user""" - return "dojo/import_scan_results.html" - - def get_form( - self, - request: HttpRequest, - test: Test, - **kwargs: dict, - ) -> ReImportScanForm: - """Returns the default import form for importing findings""" - if request.method == "POST": - return ReImportScanForm(request.POST, request.FILES, test=test, **kwargs) - return ReImportScanForm(test=test, **kwargs) - - def get_jira_form( - self, - request: HttpRequest, - test: Test, - ) -> tuple[JIRAImportScanForm | None, bool]: - """Returns a JiraImportScanForm if jira is enabled""" - jira_form = None - push_all_jira_issues = False - # Decide if we need to present the Push to JIRA form - if get_system_setting("enable_jira"): - # Determine if jira issues should be pushed automatically - push_all_jira_issues = jira_services.is_push_all_issues(test) - # Only return the form if the jira is enabled on this engagement or product - if jira_services.get_project(test): - if request.method == "POST": - jira_form = JIRAImportScanForm( - request.POST, - push_all=push_all_jira_issues, - prefix="jiraform", - ) - else: - jira_form = JIRAImportScanForm( - push_all=push_all_jira_issues, - prefix="jiraform", - ) - return jira_form, push_all_jira_issues - - def handle_request( - self, - request: HttpRequest, - test_id: int, - ) -> tuple[HttpRequest, dict]: - """ - Process the common behaviors between request types, and then return - the request and context dict back to be rendered - """ - # Get the test object - test = get_object_or_404(Test, id=test_id) - # Ensure the supplied user has access to import to the engagement or product - user_has_permission_or_403(request.user, test, "import") - # by default we keep a trace of the scan_type used to create the test - # if it's not here, we use the "name" of the test type - # this feature exists to provide custom label for tests for some parsers - scan_type = test.scan_type or test.test_type.name - # Set the product tab - product_tab = Product_Tab(test.engagement.product, title=_("Re-upload a %s") % scan_type, tab="engagements") - product_tab.setEngagement(test.engagement) - # Get the import form with some initial data in place - if settings.V3_FEATURE_LOCATIONS: - endpoints = Location.objects.filter(products__product__id=product_tab.product.id) - else: - # TODO: Delete this after the move to Locations - endpoints = Endpoint.objects.filter(product__id=product_tab.product.id) - form = self.get_form( - request, - test, - endpoints=endpoints, - api_scan_configuration=test.api_scan_configuration, - api_scan_configuration_queryset=Product_API_Scan_Configuration.objects.filter(product__id=product_tab.product.id), - ) - # Get the jira form - jira_form, push_all_jira_issues = self.get_jira_form(request, test) - # Return the request and the context - return request, { - "test": test, - "form": form, - "product_tab": product_tab, - "eid": test.engagement.id, - "jform": jira_form, - "scan_type": scan_type, - "scan_types": get_scan_types_sorted(), - "push_all_jira_issues": push_all_jira_issues, - "additional_message": ( - "When re-uploading a scan, any findings not found in original scan will be updated as " - "mitigated. The process attempts to identify the differences, however manual verification " - "is highly recommended." - ), - } - - def validate_forms( - self, - context: dict, - ) -> bool: - """ - Validates each of the forms to ensure all errors from the form - level are bubbled up to the user first before we process too much - """ - form_validation_list = [] - for form_name in ["form", "jform"]: - if (form := context.get(form_name)) is not None: - if errors := form.errors: - form_validation_list.append(errors) - return form_validation_list - - def process_form( - self, - request: HttpRequest, - form: ReImportScanForm, - context: dict, - ) -> str | None: - """Process the form and manipulate the input in any way that is appropriate""" - # Update the running context dict with cleaned form input - context.update({ - "scan": request.FILES.get("file", None), - "scan_date": form.cleaned_data.get("scan_date"), - "minimum_severity": form.cleaned_data.get("minimum_severity"), - "do_not_reactivate": form.cleaned_data.get("do_not_reactivate"), - "tags": form.cleaned_data.get("tags"), - "version": form.cleaned_data.get("version") or None, - "branch_tag": form.cleaned_data.get("branch_tag") or None, - "build_id": form.cleaned_data.get("build_id") or None, - "commit_hash": form.cleaned_data.get("commit_hash") or None, - "api_scan_configuration": form.cleaned_data.get("api_scan_configuration") or None, - "service": form.cleaned_data.get("service") or None, - "apply_tags_to_findings": form.cleaned_data.get("apply_tags_to_findings", False), - "apply_tags_to_endpoints": form.cleaned_data.get("apply_tags_to_endpoints", False), - "group_by": form.cleaned_data.get("group_by") or None, - "close_old_findings": form.cleaned_data.get("close_old_findings", None), - "create_finding_groups_for_all_findings": form.cleaned_data.get("create_finding_groups_for_all_findings", None), - }) - # Override the form values of active and verified - if activeChoice := form.cleaned_data.get("active", None): - if activeChoice == "force_to_true": - context["active"] = True - elif activeChoice == "force_to_false": - context["active"] = False - if verifiedChoice := form.cleaned_data.get("verified", None): - if verifiedChoice == "force_to_true": - context["verified"] = True - elif verifiedChoice == "force_to_false": - context["verified"] = False - # Override the tags and version - context.get("test").tags = context.get("tags") - context.get("test").version = context.get("version") - return None - - def process_jira_form( - self, - request: HttpRequest, - form: JIRAImportScanForm, - context: dict, - ) -> str | None: - """ - Process the jira form by first making sure one was supplied - and then setting any values supplied by the user. An error - may be returned and will be bubbled up in the form of a message - """ - # Determine if push all issues is enabled - push_all_jira_issues = context.get("push_all_jira_issues", False) - context["push_to_jira"] = push_all_jira_issues or (form and form.cleaned_data.get("push_to_jira")) - return None - - def get_reimporter( - self, - context: dict, - ) -> BaseImporter: - """Gets the reimporter to use""" - return DefaultReImporter(**context) - - def reimport_findings( - self, - context: dict, - ) -> str | None: - """Attempt to import with all the supplied information""" - try: - # Log only user-entered form values, excluding internal objects - user_values = { - "scan_type": context.get("scan_type"), - "scan_date": context.get("scan_date"), - "minimum_severity": context.get("minimum_severity"), - "active": context.get("active"), - "verified": context.get("verified"), - "tags": context.get("tags"), - "version": context.get("version"), - "branch_tag": context.get("branch_tag"), - "build_id": context.get("build_id"), - "commit_hash": context.get("commit_hash"), - "service": context.get("service"), - "close_old_findings": context.get("close_old_findings"), - "apply_tags_to_findings": context.get("apply_tags_to_findings"), - "apply_tags_to_endpoints": context.get("apply_tags_to_endpoints"), - "close_old_findings_product_scope": context.get("close_old_findings_product_scope"), - "group_by": context.get("group_by"), - "create_finding_groups_for_all_findings": context.get("create_finding_groups_for_all_findings"), - "push_to_jira": context.get("push_to_jira"), - "push_all_jira_issues": context.get("push_all_jira_issues"), - "do_not_reactivate": context.get("do_not_reactivate"), - } - logger.debug(f"reimport_findings called with user values: {user_values}") - importer_client = self.get_reimporter(context) - ( - context["test"], - finding_count, - new_finding_count, - closed_finding_count, - reactivated_finding_count, - untouched_finding_count, - _, - ) = importer_client.process_scan( - context.pop("scan", None), - ) - # Add a message to the view for the user to see the results - add_success_message_to_response(importer_client.construct_imported_message( - finding_count=finding_count, - new_finding_count=new_finding_count, - closed_finding_count=closed_finding_count, - reactivated_finding_count=reactivated_finding_count, - untouched_finding_count=untouched_finding_count, - )) - except Exception as e: - logger.exception("An exception error occurred during the report import") - return f"An exception error occurred during the report import: {e}" - return None - - def success_redirect( - self, - request: HttpRequest, - context: dict, - ) -> HttpResponseRedirect: - """Redirect the user to a place that indicates a successful import""" - duration = time.perf_counter() - request._start_time - LargeScanSizeProductAnnouncement(request=request, duration=duration) - ScanTypeProductAnnouncement(request=request, scan_type=context.get("scan_type")) - return HttpResponseRedirect(reverse("view_test", args=(context.get("test").id, ))) - - def failure_redirect( - self, - request: HttpRequest, - context: dict, - ) -> HttpResponseRedirect: - """Redirect the user to a place that indicates a failed import""" - ErrorPageProductAnnouncement(request=request) - return HttpResponseRedirect(reverse( - "re_import_scan_results", - args=(context.get("test").id, ), - )) - - def get( - self, - request: HttpRequest, - test_id: int, - ) -> HttpResponse: - """Process GET requests for the ReImport View""" - # process the request and path parameters - request, context = self.handle_request( - request, - test_id=test_id, - ) - # Render the form - return render(request, self.get_template(), context) - - def post( - self, - request: HttpRequest, - test_id: int, - ) -> HttpResponse: - """Process POST requests for the ReImport View""" - # process the request and path parameters - request, context = self.handle_request( - request, - test_id=test_id, - ) - request._start_time = time.perf_counter() - # ensure all three forms are valid first before moving forward - if form_errors := self.validate_forms(context): - for form_error in form_errors: - add_error_message_to_response(form_error) - return self.failure_redirect(request, context) - # Process the jira form if it is present - if form_error := self.process_jira_form(request, context.get("jform"), context): - add_error_message_to_response(form_error) - return self.failure_redirect(request, context) - # Process the import form - if form_error := self.process_form(request, context.get("form"), context): - add_error_message_to_response(form_error) - return self.failure_redirect(request, context) - # Add pghistory context for audit trail (adds to existing middleware context) - pghistory.context( - source="reimport", - test_id=context.get("test").id, - scan_type=context.get("scan_type"), - ) - # Kick off the import process - if import_error := self.reimport_findings(context): - add_error_message_to_response(import_error) - return self.failure_redirect(request, context) - # Otherwise return the user back to the engagement (if present) or the product - return self.success_redirect(request, context) +# Backward-compat shim: the view logic moved to dojo.test.ui.views during the +# module reorg. External consumers (e.g. dojo-pro) still import from +# dojo.test.views, so re-export the public names from their new location. +from dojo.test.ui.views import * # noqa: F403 -- backward compat re-export diff --git a/dojo/test_type/views.py b/dojo/test_type/views.py index 5a25e9ed00a..c025761ba76 100644 --- a/dojo/test_type/views.py +++ b/dojo/test_type/views.py @@ -7,9 +7,9 @@ from django.shortcuts import get_object_or_404, render from django.urls import reverse -from dojo.filters import TestTypeFilter from dojo.forms import Test_TypeForm from dojo.models import Test_Type +from dojo.test.ui.filters import TestTypeFilter from dojo.utils import add_breadcrumb, get_page_items logger = logging.getLogger(__name__) diff --git a/dojo/tool_config/__init__.py b/dojo/tool_config/__init__.py index e69de29bb2d..3f63d29bec4 100644 --- a/dojo/tool_config/__init__.py +++ b/dojo/tool_config/__init__.py @@ -0,0 +1 @@ +import dojo.tool_config.admin # noqa: F401 diff --git a/dojo/tool_config/admin.py b/dojo/tool_config/admin.py new file mode 100644 index 00000000000..cb07719c546 --- /dev/null +++ b/dojo/tool_config/admin.py @@ -0,0 +1,40 @@ +from django import forms +from django.contrib import admin + +from dojo.tool_config.models import Tool_Configuration + + +# declare form here as we can't import forms.py due to circular imports not even locally +class ToolConfigForm_Admin(forms.ModelForm): + password = forms.CharField(widget=forms.PasswordInput, required=False) + api_key = forms.CharField(widget=forms.PasswordInput, required=False) + ssh = forms.CharField(widget=forms.PasswordInput, required=False) + + # django doesn't seem to have an easy way to handle password fields as PasswordInput requires reentry of passwords + password_from_db = None + ssh_from_db = None + api_key_from_db = None + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + if self.instance: + # keep password from db to use if the user entered no password + self.password_from_db = self.instance.password + self.ssh_from_db = self.instance.ssh + self.api_key = self.instance.api_key + + def clean(self): + cleaned_data = super().clean() + if not cleaned_data["password"] and not cleaned_data["ssh"] and not cleaned_data["api_key"]: + cleaned_data["password"] = self.password_from_db + cleaned_data["ssh"] = self.ssh_from_db + cleaned_data["api_key"] = self.api_key_from_db + + return cleaned_data + + +class Tool_Configuration_Admin(admin.ModelAdmin): + form = ToolConfigForm_Admin + + +admin.site.register(Tool_Configuration, Tool_Configuration_Admin) diff --git a/dojo/tool_config/api/__init__.py b/dojo/tool_config/api/__init__.py new file mode 100644 index 00000000000..3f12ce59e94 --- /dev/null +++ b/dojo/tool_config/api/__init__.py @@ -0,0 +1 @@ +path = "tool_configurations" # noqa: RUF067 diff --git a/dojo/tool_config/api/serializer.py b/dojo/tool_config/api/serializer.py new file mode 100644 index 00000000000..f80dc1b782d --- /dev/null +++ b/dojo/tool_config/api/serializer.py @@ -0,0 +1,14 @@ +from rest_framework import serializers + +from dojo.tool_config.models import Tool_Configuration + + +class ToolConfigurationSerializer(serializers.ModelSerializer): + class Meta: + model = Tool_Configuration + fields = "__all__" + extra_kwargs = { + "password": {"write_only": True}, + "ssh": {"write_only": True}, + "api_key": {"write_only": True}, + } diff --git a/dojo/tool_config/api/urls.py b/dojo/tool_config/api/urls.py new file mode 100644 index 00000000000..cc4620473a7 --- /dev/null +++ b/dojo/tool_config/api/urls.py @@ -0,0 +1,6 @@ +from dojo.tool_config.api.views import ToolConfigurationsViewSet + + +def add_tool_config_urls(router): + router.register(r"tool_configurations", ToolConfigurationsViewSet, basename="tool_configuration") + return router diff --git a/dojo/tool_config/api/views.py b/dojo/tool_config/api/views.py new file mode 100644 index 00000000000..91a740610b7 --- /dev/null +++ b/dojo/tool_config/api/views.py @@ -0,0 +1,32 @@ +import logging + +from django_filters.rest_framework import DjangoFilterBackend +from drf_spectacular.utils import extend_schema_view + +from dojo.api_v2.views import PrefetchDojoModelViewSet, schema_with_prefetch +from dojo.authorization import api_permissions as permissions +from dojo.tool_config.api.serializer import ToolConfigurationSerializer +from dojo.tool_config.models import Tool_Configuration + +logger = logging.getLogger(__name__) + + +# Authorization: configurations +@extend_schema_view(**schema_with_prefetch()) +class ToolConfigurationsViewSet( + PrefetchDojoModelViewSet, +): + serializer_class = ToolConfigurationSerializer + queryset = Tool_Configuration.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = [ + "id", + "name", + "tool_type", + "url", + "authentication_type", + ] + permission_classes = (permissions.UserHasConfigurationPermissionSuperuser,) + + def get_queryset(self): + return Tool_Configuration.objects.all().order_by("id") diff --git a/dojo/tool_config/models.py b/dojo/tool_config/models.py new file mode 100644 index 00000000000..6190fe839ce --- /dev/null +++ b/dojo/tool_config/models.py @@ -0,0 +1,31 @@ +from django.db import models +from django.utils.translation import gettext_lazy as _ + + +class Tool_Configuration(models.Model): + name = models.CharField(max_length=200, null=False) + description = models.CharField(max_length=2000, null=True, blank=True) + url = models.CharField(max_length=2000, null=True, blank=True) + tool_type = models.ForeignKey("dojo.Tool_Type", related_name="tool_type", on_delete=models.CASCADE) + authentication_type = models.CharField(max_length=15, + choices=( + ("API", "API Key"), + ("Password", + "Username/Password"), + ("SSH", "SSH")), + null=True, blank=True) + extras = models.CharField(max_length=255, null=True, blank=True, help_text=_("Additional definitions that will be " + "consumed by scanner")) + username = models.CharField(max_length=200, null=True, blank=True) + password = models.CharField(max_length=600, null=True, blank=True) + auth_title = models.CharField(max_length=200, null=True, blank=True, + verbose_name=_("Title for SSH/API Key")) + ssh = models.CharField(max_length=6000, null=True, blank=True) + api_key = models.CharField(max_length=600, null=True, blank=True, + verbose_name=_("API Key")) + + class Meta: + ordering = ["name"] + + def __str__(self): + return self.name diff --git a/dojo/tool_config/ui/__init__.py b/dojo/tool_config/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/tool_config/ui/forms.py b/dojo/tool_config/ui/forms.py new file mode 100644 index 00000000000..cc769725d78 --- /dev/null +++ b/dojo/tool_config/ui/forms.py @@ -0,0 +1,27 @@ +from django import forms +from django.core.validators import URLValidator + +from dojo.tool_config.models import Tool_Configuration +from dojo.tool_type.models import Tool_Type + + +class ToolConfigForm(forms.ModelForm): + tool_type = forms.ModelChoiceField(queryset=Tool_Type.objects.all(), label="Tool Type") + ssh = forms.CharField(widget=forms.Textarea(attrs={}), required=False, label="SSH Key") + + class Meta: + model = Tool_Configuration + exclude = ["product"] + + def clean(self): + form_data = self.cleaned_data + + try: + if form_data["url"] is not None: + url_validator = URLValidator(schemes=["ssh", "http", "https"]) + url_validator(form_data["url"]) + except forms.ValidationError: + msg = "It does not appear as though this endpoint is a valid URL/SSH or IP address." + raise forms.ValidationError(msg, code="invalid") + + return form_data diff --git a/dojo/tool_config/urls.py b/dojo/tool_config/ui/urls.py similarity index 89% rename from dojo/tool_config/urls.py rename to dojo/tool_config/ui/urls.py index 263142742e6..c2d5862e460 100644 --- a/dojo/tool_config/urls.py +++ b/dojo/tool_config/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from . import views +from dojo.tool_config.ui import views urlpatterns = [ re_path(r"^tool_config/add", views.new_tool_config, name="add_tool_config"), diff --git a/dojo/tool_config/views.py b/dojo/tool_config/ui/views.py similarity index 97% rename from dojo/tool_config/views.py rename to dojo/tool_config/ui/views.py index cb32d3203cf..a0807917925 100644 --- a/dojo/tool_config/views.py +++ b/dojo/tool_config/ui/views.py @@ -6,9 +6,9 @@ from django.shortcuts import render from django.urls import reverse -from dojo.forms import ToolConfigForm -from dojo.models import Tool_Configuration from dojo.tool_config.factory import create_API +from dojo.tool_config.models import Tool_Configuration +from dojo.tool_config.ui.forms import ToolConfigForm from dojo.utils import add_breadcrumb, dojo_crypto_encrypt, prepare_for_view logger = logging.getLogger(__name__) diff --git a/dojo/tool_product/__init__.py b/dojo/tool_product/__init__.py index e69de29bb2d..f145962c51d 100644 --- a/dojo/tool_product/__init__.py +++ b/dojo/tool_product/__init__.py @@ -0,0 +1 @@ +import dojo.tool_product.admin # noqa: F401 diff --git a/dojo/tool_product/admin.py b/dojo/tool_product/admin.py new file mode 100644 index 00000000000..19d8890eff1 --- /dev/null +++ b/dojo/tool_product/admin.py @@ -0,0 +1,6 @@ +from django.contrib import admin + +from dojo.tool_product.models import Tool_Product_History, Tool_Product_Settings + +admin.site.register(Tool_Product_Settings) +admin.site.register(Tool_Product_History) diff --git a/dojo/tool_product/api/__init__.py b/dojo/tool_product/api/__init__.py new file mode 100644 index 00000000000..1a68ec15513 --- /dev/null +++ b/dojo/tool_product/api/__init__.py @@ -0,0 +1 @@ +path = "tool_product_settings" # noqa: RUF067 diff --git a/dojo/tool_product/api/serializer.py b/dojo/tool_product/api/serializer.py new file mode 100644 index 00000000000..12aafcdfec7 --- /dev/null +++ b/dojo/tool_product/api/serializer.py @@ -0,0 +1,15 @@ +from rest_framework import serializers + +from dojo.models import Product +from dojo.tool_product.models import Tool_Product_Settings + + +class ToolProductSettingsSerializer(serializers.ModelSerializer): + setting_url = serializers.CharField(source="url") + product = serializers.PrimaryKeyRelatedField( + queryset=Product.objects.all(), required=True, + ) + + class Meta: + model = Tool_Product_Settings + fields = "__all__" diff --git a/dojo/tool_product/api/urls.py b/dojo/tool_product/api/urls.py new file mode 100644 index 00000000000..f3fca7fa2f2 --- /dev/null +++ b/dojo/tool_product/api/urls.py @@ -0,0 +1,6 @@ +from dojo.tool_product.api.views import ToolProductSettingsViewSet + + +def add_tool_product_urls(router): + router.register(r"tool_product_settings", ToolProductSettingsViewSet, basename="tool_product_settings") + return router diff --git a/dojo/tool_product/api/views.py b/dojo/tool_product/api/views.py new file mode 100644 index 00000000000..33a6278841d --- /dev/null +++ b/dojo/tool_product/api/views.py @@ -0,0 +1,38 @@ +import logging + +from django_filters.rest_framework import DjangoFilterBackend +from drf_spectacular.utils import extend_schema_view +from rest_framework.permissions import IsAuthenticated + +from dojo.api_v2.views import PrefetchDojoModelViewSet, schema_with_prefetch +from dojo.authorization import api_permissions as permissions +from dojo.tool_product.api.serializer import ToolProductSettingsSerializer +from dojo.tool_product.models import Tool_Product_Settings +from dojo.tool_product.queries import get_authorized_tool_product_settings + +logger = logging.getLogger(__name__) + + +# Authorization: object-based +@extend_schema_view(**schema_with_prefetch()) +class ToolProductSettingsViewSet( + PrefetchDojoModelViewSet, +): + serializer_class = ToolProductSettingsSerializer + queryset = Tool_Product_Settings.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = [ + "id", + "name", + "product", + "tool_configuration", + "tool_project_id", + "url", + ] + permission_classes = ( + IsAuthenticated, + permissions.UserHasToolProductSettingsPermission, + ) + + def get_queryset(self): + return get_authorized_tool_product_settings("view") diff --git a/dojo/tool_product/models.py b/dojo/tool_product/models.py new file mode 100644 index 00000000000..2f77ea89a6e --- /dev/null +++ b/dojo/tool_product/models.py @@ -0,0 +1,25 @@ +from django.db import models +from django.utils.timezone import now +from django.utils.translation import gettext_lazy as _ + + +class Tool_Product_Settings(models.Model): + name = models.CharField(max_length=200, null=False) + description = models.CharField(max_length=2000, null=True, blank=True) + url = models.CharField(max_length=2000, null=True, blank=True) + product = models.ForeignKey("dojo.Product", default=1, editable=False, on_delete=models.CASCADE) + tool_configuration = models.ForeignKey("dojo.Tool_Configuration", null=False, + related_name="tool_configuration", on_delete=models.CASCADE) + tool_project_id = models.CharField(max_length=200, null=True, blank=True) + notes = models.ManyToManyField("dojo.Notes", blank=True, editable=False) + + class Meta: + ordering = ["name"] + + +class Tool_Product_History(models.Model): + product = models.ForeignKey("dojo.Tool_Product_Settings", editable=False, on_delete=models.CASCADE) + last_scan = models.DateTimeField(null=False, editable=False, default=now) + succesfull = models.BooleanField(default=True, verbose_name=_("Succesfully")) + configuration_details = models.CharField(max_length=2000, null=True, + blank=True) diff --git a/dojo/tool_product/queries.py b/dojo/tool_product/queries.py index 6ae66429fdc..45dd338b5b3 100644 --- a/dojo/tool_product/queries.py +++ b/dojo/tool_product/queries.py @@ -3,8 +3,8 @@ except ImportError: def get_auth_filter(key): return None -from dojo.models import Tool_Product_Settings from dojo.request_cache import cache_for_request +from dojo.tool_product.models import Tool_Product_Settings # Cached: all parameters are hashable, no dynamic queryset filtering diff --git a/dojo/tool_product/signals.py b/dojo/tool_product/signals.py index 96dd881ff45..391189cfb26 100644 --- a/dojo/tool_product/signals.py +++ b/dojo/tool_product/signals.py @@ -3,8 +3,8 @@ from django.db.models.signals import pre_delete from django.dispatch import receiver -from dojo.models import Tool_Product_Settings from dojo.notes.helper import delete_related_notes +from dojo.tool_product.models import Tool_Product_Settings logger = logging.getLogger(__name__) diff --git a/dojo/tool_product/ui/__init__.py b/dojo/tool_product/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/tool_product/ui/forms.py b/dojo/tool_product/ui/forms.py new file mode 100644 index 00000000000..f62dac4bb9c --- /dev/null +++ b/dojo/tool_product/ui/forms.py @@ -0,0 +1,37 @@ +from django import forms +from django.core.validators import URLValidator + +from dojo.tool_config.models import Tool_Configuration +from dojo.tool_product.models import Tool_Product_Settings + + +class DeleteToolProductSettingsForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = Tool_Product_Settings + fields = ["id"] + + +class ToolProductSettingsForm(forms.ModelForm): + tool_configuration = forms.ModelChoiceField(queryset=Tool_Configuration.objects.all(), label="Tool Configuration") + + class Meta: + model = Tool_Product_Settings + fields = ["name", "description", "url", "tool_configuration", "tool_project_id"] + exclude = ["tool_type"] + order = ["name"] + + def clean(self): + form_data = self.cleaned_data + + try: + if form_data["url"] is not None: + url_validator = URLValidator(schemes=["ssh", "http", "https"]) + url_validator(form_data["url"]) + except forms.ValidationError: + msg = "It does not appear as though this endpoint is a valid URL/SSH or IP address." + raise forms.ValidationError(msg, code="invalid") + + return form_data diff --git a/dojo/tool_product/urls.py b/dojo/tool_product/ui/urls.py similarity index 93% rename from dojo/tool_product/urls.py rename to dojo/tool_product/ui/urls.py index 9acc6cdb139..943647d5d49 100644 --- a/dojo/tool_product/urls.py +++ b/dojo/tool_product/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from . import views +from dojo.tool_product.ui import views urlpatterns = [ re_path(r"^product/(?P\d+)/tool_product/add$", views.new_tool_product, name="new_tool_product"), diff --git a/dojo/tool_product/views.py b/dojo/tool_product/ui/views.py similarity index 95% rename from dojo/tool_product/views.py rename to dojo/tool_product/ui/views.py index de142b1bcf8..39afab79e28 100644 --- a/dojo/tool_product/views.py +++ b/dojo/tool_product/ui/views.py @@ -8,8 +8,9 @@ from django.urls import reverse from django.utils.translation import gettext as _ -from dojo.forms import DeleteToolProductSettingsForm, ToolProductSettingsForm -from dojo.models import Product, Tool_Product_Settings +from dojo.models import Product +from dojo.tool_product.models import Tool_Product_Settings +from dojo.tool_product.ui.forms import DeleteToolProductSettingsForm, ToolProductSettingsForm from dojo.utils import Product_Tab logger = logging.getLogger(__name__) diff --git a/dojo/tool_type/__init__.py b/dojo/tool_type/__init__.py index e69de29bb2d..0235ecd395e 100644 --- a/dojo/tool_type/__init__.py +++ b/dojo/tool_type/__init__.py @@ -0,0 +1 @@ +import dojo.tool_type.admin # noqa: F401 diff --git a/dojo/tool_type/admin.py b/dojo/tool_type/admin.py new file mode 100644 index 00000000000..2196308ff07 --- /dev/null +++ b/dojo/tool_type/admin.py @@ -0,0 +1,5 @@ +from django.contrib import admin + +from dojo.tool_type.models import Tool_Type + +admin.site.register(Tool_Type) diff --git a/dojo/tool_type/api/__init__.py b/dojo/tool_type/api/__init__.py new file mode 100644 index 00000000000..7af3572bf6b --- /dev/null +++ b/dojo/tool_type/api/__init__.py @@ -0,0 +1 @@ +path = "tool_types" # noqa: RUF067 diff --git a/dojo/tool_type/api/serializer.py b/dojo/tool_type/api/serializer.py new file mode 100644 index 00000000000..5d056dcb8fc --- /dev/null +++ b/dojo/tool_type/api/serializer.py @@ -0,0 +1,18 @@ +from rest_framework import serializers + +from dojo.tool_type.models import Tool_Type + + +class ToolTypeSerializer(serializers.ModelSerializer): + class Meta: + model = Tool_Type + fields = "__all__" + + def validate(self, data): + if self.context["request"].method == "POST": + name = data.get("name") + # Make sure this will not create a duplicate test type + if Tool_Type.objects.filter(name=name).count() > 0: + msg = "A Tool Type with the name already exists" + raise serializers.ValidationError(msg) + return data diff --git a/dojo/tool_type/api/urls.py b/dojo/tool_type/api/urls.py new file mode 100644 index 00000000000..044c34fe6cc --- /dev/null +++ b/dojo/tool_type/api/urls.py @@ -0,0 +1,6 @@ +from dojo.tool_type.api.views import ToolTypesViewSet + + +def add_tool_type_urls(router): + router.register(r"tool_types", ToolTypesViewSet, basename="tool_type") + return router diff --git a/dojo/tool_type/api/views.py b/dojo/tool_type/api/views.py new file mode 100644 index 00000000000..b84af213ad9 --- /dev/null +++ b/dojo/tool_type/api/views.py @@ -0,0 +1,24 @@ +import logging + +from django_filters.rest_framework import DjangoFilterBackend + +from dojo.api_v2.views import DojoModelViewSet +from dojo.authorization import api_permissions as permissions +from dojo.tool_type.api.serializer import ToolTypeSerializer +from dojo.tool_type.models import Tool_Type + +logger = logging.getLogger(__name__) + + +# Authorization: configuration +class ToolTypesViewSet( + DojoModelViewSet, +): + serializer_class = ToolTypeSerializer + queryset = Tool_Type.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = ["id", "name", "description"] + permission_classes = (permissions.UserHasConfigurationPermissionSuperuser,) + + def get_queryset(self): + return Tool_Type.objects.all().order_by("id") diff --git a/dojo/tool_type/models.py b/dojo/tool_type/models.py new file mode 100644 index 00000000000..a5c55213d32 --- /dev/null +++ b/dojo/tool_type/models.py @@ -0,0 +1,12 @@ +from django.db import models + + +class Tool_Type(models.Model): + name = models.CharField(max_length=200) + description = models.CharField(max_length=2000, null=True, blank=True) + + class Meta: + ordering = ["name"] + + def __str__(self): + return self.name diff --git a/dojo/tool_type/ui/__init__.py b/dojo/tool_type/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/tool_type/ui/forms.py b/dojo/tool_type/ui/forms.py new file mode 100644 index 00000000000..8e7ff33f90f --- /dev/null +++ b/dojo/tool_type/ui/forms.py @@ -0,0 +1,27 @@ +from django import forms + +from dojo.tool_type.models import Tool_Type + + +class ToolTypeForm(forms.ModelForm): + class Meta: + model = Tool_Type + exclude = ["product"] + + def __init__(self, *args, **kwargs): + instance = kwargs.get("instance") + self.newly_created = True + if instance is not None: + self.newly_created = instance.pk is None + super().__init__(*args, **kwargs) + + def clean(self): + form_data = self.cleaned_data + if self.newly_created: + name = form_data.get("name") + # Make sure this will not create a duplicate test type + if Tool_Type.objects.filter(name=name).count() > 0: + msg = "A Tool Type with the name already exists" + raise forms.ValidationError(msg) + + return form_data diff --git a/dojo/tool_type/urls.py b/dojo/tool_type/ui/urls.py similarity index 89% rename from dojo/tool_type/urls.py rename to dojo/tool_type/ui/urls.py index 3b79b58d1b5..68d82c15be4 100644 --- a/dojo/tool_type/urls.py +++ b/dojo/tool_type/ui/urls.py @@ -1,6 +1,6 @@ from django.urls import re_path -from . import views +from dojo.tool_type.ui import views urlpatterns = [ re_path(r"^tool_type/add", views.new_tool_type, name="add_tool_type"), diff --git a/dojo/tool_type/views.py b/dojo/tool_type/ui/views.py similarity index 95% rename from dojo/tool_type/views.py rename to dojo/tool_type/ui/views.py index 3f9e8218136..551ed7c0f4e 100644 --- a/dojo/tool_type/views.py +++ b/dojo/tool_type/ui/views.py @@ -7,8 +7,8 @@ from django.urls import reverse from django.utils.translation import gettext as _ -from dojo.forms import ToolTypeForm -from dojo.models import Tool_Type +from dojo.tool_type.models import Tool_Type +from dojo.tool_type.ui.forms import ToolTypeForm from dojo.utils import add_breadcrumb logger = logging.getLogger(__name__) diff --git a/dojo/url/ui/views.py b/dojo/url/ui/views.py index 13eb6521286..3d9845fb561 100644 --- a/dojo/url/ui/views.py +++ b/dojo/url/ui/views.py @@ -24,7 +24,7 @@ from dojo.location.queries import annotate_location_counts_and_status, get_authorized_locations from dojo.location.status import FindingLocationStatus, ProductLocationStatus from dojo.models import DojoMeta, Finding, Product -from dojo.reports.views import generate_report +from dojo.reports.ui.views import generate_report from dojo.url.filters import URLFilter from dojo.url.models import URL from dojo.url.queries import annotate_host_contents diff --git a/dojo/urls.py b/dojo/urls.py index 9b9a8d6a399..063d65ef220 100644 --- a/dojo/urls.py +++ b/dojo/urls.py @@ -10,22 +10,13 @@ from rest_framework.routers import DefaultRouter from dojo import views -from dojo.announcement.urls import urlpatterns as announcement_urls +from dojo.announcement.api.urls import add_announcement_urls +from dojo.announcement.ui.urls import urlpatterns as announcement_urls from dojo.api_v2.views import ( - AnnouncementViewSet, AppAnalysisViewSet, - BurpRawRequestResponseViewSet, CeleryViewSet, ConfigurationPermissionViewSet, - DevelopmentEnvironmentViewSet, DojoMetaViewSet, - EndpointMetaImporterView, - EndpointStatusViewSet, - EndPointViewSet, - EngagementPresetsViewset, - EngagementViewSet, - FindingTemplatesViewSet, - FindingViewSet, ImportLanguagesView, ImportScanView, JiraInstanceViewSet, @@ -34,38 +25,25 @@ LanguageTypeViewSet, LanguageViewSet, NetworkLocationsViewset, - NotesViewSet, - NoteTypeViewSet, - ProductAPIScanConfigurationViewSet, - ProductTypeViewSet, - ProductViewSet, - RegulationsViewSet, ReImportScanView, - RiskAcceptanceViewSet, SLAConfigurationViewset, SonarqubeIssueTransitionViewSet, SonarqubeIssueViewSet, - SystemSettingsViewSet, - TestImportViewSet, - TestsViewSet, - TestTypesViewSet, - ToolConfigurationsViewSet, - ToolProductSettingsViewSet, - ToolTypesViewSet, - UserContactInfoViewSet, - UserProfileView, - UsersViewSet, ) from dojo.api_v2.views import DojoSpectacularAPIView as SpectacularAPIView from dojo.asset.api.urls import add_asset_urls from dojo.asset.urls import urlpatterns as asset_urls -from dojo.banner.urls import urlpatterns as banner_urls -from dojo.benchmark.urls import urlpatterns as benchmark_urls +from dojo.banner.ui.urls import urlpatterns as banner_urls +from dojo.benchmark.ui.urls import urlpatterns as benchmark_urls from dojo.components.urls import urlpatterns as component_urls -from dojo.development_environment.urls import urlpatterns as dev_env_urls -from dojo.endpoint.urls import urlpatterns as endpoint_urls -from dojo.engagement.urls import urlpatterns as eng_urls -from dojo.finding.urls import urlpatterns as finding_urls +from dojo.development_environment.api.urls import add_development_environment_urls +from dojo.development_environment.ui.urls import urlpatterns as dev_env_urls +from dojo.endpoint.api.urls import add_endpoint_urls, register_endpoint_meta_import +from dojo.endpoint.ui.urls import urlpatterns as endpoint_urls +from dojo.engagement.api.urls import add_engagement_urls +from dojo.engagement.ui.urls import urlpatterns as eng_urls +from dojo.finding.api.urls import add_finding_urls +from dojo.finding.ui.urls import urlpatterns as finding_urls from dojo.finding_group.urls import urlpatterns as finding_group_urls from dojo.github.ui.urls import urlpatterns as github_urls from dojo.home.urls import urlpatterns as home_urls @@ -73,27 +51,40 @@ from dojo.location.api.endpoint_compat import V3EndpointCompatibleViewSet, V3EndpointStatusCompatibleViewSet from dojo.location.api.urls import add_locations_urls from dojo.metrics.urls import urlpatterns as metrics_urls -from dojo.note_type.urls import urlpatterns as note_type_urls -from dojo.notes.urls import urlpatterns as notes_urls +from dojo.note_type.api.urls import add_note_type_urls +from dojo.note_type.ui.urls import urlpatterns as note_type_urls +from dojo.notes.api.urls import add_notes_urls +from dojo.notes.ui.urls import urlpatterns as notes_urls from dojo.notifications.api.urls import add_notifications_urls from dojo.notifications.ui.urls import urlpatterns as notifications_urls -from dojo.object.urls import urlpatterns as object_urls +from dojo.object.ui.urls import urlpatterns as object_urls from dojo.organization.api.urls import add_organization_urls from dojo.organization.urls import urlpatterns as organization_urls -from dojo.regulations.urls import urlpatterns as regulations -from dojo.reports.urls import urlpatterns as reports_urls +from dojo.product.api.urls import add_product_urls +from dojo.product_type.api.urls import add_product_type_urls +from dojo.regulations.api.urls import add_regulations_urls +from dojo.regulations.ui.urls import urlpatterns as regulations +from dojo.reports.ui.urls import urlpatterns as reports_urls +from dojo.risk_acceptance.api.urls import add_risk_acceptance_urls from dojo.search.urls import urlpatterns as search_urls from dojo.sla_config.urls import urlpatterns as sla_urls -from dojo.survey.urls import urlpatterns as survey_urls -from dojo.system_settings.urls import urlpatterns as system_settings_urls -from dojo.test.urls import urlpatterns as test_urls +from dojo.survey.ui.urls import urlpatterns as survey_urls +from dojo.system_settings.api.urls import add_system_settings_urls +from dojo.system_settings.ui.urls import urlpatterns as system_settings_urls +from dojo.test.api.urls import add_test_urls +from dojo.test.ui.urls import urlpatterns as test_urls from dojo.test_type.urls import urlpatterns as test_type_urls -from dojo.tool_config.urls import urlpatterns as tool_config_urls -from dojo.tool_product.urls import urlpatterns as tool_product_urls -from dojo.tool_type.urls import urlpatterns as tool_type_urls +from dojo.tool_config.api.urls import add_tool_config_urls +from dojo.tool_config.ui.urls import urlpatterns as tool_config_urls +from dojo.tool_product.api.urls import add_tool_product_urls +from dojo.tool_product.ui.urls import urlpatterns as tool_product_urls +from dojo.tool_type.api.urls import add_tool_type_urls +from dojo.tool_type.ui.urls import urlpatterns as tool_type_urls from dojo.url.api.urls import add_url_urls from dojo.url.ui.urls import urlpatterns as url_patterns -from dojo.user.urls import urlpatterns as user_urls +from dojo.user.api.urls import add_user_urls +from dojo.user.api.views import UserProfileView +from dojo.user.ui.urls import urlpatterns as user_urls from dojo.utils import get_system_setting logger = logging.getLogger(__name__) @@ -107,16 +98,12 @@ # v2 api written in django-rest-framework v2_api = DefaultRouter() -v2_api.register(r"announcements", AnnouncementViewSet, basename="announcement") +v2_api = add_announcement_urls(v2_api) v2_api.register(r"configuration_permissions", ConfigurationPermissionViewSet, basename="permission") -v2_api.register(r"development_environments", DevelopmentEnvironmentViewSet, basename="development_environment") +v2_api = add_development_environment_urls(v2_api) # RBAC endpoints moved to Pro under legacy authorization: # dojo_groups, dojo_group_members → pro/groups, pro/group_members -v2_api.register(r"endpoint_meta_import", EndpointMetaImporterView, basename="endpointmetaimport") -v2_api.register(r"engagements", EngagementViewSet, basename="engagement") -v2_api.register(r"engagement_presets", EngagementPresetsViewset, basename="engagement_presets") -v2_api.register(r"finding_templates", FindingTemplatesViewSet, basename="finding_template") -v2_api.register(r"findings", FindingViewSet, basename="finding") +v2_api = register_endpoint_meta_import(v2_api) # RBAC endpoint moved to Pro under legacy authorization: global_roles → pro/global_roles v2_api.register(r"import-languages", ImportLanguagesView, basename="importlanguages") v2_api.register(r"import-scan", ImportScanView, basename="importscan") @@ -129,34 +116,31 @@ v2_api.register(r"language_types", LanguageTypeViewSet, basename="language_type") v2_api.register(r"metadata", DojoMetaViewSet, basename="metadata") v2_api.register(r"network_locations", NetworkLocationsViewset, basename="network_locations") -v2_api.register(r"notes", NotesViewSet, basename="notes") -v2_api.register(r"note_type", NoteTypeViewSet, basename="note_type") +v2_api = add_notes_urls(v2_api) +v2_api = add_note_type_urls(v2_api) add_notifications_urls(v2_api) -v2_api.register(r"products", ProductViewSet, basename="product") -v2_api.register(r"product_api_scan_configurations", ProductAPIScanConfigurationViewSet, basename="product_api_scan_configuration") +v2_api = add_product_urls(v2_api) # RBAC endpoints moved to Pro under legacy authorization: # product_groups, product_members → pro/product_groups, pro/product_members -v2_api.register(r"product_types", ProductTypeViewSet, basename="product_type") +v2_api = add_product_type_urls(v2_api) +v2_api = add_engagement_urls(v2_api) +v2_api = add_finding_urls(v2_api) # RBAC endpoints moved to Pro under legacy authorization: # product_type_members, product_type_groups → pro/product_type_members, pro/product_type_groups -v2_api.register(r"regulations", RegulationsViewSet, basename="regulations") +v2_api = add_regulations_urls(v2_api) v2_api.register(r"reimport-scan", ReImportScanView, basename="reimportscan") -v2_api.register(r"request_response_pairs", BurpRawRequestResponseViewSet, basename="request_response_pairs") -v2_api.register(r"risk_acceptance", RiskAcceptanceViewSet, basename="risk_acceptance") +v2_api = add_risk_acceptance_urls(v2_api) # RBAC endpoint moved to Pro under legacy authorization: roles → pro/roles v2_api.register(r"sla_configurations", SLAConfigurationViewset, basename="sla_configurations") v2_api.register(r"sonarqube_issues", SonarqubeIssueViewSet, basename="sonarqube_issue") v2_api.register(r"sonarqube_transitions", SonarqubeIssueTransitionViewSet, basename="sonarqube_issue_transition") -v2_api.register(r"system_settings", SystemSettingsViewSet, basename="system_settings") +v2_api = add_system_settings_urls(v2_api) v2_api.register(r"technologies", AppAnalysisViewSet, basename="app_analysis") -v2_api.register(r"tests", TestsViewSet, basename="test") -v2_api.register(r"test_types", TestTypesViewSet, basename="test_type") -v2_api.register(r"test_imports", TestImportViewSet, basename="test_imports") -v2_api.register(r"tool_configurations", ToolConfigurationsViewSet, basename="tool_configuration") -v2_api.register(r"tool_product_settings", ToolProductSettingsViewSet, basename="tool_product_settings") -v2_api.register(r"tool_types", ToolTypesViewSet, basename="tool_type") -v2_api.register(r"users", UsersViewSet, basename="user") -v2_api.register(r"user_contact_infos", UserContactInfoViewSet, basename="usercontactinfo") +v2_api = add_test_urls(v2_api) +v2_api = add_tool_config_urls(v2_api) +v2_api = add_tool_product_urls(v2_api) +v2_api = add_tool_type_urls(v2_api) +v2_api = add_user_urls(v2_api) # Add the location routes if settings.V3_FEATURE_LOCATIONS: # Endpoints -> Locations @@ -165,8 +149,7 @@ v2_api.register(r"endpoints", V3EndpointCompatibleViewSet, basename="endpoint") v2_api.register(r"endpoint_status", V3EndpointStatusCompatibleViewSet, basename="endpoint_status") else: - v2_api.register(r"endpoints", EndPointViewSet, basename="endpoint") - v2_api.register(r"endpoint_status", EndpointStatusViewSet, basename="endpoint_status") + v2_api = add_endpoint_urls(v2_api) v2_api.register(r"celery", CeleryViewSet, basename="celery") # V3 add_asset_urls(v2_api) diff --git a/dojo/user/__init__.py b/dojo/user/__init__.py index e69de29bb2d..e1885283340 100644 --- a/dojo/user/__init__.py +++ b/dojo/user/__init__.py @@ -0,0 +1 @@ +import dojo.user.admin # noqa: F401 diff --git a/dojo/user/admin.py b/dojo/user/admin.py new file mode 100644 index 00000000000..c8d20a46344 --- /dev/null +++ b/dojo/user/admin.py @@ -0,0 +1,6 @@ +from django.contrib import admin + +from dojo.user.models import Contact, UserContactInfo + +admin.site.register(UserContactInfo) +admin.site.register(Contact) diff --git a/dojo/user/api/__init__.py b/dojo/user/api/__init__.py new file mode 100644 index 00000000000..06ffb66484b --- /dev/null +++ b/dojo/user/api/__init__.py @@ -0,0 +1 @@ +path = "users" # noqa: RUF067 diff --git a/dojo/user/api/filters.py b/dojo/user/api/filters.py new file mode 100644 index 00000000000..0e4bb8b8e14 --- /dev/null +++ b/dojo/user/api/filters.py @@ -0,0 +1,44 @@ +from django.contrib.auth import get_user_model +from django_filters import OrderingFilter +from django_filters import rest_framework as filters + +User = get_user_model() + + +class ApiUserFilter(filters.FilterSet): + last_login = filters.DateFromToRangeFilter() + date_joined = filters.DateFromToRangeFilter() + is_active = filters.BooleanFilter() + is_superuser = filters.BooleanFilter() + username = filters.CharFilter(lookup_expr="icontains") + first_name = filters.CharFilter(lookup_expr="icontains") + last_name = filters.CharFilter(lookup_expr="icontains") + email = filters.CharFilter(lookup_expr="icontains") + + class Meta: + model = User + fields = [ + "id", + "username", + "first_name", + "last_name", + "email", + "is_active", + "is_superuser", + "last_login", + "date_joined", + ] + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("username", "username"), + ("last_name", "last_name"), + ("first_name", "first_name"), + ("email", "email"), + ("is_active", "is_active"), + ("is_superuser", "is_superuser"), + ("date_joined", "date_joined"), + ("last_login", "last_login"), + ), + ) diff --git a/dojo/user/api/serializer.py b/dojo/user/api/serializer.py new file mode 100644 index 00000000000..b74b13c8742 --- /dev/null +++ b/dojo/user/api/serializer.py @@ -0,0 +1,196 @@ +from django.conf import settings +from django.contrib.auth import get_user_model +from django.contrib.auth.models import Permission +from django.contrib.auth.password_validation import validate_password +from django.core.exceptions import ValidationError +from drf_spectacular.utils import extend_schema_field +from rest_framework import serializers + +from dojo.models import Dojo_User, UserContactInfo +from dojo.user.utils import get_configuration_permissions_codenames + +User = get_user_model() + + +class UserSerializer(serializers.ModelSerializer): + date_joined = serializers.DateTimeField(read_only=True) + last_login = serializers.DateTimeField(read_only=True, allow_null=True) + email = serializers.EmailField(required=True) + token_last_reset = serializers.SerializerMethodField() + password_last_reset = serializers.SerializerMethodField() + password = serializers.CharField( + write_only=True, + style={"input_type": "password"}, + required=False, + validators=[validate_password], + ) + configuration_permissions = serializers.PrimaryKeyRelatedField( + allow_null=True, + queryset=Permission.objects.filter( + codename__in=get_configuration_permissions_codenames(), + ), + many=True, + required=False, + source="user_permissions", + ) + + class Meta: + model = Dojo_User + fields = ( + "id", + "username", + "first_name", + "last_name", + "email", + "date_joined", + "last_login", + "is_active", + "is_staff", + "is_superuser", + "token_last_reset", + "password_last_reset", + "password", + "configuration_permissions", + ) + + @extend_schema_field(serializers.DateTimeField(allow_null=True)) + def get_token_last_reset(self, instance): + uci = getattr(instance, "usercontactinfo", None) + return getattr(uci, "token_last_reset", None) + + @extend_schema_field(serializers.DateTimeField(allow_null=True)) + def get_password_last_reset(self, instance): + uci = getattr(instance, "usercontactinfo", None) + return getattr(uci, "password_last_reset", None) + + def to_representation(self, instance): + ret = super().to_representation(instance) + + # This will show only "configuration_permissions" even if user has also + # other permissions + all_permissions = set(ret["configuration_permissions"]) + allowed_configuration_permissions = set( + self.fields[ + "configuration_permissions" + ].child_relation.queryset.values_list("id", flat=True), + ) + ret["configuration_permissions"] = list( + all_permissions.intersection(allowed_configuration_permissions), + ) + + return ret + + def update(self, instance, validated_data): + permissions_in_payload = None + new_configuration_permissions = None + if ( + "user_permissions" in validated_data + ): # This field was renamed from "configuration_permissions" in the meantime + permissions_in_payload = validated_data.pop("user_permissions") + new_configuration_permissions = set(permissions_in_payload) + + instance = super().update(instance, validated_data) + + # This will update only Permissions from category + # "configuration_permissions". Others will be untouched + if new_configuration_permissions: + allowed_configuration_permissions = set( + self.fields[ + "configuration_permissions" + ].child_relation.queryset.all(), + ) + non_configuration_permissions = ( + set(instance.user_permissions.all()) + - allowed_configuration_permissions + ) + new_permissions = non_configuration_permissions.union( + new_configuration_permissions, + ) + instance.user_permissions.set(new_permissions) + + # Clear all configuration permissions if an empty list is provided + if isinstance(permissions_in_payload, list) and len(permissions_in_payload) == 0: + instance.user_permissions.clear() + + return instance + + def create(self, validated_data): + password = validated_data.pop("password", None) + + new_configuration_permissions = None + if ( + "user_permissions" in validated_data + ): # This field was renamed from "configuration_permissions" in the meantime + new_configuration_permissions = set( + validated_data.pop("user_permissions"), + ) + + user = Dojo_User.objects.create(**validated_data) + + if password: + user.set_password(password) + else: + user.set_unusable_password() + + # This will create only Permissions from category + # "configuration_permissions". There are no other Permissions. + if new_configuration_permissions: + user.user_permissions.set(new_configuration_permissions) + + user.save() + return user + + def validate(self, data): + instance_is_superuser = self.instance.is_superuser if self.instance is not None else False + data_is_superuser = data.get("is_superuser", False) + if not self.context["request"].user.is_superuser and ( + instance_is_superuser or data_is_superuser + ): + msg = "Only superusers are allowed to add or edit superusers." + raise ValidationError(msg) + + instance_is_staff = self.instance.is_staff if self.instance is not None else False + data_is_staff = data.get("is_staff", instance_is_staff) + if not self.context["request"].user.is_superuser and data_is_staff != instance_is_staff: + msg = "Only superusers are allowed to add or edit staff users." + raise ValidationError(msg) + + if self.context["request"].method in {"PATCH", "PUT"} and "password" in data: + msg = "Update of password though API is not allowed" + raise ValidationError(msg) + if self.context["request"].method == "POST" and "password" not in data and settings.REQUIRE_PASSWORD_ON_USER: + msg = "Passwords must be supplied for new users" + raise ValidationError(msg) + return super().validate(data) + + +class UserContactInfoSerializer(serializers.ModelSerializer): + user_profile = UserSerializer(many=False, source="user", read_only=True) + + class Meta: + model = UserContactInfo + fields = "__all__" + + def validate(self, data): + user = data.get("user", None) or self.instance.user + if data.get("force_password_reset", False) and not user.has_usable_password(): + msg = "Password resets are not allowed for users authorized through SSO." + raise ValidationError(msg) + return super().validate(data) + + +class UserStubSerializer(serializers.ModelSerializer): + class Meta: + model = Dojo_User + fields = ("id", "username", "first_name", "last_name") + + +class AddUserSerializer(serializers.ModelSerializer): + class Meta: + model = User + fields = ("id", "username") + + +class UserProfileSerializer(serializers.Serializer): + user = UserSerializer(many=False) + user_contact_info = UserContactInfoSerializer(many=False, required=False) diff --git a/dojo/user/api/urls.py b/dojo/user/api/urls.py new file mode 100644 index 00000000000..cb8e8a909b5 --- /dev/null +++ b/dojo/user/api/urls.py @@ -0,0 +1,7 @@ +from dojo.user.api.views import UserContactInfoViewSet, UsersViewSet + + +def add_user_urls(router): + router.register(r"users", UsersViewSet, basename="user") + router.register(r"user_contact_infos", UserContactInfoViewSet, basename="usercontactinfo") + return router diff --git a/dojo/user/api/views.py b/dojo/user/api/views.py new file mode 100644 index 00000000000..c1eb3640442 --- /dev/null +++ b/dojo/user/api/views.py @@ -0,0 +1,102 @@ +import logging + +from crum import get_current_user +from django.contrib.auth import get_user_model +from django_filters.rest_framework import DjangoFilterBackend +from drf_spectacular.utils import extend_schema_view +from rest_framework import status +from rest_framework.decorators import action +from rest_framework.generics import GenericAPIView +from rest_framework.permissions import DjangoModelPermissions, IsAuthenticated +from rest_framework.response import Response + +from dojo.api_v2.views import DojoModelViewSet, PrefetchDojoModelViewSet, schema_with_prefetch +from dojo.authorization import api_permissions as permissions +from dojo.models import UserContactInfo +from dojo.user.api.filters import ApiUserFilter +from dojo.user.api.serializer import ( + UserContactInfoSerializer, + UserProfileSerializer, + UserSerializer, +) +from dojo.user.authentication import reset_token_for_user + +logger = logging.getLogger(__name__) + +User = get_user_model() + + +# Authorization: configuration +class UsersViewSet( + DojoModelViewSet, +): + serializer_class = UserSerializer + queryset = User.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_class = ApiUserFilter + permission_classes = (permissions.UserHasConfigurationPermissionSuperuser,) + + def get_queryset(self): + return User.objects.all().order_by("id") + + def destroy(self, request, *args, **kwargs): + instance = self.get_object() + if request.user == instance: + return Response( + "Users may not delete themselves", + status=status.HTTP_400_BAD_REQUEST, + ) + self.perform_destroy(instance) + return Response(status=status.HTTP_204_NO_CONTENT) + + @action( + detail=True, + methods=["post"], + url_path="reset_api_token", + permission_classes=(IsAuthenticated, permissions.IsSuperUserOrGlobalOwner), + filter_backends=[], + pagination_class=None, + ) + def reset_api_token(self, request, pk=None): + target_user = self.get_object() + reset_token_for_user(acting_user=request.user, target_user=target_user) + return Response(status=status.HTTP_204_NO_CONTENT) + + +# Authorization: superuser +@extend_schema_view(**schema_with_prefetch()) +class UserContactInfoViewSet( + PrefetchDojoModelViewSet, +): + serializer_class = UserContactInfoSerializer + queryset = UserContactInfo.objects.none() + filter_backends = (DjangoFilterBackend,) + filterset_fields = "__all__" + permission_classes = (permissions.IsSuperUser, DjangoModelPermissions) + + def get_queryset(self): + return UserContactInfo.objects.all().order_by("id") + + +# Authorization: authenticated users +class UserProfileView(GenericAPIView): + permission_classes = (IsAuthenticated,) + pagination_class = None + serializer_class = UserProfileSerializer + + @action( + detail=True, methods=["get"], filter_backends=[], pagination_class=None, + ) + def get(self, request, _=None): + user = get_current_user() + user_contact_info = ( + user.usercontactinfo if hasattr(user, "usercontactinfo") else None + ) + serializer = UserProfileSerializer( + { + "user": user, + "user_contact_info": user_contact_info, + }, + many=False, + ) + return Response(serializer.data) diff --git a/dojo/user/models.py b/dojo/user/models.py new file mode 100644 index 00000000000..7d88c731fce --- /dev/null +++ b/dojo/user/models.py @@ -0,0 +1,78 @@ +from django.contrib.auth import get_user_model +from django.core.validators import RegexValidator +from django.db import models +from django.utils.translation import gettext as _ + +User = get_user_model() + + +# proxy class for convenience and UI +class Dojo_User(User): + class Meta: + proxy = True + ordering = ["first_name"] + + def get_full_name(self): + return Dojo_User.generate_full_name(self) + + def __str__(self): + return self.get_full_name() + + @staticmethod + def wants_block_execution(user): + # this return False if there is no user, i.e. in celery processes, unittests, etc. + return hasattr(user, "usercontactinfo") and user.usercontactinfo.block_execution + + @staticmethod + def force_password_reset(user): + return hasattr(user, "usercontactinfo") and user.usercontactinfo.force_password_reset + + def disable_force_password_reset(self): + if hasattr(self, "usercontactinfo"): + self.usercontactinfo.force_password_reset = False + self.usercontactinfo.save() + + def enable_force_password_reset(self): + if hasattr(self, "usercontactinfo"): + self.usercontactinfo.force_password_reset = True + self.usercontactinfo.save() + + @staticmethod + def generate_full_name(user): + """Returns the first_name plus the last_name, with a space in between.""" + full_name = f"{user.first_name} {user.last_name} ({user.username})" + return full_name.strip() + + +class UserContactInfo(models.Model): + user = models.OneToOneField("dojo.Dojo_User", on_delete=models.CASCADE) + title = models.CharField(blank=True, null=True, max_length=150) + phone_regex = RegexValidator(regex=r"^\+?1?\d{9,15}$", + message=_("Phone number must be entered in the format: '+999999999'. " + "Up to 15 digits allowed.")) + phone_number = models.CharField(validators=[phone_regex], blank=True, + max_length=15, + help_text=_("Phone number must be entered in the format: '+999999999'. " + "Up to 15 digits allowed.")) + cell_number = models.CharField(validators=[phone_regex], blank=True, + max_length=15, + help_text=_("Phone number must be entered in the format: '+999999999'. " + "Up to 15 digits allowed.")) + twitter_username = models.CharField(blank=True, null=True, max_length=150) + github_username = models.CharField(blank=True, null=True, max_length=150) + slack_username = models.CharField(blank=True, null=True, max_length=150, help_text=_("Email address associated with your slack account"), verbose_name=_("Slack Email Address")) + slack_user_id = models.CharField(blank=True, null=True, max_length=25) + block_execution = models.BooleanField(default=False, help_text=_("Instead of async deduping a finding the findings will be deduped synchronously and will 'block' the user until completion.")) + force_password_reset = models.BooleanField(default=False, help_text=_("Forces this user to reset their password on next login.")) + ui_use_tailwind = models.BooleanField(default=False, verbose_name=_("Use new UI (beta)"), help_text=_("Opt in to the new Tailwind-based UI. Leave off for the classic UI.")) + token_last_reset = models.DateTimeField(null=True, blank=True, help_text=_("Timestamp of the most recent API token reset for this user.")) + password_last_reset = models.DateTimeField(null=True, blank=True, help_text=_("Timestamp of the most recent password reset for this user.")) + + +class Contact(models.Model): + name = models.CharField(max_length=100) + email = models.EmailField() + team = models.CharField(max_length=100) + is_admin = models.BooleanField(default=False) + is_globally_read_only = models.BooleanField(default=False) + updated = models.DateTimeField(auto_now=True) diff --git a/dojo/user/ui/__init__.py b/dojo/user/ui/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/dojo/user/ui/filters.py b/dojo/user/ui/filters.py new file mode 100644 index 00000000000..91ab303f69f --- /dev/null +++ b/dojo/user/ui/filters.py @@ -0,0 +1,36 @@ +from django_filters import CharFilter, OrderingFilter + +from dojo.filters import DojoFilter +from dojo.models import Dojo_User + + +class UserFilter(DojoFilter): + first_name = CharFilter(lookup_expr="icontains") + last_name = CharFilter(lookup_expr="icontains") + username = CharFilter(lookup_expr="icontains") + email = CharFilter(lookup_expr="icontains") + + o = OrderingFilter( + # tuple-mapping retains order + fields=( + ("username", "username"), + ("last_name", "last_name"), + ("first_name", "first_name"), + ("email", "email"), + ("is_active", "is_active"), + ("is_superuser", "is_superuser"), + ("is_staff", "is_staff"), + ("date_joined", "date_joined"), + ("last_login", "last_login"), + ), + field_labels={ + "username": "User Name", + "is_active": "Active", + "is_superuser": "Superuser", + "is_staff": "Staff", + }, + ) + + class Meta: + model = Dojo_User + fields = ["is_superuser", "is_staff", "is_active", "first_name", "last_name", "username", "email"] diff --git a/dojo/user/ui/forms.py b/dojo/user/ui/forms.py new file mode 100644 index 00000000000..d32c3da187e --- /dev/null +++ b/dojo/user/ui/forms.py @@ -0,0 +1,114 @@ +from crum import get_current_user +from django import forms +from django.conf import settings +from django.contrib.auth.password_validation import validate_password +from django.utils.translation import gettext_lazy as _ + +from dojo.authorization.authorization import user_has_configuration_permission, user_is_superuser_or_global_owner +from dojo.models import Dojo_User, User, UserContactInfo +from dojo.utils import get_password_requirements_string, get_system_setting + + +class DojoUserForm(forms.ModelForm): + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + if not get_current_user().is_superuser and not get_system_setting("enable_user_profile_editable"): + for field in self.fields: + self.fields[field].disabled = True + + class Meta: + model = Dojo_User + exclude = ["password", "last_login", "is_superuser", "groups", + "username", "is_staff", "is_active", "date_joined", + "user_permissions"] + + +class AddDojoUserForm(forms.ModelForm): + email = forms.EmailField(required=True) + password = forms.CharField(widget=forms.PasswordInput, + required=settings.REQUIRE_PASSWORD_ON_USER, + validators=[validate_password], + help_text="") + + class Meta: + model = Dojo_User + fields = ["username", "password", "first_name", "last_name", "email", "is_active", "is_staff", "is_superuser"] + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + current_user = get_current_user() + if not current_user.is_superuser: + self.fields["is_staff"].disabled = True + self.fields["is_superuser"].disabled = True + self.fields["password"].help_text = get_password_requirements_string() + + +class EditDojoUserForm(forms.ModelForm): + email = forms.EmailField(required=True) + + class Meta: + model = Dojo_User + fields = ["username", "first_name", "last_name", "email", "is_active", "is_staff", "is_superuser"] + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + current_user = get_current_user() + if not current_user.is_superuser: + self.fields["is_staff"].disabled = True + self.fields["is_superuser"].disabled = True + + +class DeleteUserForm(forms.ModelForm): + id = forms.IntegerField(required=True, + widget=forms.widgets.HiddenInput()) + + class Meta: + model = User + fields = ["id"] + + +class UserContactInfoForm(forms.ModelForm): + reset_api_token = forms.BooleanField( + required=False, + label=_("Reset API token"), + help_text=_("Upon saving, a new token will be generated and a notification of category 'Other' is triggered."), + ) + + class Meta: + model = UserContactInfo + exclude = ["user", "slack_user_id"] + # Swap order: password_last_reset before token_last_reset + field_order = [ + "title", "phone_number", "cell_number", "twitter_username", "github_username", + "slack_username", "ui_use_tailwind", "block_execution", "force_password_reset", "reset_api_token", + "password_last_reset", "token_last_reset", + ] + + def __init__(self, *args, **kwargs): + user = kwargs.pop("user", None) + super().__init__(*args, **kwargs) + # Make timestamp fields readonly. + # NOTE: `disabled=True` is enforced server-side by Django forms: posted values for disabled fields + # are ignored during binding/cleaning, so these timestamps cannot be modified via this form. + if "password_last_reset" in self.fields: + self.fields["password_last_reset"].disabled = True + if "token_last_reset" in self.fields: + self.fields["token_last_reset"].disabled = True + # Do not expose force password reset if the current user does not have a password to reset + if user is not None: + if not user.has_usable_password(): + self.fields["force_password_reset"].disabled = True + self.fields["force_password_reset"].help_text = "This user is authorized through SSO, and does not have a password to reset" + # Determine some other settings based on the current user + current_user = get_current_user() + if not current_user.is_superuser: + if not user_has_configuration_permission(current_user, "auth.change_user") and \ + not user_has_configuration_permission(current_user, "auth.add_user"): + self.fields.pop("force_password_reset", None) + if not get_system_setting("enable_user_profile_editable"): + for field in self.fields: + self.fields[field].disabled = True + + # Only show reset_api_token to superusers or global owners, and only if API tokens are enabled + if not settings.API_TOKENS_ENABLED or not user_is_superuser_or_global_owner(current_user): + self.fields.pop("reset_api_token", None) diff --git a/dojo/user/urls.py b/dojo/user/ui/urls.py similarity index 99% rename from dojo/user/urls.py rename to dojo/user/ui/urls.py index b3c97bea8ea..395954f7679 100644 --- a/dojo/user/urls.py +++ b/dojo/user/ui/urls.py @@ -2,7 +2,7 @@ from django.contrib.auth import views as auth_views from django.urls import re_path, reverse_lazy -from dojo.user import views +from dojo.user.ui import views urlpatterns = [ # user specific diff --git a/dojo/user/ui/views.py b/dojo/user/ui/views.py new file mode 100644 index 00000000000..9ba12c044c8 --- /dev/null +++ b/dojo/user/ui/views.py @@ -0,0 +1,651 @@ +import contextlib +import logging +from datetime import timedelta + +import hyperlink +from django.conf import settings +from django.contrib import messages +from django.contrib.admin.utils import NestedObjects +from django.contrib.auth import logout +from django.contrib.auth.decorators import login_required, user_passes_test +from django.contrib.auth.forms import AuthenticationForm, PasswordResetForm +from django.contrib.auth.views import LoginView, PasswordResetConfirmView, PasswordResetView +from django.contrib.humanize.templatetags.humanize import naturaltime +from django.core import serializers +from django.core.exceptions import PermissionDenied, ValidationError +from django.core.mail import get_connection +from django.core.mail.backends.smtp import EmailBackend +from django.db import DEFAULT_DB_ALIAS +from django.db.models import Q +from django.db.models.deletion import RestrictedError +from django.http import HttpResponse, HttpResponseRedirect, JsonResponse +from django.shortcuts import get_object_or_404, render +from django.urls import reverse +from django.utils.timezone import now +from django.utils.translation import gettext as _ +from rest_framework.authtoken.models import Token +from rest_framework.exceptions import PermissionDenied as RFPermissionDenied +from rest_framework.exceptions import ValidationError as RFValidationError + +from dojo.authorization.authorization import user_is_superuser_or_global_owner +from dojo.decorators import dojo_ratelimit +from dojo.forms import ( + APIKeyForm, + Authorize_User_For_ProductsForm, + Authorize_User_For_ProductTypesForm, + ChangePasswordForm, + ConfigurationPermissionsForm, +) +from dojo.labels import get_labels +from dojo.models import Alerts, Dojo_User, Product, Product_Type, UserContactInfo +from dojo.user.authentication import reset_token_for_user +from dojo.user.ui.filters import UserFilter +from dojo.user.ui.forms import ( + AddDojoUserForm, + DeleteUserForm, + DojoUserForm, + EditDojoUserForm, + UserContactInfoForm, +) +from dojo.utils import add_breadcrumb, get_page_items, get_setting, get_system_setting + +logger = logging.getLogger(__name__) + +labels = get_labels() + + +class DojoLoginView(LoginView): + template_name = "dojo/login.html" + authentication_form = AuthenticationForm + + def form_valid(self, form): + last_login = None + with contextlib.suppress(Exception): + username = form.cleaned_data.get("username") + user = Dojo_User.objects.get(username=username) + last_login = user.last_login + response = super().form_valid(form) + name = self.request.user.first_name or self.request.user.username + last_login = last_login or self.request.user.last_login + messages.add_message( + self.request, + messages.SUCCESS, + _("Hello %s! Your last login was %s (%s)") % (name, naturaltime(last_login), last_login.strftime("%Y-%m-%d %I:%M:%S %p")), + extra_tags="alert-success") + return response + +# # Django Rest Framework API v2 + + +def api_v2_key(request): + # This check should not be necessary because url should not be in 'urlpatterns' but we never know + if not settings.API_TOKENS_ENABLED: + raise PermissionDenied + api_key = "" + form = APIKeyForm(instance=request.user) + if request.method == "POST": # new key requested + form = APIKeyForm(request.POST, instance=request.user) + if form.is_valid() and form.cleaned_data["id"] == request.user.id: + try: + reset_token_for_user(acting_user=request.user, target_user=request.user, allow_self_reset=True) + except (RFPermissionDenied, RFValidationError) as e: + messages.add_message(request, + messages.ERROR, + _("API Key generation failed: %s") % str(e), + extra_tags="alert-danger") + else: + messages.add_message(request, + messages.SUCCESS, + _("API Key generated successfully."), + extra_tags="alert-success") + else: + raise PermissionDenied + else: + try: + api_key = Token.objects.get(user=request.user) + except Token.DoesNotExist: + api_key = Token.objects.create(user=request.user) + add_breadcrumb(title=_("API Key"), top_level=True, request=request) + + return render(request, "dojo/api_v2_key.html", + {"name": _("API v2 Key"), + "metric": False, + "user": request.user, + "key": api_key, + "form": form, + }) + + +# # user specific +@dojo_ratelimit(key="post:username") +@dojo_ratelimit(key="post:password") +def login_view(request): + return DojoLoginView.as_view(template_name="dojo/login.html", authentication_form=AuthenticationForm)(request) + + +def logout_view(request): + logout(request) + messages.add_message(request, + messages.SUCCESS, + _("You have logged out successfully."), + extra_tags="alert-success") + + return HttpResponseRedirect(reverse("login")) + + +@user_passes_test(lambda u: u.is_active) +def alerts(request): + alerts = Alerts.objects.filter(user_id=request.user).order_by("-id") + + if request.method == "POST": + removed_alerts = request.POST.getlist("alert_select") + alerts.filter(id__in=removed_alerts).delete() + alerts = alerts.filter(~Q(id__in=removed_alerts)) + + paged_alerts = get_page_items(request, alerts, 25) + alert_title = "Alerts" + if request.user.get_full_name(): + alert_title += " for " + request.user.get_full_name() + + add_breadcrumb(title=alert_title, top_level=True, request=request) + return render(request, + "notifications/alerts.html", + {"alerts": paged_alerts}) + + +def delete_alerts(request): + alerts = Alerts.objects.filter(user_id=request.user).order_by("-id") + + if request.method == "POST": + alerts.filter().delete() + messages.add_message( + request, + messages.SUCCESS, + _("Alerts removed."), + extra_tags="alert-success") + return HttpResponseRedirect("alerts") + + return render(request, "notifications/delete_alerts.html", { + "alerts": alerts, + "delete_preview": get_setting("DELETE_PREVIEW"), + }) + + +@login_required +def alerts_json(request, limit=None): + limit = request.GET.get("limit") + if limit: + alerts = serializers.serialize("json", Alerts.objects.filter(user_id=request.user)[:int(limit)]) + else: + alerts = serializers.serialize("json", Alerts.objects.filter(user_id=request.user)) + return HttpResponse(alerts, content_type="application/json") + + +def alertcount(request): + if not settings.DISABLE_ALERT_COUNTER: + count = Alerts.objects.filter(user_id=request.user).count() + return JsonResponse({"count": count}) + return JsonResponse({"count": 0}) + + +def alertcount_text(request): + """Return alert count as plain text for htmx polling.""" + count = Alerts.objects.filter(user_id=request.user).count() if not settings.DISABLE_ALERT_COUNTER else 0 + return HttpResponse(str(count), content_type="text/plain") + + +@login_required +def alerts_partial(request): + """Return alert dropdown HTML partial for htmx.""" + limit = request.GET.get("limit") + if limit: + alerts = Alerts.objects.filter(user_id=request.user)[:int(limit)] + else: + alerts = Alerts.objects.filter(user_id=request.user) + return render(request, "dojo/partials/alerts_dropdown.html", {"alerts": alerts}) + + +def view_profile(request): + user = get_object_or_404(Dojo_User, pk=request.user.id) + form = DojoUserForm(instance=user) + + user_contact = user.usercontactinfo if hasattr(user, "usercontactinfo") else None + contact_form = UserContactInfoForm(user=user) if user_contact is None else UserContactInfoForm(instance=user_contact, user=user) + + if request.method == "POST": + form = DojoUserForm(request.POST, instance=user) + contact_form = UserContactInfoForm(request.POST, instance=user_contact, user=user) + if form.is_valid() and contact_form.is_valid(): + form.save() + contact = contact_form.save(commit=False) + contact.user = user + contact.save() + + messages.add_message(request, + messages.SUCCESS, + _("Profile updated successfully."), + extra_tags="alert-success") + # Redirect so the response renders against a fresh request — this + # ensures UIPreferenceLoader and the UI-toggle banner read the + # just-saved usercontactinfo (e.g. ui_use_tailwind) instead of any + # state cached on the POST request. Also prevents form + # resubmission on refresh. + return HttpResponseRedirect(reverse("view_profile")) + add_breadcrumb(title=_("User Profile - %(user_full_name)s") % {"user_full_name": user.get_full_name()}, top_level=True, request=request) + return render(request, "dojo/profile.html", { + "user": user, + "form": form, + "contact_form": contact_form}) + + +def change_password(request): + user = get_object_or_404(Dojo_User, pk=request.user.id) + form = ChangePasswordForm(user=user) + + if request.method == "POST": + form = ChangePasswordForm(request.POST, user=user) + if form.is_valid(): + new_password = form.cleaned_data["new_password"] + + user.set_password(new_password) + user.disable_force_password_reset() + user.save() + # Case: user is logged in and changes their password via the profile UI. + # We stamp password_last_reset here so this flow is tracked independently from + # the "forgot password" reset flow (handled in DojoPasswordResetConfirmView). + uci, _created = UserContactInfo.objects.get_or_create(user=user) + uci.password_last_reset = now() + uci.save(update_fields=["password_last_reset"]) + + messages.add_message(request, + messages.SUCCESS, + _("Your password has been changed."), + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_profile")) + + add_breadcrumb(title=_("Change Password"), top_level=False, request=request) + return render(request, "dojo/change_pwd.html", {"form": form}) + + +def user(request): + page_name = _("All Users") + users = Dojo_User.objects.all() \ + .select_related("usercontactinfo") \ + .order_by("username", "last_name", "first_name") + users = UserFilter(request.GET, queryset=users) + paged_users = get_page_items(request, users.qs, 25) + add_breadcrumb(title=page_name, top_level=True, request=request) + return render(request, "dojo/users.html", { + "users": paged_users, + "filtered": users, + "name": page_name, + }) + + +def add_user(request): + page_name = _("Add User") + form = AddDojoUserForm() + contact_form = UserContactInfoForm() + user = None + + if request.method == "POST": + form = AddDojoUserForm(request.POST) + contact_form = UserContactInfoForm(request.POST) + if form.is_valid() and contact_form.is_valid(): + if not request.user.is_superuser and form.cleaned_data["is_superuser"]: + messages.add_message(request, + messages.ERROR, + _("Only superusers are allowed to add superusers. User was not saved."), + extra_tags="alert-danger") + elif not request.user.is_superuser and form.cleaned_data["is_staff"]: + messages.add_message(request, + messages.ERROR, + _("Only superusers are allowed to grant staff status. User was not saved."), + extra_tags="alert-danger") + else: + user = form.save(commit=False) + password = request.POST["password"] + if password: + user.set_password(password) + else: + user.set_unusable_password() + user.active = True + user.save() + contact = contact_form.save(commit=False) + contact.user = user + contact.save() + messages.add_message(request, + messages.SUCCESS, + _("User added successfully."), + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_user", args=(user.id,))) + else: + messages.add_message(request, + messages.ERROR, + _("User was not added successfully."), + extra_tags="alert-danger") + add_breadcrumb(title=page_name, top_level=False, request=request) + return render(request, "dojo/add_user.html", { + "name": page_name, + "form": form, + "contact_form": contact_form, + "to_add": True}) + + +def view_user(request, uid): + user = get_object_or_404(Dojo_User, id=uid) + # Legacy access lists: Product / Product_Type the user is on + # via authorized_users (with cascade Product_Type → Product). + accessible_product_types = Product_Type.objects.filter( + authorized_users=user, + ).order_by("name") + accessible_products = Product.objects.filter( + Q(authorized_users=user) | Q(prod_type__authorized_users=user), + ).distinct().order_by("name") + configuration_permission_form = ConfigurationPermissionsForm(user=user) + + add_breadcrumb(title=_("View User"), top_level=False, request=request) + return render(request, "dojo/view_user.html", { + "user": user, + "accessible_product_types": accessible_product_types, + "accessible_products": accessible_products, + "configuration_permission_form": configuration_permission_form}) + + +def edit_user(request, uid): + page_name = _("Edit User") + user = get_object_or_404(Dojo_User, id=uid) + form = EditDojoUserForm(instance=user) + + user_contact = user.usercontactinfo if hasattr(user, "usercontactinfo") else None + contact_form = UserContactInfoForm(user=user) if user_contact is None else UserContactInfoForm(instance=user_contact, user=user) + + if request.method == "POST": + form = EditDojoUserForm(request.POST, instance=user) + if user_contact is None: + contact_form = UserContactInfoForm(request.POST, user=user) + else: + contact_form = UserContactInfoForm(request.POST, instance=user_contact, user=user) + + if form.is_valid() and contact_form.is_valid(): + if not request.user.is_superuser and form.cleaned_data["is_superuser"]: + messages.add_message(request, + messages.ERROR, + _("Only superusers are allowed to edit superusers. User was not saved."), + extra_tags="alert-danger") + elif not request.user.is_superuser and form.cleaned_data["is_staff"] != user.is_staff: + messages.add_message(request, + messages.ERROR, + _("Only superusers are allowed to change staff status. User was not saved."), + extra_tags="alert-danger") + else: + form.save() + contact = contact_form.save(commit=False) + contact.user = user + contact.save() + + # Handle API token reset if checkbox is checked + # Only allow superusers or global owners to reset tokens + token_reset_success = False + if user_is_superuser_or_global_owner(request.user): + reset_token = contact_form.cleaned_data.get("reset_api_token", False) + if reset_token: + try: + reset_token_for_user(acting_user=request.user, target_user=user) + token_reset_success = True + messages.add_message(request, + messages.SUCCESS, + _("API token reset successfully."), + extra_tags="alert-success") + except (RFPermissionDenied, RFValidationError) as e: + # If permission denied or validation error, log but don't fail the user save + messages.add_message(request, + messages.WARNING, + _("User saved successfully, but API token reset failed: %s") % str(e), + extra_tags="alert-warning") + + messages.add_message(request, + messages.SUCCESS, + _("User saved successfully."), + extra_tags="alert-success") + + # Re-instantiate forms to uncheck the checkbox after successful save + if token_reset_success: + # Reload contact from database to get updated token_last_reset timestamp + contact.refresh_from_db() + contact_form = UserContactInfoForm(instance=contact, user=user) + else: + messages.add_message(request, + messages.ERROR, + _("User was not saved successfully."), + extra_tags="alert-danger") + add_breadcrumb(title=page_name, top_level=False, request=request) + return render(request, "dojo/add_user.html", { + "name": page_name, + "form": form, + "contact_form": contact_form, + "to_edit": user}) + + +def delete_user(request, uid): + user = get_object_or_404(Dojo_User, id=uid) + form = DeleteUserForm(instance=user) + + if user.id == request.user.id: + messages.add_message(request, + messages.ERROR, + _("You may not delete yourself."), + extra_tags="alert-danger") + return HttpResponseRedirect(reverse("edit_user", args=(user.id,))) + + if request.method == "POST": + if "id" in request.POST and str(user.id) == request.POST["id"]: + form = DeleteUserForm(request.POST, instance=user) + if form.is_valid(): + if not request.user.is_superuser and user.is_superuser: + messages.add_message(request, + messages.ERROR, + _("Only superusers are allowed to delete superusers. User was not removed."), + extra_tags="alert-danger") + else: + try: + user.delete() + messages.add_message(request, + messages.SUCCESS, + _("User and relationships removed."), + extra_tags="alert-success") + except RestrictedError as err: + messages.add_message(request, + messages.WARNING, + _("User cannot be deleted: %(error)s") % {"error": err}, + extra_tags="alert-warning") + return HttpResponseRedirect(reverse("users")) + + rels = ["Previewing the relationships has been disabled.", ""] + display_preview = get_setting("DELETE_PREVIEW") + if display_preview: + collector = NestedObjects(using=DEFAULT_DB_ALIAS) + collector.collect([user]) + rels = collector.nested() + + add_breadcrumb(title=_("Delete User"), top_level=False, request=request) + return render(request, "dojo/delete_user.html", + {"to_delete": user, + "form": form, + "rels": rels, + }) + + +@user_passes_test(lambda u: u.is_staff) +def authorize_user_for_products(request, uid): + """OS legacy: add this user to one or more products' authorized_users.""" + page_name = _("Authorize User for Products") + user = get_object_or_404(Dojo_User, id=uid) + form = Authorize_User_For_ProductsForm(user=user) + if request.method == "POST": + form = Authorize_User_For_ProductsForm(request.POST, user=user) + if form.is_valid(): + products = form.cleaned_data["products"] + for product in products: + product.authorized_users.add(user) + messages.add_message( + request, messages.SUCCESS, + _("Authorized %(username)s for %(count)d product(s).") % { + "username": user.username, "count": len(products), + }, + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_user", args=(uid,))) + add_breadcrumb(title=page_name, top_level=False, request=request) + return render(request, "dojo/authorize_user_for_products.html", { + "name": page_name, "user": user, "form": form, + }) + + +@user_passes_test(lambda u: u.is_staff) +def authorize_user_for_product_types(request, uid): + """OS legacy: add this user to one or more product_types' authorized_users.""" + page_name = _("Authorize User for Product Types") + user = get_object_or_404(Dojo_User, id=uid) + form = Authorize_User_For_ProductTypesForm(user=user) + if request.method == "POST": + form = Authorize_User_For_ProductTypesForm(request.POST, user=user) + if form.is_valid(): + product_types = form.cleaned_data["product_types"] + for pt in product_types: + pt.authorized_users.add(user) + messages.add_message( + request, messages.SUCCESS, + _("Authorized %(username)s for %(count)d product type(s).") % { + "username": user.username, "count": len(product_types), + }, + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_user", args=(uid,))) + add_breadcrumb(title=page_name, top_level=False, request=request) + return render(request, "dojo/authorize_user_for_product_types.html", { + "name": page_name, "user": user, "form": form, + }) + + +@user_passes_test(lambda u: u.is_staff) +def revoke_user_from_product(request, uid, pid): + """OS legacy: remove user from a product's authorized_users.""" + if request.method != "POST": + raise PermissionDenied + user = get_object_or_404(Dojo_User, id=uid) + product = get_object_or_404(Product, id=pid) + product.authorized_users.remove(user) + messages.add_message( + request, messages.SUCCESS, + _("Revoked %(username)s from %(product)s.") % { + "username": user.username, "product": product.name, + }, + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_user", args=(uid,))) + + +@user_passes_test(lambda u: u.is_staff) +def revoke_user_from_product_type(request, uid, ptid): + """OS legacy: remove user from a product_type's authorized_users.""" + if request.method != "POST": + raise PermissionDenied + user = get_object_or_404(Dojo_User, id=uid) + pt = get_object_or_404(Product_Type, id=ptid) + pt.authorized_users.remove(user) + messages.add_message( + request, messages.SUCCESS, + _("Revoked %(username)s from %(pt)s.") % { + "username": user.username, "pt": pt.name, + }, + extra_tags="alert-success", + ) + return HttpResponseRedirect(reverse("view_user", args=(uid,))) + + +def edit_permissions(request, uid): + user = get_object_or_404(Dojo_User, id=uid) + if request.method == "POST": + form = ConfigurationPermissionsForm(request.POST, user=user) + if form.is_valid(): + form.save() + messages.add_message(request, + messages.SUCCESS, + _("Permissions updated."), + extra_tags="alert-success") + return HttpResponseRedirect(reverse("view_user", args=(uid,))) + + +class DojoForgotUsernameForm(PasswordResetForm): + def send_mail(self, subject_template_name, email_template_name, + context, from_email, to_email, html_email_template_name=None): + + from_email = get_system_setting("email_from") + + url = hyperlink.parse(settings.SITE_URL) + subject_template_name = "login/forgot_username_subject.html" + email_template_name = "login/forgot_username.tpl" + context["site_name"] = url.host + context["protocol"] = url.scheme + context["domain"] = settings.SITE_URL[len(f"{url.scheme}://"):] + + super().send_mail(subject_template_name, email_template_name, context, from_email, to_email, html_email_template_name) + + def clean(self): + try: + connection = get_connection() + if isinstance(connection, EmailBackend): + connection.open() + connection.close() + except Exception: + msg = "SMTP server is not configured correctly..." + raise ValidationError(msg) + + +class DojoPasswordResetForm(PasswordResetForm): + def send_mail(self, subject_template_name, email_template_name, + context, from_email, to_email, html_email_template_name=None): + + from_email = get_system_setting("email_from") + + url = hyperlink.parse(settings.SITE_URL) + email_template_name = "login/forgot_password.tpl" + context["site_name"] = url.host + context["protocol"] = url.scheme + context["domain"] = settings.SITE_URL[len(f"{url.scheme}://"):] + context["link_expiration_date"] = naturaltime(now() + timedelta(seconds=settings.PASSWORD_RESET_TIMEOUT)) + + super().send_mail(subject_template_name, email_template_name, context, from_email, to_email, html_email_template_name) + + def clean(self): + try: + connection = get_connection() + if isinstance(connection, EmailBackend): + connection.open() + connection.close() + except Exception as e: + logger.error("SMTP Server Connection Failure: %s", e) + msg = "SMTP server is not configured correctly..." + raise ValidationError(msg) + + +class DojoPasswordResetView(PasswordResetView): + form_class = DojoPasswordResetForm + + +class DojoForgotUsernameView(PasswordResetView): + form_class = DojoForgotUsernameForm + + +class DojoPasswordResetConfirmView(PasswordResetConfirmView): + def form_valid(self, form): + response = super().form_valid(form) + # Flow: user resets password via the emailed "forgot password" link. + # This uses PasswordResetConfirmView, so we stamp password_last_reset here + # because this flow does not pass through change_password(). + user = form.user + uci, _created = UserContactInfo.objects.get_or_create(user=user) + uci.password_last_reset = now() + uci.save(update_fields=["password_last_reset"]) + return response diff --git a/dojo/user/views.py b/dojo/user/views.py index ce3d8449a39..d193f7e5dbe 100644 --- a/dojo/user/views.py +++ b/dojo/user/views.py @@ -1,649 +1,4 @@ -import contextlib -import logging -from datetime import timedelta - -import hyperlink -from django.conf import settings -from django.contrib import messages -from django.contrib.admin.utils import NestedObjects -from django.contrib.auth import logout -from django.contrib.auth.decorators import login_required, user_passes_test -from django.contrib.auth.forms import AuthenticationForm, PasswordResetForm -from django.contrib.auth.views import LoginView, PasswordResetConfirmView, PasswordResetView -from django.contrib.humanize.templatetags.humanize import naturaltime -from django.core import serializers -from django.core.exceptions import PermissionDenied, ValidationError -from django.core.mail import get_connection -from django.core.mail.backends.smtp import EmailBackend -from django.db import DEFAULT_DB_ALIAS -from django.db.models import Q -from django.db.models.deletion import RestrictedError -from django.http import HttpResponse, HttpResponseRedirect, JsonResponse -from django.shortcuts import get_object_or_404, render -from django.urls import reverse -from django.utils.timezone import now -from django.utils.translation import gettext as _ -from rest_framework.authtoken.models import Token -from rest_framework.exceptions import PermissionDenied as RFPermissionDenied -from rest_framework.exceptions import ValidationError as RFValidationError - -from dojo.authorization.authorization import user_is_superuser_or_global_owner -from dojo.decorators import dojo_ratelimit -from dojo.filters import UserFilter -from dojo.forms import ( - AddDojoUserForm, - APIKeyForm, - Authorize_User_For_ProductsForm, - Authorize_User_For_ProductTypesForm, - ChangePasswordForm, - ConfigurationPermissionsForm, - DeleteUserForm, - DojoUserForm, - EditDojoUserForm, - UserContactInfoForm, -) -from dojo.labels import get_labels -from dojo.models import Alerts, Dojo_User, Product, Product_Type, UserContactInfo -from dojo.user.authentication import reset_token_for_user -from dojo.utils import add_breadcrumb, get_page_items, get_setting, get_system_setting - -logger = logging.getLogger(__name__) - -labels = get_labels() - - -class DojoLoginView(LoginView): - template_name = "dojo/login.html" - authentication_form = AuthenticationForm - - def form_valid(self, form): - last_login = None - with contextlib.suppress(Exception): - username = form.cleaned_data.get("username") - user = Dojo_User.objects.get(username=username) - last_login = user.last_login - response = super().form_valid(form) - name = self.request.user.first_name or self.request.user.username - last_login = last_login or self.request.user.last_login - messages.add_message( - self.request, - messages.SUCCESS, - _("Hello %s! Your last login was %s (%s)") % (name, naturaltime(last_login), last_login.strftime("%Y-%m-%d %I:%M:%S %p")), - extra_tags="alert-success") - return response - -# # Django Rest Framework API v2 - - -def api_v2_key(request): - # This check should not be necessary because url should not be in 'urlpatterns' but we never know - if not settings.API_TOKENS_ENABLED: - raise PermissionDenied - api_key = "" - form = APIKeyForm(instance=request.user) - if request.method == "POST": # new key requested - form = APIKeyForm(request.POST, instance=request.user) - if form.is_valid() and form.cleaned_data["id"] == request.user.id: - try: - reset_token_for_user(acting_user=request.user, target_user=request.user, allow_self_reset=True) - except (RFPermissionDenied, RFValidationError) as e: - messages.add_message(request, - messages.ERROR, - _("API Key generation failed: %s") % str(e), - extra_tags="alert-danger") - else: - messages.add_message(request, - messages.SUCCESS, - _("API Key generated successfully."), - extra_tags="alert-success") - else: - raise PermissionDenied - else: - try: - api_key = Token.objects.get(user=request.user) - except Token.DoesNotExist: - api_key = Token.objects.create(user=request.user) - add_breadcrumb(title=_("API Key"), top_level=True, request=request) - - return render(request, "dojo/api_v2_key.html", - {"name": _("API v2 Key"), - "metric": False, - "user": request.user, - "key": api_key, - "form": form, - }) - - -# # user specific -@dojo_ratelimit(key="post:username") -@dojo_ratelimit(key="post:password") -def login_view(request): - return DojoLoginView.as_view(template_name="dojo/login.html", authentication_form=AuthenticationForm)(request) - - -def logout_view(request): - logout(request) - messages.add_message(request, - messages.SUCCESS, - _("You have logged out successfully."), - extra_tags="alert-success") - - return HttpResponseRedirect(reverse("login")) - - -@user_passes_test(lambda u: u.is_active) -def alerts(request): - alerts = Alerts.objects.filter(user_id=request.user).order_by("-id") - - if request.method == "POST": - removed_alerts = request.POST.getlist("alert_select") - alerts.filter(id__in=removed_alerts).delete() - alerts = alerts.filter(~Q(id__in=removed_alerts)) - - paged_alerts = get_page_items(request, alerts, 25) - alert_title = "Alerts" - if request.user.get_full_name(): - alert_title += " for " + request.user.get_full_name() - - add_breadcrumb(title=alert_title, top_level=True, request=request) - return render(request, - "notifications/alerts.html", - {"alerts": paged_alerts}) - - -def delete_alerts(request): - alerts = Alerts.objects.filter(user_id=request.user).order_by("-id") - - if request.method == "POST": - alerts.filter().delete() - messages.add_message( - request, - messages.SUCCESS, - _("Alerts removed."), - extra_tags="alert-success") - return HttpResponseRedirect("alerts") - - return render(request, "notifications/delete_alerts.html", { - "alerts": alerts, - "delete_preview": get_setting("DELETE_PREVIEW"), - }) - - -@login_required -def alerts_json(request, limit=None): - limit = request.GET.get("limit") - if limit: - alerts = serializers.serialize("json", Alerts.objects.filter(user_id=request.user)[:int(limit)]) - else: - alerts = serializers.serialize("json", Alerts.objects.filter(user_id=request.user)) - return HttpResponse(alerts, content_type="application/json") - - -def alertcount(request): - if not settings.DISABLE_ALERT_COUNTER: - count = Alerts.objects.filter(user_id=request.user).count() - return JsonResponse({"count": count}) - return JsonResponse({"count": 0}) - - -def alertcount_text(request): - """Return alert count as plain text for htmx polling.""" - count = Alerts.objects.filter(user_id=request.user).count() if not settings.DISABLE_ALERT_COUNTER else 0 - return HttpResponse(str(count), content_type="text/plain") - - -@login_required -def alerts_partial(request): - """Return alert dropdown HTML partial for htmx.""" - limit = request.GET.get("limit") - if limit: - alerts = Alerts.objects.filter(user_id=request.user)[:int(limit)] - else: - alerts = Alerts.objects.filter(user_id=request.user) - return render(request, "dojo/partials/alerts_dropdown.html", {"alerts": alerts}) - - -def view_profile(request): - user = get_object_or_404(Dojo_User, pk=request.user.id) - form = DojoUserForm(instance=user) - - user_contact = user.usercontactinfo if hasattr(user, "usercontactinfo") else None - contact_form = UserContactInfoForm(user=user) if user_contact is None else UserContactInfoForm(instance=user_contact, user=user) - - if request.method == "POST": - form = DojoUserForm(request.POST, instance=user) - contact_form = UserContactInfoForm(request.POST, instance=user_contact, user=user) - if form.is_valid() and contact_form.is_valid(): - form.save() - contact = contact_form.save(commit=False) - contact.user = user - contact.save() - - messages.add_message(request, - messages.SUCCESS, - _("Profile updated successfully."), - extra_tags="alert-success") - # Redirect so the response renders against a fresh request — this - # ensures UIPreferenceLoader and the UI-toggle banner read the - # just-saved usercontactinfo (e.g. ui_use_tailwind) instead of any - # state cached on the POST request. Also prevents form - # resubmission on refresh. - return HttpResponseRedirect(reverse("view_profile")) - add_breadcrumb(title=_("User Profile - %(user_full_name)s") % {"user_full_name": user.get_full_name()}, top_level=True, request=request) - return render(request, "dojo/profile.html", { - "user": user, - "form": form, - "contact_form": contact_form}) - - -def change_password(request): - user = get_object_or_404(Dojo_User, pk=request.user.id) - form = ChangePasswordForm(user=user) - - if request.method == "POST": - form = ChangePasswordForm(request.POST, user=user) - if form.is_valid(): - new_password = form.cleaned_data["new_password"] - - user.set_password(new_password) - user.disable_force_password_reset() - user.save() - # Case: user is logged in and changes their password via the profile UI. - # We stamp password_last_reset here so this flow is tracked independently from - # the "forgot password" reset flow (handled in DojoPasswordResetConfirmView). - uci, _created = UserContactInfo.objects.get_or_create(user=user) - uci.password_last_reset = now() - uci.save(update_fields=["password_last_reset"]) - - messages.add_message(request, - messages.SUCCESS, - _("Your password has been changed."), - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_profile")) - - add_breadcrumb(title=_("Change Password"), top_level=False, request=request) - return render(request, "dojo/change_pwd.html", {"form": form}) - - -def user(request): - page_name = _("All Users") - users = Dojo_User.objects.all() \ - .select_related("usercontactinfo") \ - .order_by("username", "last_name", "first_name") - users = UserFilter(request.GET, queryset=users) - paged_users = get_page_items(request, users.qs, 25) - add_breadcrumb(title=page_name, top_level=True, request=request) - return render(request, "dojo/users.html", { - "users": paged_users, - "filtered": users, - "name": page_name, - }) - - -def add_user(request): - page_name = _("Add User") - form = AddDojoUserForm() - contact_form = UserContactInfoForm() - user = None - - if request.method == "POST": - form = AddDojoUserForm(request.POST) - contact_form = UserContactInfoForm(request.POST) - if form.is_valid() and contact_form.is_valid(): - if not request.user.is_superuser and form.cleaned_data["is_superuser"]: - messages.add_message(request, - messages.ERROR, - _("Only superusers are allowed to add superusers. User was not saved."), - extra_tags="alert-danger") - elif not request.user.is_superuser and form.cleaned_data["is_staff"]: - messages.add_message(request, - messages.ERROR, - _("Only superusers are allowed to grant staff status. User was not saved."), - extra_tags="alert-danger") - else: - user = form.save(commit=False) - password = request.POST["password"] - if password: - user.set_password(password) - else: - user.set_unusable_password() - user.active = True - user.save() - contact = contact_form.save(commit=False) - contact.user = user - contact.save() - messages.add_message(request, - messages.SUCCESS, - _("User added successfully."), - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_user", args=(user.id,))) - else: - messages.add_message(request, - messages.ERROR, - _("User was not added successfully."), - extra_tags="alert-danger") - add_breadcrumb(title=page_name, top_level=False, request=request) - return render(request, "dojo/add_user.html", { - "name": page_name, - "form": form, - "contact_form": contact_form, - "to_add": True}) - - -def view_user(request, uid): - user = get_object_or_404(Dojo_User, id=uid) - # Legacy access lists: Product / Product_Type the user is on - # via authorized_users (with cascade Product_Type → Product). - accessible_product_types = Product_Type.objects.filter( - authorized_users=user, - ).order_by("name") - accessible_products = Product.objects.filter( - Q(authorized_users=user) | Q(prod_type__authorized_users=user), - ).distinct().order_by("name") - configuration_permission_form = ConfigurationPermissionsForm(user=user) - - add_breadcrumb(title=_("View User"), top_level=False, request=request) - return render(request, "dojo/view_user.html", { - "user": user, - "accessible_product_types": accessible_product_types, - "accessible_products": accessible_products, - "configuration_permission_form": configuration_permission_form}) - - -def edit_user(request, uid): - page_name = _("Edit User") - user = get_object_or_404(Dojo_User, id=uid) - form = EditDojoUserForm(instance=user) - - user_contact = user.usercontactinfo if hasattr(user, "usercontactinfo") else None - contact_form = UserContactInfoForm(user=user) if user_contact is None else UserContactInfoForm(instance=user_contact, user=user) - - if request.method == "POST": - form = EditDojoUserForm(request.POST, instance=user) - if user_contact is None: - contact_form = UserContactInfoForm(request.POST, user=user) - else: - contact_form = UserContactInfoForm(request.POST, instance=user_contact, user=user) - - if form.is_valid() and contact_form.is_valid(): - if not request.user.is_superuser and form.cleaned_data["is_superuser"]: - messages.add_message(request, - messages.ERROR, - _("Only superusers are allowed to edit superusers. User was not saved."), - extra_tags="alert-danger") - elif not request.user.is_superuser and form.cleaned_data["is_staff"] != user.is_staff: - messages.add_message(request, - messages.ERROR, - _("Only superusers are allowed to change staff status. User was not saved."), - extra_tags="alert-danger") - else: - form.save() - contact = contact_form.save(commit=False) - contact.user = user - contact.save() - - # Handle API token reset if checkbox is checked - # Only allow superusers or global owners to reset tokens - token_reset_success = False - if user_is_superuser_or_global_owner(request.user): - reset_token = contact_form.cleaned_data.get("reset_api_token", False) - if reset_token: - try: - reset_token_for_user(acting_user=request.user, target_user=user) - token_reset_success = True - messages.add_message(request, - messages.SUCCESS, - _("API token reset successfully."), - extra_tags="alert-success") - except (RFPermissionDenied, RFValidationError) as e: - # If permission denied or validation error, log but don't fail the user save - messages.add_message(request, - messages.WARNING, - _("User saved successfully, but API token reset failed: %s") % str(e), - extra_tags="alert-warning") - - messages.add_message(request, - messages.SUCCESS, - _("User saved successfully."), - extra_tags="alert-success") - - # Re-instantiate forms to uncheck the checkbox after successful save - if token_reset_success: - # Reload contact from database to get updated token_last_reset timestamp - contact.refresh_from_db() - contact_form = UserContactInfoForm(instance=contact, user=user) - else: - messages.add_message(request, - messages.ERROR, - _("User was not saved successfully."), - extra_tags="alert-danger") - add_breadcrumb(title=page_name, top_level=False, request=request) - return render(request, "dojo/add_user.html", { - "name": page_name, - "form": form, - "contact_form": contact_form, - "to_edit": user}) - - -def delete_user(request, uid): - user = get_object_or_404(Dojo_User, id=uid) - form = DeleteUserForm(instance=user) - - if user.id == request.user.id: - messages.add_message(request, - messages.ERROR, - _("You may not delete yourself."), - extra_tags="alert-danger") - return HttpResponseRedirect(reverse("edit_user", args=(user.id,))) - - if request.method == "POST": - if "id" in request.POST and str(user.id) == request.POST["id"]: - form = DeleteUserForm(request.POST, instance=user) - if form.is_valid(): - if not request.user.is_superuser and user.is_superuser: - messages.add_message(request, - messages.ERROR, - _("Only superusers are allowed to delete superusers. User was not removed."), - extra_tags="alert-danger") - else: - try: - user.delete() - messages.add_message(request, - messages.SUCCESS, - _("User and relationships removed."), - extra_tags="alert-success") - except RestrictedError as err: - messages.add_message(request, - messages.WARNING, - _("User cannot be deleted: %(error)s") % {"error": err}, - extra_tags="alert-warning") - return HttpResponseRedirect(reverse("users")) - - rels = ["Previewing the relationships has been disabled.", ""] - display_preview = get_setting("DELETE_PREVIEW") - if display_preview: - collector = NestedObjects(using=DEFAULT_DB_ALIAS) - collector.collect([user]) - rels = collector.nested() - - add_breadcrumb(title=_("Delete User"), top_level=False, request=request) - return render(request, "dojo/delete_user.html", - {"to_delete": user, - "form": form, - "rels": rels, - }) - - -@user_passes_test(lambda u: u.is_staff) -def authorize_user_for_products(request, uid): - """OS legacy: add this user to one or more products' authorized_users.""" - page_name = _("Authorize User for Products") - user = get_object_or_404(Dojo_User, id=uid) - form = Authorize_User_For_ProductsForm(user=user) - if request.method == "POST": - form = Authorize_User_For_ProductsForm(request.POST, user=user) - if form.is_valid(): - products = form.cleaned_data["products"] - for product in products: - product.authorized_users.add(user) - messages.add_message( - request, messages.SUCCESS, - _("Authorized %(username)s for %(count)d product(s).") % { - "username": user.username, "count": len(products), - }, - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_user", args=(uid,))) - add_breadcrumb(title=page_name, top_level=False, request=request) - return render(request, "dojo/authorize_user_for_products.html", { - "name": page_name, "user": user, "form": form, - }) - - -@user_passes_test(lambda u: u.is_staff) -def authorize_user_for_product_types(request, uid): - """OS legacy: add this user to one or more product_types' authorized_users.""" - page_name = _("Authorize User for Product Types") - user = get_object_or_404(Dojo_User, id=uid) - form = Authorize_User_For_ProductTypesForm(user=user) - if request.method == "POST": - form = Authorize_User_For_ProductTypesForm(request.POST, user=user) - if form.is_valid(): - product_types = form.cleaned_data["product_types"] - for pt in product_types: - pt.authorized_users.add(user) - messages.add_message( - request, messages.SUCCESS, - _("Authorized %(username)s for %(count)d product type(s).") % { - "username": user.username, "count": len(product_types), - }, - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_user", args=(uid,))) - add_breadcrumb(title=page_name, top_level=False, request=request) - return render(request, "dojo/authorize_user_for_product_types.html", { - "name": page_name, "user": user, "form": form, - }) - - -@user_passes_test(lambda u: u.is_staff) -def revoke_user_from_product(request, uid, pid): - """OS legacy: remove user from a product's authorized_users.""" - if request.method != "POST": - raise PermissionDenied - user = get_object_or_404(Dojo_User, id=uid) - product = get_object_or_404(Product, id=pid) - product.authorized_users.remove(user) - messages.add_message( - request, messages.SUCCESS, - _("Revoked %(username)s from %(product)s.") % { - "username": user.username, "product": product.name, - }, - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_user", args=(uid,))) - - -@user_passes_test(lambda u: u.is_staff) -def revoke_user_from_product_type(request, uid, ptid): - """OS legacy: remove user from a product_type's authorized_users.""" - if request.method != "POST": - raise PermissionDenied - user = get_object_or_404(Dojo_User, id=uid) - pt = get_object_or_404(Product_Type, id=ptid) - pt.authorized_users.remove(user) - messages.add_message( - request, messages.SUCCESS, - _("Revoked %(username)s from %(pt)s.") % { - "username": user.username, "pt": pt.name, - }, - extra_tags="alert-success", - ) - return HttpResponseRedirect(reverse("view_user", args=(uid,))) - - -def edit_permissions(request, uid): - user = get_object_or_404(Dojo_User, id=uid) - if request.method == "POST": - form = ConfigurationPermissionsForm(request.POST, user=user) - if form.is_valid(): - form.save() - messages.add_message(request, - messages.SUCCESS, - _("Permissions updated."), - extra_tags="alert-success") - return HttpResponseRedirect(reverse("view_user", args=(uid,))) - - -class DojoForgotUsernameForm(PasswordResetForm): - def send_mail(self, subject_template_name, email_template_name, - context, from_email, to_email, html_email_template_name=None): - - from_email = get_system_setting("email_from") - - url = hyperlink.parse(settings.SITE_URL) - subject_template_name = "login/forgot_username_subject.html" - email_template_name = "login/forgot_username.tpl" - context["site_name"] = url.host - context["protocol"] = url.scheme - context["domain"] = settings.SITE_URL[len(f"{url.scheme}://"):] - - super().send_mail(subject_template_name, email_template_name, context, from_email, to_email, html_email_template_name) - - def clean(self): - try: - connection = get_connection() - if isinstance(connection, EmailBackend): - connection.open() - connection.close() - except Exception: - msg = "SMTP server is not configured correctly..." - raise ValidationError(msg) - - -class DojoPasswordResetForm(PasswordResetForm): - def send_mail(self, subject_template_name, email_template_name, - context, from_email, to_email, html_email_template_name=None): - - from_email = get_system_setting("email_from") - - url = hyperlink.parse(settings.SITE_URL) - email_template_name = "login/forgot_password.tpl" - context["site_name"] = url.host - context["protocol"] = url.scheme - context["domain"] = settings.SITE_URL[len(f"{url.scheme}://"):] - context["link_expiration_date"] = naturaltime(now() + timedelta(seconds=settings.PASSWORD_RESET_TIMEOUT)) - - super().send_mail(subject_template_name, email_template_name, context, from_email, to_email, html_email_template_name) - - def clean(self): - try: - connection = get_connection() - if isinstance(connection, EmailBackend): - connection.open() - connection.close() - except Exception as e: - logger.error("SMTP Server Connection Failure: %s", e) - msg = "SMTP server is not configured correctly..." - raise ValidationError(msg) - - -class DojoPasswordResetView(PasswordResetView): - form_class = DojoPasswordResetForm - - -class DojoForgotUsernameView(PasswordResetView): - form_class = DojoForgotUsernameForm - - -class DojoPasswordResetConfirmView(PasswordResetConfirmView): - def form_valid(self, form): - response = super().form_valid(form) - # Flow: user resets password via the emailed "forgot password" link. - # This uses PasswordResetConfirmView, so we stamp password_last_reset here - # because this flow does not pass through change_password(). - user = form.user - uci, _created = UserContactInfo.objects.get_or_create(user=user) - uci.password_last_reset = now() - uci.save(update_fields=["password_last_reset"]) - return response +# Backward-compat shim: the view logic moved to dojo.user.ui.views during the +# module reorg. External consumers (e.g. dojo-pro) still import from +# dojo.user.views, so re-export the public names from their new location. +from dojo.user.ui.views import * # noqa: F403 -- backward compat re-export diff --git a/unittests/test_apply_finding_template.py b/unittests/test_apply_finding_template.py index f2fd228a7c0..468dacbd36f 100644 --- a/unittests/test_apply_finding_template.py +++ b/unittests/test_apply_finding_template.py @@ -13,8 +13,8 @@ Product_Member, Role, ) -from dojo.finding import views from dojo.finding.helper import save_endpoints_template, save_vulnerability_ids_template +from dojo.finding.ui import views from dojo.models import ( Dojo_User, Engagement, @@ -28,7 +28,7 @@ Test_Type, Vulnerability_Id, ) -from dojo.test import views as test_views +from dojo.test.ui import views as test_views from unittests.dojo_test_case import DojoTestCase, versioned_fixtures diff --git a/unittests/test_copy_model.py b/unittests/test_copy_model.py index f36348753b2..2b262847ac5 100644 --- a/unittests/test_copy_model.py +++ b/unittests/test_copy_model.py @@ -1,6 +1,10 @@ +from unittest.mock import patch + +from dojo.engagement.services import copy_engagement from dojo.location.models import Location, LocationFindingReference from dojo.models import Endpoint, Endpoint_Status, Engagement, Finding, Product, Test, User +from dojo.test.services import copy_test from dojo.url.models import URL from .dojo_test_case import DojoTestCase, skip_unless_v2, skip_unless_v3 @@ -276,6 +280,34 @@ def test_duplicate_test_with_tags_and_notes(self): self.assertEqual(test.tags, test_copy.tags) +class TestCopyTestService(DojoTestCase): + + """Phase 2: the copy_test service holds the copy workflow extracted from the UI view.""" + + @patch("dojo.test.services.create_notification") + @patch("dojo.test.services.dojo_dispatch_task") + def test_copy_test_service(self, mock_dispatch, mock_notification): + user, _ = User.objects.get_or_create(username="admin") + product_type = self.create_product_type("svc_pt_test") + product = self.create_product("svc_copy_test_product", prod_type=product_type) + engagement = self.create_engagement("svc_eng_test", product) + test = self.create_test(engagement=engagement, scan_type="NPM Audit Scan", title="test") + _ = Finding.objects.create(test=test, reporter=user) + before_tests = Test.objects.filter(engagement=engagement).count() + before_findings = Finding.objects.filter(test__engagement=engagement).count() + # Run the service (copy into the same engagement) + test_copy = copy_test(test, engagement, user) + # A new test was created under the engagement, with its findings + self.assertEqual(before_tests + 1, Test.objects.filter(engagement=engagement).count()) + self.assertNotEqual(test.id, test_copy.id) + self.assertEqual(engagement, test_copy.engagement) + self.assertEqual(before_findings + 1, Finding.objects.filter(test__engagement=engagement).count()) + # Side effects: grade recalculation dispatched and a notification raised + mock_dispatch.assert_called_once() + mock_notification.assert_called_once() + self.assertEqual(mock_notification.call_args.kwargs["event"], "test_copied") + + class TestCopyEngagementModel(DojoTestCase): def test_duplicate_engagement(self): @@ -358,3 +390,32 @@ def test_duplicate_engagement_with_tags_and_notes(self): self.assertQuerySetEqual(engagement.notes.all(), engagement_copy.notes.all()) # Do the tags match self.assertEqual(engagement.tags, engagement_copy.tags) + + +class TestCopyEngagementService(DojoTestCase): + + """Phase 2: the copy_engagement service holds the copy workflow extracted from the UI view.""" + + @patch("dojo.engagement.services.create_notification") + @patch("dojo.engagement.services.dojo_dispatch_task") + def test_copy_engagement_service(self, mock_dispatch, mock_notification): + user, _ = User.objects.get_or_create(username="admin") + product_type = self.create_product_type("svc_prod_type") + product = self.create_product("svc_copy_product", prod_type=product_type) + engagement = self.create_engagement("svc_eng", product) + test = self.create_test(engagement=engagement, scan_type="NPM Audit Scan", title="test") + _ = Finding.objects.create(test=test, reporter=user) + before = Engagement.objects.filter(product=product).count() + before_findings = Finding.objects.filter(test__engagement__product=product).count() + # Run the service + engagement_copy = copy_engagement(engagement, user) + # A new engagement was created under the same product + self.assertEqual(before + 1, Engagement.objects.filter(product=product).count()) + self.assertNotEqual(engagement.id, engagement_copy.id) + self.assertEqual(product, engagement_copy.product) + # Findings were duplicated along with the engagement + self.assertEqual(before_findings + 1, Finding.objects.filter(test__engagement__product=product).count()) + # Side effects: grade recalculation dispatched and a notification raised + mock_dispatch.assert_called_once() + mock_notification.assert_called_once() + self.assertEqual(mock_notification.call_args.kwargs["event"], "engagement_copied") diff --git a/unittests/test_false_positive_history_logic.py b/unittests/test_false_positive_history_logic.py index 8748239bedd..5975348e14e 100644 --- a/unittests/test_false_positive_history_logic.py +++ b/unittests/test_false_positive_history_logic.py @@ -6,7 +6,7 @@ from django.conf import settings from dojo.finding.deduplication import do_false_positive_history_batch -from dojo.finding.views import EditFinding +from dojo.finding.ui.views import EditFinding from dojo.location.models import Location, LocationFindingReference from dojo.models import ( Endpoint, diff --git a/unittests/test_filter_finding_mitigation.py b/unittests/test_filter_finding_mitigation.py index 31b3fd5ce95..d21e7db7693 100644 --- a/unittests/test_filter_finding_mitigation.py +++ b/unittests/test_filter_finding_mitigation.py @@ -3,7 +3,8 @@ from django.test import TestCase from django.utils import timezone -from dojo.filters import ApiFindingFilter, FindingFilterHelper +from dojo.finding.api.filters import ApiFindingFilter +from dojo.finding.ui.filters import FindingFilterHelper from dojo.models import ( Dojo_User, Engagement, diff --git a/unittests/test_finding_group_filter_context.py b/unittests/test_finding_group_filter_context.py index f9811aa5942..6af9a7028b0 100644 --- a/unittests/test_finding_group_filter_context.py +++ b/unittests/test_finding_group_filter_context.py @@ -1,6 +1,6 @@ from django.utils.timezone import now -from dojo.filters import FindingFilter, FindingFilterWithoutObjectLookups +from dojo.finding.ui.filters import FindingFilter, FindingFilterWithoutObjectLookups from dojo.models import ( Dojo_User, Engagement, diff --git a/unittests/test_notifications.py b/unittests/test_notifications.py index 1980f73504d..504e168e70b 100644 --- a/unittests/test_notifications.py +++ b/unittests/test_notifications.py @@ -478,7 +478,7 @@ def test_auditlog_on(self, mock): self.client.delete(reverse("product_type-detail", args=(prod_type.pk,)), format="json") self.assertEqual(mock.call_args_list[-1].kwargs["description"], 'The Organization "notif prod type API" was deleted by admin') - @patch("dojo.api_v2.serializers.dojo_dispatch_task") + @patch("dojo.finding.api.serializer.dojo_dispatch_task") def test_create_calls_notification_with_auto_assigned_reporter(self, mock_dispatch): """Dispatch of async_create_notification when creating a finding without explicit reporter.""" payload = self._minimal_create_payload("Finding with auto-assigned reporter notification") @@ -504,7 +504,7 @@ def test_create_calls_notification_with_auto_assigned_reporter(self, mock_dispat created_finding = Finding.objects.get(id=created_id) self.assertEqual(created_finding.reporter, self.admin) - @patch("dojo.api_v2.serializers.dojo_dispatch_task") + @patch("dojo.finding.api.serializer.dojo_dispatch_task") def test_create_calls_notification_with_explicit_reporter(self, mock_dispatch): """Dispatch of async_create_notification when creating a finding with explicit reporter.""" explicit_reporter = User.objects.create(username="explicit_reporter", email="reporter@test.com") @@ -533,7 +533,7 @@ def test_create_calls_notification_with_explicit_reporter(self, mock_dispatch): created_finding = Finding.objects.get(id=created_id) self.assertEqual(created_finding.reporter, explicit_reporter) - @patch("dojo.api_v2.serializers.dojo_dispatch_task") + @patch("dojo.finding.api.serializer.dojo_dispatch_task") def test_notification_parameters_are_correct(self, mock_dispatch): """All dispatch parameters for finding_added are properly formatted and passed.""" payload = self._minimal_create_payload("Test Finding for Parameter Validation") diff --git a/unittests/test_product_metrics_closed_count.py b/unittests/test_product_metrics_closed_count.py index 80b9c4dbd62..31b978b5b6a 100644 --- a/unittests/test_product_metrics_closed_count.py +++ b/unittests/test_product_metrics_closed_count.py @@ -18,7 +18,7 @@ from django.test import RequestFactory from dojo.models import Engagement, Finding, Product, Product_Type, Test, Test_Type, User -from dojo.product.views import finding_queries +from dojo.product.ui.views import finding_queries from .dojo_test_case import DojoTestCase diff --git a/unittests/test_product_type_counts.py b/unittests/test_product_type_counts.py index 5bac04f3d1c..9ea01cfdcc3 100644 --- a/unittests/test_product_type_counts.py +++ b/unittests/test_product_type_counts.py @@ -1,5 +1,5 @@ from dojo.models import Product, Product_Type -from dojo.product_type.views import prefetch_for_product_type +from dojo.product_type.ui.views import prefetch_for_product_type from unittests.dojo_test_case import DojoTestCase, versioned_fixtures diff --git a/unittests/test_query_utils.py b/unittests/test_query_utils.py index e953efd1df9..5e98b507fac 100644 --- a/unittests/test_query_utils.py +++ b/unittests/test_query_utils.py @@ -1,6 +1,6 @@ from django.db.models import Count -from dojo.engagement.views import prefetch_for_view_tests +from dojo.engagement.ui.views import prefetch_for_view_tests from dojo.models import Finding, Test from unittests.dojo_test_case import DojoTestCase, versioned_fixtures diff --git a/unittests/test_rest_framework.py b/unittests/test_rest_framework.py index 4455dba1374..f82fe0a10fc 100644 --- a/unittests/test_rest_framework.py +++ b/unittests/test_rest_framework.py @@ -33,20 +33,13 @@ ) from rest_framework.test import APIClient +from dojo.announcement.api.views import AnnouncementViewSet from dojo.api_v2.mixins import DeletePreviewModelMixin from dojo.api_v2.prefetch import PrefetchListMixin, PrefetchRetrieveMixin from dojo.api_v2.prefetch.utils import get_prefetchable_fields from dojo.api_v2.views import ( - AnnouncementViewSet, AppAnalysisViewSet, - BurpRawRequestResponseViewSet, ConfigurationPermissionViewSet, - DevelopmentEnvironmentViewSet, - EndpointStatusViewSet, - EndPointViewSet, - EngagementViewSet, - FindingTemplatesViewSet, - FindingViewSet, ImportLanguagesView, ImportScanView, JiraInstanceViewSet, @@ -56,24 +49,21 @@ LanguageViewSet, NotesViewSet, NoteTypeViewSet, - ProductAPIScanConfigurationViewSet, - ProductTypeViewSet, - ProductViewSet, - RiskAcceptanceViewSet, SonarqubeIssueViewSet, - TestsViewSet, - TestTypesViewSet, - ToolConfigurationsViewSet, - ToolProductSettingsViewSet, - ToolTypesViewSet, - UserContactInfoViewSet, - UsersViewSet, ) from dojo.asset.api.views import ( AssetAPIScanConfigurationViewSet, AssetViewSet, ) from dojo.authorization.roles_permissions import Permissions, permission_to_action +from dojo.development_environment.api.views import DevelopmentEnvironmentViewSet +from dojo.endpoint.api.views import EndpointStatusViewSet, EndPointViewSet +from dojo.engagement.api.views import EngagementViewSet +from dojo.finding.api.views import ( + BurpRawRequestResponseViewSet, + FindingTemplatesViewSet, + FindingViewSet, +) from dojo.location.api.endpoint_compat import V3EndpointCompatibleViewSet, V3EndpointStatusCompatibleViewSet from dojo.location.api.views import LocationFindingReferenceViewSet, LocationProductReferenceViewSet, LocationViewSet from dojo.location.models import Location, LocationFindingReference, LocationProductReference @@ -116,8 +106,16 @@ from dojo.organization.api.views import ( OrganizationViewSet, ) +from dojo.product.api.views import ProductAPIScanConfigurationViewSet, ProductViewSet +from dojo.product_type.api.views import ProductTypeViewSet +from dojo.risk_acceptance.api.views import RiskAcceptanceViewSet +from dojo.test.api.views import TestsViewSet, TestTypesViewSet +from dojo.tool_config.api.views import ToolConfigurationsViewSet +from dojo.tool_product.api.views import ToolProductSettingsViewSet +from dojo.tool_type.api.views import ToolTypesViewSet from dojo.url.api.views import URLViewSet from dojo.url.models import URL +from dojo.user.api.views import UserContactInfoViewSet, UsersViewSet from .dojo_test_case import ( DojoAPITestCase, diff --git a/unittests/test_survey_forms.py b/unittests/test_survey_forms.py index 9526c44424a..1945f2d2245 100644 --- a/unittests/test_survey_forms.py +++ b/unittests/test_survey_forms.py @@ -1,6 +1,6 @@ import json -from dojo.forms import MultiExampleField, MultiWidgetBasic +from dojo.survey.ui.forms import MultiExampleField, MultiWidgetBasic from unittests.dojo_test_case import DojoTestCase diff --git a/unittests/test_test_type_active_toggle.py b/unittests/test_test_type_active_toggle.py index 1d0e8a55644..dd2e98d3f04 100644 --- a/unittests/test_test_type_active_toggle.py +++ b/unittests/test_test_type_active_toggle.py @@ -1,7 +1,7 @@ from django.test import TestCase -from dojo.filters import FindingFilter +from dojo.finding.ui.filters import FindingFilter from dojo.models import Test_Type from dojo.utils import get_visible_scan_types diff --git a/unittests/test_user_ui_timestamps.py b/unittests/test_user_ui_timestamps.py index e2296368d14..ad2b7ebe145 100644 --- a/unittests/test_user_ui_timestamps.py +++ b/unittests/test_user_ui_timestamps.py @@ -49,7 +49,7 @@ def test_change_password_stamps_password_last_reset(self): user.save() self.client.force_login(user) - with patch("dojo.user.views.now", return_value=fixed): + with patch("dojo.user.ui.views.now", return_value=fixed): resp = self.client.post( reverse("change_password"), data={ @@ -74,7 +74,7 @@ def test_password_reset_confirm_stamps_password_last_reset(self): token = default_token_generator.make_token(user) url = reverse("password_reset_confirm", kwargs={"uidb64": uidb64, "token": token}) - with patch("dojo.user.views.now", return_value=fixed): + with patch("dojo.user.ui.views.now", return_value=fixed): # Django's PasswordResetConfirmView typically requires a GET to the tokenized URL, # which sets a session token and redirects to the "set-password" URL. resp_get = self.client.get(url)