From 71891344fecdedd303b158cc0f463f5bee5137f2 Mon Sep 17 00:00:00 2001 From: NWarila <33955773+NWarila@users.noreply.github.com> Date: Tue, 2 Jun 2026 19:23:58 +0000 Subject: [PATCH] docs: add SECURITY.md, ADRs, diagram, and security badge Add SECURITY.md with supported-version policy, private vulnerability reporting via the Security tab, and in/out-of-scope classification. Add docs/decision-records/0001 (standalone stdlib-only scripts) and 0002 (pull-based manifest-driven template sync), capturing decisions already shipped and reasoned in PLAN.md. Add docs/diagrams/qa-template-sync-flow.mmd showing the QA and template-sync flow from scripts/ through auto-release to downstream sync PRs. Add Security Policy badge to the README badge row alongside the existing CI, Coverage, Python, Platform, and License badges. Update .gitignore allowlist to track SECURITY.md and the new docs/ directory tree. --- .gitignore | 5 + README.md | 1 + SECURITY.md | 59 +++++++++ ...-scripts-are-standalone-and-stdlib-only.md | 111 ++++++++++++++++ ...ull-based-manifest-driven-template-sync.md | 125 ++++++++++++++++++ docs/diagrams/qa-template-sync-flow.mmd | 39 ++++++ 6 files changed, 340 insertions(+) create mode 100644 SECURITY.md create mode 100644 docs/decision-records/0001-scripts-are-standalone-and-stdlib-only.md create mode 100644 docs/decision-records/0002-pull-based-manifest-driven-template-sync.md create mode 100644 docs/diagrams/qa-template-sync-flow.mmd diff --git a/.gitignore b/.gitignore index 5fb586a..9e42eb3 100644 --- a/.gitignore +++ b/.gitignore @@ -12,8 +12,13 @@ !/LICENSE !/README.md !/PLAN.md +!/SECURITY.md !/sync-manifest.json +# Allow docs directory and its contents. +!/docs/ +!/docs/** + # Allow the org-standard automation and reference directories. !/.github/ !/.github/** diff --git a/README.md b/README.md index 78d69ab..2eba810 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,7 @@ [![Coverage](https://codecov.io/gh/NWarila/python-template/graph/badge.svg)](https://codecov.io/gh/NWarila/python-template) [![Python](https://img.shields.io/badge/python-%E2%89%A53.11-3776ab?logo=python&logoColor=white)](https://www.python.org) [![Platform](https://img.shields.io/badge/platform-linux%20%7C%20macos%20%7C%20windows-lightgrey)](https://github.com/NWarila/python-template) +[![Security Policy](https://img.shields.io/badge/security-policy-informational)](SECURITY.md) [![License](https://img.shields.io/github/license/NWarila/python-template)](LICENSE) Reusable Python quality-gate scripts, a reusable CI workflow, and reference configurations that define a consistent developer experience across all Python repositories in the **NWarila** GitHub account. diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..46aa644 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,59 @@ +# Security Policy + +## Reporting a vulnerability + +**Do not file public issues for security vulnerabilities.** + +### Preferred: GitHub private vulnerability reporting + +Use [GitHub's private vulnerability reporting](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing-information-about-vulnerabilities/privately-reporting-a-security-vulnerability) to report vulnerabilities directly through the affected repository's Security tab. + +### Fallback contact + +If private vulnerability reporting is not available on the affected repository, contact the maintainer through their [GitHub profile](https://github.com/NWarila). + +## What to include + +- Description of the vulnerability +- Steps to reproduce or proof of concept +- Affected repository and version (or "latest default branch" if unsure) +- Potential impact + +## Response timeline + +| Stage | Target | +|-------|--------| +| Initial acknowledgement | 7 business days | +| Validation | 14 days | +| Remediation or mitigation | 90 days when reasonable | + +These are targets, not guarantees. Complex issues may take longer. You will be kept informed of progress. + +## Supported versions + +Only the latest release of `python-template` is supported. Downstream repositories that pin an older release tag should upgrade to receive fixes. The `v1` floating tag always resolves to the current supported release. + +## Scope + +### In scope + +- Vulnerabilities in scripts, workflows, or reference configurations maintained in this repository +- Misconfigurations in GitHub Actions workflows that could lead to secret exposure or privilege escalation +- Supply-chain weaknesses introduced by synced files that propagate to downstream repositories + +### Out of scope + +- Vulnerabilities in third-party tools (`ruff`, `mypy`, `pytest`, `pip-audit`, etc.) — report those to the respective upstream projects +- Social engineering attacks +- Denial of service attacks +- Issues in archived repositories + +## Coordinated disclosure + +We follow coordinated disclosure practices. We ask that you: + +- Give us reasonable time to investigate and address the issue before public disclosure +- Act in good faith and avoid accessing or modifying data that does not belong to you +- Do not exploit the vulnerability beyond what is necessary to demonstrate it + +We will credit researchers who report valid vulnerabilities unless they prefer to remain anonymous. diff --git a/docs/decision-records/0001-scripts-are-standalone-and-stdlib-only.md b/docs/decision-records/0001-scripts-are-standalone-and-stdlib-only.md new file mode 100644 index 0000000..9d61bc2 --- /dev/null +++ b/docs/decision-records/0001-scripts-are-standalone-and-stdlib-only.md @@ -0,0 +1,111 @@ +# ADR-0001: QA Scripts Are Standalone and Stdlib-Only + +| Field | Value | +| -------------- | --------------------------------------- | +| Status | Accepted | +| Date | 2026-04-08 | +| Authors | Nick Warila (@NWarila) | +| Decision-maker | Nick Warila (sole portfolio maintainer) | +| Consulted | Python packaging and tooling docs. | +| Informed | Downstream Python repositories. | +| Reversibility | Medium | +| Review-by | N/A (Accepted) | + +## TL;DR + +Each QA script (`check_*.py`, `qa.py`) is a fully self-contained Python file with no shared helper module and no imports beyond the standard library. Cross-script logic duplication is acceptable. + +## Context and Problem Statement + +The template ships QA scripts that downstream repositories sync into `.github/scripts/`. Those scripts must be runnable on any OS (Linux, macOS, Windows) from a plain activated venv, regardless of whether the repo uses `pip`+`venv` or `uv`. They must also be readable and debuggable in isolation without needing to understand the whole template. + +Two design paths were considered: + +1. A shared helper module (`_common.py`) imported by each script, reducing duplication. +2. Each script as a self-contained file, duplicating small helpers (config reading, path resolution) independently. + +## Decision Drivers + +1. Downstream repos sync only the files they need, not the whole template. A shared module would require every consumer to also sync `_common.py` and keep it aligned. +2. Scripts must run from two different depths (`scripts/` and `.github/scripts/`) without path assumptions breaking. +3. Each script should be independently readable without tracing imports. +4. No third-party install overhead from template infrastructure should land in downstream dev dependencies. + +## Considered Options + +1. Standalone, stdlib-only scripts — no shared module, no third-party deps. +2. Shared helper module (`_common.py`) imported by each check script. +3. Template Python package installed as a dev dependency. + +## Decision Outcome + +Chosen option: **Option 1, standalone stdlib-only scripts.** + +Each `check_*.py` file reads `pyproject.toml` via `tomllib` (stdlib since 3.11), resolves paths, invokes the tool via `subprocess`, and reports results — all inline. The `~10-line config-reading pattern is duplicated across scripts rather than shared. + +Scripts shell out to the actual tools (`ruff`, `mypy`, `pytest`, `pip-audit`, `codespell`, `build`, `twine`) and do not import them. This means the scripts carry zero non-stdlib import dependencies of their own. + +## Pros and Cons of the Options + +### Option 1: Standalone stdlib-only scripts + +- Good, because each file can be read, copied, and debugged in isolation. +- Good, because downstream repos need no extra sync target beyond the scripts themselves. +- Good, because no third-party imports pollute downstream dev-dependency graphs. +- Bad, because `~10-line` patterns (config reading, path resolution) are duplicated. + +### Option 2: Shared helper module + +- Good, because it reduces duplication. +- Bad, because downstream sync must include `_common.py` and version it carefully. +- Bad, because a change to `_common.py` affects every script simultaneously, widening the blast radius. + +### Option 3: Template as installable package + +- Good, because version management is explicit via `pip install`. +- Bad, because it adds a template-infrastructure install step to every downstream developer setup. +- Bad, because it conflates template tooling with project dependencies. + +## Confirmation + +1. No `check_*.py` or `qa.py` file contains a cross-script import. +2. The only stdlib module used for config reading is `tomllib` (Python 3.11+), consistent with the org's minimum supported version (ADR-0003). +3. `mypy` and `ruff` pass on all scripts in CI (`template-ci.yml`). + +## Consequences + +### Positive + +- Any script can be extracted, read, or replaced independently. +- Downstream repos remain decoupled from template internals. + +### Negative + +- Config-reading code is duplicated across scripts; a structural change to how `pyproject.toml` sections are read must be made in each file. + +### Neutral + +- The duplication is bounded: the pattern is ~10 lines and changes rarely. + +## Assumptions + +1. Python 3.11 remains the minimum version for the org (see PLAN.md Resolved Decision 15). +2. The set of check scripts stays small enough that per-file duplication is manageable. + +## Supersedes + +None. + +## Superseded by + +None (current). + +## Implementing PRs + +- Initial implementation: standalone `check_*.py` scripts in `scripts/`. +- Confirmed in Phase 1 cleanup: every `PROJECT_ROOT` resolution was migrated to walk-up-to-`pyproject.toml` (PR #5) after the `SCRIPT_DIR.parent` assumption broke at `.github/scripts/` depth. + +## Related ADRs + +- ADR-0002: Pull-based manifest-driven template sync +- ADR-0003: Python minimum version floor diff --git a/docs/decision-records/0002-pull-based-manifest-driven-template-sync.md b/docs/decision-records/0002-pull-based-manifest-driven-template-sync.md new file mode 100644 index 0000000..faefe59 --- /dev/null +++ b/docs/decision-records/0002-pull-based-manifest-driven-template-sync.md @@ -0,0 +1,125 @@ +# ADR-0002: Pull-Based, Manifest-Driven Template Sync + +| Field | Value | +| -------------- | --------------------------------------- | +| Status | Accepted | +| Date | 2026-04-08 | +| Authors | Nick Warila (@NWarila) | +| Decision-maker | Nick Warila (sole portfolio maintainer) | +| Consulted | GitHub Actions reusable workflow docs, git submodule docs. | +| Informed | Downstream Python repositories. | +| Reversibility | Medium | +| Review-by | N/A (Accepted) | + +## TL;DR + +Template updates are delivered to downstream repositories as reviewable pull requests triggered by each repo on its own schedule, not pushed from the template. File mappings are defined in a machine-readable manifest (`sync-manifest.json`), not hardcoded in workflow YAML. + +## Context and Problem Statement + +A central Python QA template must deliver updated scripts, configs, and reference files to downstream repositories without requiring cross-repo credentials, without bypassing PR review, and without coupling downstream release cadences to the template. + +Three distribution mechanisms were evaluated: git submodules, a versioned Python package, and a manifest-driven file-sync workflow. + +An earlier prototype used a push-based `sync-downstream.yml` workflow that required a fine-grained PAT (`TEMPLATE_SYNC_PAT`) with write access to every downstream repo. That PAT was never configured, making the workflow inoperative, and the design is fundamentally incompatible with keeping each repo's update pace self-controlled. + +## Decision Drivers + +1. Downstream repos must be able to review and merge template updates at their own pace. +2. No cross-repo write credentials (PATs or push permissions) from the template side. +3. Both scripts and config files must be synced; package installs only handle code. +4. Some files need partial replacement (marker-preserving merge) rather than full overwrite. +5. Every sync PR must be attributable to a specific template release tag for audit. + +## Considered Options + +1. Push-based workflow with cross-repo PAT. +2. Git submodule pinned to a template commit. +3. Versioned Python package installed as a dev dependency. +4. Pull-based reusable workflow with `sync-manifest.json`. + +## Decision Outcome + +Chosen option: **Option 4, pull-based reusable workflow with `sync-manifest.json`.** + +The template publishes a GitHub release whenever `scripts/` changes (via `auto-release.yml`). Each downstream repository owns a thin `template-sync.yml` wrapper that calls the reusable `self-update.yml` from a pinned template release. The reusable workflow checks for a newer release, reads `sync-manifest.json` for source-to-destination mappings, and opens a pull request using the downstream repo's own `GITHUB_TOKEN`. No cross-repo credentials are required. + +`sync-manifest.json` declares per-file `mode`: `overwrite` for fully managed files (scripts, pre-commit config, VSCode settings) and `marker-preserve` for files where template-owned regions must be updated while repo-specific sections are retained (e.g., `tasks.json`). + +## Pros and Cons of the Options + +### Option 1: Push-based workflow with cross-repo PAT + +- Good, because the template controls delivery timing. +- Bad, because it requires a PAT with write access to every downstream repo. +- Bad, because PATs are tied to a personal account and create a single point of failure. +- Bad, because downstream repos cannot defer or review updates before they land. + +### Option 2: Git submodule + +- Good, because it uses native git tooling. +- Bad, because submodules pin to a commit, not a release; there is no selective file mapping. +- Bad, because downstream repos get no PR with migration notes — the submodule bump is one diff line. +- Bad, because submodule workflows are brittle across clone contexts. + +### Option 3: Versioned Python package + +- Good, because version management is explicit via `pip install`. +- Bad, because package installs cannot deliver non-Python config files (`.gitignore`, VSCode JSON, pre-commit YAML). +- Bad, because it adds a template-infrastructure install to every downstream developer setup. + +### Option 4: Pull-based reusable workflow with manifest + +- Good, because downstream repos retain full review control. +- Good, because no cross-repo credentials are required from the template side. +- Good, because scripts and config files are both covered by the manifest. +- Good, because per-file merge strategies (`overwrite` vs `marker-preserve`) are explicit. +- Good, because every sync PR references the source release tag. +- Bad, because downstream repos must own a thin wrapper workflow and can fall behind if they skip updates. + +## Confirmation + +1. `sync-manifest.json` at the repository root defines all synced source-to-destination mappings. +2. `self-update.yml` supports `workflow_call` so downstream repos can call it as a reusable workflow. +3. The reusable workflow uses `GITHUB_TOKEN` (downstream repo's own token) for PR creation — no PAT. +4. `auto-release.yml` creates a new release when `scripts/` changes merge to `main`. +5. This repo dogfoods the sync mechanism via a nightly scheduled run of `self-update.yml`. + +## Consequences + +### Positive + +- Each downstream repo controls its own update cadence. +- Template releases are reviewable and rollback-able. +- No cross-repo credentials are held by the template. + +### Negative + +- Downstream repos can drift from the template if they skip sync PRs. +- Workflow-created PRs do not trigger CI automatically (see `docs/GITHUB_TOKEN_LIMITATION.md`). + +### Neutral + +- The per-file manifest is the single source of truth for what the template owns; adding a new synced file requires a manifest entry and a `.gitignore` allowlist update in downstream repos. + +## Assumptions + +1. Downstream repos maintain their own `template-sync.yml` wrapper workflow. +2. The template maintains backward-compatible `self-update.yml` behavior within a major version. + +## Supersedes + +- Push-based `sync-downstream.yml` prototype (removed in PR #4). + +## Superseded by + +None (current). + +## Implementing PRs + +- PR #4: Replace push-based sync with pull-based reusable workflow. + +## Related ADRs + +- ADR-0001: QA scripts are standalone and stdlib-only +- ADR-0004: Use Renovate for dependency updates (`.github/renovate.json5`) diff --git a/docs/diagrams/qa-template-sync-flow.mmd b/docs/diagrams/qa-template-sync-flow.mmd new file mode 100644 index 0000000..f2e143c --- /dev/null +++ b/docs/diagrams/qa-template-sync-flow.mmd @@ -0,0 +1,39 @@ +--- +title: QA and Template-Sync Flow +--- +flowchart TD + subgraph python-template ["NWarila/python-template"] + SRC["scripts/ (canonical source)"] + REL["auto-release.yml\ncreates release tag"] + SELFUP["self-update.yml\n(nightly, pull released files)"] + TPLCI["template-ci.yml\nruns .github/scripts/"] + GHSCRIPTS[".github/scripts/\n(released copies)"] + MANIFEST["sync-manifest.json\n(source → dest mappings)"] + end + + subgraph downstream ["Downstream Repo"] + DSCRIPTS[".github/scripts/\n(synced from release)"] + DSYNC["template-sync.yml\ncalls self-update.yml@v1"] + DCI["repo-ci.yml\ncalls python-qa.yml@v1"] + DPYPROJECT["pyproject.toml\n(tool config: ruff, mypy, pytest)"] + DLOCAL["Developer local run\nqa.py / VSCode tasks"] + end + + subgraph GitHub_Actions ["GitHub Actions (reusable)"] + PYQA["python-qa.yml\nlint / types / tests / security\nspelling / package / ci-passed"] + end + + SRC -->|"merge to main"| REL + REL -->|"new tag"| SELFUP + SELFUP -->|"reads"| MANIFEST + SELFUP -->|"writes"| GHSCRIPTS + GHSCRIPTS --> TPLCI + + SELFUP -.->|"reusable via uses:"| DSYNC + DSYNC -->|"PR with synced files"| DSCRIPTS + DSCRIPTS --> DLOCAL + DSCRIPTS --> DCI + + DCI -->|"calls"| PYQA + PYQA -->|"checks out caller repo\nruns .github/scripts/"| DSCRIPTS + DPYPROJECT -->|"tool config read at runtime"| PYQA