diff --git a/.gitignore b/.gitignore index 5fb586a..9e42eb3 100644 --- a/.gitignore +++ b/.gitignore @@ -12,8 +12,13 @@ !/LICENSE !/README.md !/PLAN.md +!/SECURITY.md !/sync-manifest.json +# Allow docs directory and its contents. +!/docs/ +!/docs/** + # Allow the org-standard automation and reference directories. !/.github/ !/.github/** diff --git a/README.md b/README.md index 78d69ab..2eba810 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,7 @@ [![Coverage](https://codecov.io/gh/NWarila/python-template/graph/badge.svg)](https://codecov.io/gh/NWarila/python-template) [![Python](https://img.shields.io/badge/python-%E2%89%A53.11-3776ab?logo=python&logoColor=white)](https://www.python.org) [![Platform](https://img.shields.io/badge/platform-linux%20%7C%20macos%20%7C%20windows-lightgrey)](https://github.com/NWarila/python-template) +[![Security Policy](https://img.shields.io/badge/security-policy-informational)](SECURITY.md) [![License](https://img.shields.io/github/license/NWarila/python-template)](LICENSE) Reusable Python quality-gate scripts, a reusable CI workflow, and reference configurations that define a consistent developer experience across all Python repositories in the **NWarila** GitHub account. diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..46aa644 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,59 @@ +# Security Policy + +## Reporting a vulnerability + +**Do not file public issues for security vulnerabilities.** + +### Preferred: GitHub private vulnerability reporting + +Use [GitHub's private vulnerability reporting](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing-information-about-vulnerabilities/privately-reporting-a-security-vulnerability) to report vulnerabilities directly through the affected repository's Security tab. + +### Fallback contact + +If private vulnerability reporting is not available on the affected repository, contact the maintainer through their [GitHub profile](https://github.com/NWarila). + +## What to include + +- Description of the vulnerability +- Steps to reproduce or proof of concept +- Affected repository and version (or "latest default branch" if unsure) +- Potential impact + +## Response timeline + +| Stage | Target | +|-------|--------| +| Initial acknowledgement | 7 business days | +| Validation | 14 days | +| Remediation or mitigation | 90 days when reasonable | + +These are targets, not guarantees. Complex issues may take longer. You will be kept informed of progress. + +## Supported versions + +Only the latest release of `python-template` is supported. Downstream repositories that pin an older release tag should upgrade to receive fixes. The `v1` floating tag always resolves to the current supported release. + +## Scope + +### In scope + +- Vulnerabilities in scripts, workflows, or reference configurations maintained in this repository +- Misconfigurations in GitHub Actions workflows that could lead to secret exposure or privilege escalation +- Supply-chain weaknesses introduced by synced files that propagate to downstream repositories + +### Out of scope + +- Vulnerabilities in third-party tools (`ruff`, `mypy`, `pytest`, `pip-audit`, etc.) — report those to the respective upstream projects +- Social engineering attacks +- Denial of service attacks +- Issues in archived repositories + +## Coordinated disclosure + +We follow coordinated disclosure practices. We ask that you: + +- Give us reasonable time to investigate and address the issue before public disclosure +- Act in good faith and avoid accessing or modifying data that does not belong to you +- Do not exploit the vulnerability beyond what is necessary to demonstrate it + +We will credit researchers who report valid vulnerabilities unless they prefer to remain anonymous. diff --git a/docs/decision-records/0001-scripts-are-standalone-and-stdlib-only.md b/docs/decision-records/0001-scripts-are-standalone-and-stdlib-only.md new file mode 100644 index 0000000..9d61bc2 --- /dev/null +++ b/docs/decision-records/0001-scripts-are-standalone-and-stdlib-only.md @@ -0,0 +1,111 @@ +# ADR-0001: QA Scripts Are Standalone and Stdlib-Only + +| Field | Value | +| -------------- | --------------------------------------- | +| Status | Accepted | +| Date | 2026-04-08 | +| Authors | Nick Warila (@NWarila) | +| Decision-maker | Nick Warila (sole portfolio maintainer) | +| Consulted | Python packaging and tooling docs. | +| Informed | Downstream Python repositories. | +| Reversibility | Medium | +| Review-by | N/A (Accepted) | + +## TL;DR + +Each QA script (`check_*.py`, `qa.py`) is a fully self-contained Python file with no shared helper module and no imports beyond the standard library. Cross-script logic duplication is acceptable. + +## Context and Problem Statement + +The template ships QA scripts that downstream repositories sync into `.github/scripts/`. Those scripts must be runnable on any OS (Linux, macOS, Windows) from a plain activated venv, regardless of whether the repo uses `pip`+`venv` or `uv`. They must also be readable and debuggable in isolation without needing to understand the whole template. + +Two design paths were considered: + +1. A shared helper module (`_common.py`) imported by each script, reducing duplication. +2. Each script as a self-contained file, duplicating small helpers (config reading, path resolution) independently. + +## Decision Drivers + +1. Downstream repos sync only the files they need, not the whole template. A shared module would require every consumer to also sync `_common.py` and keep it aligned. +2. Scripts must run from two different depths (`scripts/` and `.github/scripts/`) without path assumptions breaking. +3. Each script should be independently readable without tracing imports. +4. No third-party install overhead from template infrastructure should land in downstream dev dependencies. + +## Considered Options + +1. Standalone, stdlib-only scripts — no shared module, no third-party deps. +2. Shared helper module (`_common.py`) imported by each check script. +3. Template Python package installed as a dev dependency. + +## Decision Outcome + +Chosen option: **Option 1, standalone stdlib-only scripts.** + +Each `check_*.py` file reads `pyproject.toml` via `tomllib` (stdlib since 3.11), resolves paths, invokes the tool via `subprocess`, and reports results — all inline. The `~10-line config-reading pattern is duplicated across scripts rather than shared. + +Scripts shell out to the actual tools (`ruff`, `mypy`, `pytest`, `pip-audit`, `codespell`, `build`, `twine`) and do not import them. This means the scripts carry zero non-stdlib import dependencies of their own. + +## Pros and Cons of the Options + +### Option 1: Standalone stdlib-only scripts + +- Good, because each file can be read, copied, and debugged in isolation. +- Good, because downstream repos need no extra sync target beyond the scripts themselves. +- Good, because no third-party imports pollute downstream dev-dependency graphs. +- Bad, because `~10-line` patterns (config reading, path resolution) are duplicated. + +### Option 2: Shared helper module + +- Good, because it reduces duplication. +- Bad, because downstream sync must include `_common.py` and version it carefully. +- Bad, because a change to `_common.py` affects every script simultaneously, widening the blast radius. + +### Option 3: Template as installable package + +- Good, because version management is explicit via `pip install`. +- Bad, because it adds a template-infrastructure install step to every downstream developer setup. +- Bad, because it conflates template tooling with project dependencies. + +## Confirmation + +1. No `check_*.py` or `qa.py` file contains a cross-script import. +2. The only stdlib module used for config reading is `tomllib` (Python 3.11+), consistent with the org's minimum supported version (ADR-0003). +3. `mypy` and `ruff` pass on all scripts in CI (`template-ci.yml`). + +## Consequences + +### Positive + +- Any script can be extracted, read, or replaced independently. +- Downstream repos remain decoupled from template internals. + +### Negative + +- Config-reading code is duplicated across scripts; a structural change to how `pyproject.toml` sections are read must be made in each file. + +### Neutral + +- The duplication is bounded: the pattern is ~10 lines and changes rarely. + +## Assumptions + +1. Python 3.11 remains the minimum version for the org (see PLAN.md Resolved Decision 15). +2. The set of check scripts stays small enough that per-file duplication is manageable. + +## Supersedes + +None. + +## Superseded by + +None (current). + +## Implementing PRs + +- Initial implementation: standalone `check_*.py` scripts in `scripts/`. +- Confirmed in Phase 1 cleanup: every `PROJECT_ROOT` resolution was migrated to walk-up-to-`pyproject.toml` (PR #5) after the `SCRIPT_DIR.parent` assumption broke at `.github/scripts/` depth. + +## Related ADRs + +- ADR-0002: Pull-based manifest-driven template sync +- ADR-0003: Python minimum version floor diff --git a/docs/decision-records/0002-pull-based-manifest-driven-template-sync.md b/docs/decision-records/0002-pull-based-manifest-driven-template-sync.md new file mode 100644 index 0000000..faefe59 --- /dev/null +++ b/docs/decision-records/0002-pull-based-manifest-driven-template-sync.md @@ -0,0 +1,125 @@ +# ADR-0002: Pull-Based, Manifest-Driven Template Sync + +| Field | Value | +| -------------- | --------------------------------------- | +| Status | Accepted | +| Date | 2026-04-08 | +| Authors | Nick Warila (@NWarila) | +| Decision-maker | Nick Warila (sole portfolio maintainer) | +| Consulted | GitHub Actions reusable workflow docs, git submodule docs. | +| Informed | Downstream Python repositories. | +| Reversibility | Medium | +| Review-by | N/A (Accepted) | + +## TL;DR + +Template updates are delivered to downstream repositories as reviewable pull requests triggered by each repo on its own schedule, not pushed from the template. File mappings are defined in a machine-readable manifest (`sync-manifest.json`), not hardcoded in workflow YAML. + +## Context and Problem Statement + +A central Python QA template must deliver updated scripts, configs, and reference files to downstream repositories without requiring cross-repo credentials, without bypassing PR review, and without coupling downstream release cadences to the template. + +Three distribution mechanisms were evaluated: git submodules, a versioned Python package, and a manifest-driven file-sync workflow. + +An earlier prototype used a push-based `sync-downstream.yml` workflow that required a fine-grained PAT (`TEMPLATE_SYNC_PAT`) with write access to every downstream repo. That PAT was never configured, making the workflow inoperative, and the design is fundamentally incompatible with keeping each repo's update pace self-controlled. + +## Decision Drivers + +1. Downstream repos must be able to review and merge template updates at their own pace. +2. No cross-repo write credentials (PATs or push permissions) from the template side. +3. Both scripts and config files must be synced; package installs only handle code. +4. Some files need partial replacement (marker-preserving merge) rather than full overwrite. +5. Every sync PR must be attributable to a specific template release tag for audit. + +## Considered Options + +1. Push-based workflow with cross-repo PAT. +2. Git submodule pinned to a template commit. +3. Versioned Python package installed as a dev dependency. +4. Pull-based reusable workflow with `sync-manifest.json`. + +## Decision Outcome + +Chosen option: **Option 4, pull-based reusable workflow with `sync-manifest.json`.** + +The template publishes a GitHub release whenever `scripts/` changes (via `auto-release.yml`). Each downstream repository owns a thin `template-sync.yml` wrapper that calls the reusable `self-update.yml` from a pinned template release. The reusable workflow checks for a newer release, reads `sync-manifest.json` for source-to-destination mappings, and opens a pull request using the downstream repo's own `GITHUB_TOKEN`. No cross-repo credentials are required. + +`sync-manifest.json` declares per-file `mode`: `overwrite` for fully managed files (scripts, pre-commit config, VSCode settings) and `marker-preserve` for files where template-owned regions must be updated while repo-specific sections are retained (e.g., `tasks.json`). + +## Pros and Cons of the Options + +### Option 1: Push-based workflow with cross-repo PAT + +- Good, because the template controls delivery timing. +- Bad, because it requires a PAT with write access to every downstream repo. +- Bad, because PATs are tied to a personal account and create a single point of failure. +- Bad, because downstream repos cannot defer or review updates before they land. + +### Option 2: Git submodule + +- Good, because it uses native git tooling. +- Bad, because submodules pin to a commit, not a release; there is no selective file mapping. +- Bad, because downstream repos get no PR with migration notes — the submodule bump is one diff line. +- Bad, because submodule workflows are brittle across clone contexts. + +### Option 3: Versioned Python package + +- Good, because version management is explicit via `pip install`. +- Bad, because package installs cannot deliver non-Python config files (`.gitignore`, VSCode JSON, pre-commit YAML). +- Bad, because it adds a template-infrastructure install to every downstream developer setup. + +### Option 4: Pull-based reusable workflow with manifest + +- Good, because downstream repos retain full review control. +- Good, because no cross-repo credentials are required from the template side. +- Good, because scripts and config files are both covered by the manifest. +- Good, because per-file merge strategies (`overwrite` vs `marker-preserve`) are explicit. +- Good, because every sync PR references the source release tag. +- Bad, because downstream repos must own a thin wrapper workflow and can fall behind if they skip updates. + +## Confirmation + +1. `sync-manifest.json` at the repository root defines all synced source-to-destination mappings. +2. `self-update.yml` supports `workflow_call` so downstream repos can call it as a reusable workflow. +3. The reusable workflow uses `GITHUB_TOKEN` (downstream repo's own token) for PR creation — no PAT. +4. `auto-release.yml` creates a new release when `scripts/` changes merge to `main`. +5. This repo dogfoods the sync mechanism via a nightly scheduled run of `self-update.yml`. + +## Consequences + +### Positive + +- Each downstream repo controls its own update cadence. +- Template releases are reviewable and rollback-able. +- No cross-repo credentials are held by the template. + +### Negative + +- Downstream repos can drift from the template if they skip sync PRs. +- Workflow-created PRs do not trigger CI automatically (see `docs/GITHUB_TOKEN_LIMITATION.md`). + +### Neutral + +- The per-file manifest is the single source of truth for what the template owns; adding a new synced file requires a manifest entry and a `.gitignore` allowlist update in downstream repos. + +## Assumptions + +1. Downstream repos maintain their own `template-sync.yml` wrapper workflow. +2. The template maintains backward-compatible `self-update.yml` behavior within a major version. + +## Supersedes + +- Push-based `sync-downstream.yml` prototype (removed in PR #4). + +## Superseded by + +None (current). + +## Implementing PRs + +- PR #4: Replace push-based sync with pull-based reusable workflow. + +## Related ADRs + +- ADR-0001: QA scripts are standalone and stdlib-only +- ADR-0004: Use Renovate for dependency updates (`.github/renovate.json5`) diff --git a/docs/diagrams/qa-template-sync-flow.mmd b/docs/diagrams/qa-template-sync-flow.mmd new file mode 100644 index 0000000..f2e143c --- /dev/null +++ b/docs/diagrams/qa-template-sync-flow.mmd @@ -0,0 +1,39 @@ +--- +title: QA and Template-Sync Flow +--- +flowchart TD + subgraph python-template ["NWarila/python-template"] + SRC["scripts/ (canonical source)"] + REL["auto-release.yml\ncreates release tag"] + SELFUP["self-update.yml\n(nightly, pull released files)"] + TPLCI["template-ci.yml\nruns .github/scripts/"] + GHSCRIPTS[".github/scripts/\n(released copies)"] + MANIFEST["sync-manifest.json\n(source → dest mappings)"] + end + + subgraph downstream ["Downstream Repo"] + DSCRIPTS[".github/scripts/\n(synced from release)"] + DSYNC["template-sync.yml\ncalls self-update.yml@v1"] + DCI["repo-ci.yml\ncalls python-qa.yml@v1"] + DPYPROJECT["pyproject.toml\n(tool config: ruff, mypy, pytest)"] + DLOCAL["Developer local run\nqa.py / VSCode tasks"] + end + + subgraph GitHub_Actions ["GitHub Actions (reusable)"] + PYQA["python-qa.yml\nlint / types / tests / security\nspelling / package / ci-passed"] + end + + SRC -->|"merge to main"| REL + REL -->|"new tag"| SELFUP + SELFUP -->|"reads"| MANIFEST + SELFUP -->|"writes"| GHSCRIPTS + GHSCRIPTS --> TPLCI + + SELFUP -.->|"reusable via uses:"| DSYNC + DSYNC -->|"PR with synced files"| DSCRIPTS + DSCRIPTS --> DLOCAL + DSCRIPTS --> DCI + + DCI -->|"calls"| PYQA + PYQA -->|"checks out caller repo\nruns .github/scripts/"| DSCRIPTS + DPYPROJECT -->|"tool config read at runtime"| PYQA