From 40c05a6747623fbc7a32fd6cd8ce171ba9f6a0fb Mon Sep 17 00:00:00 2001 From: Frank Lesniak Date: Mon, 4 May 2026 16:19:59 -0500 Subject: [PATCH 01/10] Copy reference docs --- .../macOS-imaging-01a-CFP-submission.md | 18 + .../macOS-imaging-03a-bolstered-outline.md | 1742 ++++++++++++++++ .../macOS-imaging-05-recommended-software.md | 615 ++++++ .../macOS-imaging-08c-repo-spec-final.md | 1781 +++++++++++++++++ docs/planning/macOS-imaging-08e-ADRs.md | 290 +++ 5 files changed, 4446 insertions(+) create mode 100644 docs/planning/macOS-imaging-01a-CFP-submission.md create mode 100644 docs/planning/macOS-imaging-03a-bolstered-outline.md create mode 100644 docs/planning/macOS-imaging-05-recommended-software.md create mode 100644 docs/planning/macOS-imaging-08c-repo-spec-final.md create mode 100644 docs/planning/macOS-imaging-08e-ADRs.md diff --git a/docs/planning/macOS-imaging-01a-CFP-submission.md b/docs/planning/macOS-imaging-01a-CFP-submission.md new file mode 100644 index 0000000..c37eb93 --- /dev/null +++ b/docs/planning/macOS-imaging-01a-CFP-submission.md @@ -0,0 +1,18 @@ +## CFP Submission (Submitted and Accepted) + +- Session title: Don't Brick the CEO's Mac: Building and Automating macOS Labs for Risk-Free Policy Testing +- Primary Speaker/Submitter: Frank Lesniak +- Co-Speaker Name: Michael Niehaus +- Description (NOTE: 600 character limit!): + + You have a lab for testing Windows--that's easy--but do you have the same for Macs? Deploying untested macOS policies is a ticking time bomb. One wrong move with FileVault, and you've locked out an executive. This session hands you a safety net: a reproducible macOS VM lab on Apple Silicon, orchestrated with PowerShell 7. We will use Parallels or UTM to spin up VMs, enroll them in Intune, apply policies, and roll back. With live demos and a GitHub starter kit, you'll leave ready to test configurations without the anxiety. +- Session Length: 1 hour 45 minutes, including 30 minutes of “extended Q&A” +- Session Takeaway 1: Analyze the virtualization constraints of Apple Silicon and select the appropriate hypervisor (Parallels vs. UTM) that aligns with your budget and automation needs. +- Session Takeaway 2: Construct a fully automated, reproducible macOS test lab using PowerShell 7 to fetch restore images/install media and control VM states. +- Session Takeaway 3: Execute end-to-end validation of high-risk policies (FileVault, Defender) by enrolling, breaking, and rolling back VMs via script—keeping production safe. +- Session Takeaway 4 (optional): Implement the provided GitHub starter kit immediately to bridge the gap between your Windows automation skills and macOS requirements. +- Primary Technology (choose one): Intune +- Session Type (multi-select): + - Automation + - Provisioning & Deployment + - Setup & Configuration diff --git a/docs/planning/macOS-imaging-03a-bolstered-outline.md b/docs/planning/macOS-imaging-03a-bolstered-outline.md new file mode 100644 index 0000000..0eb7601 --- /dev/null +++ b/docs/planning/macOS-imaging-03a-bolstered-outline.md @@ -0,0 +1,1742 @@ + +# Don't Brick the CEO's Mac: MMSMOA 2026 Session Plan and Speaker Runbook + +## Metadata + +- **Status:** Draft +- **Owner:** Frank Lesniak +- **Last Updated:** 2026-05-04 +- **Scope:** Talk-development working artifact for the MMSMOA 2026 session: "Don't Brick the CEO's Mac: MMSMOA 2026 Session Plan and Speaker Runbook". Captures interim concepting, prioritization, outline, or runbook content for that session; not a final published deliverable. +- **Related:** [Merged repository specification](macOS-imaging-08c-merged.md), [Architecture decision records](macOS-imaging-08e-ADRs.md), [Closed questions archive](macOS-imaging-08d-closed-questions-archive.md), [Repository Copilot Instructions](../../.github/copilot-instructions.md), [Documentation Writing Style](../../.github/instructions/docs.instructions.md) + +## Document Conventions + +- File-path references are written as plain inline code with backticks — for example, `Start-Here.md`. They refer to files inside the GitHub starter kit. If your editor auto-converts `.md` filenames into hyperlinks (some notetaking and writing tools do this silently), undo that conversion before sharing, printing, or pasting this runbook. The intended rendering is plain inline code, not a hyperlink. +- Code samples that include version-specific values use angle-bracket placeholders. Replace each placeholder with the actual pinned value during rehearsal, and keep them current as the event approaches: + - `` and `` for guest builds. + - `` for the demo Mac's host OS. + - `` for the Parallels Desktop release in use. + - `` for the UTM release in use. + - `` for the PowerShell 7.4-or-newer build. + - `` for Microsoft Defender for Endpoint on macOS. + - `` for the snapshot of demo-tenant policies in use. +- Times in the timeline are wall-clock minutes from the start of the session. Demo durations should be treated as keyboard-time targets, not guarantees. +- "Provider" means the VM provider unless explicitly qualified as a media source. Use terms like `-Source Mist` for media acquisition to avoid confusing the media source with the hypervisor provider. +- Evidence examples must be redacted before they appear in slides, recordings, screenshots, or the public repo. Prove that escrow, assignment, and health states exist; do not expose recovery keys, tenant secrets, bearer tokens, app IDs/secrets, private device IDs, tenant names, tenant domains, user principal names, or other user-identifying data. + +## Final Repository Decisions Reflected Here + +The session outline now aligns with the final `macOSLab` repository decisions: + +- The companion repository is `franklesniak/macOSLab`, public, MIT-licensed, and initialized from `franklesniak/copilot-repo-template`. +- The automation floor is PowerShell 7.4 or newer. PowerShell 5.1 compatibility is not part of the talk or repository promise. +- The initial Pester version is 5.7.1. The inherited template's PowerShell CI should use `macos-latest`; Python CI can be removed if no Python sample content remains. +- The live demo target is macOS Tahoe 26.4.1. The repo should also support currently Apple-supported macOS compatibility targets, initially macOS Sequoia 15.7.5 and macOS Sonoma 14.8.5. +- The primary live provider remains Parallels Desktop Pro Edition. UTM remains the provider-swap path. Tart is optional and stubbed in v1 unless explicitly approved later. +- Tart and Orchard are discussed as Fair Source/free-tier tools: Tart's free-tier documentation describes a 100 CPU-core limit, and Orchard's free-tier documentation describes a 4-worker limit. This is practical planning guidance, not legal advice. +- Apple licensing should be framed around Apple-branded host hardware, permitted purposes, and the commonly relevant boundary of up to two additional macOS virtual instances per Apple-branded host under Apple's current macOS terms. Do not present that as blanket enterprise legal advice. +- The public v1 repo should not ship checked-in example screenshots. Local rehearsal screenshots and deck screenshots are still allowed if fully redacted and kept out of the repo unless a later Phase 10 TODO explicitly approves visual artifacts. +- `Reset-IntuneMacLabDevice.ps1` is report-only in v1. It explains candidate Intune, Entra, and Defender cleanup work; it does not retire, soft-delete, or hard-delete cloud records. +- Deferred implementation work must be tracked in root per-phase TODO files, such as `TODO-Phase-04-Media-Acquisition.md`, `TODO-Phase-05-Parallels-Provider.md`, `TODO-Phase-06-UTM-Provider.md`, `TODO-Phase-08-Validation-Loop.md`, and `TODO-Phase-10-Deferred-Work.md`. Omit a file only when that phase has no deferred work. + +## Session Identity + +| Field | Value | +| --- | --- | +| Session title | Don't Brick the CEO's Mac: Building and Automating macOS Labs for Risk-Free Policy Testing | +| Primary speaker | Frank Lesniak | +| Co-speaker | Michael Niehaus | +| Event | Midwest Management Summit at the Mall of America 2026 | +| Total length | 105 minutes | +| Core content | 75 minutes | +| Extended Q&A | 30 minutes | +| Q&A format | Raise-hand questions; mic runner if the room is large; no live polling | +| Primary technology | Intune | +| Session types | Automation; Provisioning & Deployment; Setup & Configuration | +| Primary audience | Microsoft endpoint admins, Intune admins, ConfigMgr admins, Windows-first automation people who now need to support Macs | +| Attendee promise | A reproducible Apple-silicon macOS VM lab, automated with PowerShell 7.4 or newer, that lets Microsoft admins test risky macOS Intune policies before production users discover the mistake | + +## Accepted Session Contract + +This is an **Intune risk-reduction session** that uses macOS virtualization as the lab substrate. + +It is not a general Mac virtualization overview. It is not a Mac hobbyist session. It is not a complete macOS management boot camp. It is a practical, technical session for Microsoft endpoint people who already understand the value of Windows testing rings, checkpoints, automation, evidence, and rollback, and who need the same safety net for Macs. + +The central story is: + +1. Enterprises are buying and managing Macs. +2. Microsoft admins are being asked to manage them with Intune. +3. Untested macOS policies can break very visible users very quickly. +4. Apple-silicon virtualization has real constraints, but it is useful if you respect those constraints. +5. PowerShell 7.4 or newer can be the familiar automation layer. +6. A VM lab is a safe place to fail before a risky policy reaches real users. +7. Some validation still belongs on physical Mac hardware, and the session must say that clearly. + +## Accepted Takeaways and Where They Are Delivered + +| # | Accepted takeaway | What the session must prove | Where it is delivered | +| --- | --- | --- | --- | +| 1 | Analyze Apple-silicon virtualization constraints and select the appropriate hypervisor, Parallels vs. UTM, for budget and automation needs. | Attendees understand the practical constraints and can make a defensible tool choice. | Technical constraints, hypervisor decision matrix, Demo 2, Demo 3, Q&A buckets. | +| 2 | Construct a fully automated, reproducible macOS test lab using PowerShell 7.4 or newer to fetch restore images/install media and control VM states. | Attendees see a repeatable flow that pins a build, acquires media, creates/starts VMs, snapshots, restores, and emits evidence. | Architecture section, Demos 1-3, starter kit, Windows-admin translation cheat sheet. | +| 3 | Execute end-to-end validation of high-risk policies, including FileVault and Defender, by enrolling, breaking, and rolling back VMs via script. | Attendees see an Intune validation loop, understand what can be proven in a VM, and know what still requires physical Mac sign-off. | Demo 4, FileVault proof path, Defender proof path, fidelity traffic light, evidence model. | +| 4 | Implement the GitHub starter kit immediately to bridge Windows automation skills and macOS requirements. | Attendees know the repo path, first command, first test case, and Monday-morning plan. | Windows-admin translation segment, repo handoff, `Start-Here.md`, Windows-admin cheat sheet, Q&A framework, close-out. | + +## Narrative Thesis + +> Your Windows lab instincts are still right. We are going to translate them to macOS, automate them with PowerShell, and prove that risky Intune policies can fail safely before they fail publicly. + +Use this phrase early and late: + +> Production is a terrible place to discover your Mac policy assumptions. The lab is the safe place to fail. + +The session should feel like a bridge, not a scolding. The audience should not leave thinking, "Macs are weird and I am behind." They should leave thinking, "I already know how to operate safely; now I have a pattern for doing that with Macs." + +## Audience Model + +### Primary Attendee + +The primary attendee is a Windows-first Microsoft endpoint admin. They likely know some combination of: + +- Intune. +- ConfigMgr. +- PowerShell. +- Windows deployment. +- Application packaging. +- Patch management. +- Compliance policies. +- Conditional Access. +- Change management. +- Endpoint security tooling. + +They may now own Macs because: + +- Executives use them. +- Developers use them. +- A business unit bought them. +- A security requirement brought them into Intune. +- Leadership expects "one endpoint management story." + +They are probably thinking: + +- "I already have a Windows lab. Why does Mac testing feel so much more fragile?" +- "I do not want to become a full-time Mac engineer just to avoid breaking the CEO's laptop." +- "I need evidence before change control approves this." +- "I need a workflow my Windows-heavy team can understand." + +### Secondary Attendee + +The secondary attendee already manages Macs and wants: + +- A faster local regression loop. +- Better lab automation. +- A way to explain macOS testing to Windows-first colleagues. +- A repeatable starter kit they can adapt. + +### Tone to Use + +Use: + +- Calm technical confidence. +- Specific failure modes. +- Practical tradeoffs. +- Occasional humor. +- Respect for both Windows and Mac admin work. + +Avoid: + +- "Macs are weird" as the main joke. +- Pretending VMs replace real hardware. +- Tool religious wars. +- Live-service heroics. +- Progress-bar theater. + +## Scope + +### Core Scope + +These are the center of gravity: + +- Apple-silicon macOS VM lab design. +- Parallels Desktop and UTM as the two primary local hypervisor paths. +- PowerShell 7.4 or newer as the automation and abstraction layer. +- Intune enrollment, policy delivery, validation, evidence, and rollback. +- High-risk macOS management areas: + - FileVault. + - Microsoft Defender for Endpoint. + - PPPC/TCC. + - Compliance and Conditional Access timing. +- GitHub starter kit that attendees can clone after the session. + +### Secondary Scope + +These are useful but should not steal time from the accepted abstract: + +- Tart as an advanced CLI/CI path. +- Full CI/CD pipeline design. +- Deep Microsoft Graph automation. +- ConfigMgr inventory bridge. +- Azure Log Analytics evidence ingestion. +- ADE/ABM test strategy. +- Platform SSO. +- Secure Enclave-dependent behavior. +- Physical hardware final sign-off. + +Mention these where relevant, but keep most of the detail in the repo and Q&A. + +### Out-of-Scope Boundaries + +Say these clearly enough that attendees trust the rest of the session: + +- A VM is not a perfect substitute for final sign-off on physical corporate Mac hardware. +- The session will not teach every part of macOS management. +- The session will not compare every virtualization product. +- The session will not depend on venue Wi-Fi, live cloud timing, or live policy changes as the only success path. +- The session will not promise that all Apple hardware/security-model behavior can be validated in a VM. +- The session will not provide legal advice on Apple licensing, vendor licensing, or enterprise procurement terms. + +## Design Decisions + +### Opening and Risk Framing + +**Decision:** Keep the opening short: hook, one risk slide, and then into constraints/tools/demos. + +**Why:** MMS audiences are deeply technical. A long intro will feel padded and will create demo time pressure. + +The canonical risk map for this session — referenced throughout the rest of this document: + +| Risk | What breaks | Why leadership notices | +| --- | --- | --- | +| FileVault | Unlock, recovery, escrow, user prompts. | Executive lockout, urgent helpdesk escalation, security visibility. | +| PPPC/TCC | Required tools cannot access protected data. | Broken screen sharing, recording, accessibility, backup, EDR, or remote support workflows. | +| Defender | System extension, network extension, Full Disk Access, onboarding. | Security gap, false-positive noise, SOC escalation, unhealthy endpoint posture. | +| Compliance/CA | Device marked noncompliant or access blocked. | Users lose access to Outlook, SharePoint, Teams, approval apps, or business workflows. | + +Optional 10-second calibration: + +> Who here manages Macs in Intune today? + +Use this once. Do not turn the session into polling. + +### Hypervisor Scope + +**Decision:** Make **Parallels Desktop** and **UTM** the on-stage core. Mention **Tart** as the advanced CLI/CI path. + +**Why:** The accepted abstract names Parallels and UTM. Keeping them central preserves abstract fidelity and reduces time risk. + +| Tool | Core positioning | Best fit | Stage treatment | +| --- | --- | --- | --- | +| Parallels Desktop Pro/Business | Polished commercial workflow with `prlctl` automation and strong desktop admin UX. | Enterprise admins who want a supported-feeling local workflow and reliable stage demos. | Primary live build/snapshot demo. | +| UTM | Cost-free experimentation path with Apple Virtualization and QEMU options; automation exists but does not mirror Parallels one-for-one. | Budget-sensitive labs, learning, and teams comfortable with templates/configuration artifacts. | Provider-swap demo. | +| Tart | CLI-first Apple Silicon VM tooling built for automation, CI, and image distribution. | Teams building repeatable macOS runners or pipeline-based test loops. | Repo/Q&A/appendix path, not a third equal branch in the core. | + +Do not frame the section as "which product is best." Frame it as: + +> Which tool makes the safe behavior easiest for your team? + +### Media Acquisition Language + +**Decision:** Use "restore image or install artifact" as the main language, not "ISO" as the default Apple-silicon Mac path. + +**Why:** For Apple-silicon macOS VM workflows, especially Parallels macOS Arm VMs, the practical and documented path is restore-image/IPSW-oriented. ISO creation can still exist in repo docs where appropriate, but it should not be the stage headline. + +Recommended wording: + +> We pin a macOS build, acquire the correct restore image or provider-appropriate install artifact, cache it, and record the metadata. The important thing is not the file extension. The important thing is reproducibility. + +### No Parallels Coherence Demo for macOS Arm VMs + +**Decision:** Do not use Coherence as the stage wow for the macOS VM. + +**Why:** It is not a reliable or accurate promise for macOS Arm VMs. It also distracts from the accepted session promise. The memorable moment should be rollback and evidence, not desktop visual integration. + +The actual wow: + +1. The validation run is red. +2. Snapshot rollback runs. +3. The validation run is green again. +4. Evidence is exported. + +That is more meaningful to this audience. + +### Intune Depth + +**Decision:** Demo 4 is workflow-first, with one or two deep proof points. + +The core workflow is: + +```text +enroll -> apply policy -> validate -> intentionally fail -> collect evidence -> roll back -> prove known-good +``` + +The proof points should be: + +- FileVault: policy assignment, local `fdesetup status`, escrow/recovery-key evidence, redacted proof, and hardware sign-off boundary. +- Defender: system extension, network extension, Full Disk Access/PPPC, onboarding, `mdatp health`, and evidence export. + +Avoid trying to show every Graph query, every profile payload, and every Intune page. Put deeper Graph/payload detail in the repo. + +### Snapshot Strategy + +**Decision:** Teach multiple snapshot types with clear guardrails. There is no universal best snapshot. + +| Snapshot | Use when | Benefits | Risks | +| --- | --- | --- | --- | +| `Clean-OS` | You need a freshly installed guest before identity, enrollment, or user state. | Maximum cleanliness and repeatability. | Slowest path back to policy testing. | +| `Pre-Enroll` | You want a clean device identity path for each enrollment test. | Best default for enrollment and MDM identity fidelity. | Slower because enrollment and sync rerun. | +| `Post-Enroll-Baseline` | You need a fast repeated policy regression loop. | Fastest and most stage-reliable. | Intune/Entra cloud state keeps moving while the VM rolls backward. | +| `Broken-Policy-State` | You need a deterministic demo failure. | Excellent for stage reliability. | Must be explained honestly as a checkpointed failure. | +| `Recovered-Known-Good` | You need proof that rollback returns the VM to a useful baseline. | Makes recovery visible. | Can hide cloud cleanup problems if not documented. | + +Recommended operational rule: + +- Use `Pre-Enroll` as the gold-standard identity baseline. +- Use `Post-Enroll-Baseline` for speed and demo reliability. +- Pair post-enroll rollback with a documented cloud cleanup routine. In v1, that routine is report-only: it identifies candidate cloud records and manual cleanup steps, but it does not mutate Intune, Entra, or Defender records. +- Teach "snapshot time travel" as a real anti-pattern. + +Important stage warning: + +> Rollback restores the VM. It does not rewind Intune, Entra, Defender portal state, audit logs, compliance history, or cloud reporting. The report-only cleanup routine is part of the lab, not an optional afterthought. + +### Failure Style + +**Decision:** Use a deterministic, scripted failure and narrate it calmly. + +Good stage language: + +> This policy state is intentionally wrong. We want the test to fail here, because failure in the lab is the point. + +Avoid: + +> Let's see what happens. + +The audience should believe that even the failure was engineered. + +### Repo Timing + +**Decision:** Publish the repo before the session, but reveal the QR/short URL near the end of the core. + +Recommended stage pattern: + +1. Show the command you would run. +2. Open a pre-cloned local copy. +3. Show the exact demo paths. +4. Reveal the QR/short URL at 73:00. +5. Keep attention on the stage until the core content is done. + +Do not depend on a live `git clone` over conference Wi-Fi. + +### Demo Timing + +Treat demo durations as **keyboard-time targets**, not guaranteed wall-clock time. + +Every demo needs: + +- A live path. +- A checkpoint path. +- A local, fully redacted screenshot path for the deck/rehearsal assets. Do not commit example screenshots to the public v1 repo. +- A screen-recording path for cloud-dependent parts. +- A practiced pivot sentence. + +Pivot sentence: + +> Watching a progress bar is not educational. The artifact is pinned, cached, and recorded, so I am going to jump to the completed checkpoint and show you what matters. + +## Technical Correctness Guardrails + +### Apple Software License and Concurrency + +Do not give legal advice from the stage. + +Safe framing: + +- macOS virtualization is tied to Apple-branded hardware and Apple's software license terms. +- For this talk and repo, use Apple's current macOS Tahoe software license agreement as the plain-language source to summarize. A commonly relevant boundary is up to two additional macOS virtual instances per Apple-branded host, subject to Apple's terms and permitted purposes. +- Do not paraphrase this as "every enterprise automatically gets two free macOS test VMs for any purpose." +- Enterprises should have legal/procurement confirm their own use case. + +Stage phrase: + +> The licensing story is friendlier than Windows licensing in some ways, but the engineering constraints are where the real design work starts. + +### Host and Guest Version Caution + +Avoid an oversimplified rule like "host must always be same or newer" unless you qualify it. + +Better wording: + +> On Apple Silicon, guest support is constrained by the host macOS version, restore image, Virtualization framework, and provider. Treat the lab host like a build server: do not auto-upgrade it right before a regression cycle. + +Practical rule: + +- Keep the host on a known-good build for the life of a demo or regression cycle. +- Record host macOS version, hypervisor version, guest macOS version, and guest build in every evidence bundle. +- Re-run the lab readiness tests after host updates, hypervisor updates, and major Intune/Defender policy changes. + +Initial guest-version policy: + +| Purpose | macOS target | Stage meaning | +| --- | --- | --- | +| Live MMSMOA demo | macOS Tahoe 26.4.1 | Primary version for the demo VMs. | +| Compatibility target | macOS Sequoia 15.7.5 | Keep the repo usable for supported 15.x environments. | +| Compatibility target | macOS Sonoma 14.8.5 | Keep the repo usable for supported 14.x environments. | + +The rule is not "support only these three patch versions forever." The rule is "support the macOS versions Apple currently supports, and record the exact host/guest/build values used for each evidence run." + +### Apple Virtualization Framework and QEMU + +Keep this to one practical slide. + +- Apple's Virtualization framework is the first-party foundation for supported macOS guest virtualization on Apple Silicon. +- Parallels macOS Arm guests use Apple's Virtualization framework. +- UTM can use Apple Virtualization and QEMU depending on the guest and configuration. +- Tart uses Apple's Virtualization framework and is designed for automation-heavy workflows. +- The practical impact is that provider automation surfaces differ. + +Do not make attendees feel they must become Virtualization framework developers. + +### Provider Version Guardrails + +Add a small support matrix to the repo and keep it current for the event. + +Example: + +| Component | Version tested | Why it matters | +| --- | --- | --- | +| Host macOS | `` | Determines restore-image compatibility and Virtualization framework behavior. | +| Parallels Desktop | `` | macOS Arm snapshot and CLI behavior can vary by version. | +| UTM | `` | Automation and template behavior can vary by version. | +| PowerShell | ``; must be 7.4 or newer | Cross-platform module behavior and remoting behavior. | +| Pester | 5.7.1 initially | Test behavior and CI output must be reproducible. | +| Defender for Endpoint | `` | `mdatp health` fields and extension behavior can change. | +| Intune policy set | `` | Assignment and reporting behavior may change over time. | + +Stage point: + +> "Works on my Mac" is not evidence. "Works on this host build, this hypervisor version, this guest build, and this policy set" is evidence. + +### VM Fidelity Traffic Light + +Use this as a memorable credibility slide. + +| Color | Meaning | Examples | +| --- | --- | --- | +| Green | Good VM tests. | Script syntax, package install behavior, Intune assignment logic, profile receipt, basic PPPC payload behavior, Defender health checks, rollback routines, evidence export. | +| Yellow | Good VM iteration, then physical hardware sign-off. | FileVault rollout behavior, recovery-key process, compliance experience, user prompts, performance-sensitive Defender behavior. | +| Red | Physical hardware or production-like enrollment required. | ADE/ABM zero-touch flows, serial-number-dependent workflows, Platform SSO sign-in/unlock experience, Touch ID, Secure Enclave-dependent behavior, physical Wi-Fi behavior, final executive pilot sign-off. | + +The point: + +> A lab that cannot explain its fidelity boundary is not a lab. It is a superstition with screenshots. + +### APNs and Network Reality + +macOS management depends on APNs and cloud service timing. Conference networks may block, proxy, throttle, or shape the traffic you care about. Corporate networks can also break things in non-obvious ways: many enterprises run TLS/SSL inspection appliances that intercept Microsoft and Apple management traffic. APNs traffic should bypass TLS decryption/inspection, and Intune/Defender check-in traffic should be validated on the same network you plan to use for the demo. + +Stage phrase: + +> If APNs is not happy, Intune is not instant. That is why we cache, checkpoint, and prove state instead of trusting a spinner. + +Pre-session checks must include: + +- APNs connectivity. +- Intune portal access. +- Graph access if used. +- Defender cloud connectivity if used. +- Hotspot fallback if permitted. +- Local-only fallback. +- Confirmation that proxy or SSL inspection behavior is not breaking Apple or Microsoft management traffic. + +Credibility moment to land verbally: + +> If you have ever had Intune check-in fail only on certain networks, look at proxy and TLS inspection before you blame Apple. APNs traffic needs to pass cleanly, and your lab should prove the network path before showtime. + +### FileVault Validation Model + +FileVault must be explicit because it is in the title and accepted takeaway. + +Minimum proof path: + +1. Show the policy assignment or lab configuration. +2. Show local state with `fdesetup status`. +3. Show Intune encryption or recovery-key evidence. +4. Redact or mask any recovery-key value. +5. Explain escrow timing. +6. Explain what VM testing can prove. +7. Explain what still requires physical hardware sign-off. + +Evidence to collect: + +- macOS version and build. +- VM provider and snapshot. +- Intune device name and ID, redacted if needed. +- FileVault policy name and assignment scope. +- `fdesetup status`. +- Recovery-key escrow status or report path. +- Redacted recovery-key existence proof, not the recovery key itself. +- Known limitations. +- Rollback result. + +Important nuance: + +> Escrow preparation and encryption state are related, but they are not the same thing. Test both. + +Stage safety rule: + +> Never display a full FileVault recovery key on the projector. Show that the key exists, show where it is retrieved, show that access is audited, and redact the value. + +Practical consequence: this rules out any live demo of the Intune portal page that retrieves a recovery key, because that page genuinely shows the value once the right role clicks "Show recovery key." Use pre-redacted screenshots of the retrieval flow instead, narrate them as if they are live, and confirm during rehearsal that no demo path leads to a portal page where the key would be exposed. + +### Defender Validation Model + +Defender should be more than "the app installed." + +Minimum proof path: + +1. Package installed. +2. System extension approved. +3. Network extension approved if network protection is used. +4. Full Disk Access/PPPC delivered. +5. Onboarding completed. +6. `mdatp health` captured. +7. Optional portal visibility or test signal. +8. Evidence exported. + +Evidence to collect: + +- Defender version. +- System extension state. +- Network extension state. +- Full Disk Access/PPPC profile receipt. +- `mdatp health`. +- Policy assignment scope. +- Local logs or health output. +- Rollback result. + +Stage point: + +> Defender on macOS is not just "install the app." The profiles are the deployment. + +### PPPC/TCC Validation Model + +PPPC is a precision problem. + +Teach: + +- Bundle ID matters. +- Code requirement matters. +- App path can matter. +- User-visible privacy UI may not tell the whole story. +- Evidence should come from profiles, logs, app behavior, and targeted scripts. + +Do not rely on "I looked in System Settings and it seemed right." + +### Compliance and Conditional Access Timing + +Compliance and Conditional Access should be framed as eventually consistent. + +Recommended stage language: + +> The portal is not a real-time debugger. It is a management system with check-in, evaluation, reporting, and access-decision timing. Our job is to collect enough evidence to know what stage we are actually waiting on. + +For the live demo, prefer a deterministic compliance smoke test or pre-created state over a live Conditional Access block. + +## Speaker Choreography + +### Ownership Model + +Use the two-speaker format intentionally. MMS wants conversation, not two disconnected monologues. + +| Segment | Driver / lead | Narrator / color | +| --- | --- | --- | +| Hook and risk map | Frank | Michael adds enterprise reality. | +| Windows-admin translation | Frank | Michael affirms with a one-line "this is how my team would think about it." | +| Apple Silicon constraints | Michael | Frank asks Windows-admin translation questions. | +| Hypervisor decision | Frank | Michael challenges tradeoffs and keeps it Intune-relevant. | +| PowerShell architecture | Frank | Michael asks "how would an endpoint team maintain this?" | +| Demo 1: media pinning | Frank | Michael explains why build pinning matters in enterprise change control. | +| Demo 2: Parallels | Frank | Michael explains why snapshot discipline matters. | +| Demo 3: UTM provider swap | Frank | Michael compares the operating model without dunking on the tool. | +| Demo 4: Intune validation | Michael | Frank narrates lab scaffold, rollback, and evidence. | +| Dragons and wrap | Both | Alternate lines. | +| Extended Q&A | Both | Route questions by domain. | + +Why this matters: + +- Frank owns the lab automation and PowerShell narrative. +- Michael owns the Intune/enrollment/policy behavior narrative. +- Demo 4 should feel like a real Intune session, not just a virtualization session. + +### Driver/Narrator Pattern + +For every demo: + +| Role | Responsibility | +| --- | --- | +| Driver | Runs keyboard and mouse, keeps windows readable, executes pivots. | +| Narrator | Explains what is happening, fills latency, watches time, names gotchas. | + +The narrator has permission to interrupt with: + +- "Pause there for a second." +- "This is the part Windows admins should notice." +- "This is where I would not trust the VM alone." +- "Let's checkpoint that before we make it messy." + +### Planned Handoff Lines + +Script these in speaker notes: + +- "Frank showed why this matters. Michael, let's make the Apple Silicon constraints concrete." +- "That is the constraint side. Now let's turn it into a tool choice instead of a religious debate." +- "We have two providers, but we do not want two automation models. Frank, show the PowerShell contract." +- "This is the part where the VM stops being cute and starts being useful." +- "Now we are going to break the lab on purpose, because production should never be the first place this mistake appears." +- "Before we go to Q&A, here is the Monday-morning path in the repo." + +## Pre-Session Stage-Readiness Runbook + +### Hardware + +Use one primary Apple-silicon Mac for the full demo chain. Bring a second Mac if available, but do not design the session around needing two. + +Minimum practical recommendation: + +- Apple-silicon Mac with enough CPU and RAM for: + - host OS. + - macOS guest. + - browser. + - PowerShell terminal. + - VS Code. + - PowerPoint. + - screen recording. +- 150-250 GB free SSD before rehearsal snapshots. +- Stable power. +- Known-good A/V adapter. +- Spare A/V adapter. +- Offline copy of the deck. +- Offline copy of the repo. +- Offline copy of critical screenshots and recording. + +### Display Readability + +- Terminal font around 24 pt, adjusted after projector testing. +- VS Code font around 22-24 pt. +- Browser zoom at 125-150 percent for Intune portal demos. +- Use a consistent window layout. +- Disable notifications. +- Disable sleep. +- Disable screen saver. +- Disable auto-updates. +- Use light theme if the room is bright and projector contrast is poor. + +### Host macOS Permissions + +Grant privacy permissions to the **exact app** used for automation. + +Check: + +- Automation. +- Full Disk Access. +- Files and folders access. +- Screen recording if recording or presenting. +- Accessibility if window automation requires it. + +Important: + +> TCC is app-specific. A permission granted to Terminal does not automatically apply to iTerm2 or VS Code. + +Avoid switching terminal apps mid-demo. + +### Local Assets + +Have these locally: + +- Restore images or provider-appropriate install artifacts. +- Pre-created VMs. +- All demo scripts. +- Redacted evidence examples. +- Screenshots of Intune portal states with secrets and user-identifying details masked. +- 60-90 second screen recording of Demo 4. +- Full offline copy of the GitHub starter kit. + +### Required VM Checkpoints + +Prepare: + +- `Clean-OS` +- `Pre-Enroll` +- `Post-Enroll-Baseline` +- `Broken-Policy-State` +- `Recovered-Known-Good` + +Document what each contains. Do not rely on memory. + +### Tenant and Intune Setup + +Use a dedicated demo tenant or an isolated lab scope. + +Minimum setup: + +- Lab test user. Use a cloud-only account in the demo tenant with no real mailbox, no real licenses beyond what the demo requires, and no group memberships outside the lab scope. The point is that a Conditional Access misfire during the demo cannot lock a real person out of real systems. +- Lab device naming convention. +- Lab-only groups. +- Lab-only filters if filters are used. +- FileVault policy. +- Defender policy set. +- Compliance policy for visible pass/fail. +- Optional Conditional Access policy only if isolated and safe. +- Graph permissions if using a Graph proof point. +- Screenshots of all portal states in case network/cloud timing misbehaves. +- A redaction pass for all screenshots, JSON outputs, logs, and evidence bundles that might be shown publicly. + +Avoid using broad production assignments. + +### Network Contingency + +Before the session: + +- Validate APNs connectivity. +- Validate Intune portal access. +- Validate Graph access if used. +- Validate Defender cloud connectivity if used. +- Validate hotspot fallback if permitted. +- Pre-trust fallback SSID if allowed. +- Prepare a local-only fallback path. +- Confirm proxy or SSL inspection on every network you might use is not breaking management traffic. + +### Break-Glass Recording + +Record a polished 60-90 second clip of Demo 4: + +```text +baseline -> apply/reveal failure -> collect evidence -> roll back -> known good +``` + +Use it only if there is a real service, APNs, network, or lab failure. + +Practiced line: + +> Rather than fight a service issue live, here is the exact run from rehearsal. Then we will jump back to the live machine for the rollback and repo path. + +Rehearse the narration over the recording at least once before the event so the audio that comes out of your mouth matches what is happening on screen. The recording is only useful if the words land in time with the visuals. + +### T-15 Minute Smoke Test + +Run this before going on stage: + +1. `Test-LabReadiness.ps1` returns green. +2. Parallels opens and required VMs are visible. +3. UTM opens or `utmctl` responds if used. +4. `prlctl list --all` returns expected output. +5. Demo VM boots from `Post-Enroll-Baseline`. +6. Intune admin center loads. +7. Demo tenant device exists. +8. Evidence script can write to `_evidence`. +9. Screen recording file opens. +10. QR code resolves. +11. Browser zoom and terminal font are readable. +12. Notifications and auto-updates are disabled. +13. Evidence screenshots are redacted. +14. No recovery key, app secret, token, tenant secret, or personal user data is visible in the demo path. + +## Core Timeline: 75 Minutes + +### 0:00-1:00: Title and Micro-Credibility + +Goal: establish who you are and what happens. + +Say: + +- We manage and automate Microsoft endpoint environments. +- We know this audience already has Windows lab instincts. +- Today we translate those instincts to macOS Intune policy testing. + +Two presenters, brief introductions only. No bios. Save personal background for the speaker bio link in Sched. + +### 1:00-3:00: CEO Lockout Hook + +Goal: create urgency without fearmongering. + +Visual: a Mac at a FileVault unlock/recovery moment or a clean mockup of an executive support escalation. + +Line: + +> You would not ship a BitLocker policy to the CEO's laptop without testing recovery. But many organizations are still shipping macOS FileVault, PPPC, and Defender policies with far less lab confidence. Today we build the safety net. + +Promise: + +> By the end, you will see a Mac policy fail safely in a lab, roll back, and produce evidence you could attach to a change ticket. + +### 3:00-6:00: Risk Map + +Goal: show why this matters. + +Use the canonical four-bucket risk slide from the Design Decisions section. Walk through each row in two or three sentences. Tie each row to the user persona it would hurt — executive lockout for FileVault, "the recording we needed for the board" for PPPC, security visibility for Defender, "I cannot get into Outlook" for Compliance/CA. + +Optional hand raise: + +> Who here manages Macs in Intune today? + +### 6:00-7:30: Windows-Admin Translation + +Goal: make Takeaway 4's bridge promise explicit before the constraints content lands. + +Show the Windows-admin translation cheat sheet as a single slide. Walk through three or four lines, not all of them. The point is signaling, not exhaustive coverage. + +Stage line: + +> Your Windows lab instincts are still right. The language stays the same — PowerShell 7.4+ is the conductor. Only the instruments change. + +This segment is also where the audience starts to relax. Constraints land better when the audience already feels invited rather than scolded. Keep it tight; the full cheat sheet is in the deck and the repo for later reference. + +### 7:30-15:00: Constraints That Drive Lab Design + +Goal: earn credibility and prevent overpromising. + +Cover: + +- Why Hyper-V/ESXi mental models do not directly apply. +- Apple software license and concurrency caution. +- Restore images and build pinning. +- Host/guest/provider compatibility. +- Apple Virtualization framework vs. QEMU. +- Provider feature differences. +- APNs and Intune timing. +- Corporate proxy and SSL inspection breaking APNs or Microsoft management traffic in non-obvious ways. +- VM fidelity boundaries. +- Host upgrade danger. +- Provider version guardrails. + +400-level anchor: + +> A lab that cannot tell you what it cannot prove is not a safe lab. + +Credibility moment to land verbally: + +> If you have ever had Intune check-in fail on Mondays only, look at your TLS-inspecting proxy before you blame Apple. APNs does not tolerate man-in-the-middle, and many corporate networks accidentally are one. + +### 15:00-24:30: Hypervisor Decision Matrix + +Goal: help attendees choose Parallels or UTM without a product fight. + +Slide: "Choose based on operating model." + +| Decision factor | Parallels Desktop Pro/Business | UTM | +| --- | --- | --- | +| Cost | Commercial subscription. | Free direct download; optional paid App Store convenience. | +| Admin UX | Highest polish for local desktop workflows. | Good experimentation UX. | +| Automation surface | `prlctl` is strong for many lifecycle operations in Pro/Business workflows. | `utmctl`, AppleScript, Shortcuts, and templates; automation coverage differs by feature. | +| Snapshots/clones | Useful in current versions; confirm Parallels Desktop 20+ for macOS Arm VM snapshots. | Useful, but workflow differs and may require template/config discipline. | +| Stage reliability | Strong local demo choice when version-checked. | Good provider-swap proof if prepared. | +| Procurement story | Easier for many enterprises. | Easier where no tool budget exists. | +| CI path | Possible but not the main story. | Possible with effort; Tart is usually the cleaner advanced CI discussion. | + +Say: + +> If I need the most polished local admin workflow and I can buy a tool, I start with Parallels. If I need no-cost experimentation and I can accept more template/config work, I start with UTM. If I want pipeline-native image distribution, Tart becomes a phase-two conversation. + +### 24:30-30:00: PowerShell 7.4+ Provider-Model Architecture + +Goal: prove this is automation-first. + +Flow diagram: + +1. Pin build. +2. Acquire restore image or provider-appropriate install artifact. +3. Create or register VM. +4. Apply sizing profile. +5. Start VM. +6. Create snapshot. +7. Enroll in Intune. +8. Apply policy. +9. Validate or intentionally fail. +10. Collect evidence. +11. Roll back. +12. Clean up local and cloud state. + +Stable interface example: + +```powershell +# Replace and with your pinned values during rehearsal. +Get-MacLabMedia ` + -Version '' ` + -Build '' ` + -Architecture arm64 ` + -Source Mist + +New-MacLabVm ` + -Provider Parallels ` + -Name 'mms-fv-01' ` + -SizingProfile Baseline + +Checkpoint-MacLabVm ` + -Name 'mms-fv-01' ` + -CheckpointName 'Pre-Enroll' + +Invoke-MacPolicyValidation ` + -Name 'mms-fv-01' ` + -TestPlan './examples/TestCases/MMS-Demo4.yml' + +Restore-MacLabVmCheckpoint ` + -Name 'mms-fv-01' ` + -CheckpointName 'Post-Enroll-Baseline' + +Export-MacLabEvidence ` + -Name 'mms-fv-01' ` + -OutputPath './_evidence/mms-fv-01' +``` + +400-level anchor: + +> The abstraction boundary is not "hide all provider details." The boundary is "make the safe workflow consistent and surface provider-specific failure modes clearly." + +### 30:00-31:00: Demo Rules and Gotchas + +Goal: set expectations. + +Say: + +- Downloads are cached. +- Cloud sync is not instant. +- Checkpoints are intentional. +- Rollback restores the VM, not the cloud. +- The point is evidence and repeatability, not watching progress bars. + +## Demo Block: 31:00-70:00 + +### Demo 1: Pin and Acquire Media, 31:00-38:00 + +Goal: show reproducible acquisition of a specific macOS build. + +Live path: + +- Show a build-pinning config file. +- Run or display the media acquisition command. +- Show cached artifact metadata. +- Show that the VM build process consumes the pinned artifact. + +Example config: + +```yaml +# Replace and with your pinned values during rehearsal. +# Choose a build that exists in Apple's restore catalog and cache it locally before the session. +name: mms-demo-macos +macOS: + version: '' + build: '' + architecture: arm64 + artifactType: restoreImage + source: mist-cli +providerDefaults: + parallels: + cpus: 4 + memoryGB: 8 + diskGB: 96 + utm: + cpus: 4 + memoryGB: 8 + diskGB: 96 +``` + +Evidence shown: + +```text +_evidence/media/-/ + metadata.json + acquisition.log + checksum.txt +``` + +Narration cue: + +> This is where repeatability starts. If you cannot say which macOS build you tested, you cannot say what your test means. + +Recovery path: + +- Leave the command visible. +- Open cached output. +- Explain that downloading is not the educational part. + +### Demo 2: Parallels VM Build and Snapshot, 38:00-50:00 + +Goal: show the polished commercial path and PowerShell driving provider-specific automation. + +Live path: + +- Confirm Parallels version supports the macOS Arm VM features used in the demo. +- Create or register the VM from a prepared restore image. +- Apply sizing profile. +- Start the VM or jump to a created VM. +- Create or show checkpoint named `Pre-Enroll`. +- Show provider logs and evidence output. + +Example commands: + +```powershell +# Replace and with your pinned values during rehearsal. +New-MacLabVm ` + -Provider Parallels ` + -Name 'mms-parallels-01' ` + -MediaId '-' ` + -SizingProfile Baseline + +Checkpoint-MacLabVm ` + -Provider Parallels ` + -Name 'mms-parallels-01' ` + -CheckpointName 'Pre-Enroll' +``` + +Narration points: + +- `prlctl` is the provider-specific engine. +- PowerShell is the stable operator interface. +- The snapshot is not decoration. It is the safety mechanism. +- The evidence bundle records what happened. +- The Parallels version is part of the evidence, because macOS Arm VM features are version-sensitive. + +Do not: + +- Promise macOS Apple-silicon Coherence Mode. +- Spend time on visual polish instead of risk reduction. +- Show an `.app` installer path as the primary Parallels Apple-silicon macOS VM path. +- Assume snapshots exist unless the Parallels version has been confirmed during rehearsal. + +Recovery path: + +- Pivot to prebuilt VM at `Pre-Enroll` or `Post-Enroll-Baseline`. +- Show logs from the successful creation. + +### Demo 3: UTM Provider Swap, 50:00-57:00 + +Goal: prove that the automation pattern survives a provider change. + +Live path: + +- Switch provider parameter from Parallels to UTM. +- Show UTM template/config artifact. +- Start or inspect the UTM VM. +- Show that the same higher-level validation flow still applies. + +Example commands: + +```powershell +# Replace and with your pinned values during rehearsal. +New-MacLabVm ` + -Provider UTM ` + -Name 'mms-utm-01' ` + -MediaId '-' ` + -TemplatePath './examples/utm/macos-lab-template.utm' + +Start-MacLabVm -Provider UTM -Name 'mms-utm-01' +Get-MacLabVm -Provider UTM -Name 'mms-utm-01' +``` + +Narration points: + +- UTM is a serious lab option, not a toy. +- UTM automation is different from Parallels automation. +- The provider abstraction prevents tool differences from changing the entire workflow. +- Not every UTM feature has the same automation parity, so the provider wrapper should make gaps explicit. + +Optional Tart cameo, one slide only: + +```bash +# Replace with the tag your team publishes for the pinned build. +tart clone ghcr.io/example/macos-runner: mms-tart-01 +tart run mms-tart-01 +``` + +Say: + +> Tart is where this pattern can grow when you want a CI runner conversation. It is not the core promise of this accepted session, so the v1 repo keeps Tart stubbed unless we explicitly approve more. Tart and Orchard publish Fair Source/free-tier terms; treat the free-tier CPU-core and worker limits as planning constraints and re-check your company's use case before treating Tart as an enterprise standard. + +Recovery path: + +- Show the UTM template artifact. +- Show a pre-created UTM VM. +- Move on rather than debugging UTM live. + +### Demo 4: Intune Validation Loop, 57:00-70:00 + +Goal: prove risk-free policy testing. + +Start state: + +- VM at `Post-Enroll-Baseline`. +- Device already enrolled in the demo tenant. +- Recent sync completed. +- Lab-only policies assigned. +- Evidence script ready. +- Evidence output redaction already tested. + +Primary live flow: + +1. Show enrollment state. +2. Run baseline evidence script. +3. Show FileVault policy evidence. +4. Show Defender health evidence. +5. Trigger or reveal a controlled failure. +6. Collect failure evidence. +7. Restore snapshot. +8. Rerun evidence script. +9. Show return to known good. +10. State clearly what cloud cleanup is still required. + +Example commands: + +```powershell +Invoke-MacPolicyValidation ` + -Name 'mms-parallels-01' ` + -TestPlan './examples/TestCases/FileVault-Defender-Smoke.yml' ` + -OutputPath './_evidence/runs/mms-demo4-before' ` + -RedactSecrets + +Restore-MacLabVmCheckpoint ` + -Provider Parallels ` + -Name 'mms-parallels-01' ` + -CheckpointName 'Post-Enroll-Baseline' + +Invoke-MacPolicyValidation ` + -Name 'mms-parallels-01' ` + -TestPlan './examples/TestCases/FileVault-Defender-Smoke.yml' ` + -OutputPath './_evidence/runs/mms-demo4-after' ` + -RedactSecrets +``` + +Minimum evidence to show. The list below is intended as terminal output during the live demo. The slide companion is a condensed six-line version highlighting MDM enrollment, FileVault evidence (with redaction note), Defender health, the intentional compliance failure, the rollback result, and the cloud-cleanup warning. The full bundle is reflected in the JSON example in the GitHub Starter Kit section. + +```text +PASS MDM enrollment profile present +PASS Device name matches lab naming convention +PASS FileVault policy detected +PASS fdesetup status captured +PASS FileVault escrow evidence captured +PASS FileVault recovery key value redacted +PASS Defender system extension present +PASS Defender network extension state captured +PASS Defender Full Disk Access profile present +PASS mdatp health captured +FAIL Compliance smoke test in intentional broken state +PASS Rollback restored known-good VM checkpoint +WARN Cloud state cleanup still required +PASS Evidence bundle exported +``` + +Controlled failure options: + +| Option | Reliability | Notes | +| --- | --- | --- | +| Pre-created `Broken-Policy-State` checkpoint | Highest | Best stage path. Explain exactly what is broken. | +| Lab-only compliance policy with deterministic failure | Medium | Good if sync is behaving. Avoid waiting on CA. | +| Live policy reassignment | Lower | Good for workshops, risky for core session. | +| Real Conditional Access block | Lowest | Prefer screenshots or recording unless fully isolated. | + +Recommended failure: + +- Use a deterministic compliance or Defender-health test failure. +- Keep FileVault and Defender evidence in the same validation run. +- Explain where physical hardware validation begins. + +Narration points: + +- Intune state is eventually consistent. +- The lab proves policy delivery, device-side behavior, rollback, and evidence collection. +- Hardware/security-model-dependent behavior still requires physical Mac sign-off. +- Rollback is useful only if cloud cleanup is understood. +- VM rollback does not erase Intune/Entra/Defender audit history or instantly reverse portal state. + +Recovery playbook: + +- If enrollment is slow, pivot to pre-synced state. +- If compliance lags, show local evidence and return later. +- If Intune is unavailable, use the break-glass recording. +- If rollback fails, show snapshot tree and restore from `Pre-Enroll`. +- If a secret appears on screen, stop, switch to redacted screenshots, and continue calmly. +- Stay calm. The whole topic is safe failure. + +## Wrap-Up: 70:00-75:00 + +### Dragons Checklist, 70:00-73:00 + +Show a prebuilt triage table. + +| Symptom | Likely cause | First check | +| --- | --- | --- | +| VM enrolls but policy never arrives. | APNs, network, assignment, or sync delay. | APNs connectivity, Company Portal sync, Intune check-in, group/filter membership. | +| FileVault evidence is confusing. | Escrow and encryption stages are not the same. | Intune encryption report plus `fdesetup status`. | +| Recovery key not visible. | Escrow not complete, device not corporate, role permissions, or wrong report path. | Encryption report, device ownership, RBAC, policy timing. | +| Recovery key appears on screen. | Evidence redaction failed. | Stop showing live output; switch to redacted screenshot or sanitized JSON. | +| Defender installed but unhealthy. | System extension, network extension, Full Disk Access, or onboarding missing. | `mdatp health`, system extension list, PPPC profile. | +| Rollback causes duplicate/stale devices. | VM identity rolled back while cloud state moved forward. | Report-only cloud cleanup routine, then manual reconciliation. | +| Clones behave strangely. | MAC address, name, or identity collision. | Naming convention, clone settings, DHCP, device record. | +| Old guest no longer runs after host upgrade. | Host/provider compatibility changed. | Host version, provider release notes, restore image support. | +| UI prompts appear despite policy. | PPPC/TCC nuance or missing payload detail. | Bundle ID, code requirement, profile receipt, app logs. | +| Disk fills unexpectedly. | Snapshots and cached media sprawl. | Cleanup script and storage report. | + +Explicit dragons to name: + +- Apple ID and Setup Assistant friction. +- Host TCC permissions are app-specific. +- NAT vs. bridged networking. +- MAC address changes on clone. +- Host upgrade danger. +- Snapshot timing and MDM identity drift. +- FileVault escrow verification. +- Evidence redaction and recovery-key handling. +- PPPC/TCC prompt interpretation. +- Defender system extension and network extension approvals. +- Disk and snapshot sprawl. + +### Repo Handoff and Q&A Rules, 73:00-75:00 + +Show: + +- QR code. +- Short URL. +- Repo tree. +- `docs/Start-Here.md` highlighted. +- `examples/MMSMOA-2026` highlighted. + +Say: + +> Your first Monday-morning target is not your entire Mac program. It is one risky policy, one VM, one evidence bundle, and one rollback. + +Q&A rules: + +- Raise your hand. +- Mic runner if needed. +- One question per person first. +- Follow-ups if time permits. +- Include hypervisor and enrollment method if relevant. + +## Extended Q&A: 75:00-105:00 + +### Mechanics, 75:00-78:00 + +Leave the Q&A bucket slide up. + +No live polling. No Q&A app. Raise hands only. + +Say: + +> We will answer with the pattern, the gotcha, what to automate, and where it lives in the repo. + +### Q&A Buckets + +Six buckets. Few enough that the slide is a real menu rather than a wall of text, broad enough that almost any reasonable question lands in one. + +| Bucket | Example questions | +| --- | --- | +| Lab construction and lifecycle | How much RAM? Parallels or UTM? How do I keep old builds across host upgrades? How do I size for two concurrent VMs? | +| Enrollment, tenant strategy, and cleanup | Demo tenant or production? How do I avoid stale Intune/Entra records after a rollback? What is the right teardown ritual? | +| Risky-policy validation | What can I prove for FileVault in a VM? How do I really test Defender? What is the PPPC nuance I keep missing? | +| Compliance and Conditional Access timing | Why does Intune say one thing and Entra another? How do I avoid waiting on CA during a live demo? | +| Evidence and change-board outputs | What do I attach to a change ticket? What does Log Analytics ingestion look like? How do I redact recovery-key proof safely? | +| CI, Tart, and inventory adjacencies | When do I move beyond desktop labs? Should I bridge any of this into ConfigMgr inventory? | + +### Structured Answer Framework, 78:00-102:00 + +Use this pattern: + +1. Restate the scenario. +2. Recommend an approach. +3. Name the gotcha. +4. Say what to automate. +5. Point to the repo path. + +Example answer: + +> If you are using a post-enroll snapshot, the fast loop is great for policy regression. The gotcha is that Intune and Entra do not roll back with the disk. In v1, use the report-only cleanup routine to identify stale records and manual cleanup steps. Retire, delete, wipe, or other cloud mutation behavior belongs in a later owner-approved Phase 10 change. The repo path is `docs/Snapshot-Strategy.md` plus `scripts/Reset-IntuneMacLabDevice.ps1`. + +### Seed Questions if the Room Is Quiet + +Use these if needed: + +- "What is the scariest macOS policy in your environment right now?" +- "Who has a Mac policy that only one person on the team understands?" +- "Who has had compliance take longer than expected to show up?" +- "Who has a change board that wants more than 'the portal says assigned'?" +- "Who is trying to decide whether Tart belongs in a CI path?" + +### Close-Out, 102:00-105:00 + +End with the Monday plan: + +1. Clone the repo. +2. Build one VM. +3. Take a `Pre-Enroll` snapshot. +4. Enroll into a lab scope. +5. Test one risky policy. +6. Export redacted evidence. +7. Roll back. +8. Clean up the cloud record. + +Final line: + +> The win is not that you have a Mac VM. The win is that your next scary Mac policy has somewhere safe to fail. + +## Recommended Slide Skeleton + +| Slide | Purpose | Time | +| --- | --- | --- | +| 1 | Title and speaker credibility. | 0:00 | +| 2 | CEO Mac scenario. | 1:00 | +| 3 | Risk map. | 3:00 | +| 4 | Windows-admin translation table. | 6:00 | +| 5 | Apple Silicon lab constraints. | 7:30 | +| 6 | Licensing and concurrency caution. | 9:00 | +| 7 | Restore images, compatibility, build pinning. | 10:30 | +| 8 | AVF vs. QEMU practical explanation. | 12:00 | +| 9 | Fidelity traffic light. | 13:30 | +| 10 | Hypervisor decision matrix. | 15:00 | +| 11 | Parallels path. | 18:00 | +| 12 | UTM path. | 20:30 | +| 13 | Tart as advanced path. | 23:00 | +| 14 | PowerShell provider model. | 24:30 | +| 15 | Snapshot taxonomy. | 27:00 | +| 16 | Demo rules and checkpoint philosophy. | 30:00 | +| 17 | Demo 1 title card. | 31:00 | +| 18 | Demo 2 title card. | 38:00 | +| 19 | Demo 3 title card. | 50:00 | +| 20 | Demo 4 title card. | 57:00 | +| 21 | FileVault and Defender evidence model. | During Demo 4 | +| 22 | Dragons checklist. | 70:00 | +| 23 | Repo tree and Start Here. | 73:00 | +| 24 | Q&A buckets. | 75:00 | +| 25 | Monday plan and final reminder. | 102:00 | + +## Windows-Admin Translation Cheat Sheet + +Put this on one slide and ship it as `docs/Windows-Admin-Cheat-Sheet.md`. + +| What Windows admins know | macOS lab equivalent | Point to make | +| --- | --- | --- | +| Hyper-V checkpoint | Parallels/UTM snapshot through `Checkpoint-MacLabVm` | Same safety habit, different provider. | +| Restore-VMCheckpoint | `Restore-MacLabVmCheckpoint` | Rollback is the emotional payoff. | +| Windows ISO / WIM | macOS restore image or provider-appropriate install artifact | Pin exact builds. | +| BitLocker recovery key escrow | FileVault recovery-key escrow | Verify escrow before broad rollout. | +| Group Policy / gpupdate | Intune sync and compliance re-evaluation | Timing is different; evidence matters. | +| ConfigMgr collection / Intune group | Intune assignment group/filter | Keep lab rings isolated. | +| Event Viewer / Get-WinEvent | `log show`, profile output, app logs | Use PowerShell to collect both worlds. | +| Defender health on Windows | `mdatp health` on macOS | Health evidence is more useful than "app exists." | +| Pester tests | Pester tests for Mac lab readiness and policy validation | Same testing idiom. | +| Change-ticket evidence | Redacted JSON/CSV/screenshots/log bundle | Make proof portable without leaking secrets. | + +Verbal point: + +> The language does not change. PowerShell 7.4+ is the conductor. The instruments change. + +## GitHub Starter Kit + +The repo should be real, runnable, and usable after the session. + +### Recommended Repo Structure + +```text +/src/Modules/MacLab + MacLab.psd1 + MacLab.psm1 + Public/ + Get-MacLabMedia.ps1 + New-MacLabVm.ps1 + Get-MacLabVm.ps1 + Start-MacLabVm.ps1 + Stop-MacLabVm.ps1 + Checkpoint-MacLabVm.ps1 + Restore-MacLabVmCheckpoint.ps1 + Remove-MacLabVm.ps1 + Invoke-MacPolicyValidation.ps1 + Export-MacLabEvidence.ps1 + Private/ + Invoke-LoggedCommand.ps1 + Write-EvidenceRecord.ps1 + Resolve-MacLabConfig.ps1 + Protect-MacLabEvidence.ps1 + Providers/ + Parallels.ps1 + UTM.ps1 + Tart.ps1 +/scripts + Install-Prereqs.ps1 + Test-LabReadiness.ps1 + Get-MacOSRestoreImage.ps1 + New-MacInstallArtifact.ps1 + New-MacVm.ps1 + Checkpoint-MacVm.ps1 + Restore-MacVmCheckpoint.ps1 + Remove-MacVm.ps1 + Reset-IntuneMacLabDevice.ps1 + Send-LabEventToLogAnalytics.ps1 + Invoke-MMSDemo.ps1 +/examples + MMSMOA-2026/ + demo-config.yml + Demo1-Media.ps1 + Demo2-Parallels.ps1 + Demo3-UTM.ps1 + Demo4-IntuneValidation.ps1 + TestCases/ + FileVault-Validation.yml + Defender-Validation.yml + Compliance-SmokeTest.yml + PPPC-Validation.yml + utm/ + macos-lab-template.utm +/docs + Start-Here.md + Prereqs.md + Hypervisor-Decision-Guide.md + Apple-Silicon-Constraints.md + Provider-Version-Matrix.md + Fidelity-Boundaries.md + Snapshot-Strategy.md + Intune-Tenant-Setup.md + FileVault-Validation.md + Defender-Validation.md + PPPC-Validation.md + Evidence-and-CAB.md + Evidence-Redaction.md + Windows-Admin-Cheat-Sheet.md + Log-Analytics-Integration.md + ConfigMgr-Inventory-Bridge.md + CI-and-Tart.md + Troubleshooting.md + Demo-Runbook.md +/tests + MacLab.Tests.ps1 + Providers.Parallels.Tests.ps1 + Providers.UTM.Tests.ps1 + Validation.FileVault.Tests.ps1 + Validation.Defender.Tests.ps1 +/.github + workflows/ + powershell-ci.yml +README.md +LICENSE +SECURITY.md +TODO-Phase-04-Media-Acquisition.md +TODO-Phase-05-Parallels-Provider.md +TODO-Phase-06-UTM-Provider.md +TODO-Phase-08-Validation-Loop.md +TODO-Phase-10-Deferred-Work.md +``` + +Naming note: the redaction helper is `Protect-MacLabEvidence.ps1` rather than `Redact-MacLabEvidence.ps1` because `Protect` is on the PowerShell approved-verbs list and `Redact` is not. PSScriptAnalyzer will flag unapproved verbs, and any attendee who imports the module would get a yellow warning at load time, which undermines the "real, runnable, professional" promise. The user-facing parameter on `Invoke-MacPolicyValidation` stays `-RedactSecrets` because parameter names are not bound by the approved-verbs rule. + +Repository governance notes: + +- The future repo is initialized from `franklesniak/copilot-repo-template`. Keep the inherited instruction, security, CI, issue-template, and linting files unless a later owner-approved implementation step says otherwise. +- `SECURITY.md` stays unchanged by default. A future implementation agent may propose only the narrow no-real-secrets/no-recovery-keys paragraph already approved in the spec, and only if it fits the inherited template file. +- Root per-phase TODO files are required only when that phase still has deferred work. Omit a TODO file if the phase has no remaining action items. +- `Reset-IntuneMacLabDevice.ps1` starts report-only in v1. It can identify stale cloud records and explain manual cleanup steps; it must not mutate cloud records unless a later Phase 10 change is explicitly approved. + +### Minimum Viable Features + +Ship these even if everything else is rough: + +- `docs/Start-Here.md` +- Hypervisor decision guide. +- Provider version matrix. +- Snapshot strategy with cleanup warnings. +- One working Parallels path. +- One UTM example path or documented template path. +- FileVault validation notes. +- Defender validation notes. +- Evidence export example. +- Evidence redaction example. +- Pester 5.7.1 readiness test. +- PowerShell 7.4+ module manifest and CI path. +- GitHub Actions PowerShell CI using `macos-latest`; remove inherited Python CI only if no Python content remains. +- Troubleshooting table. +- Demo runbook. + +### Test Cases Worth Calling Out + +| Test case | What it proves | +| --- | --- | +| `FileVault-Validation.yml` | Policy receipt, local status capture, escrow evidence path, rollback note, and redacted recovery-key proof. | +| `Defender-Validation.yml` | System extension, network extension, Full Disk Access, onboarding, `mdatp health`. | +| `PPPC-Validation.yml` | Bundle ID/code requirement/profile receipt/app behavior. | +| `Compliance-SmokeTest.yml` | Fast deterministic pass/fail loop before risky tests. | + +### Evidence Output Example + +```json +{ + "runId": "2026-05-mms-demo4-001", + "vmName": "mms-parallels-01", + "provider": "Parallels", + "providerVersion": "", + "snapshot": "Post-Enroll-Baseline", + "hostMacOS": "", + "guestMacOS": "", + "guestBuild": "", + "powerShellVersion": "", + "pesterVersion": "5.7.1", + "intuneDeviceName": "MMS-MACLAB-001", + "redactionApplied": true, + "cloudStateWarning": "VM rollback does not rewind Intune, Entra, Defender portal state, audit logs, or reporting history.", + "tests": [ + { "name": "MDM enrollment profile present", "result": "Pass" }, + { "name": "FileVault status captured", "result": "Pass" }, + { "name": "FileVault escrow evidence captured", "result": "Pass" }, + { "name": "FileVault recovery key value redacted", "result": "Pass" }, + { "name": "Defender health captured", "result": "Pass" }, + { "name": "Compliance smoke test", "result": "Fail", "expectedFailure": true }, + { "name": "Rollback restored known-good VM checkpoint", "result": "Pass" }, + { "name": "Report-only cloud cleanup routine documented", "result": "Warn" } + ] +} +``` + +## Risk-to-Dollars Speaker Notes + +Do not overload the slide with numbers. Keep these in notes for conversational credibility. + +### FileVault Failure on an Executive Mac + +Possible impact: + +- 2-6 hours of executive downtime. +- Tier-3 escalation. +- Security team involvement. +- Physical recovery if recovery-key escrow was not verified. +- IT credibility hit that lasts longer than the incident. + +Speaker line: + +> The reputational cost to IT is usually bigger than the technical fix. + +### PPPC Misconfiguration + +Possible impact: + +- Remote support cannot control the screen. +- Recording fails during an important meeting. +- Accessibility-dependent tools stop working. +- EDR or backup agent loses needed access. + +Speaker line: + +> PPPC failures are quiet until the exact moment they become loud. + +### Defender Misconfiguration + +Possible impact: + +- Endpoint appears protected but is unhealthy. +- SOC gets bad data. +- Network extension issues cause user pain. +- Full Disk Access gaps reduce visibility. +- False positives or missing signals create avoidable escalations. + +Speaker line: + +> Defender on macOS is not just "install the app." The profiles are the deployment. + +### Compliance and Conditional Access + +Possible impact: + +- Outlook, Teams, SharePoint, or approval apps blocked. +- Helpdesk cannot explain why quickly. +- Change board loses trust. +- Business users see security as arbitrary. + +Speaker line: + +> Every one of these has a calendar event attached to it. Our job is to make sure the calendar event is a test run, not a Monday morning incident. + +## Memorable Patterns and Anti-Patterns + +### Patterns to Repeat + +| Pattern | Meaning | +| --- | --- | +| Safe place to fail | The lab is where mistakes belong. | +| Pin, prove, rollback | The three-part demo rhythm. | +| Evidence beats vibes | Portal assignment is not proof by itself. | +| Windows instincts still work | Testing rings, checkpoints, and evidence still matter. | +| VM first, hardware last | Iterate in VM; sign off on real hardware where needed. | +| Redacted proof | Show that the state exists without leaking the secret. | + +### Anti-Patterns to Name + +| Anti-pattern | Meaning | +| --- | --- | +| Portal faith | Trusting "assigned" without device-side evidence. | +| Snapshot time travel | Rolling back the VM while forgetting cloud state moved forward. | +| Progress-bar theater | Spending stage time watching downloads or sync. | +| Executive pilot roulette | Using the CEO's Mac as the first real test. | +| Haunted device object | A stale Intune/Entra record that no longer matches the rolled-back VM. | +| Tool-vibe selection | Picking a hypervisor by feel instead of operating model. | +| Screenshot leak | Showing a recovery key, token, tenant secret, or real user detail because the evidence path was not redacted. | + +Use these sparingly. They make the talk sticky. + +## Presenter Notes for Hard Questions + +### "Can this replace physical Mac testing?" + +Answer: + +> No. It reduces the number of mistakes that reach physical Mac testing. Use VMs for iteration, regression, payload validation, and evidence automation. Use physical hardware for final sign-off on hardware, enrollment, security, and user-experience paths. + +### "Should I use Parallels or UTM?" + +Answer: + +> If you need the most polished local enterprise workflow and can buy a commercial tool, start with Parallels. If you need no-cost experimentation and can accept more template/configuration work, start with UTM. If you want CI image distribution and CLI-first runner workflows, look at Tart as a second phase. + +### "Why PowerShell instead of shell scripts?" + +Answer: + +> Use the language your Microsoft endpoint team already operationalizes. PowerShell is not the only way. It is the bridge that lets Windows-first Intune admins own the workflow without pretending they became Mac platform engineers overnight. + +### "Can I test ADE in this lab?" + +Answer: + +> You can test pieces around enrollment and policy behavior, but zero-touch ADE behavior, serial-number-dependent flows, and production enrollment experiences need real hardware and ABM-connected test devices. + +### "Can I trust FileVault results in a VM?" + +Answer: + +> Trust the VM for early iteration, policy receipt, evidence workflow, and regression. Use real hardware for final FileVault rollout sign-off, boot unlock experience, recovery process, and executive pilot readiness. + +### "What about ConfigMgr?" + +Answer: + +> Treat ConfigMgr integration as reporting or inventory adjacency, not the primary control plane. The primary control plane for this session is Intune. If you want inventory, export lab metadata and decide whether it belongs in ConfigMgr, Log Analytics, or your CMDB. + +### "Can I run more than two macOS VMs?" + +Answer: + +> Treat two concurrent macOS VMs per Apple-branded host as the relevant boundary to design around, and verify your organization's interpretation with legal/procurement. More hosts, not just bigger hosts, may be the scaling answer. + +### "What if Intune takes too long during the demo?" + +Answer: + +> That is exactly why the lab uses checkpoints and evidence. Cloud timing is part of the system. We do not pretend it is instant; we design around it. + +### "Does rollback make Intune forget the failure?" + +Answer: + +> No. Snapshot rollback restores the VM state. It does not erase Intune, Entra, Defender portal state, audit logs, or reporting history. The starter kit treats cloud cleanup as part of the lab lifecycle because otherwise you create haunted device objects. + +### "Can you show the FileVault recovery key?" + +Answer: + +> Not on the projector. The proof we need is that escrow exists, the right role can retrieve it, access is audited, and the value is redacted in exported evidence. + +## Final Rehearsal Checklist + +### Two Weeks Before + +- Freeze demo host macOS version. +- Freeze hypervisor versions. +- Freeze demo tenant policies. +- Update placeholder build values throughout the deck and demo configs to your actual pinned build. +- Publish repo privately or publicly enough to validate links. +- Run full 105-minute rehearsal. +- Record Demo 4 success path. +- Rehearse the break-glass narration over the recording at least once. +- Validate screenshots. +- Validate QR code and short URL. +- Confirm both speakers know handoffs. +- Confirm evidence redaction behavior. +- Confirm no full recovery key, token, secret, or private tenant detail appears in any public artifact. + +### One Week Before + +- Run the full deck at projected resolution. +- Validate font sizes. +- Confirm all checkpoints. +- Export evidence from a clean run. +- Test hotspot or fallback network. +- Confirm repo short URL. +- Re-run `Test-LabReadiness.ps1`. +- Rehearse failure pivots. +- Re-check Parallels, UTM, Tart, Apple, and Microsoft documentation for any changed behavior or version notes. + +### Day Before + +- Disable auto-updates. +- Charge everything. +- Confirm local media cache. +- Confirm VMs start without prompts. +- Confirm Intune portal access. +- Confirm APNs/network path if possible. +- Close unrelated apps. +- Turn on Focus or Do Not Disturb. +- Confirm break-glass recording plays and the rehearsed narration still matches. +- Confirm evidence screenshots are redacted. + +### In the Room + +- Check projector scaling. +- Check terminal readability from the back if possible. +- Check mic handoff plan. +- Open all demo windows in order. +- Start from known snapshots. +- Keep Q&A bucket slide ready. +- Run T-15 smoke test. +- Do not improvise a new demo path. +- Do not show raw recovery-key, token, or secret output. + +## Final Self-Check Before Locking the Deck + +1. Every accepted takeaway maps to visible content. +2. FileVault is not just a scary intro story. +3. Defender validation is more than "app installed." +4. The Parallels demo does not depend on macOS Arm Coherence. +5. Media language says restore image or provider-appropriate artifact, not ISO as the default. +6. The media acquisition command uses `-Source Mist`, not `-Provider Mist`, to avoid overloaded terminology. +7. The media acquisition command or adjacent config explicitly states `arm64`. +8. The hypervisor matrix is a real operating-model comparison. +9. The fidelity traffic light is present. +10. Snapshot cleanup is documented. +11. Demo 4 says rollback restores the VM, not Intune/Entra/Defender cloud state. +12. The Windows-admin translation cheat sheet exists in the deck and the repo, and has its own minute on stage. +13. The repo URL resolves. +14. `Test-LabReadiness.ps1` returns green. +15. The break-glass recording exists, plays, and has been narrated against at least once. +16. Both speakers know who drives each segment. +17. At least three handoff lines have been rehearsed. +18. There is at least one seed question ready for Q&A. +19. The final line is memorized. +20. All angle-bracket placeholders in the deck and demo configs have been replaced with the pinned values for the event. The full set is ``, ``, ``, ``, ``, ``, ``, and ``. +21. All screenshots, recordings, logs, and evidence examples have been redacted. +22. Tenant names, tenant domains, UPNs, device IDs, serial numbers, recovery keys, tokens, and secrets are masked anywhere they appear. +23. All `.md` filename references in the runbook and any derivative artifacts render as plain inline code, not as auto-converted hyperlinks. +24. The redaction helper in the module is named `Protect-MacLabEvidence.ps1` (approved verb), and PowerShell tooling reports no verb warnings on module import or analysis. + +## Source Notes to Re-Check Before the Event + +Re-check these close to the event because Apple, Parallels, UTM, Tart, and Microsoft Intune change quickly. + +- Apple Software License Agreements. +- Apple Developer Virtualization framework documentation. +- Apple Support APNs and enterprise network requirements. +- Apple Platform Deployment PPPC payload documentation. +- Apple Platform Security notes for FileVault and Secure Enclave behavior. +- Parallels KB: installing macOS VMs on Apple Silicon. +- Parallels KB: known limitations of macOS Arm VMs. +- Parallels command-line interface documentation. +- UTM scripting and `utmctl` documentation. +- Tart documentation and license. +- mist-cli documentation. +- Microsoft Learn: macOS management in Intune. +- Microsoft Learn: FileVault with Intune. +- Microsoft Learn: Defender for Endpoint on macOS with Intune. +- Microsoft Learn: Defender macOS system extension and network extension troubleshooting. +- Microsoft Learn: Defender `mdatp health` output and troubleshooting. +- PowerShell approved verbs reference (verify any cmdlet rename against the current list). diff --git a/docs/planning/macOS-imaging-05-recommended-software.md b/docs/planning/macOS-imaging-05-recommended-software.md new file mode 100644 index 0000000..73ccc96 --- /dev/null +++ b/docs/planning/macOS-imaging-05-recommended-software.md @@ -0,0 +1,615 @@ +## Metadata + +- **Status:** Draft +- **Owner:** Frank Lesniak +- **Last Updated:** 2026-04-30 +- **Scope:** Talk-development working artifact for the MMSMOA 2026 session: "Submitting to the Midwest Management Summit at the Mall of America (MMSMOA) 2026". Captures interim concepting, prioritization, outline, or runbook content for that session; not a final published deliverable. +- **Related:** [Repository Copilot Instructions](../../.github/copilot-instructions.md), [Documentation Writing Style](../../.github/instructions/docs.instructions.md) + +## Submitting to the Midwest Management Summit at the Mall of America (MMSMOA) 2026 + +I thought of a high-level concept that I want to refine into a speaking proposal for the Midwest Management Summit at the Mall of America (MMSMOA). I spoke in three sessions at MMSMOA 2025, so I know the general expectations and feel good about my chances of being selected. + +### Midwest Management Summit at the Mall of America (MMSMOA) 2026 Call for Proposals (CFP) + +The following is the call for proposals (CFP) for the 2026 Midwest Management Summit at the Mall of America (MMSMOA): + +--- + +Attendees are seeking deep technical advise on all things related to systems management. + +Join us for an unforgettable experience! MMS 2026 at MOA is more than just an event; it's a community coming together to learn, network, and make a difference. We can't wait to welcome our 750 speakers, sponsors, and attendees. + +#### How to Submit + +Submit 2–5 sessions via Sessionize. + +Include: + +- Short, descriptive title +- Detailed description +- 3–4 key takeaways +- Optional: Request a co-speaker (not guaranteed). For best pairing, ask them to submit too. + +#### What We Look For + +- Topic originality and relevance to systems/endpoint management. +- Your expertise and presentation skills. +- Online visibility (e.g., blog, X, prior talks). + +#### Speaker Expectations + +- Attend the full event (Sunday–last session). +- Engage with attendees during sessions and networking. + +Questions? Contact info@mmsmoa.com + +#### Flight Reimbursement + +- We will reimburse up to $2,000 international & $1,000 US. +- You must submit your airline receipt and upload your PPT before deadlines. +- If you miss the deadlines, there will be no reimbursement. + +#### Hotel + +- MMS will cover all nights of the event plus one extra for US and two for international. +- You will get a code to book hotel. Expect to see a holding charge (not a full charge). + +#### FAQ + +##### Good Example of a Submission + +Title: 10 Intune Mistakes to Avoid for Optimal Endpoint Management + +Session Description: + +Unlock Intune's full potential! Learn about the top 10 common mistakes in Intune and how to avoid them, leveraging cutting-edge Intune technologies. Discover best practices for optimal endpoint management, application delivery, and robust security across PCs, macOS, and mobile devices. Walk away with inspirational ideas ready to use in your environment. + +Session Take-Aways: + +- Avoid common Intune mistakes from real-world experience +- Discover best practices for optimal endpoint management, application delivery, and robust security +- Learn how to leverage cutting-edge Intune technologies to replace outdated methods +- Gain ideas to implement in your environment + +##### Tips for Session Submission + +Plan for 75 min of content and 30 min of Q & A + +How deep is your content? **Give us (and attendees) an idea of the target audience.** Think of use cases on how your content will apply to the target audience. + +**Goals of the session** - we find that it's helpful to include information in your description that helps attendees understand what they will learn and the value they will obtain. + +We know that tech evolves fast - if you want to submit content on something that is not yet released, or expected to change before MMS, please include this information in your proposal - we will work with you to refine the session description as we approach the conference. + +**We require each session to have two presenters.** While some presenters prefer to present alone, we have seen many of our presenters really enjoy the opportunity to work with someone else. Many times, it was someone that they had known (or known of) for several years, but never really worked together or got to know. If you wish, we can pair you up with someone (and might have to if they're booked for another session at the same time). + +**Four (co-presented) sessions max for each speaker!** Many of our speakers have been overworked in the past – we are limiting speakers session count (not including Nerds of a Feather and Ask-the-Expert sessions). + +**Help us get to know you.** We know most of you, but we don't know all of you. When you submit your sessions, include information to help us learn more about you - your LinkedIn profile, blog posts, and other places on the web that you share content. This will help us determine if you (and your content) are a good fit as a presenter at MMS. + +MMS is a deep technical conference. We expect level 300, 400, and deeper content. Of course, 'levels' are subjective. **If you say it's a 400 level session, tell us how it's a 400 level session.** + +##### What is the difference between "Attendee Preview" and "Final PPT"? + +Attendees like to review session slides to help choose between conflicting sessions. Speakers sometimes prefer to not post their entire deck before presenting. The Attendee Preview version should contain enough information about your session for attendees to get the gist, and goes into more detail than the title and description on Sched. Your Attendee Preview deck must be posted to your session on sched.com according to the timeline on this page. (As it's visible on sched, it must be polished, and using our template). + +##### Session Selection + +We start session selections in December (for our May event) and May/June (for our fall event), meeting weekly until all slots are filled. If none of your sessions are selected, we might come ask you to co-present with someone else's session. + +##### What is a Co-speaker? + +A co-speaker is simply a second speaker in a session. At MMS, there's no concept of a "lead speaker." Both speakers share equal status and responsibility for meeting deadlines. MMS always features two speakers to foster a conversational tone in sessions, signaling to the audience that it's more of an exchange than a lecture. Feel free to present differing perspectives during a session; it helps attendees see that no single answer fully resolves a question. You can enjoy yourself and dive into technical depth simultaneously. + +##### Why Do You Pair Up Speakers? + +Our attendees love multiple perspectives, and as a speaker, it gives you a chance to collaborate with someone as well as share the load of a session, giving you a chance to pause, think, and add perspective while your co-speaker is talking. + +##### Why Can’t You Guarantee my Requested Co-speaker? + +For many reasons, but at its core, it’s a numbers thing – we simply can’t support adding all requested co-speakers, because we would have three times as many speakers and 25% fewer attendees if we did that. We will try to pair you with whom you requested, but if we can’t, we ask you to be flexible. If you consider not selecting your co-speaker a show-stopper, reconsider the sessions your submitting. + +##### "Be Present" Means What Exactly? + +Look at our attendee feedback each year and talk with speakers from previous MMS. We really want you to engage, before, during, and after your sessions, at lunch, in social gatherings, and anywhere else. Therefore, we ask you to wear your jacket, and why we don’t have a speaker room. The point of our conference is for attendees to connect with both you and other attendees – your job as a speaker is to help facilitate that. + +##### Must I Attend the Entire Conference? + +Absolutely! From the speaker meeting on Sunday to the end of the last session on the last day. As mentioned, this isn’t about a one-way presentation from you to attendees – this is about being present, engaging, and being available for the duration of the conference. Help us help attendees make MMS the best conference they can attend! + +##### I’m Not Social, Is That OK? + +Well, it will be difficult. If we don’t know anything about you, and you don’t appear to be stepping out, there’s little chance you’ll be selected. We acknowledge that sometimes it’s hard to come out of your shell, but we really need you to at MMS. + +##### How Can I Best Prepare? + +User groups! Go speak at user groups and encourage attendees to be social – we love seeing tweet storms where people talk about a presenter knocking it out of the park! + +##### MMS Code of Conduct + +MMS seeks to create a respectful, friendly, and professional experience for all participants. As such, we do not tolerate harassing or disrespectful behavior, messages, images, or interactions by any event participant, in any form, at any aspect of the program including business and social activities, regardless of location. + +We encourage everyone to assist in creating a welcoming and safe environment. Please report any concerns, harassing behavior, suspicious or disruptive activity to the nearest security guard or event staff. MMS reserves the right to refuse admittance to, or remove any person from, MMS at any time in its sole discretion. + +#### Deadlines + +| MOA | Midway | Task | +| --- | --- | --- | +| 7 days after acceptance | 7 days after acceptance | Complete your attendee profile (bio & picture) on sched.com | +| 15 MAR 2026 | 24 AUG 2026 | Submit your airline receipt to finances@mmsmoa.com. No airline reimbursement after this date! | +| 01 APR 2026 | 22 SEP 2026 | Attendee Preview version of PPT due. Send to info@mmsmoa.com and post to your session in sched. | +| 21 APR 2026 | 19 OCT 2026 | Final PPT. Upload to sched.com & email attendees from sched that they're posted. | + +#### Notes + +- There is no guarantee your co-speaker will be accepted, although we'll do our best. +- Review our speaker agreement for reimbursement rates, deadlines, expectations, etc. +- Your submission should be technical (generally 300-400 level) and will include 65-75 minutes of presentation (with your agreed-upon co-speaker) and 30-40 minutes for questions and answers. +- We expect you to actively encourage dialog between the audience and your co-speaker—our attendees love this format! +- If you submit a part 1/part 2 set of sessions, we'll do our best to schedule them before and after lunch. + +### Unofficial Information About the Audience at MMSMOA + +MMSMOA is a Microsoft large enterprise sysadmin/endpoint engineer-focused conference. The attendees at this event are Microsoft sysadmins or endpoint engineers. 70% of the audience work for huge companies and work with Microsoft Intune or ConfigMgr, deploying/configuring/patching Windows and software packages. The rest are a combination of Windows Server sysadmins, Azure engineers, developers, IT managers (ex-sysadmins or ex-endpoint engineers), PowerShell enthusiasts, and an occasional cybersecurity-interested person. There are also some Microsoft technology generalists in the audience. + +Some developer-oriented folks may use a Mac, but most use Windows. + +## Talk Concept + +The *draft* concept for this talk is as follows: + +Enterprises keep buying Macs, but most Microsoft admins still deploy Intune scripts and ConfigMgr payloads to macOS with fingers crossed—and one bad FileVault profile can lock out an executive in seconds. This session hands those admins a safety net: a reproducible macOS VM lab that lives on a single Apple-silicon Mac yet is orchestrated entirely with PowerShell 7, the language they already wield every day. + +We open by quantifying the business risk of untested macOS policies—lost user trust, ticket volume, conditional-access outages—and link each risk to dollars and executive visibility. With stakes clear, we introduce the three viable virtualization paths on Apple silicon: **Parallels Desktop Pro** for polished enterprise workflows, **UTM** for cost-free experimentation, and **Tart** for pipeline-grade CLI control. We decode Apple’s virtualization EULA (two concurrent macOS VMs per host, host-OS-must-be-same-or-newer rule, no Secure Enclave access) and the licensing math (macOS is license-free on Apple hardware but Parallels is commercial). + +PowerShell becomes the universal conductor. A single cross-platform script: + +1. Fetches an *exact* macOS build via `softwareupdate` or `mist-cli`. +2. Builds an ISO with `hdiutil`. +3. Drives `prlctl`, `utmctl`, or `tart` to spin up parameterized VMs, set CPUs/RAM/disk, and tag snapshots. +4. Boots an “Intune-ready” desktop, enrolls to a demo tenant, applies a compliance policy, deliberately violates it, and then rolls back—all from PowerShell. + +Four timed demos punctuate the talk: + +- **Demo 1 (3 min):** Image fetch + ISO build—“from Apple CDN to bootable media” live. +- **Demo 2 (5 min):** Parallels VM spun up and snapshot in under 60 seconds; delivers the Gr-inspired Coherence Mode “wow” by running macOS and Windows apps side-by-side. +- **Demo 3 (4 min):** Same ISO booted in UTM/Tart to highlight tooling choice and fully CLI-driven pipelines. +- **Demo 4 (8 min):** Intune enrollment, config push, break-and-rollback—showing exactly how to test FileVault, PPPC, and Defender before production. + +Throughout, code snippets map directly to Windows-admin muscle memory: `Invoke-Command`, error-handled `prlctl` calls, and logging to Azure Log Analytics. Practical gotchas—Apple-ID prompts, virtualization-specific network tuning, ADE limits—are surfaced in a concise checklist so attendees know where dragons hide. + +Every artifact—scripts, Pester tests, and a Makefile-style wrapper—lands in a public GitHub “starter kit” released at session end, ensuring Monday-morning applicability. The 30-minute extended Q\&A that follows uses classic raise-your-hand interaction (or mic runners in a large room) to tackle niche scenarios like integrating the lab with ConfigMgr hardware inventory or automating VM teardown in CI. + +By marrying a familiar automation language with hard Apple constraints, delivering high-energy demos, and handing over production-quality scripts, this session promises concrete Monday-morning value, lively engagement, and the technical depth that conference reviewers look for in a 105-minute flagship slot. + +## Submitted and Accepted Session + +- Session title: Don't Brick the CEO's Mac: Building and Automating macOS Labs for Risk-Free Policy Testing +- Primary Speaker/Submitter: Frank Lesniak +- Co-Speaker Name: Michael Niehaus +- Description (NOTE: 600 character limit!): + + You have a lab for testing Windows--that's easy--but do you have the same for Macs? Deploying untested macOS policies is a ticking time bomb. One wrong move with FileVault, and you've locked out an executive. This session hands you a safety net: a reproducible macOS VM lab on Apple Silicon, orchestrated with PowerShell 7. We will use Parallels or UTM to spin up VMs, enroll them in Intune, apply policies, and roll back. With live demos and a GitHub starter kit, you'll leave ready to test configurations without the anxiety. +- Session Length: 1 hour 45 minutes, including 30 minutes of “extended Q&A” +- Session Takeaway 1: Analyze the virtualization constraints of Apple Silicon and select the appropriate hypervisor (Parallels vs. UTM) that aligns with your budget and automation needs. +- Session Takeaway 2: Construct a fully automated, reproducible macOS test lab using PowerShell 7 to fetch restore images/install media and control VM states. +- Session Takeaway 3: Execute end-to-end validation of high-risk policies (FileVault, Defender) by enrolling, breaking, and rolling back VMs via script—keeping production safe. +- Session Takeaway 4 (optional): Implement the provided GitHub starter kit immediately to bridge the gap between your Windows automation skills and macOS requirements. +- Primary Technology (choose one): Intune +- Session Type (multi-select): + - Automation + - Provisioning & Deployment + - Setup & Configuration + +## Session Length + +105 minutes, including 30 minutes of "extended Q&A". + +## More Information and Constraints + +- I cannot change the session length or the division of time. The core session must be at least 60 and up to 75 minutes, plus 30 minutes of "extended Q&A." +- When it comes to asking questions during extended Q&A, the audience will raise their hands and then ask when called on (or, if we are in a very large room, a helper will run a microphone over to them); we will not use any sort of live polling to facilitate the extended Q&A. + +## Draft Session Outline + +Here is a draft outline of the talk: + +`````markdown + + +# Don't Brick the CEO's Mac — Session Outline + +## 1) Design decisions (explicitly resolving disagreements) + +### A) Opening/risk framing + +**Decision:** Keep it short (hook + one risk slide) and move quickly into constraints, tools, and demos. +**Why:** MMS is deep technical and demo-heavy; long intros cause overruns. + +**Optional:** One brief hand-raise calibration ("Who manages Macs in Intune?") but don't overuse audience interaction. + +### B) Hypervisor scope + +**Decision:** Make **Parallels + UTM** the core (matches accepted abstract). Mention **Tart** as optional/advanced (pipeline path) with deeper content in the repo/Q&A. +**Pros:** Abstract fidelity, lower time risk. +**Cons:** Less stage time for CI/pipeline; solve via repo docs and Q&A buckets. + +### C) Intune depth: workflow-first + one "proof point" + +**Decision:** Live demo focuses on **workflow** (enroll → apply → validate/break → rollback). Include **one** advanced verification snippet (Graph query or payload excerpt) only if time permits. Keep the deeper Graph/payload automation in the repo. +**Why:** Reliability > breadth in a 75-minute demo-heavy core. + +### D) Snapshot strategy (pre-enroll vs post-enroll baselines) + +There is **no one universal right answer**, so teach both, with clear pros/cons and guardrails: + +- **Pre-Enroll snapshot (clean identity baseline)** + - **Pros:** Least weird MDM identity behavior; closest to "clean device" semantics + - **Cons:** Slower; relies on enrollment/sync timing + +- **Post-Enroll-Baseline snapshot (fast regression loop)** + - **Pros:** Best for rapid repeated tests and stage demo reliability + - **Cons:** Potential MDM "time travel" issues (cloud state moves forward while VM rewinds) + +**Recommended operational rule to teach:** + +- Use **Pre-Enroll** as the "gold standard" baseline. +- Use **Post-Enroll-Baseline** for speed, but pair it with a documented **reset/cleanup routine** appropriate to your enrollment method (e.g., remove/retire/wipe device record, ensure certificate/device identity consistency, etc.). Document this explicitly in `Snapshot-Strategy.md`. + +### E) "Theatrical oops" vs calm controlled failure + +**Decision:** Use a controlled, deterministic failure and narrate it calmly. +**Why:** Reduces risk of seeming gimmicky; preserves credibility if something genuinely fails. + +### F) Repo timing and live `git clone` + +**Decision:** Publish repo **before** the session (link reliability). Reveal QR/URL late (final minutes of core) to avoid distraction. + +**Live `git clone` options:** + +- **Option 1 (recommended hybrid):** Show the command you would run, then open a **pre-cloned local copy** to demonstrate repo contents. + - **Pros:** Persuasive + no network dependency + - **Cons:** Slightly less "live internet proof" +- **Option 2:** Do a real `git clone` on stage. + - **Pros:** Very convincing + - **Cons:** Can fail due to network/DNS/portal issues + +### G) Demos: "keyboard-time targets" vs wall-clock reality + +**Decision:** Treat the 3/5/4/8 minute demo durations as **keyboard-time targets**. Allocate **buffered demo blocks** and use prebuilt checkpoints to stay on schedule. + +--- + +## 2) Speaker choreography (two presenters, intentional) + +### Driver/Narrator model (for every demo) + +- **Driver:** controls keyboard/mouse for that demo segment. +- **Narrator:** explains what's happening, calls out flags/gotchas, fills dead air during latency. + +### Segment ownership + +- Each section has one "thread owner." +- The other speaker interjects with: + - enterprise reality checks + - alternative viewpoints + - failure modes and how to recover + +### Planned handoff lines (script these) + +Prepare 5–6 handoff sentences in speaker notes to avoid awkward transitions. + +--- + +## 3) Pre-session stage-readiness checklist (internal runbook) + +### Hardware + +- **One Apple silicon Mac** for the entire demo chain. +- Prioritize **RAM and SSD free space** (snapshots can consume tens of GB quickly). +- Ensure stable power and a known-good A/V adapter. + +### Readability + +- Terminal/VS Code font size: target **~24pt** (or whatever is readable from the back row). +- Use a window manager or consistent snapping so split-screen layouts are identical across demos. + +### Host macOS permissions (TCC) + +- Grant **Automation** and **Full Disk Access** (as needed) to the **exact app** running PowerShell (Terminal/iTerm2/VS Code). +- Avoid switching between apps mid-demo. + +### Assets and checkpoints (non-negotiable) + +- Cache installers/media locally. +- Pre-bake VMs at: + - `Clean-OS` + - `Pre-Enroll` + - `Post-Enroll-Baseline` (enrolled + recently synced) + +### Venue network risk planning (APNs + Intune) + +- macOS management relies on **APNs**, commonly **TCP 5223**; conference networks can block/shape traffic. +- Do not rely on venue Wi‑Fi for critical steps. +- If allowed, have a fallback network plan (e.g., hotspot). + +### "Break-glass" contingency + +- Optional but recommended: a **60–90 second screen recording** of Demo 4 working perfectly (apply → fail → rollback). Use only if there's a confirmed service outage. + +--- + +## 4) Technical content that must be correct/explicit (400-level credibility) + +### Apple virtualization internals (keep it to one slide) + +- Explain **Apple Virtualization Framework (AVF)** vs **QEMU-based** approaches at a high level. +- Key impact: tool automation surfaces differ; security model differs; Secure Enclave is not exposed to macOS VMs. + +### Fidelity boundaries: name at least one example + +- Explicitly state: you cannot fully validate some hardware/security-model-dependent flows in a VM (example: **Platform SSO**). +- Guidance: VM lab for iteration/regression; real hardware for final sign-off where needed. + +### Host upgrade danger (regression testing foot-gun) + +- Host OS version rules can limit which older macOS guests you can run. +- Treat the lab host like a build server: don't auto-upgrade immediately; maintain a stable host for regression. + +--- + +## 5) Core timeline (75 minutes) + +### 0:00–0:30 — Title + micro-credibility (30s) + +- One sentence each: why you're qualified and what you cover. + +### 0:30–2:30 — Hook: CEO lockout (2m) + +- Stakes + promise: "We'll build a safety net and prove it with rollback." + +### 2:30–6:30 — Risk map (4m) + +- One slide: FileVault / PPPC / Defender / Compliance+CA. +- Optional 10-second hand raise: "Who manages Macs in Intune?" + +### 6:30–14:00 — Constraints that drive lab design (7m 30s) + +- Why not Hyper-V/ESXi? +- Apple silicon constraints (capacity rules, host/guest OS) +- AVF vs QEMU (one slide) +- No Secure Enclave; fidelity boundary example: Platform SSO +- Practical friction: Apple ID prompts, networking, ADE realities, host TCC permissions +- Host upgrade danger warning + +### 14:00–24:00 — Hypervisor decision (10m) + +- Parallels vs UTM as core choices +- Tart as optional "CI path" +- Decision rubric: budget, UX vs automation, snapshots/clones, update cadence + +### 24:00–30:00 — PowerShell provider-model architecture (6m) + +- End-to-end flow diagram: + 1) pin build → 2) acquire media → 3) create VM (params) → 4) snapshot → 5) enroll → 6) apply policy → 7) validate/break → 8) rollback → 9) evidence output → 10) teardown +- Snapshot taxonomy: + - `Clean-OS` + - `Pre-Enroll` + - `Post-Enroll-Baseline` (with caveats + cleanup routine) +- Naming convention + tagging +- Sizing profiles (`Baseline` vs `Performance`) +- "Tech depth marker" cadence: include one advanced anchor per segment (flag/error mode/verification/Pester check) + +### 30:00–31:00 — Gotchas heads-up (1m) + +- Downloads/sync delays; we'll pivot to checkpoints. +- CA/compliance delay exists. +- Host TCC permissions can block automation. + +--- + +## 6) Demo block (31:00–70:00) + +### 31:00–38:00 — Demo 1: Fetch exact macOS build + create media (7m) + +- Show pinned build acquisition (`softwareupdate` and/or `mist-cli`) +- Show ISO/media creation (`hdiutil`) +- Emphasize caching and reproducibility + +**Contingency:** leave command visible; pivot to cached output. + +### 38:00–50:00 — Demo 2: Parallels VM build + snapshot + brief Coherence (12m) + +- PowerShell wrapper drives `prlctl` +- Apply sizing profile (`Baseline` or `Performance`) +- Snapshot to `Pre-Enroll` (and/or show taxonomy) +- Time-box Coherence "wow" to 60–90 seconds + +**Contingency:** pivot to prebuilt VM at `Pre-Enroll` or `Post-Enroll-Baseline`. + +### 50:00–57:00 — Demo 3: UTM provider swap (or Tart cameo) (7m) + +- Demonstrate provider-agnostic pattern +- Explain UTM automation may rely on templates/config artifacts (not identical CLI parity) +- Optional Tart cameo: 1–2 CLI commands + "details in repo" + +### 57:00–70:00 — Demo 4: Intune validation loop (13m) + +**Goal:** prove risk-free policy testing with rollback. + +Recommended reliable path: + +- Start from `Post-Enroll-Baseline` (pre-synced) +- Apply risky policies (FileVault/PPPC/Defender) + deterministic compliance failure +- Show noncompliance and/or CA impact (if configured) +- Roll back snapshot, prove return to known good + +Narration: + +- Explain Intune → compliance → Entra CA evaluation delay +- Show evidence outputs (logs, pass/fail, screenshots) + +Optional single advanced proof point: + +- One Graph query verifying a key behavior (e.g., FileVault recovery key escrow) OR one payload excerpt—keep to 60 seconds. + +**Contingency:** pivot to pre-synced state; use break-glass recording only if necessary. + +--- + +## 7) Wrap-up (70:00–75:00) + +### 70:00–73:00 — Dragons checklist + mini triage table (3m) + +Include specific gotchas: + +- Apple ID/setup assistant friction +- Host TCC permissions (app-specific) +- NAT/bridged networking +- **MAC address regeneration on clone** and why it matters +- Host upgrade danger for regression testing +- Snapshot timing and MDM identity drift +- FileVault escrow verification +- PPPC/TCC prompts interpretation +- Defender extension approvals +- Disk/snapshot sprawl + cleanup/teardown scripts + +Mini triage: symptom → likely cause → first check. + +### 73:00–75:00 — Repo handoff + Q&A rules (2m) + +- QR + short URL +- Repo tree and "Start Here" +- Explain Q&A rules: one question per person; follow-ups only if time permits. + +--- + +## 8) Extended Q&A (75:00–105:00) + +### 75:00–78:00 — Mechanics + topic buckets (3m) + +Leave buckets slide up: + +- Hypervisor choice/sizing +- Image pinning/updates +- Enrollment & tenant strategy (Dev/Test/Prod) +- FileVault/PPPC/Defender testing +- Compliance/CA timing +- Evidence/CAB outputs +- CI/pipelines (Tart optional) +- Teardown/cleanup + +### 78:00–102:00 — Q&A (24m) + +Use a structured answer framework: + +1) clarify scenario +2) recommended approach +3) gotchas + automate vs manual +4) "here's the repo path" + +Seed topics (tiered): + +- Tier 1: blockers, scariest policy, Parallels vs UTM choice +- Tier 2: CAB evidence, ABM/ADE strategy, CI wiring, ConfigMgr integration + +### 102:00–105:00 — Close-out (3m) + +- Monday plan: "Clone → build VM → snapshot → test one risky policy → rollback." +- Repo reminder +- Where to find speakers after session; evaluation reminder (brief) + +--- + +## 9) GitHub starter kit contents (complete) + +### Repo structure (suggested) + +```text +/src/Modules/MacLab + Providers/Parallels.ps1 + Providers/UTM.ps1 + Providers/Tart.ps1 (optional) +/scripts + New-MacosInstallMedia.ps1 + New-MacVm.ps1 + Checkpoint-MacVm.ps1 + Restore-MacVmCheckpoint.ps1 + Remove-MacVm.ps1 + Test-LabReadiness.ps1 +/examples/TestCases + Compliance-SmokeTest.ps1 + FileVault-Validation.ps1 + PPPC-Validation.ps1 + Defender-Validation.ps1 +/docs + Start-Here.md + Prereqs.md (incl. TCC + APNs notes) + Snapshot-Strategy.md (Pre-Enroll vs Post-Enroll caveats + cleanup) + Naming-Conventions.md + Troubleshooting.md + Evidence-and-CAB.md + CI-and-Tart.md (optional) +/tests + MacLab.Tests.ps1 +``` + +### Key features + +- Provider model with stable PowerShell interface +- Snapshot taxonomy + naming conventions +- Sizing profiles selectable by parameter +- Readiness gate (Pester/prechecks) +- Evidence outputs (CAB-friendly) +- Cleanup/teardown scripts +- Optional PlatyPS docs generation (if implemented) +- Optional Makefile-style wrapper (or equivalent) for one-command flows +- Optional advanced: Tart + OCI/registry distribution notes (documented, verified) + +--- + +This is the complete blended plan: abstract-aligned, MMS-safe, demo-reliable, with all unique expert insights incorporated and disagreements resolved with explicit decisions and guardrails. +````` + +## Hardware Procurement + +I am procuring the following hardware to support this talk: + +```markdown + + +## Recommended Hardware for Shared Use, Outdoor Use, and VM Lab Work + +- **Device:** 14-inch MacBook Pro +- **Chip:** Apple M5 Pro +- **CPU/GPU:** 18-core CPU / 20-core GPU +- **Memory:** 64GB unified memory +- **Storage:** 2TB SSD +- **Display:** Nano-texture display + +### Rationale + +- **14-inch size:** Best fit for the portability constraint while still giving enough performance and ports for serious work. +- **M5 Pro (18-core CPU / 20-core GPU):** Best balance of performance, battery life, portability, and cost for your use case. +- **64GB memory:** The safest recommendation for macOS VM work, Docker/container use, browser/admin portals, coding tools, and normal multitasking. +- **2TB SSD:** Best balance for VM images, snapshots, repos, package caches, logs, apps, and normal shared-family use. +- **Nano-texture display:** Worth it here because the laptop is likely to be used outdoors, where glare/reflections matter more. +- **Why not M5 Max:** It mostly buys extra headroom rather than a clearly necessary capability for this workload. +- **Why not 4TB SSD:** Useful only if the laptop will be treated as a full-time all-in-one lab box and you want to avoid storage cleanup entirely. + +### Bottom line + +**Recommended purchase:** + +- 14-inch MacBook Pro +- M5 Pro +- 18-core CPU / 20-core GPU +- 64GB unified memory +- 2TB SSD +- Nano-texture display +``` + +## What I Need Help With + +Can you identify what software I should procure and install to successfully deliver this talk? Be specific--for any commercial software, tell me what "edition" or license level is recommended. diff --git a/docs/planning/macOS-imaging-08c-repo-spec-final.md b/docs/planning/macOS-imaging-08c-repo-spec-final.md new file mode 100644 index 0000000..9c6c924 --- /dev/null +++ b/docs/planning/macOS-imaging-08c-repo-spec-final.md @@ -0,0 +1,1781 @@ + +# macOSLab Repository Specification + +## Metadata + +- **Status:** Draft for owner approval +- **Owner:** Frank Lesniak +- **Last Updated:** 2026-05-04 +- **Scope:** Human-readable implementation specification for creating the public `franklesniak/macOSLab` repository from the [`franklesniak/copilot-repo-template`](https://github.com/franklesniak/copilot-repo-template). It defines repository identity, authority hierarchy, safety rules, functional requirements, file layout, PowerShell module contracts, provider contracts, scripts, examples, documentation, tests, CI, phase gates, and definition of done. +- **Related:** [Original prompt](macOS-imaging-08-repo-spec.md), [Bolstered outline](macOS-imaging-03a-bolstered-outline.md), [Software runbook](macos-imaging-05c-recommended-software-step-by-step-revised.md), [Repository description](macos-imaging-06a-repo-description.md), [CFP submission](macOS-imaging-01a-CFP-submission.md), [Closed questions archive](macOS-imaging-08d-closed-questions-archive.md), [Architecture decision records](macOS-imaging-08e-ADRs.md), [Repository Copilot Instructions](../../.github/copilot-instructions.md), [Documentation Writing Style](../../.github/instructions/docs.instructions.md), [PowerShell Writing Style](../../.github/instructions/powershell.instructions.md) + +## 1. Approval Summary + +This document describes the repository that should be built later. It is not the repository itself. + +The future repository is named `macOSLab` and will be hosted at `franklesniak/macOSLab`. It will be initialized from [`franklesniak/copilot-repo-template`](https://github.com/franklesniak/copilot-repo-template), then extended into a PowerShell 7.4-or-newer starter kit for building reproducible macOS VM labs on Apple-silicon Macs. + +The repository has one central promise: + +> Help Microsoft endpoint administrators safely test risky macOS Intune policies in a reproducible VM lab before those policies reach real users. + +The repository MUST help a user: + +1. Install and verify host prerequisites. +2. Pin and acquire macOS restore images or provider-appropriate install artifacts. +3. Create or register macOS VMs through a provider abstraction. +4. Capture and restore named checkpoints. +5. Enroll VMs into Intune for lab-only validation. +6. Validate FileVault, Defender, PPPC/TCC, and compliance signals within documented fidelity limits. +7. Export redacted evidence suitable for demos, change review, and audit-style review. +8. Explain clearly what the VM lab can prove and what still needs physical Mac sign-off. + +The repository MUST NOT pretend that macOS VMs replace real hardware validation. It MUST NOT expose secrets, recovery keys, tenant identifiers, credentials, or personal data in logs, screenshots, examples, tests, or evidence bundles. + +## 2. Design Posture + +This repository is intended to be: + +- A clear starter-kit repository, not a full enterprise product. +- PowerShell 7.4-or-newer automation for Apple-silicon macOS hosts. +- Parallels first, UTM second, and Tart optional/stubbed unless owner-approved later. +- Evidence-first and redaction-first. +- VM-first and hardware-last: VMs speed up iteration, but physical Macs remain required for Red-bucket and some Yellow-bucket outcomes. +- Phase-gated so the owner can approve each step before the agent continues. + +## 3. Reader Guide + +Use this document in three passes: + +1. Read Sections 1 through 10 to decide whether this is the right repository to build. +2. Read Sections 11 through 20 to approve the functional surface and implementation contract. +3. Give the entire file to the future coding agent and require them to treat Sections 21 through 25 as the build plan, acceptance criteria, and sign-off mechanism. + +If you only have 15 minutes: + +1. Read Section 4, "Approval Checklist." +2. Confirm Section 5, "Repository Identity." +3. Read Section 9, "Security and Redaction Requirements." +4. Skim Section 14, "Repository File Map." +5. Read Section 21, "Implementation Phases." +6. Consult the [closed questions archive](macOS-imaging-08d-closed-questions-archive.md) only if you need historical rationale for a decision. + +## 4. Approval Checklist + +Before the future agent begins work, the owner SHOULD initial each box that matches expectation. If an item cannot be accepted, strike it through, add a note, or create a new explicit decision item before implementation starts. + +### 4.1 Identity and Visibility + +- [ ] The repository will be created at [`franklesniak/macOSLab`](https://github.com/franklesniak/macOSLab). +- [ ] The repository will be public. +- [ ] The license will be MIT. +- [ ] The default branch will be `main`. +- [ ] The initial CODEOWNERS value will be `* @franklesniak` unless the owner names additional code owners. + +### 4.2 Source of Truth and Standards + +- [ ] The new repository will be initialized from [`franklesniak/copilot-repo-template`](https://github.com/franklesniak/copilot-repo-template). +- [ ] The new repository's `.github/copilot-instructions.md` will be its canonical constitution. +- [ ] PowerShell coding standards will be inherited from the template and the referenced PowerShell style guide. +- [ ] The future agent is not authorized to modify protected instruction files unless that exact instruction-file change is explicitly authorized in the current task. + +### 4.3 Scope and Posture + +- [ ] The repository is a starter kit/scaffold, not a finished enterprise product. +- [ ] The repository is VM-first, hardware-last: it does not promise to validate ADE/ABM zero-touch enrollment, Platform SSO sign-in/unlock, Touch ID, Secure Enclave-dependent behavior, or final FileVault rollout. +- [ ] The supported host is Apple silicon running macOS. Intel Mac hosts are out of scope. +- [ ] The primary hypervisor is Parallels Desktop Pro Edition. +- [ ] The secondary hypervisor is UTM. +- [ ] Tart is optional and may ship as a documented stub provider in v1. +- [ ] The orchestration language is PowerShell 7.4 or newer. PowerShell 5.1 compatibility is not a goal. + +### 4.4 Security and Redaction + +- [ ] No real tenant identifiers, UPNs, device IDs, serial numbers, recovery keys, tokens, app secrets, or personally identifying data may appear in any committed artifact. +- [ ] All evidence output must pass through `Protect-MacLabEvidence` before being written to disk. +- [ ] No demo path may instruct the user to display a raw FileVault recovery key on a projector or commit one to the repository. +- [ ] No secret-management service, telemetry service, or external logging service may be added without explicit owner approval. + +### 4.5 Module Surface + +- [ ] The PowerShell module is named `MacLab`. +- [ ] The public cmdlets are exactly the ten cmdlets listed in Section 16. +- [ ] The redaction helper is `Protect-MacLabEvidence`; `Redact-MacLabEvidence` must not be introduced. +- [ ] Media acquisition uses `-Source Mist`; `-Provider` is reserved for hypervisors. +- [ ] The five-checkpoint vocabulary is `Clean-OS`, `Pre-Enroll`, `Post-Enroll-Baseline`, `Broken-Policy-State`, and `Recovered-Known-Good`. + +### 4.6 Phasing and Gates + +- [ ] The build is delivered in phases. +- [ ] The agent must stop at each phase gate and wait for owner approval before continuing. +- [ ] The owner accepts that v1 may ship without full Tart, ADE, Log Analytics, or ConfigMgr inventory adapters if those phases are deferred. +- [ ] The owner accepts that the agent will open `open-question` issues rather than guess when requirements are ambiguous. + +## 5. Repository Identity + +| Field | Value | +| --- | --- | +| Repository slug | `macOSLab` | +| Owner / GitHub user | `franklesniak` | +| Canonical URL | [`franklesniak/macOSLab`](https://github.com/franklesniak/macOSLab) | +| Visibility | Public | +| Default branch | `main` | +| License | MIT, with copyright line `Copyright (c) 2026 Frank Lesniak` | +| GitHub description | `Reproducible Apple-silicon macOS VM lab starter kit, automated with PowerShell 7.4+. Pin macOS media, build snapshots in Parallels or UTM, enroll into Intune, validate FileVault/Defender/PPPC, fail safely, roll back, and export redacted evidence.` | +| GitHub topics | `macos`, `apple-silicon`, `intune`, `powershell`, `pwsh`, `parallels-desktop`, `utm`, `vm-lab`, `policy-testing`, `filevault`, `defender-for-endpoint`, `pppc`, `tcc`, `mdm`, `microsoft-graph`, `pester`, `endpoint-management`, `change-management`, `rollback`, `evidence` | +| Initial release tag | `v0.1.0-mmsmoa-preview`, cut after Phase 9 with owner approval | + +### 5.1 Companion Identifiers + +| Field | Value | +| --- | --- | +| PowerShell module name | `MacLab` | +| Module manifest path | `src/Modules/MacLab/MacLab.psd1` | +| Module loader path | `src/Modules/MacLab/MacLab.psm1` | +| Module GUID | Owner decision. Generate once with `[guid]::NewGuid()` and pin for the life of the module. | +| Initial module version | `0.1.0` | +| PowerShell floor | PowerShell `7.4` minimum; support newer PowerShell 7.x releases unless a concrete incompatibility is documented. | +| Pester dependency | Pester `5.7.1` pinned in CI for the initial implementation. | +| Microsoft Graph dependency | Individual submodules only: `Microsoft.Graph.Authentication`, `Microsoft.Graph.DeviceManagement`, and optionally `Microsoft.Graph.Beta.DeviceManagement` for FileVault escrow proof. Do not require the `Microsoft.Graph` meta-module by default. | + +### 5.2 Reserved Names Not Introduced in v1 + +The following names are reserved or forbidden to prevent drift: + +- `Redact-MacLabEvidence`: forbidden because `Redact` is not an approved PowerShell verb. +- `Get-MacLabMedia -Provider Mist`: forbidden because `-Provider` is for hypervisors. +- "Three required snapshots" or any synonym implying fewer than five canonical checkpoints. +- `MacLabPro`, `MacLabStarterKit`, or any other v1 repo-name variant. + +## 6. Authority Hierarchy + +The future coding agent MUST resolve conflicts in this order, top wins: + +1. The new repository's `.github/copilot-instructions.md`. +2. The new repository's modular instruction files under `.github/instructions/`. +3. The new repository's root agent instruction files: `AGENTS.md`, `CLAUDE.md`, and `GEMINI.md`. +4. This specification. +5. The bolstered outline and the software runbook. +6. External vendor documentation, when a current command, API, license, or behavior must be verified. + +If the new repository's instructions differ from the planning repository's instructions, the new repository's instructions win for code that lives in the new repository. + +### 6.1 Protected Files + +The agent MUST NOT create, edit, delete, rename, or otherwise change these protected files in the new repository unless the owner explicitly authorizes that exact instruction-file change in the current task: + +- `.github/copilot-instructions.md` +- Anything under `.github/instructions/` +- `AGENTS.md` +- `CLAUDE.md` +- `GEMINI.md` + +Implied consent is insufficient. A general instruction such as "fix anything that fails CI" or "update docs as needed" MUST NOT be interpreted as authorization to modify protected files. If the agent identifies a warranted protected-file change, it MUST open an issue or add a new explicit decision item and wait. + +## 7. Background Context + +This specification is intended to be self-contained. The future coding agent MAY use these planning files as background context when a requirement needs more narrative detail: + +| Source | What it contributes | +| --- | --- | +| `2026/MMSMOA/macOS-imaging-08-repo-spec.md` | Primary prompt, MVP feature list, validation guardrails, and selected repo name. | +| `2026/MMSMOA/macOS-imaging-03a-bolstered-outline.md` | Session contract, audience model, demo flow, fidelity boundaries, and stage runbook. | +| `2026/MMSMOA/macos-imaging-05c-recommended-software-step-by-step-revised.md` | Host setup sequence, software acquisition, Graph module guidance, and starter-kit verification expectations. | +| `2026/MMSMOA/macos-imaging-06a-repo-description.md` | Repository prose description and implementation posture. | +| `2026/MMSMOA/macOS-imaging-01a-CFP-submission.md` | Accepted abstract and attendee takeaways. | +| `2026/MMSMOA/macOS-imaging-08d-closed-questions-archive.md` | Historical archive of owner decisions and implementation uncertainties that have been closed. | + +If background files disagree with this specification, this specification controls repository implementation details unless the owner approves a later replacement. + +## 8. Product Boundaries + +### 8.1 In Scope + +- Apple-silicon host Macs. +- macOS guest VMs. +- PowerShell 7.4 or newer as the automation layer. +- Parallels Desktop Pro Edition as the primary provider. +- UTM as a secondary provider and provider-swap example. +- Tart as an optional advanced CLI/CI path, stubbed in v1 unless explicitly approved. +- `mist-cli`-based media discovery and acquisition. +- Intune enrollment and validation workflows for lab-only devices and lab-only users. +- FileVault, Defender for Endpoint, PPPC/TCC, compliance, and Conditional Access timing guidance. +- Redacted evidence export. +- Pester tests and GitHub Actions validation. +- Human-readable documentation for Windows-first Microsoft endpoint administrators. + +### 8.1.1 macOS Guest Version Support + +The repository MUST use an Apple-supported-macOS policy rather than hardcoding support to one demo build. + +For initial implementation and rehearsal, the support matrix is: + +| Purpose | macOS major version | Initial version target | Notes | +| --- | --- | --- | --- | +| Live MMSMOA demo path | macOS Tahoe 26.x | `26.4.1` | Primary demo VM version. | +| Compatibility target | macOS Sequoia 15.x | `15.7.5` | Supported compatibility target for users who are not ready for 26.x. | +| Compatibility target | macOS Sonoma 14.x | `14.8.5` | Supported compatibility target for users still on 14.x. | + +Future updates SHOULD track the macOS versions Apple currently supports with security updates. The implementation MUST record exact host macOS, guest macOS, and build numbers in the Provider Version Matrix rather than relying on marketing version names alone. + +### 8.2 Out of Scope + +- Intel Mac host support. +- Windows-host support. +- Production Mac fleet management. +- A replacement for Intune, ConfigMgr, Jamf, Munki, AutoPkg, or any patch/app deployment platform. +- A complete macOS management curriculum. +- Legal advice about Apple licensing, vendor licensing, or enterprise procurement. +- A guarantee that every Apple hardware or security-model behavior can be tested in a VM. +- Full ADE/ABM zero-touch validation in a VM. +- Platform SSO sign-in/unlock validation. +- Touch ID and Secure Enclave-dependent validation. +- Code signing or notarization of the module. +- Publishing the module to PowerShell Gallery. +- Telemetry, anonymous usage reporting, automatic crash submission, or external logging by default. +- A GUI, Electron app, or web front end. +- Full ConfigMgr inventory bridge implementation. +- Full Log Analytics ingestion pipeline implementation. +- Live Conditional Access blocks as a demo dependency. +- macOS upgrade automation, Time Machine integration, or iCloud sign-in flows. +- Multi-user or multi-host lab orchestration. +- Storing or publishing real tenant data, device identifiers, recovery keys, tokens, secrets, or personal data. + +## 9. Security and Redaction Requirements + +This section is non-negotiable. If another section conflicts with this section, this section wins. + +### 9.1 Hard Prohibitions + +The following MUST NEVER appear in any committed artifact, including source code, docs, examples, screenshots, recordings, JSON fixtures, log captures, evidence bundles, tests, README content, or issue templates: + +- A real or example FileVault personal recovery key value. +- A real or example Apple ID, Apple Push Certificates Portal credential, or APNs certificate value. +- A real or example Microsoft Entra tenant ID, tenant domain, application/client ID, client secret, certificate thumbprint, or refresh token. +- A real or example Microsoft Graph access token, Defender API token, or Intune API token. +- A real or example device identifier, including Intune device ID, Entra device ID, Defender machine ID, or hardware serial number. +- A real user UPN, email address, full name, employee number, or other identifier that points at a specific human. +- A production tenant name or production policy name. +- Any private key in any encoding. +- The contents of `.env`, `.envrc`, or similar credential-bearing files. + +### 9.2 Allowed Placeholder Values + +When code, docs, fixtures, or examples need placeholders, use these patterns: + +| Concept | Allowed placeholder pattern | Example | +| --- | --- | --- | +| Tenant ID | `00000000-0000-0000-0000-000000000000` | `tenantId: 00000000-0000-0000-0000-000000000000` | +| Tenant domain | `.onmicrosoft.example` | `tenant: contoso-lab.onmicrosoft.example` | +| User UPN | `lab.user@.onmicrosoft.example` | `lab.user@contoso-lab.onmicrosoft.example` | +| Device name | `MMS-MACLAB-001` | `MMS-MACLAB-001` | +| Intune device ID | `00000000-0000-0000-0000-000000000001` | `00000000-0000-0000-0000-000000000001` | +| App/client ID | `00000000-0000-0000-0000-00000000ABCD` | `00000000-0000-0000-0000-00000000ABCD` | +| FileVault recovery key | `***REDACTED***` | `recoveryKey: ***REDACTED***` | +| Token/secret | `***REDACTED***` | `accessToken: ***REDACTED***` | +| Hardware serial | `LAB000000001` | `LAB000000001` | +| MAC address | `02:00:00:00:00:01` | `02:00:00:00:00:01` | +| macOS version | `` until pinned for rehearsal | `` | +| macOS build | `` until pinned for rehearsal | `` | + +The repo MUST NOT ship any real value masked behind a redacted placeholder. If a real value was ever placed in a file, the file is not safe to commit because the value may exist in local history or tool caches. + +### 9.3 Redaction Helper Contract + +The redaction helper is `Protect-MacLabEvidence` and lives at `src/Modules/MacLab/Private/Protect-MacLabEvidence.ps1`. + +Requirements: + +- All evidence written by the module to disk MUST flow through `Protect-MacLabEvidence`. +- The helper accepts an in-memory evidence object plus optional additional sensitive field names. +- The helper MUST return a redacted clone and MUST NOT mutate the input object in place. +- The helper MUST walk nested objects recursively. +- The helper MUST replace sensitive values with the literal string `***REDACTED***`. +- The helper MUST stamp the clone with `redactionApplied: true` and `redactionVersion: `. + +The sensitive field-name match list MUST include, at minimum: + +- `recoveryKey` +- `personalRecoveryKey` +- `bitLockerRecoveryKey` +- `accessToken` +- `refreshToken` +- `idToken` +- `clientSecret` +- `apiKey` +- `password` +- `secret` +- `bearer` +- `authorization` +- `serial` +- `serialNumber` +- `tenantId` +- `deviceId` +- `objectId` +- `mail` +- `userPrincipalName` +- `accountName` +- `phoneNumber` +- `imei` +- `meid` +- `wifiPassword` + +The helper MUST also redact: + +- Recovery-key-shaped strings, even when the field name looks harmless. +- JWT-shaped strings beginning with `eyJ` and containing base64url segments separated by periods. +- Caller-specified additional sensitive fields. + +For strings longer than 200 characters that appear high entropy, the helper SHOULD emit a `Write-Warning` advising review. That warning is a backstop, not the primary redaction mechanism. + +### 9.4 Demo Safety Rules + +Examples and docs MUST: + +1. Never instruct the user to display a raw recovery key on a projector. +2. Show FileVault escrow as existence and retrieval-path proof, with the value redacted before persistence. +3. Avoid committed screenshots in v1. +4. Use realistic but synthetic output for `mdatp health`, `fdesetup status`, and `log show`. +5. Avoid transcripts unless a transcript is generated locally and reviewed for secrets before sharing. + +### 9.5 Secret Scanning and Ignore Rules + +- The agent MUST NOT disable GitHub secret scanning or push protection. +- The repository MUST NOT commit `.env`, `.envrc`, `*.pem`, `*.p12`, `*.pfx`, `*.key`, `id_rsa*`, `*.token`, `.app`, `.ipsw`, `.iso`, `.dmg`, binary `.utm` bundles, screenshots, or recordings. +- The template `.gitignore` SHOULD be preserved and extended only when needed for this project. + +### 9.6 Root Phase TODO Files + +The future repository MUST use root-level per-phase TODO files for deferred work, current-tool verification, and owner follow-up items that are easy to forget. This is intentional: the owner wants these TODOs in the repository root where they are visible before implementation or rehearsal. + +Rules: + +- File naming pattern: `TODO-Phase--.md`, for example `TODO-Phase-04-Media-Acquisition.md`. +- Create a TODO file only when that phase has unresolved or deferred work. Omit the file when there are no phase TODOs. +- Each TODO file MUST include enough context for a future reader who does not remember the planning conversation. +- Each TODO item MUST include owner, why it matters, phase gate affected, exact action, acceptance condition, source/link if applicable, and status. +- README MUST link to any root TODO files that exist. +- Phase completion summaries MUST state whether the matching root TODO file is empty, closed, or still has deferred items. + +Known root TODO files required by this specification: + +| File | Required initial content | +| --- | --- | +| `TODO-Phase-04-Media-Acquisition.md` | Verify current `mist-cli` list/download syntax and record exact commands before implementing `Get-MacLabMedia`. | +| `TODO-Phase-05-Parallels-Provider.md` | Verify current `prlctl` syntax, version output, and edition detection on the owner/demo host. | +| `TODO-Phase-06-UTM-Provider.md` | Verify current UTM/`utmctl` automation surface and document any manual-step-required gaps. | +| `TODO-Phase-08-Validation-Loop.md` | Verify current Defender `mdatp health` output shape and sanitize fixtures before evidence tests. | +| `TODO-Phase-10-Deferred-Work.md` | Explain every deferred Phase 10 item, why it was deferred, what would be needed to resume it, and how to validate it safely. Include any future screenshot/recording follow-up if the owner later authorizes checked-in visual artifacts. | + +## 10. VM Fidelity Traffic Light + +This is the honesty boundary for the talk and the repository. + +### 10.0 Apple Virtualization and Licensing Boundary + +The repository documentation MUST include current public Apple license links and MUST avoid giving legal advice. + +For initial documentation, use the macOS Tahoe software license agreement as the primary Apple source: [macOS Tahoe SLA](https://www.apple.com/legal/sla/docs/macOSTahoe.pdf). The docs may summarize the practical lab boundary this way: + +- The lab is designed for Apple-branded computers running macOS. +- The Apple license language allows limited additional macOS virtual instances on Apple-branded computers for specific uses such as software development, testing during software development, macOS Server, or personal non-commercial use. +- The repo's default posture is "two concurrent macOS guests per Apple-branded host unless the owner verifies a different license posture." +- The repo MUST NOT claim to provide legal advice or replace the user's vendor/procurement review. + +This boundary belongs in `docs/Apple-Silicon-Constraints.md`, `docs/Hypervisor-Decision-Guide.md`, and `docs/CI-and-Tart.md`. + +| Color | Meaning | Examples | +| --- | --- | --- | +| Green | The VM lab is sufficient evidence on its own. | Script syntax, package install behavior, Intune assignment logic, profile receipt, basic PPPC payload behavior, Defender health checks, rollback routines, evidence export. | +| Yellow | VM is good for iteration, but physical hardware sign-off is still required. | FileVault rollout behavior, recovery-key process end to end, compliance experience under realistic timing, user prompts, UI flows, performance-sensitive Defender behavior. | +| Red | The VM cannot validate this; physical hardware or production-like enrollment is required. | ADE/ABM zero-touch enrollment, serial-number-dependent workflows, Platform SSO sign-in/unlock, Touch ID, Secure Enclave-dependent behavior, final executive pilot sign-off. | + +### 10.1 Cmdlet Honesty Rules + +- Evidence-producing cmdlets MUST stamp evidence with the fidelity color of the test. +- Yellow evidence MUST include `hardwareSignoffRequired: true` when physical hardware remains required. +- A test plan that asserts a Red-bucket outcome MUST be rejected at parse time by `Invoke-MacPolicyValidation`. +- Documentation MUST NOT claim the lab validates or proves Red-bucket outcomes. + +### 10.2 Cloud-State Rollback Warning + +This sentence, or a stable equivalent, MUST appear in every restore warning and evidence record: + +> VM rollback does not rewind Intune, Entra, Defender portal state, audit logs, or reporting history. + +Every cmdlet or script that restores a VM checkpoint MUST emit a runtime warning containing that statement. The docs MUST repeat it in the snapshot, Intune, and demo runbook sections. + +## 11. Requirements + +Each requirement is normative and testable. Later sections define the implementation contracts that satisfy these requirements. + +### MLAB-REQ-001: Template Initialization + +The repository MUST be initialized from [`franklesniak/copilot-repo-template`](https://github.com/franklesniak/copilot-repo-template). + +**Rationale:** The template supplies coding-agent instructions, CI conventions, issue templates, pre-commit configuration, and baseline repo hygiene. + +**Acceptance Criteria:** + +- Template governance files remain intact unless this spec explicitly authorizes a narrow change. +- Protected instruction files are not weakened. +- Project-specific docs and source files are added without deleting template governance. + +**Verification:** Review the initial commit diff and run the template's configured checks. + +### MLAB-REQ-002: Repository and Module Names + +The GitHub repository MUST be named `macOSLab`. The PowerShell module MUST be named `MacLab`. + +**Acceptance Criteria:** + +- Repo references use `franklesniak/macOSLab`. +- The module manifest is `src/Modules/MacLab/MacLab.psd1`. +- The module root file is `src/Modules/MacLab/MacLab.psm1`. +- Exported commands use the intended `MacLab` nouns. + +**Verification:** Import the module locally and confirm exported commands with `Get-Command -Module MacLab`. + +### MLAB-REQ-003: PowerShell 7.4 Baseline + +The repository MUST target PowerShell 7.4 or newer on macOS as the primary runtime. + +**Acceptance Criteria:** + +- Scripts declare or document the supported PowerShell version. +- macOS-only scripts check that they are running on macOS. +- PowerShell follows the inherited style guide. +- No Windows PowerShell-only assumptions are introduced. + +**Verification:** Run PSScriptAnalyzer and Pester in PowerShell 7.4 or newer. + +### MLAB-REQ-004: Provider Abstraction + +The `MacLab` module MUST expose one user-facing automation model over multiple VM providers. + +**Acceptance Criteria:** + +- Provider values are `Parallels`, `UTM`, and `Tart`. +- Parallels is implemented as the primary working path. +- UTM has either a working automation path or a clearly documented template/manual path. +- Tart is optional and explicitly marked advanced or stubbed. +- Unsupported provider capabilities return a clear warning or terminating error. + +**Verification:** Provider tests cover capability discovery, supported calls, and unsupported calls. + +### MLAB-REQ-005: Reproducible Media Acquisition + +The repository MUST treat macOS media pinning as a first-class workflow. + +**Acceptance Criteria:** + +- Media acquisition uses `mist-cli` as the default source. +- The public surface uses `Get-MacLabMedia -Source Mist`. +- `-Provider` is reserved for hypervisor selection. +- Media metadata records version, build, artifact path, source, acquisition time, and checksum where practical. +- Docs use "restore image or provider-appropriate install artifact" instead of treating ISO as the Apple-silicon default. + +**Verification:** Unit tests validate metadata shape and parameter behavior. A manual live validation confirms a pinned artifact can be discovered and cached. + +### MLAB-REQ-006: Five-Checkpoint Snapshot Model + +The repository MUST encode the five named checkpoints used by the talk. + +**Acceptance Criteria:** + +- Supported checkpoint names are `Clean-OS`, `Pre-Enroll`, `Post-Enroll-Baseline`, `Broken-Policy-State`, and `Recovered-Known-Good`. +- Docs explain when to use each checkpoint. +- Restore workflows warn when cloud state may no longer match the restored VM. +- A readiness workflow verifies required checkpoints before rehearsal. + +**Verification:** Pester tests validate accepted checkpoint names and warning behavior. + +### MLAB-REQ-007: Intune Lab Scope + +The repository MUST assume lab-only Intune devices, users, and groups. + +**Acceptance Criteria:** + +- Documentation tells users to create lab-only users, groups, and assignments. +- Example config values use lab-only names. +- Any Graph helper that finds devices requires explicit scope filters or confirmation before acting on cloud objects. +- Cleanup workflows are documented before destructive cloud-side actions are implemented. + +**Verification:** Tests verify filters or confirmations are required for cleanup routines. Docs review confirms no production assignment examples are present. + +### MLAB-REQ-008: FileVault Validation + +The repository MUST include a FileVault validation model that proves policy receipt and escrow evidence without exposing recovery keys. + +**Acceptance Criteria:** + +- Docs define the minimum FileVault proof path. +- Validation captures local `fdesetup status`. +- Validation captures escrow existence or retrieval-path evidence. +- Raw recovery key values are redacted before persistence. +- Docs clearly separate escrow preparation from encryption state. +- Docs say which FileVault results require physical Mac sign-off. + +**Verification:** Pester tests prove the redaction helper masks recovery-key-shaped values and evidence export marks redaction as applied. + +### MLAB-REQ-009: Defender Validation + +The repository MUST validate Defender for Endpoint on macOS as more than an app install. + +**Acceptance Criteria:** + +- Docs and examples include package install, system extension approval, network extension approval when used, Full Disk Access/PPPC delivery, onboarding, and `mdatp health`. +- Evidence records Defender version and health output after redaction. +- Provider rollback result is included in the evidence model. + +**Verification:** Tests validate expected evidence fields. Manual demo validation captures `mdatp health` in a redacted evidence bundle. + +### MLAB-REQ-010: PPPC/TCC Validation + +The repository MUST explain PPPC/TCC validation as a precision problem. + +**Acceptance Criteria:** + +- Docs explain bundle ID, code requirement, app path, profile receipt, app behavior, and log evidence. +- Examples do not rely only on System Settings screenshots. +- Evidence can include profile receipt and targeted behavior checks. + +**Verification:** Docs review and example test-case review. + +### MLAB-REQ-011: Evidence as a First-Class Output + +Every meaningful validation run MUST produce structured evidence. + +**Acceptance Criteria:** + +- Evidence is JSON by default. +- Evidence includes run ID, provider, provider version, host macOS version, guest macOS version/build, snapshot, Intune device name or redacted ID, test results, cloud-state warning, fidelity, hardware sign-off flag, and redaction status. +- Evidence export creates a predictable folder structure. +- Evidence export fails closed when redaction cannot be applied to sensitive fields. + +**Verification:** Pester tests validate evidence schema and redaction-required behavior. + +### MLAB-REQ-012: Readiness Gate + +The repository MUST include `scripts/Test-LabReadiness.ps1` as the canonical T-15 readiness gate. + +**Acceptance Criteria:** + +- The script checks host tools, provider tools, PowerShell modules, configured paths, detectable permissions, media cache, required checkpoints, and optional network/cloud reachability. +- Required checks fail the run. +- Optional checks are reported as warnings or skipped checks with reasons. +- Output is terminal-readable and exportable as evidence. + +**Verification:** Pester tests cover pass, fail, warn, and skipped cases. + +### MLAB-REQ-013: Documentation-First Handoff + +The repository MUST be useful to an attendee who starts at `docs/Start-Here.md`. + +**Acceptance Criteria:** + +- `docs/Start-Here.md` gives the first command, first validation target, and first expected evidence artifact. +- README gives the same starting path at a higher level. +- Docs are written for Windows-first Microsoft endpoint administrators. +- Docs include examples, expected outputs, and failure/ambiguous-input behavior where appropriate. + +**Verification:** Manual docs review from a fresh-reader perspective. + +### MLAB-REQ-014: Tests and CI + +The repository MUST include Pester tests and GitHub Actions validation. + +**Acceptance Criteria:** + +- Tests use Pester 5.x syntax; CI initially pins Pester `5.7.1`. +- Tests are located under `tests/`. +- GitHub Actions runs Pester and PowerShell linting. +- Markdown linting runs through the template's existing tooling. +- Default tests avoid requiring real Parallels, UTM, Graph, Defender, or a real macOS VM. + +**Verification:** CI passes on a clean clone without secrets. + +### MLAB-REQ-015: No Stage Reliance on Live Cloud Timing + +The repository MUST support live, checkpoint, screenshot, and screen-recording fallback paths for cloud-dependent demos. + +**Acceptance Criteria:** + +- Demo docs include live path, checkpoint path, screenshot path, screen-recording path, and pivot sentence. +- Demo scripts can run against prepared state. +- Evidence examples can be generated from fixture data for documentation and rehearsal. + +**Verification:** Demo runbook review and fixture-based evidence tests. + +## 12. Initialization From the Template + +The repository MUST be created from [`franklesniak/copilot-repo-template`](https://github.com/franklesniak/copilot-repo-template). The agent SHOULD use GitHub's template flow or an equivalent `gh repo create --template` flow, then clone the new templated repository locally. + +Before Phase 0 begins, the owner will create or provide a template initialization implementation-steps guide. The future coding agent MUST use that guide as the concrete Phase 0 checklist, while still following this specification and the new repository's protected instruction files. + +### 12.1 Keep From the Template + +The agent MUST keep template governance files unless this spec explicitly says otherwise. At minimum, preserve: + +- `.github/copilot-instructions.md` +- `.github/instructions/` +- `.github/linting/PSScriptAnalyzerSettings.psd1` +- `.github/workflows/auto-fix-precommit.yml` +- `.github/dependabot.yml` +- `.github/ISSUE_TEMPLATE/` +- `.github/pull_request_template.md` +- `.github/scripts/lint-nested-markdown.js` +- `.gitattributes` +- `.gitignore` +- `.markdownlint.jsonc` +- `.pre-commit-config.yaml` +- `AGENTS.md` +- `CLAUDE.md` +- `GEMINI.md` +- `CONTRIBUTING.md` +- `SECURITY.md` + +If the template has additional governance files not listed here, preserve them unless the owner authorizes removal. + +`SECURITY.md` remains unchanged by default. The implementation agent MUST NOT rewrite it or replace the template's responsible-disclosure process. During Phase 1, after the inherited template file is visible, the agent may propose only this exact project-specific paragraph if it is compatible with the inherited file: + +> This repository ships no real tenant identifiers, no recovery keys, no tokens, and no production credentials. If you discover content that appears to be a real secret, recovery key, tenant identifier, or personal data, please report it privately through the repository's security advisory process rather than opening a public issue. + +No other `SECURITY.md` customization is approved by this specification. + +### 12.2 Phase 0 Customizations + +During Phase 0, the agent MUST make only identity/bootstrap changes: + +| Path | Required customization | +| --- | --- | +| `LICENSE` | Set the copyright line to `Copyright (c) 2026 Frank Lesniak`; leave MIT license body untouched. | +| `README.md` | Replace the template README with a minimal Phase 0 `macOSLab` README. Full content lands in Phase 1. | +| `.github/CODEOWNERS` | Confirm it contains `* @franklesniak`, unless the owner names additional launch code owners. | +| `package.json` | Set repository identity fields for `macOSLab`; keep markdownlint scripts from the template; keep `"private": true`. | +| `package-lock.json` | Regenerate after `package.json` edits. | + +The agent MAY remove template sample artifacts for languages this repo does not use, such as Python sample source/tests, only if removal does not break validation. If unsure, record a new explicit decision item before removing them. + +### 12.3 Phase 0 Acceptance + +Phase 0 is complete only when: + +1. The customizations in Section 12.2 are committed. +2. `pre-commit run --all-files` passes locally. +3. `npm run lint:md` passes. +4. `npm run lint:md:nested` passes if that script exists. +5. GitHub Actions is green for the most recent commit. +6. The repo is publicly visible at [`franklesniak/macOSLab`](https://github.com/franklesniak/macOSLab). +7. No protected instruction file was modified. +8. The agent has posted a Phase 0 summary and stopped for owner approval. + +## 13. Coding Standards + +The repository inherits coding standards from the template. The agent MUST NOT fork those standards into new custom style guides. + +### 13.1 PowerShell + +- All PowerShell is Modern style: PowerShell 7.4 or newer, `[CmdletBinding()]`, `[OutputType()]`, structured error handling, streaming output, and comment-based help. +- Public identifiers use PascalCase. +- Local variables use the inherited style guide's typed-prefix camelCase convention. +- Approved verbs only. PSScriptAnalyzer `PSUseApprovedVerbs` must report zero findings. +- No aliases in code. +- Use 4-space indentation, OTBS braces, UTF-8 without BOM, no trailing whitespace, and no vertical alignment. +- Use `try`/`catch`; empty `catch` blocks are forbidden. +- Use `$PSCmdlet.ThrowTerminatingError()` when wrapping errors. +- Never emit raw PII, credentials, tokens, or recovery keys to any output stream. +- Shared module state, where unavoidable, uses `$script:` scope. Do not mutate `$global:`. +- Path inputs MUST be resolved and constrained before use. + +### 13.2 Markdown + +- All non-trivial docs include the metadata block required by the inherited docs style. +- Markdown must pass `npm run lint:md` and `npm run lint:md:nested` where available. +- Docs use plain, technical language for Windows-first Microsoft endpoint administrators. +- Docs are honest about VM fidelity boundaries. +- Docs use placeholders from Section 9.2. + +### 13.3 Pester + +- Tests use Pester 5.x syntax; CI initially pins Pester `5.7.1`. +- Test files end in `.Tests.ps1`. +- Use `BeforeAll`, `Describe`, `Context`, and `It`. +- Follow Arrange-Act-Assert. +- Mock external commands and cloud calls by default. + +### 13.4 CI and Pre-Commit + +- Run `pre-commit run --all-files` before every commit. +- Commit auto-fixes with the related change. +- Do not skip hooks without explicit owner authorization. +- CI is a safety net, not a substitute for local checks. + +## 14. Repository File Map + +The project-owned v1 file tree SHOULD match this map. Template-owned files may exist in addition to this map when they are inherited from the template and are not harmful. New project-owned files outside this map require owner approval or a new explicit decision item. + +```text +macOSLab/ +├── .devcontainer/ # inherited if present +├── .github/ +│ ├── CODEOWNERS # customized in Phase 0 +│ ├── ISSUE_TEMPLATE/ # inherited +│ ├── copilot-instructions.md # inherited, protected +│ ├── dependabot.yml # inherited +│ ├── instructions/ # inherited, protected +│ │ ├── docs.instructions.md +│ │ └── powershell.instructions.md +│ ├── linting/ +│ │ └── PSScriptAnalyzerSettings.psd1 +│ ├── pull_request_template.md +│ ├── scripts/ +│ │ └── lint-nested-markdown.js +│ └── workflows/ +│ ├── auto-fix-precommit.yml +│ └── pester.yml +├── .gitattributes +├── .gitignore +├── .markdownlint.jsonc +├── .pre-commit-config.yaml +├── .vscode/ # inherited if present +├── AGENTS.md # inherited, protected +├── CLAUDE.md # inherited, protected +├── CONTRIBUTING.md +├── GEMINI.md # inherited, protected +├── LICENSE +├── README.md +├── SECURITY.md +├── TODO-Phase-04-Media-Acquisition.md # if Phase 4 TODOs remain +├── TODO-Phase-05-Parallels-Provider.md # if Phase 5 TODOs remain +├── TODO-Phase-06-UTM-Provider.md # if Phase 6 TODOs remain +├── TODO-Phase-08-Validation-Loop.md # if Phase 8 TODOs remain +├── TODO-Phase-10-Deferred-Work.md # if Phase 10 work is deferred +├── docs/ +│ ├── Apple-Silicon-Constraints.md +│ ├── CI-and-Tart.md +│ ├── ConfigMgr-Inventory-Bridge.md +│ ├── Defender-Validation.md +│ ├── Demo-Runbook.md +│ ├── Evidence-and-CAB.md +│ ├── Evidence-Redaction.md +│ ├── Fidelity-Boundaries.md +│ ├── FileVault-Validation.md +│ ├── Hypervisor-Decision-Guide.md +│ ├── Intune-Tenant-Setup.md +│ ├── Log-Analytics-Integration.md +│ ├── PPPC-Validation.md +│ ├── Prereqs.md +│ ├── Provider-Version-Matrix.md +│ ├── Snapshot-Strategy.md +│ ├── Start-Here.md +│ ├── Troubleshooting.md +│ └── Windows-Admin-Cheat-Sheet.md +├── examples/ +│ ├── MMSMOA-2026/ +│ │ ├── demo-config.yml +│ │ ├── Demo1-Media.ps1 +│ │ ├── Demo2-Parallels.ps1 +│ │ ├── Demo3-UTM.ps1 +│ │ └── Demo4-IntuneValidation.ps1 +│ ├── TestCases/ +│ │ ├── Compliance-SmokeTest.yml +│ │ ├── Defender-Validation.yml +│ │ ├── FileVault-Validation.yml +│ │ └── PPPC-Validation.yml +│ └── utm/ +│ └── macos-lab-template.utm.json +├── package-lock.json +├── package.json +├── scripts/ +│ ├── Checkpoint-MacVm.ps1 +│ ├── Get-MacOSRestoreImage.ps1 +│ ├── Install-Prereqs.ps1 +│ ├── Invoke-MMSDemo.ps1 +│ ├── New-MacInstallArtifact.ps1 +│ ├── New-MacVm.ps1 +│ ├── Remove-MacVm.ps1 +│ ├── Reset-IntuneMacLabDevice.ps1 +│ ├── Restore-MacVmCheckpoint.ps1 +│ ├── Send-LabEventToLogAnalytics.ps1 +│ └── Test-LabReadiness.ps1 +├── src/ +│ └── Modules/ +│ └── MacLab/ +│ ├── MacLab.psd1 +│ ├── MacLab.psm1 +│ ├── Private/ +│ │ ├── Invoke-LoggedCommand.ps1 +│ │ ├── Protect-MacLabEvidence.ps1 +│ │ ├── Resolve-MacLabConfig.ps1 +│ │ └── Write-EvidenceRecord.ps1 +│ ├── Providers/ +│ │ ├── Parallels.ps1 +│ │ ├── Tart.ps1 +│ │ └── UTM.ps1 +│ └── Public/ +│ ├── Checkpoint-MacLabVm.ps1 +│ ├── Export-MacLabEvidence.ps1 +│ ├── Get-MacLabMedia.ps1 +│ ├── Get-MacLabVm.ps1 +│ ├── Invoke-MacPolicyValidation.ps1 +│ ├── New-MacLabVm.ps1 +│ ├── Remove-MacLabVm.ps1 +│ ├── Restore-MacLabVmCheckpoint.ps1 +│ ├── Start-MacLabVm.ps1 +│ └── Stop-MacLabVm.ps1 +└── tests/ + ├── MacLab.Tests.ps1 + ├── Providers.Parallels.Tests.ps1 + ├── Providers.UTM.Tests.ps1 + ├── Validation.Defender.Tests.ps1 + └── Validation.FileVault.Tests.ps1 +``` + +### 14.1 UTM Template Format + +The repository ships `examples/utm/macos-lab-template.utm.json`, not a real `.utm` bundle. A `.utm` bundle is a directory/package that can include binary disk content and MUST NOT be committed. The JSON descriptor records the intended CPU, memory, disk, network, virtualization mode, display, and sharing settings. Docs explain how to reconstruct the actual UTM VM locally. + +### 14.2 What Does Not Ship + +The repo MUST NOT ship: + +- `.app` bundles +- `.ipsw` files +- `.iso` files +- `.dmg` files +- Binary `.utm` bundles +- Screenshots +- Recordings +- Production policy exports +- Graph payloads with real tenant data +- Credential-bearing files +- Vendor SDK redistributables + +## 15. Naming Conventions + +### 15.1 Module and File Names + +- Module name: `MacLab` +- Repo name: `macOSLab` +- Manifest: `src/Modules/MacLab/MacLab.psd1` +- Loader: `src/Modules/MacLab/MacLab.psm1` +- Public cmdlet files: one cmdlet per file under `Public/`. +- Private helper files: one helper per file under `Private/`. +- Provider files: `Parallels.ps1`, `UTM.ps1`, and `Tart.ps1`. + +### 15.2 Parameter Names + +- `-Provider` is only for hypervisors: `Parallels`, `UTM`, and `Tart`. +- `-Source` is only for media sources. V1 valid value: `Mist`. +- `-OutputPath` is preferred over `-Path` for write destinations. +- Evidence-producing cmdlets include `-RedactSecrets`, defaulting to `$true`. In PowerShell this may be implemented as a switch with default `$true`, so callers can explicitly pass `-RedactSecrets:$false` only for non-public lab paths. Even then, raw recovery keys, tokens, and tenant secrets MUST NOT be written to disk. + +### 15.3 Checkpoint Vocabulary + +| Canonical name | Meaning | +| --- | --- | +| `Clean-OS` | Freshly installed guest, Setup Assistant complete, no demo software, never enrolled. | +| `Pre-Enroll` | `Clean-OS` plus Intune Company Portal at the sign-in screen; not enrolled. | +| `Post-Enroll-Baseline` | Fully enrolled, recently synced, deterministic healthy baseline. | +| `Broken-Policy-State` | Deterministic intentionally broken state for the engineered demo failure. | +| `Recovered-Known-Good` | Post-rollback healthy state, captured after cloud cleanup/reconciliation. | + +Alternative names are not allowed in v1. + +## 16. Module Specification + +The agent MUST generate one `.ps1` file per public cmdlet under `src/Modules/MacLab/Public/`. Each file contains exactly one function whose name matches the file name. Each public function includes full comment-based help and follows the inherited PowerShell style guide. + +### 16.1 Module Manifest + +The manifest MUST include at minimum: + +```powershell +@{ + RootModule = 'MacLab.psm1' + ModuleVersion = '0.1.0' + GUID = '' + Author = 'Frank Lesniak' + CompanyName = 'Frank Lesniak' + Copyright = '(c) 2026 Frank Lesniak. MIT License.' + Description = 'Reproducible Apple-silicon macOS VM lab for risk-free Intune policy testing. Pin macOS media, build snapshots, enroll, validate FileVault/Defender/PPPC, fail safely, roll back, and export redacted evidence.' + PowerShellVersion = '7.4' + CompatiblePSEditions = @('Core') + FunctionsToExport = @( + 'Get-MacLabMedia', + 'New-MacLabVm', + 'Get-MacLabVm', + 'Start-MacLabVm', + 'Stop-MacLabVm', + 'Checkpoint-MacLabVm', + 'Restore-MacLabVmCheckpoint', + 'Remove-MacLabVm', + 'Invoke-MacPolicyValidation', + 'Export-MacLabEvidence' + ) + CmdletsToExport = @() + VariablesToExport = @() + AliasesToExport = @() +} +``` + +Rules: + +- The GUID must be generated once and must not remain a placeholder. +- `FunctionsToExport` lists exactly the ten public cmdlets. +- Private helpers and provider functions are never exported. +- `PowerShellVersion` is `7.4`. The repo supports PowerShell 7.4.x and newer PowerShell 7.x releases unless a concrete incompatibility is documented. + +### 16.2 Module Loader + +`MacLab.psm1` MUST: + +1. Set `Set-StrictMode -Version Latest` as the first executable statement. +2. Resolve the module root directory once and store it in `$script:` scope. +3. Dot-source `Private/`, then `Providers/`, then `Public/`. +4. Export only functions listed in the manifest. +5. Avoid `$global:` mutations. +6. Avoid network calls at module-load time. +7. Avoid credential prompts at module-load time. +8. Fail loudly if any provider or function file cannot be loaded. + +### 16.3 Public Cmdlet Contracts + +All public cmdlets share these rules: + +- Use `[CmdletBinding()]`. +- Use typed parameters. +- Declare `[OutputType()]`. +- Stream objects to the pipeline. +- Use `try`/`catch`. +- Avoid secrets in verbose/debug/warning/error output. +- Support `ShouldProcess` when writing to disk, mutating a VM, contacting cloud services, or performing destructive actions. + +#### 16.3.1 `Get-MacLabMedia` + +**Purpose:** Discover, acquire, cache, and record pinned macOS install artifacts through `mist-cli`. + +**Parameters:** + +| Name | Type | Required | Default | Notes | +| --- | --- | --- | --- | --- | +| `Version` | `string` | Yes | None | macOS marketing version. | +| `Build` | `string` | No | `$null` | Build number. If supplied, build is the stronger match key. | +| `Architecture` | `arm64` validate set | No | `arm64` | V1 supports Apple silicon only. | +| `Source` | `Mist` validate set | No | `Mist` | Media source, not hypervisor. | +| `ArtifactType` | `Installer`, `Firmware`, `Both` | No | `Both` | `.app`, `.ipsw`, or both. | +| `CacheRoot` | `string` | No | `~/Demo/Installers` | Resolved before use. | +| `Force` | `switch` | No | `$false` | Re-download even if cached. | +| `RedactSecrets` | `switch` | No | `$true` | Consistent evidence/logging posture. | + +**Output:** Object with `Version`, `Build`, `Architecture`, `Source`, `ArtifactType`, artifact paths, per-artifact SHA-256 values, sizes, `AcquiredAt`, and `MetadataJsonPath`. + +**Behavior:** + +1. Verify host is macOS. +2. Verify `mist` is available. +3. Resolve and create `CacheRoot`. +4. Use `mist-cli` to list and download the requested artifacts. +5. Verify current `mist-cli` syntax during implementation instead of relying blindly on stale examples. +6. Compute SHA-256 for acquired files. +7. Write a sidecar `metadata.json` under a deterministic cache folder. +8. Stream the metadata object. + +#### 16.3.2 `New-MacLabVm` + +**Purpose:** Create or register a VM under a selected provider using cached media. + +**Parameters:** `Provider`, `Name`, `MediaId`, optional `CacheRoot`, `SizingProfile`, `VmRoot`, `RedactSecrets`. + +**Output:** Object with provider, name, media ID, sizing profile, state, config path, creation time, and provider version. + +**Behavior:** + +1. Validate VM name with a cross-provider-safe regex. +2. Load media metadata from cache. +3. Dispatch to the chosen provider. +4. Create/register but do not start the VM. +5. Stream the provider result. + +**ShouldProcess:** Yes, `ConfirmImpact = 'Medium'`. + +#### 16.3.3 `Get-MacLabVm` + +**Purpose:** List or describe lab VMs. + +**Parameters:** Optional `Provider`, optional `Name`. + +**Output:** One object per VM with provider, name, state, media ID, sizing profile, CPU, memory, disk, MAC address, last seen time, snapshots, capabilities, and provider version where available. + +**Behavior:** If no provider is supplied, query installed providers and skip missing providers without error. + +#### 16.3.4 `Start-MacLabVm` + +**Purpose:** Start a lab VM. + +**Parameters:** `Provider`, `Name`, optional `WaitForReady`. + +**Output:** VM state object. + +**Behavior:** Dispatch to provider. Warn if network/cloud readiness has not been run recently when the next workflow depends on APNs, Intune, Graph, or Defender. + +**ShouldProcess:** Yes, `ConfirmImpact = 'Low'`. + +#### 16.3.5 `Stop-MacLabVm` + +**Purpose:** Stop a lab VM. + +**Parameters:** `Provider`, `Name`, optional `Force`. + +**Output:** VM state object. + +**Behavior:** Clean shutdown by default. Force powers off. Warn that snapshotting a running or paused VM increases MDM identity-drift risk. + +**ShouldProcess:** Yes, `ConfirmImpact = 'Medium'`. + +#### 16.3.6 `Checkpoint-MacLabVm` + +**Purpose:** Capture a named checkpoint. + +**Parameters:** `Provider`, `Name`, `CheckpointName`, optional `AllowNonCanonicalCheckpoint`, `Description`, and `RequireCleanShutdown`. + +**Output:** Checkpoint object with provider, VM name, checkpoint name, parent checkpoint if known, created-at timestamp, description, and canonical-name flag. + +**Behavior:** + +1. Require one of the five canonical checkpoint names unless `AllowNonCanonicalCheckpoint` is supplied. +2. Refuse to snapshot a running or paused VM when `RequireCleanShutdown` is true. +3. Auto-include macOS version, build, and provider version in checkpoint metadata where available. +4. Dispatch to provider. + +**ShouldProcess:** Yes, `ConfirmImpact = 'Medium'`. + +#### 16.3.7 `Restore-MacLabVmCheckpoint` + +**Purpose:** Restore a VM to a named checkpoint. + +**Parameters:** `Provider`, `Name`, `CheckpointName`, optional `AcknowledgeCloudStateWarning`. + +**Output:** VM state object plus `RestoredFromCheckpoint` and `RestoredAt`. + +**Behavior:** + +1. Require explicit acknowledgement in non-interactive or unattended paths. +2. Emit the cloud-state warning before restore. +3. Dispatch to provider. +4. Emit the cloud-state warning after restore. +5. Record an evidence entry for the restore event. + +**ShouldProcess:** Yes, `ConfirmImpact = 'High'`. + +#### 16.3.8 `Remove-MacLabVm` + +**Purpose:** Remove a lab VM and optionally remove local disk files. + +**Parameters:** `Provider`, `Name`, optional `RemoveDiskFiles`, optional `Force`. + +**Output:** Removal result object. + +**Behavior:** Dispatch to provider. Warn that local removal does not delete Intune, Entra, or Defender records. + +**ShouldProcess:** Yes, `ConfirmImpact = 'High'`. + +#### 16.3.9 `Invoke-MacPolicyValidation` + +**Purpose:** Run a YAML test plan against an enrolled lab VM and emit structured evidence. + +**Parameters:** `Provider`, `Name`, `TestPlan`, optional `OutputPath`, `RedactSecrets`, `Fidelity`, and `GraphScope`. + +**Output:** Evidence object matching Appendix B. + +**Behavior:** + +1. Load and validate the YAML test plan. +2. Reject Red-bucket assertions. +3. Verify the VM and checkpoint state expected by the plan. +4. Dispatch each step to the relevant proof helper. +5. Capture local and cloud state where the plan requires both. +6. Run the evidence through `Protect-MacLabEvidence`. +7. Persist evidence with `Write-EvidenceRecord`. +8. Stream the redacted evidence object. + +Test assertion failures are recorded as evidence `Fail` results and are not terminating errors. Schema violations, missing VM state, missing required scopes, and redaction failures are terminating errors. + +**ShouldProcess:** Yes, `ConfirmImpact = 'Medium'`. + +#### 16.3.10 `Export-MacLabEvidence` + +**Purpose:** Export an evidence run to a portable directory or zip bundle. + +**Parameters:** Optional `Name`, optional `RunId`, optional `EvidenceRoot`, required `OutputPath`, optional `Format`, optional `RedactSecrets`. + +**Output:** Object with `BundlePath`, `RunId`, `RedactionApplied`, and `BundleSha256`. + +**Behavior:** + +1. Locate the requested run. +2. Re-run redaction on every JSON artifact before bundling. +3. Include `evidence.json`, `evidence.summary.txt`, the YAML test plan, Provider Version Matrix snapshot, and `MANIFEST.json`. +4. Compute SHA-256 over the bundle. + +**ShouldProcess:** Yes, `ConfirmImpact = 'Low'`. + +## 17. Private Helpers and Providers + +### 17.1 Private Helpers + +Private helpers live under `src/Modules/MacLab/Private/`, are not exported, and each file contains exactly one function. + +| Helper | Required purpose | +| --- | --- | +| `Invoke-LoggedCommand` | Run external commands, capture stdout/stderr/exit code/timing, redact sensitive arguments in logs, and enforce timeouts. | +| `Write-EvidenceRecord` | Write redacted evidence to `evidence.json` and `evidence.summary.txt`, embed Provider Version Matrix, verify `redactionApplied`, use deterministic JSON formatting, and return file paths plus SHA-256. | +| `Resolve-MacLabConfig` | Merge embedded defaults, optional config file, and explicit overrides without creating config files as a side effect. | +| `Protect-MacLabEvidence` | Redact secrets from evidence as described in Section 9.3. | + +### 17.2 Provider Primitive Contract + +Each provider file MUST expose provider-specific primitives that the public cmdlets can call through a uniform dispatch layer: + +| Primitive | Output expectation | +| --- | --- | +| `New-MacLabVm_` | Created VM record. | +| `Get-MacLabVm_` | VM records. | +| `Start-MacLabVm_` | VM state record. | +| `Stop-MacLabVm_` | VM state record. | +| `Checkpoint-MacLabVm_` | Checkpoint record. | +| `Restore-MacLabVmCheckpoint_` | VM state plus restored-from data. | +| `Remove-MacLabVm_` | Removal result. | +| `Get-ProviderVersion_` | Version string and diagnostic data. | +| `Test-ProviderInstalled_` | Boolean or structured installed/missing state. | + +### 17.3 Shared Provider Rules + +- Providers declare capabilities such as `CanCreateVm`, `CanStartVm`, `CanStopVm`, `CanCheckpoint`, `CanRestoreCheckpoint`, `CanListSnapshots`, and `SupportsTemplateImport`. +- Providers return structured objects, not plain strings. +- Providers capture executable versions where available. +- Providers fail clearly when tools are missing. +- Providers do not delete VMs, snapshots, or media without confirmation/`ShouldProcess`. +- Providers use shared networking, not bridged networking, by default. +- Providers disable host-to-guest sharing features that blur identity boundaries unless a future owner-approved change explicitly permits them. + +### 17.4 Parallels Provider + +Parallels is the primary provider. + +Required behavior: + +- Detect `prlctl`. +- Capture Parallels version and edition where possible. +- Warn if Standard edition is detected because the demo assumes Pro or Business automation. +- Create/register VMs from the chosen artifact path. +- Start and stop VMs. +- Create, list, and restore snapshots/checkpoints. +- Return stable VM identity metadata. +- Use Shared networking. +- Disable Coherence, shared clipboard, shared folders, shared cameras, shared Bluetooth, and similar host integrations when creating VMs. + +### 17.5 UTM Provider + +UTM is the secondary provider. + +Required behavior: + +- Detect UTM and `utmctl` where available. +- Enforce Apple Virtualization for macOS guests, not QEMU emulation. +- Use Shared Network, not bridged. +- Reference `examples/utm/macos-lab-template.utm.json`. +- Where `utmctl` cannot perform a primitive, throw a clear "manual step required" error and link to the documented procedure. +- Support the provider-swap demo path honestly, even if some steps are manual. + +### 17.6 Tart Provider + +Tart is optional in v1. + +The v1 provider MAY be a stub that: + +- Implements `Test-ProviderInstalled_Tart`. +- Implements `Get-ProviderVersion_Tart`. +- Throws a clear "Tart provider is documented but not implemented in v1" error for all mutating primitives. +- Points to `docs/CI-and-Tart.md`. + +The agent MUST NOT silently expand Tart beyond the stub unless the owner explicitly approves that work. + +`docs/CI-and-Tart.md` MUST explain the initial Tart licensing posture in plain language: + +- Tart and Orchard currently publish Fair Source license files in their public repositories. +- Tart's free-tier documentation describes a 100 CPU-core limit. +- Orchard's free-tier documentation describes a 4-worker limit; the Orchard license file defines the relevant user concept as a device running macOS. +- V1 does not depend on Tart for the talk path. The provider remains stubbed unless the owner later approves full implementation. +- This is implementation guidance, not legal advice. The owner/user remains responsible for verifying license suitability before using Tart or Orchard in their own organization. + +### 17.7 Provider Version Matrix + +Every evidence-producing cmdlet MUST capture a Provider Version Matrix with: + +- Host macOS version and build. +- Guest macOS version and build where available. +- Hypervisor version. +- PowerShell version. +- Pester version, initially pinned to `5.7.1` in CI. +- Defender version when relevant. +- Intune policy-set identifier when supplied by the test plan. + +## 18. Scripts Specification + +Scripts under `scripts/` are stage-friendly wrappers and utilities. They are not the public module surface. + +Every script MUST: + +- Include full comment-based help. +- Use `[CmdletBinding()]`, `param()`, and `Set-StrictMode -Version Latest`. +- Honor `WhatIf`/`Confirm` where side effects exist. +- Import the local `MacLab` module, not a gallery module. +- Pass PSScriptAnalyzer. +- Avoid secret output. + +| Script | Required purpose | +| --- | --- | +| `Install-Prereqs.ps1` | Idempotently check/install host prerequisites. Instruct rather than blindly running risky external installer commands. | +| `Test-LabReadiness.ps1` | Canonical T-15 readiness gate; returns green/red with per-check detail and optional JSON output. | +| `Get-MacOSRestoreImage.ps1` | Thin wrapper around `Get-MacLabMedia -ArtifactType Firmware`. | +| `New-MacInstallArtifact.ps1` | Thin wrapper around `Get-MacLabMedia -ArtifactType Installer`. | +| `New-MacVm.ps1` | Thin wrapper around `New-MacLabVm`. | +| `Checkpoint-MacVm.ps1` | Thin wrapper around `Checkpoint-MacLabVm`. | +| `Restore-MacVmCheckpoint.ps1` | Thin wrapper around `Restore-MacLabVmCheckpoint`; prints cloud-state warning before and after restore. | +| `Remove-MacVm.ps1` | Thin wrapper around `Remove-MacLabVm`; prompts about cloud cleanup planning. | +| `Reset-IntuneMacLabDevice.ps1` | Cloud cleanup/reconciliation routine. V1 is report-only. Report-only output identifies candidate Intune, Entra, and Defender records; explains why they may be stale; shows portal paths or Graph commands for manual cleanup; and writes redacted evidence. Soft-delete/retire or hard-delete is deferred and requires a later owner-approved Phase 10 item. | +| `Send-LabEventToLogAnalytics.ps1` | Optional Phase 10 helper; disabled/deferred by default. | +| `Invoke-MMSDemo.ps1` | Stage-friendly Demo 4 orchestrator with interactive gates and non-interactive rehearsal mode. | + +## 19. Examples and Documentation + +### 19.1 Examples + +`examples/MMSMOA-2026/` MUST contain: + +- `demo-config.yml` +- `Demo1-Media.ps1` +- `Demo2-Parallels.ps1` +- `Demo3-UTM.ps1` +- `Demo4-IntuneValidation.ps1` + +`demo-config.yml` MUST use placeholders and include at minimum: + +```yaml +name: mms-demo-macos +macOS: + version: '' + build: '' + architecture: arm64 + artifactType: Both + source: Mist +providerDefaults: + parallels: + cpus: 4 + memoryGB: 8 + diskGB: 96 + utm: + cpus: 4 + memoryGB: 8 + diskGB: 96 +evidence: + outputRoot: './_evidence' + redactSecrets: true +network: + shared: true + bridged: false +intune: + tenant: '.onmicrosoft.example' + policySetId: '' +fidelity: + defaultBucket: Yellow +``` + +`examples/TestCases/` MUST contain: + +- `FileVault-Validation.yml` +- `Defender-Validation.yml` +- `PPPC-Validation.yml` +- `Compliance-SmokeTest.yml` + +Each test plan declares `name`, `version`, `description`, `fidelity`, supported providers, required checkpoint, steps, and expectations. + +`examples/utm/macos-lab-template.utm.json` MUST be a text descriptor, not a binary `.utm` bundle. + +### 19.2 Documentation Files + +Every non-trivial doc MUST include the inherited metadata block. Required docs: + +| File | Required content | +| --- | --- | +| `README.md` | What the repo is/is not, fastest safe starting path, badges, five-line usage snippet, talk metadata, license, acknowledgements. | +| `docs/Start-Here.md` | Canonical entry point: what kit is, who it is for, five-minute Monday plan, what to read next, where to ask questions. | +| `docs/Prereqs.md` | Apple-silicon host requirements, software, provider prerequisites, Graph module guidance, tenant prerequisites. | +| `docs/Hypervisor-Decision-Guide.md` | Parallels vs. UTM vs. Tart decision matrix and UTM descriptor reconstruction path. | +| `docs/Apple-Silicon-Constraints.md` | Virtualization framework, host/guest constraints, Apple SLA link and two-guest default posture, APNs/network caveats, physical hardware boundaries. | +| `docs/Provider-Version-Matrix.md` | Definition and sample matrix; "works on my Mac" is not evidence. | +| `docs/Fidelity-Boundaries.md` | Traffic Light table, cmdlet honesty rules, cloud-state warning, Yellow-result wording for change tickets. | +| `docs/Snapshot-Strategy.md` | Five-checkpoint model, cloud cleanup, restore testing, identity-drift warnings. | +| `docs/Intune-Tenant-Setup.md` | Lab tenant/user/group setup, Apple MDM Push Certificate, policies, assignments, filters, CA scope cautions. | +| `docs/FileVault-Validation.md` | Policy assignment, `fdesetup status`, escrow evidence, redacted proof, hardware sign-off boundary. | +| `docs/Defender-Validation.md` | Package, system extension, network extension, PPPC/FDA, onboarding, `mdatp health`, evidence. | +| `docs/PPPC-Validation.md` | Bundle ID, code requirement, app path, profile receipt, behavior tests, pitfalls. | +| `docs/Evidence-and-CAB.md` | Evidence design, schema, summary format, CAB usage, redaction levels. | +| `docs/Evidence-Redaction.md` | Redaction helper behavior, field list, shape matching, verification tests. | +| `docs/Windows-Admin-Cheat-Sheet.md` | Translation table from Windows lab instincts to macOS VM lab patterns. | +| `docs/Log-Analytics-Integration.md` | Optional Phase 10 doc; disabled by default, env-var based, redaction-first. | +| `docs/ConfigMgr-Inventory-Bridge.md` | Optional Phase 10 doc; inventory adjacency only, no secret export. | +| `docs/CI-and-Tart.md` | Optional Tart/CI guidance, Tart/Orchard Fair Source free-tier links and limits, and license verification warning. | +| `docs/Troubleshooting.md` | Symptom, likely cause, diagnostic step, safe remediation table. | +| `docs/Demo-Runbook.md` | T-15 readiness gate, pre-stage checklist, per-demo recovery paths, recording fallback rules, secret-leak response. | + +The README usage snippet MUST be: + +```powershell +Import-Module ./src/Modules/MacLab/MacLab.psd1 +Get-MacLabMedia -Version '' -Build '' +New-MacLabVm -Provider Parallels -Name 'demo-01' -MediaId '-' +Checkpoint-MacLabVm -Provider Parallels -Name 'demo-01' -CheckpointName 'Pre-Enroll' +Invoke-MacPolicyValidation -Provider Parallels -Name 'demo-01' -TestPlan ./examples/TestCases/Compliance-SmokeTest.yml +``` + +## 20. Tests and CI + +### 20.1 Test Files + +| File | Required coverage | +| --- | --- | +| `tests/MacLab.Tests.ps1` | Manifest validity, exact exports, comment-based help, output types, media mocks, invalid VM names, missing media metadata, checkpoint validation, restore warning, Red-bucket rejection, redaction defaults, `WhatIf`. | +| `tests/Providers.Parallels.Tests.ps1` | `prlctl` detection, version parsing, Shared networking, sharing disabled, checkpoint state enforcement, restore warning, synthetic `prlctl list` parsing. | +| `tests/Providers.UTM.Tests.ps1` | UTM/`utmctl` detection, version parsing, Apple Virtualization enforcement, manual-step-required errors for unsupported primitives. | +| `tests/Validation.FileVault.Tests.ps1` | Synthetic recovery-key redaction by name and shape, JWT redaction, refusal to write unredacted evidence, FileVault plan parse, Yellow fidelity, cloud warning. | +| `tests/Validation.Defender.Tests.ps1` | Defender plan parse, synthetic `mdatp health` parsing, missing system extension as evidence `Fail`, Graph token redaction. | + +Default tests MUST NOT require real Parallels, UTM, Tart, Apple ID, Microsoft 365 tenant, Microsoft Graph endpoint, Defender installation, or a real macOS VM. + +### 20.2 CI Workflow + +The new repository has: + +1. Inherited `auto-fix-precommit.yml`, unchanged. +2. New `.github/workflows/pester.yml`. + +The `pester.yml` workflow MUST: + +- Run on push to `main` and pull requests targeting `main`. +- Use `macos-latest` for macOS CI unless the template or GitHub Actions availability changes before implementation. The inherited template may also include Ubuntu and Windows matrix entries; Python-specific workflow jobs may be removed if Python sample content is trimmed from the repo. +- Checkout code using the template/Dependabot-managed checkout action version. +- Install PowerShell 7.4 or newer if needed. +- Install Pester `5.7.1`. +- Install PSScriptAnalyzer. +- Run PSScriptAnalyzer against `src/`, `scripts/`, and `examples/MMSMOA-2026/`. +- Run Pester against `tests/`. +- Upload test results as an artifact. +- Require no secrets and no authenticated network calls. + +Action versions in `uses:` lines MUST remain directly visible for Dependabot. Tool versions may be centralized in workflow `env` if needed. + +### 20.3 Validation Commands + +Before every commit, run: + +```bash +pre-commit run --all-files +npm run lint:md +npm run lint:md:nested +``` + +For PowerShell validation, run the repo's configured Pester and PSScriptAnalyzer commands as implemented in CI. + +## 21. Implementation Phases + +The build is delivered in phases. The agent MUST stop at each gate, summarize deliverables and validation, and wait for owner approval before continuing. + +### 21.1 Phase 0: Bootstrap + +**Goal:** Initialize from template and customize identity-only fields. + +**Deliverables:** + +- New public repo at [`franklesniak/macOSLab`](https://github.com/franklesniak/macOSLab). +- `LICENSE` copyright line set. +- Phase 0 README skeleton. +- CODEOWNERS validated. +- `package.json` customized and lockfile regenerated. +- Irrelevant template sample artifacts removed only if safe. + +**Acceptance:** Section 12.3. + +**Gate:** Stop for owner approval. + +### 21.2 Phase 1: Foundation Documentation + +**Goal:** Ship read-this-first documents. + +**Deliverables:** + +- `docs/Start-Here.md` +- `docs/Prereqs.md` +- `docs/Hypervisor-Decision-Guide.md` +- `docs/Apple-Silicon-Constraints.md` +- `docs/Provider-Version-Matrix.md` +- `docs/Fidelity-Boundaries.md` +- `docs/Snapshot-Strategy.md` +- `docs/Windows-Admin-Cheat-Sheet.md` +- Any required root phase TODO files from Section 9.6. +- Full `README.md` +- No `SECURITY.md` rewrite. If the inherited file supports a narrow project-specific addition, propose only the exact paragraph from Section 12.1 for owner review. + +**Acceptance:** Markdown lint green, metadata blocks present, no real secrets or identifiers. + +**Gate:** Stop for owner approval. + +### 21.3 Phase 2: Module Skeleton and CI + +**Goal:** Stand up the module surface with compliant stubs and CI. + +**Deliverables:** + +- `MacLab.psd1` with owner-approved GUID. +- `MacLab.psm1`. +- Public cmdlet stubs. +- Private helper stubs. +- Provider stubs. +- `tests/MacLab.Tests.ps1`. +- `.github/workflows/pester.yml`. + +**Acceptance:** Manifest imports, PSScriptAnalyzer is clean, Pester is green, stubs fail clearly. + +**Gate:** Stop for owner approval. + +### 21.4 Phase 3: Lab Readiness Gate + +**Goal:** Deliver readiness and prerequisite scripts. + +**Deliverables:** + +- `scripts/Install-Prereqs.ps1` +- `scripts/Test-LabReadiness.ps1` +- Tests for readiness logic. + +**Acceptance:** Owner can run on an Apple-silicon Mac; readiness reports green/red with clear detail. + +**Gate:** Stop for owner approval. + +### 21.5 Phase 4: Media Acquisition + +**Goal:** Implement pinned media acquisition. + +**Deliverables:** + +- `Get-MacLabMedia` +- `scripts/Get-MacOSRestoreImage.ps1` +- `scripts/New-MacInstallArtifact.ps1` +- `TODO-Phase-04-Media-Acquisition.md` with the current `mist-cli` verification task, unless that task is completed and closed in the same phase. +- Pester tests. + +**Acceptance:** Live owner run fetches a pinned macOS build and produces metadata sidecar; tests pass. + +**Gate:** Stop for owner approval. + +### 21.6 Phase 5: Parallels Provider + +**Goal:** Implement primary provider path. + +**Deliverables:** + +- `Providers/Parallels.ps1` +- VM lifecycle cmdlets for `-Provider Parallels` +- `tests/Providers.Parallels.Tests.ps1` +- `examples/MMSMOA-2026/Demo2-Parallels.ps1` +- `TODO-Phase-05-Parallels-Provider.md` with the `prlctl` verification task, unless that task is completed and closed in the same phase. + +**Acceptance:** Owner live demo creates, checkpoints, restores, and removes a Parallels VM; tests pass. + +**Gate:** Stop for owner approval. + +### 21.7 Phase 6: UTM Provider and Tart Stub + +**Goal:** Implement UTM provider-swap path and Tart v1 stub. + +**Deliverables:** + +- `Providers/UTM.ps1` +- `Providers/Tart.ps1` +- `tests/Providers.UTM.Tests.ps1` +- `examples/MMSMOA-2026/Demo3-UTM.ps1` +- `examples/utm/macos-lab-template.utm.json` +- `TODO-Phase-06-UTM-Provider.md` with UTM/`utmctl` verification tasks, unless those tasks are completed and closed in the same phase. + +**Acceptance:** Owner confirms UTM path and manual-step gaps are clear; tests pass. + +**Gate:** Stop for owner approval. + +### 21.8 Phase 7: Evidence Pipeline + +**Goal:** Implement redaction, evidence writing, and bundling. + +**Deliverables:** + +- `Protect-MacLabEvidence` +- `Write-EvidenceRecord` +- `Export-MacLabEvidence` +- Tests for recovery-key, JWT, nested redaction, and idempotency. +- `docs/Evidence-and-CAB.md` +- `docs/Evidence-Redaction.md` + +**Acceptance:** Redaction tests pass; evidence bundle remains redacted; schema is valid. + +**Gate:** Stop for owner approval. + +### 21.9 Phase 8: Validation Loop + +**Goal:** Implement policy validation and canonical test plans. + +**Deliverables:** + +- `Invoke-MacPolicyValidation` +- Four YAML test plans. +- FileVault, Defender, PPPC, and Intune docs. +- Validation tests. +- `TODO-Phase-08-Validation-Loop.md` with Defender `mdatp health` output-shape verification, unless that task is completed and closed in the same phase. + +**Acceptance:** Test plans parse and produce valid fixture evidence; Red-bucket assertions are rejected; redaction defaults on. + +**Gate:** Stop for owner approval. + +### 21.10 Phase 9: Demo Orchestrator and MMSMOA Examples + +**Goal:** Make the kit sufficient for the talk. + +**Deliverables:** + +- `scripts/Invoke-MMSDemo.ps1` +- `Demo1-Media.ps1` +- `Demo4-IntuneValidation.ps1` +- `demo-config.yml` +- `docs/Demo-Runbook.md` +- `docs/Troubleshooting.md` + +**Acceptance:** Owner completes a dry run of apply, break, rollback, evidence export. + +**Gate:** Stop for owner approval. Tag `v0.1.0-mmsmoa-preview` only with owner go-ahead. + +### 21.11 Phase 10: Optional Bridges and Polish + +**Goal:** Add optional items only when individually approved. + +Possible deliverables: + +- `scripts/Reset-IntuneMacLabDevice.ps1` +- `scripts/Send-LabEventToLogAnalytics.ps1` +- `docs/Log-Analytics-Integration.md` +- `docs/ConfigMgr-Inventory-Bridge.md` +- `docs/CI-and-Tart.md` +- Fuller Tart provider. +- `TODO-Phase-10-Deferred-Work.md`, required when any Phase 10 item remains deferred. It MUST explain what is deferred, why, what context the owner needs to resume it, and how to validate it safely. + +**Acceptance:** Per deliverable. + +**Gate:** Owner approval per item. + +## 22. Definition of Done + +The v1 build is complete when all of these are true on `main`: + +- Repo exists publicly at [`franklesniak/macOSLab`](https://github.com/franklesniak/macOSLab). +- `LICENSE` shows `Copyright (c) 2026 Frank Lesniak`. +- README renders correctly and points to `docs/Start-Here.md`. +- CODEOWNERS lists the approved owner(s). +- Protected instruction files were not modified without explicit authorization. +- Project-owned files in Section 14 exist with specified names. +- No binary media, screenshots, recordings, or credential-bearing files are committed. +- `Test-ModuleManifest src/Modules/MacLab/MacLab.psd1` succeeds. +- `Import-Module src/Modules/MacLab/MacLab.psd1` succeeds. +- `Get-Command -Module MacLab` lists exactly the ten public cmdlets. +- Every public and private function has required comment-based help. +- Every rollback path emits the cloud-state warning. +- PSScriptAnalyzer reports zero findings. +- `pre-commit run --all-files` passes. +- `npm run lint:md` passes. +- `npm run lint:md:nested` passes if present. +- Pester tests pass. +- CI is green. +- Default tests require no real provider tools or cloud credentials. +- Every required doc exists with metadata. +- No docs or examples contain real secrets, users, tenants, or device identifiers. +- Evidence export works from fixture data. +- Redaction is applied before sensitive evidence is written. +- FileVault docs never instruct the user to display a raw recovery key. +- Defender docs validate profiles and health, not only installation. +- VM Fidelity Traffic Light is present and clear. +- Cloud rollback boundary is repeated in snapshot, Intune, and demo docs. +- Owner has performed or explicitly waived the live Apple-silicon end-to-end run. +- Release tag `v0.1.0-mmsmoa-preview` is created only after owner approval. + +## 23. Cmdlet Quick Reference + +| Cmdlet | Purpose | Abbreviated signature | +| --- | --- | --- | +| `Get-MacLabMedia` | Pin and cache macOS media. | `-Version [-Build ] [-Architecture arm64] [-Source Mist] [-ArtifactType Installer/Firmware/Both] [-CacheRoot ] [-Force] [-RedactSecrets]` | +| `New-MacLabVm` | Create a VM under a provider. | `-Provider Parallels/UTM/Tart -Name -MediaId [-CacheRoot ] [-SizingProfile Baseline/Performance] [-VmRoot ] [-RedactSecrets]` | +| `Get-MacLabVm` | List or describe lab VMs. | `[-Provider ] [-Name ]` | +| `Start-MacLabVm` | Start a VM. | `-Provider -Name [-WaitForReady]` | +| `Stop-MacLabVm` | Stop a VM. | `-Provider -Name [-Force]` | +| `Checkpoint-MacLabVm` | Capture a checkpoint. | `-Provider -Name -CheckpointName [-AllowNonCanonicalCheckpoint] [-Description ] [-RequireCleanShutdown]` | +| `Restore-MacLabVmCheckpoint` | Restore a VM checkpoint. | `-Provider -Name -CheckpointName [-AcknowledgeCloudStateWarning]` | +| `Remove-MacLabVm` | Delete a lab VM. | `-Provider -Name [-RemoveDiskFiles] [-Force]` | +| `Invoke-MacPolicyValidation` | Run a YAML test plan and emit evidence. | `-Provider -Name -TestPlan [-OutputPath ] [-RedactSecrets] [-Fidelity Green/Yellow/Red] [-GraphScope ]` | +| `Export-MacLabEvidence` | Bundle an evidence run. | `[-Name ] [-RunId ] [-EvidenceRoot ] -OutputPath [-Format Directory/Zip] [-RedactSecrets]` | + +## 24. Owner Decisions Archive + +Closed owner decisions and implementation uncertainties are archived in [macOS-imaging-08d-closed-questions-archive.md](macOS-imaging-08d-closed-questions-archive.md). Durable decisions are recorded in [macOS-imaging-08e-ADRs.md](macOS-imaging-08e-ADRs.md). + +The future coding agent MUST use this specification and the ADR file as the active contract. The archive is historical context, not a pending work list. + +## 25. Evidence Bundle Schema + +Every evidence record produced by `Invoke-MacPolicyValidation` and bundled by `Export-MacLabEvidence` MUST conform to this shape or a backwards-compatible extension approved by the owner. + +```json +{ + "$schemaVersion": "1.0.0", + "runId": "2026-05-mms-demo4-001", + "createdAt": "2026-05-04T00:00:00Z", + "vmName": "mms-parallels-01", + "provider": "Parallels", + "providerVersion": "", + "snapshot": "Post-Enroll-Baseline", + "fidelity": "Yellow", + "hardwareSignoffRequired": true, + "providerVersionMatrix": { + "hostMacOS": "", + "hostMacOSBuild": "", + "guestMacOS": "", + "guestBuild": "", + "parallelsVersion": "", + "utmVersion": null, + "tartVersion": null, + "powerShellVersion": "7.4.0", + "pesterVersion": "5.7.1", + "defenderVersion": "", + "intunePolicySetId": "" + }, + "intuneDeviceName": "MMS-MACLAB-001", + "intuneDeviceIdRedacted": "***REDACTED***", + "tenantSuffixOnly": ".onmicrosoft.example", + "redactionApplied": true, + "redactionVersion": "1.0.0", + "cloudStateWarning": "VM rollback does not rewind Intune, Entra, Defender portal state, audit logs, or reporting history.", + "tests": [ + { + "name": "MDM enrollment profile present", + "kind": "MdmEnrollment", + "result": "Pass", + "expectedFailure": false, + "evidenceRefs": [] + }, + { + "name": "FileVault status captured", + "kind": "FileVaultStatus", + "result": "Pass", + "expectedFailure": false, + "evidenceRefs": ["fdesetup-status.txt"] + }, + { + "name": "FileVault escrow evidence captured", + "kind": "FileVaultEscrow", + "result": "Pass", + "expectedFailure": false, + "evidenceRefs": ["graph-escrow-summary.json"] + }, + { + "name": "FileVault recovery key value redacted", + "kind": "RedactionVerification", + "result": "Pass", + "expectedFailure": false, + "evidenceRefs": [] + }, + { + "name": "Defender health captured", + "kind": "DefenderHealth", + "result": "Pass", + "expectedFailure": false, + "evidenceRefs": ["mdatp-health.json"] + }, + { + "name": "Compliance smoke test", + "kind": "ComplianceState", + "result": "Fail", + "expectedFailure": true, + "evidenceRefs": ["graph-compliance-snapshot.json"] + }, + { + "name": "Rollback restored known-good VM checkpoint", + "kind": "RollbackVerification", + "result": "Pass", + "expectedFailure": false, + "evidenceRefs": [] + }, + { + "name": "Cloud cleanup routine documented", + "kind": "CloudCleanupNotice", + "result": "Warn", + "expectedFailure": false, + "evidenceRefs": ["docs/Snapshot-Strategy.md"] + } + ] +} +``` + +Schema notes: + +- Field names are lowerCamelCase except `$schemaVersion`. +- `result` is one of `Pass`, `Fail`, or `Warn`. +- `expectedFailure: true` means a `Fail` result is the intended engineered demo failure. +- `evidenceRefs` are relative paths inside the evidence run directory. +- Raw recovery keys, tokens, client secrets, and tenant secrets never appear in this object. + +## 26. Glossary + +| Term | Meaning in this repository | +| --- | --- | +| APNs | Apple Push Notification service. macOS MDM depends on it. | +| ADE/ABM | Apple Automated Device Enrollment / Apple Business Manager. Red-bucket VM fidelity. | +| CAB | Change Advisory Board. One target audience for the evidence bundle. | +| Cloud-state warning | The statement that VM rollback does not rewind Intune, Entra, Defender portal state, audit logs, or reporting history. | +| Demo 4 | The apply, break, rollback, evidence loop. | +| Evidence bundle | Portable redacted output from a validation run. | +| FileVault personal recovery key | A secret that must never appear in committed artifacts. | +| Fidelity Traffic Light | Green, Yellow, Red model for what VMs can validate. | +| Five-checkpoint model | `Clean-OS`, `Pre-Enroll`, `Post-Enroll-Baseline`, `Broken-Policy-State`, `Recovered-Known-Good`. | +| Intune | Microsoft Intune, the first-class management plane for the repo. | +| MDM identity drift | Mismatch between rolled-back local VM state and cloud-side records that kept moving forward. | +| mist-cli | macOS CLI used to acquire pinned install artifacts. | +| Parallels | Primary hypervisor provider. | +| PPPC/TCC | macOS privacy preferences and Transparency, Consent, and Control framework. | +| Provider Version Matrix | Host, guest, hypervisor, PowerShell, Pester, Defender, and policy versions captured in evidence. | +| `Protect-MacLabEvidence` | Redaction helper. Never named `Redact-MacLabEvidence`. | +| Tart | Optional CLI-first Apple-silicon VM tool; v1 may stub it. | +| `Test-LabReadiness.ps1` | Canonical T-15 readiness gate. | +| UTM | Secondary hypervisor provider using Apple Virtualization for macOS guests. | +| VM-first, hardware-last | Iterate and gather evidence in VMs; sign off on physical hardware where required. | + +## 27. Final Approval Line + +> **Owner approval:** I, Frank Lesniak, have read this specification end to end. I have initialed or struck through every box in the Approval Checklist. I have reviewed the closed questions archive and ADR file. Any future unresolved item must be recorded as a new explicit decision before implementation proceeds. The agent is authorized to begin Phase 0 as described in Section 21.1. +> +> Signature: ______________________ +> +> Date: __________________________ +> +> Pinned macOS version: __________________________ +> +> Pinned macOS build: __________________________ +> +> MacLab module GUID: __________________________ + +Reading-time estimate: full first pass, 60-75 minutes. Approval checklist plus ADR file, 10-15 minutes. diff --git a/docs/planning/macOS-imaging-08e-ADRs.md b/docs/planning/macOS-imaging-08e-ADRs.md new file mode 100644 index 0000000..2a348c6 --- /dev/null +++ b/docs/planning/macOS-imaging-08e-ADRs.md @@ -0,0 +1,290 @@ + +# macOSLab Architecture Decision Records + +## Metadata + +- **Status:** Draft for owner review +- **Owner:** Frank Lesniak +- **Last Updated:** 2026-05-04 +- **Scope:** Architecture and planning decisions for the future `macOSLab` repository specification. This file records accepted decisions and conditional follow-up decisions that affect implementation, phase gates, or repository policy. +- **Related:** [macOSLab merged specification](macOS-imaging-08c-merged.md), [Closed questions archive](macOS-imaging-08d-closed-questions-archive.md), [Original prompt](macOS-imaging-08-repo-spec.md), [Repository Copilot Instructions](../../.github/copilot-instructions.md), [Documentation Writing Style](../../.github/instructions/docs.instructions.md) + +## ADR-0001: PowerShell Runtime Floor + +- **Status:** Accepted +- **Date:** 2026-05-04 + +### Context + +The previous draft defaulted to PowerShell 7.2. The owner has decided that the repository should support only PowerShell 7.4.x and newer. + +### Decision + +The `MacLab` module manifest MUST set `PowerShellVersion = '7.4'`. Documentation and CI MUST treat PowerShell 7.4 as the minimum supported runtime. Newer PowerShell 7.x releases are supported unless a concrete incompatibility is documented. + +### Consequences + +- Implementation can use modern PowerShell 7.4 behavior without spending time on older runtime compatibility. +- Attendees on older PowerShell 7 releases must upgrade before using the kit. +- The prerequisite script and readiness gate must check for PowerShell 7.4 or newer. + +### Alternatives Considered + +- Keep PowerShell 7.2 as the floor. Rejected because the owner explicitly wants 7.4.x and newer only. +- Leave runtime unspecified. Rejected because the module should fail early and clearly on unsupported PowerShell versions. + +## ADR-0002: macOS Support Policy and Demo Version + +- **Status:** Accepted +- **Date:** 2026-05-04 + +### Context + +The owner will demo VMs running macOS 26.x and identified `26.4.1` as the current demo target. The repo should also be compatible with supported 14.x and 15.x macOS versions, specifically initial targets `14.8.5` and `15.7.5`. + +### Decision + +The repository MUST use an Apple-supported-macOS policy. Initial targets are: + +- Demo path: macOS Tahoe 26.4.1. +- Compatibility target: macOS Sequoia 15.7.5. +- Compatibility target: macOS Sonoma 14.8.5. + +The Provider Version Matrix MUST record exact host and guest version/build values for every run. + +### Consequences + +- The demo remains current while the starter kit remains useful for admins still validating supported older macOS releases. +- Tests and docs must avoid assuming that every user is on the live demo version. +- Media acquisition and provider code must handle version/build as data, not as hardcoded constants. + +### Alternatives Considered + +- Support only 26.x. Rejected because the owner wants practical compatibility with supported 14.x and 15.x. +- Track "latest" continuously. Rejected because reproducibility matters more than novelty. +- Support beta/preview macOS versions. Rejected for the v1 public path. + +## ADR-0003: Root Per-Phase TODO Files + +- **Status:** Accepted +- **Date:** 2026-05-04 + +### Context + +Several items can safely be verified later, but the owner is concerned they will be forgotten if they live only in planning notes. The owner requested root-level, per-phase TODO files in the implementation repository. + +### Decision + +The future repository MUST create root-level per-phase TODO files when a phase has unresolved or deferred work. File names follow `TODO-Phase--.md`. Known required TODO files are: + +- `TODO-Phase-04-Media-Acquisition.md` +- `TODO-Phase-05-Parallels-Provider.md` +- `TODO-Phase-06-UTM-Provider.md` +- `TODO-Phase-08-Validation-Loop.md` +- `TODO-Phase-10-Deferred-Work.md` + +The file is omitted when the phase has no outstanding TODOs. + +### Consequences + +- Deferred work is visible from the repo root. +- README and phase summaries must link to active TODO files. +- Phase 10 deferral is acceptable only when the deferred items are explained in `TODO-Phase-10-Deferred-Work.md`. + +### Alternatives Considered + +- Track TODOs only in GitHub Issues. Rejected because the owner wants root-visible context. +- Track all TODOs in one root file. Rejected because per-phase files reduce clutter and make phase gates easier to review. +- Track no TODO files. Rejected because deferred verification items are easy to miss. + +## ADR-0004: Template Initialization Guide + +- **Status:** Accepted +- **Date:** 2026-05-04 + +### Context + +The future repository will be initialized from `franklesniak/copilot-repo-template`. The owner plans to work with a coding agent to create a separate instructions/template repository implementation steps guide after this specification is complete. + +### Decision + +The implementation agent MUST use the forthcoming template initialization steps guide during Phase 0. The agent must still inspect the template at implementation time, preserve governance files, and avoid modifying protected instruction files without explicit authorization. + +### Consequences + +- Phase 0 becomes safer and more repeatable. +- This spec does not need to duplicate every template initialization step. +- If the guide and this spec conflict, the new repo's canonical instruction files and this spec still govern protected-file behavior. + +### Alternatives Considered + +- Rely only on this spec. Rejected because the owner wants a more focused template initialization guide. +- Vendor-copy the template into the planning repo. Rejected because it would drift from the actual template. + +## ADR-0005: Deferred Tool Verification TODOs + +- **Status:** Accepted +- **Date:** 2026-05-04 + +### Context + +The owner accepts later verification for `mist-cli`, `prlctl`, UTM/`utmctl`, and Defender `mdatp health`, but wants these items clearly visible in the implementation repo. + +### Decision + +The following verifications are deferred to the implementation phase but MUST be tracked in root TODO files until closed: + +| Phase | TODO file | Verification | +| --- | --- | --- | +| Phase 4 | `TODO-Phase-04-Media-Acquisition.md` | Verify current `mist-cli` list/download syntax. | +| Phase 5 | `TODO-Phase-05-Parallels-Provider.md` | Verify current `prlctl` syntax, version output, and edition detection. | +| Phase 6 | `TODO-Phase-06-UTM-Provider.md` | Verify current UTM/`utmctl` automation surface and manual-step gaps. | +| Phase 8 | `TODO-Phase-08-Validation-Loop.md` | Verify current Defender `mdatp health` output shape and sanitize fixtures. | + +### Consequences + +- The spec can proceed without pretending current external tool behavior is frozen. +- The implementation repo remains explicit about what must be verified before each phase is accepted. +- Phase summaries must close or carry forward the relevant TODOs. + +### Alternatives Considered + +- Verify every tool in this planning repo. Rejected because implementation-time installed versions matter. +- Trust planning examples without verification. Rejected because CLI syntax and output can change. + +## ADR-0006: Tart v1 Scope and License Posture + +- **Status:** Accepted +- **Date:** 2026-05-04 + +### Context + +Tart is useful for a CI/runner conversation but not required for the core talk path. The owner supplied Tart and Orchard free-tier license links. + +### Decision + +The v1 repository keeps Tart as a stub provider unless fuller implementation is later approved. `docs/CI-and-Tart.md` MUST link to: + +- [Tart license](https://github.com/cirruslabs/tart/blob/main/LICENSE) +- [Orchard license](https://github.com/cirruslabs/orchard/blob/main/LICENSE) + +The doc MUST summarize the current practical posture: + +- Tart's license file uses Fair Source License 0.9 and defines a 100-user use limitation where user means CPU core used by the product. +- Orchard's license file uses Fair Source License 0.9 and defines a 4-user use limitation where user means a device running macOS. +- The owner-supplied Tart docs language describes the free tier as 100 CPU cores for Tart and 4 Orchard Workers for Orchard. +- This is not legal advice; users must verify license suitability for their own organization. + +### Consequences + +- Tart remains visible without becoming a v1 dependency. +- The repo provides useful context without turning the starter kit into a licensing authority. +- Full Tart provider parity is deferred until explicitly approved. + +### Alternatives Considered + +- Implement full Tart support in v1. Rejected because it is outside the core talk path. +- Remove Tart entirely. Rejected because it is valuable as an advanced/CI extension point. + +## ADR-0007: Apple macOS Virtualization License Language + +- **Status:** Accepted +- **Date:** 2026-05-04 + +### Context + +The owner provided the [macOS Tahoe SLA](https://www.apple.com/legal/sla/docs/macOSTahoe.pdf) as the source for Apple/macOS virtualization and licensing language. + +### Decision + +The docs MUST link to the macOS Tahoe SLA and summarize the practical boundary without giving legal advice: + +- The repo is designed for Apple-branded computers. +- The Tahoe SLA includes language allowing up to two additional macOS virtual instances on an Apple-branded computer already running macOS for specified purposes such as software development, testing during software development, macOS Server, or personal non-commercial use. +- The repo's default posture is "two concurrent macOS guests per Apple-branded host unless the owner/user verifies a different license posture." +- The docs must tell users to verify their own license/procurement posture. + +### Consequences + +- The docs can explain the lab boundary plainly. +- The repo avoids making unsupported licensing claims. +- The readiness and hypervisor docs should design around the two-guest assumption. + +### Alternatives Considered + +- Omit licensing language. Rejected because virtualization boundaries matter for a public macOS lab repo. +- Provide legal advice. Rejected because the repo is a technical starter kit, not legal counsel. + +## ADR-0008: CI Runner and Pester Version + +- **Status:** Accepted +- **Date:** 2026-05-04 + +### Context + +The owner identified that the template already uses `macos-latest` in active PowerShell and Python workflow matrices. The owner also supplied current Pester version `5.7.1`. + +### Decision + +The v1 CI workflow uses `macos-latest` for macOS CI and pins Pester `5.7.1`. Python-specific workflow jobs may be removed if Python sample content is trimmed from the repo. + +### Consequences + +- CI aligns with the template's existing pattern. +- Tests remain deterministic through a pinned Pester version. +- The repo avoids retaining irrelevant Python CI when no Python code remains. + +### Alternatives Considered + +- Pick a version-pinned macOS runner label now. Rejected because the template uses `macos-latest` and default tests are mocked. +- Leave Pester unpinned. Rejected because deterministic CI is preferable. + +## ADR-0009: `SECURITY.md` Redaction Paragraph + +- **Status:** Accepted +- **Date:** 2026-05-04 + +### Context + +The future repository inherits `SECURITY.md` from the template. That file is a governance/security file, so the implementation should avoid unnecessary drift. The repo still has a specific risk profile: examples and evidence must not contain real tenant identifiers, recovery keys, tokens, production credentials, or personal data. + +### Decision + +Leave `SECURITY.md` unchanged by default and do not rewrite it. During Phase 1, after the inherited template file is visible, the implementation agent may propose only this exact short project-specific paragraph if it fits the existing responsible-disclosure process: + +> This repository ships no real tenant identifiers, no recovery keys, no tokens, and no production credentials. If you discover content that appears to be a real secret, recovery key, tenant identifier, or personal data, please report it privately through the repository's security advisory process rather than opening a public issue. + +### Consequences + +- Leaving the file unchanged preserves template governance and avoids accidental protected-file drift. +- Adding the paragraph improves reporter guidance for this repo's specific risk profile. +- No `SECURITY.md` rewrite is approved. + +### Alternatives Considered + +- Automatically add the paragraph before inspecting the inherited file. Rejected because it could conflict with the template's existing security policy. +- Rewrite `SECURITY.md`. Rejected because it creates unnecessary governance drift. + +## ADR-0010: Cloud Cleanup Mutation Scope + +- **Status:** Accepted +- **Date:** 2026-05-04 + +### Context + +Local VM rollback does not rewind cloud state. A lab operator may therefore see stale Intune, Entra, or Defender records after restoring or deleting a VM. Cleanup guidance is useful, but automated cloud mutation can delete or retire the wrong object if matching, scoping, or confirmation behavior is wrong. + +### Decision + +Start `scripts/Reset-IntuneMacLabDevice.ps1` as report-only in v1. A report-only run should identify candidate Intune, Entra, and Defender records, explain why they may be stale, show portal paths or Graph commands for manual cleanup, and write redacted evidence. Add soft-delete/retire or hard-delete only after a later explicit owner-approved Phase 10 change and tests for scoping, confirmation, and evidence redaction. + +### Consequences + +- V1 helps the operator understand identity drift without risking wrong-object deletion. +- Mutation remains possible later after matching logic, Graph scopes, prompts, and evidence are reviewed. +- Phase 10 must carry any future cloud mutation work in `TODO-Phase-10-Deferred-Work.md`. + +### Alternatives Considered + +- Allow soft-delete/retire by default. Deferred because matching/scoping must be reviewed first. +- Allow hard-delete in v1. Rejected as too risky for the first release. From 60cfcc5e8ec29608e6d4fc2595a58ddcd4ae2c33 Mon Sep 17 00:00:00 2001 From: Frank Lesniak Date: Mon, 4 May 2026 18:26:05 -0500 Subject: [PATCH 02/10] Fix devcontainer issue --- .devcontainer/post-create.sh | 55 ++++++++++++++++++++---------------- 1 file changed, 30 insertions(+), 25 deletions(-) diff --git a/.devcontainer/post-create.sh b/.devcontainer/post-create.sh index f983f99..c4709fe 100644 --- a/.devcontainer/post-create.sh +++ b/.devcontainer/post-create.sh @@ -1,25 +1,30 @@ -#!/usr/bin/env bash -set -euo pipefail - -if [ -f package-lock.json ]; then - npm ci -elif [ -f package.json ]; then - npm install -fi - -sudo npm install -g "@openai/codex@latest" - -python -m venv .venv -. .venv/bin/activate -python -m pip install --upgrade pip -python -m pip install -e ".[dev]" pre-commit -pre-commit install - -mkdir -p "${HOME}/.codex" -cat > "${HOME}/.codex/config.toml" <<'EOF' -approval_policy = "never" -sandbox_mode = "danger-full-access" - -[features] -goals = true -EOF +#!/usr/bin/env bash +set -euo pipefail + +if [ -f package-lock.json ]; then + npm ci +elif [ -f package.json ]; then + npm install +fi + +# `sudo` resets PATH via secure_path, which drops the Node feature's bin +# directory and causes `sudo npm` to fail with "command not found". Resolve +# npm in the calling shell and invoke it through `sudo` while preserving PATH. +NPM_BIN="$(command -v npm)" +sudo env "PATH=$PATH" "$NPM_BIN" install -g npm@latest +sudo env "PATH=$PATH" "$NPM_BIN" install -g "@openai/codex@latest" + +python -m venv .venv +. .venv/bin/activate +python -m pip install --upgrade pip +python -m pip install -e ".[dev]" pre-commit +pre-commit install + +mkdir -p "${HOME}/.codex" +cat > "${HOME}/.codex/config.toml" <<'EOF' +approval_policy = "never" +sandbox_mode = "danger-full-access" + +[features] +goals = true +EOF From 7205d9b88c943f8519d09e60a23daf644e52baf1 Mon Sep 17 00:00:00 2001 From: Frank Lesniak Date: Mon, 4 May 2026 18:31:36 -0500 Subject: [PATCH 03/10] Fix shell script error? --- .devcontainer/post-create.sh | 60 ++++++------ .gitattributes | 178 ++++++++++++++++++----------------- 2 files changed, 122 insertions(+), 116 deletions(-) diff --git a/.devcontainer/post-create.sh b/.devcontainer/post-create.sh index c4709fe..b4b296a 100644 --- a/.devcontainer/post-create.sh +++ b/.devcontainer/post-create.sh @@ -1,30 +1,30 @@ -#!/usr/bin/env bash -set -euo pipefail - -if [ -f package-lock.json ]; then - npm ci -elif [ -f package.json ]; then - npm install -fi - -# `sudo` resets PATH via secure_path, which drops the Node feature's bin -# directory and causes `sudo npm` to fail with "command not found". Resolve -# npm in the calling shell and invoke it through `sudo` while preserving PATH. -NPM_BIN="$(command -v npm)" -sudo env "PATH=$PATH" "$NPM_BIN" install -g npm@latest -sudo env "PATH=$PATH" "$NPM_BIN" install -g "@openai/codex@latest" - -python -m venv .venv -. .venv/bin/activate -python -m pip install --upgrade pip -python -m pip install -e ".[dev]" pre-commit -pre-commit install - -mkdir -p "${HOME}/.codex" -cat > "${HOME}/.codex/config.toml" <<'EOF' -approval_policy = "never" -sandbox_mode = "danger-full-access" - -[features] -goals = true -EOF +#!/usr/bin/env bash +set -euo pipefail + +if [ -f package-lock.json ]; then + npm ci +elif [ -f package.json ]; then + npm install +fi + +# `sudo` resets PATH via secure_path, which drops the Node feature's bin +# directory and causes `sudo npm` to fail with "command not found". Resolve +# npm in the calling shell and invoke it through `sudo` while preserving PATH. +NPM_BIN="$(command -v npm)" +sudo env "PATH=$PATH" "$NPM_BIN" install -g npm@latest +sudo env "PATH=$PATH" "$NPM_BIN" install -g "@openai/codex@latest" + +python -m venv .venv +. .venv/bin/activate +python -m pip install --upgrade pip +python -m pip install -e ".[dev]" pre-commit +pre-commit install + +mkdir -p "${HOME}/.codex" +cat > "${HOME}/.codex/config.toml" <<'EOF' +approval_policy = "never" +sandbox_mode = "danger-full-access" + +[features] +goals = true +EOF diff --git a/.gitattributes b/.gitattributes index 92313fe..b9c5f5f 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,86 +1,92 @@ -# --------------------------------------------------------------------------- -# Byte-exact text artifact protection -# --------------------------------------------------------------------------- -# The rules below pin committed text fixtures to LF line endings so that -# checkouts on Windows hosts with `core.autocrlf=true` do not silently -# rewrite the on-disk bytes. Any test or tool that performs byte-exact -# comparison (hash equality, signature verification, snapshot diffing) -# depends on this stability. -# -# The directory-level rules below are broad on purpose: they catch text -# fixtures regardless of extension (including ad-hoc names like `expected` -# or `baseline`). Binary file types commonly stored alongside text -# fixtures (images, archives, media, fonts, compiled artifacts) are -# declassified further down with Git's built-in `binary` macro so -# Git does not apply line-ending conversion to them. Later patterns win -# per-attribute in `.gitattributes`, so the binary overrides take -# precedence over the directory-wide `text eol=lf` pins. -# -# DO NOT DELETE these rules when customizing this template unless you are -# certain no byte-exact comparison exists in your repository. See -# .github/instructions/gitattributes.instructions.md for the normative rule -# and guidance on adding project-specific entries (including how to -# declassify additional binary types). -# --------------------------------------------------------------------------- - -# Text fixtures: pin LF line endings on checkout. -tests/**/golden/** text eol=lf -tests/**/goldens/** text eol=lf -tests/**/snapshots/** text eol=lf -tests/**/__snapshots__/** text eol=lf -tests/**/fixtures/** text eol=lf -testdata/** text eol=lf - -# Binary overrides: prevent the broad rules above (or any default -# text-detection) from treating these common binary file types as text. -# The `binary` macro expands to `-text -diff`. - -# Images -*.png binary -*.jpg binary -*.jpeg binary -*.gif binary -*.webp binary -*.bmp binary -*.ico binary -*.tif binary -*.tiff binary - -# Documents and archives -*.pdf binary -*.zip binary -*.tar binary -*.gz binary -*.tgz binary -*.bz2 binary -*.xz binary -*.7z binary -*.rar binary - -# Compiled and packaged artifacts -*.exe binary -*.dll binary -*.so binary -*.dylib binary -*.class binary -*.jar binary -*.wasm binary -*.pyc binary - -# Audio and video -*.mp3 binary -*.mp4 binary -*.mov binary -*.wav binary -*.ogg binary -*.webm binary -*.mkv binary -*.avi binary -*.flac binary - -# Fonts -*.ttf binary -*.otf binary -*.woff binary -*.woff2 binary -*.eot binary +# --------------------------------------------------------------------------- +# Byte-exact text artifact protection +# --------------------------------------------------------------------------- +# The rules below pin committed text fixtures to LF line endings so that +# checkouts on Windows hosts with `core.autocrlf=true` do not silently +# rewrite the on-disk bytes. Any test or tool that performs byte-exact +# comparison (hash equality, signature verification, snapshot diffing) +# depends on this stability. +# +# The directory-level rules below are broad on purpose: they catch text +# fixtures regardless of extension (including ad-hoc names like `expected` +# or `baseline`). Binary file types commonly stored alongside text +# fixtures (images, archives, media, fonts, compiled artifacts) are +# declassified further down with Git's built-in `binary` macro so +# Git does not apply line-ending conversion to them. Later patterns win +# per-attribute in `.gitattributes`, so the binary overrides take +# precedence over the directory-wide `text eol=lf` pins. +# +# DO NOT DELETE these rules when customizing this template unless you are +# certain no byte-exact comparison exists in your repository. See +# .github/instructions/gitattributes.instructions.md for the normative rule +# and guidance on adding project-specific entries (including how to +# declassify additional binary types). +# --------------------------------------------------------------------------- + +# Text fixtures: pin LF line endings on checkout. +tests/**/golden/** text eol=lf +tests/**/goldens/** text eol=lf +tests/**/snapshots/** text eol=lf +tests/**/__snapshots__/** text eol=lf +tests/**/fixtures/** text eol=lf +testdata/** text eol=lf + +# Shell scripts must use LF: CRLF breaks shebang resolution and option +# parsing (e.g. `set -euo pipefail\r` is rejected by bash) when the script +# is checked out on Windows with `core.autocrlf=true` and then executed +# inside a Linux container, such as the devcontainer post-create hook. +*.sh text eol=lf + +# Binary overrides: prevent the broad rules above (or any default +# text-detection) from treating these common binary file types as text. +# The `binary` macro expands to `-text -diff`. + +# Images +*.png binary +*.jpg binary +*.jpeg binary +*.gif binary +*.webp binary +*.bmp binary +*.ico binary +*.tif binary +*.tiff binary + +# Documents and archives +*.pdf binary +*.zip binary +*.tar binary +*.gz binary +*.tgz binary +*.bz2 binary +*.xz binary +*.7z binary +*.rar binary + +# Compiled and packaged artifacts +*.exe binary +*.dll binary +*.so binary +*.dylib binary +*.class binary +*.jar binary +*.wasm binary +*.pyc binary + +# Audio and video +*.mp3 binary +*.mp4 binary +*.mov binary +*.wav binary +*.ogg binary +*.webm binary +*.mkv binary +*.avi binary +*.flac binary + +# Fonts +*.ttf binary +*.otf binary +*.woff binary +*.woff2 binary +*.eot binary From dfc2e9b5ccc8f381667d2eb6f4b9e301776640fe Mon Sep 17 00:00:00 2001 From: Frank Lesniak Date: Mon, 4 May 2026 18:39:51 -0500 Subject: [PATCH 04/10] Fix access denied error in devcontainer --- .devcontainer/post-create.sh | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/.devcontainer/post-create.sh b/.devcontainer/post-create.sh index b4b296a..9ccc632 100644 --- a/.devcontainer/post-create.sh +++ b/.devcontainer/post-create.sh @@ -10,9 +10,11 @@ fi # `sudo` resets PATH via secure_path, which drops the Node feature's bin # directory and causes `sudo npm` to fail with "command not found". Resolve # npm in the calling shell and invoke it through `sudo` while preserving PATH. +# `-H` resets HOME so package postinstall scripts (notably @openai/codex) do +# not write into the calling user's home directory as root. NPM_BIN="$(command -v npm)" -sudo env "PATH=$PATH" "$NPM_BIN" install -g npm@latest -sudo env "PATH=$PATH" "$NPM_BIN" install -g "@openai/codex@latest" +sudo -H env "PATH=$PATH" "$NPM_BIN" install -g npm@latest +sudo -H env "PATH=$PATH" "$NPM_BIN" install -g "@openai/codex@latest" python -m venv .venv . .venv/bin/activate @@ -20,6 +22,11 @@ python -m pip install --upgrade pip python -m pip install -e ".[dev]" pre-commit pre-commit install +# Reclaim ownership in case a prior run created ~/.codex as root before the +# `sudo -H` fix above was in place. +if [ -e "${HOME}/.codex" ] && [ ! -O "${HOME}/.codex" ]; then + sudo chown -R "$(id -u):$(id -g)" "${HOME}/.codex" +fi mkdir -p "${HOME}/.codex" cat > "${HOME}/.codex/config.toml" <<'EOF' approval_policy = "never" From e81436cbe009e41c51d2b95f54e501780858f239 Mon Sep 17 00:00:00 2001 From: Frank Lesniak Date: Tue, 5 May 2026 00:37:36 +0000 Subject: [PATCH 05/10] chore: bootstrap macOSLab phase 0 --- .github/CODEOWNERS | 15 +- .github/ISSUE_TEMPLATE/bug_report.yml | 345 +- .github/ISSUE_TEMPLATE/config.yml | 105 +- .../ISSUE_TEMPLATE/documentation_issue.yml | 80 +- .github/ISSUE_TEMPLATE/feature_request.yml | 185 +- .github/TEMPLATE_DESIGN_DECISIONS.md | 1457 +------ .github/dependabot.yml | 17 - .github/pull_request_template.md | 126 +- .github/scripts/lint-nested-markdown.js | 6 +- .github/workflows/auto-fix-precommit.yml | 23 +- .github/workflows/data-ci.yml | 26 +- .github/workflows/powershell-ci.yml | 16 +- .github/workflows/precommit-ci.yml | 45 + .github/workflows/python-ci.yml | 241 -- .github/workflows/terraform-ci.yml | 229 -- .gitignore | 35 +- .pre-commit-config.yaml | 37 +- .tflint.hcl | 97 - .vscode/settings.json | 8 +- CODE_OF_CONDUCT.md | 2 +- CONTRIBUTING.md | 387 +- COPILOT_CHAT_PROMPTS.md | 84 - GETTING_STARTED_EXISTING_REPO.md | 2023 ---------- GETTING_STARTED_NEW_REPO.md | 2276 ----------- OPTIONAL_CONFIGURATIONS.md | 3432 ----------------- README.md | 318 +- SECURITY.md | 7 +- TEMPLATE_MAINTENANCE.md | 322 -- TODO-Phase-00-Branch-Protection.md | 13 + TODO-Phase-04-Media-Acquisition.md | 13 + TODO-Phase-05-Parallels-Provider.md | 13 + TODO-Phase-06-UTM-Provider.md | 13 + TODO-Phase-07-Evidence-Pipeline.md | 13 + TODO-Phase-08-Validation-Loop.md | 13 + TODO-Phase-10-Deferred-Work.md | 15 + docs/phase-0-bootstrap-pr-description.md | 93 + .../TERRAFORM_COPILOT_INSTRUCTIONS_GUIDE.md | 1855 --------- docs/terraform/TERRAFORM_LINTING_GUIDE.md | 1650 -------- docs/terraform/TERRAFORM_TESTING_GUIDE.md | 1845 --------- package-lock.json | 4 +- package.json | 15 +- pyproject.toml | 69 - schemas/README.md | 225 +- src/copilot_repo_template/__init__.py | 25 - src/copilot_repo_template/example.py | 58 - templates/python/README.md | 37 - templates/python/pyproject.toml | 85 - templates/python/tests/__init__.py | 2 - templates/python/tests/test_placeholder.py | 12 - .../python/tests/test_schema_examples.py | 348 -- templates/terraform/Example.tftest.hcl | 131 - tests/__init__.py | 10 - tests/test_example.py | 62 - tests/test_schema_examples.py | 302 -- 54 files changed, 558 insertions(+), 18307 deletions(-) create mode 100644 .github/workflows/precommit-ci.yml delete mode 100644 .github/workflows/python-ci.yml delete mode 100644 .github/workflows/terraform-ci.yml delete mode 100644 .tflint.hcl delete mode 100644 COPILOT_CHAT_PROMPTS.md delete mode 100644 GETTING_STARTED_EXISTING_REPO.md delete mode 100644 GETTING_STARTED_NEW_REPO.md delete mode 100644 OPTIONAL_CONFIGURATIONS.md delete mode 100644 TEMPLATE_MAINTENANCE.md create mode 100644 TODO-Phase-00-Branch-Protection.md create mode 100644 TODO-Phase-04-Media-Acquisition.md create mode 100644 TODO-Phase-05-Parallels-Provider.md create mode 100644 TODO-Phase-06-UTM-Provider.md create mode 100644 TODO-Phase-07-Evidence-Pipeline.md create mode 100644 TODO-Phase-08-Validation-Loop.md create mode 100644 TODO-Phase-10-Deferred-Work.md create mode 100644 docs/phase-0-bootstrap-pr-description.md delete mode 100644 docs/terraform/TERRAFORM_COPILOT_INSTRUCTIONS_GUIDE.md delete mode 100644 docs/terraform/TERRAFORM_LINTING_GUIDE.md delete mode 100644 docs/terraform/TERRAFORM_TESTING_GUIDE.md delete mode 100644 pyproject.toml delete mode 100644 src/copilot_repo_template/__init__.py delete mode 100644 src/copilot_repo_template/example.py delete mode 100644 templates/python/README.md delete mode 100644 templates/python/pyproject.toml delete mode 100644 templates/python/tests/__init__.py delete mode 100644 templates/python/tests/test_placeholder.py delete mode 100644 templates/python/tests/test_schema_examples.py delete mode 100644 templates/terraform/Example.tftest.hcl delete mode 100644 tests/__init__.py delete mode 100644 tests/test_example.py delete mode 100644 tests/test_schema_examples.py diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 0bf8e34..e428669 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1,22 +1,17 @@ # CODEOWNERS file - Defines code ownership for pull request reviews # -# TEMPLATE NOTE: Replace @OWNER with your GitHub username or team. +# Phase 0 bootstrap note: Frank Lesniak is the launch code owner. # See: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners # -# EXPECTED WARNINGS: GitHub will show "Unknown owner" errors for @OWNER in -# the template repository. This is expected and intentional - the placeholder -# must be replaced after cloning. The check-placeholders.yml workflow enforces -# this replacement. -# # After customizing, this file ensures the right people are automatically # requested to review pull requests. # Default owners for everything in the repo -* @OWNER +* @franklesniak # Workflow files require maintainer review -.github/workflows/ @OWNER +.github/workflows/ @franklesniak # Copilot instructions require maintainer review -.github/copilot-instructions.md @OWNER -.github/instructions/ @OWNER +.github/copilot-instructions.md @franklesniak +.github/instructions/ @franklesniak diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index c1e60e3..17389a5 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -1,131 +1,65 @@ -# ============================================================================= -# BUG REPORT TEMPLATE -# ============================================================================= -# For detailed design decisions and customization options, see: -# .github/TEMPLATE_DESIGN_DECISIONS.md > Issue Template Design Decisions > bug_report.yml -# -# CUSTOMIZE: markers below indicate actionable customization points. -# Look for `# CUSTOMIZE:` comments throughout this file. -# ============================================================================= - -name: 🐛 Bug Report -description: Report a bug or unexpected behavior -# CUSTOMIZE: Adjust the title prefix to match your project's conventions +name: Bug Report +description: Report a macOSLab bug or unexpected behavior. title: "[Bug] " -# CUSTOMIZE: Update these labels to match your repository's label taxonomy. -# Ensure these labels exist in your repository or create them before using this template. -# Common patterns include: -# - Type labels: bug, enhancement, documentation -# - Status labels: triage, confirmed, in-progress -# - Priority labels: priority:critical, priority:high, priority:medium, priority:low -# - Area labels: area:api, area:cli, area:docs labels: - bug - # - triage # ACTION ITEM: Uncomment after creating the `triage` label in your repository -# CUSTOMIZE: Uncomment and update to specify an issue type (defined at the organization level). -# See: https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms#top-level-syntax -# type: Bug -# CUSTOMIZE: Uncomment and update to pre-populate the assignees field when opening bug reports -# assignees: -# - maintainer-username -# CUSTOMIZE: Uncomment and update to auto-add bug reports to a GitHub Project (uses project number) -# projects: -# - org/1 + - triage body: - # DESIGN DECISION: REDUNDANT SECURITY WARNINGS - # This template intentionally includes multiple security warnings (top-of-form, - # checkbox, severity dropdown). This defense-in-depth approach maximizes chance - # of catching accidental security disclosure. See .github/TEMPLATE_DESIGN_DECISIONS.md for rationale. - # NOTE: The Markdown links below use absolute URLs because relative paths - # (e.g., `[SECURITY.md](blob/HEAD/SECURITY.md)`) resolve against the - # rendering URL `/{owner}/{repo}/issues/new?...` and 404. See the - # "Issue and PR templates" carve-out in `.github/instructions/docs.instructions.md` - # (and the matching section in `.github/instructions/yaml.instructions.md`). - # - # CUSTOMIZE: Replace `OWNER/REPO` in both URLs with your actual org/repo. - # If you keep `.github/workflows/check-placeholders.yml` (an optional adoption - # step), CI will fail until this substitution is made; if you do not adopt - # that workflow or you remove it after setup, no CI guardrail catches a - # missed substitution and you must verify the replacement manually. - # GHES users: Replace `github.com` with your GHES host (e.g., `github.company.com`). - type: markdown attributes: value: | - ## Thank you for reporting a bug! - Please fill out the form below to help us investigate and resolve the issue. + Thank you for reporting a bug. - ⚠️ **Security Notice:** If you are reporting a security vulnerability, do NOT use this form. - Please report security issues privately via the repository's [Security tab](https://github.com/OWNER/REPO/security) or [SECURITY.md](https://github.com/OWNER/REPO/blob/HEAD/SECURITY.md). + Security vulnerabilities must be reported privately through [GitHub private vulnerability reporting](https://github.com/franklesniak/macOSLab/security/advisories/new), not through this form. - type: checkboxes id: pre_flight attributes: label: Pre-flight Checklist - description: >- - Please confirm the following before submitting options: - - label: I have searched existing issues and this is not a duplicate - required: true - # CUSTOMIZE: Consider changing to `required: true` for projects with - # comprehensive documentation that users should consult before filing bugs - - label: I have read the project documentation - required: false - - label: I am using a supported version of the project + - label: I searched existing issues and this is not a duplicate. required: true - - label: I have tested on the latest version or main branch + - label: I reviewed the current README or planning docs that apply to this behavior. required: false - - label: This is NOT a security vulnerability (report those privately via SECURITY.md) + - label: This is not a security vulnerability. required: true - # CUSTOMIZE: Remove if your project doesn't accept community contributions - - label: I am willing to submit a pull request to fix this issue - required: false - - - type: textarea - id: description - attributes: - label: Description - description: >- - Provide a clear and concise description of the bug - placeholder: Describe what the bug is... validations: required: true - # DESIGN DECISION: Area is optional by default for template portability. - # Small or single-language downstream repos may find "Area" redundant and - # friction-inducing. Downstream repos that rely on area-based routing can - # change this to `required: true` if needed. - type: dropdown id: area attributes: label: Area - description: >- - What part of the project does this bug affect? - Tip: For documentation typos/missing info, prefer the 📚 Documentation Issue template. - # CUSTOMIZE: Update these options to match your project's languages/components (remove unused entries). options: - - Python - - PowerShell + - PowerShell module + - PowerShell scripts + - Pester tests - Markdown / Documentation - GitHub Actions / CI - - Cross-language / Integration - - Cross-cutting / Repo-wide - - Other (describe/specify in Additional Context) + - JSON / YAML / schema validation + - Parallels provider + - UTM provider + - Evidence / redaction + - Other validations: required: false - # NOTE: render: text is used here because users frequently paste commands and output - # in reproduction steps. This prevents Markdown parsing from mangling CLI commands. + - type: textarea + id: description + attributes: + label: Description + description: Describe the bug clearly. + validations: + required: true + - type: textarea id: steps_to_reproduce attributes: label: Steps to Reproduce - description: >- - Provide detailed steps to reproduce the behavior render: text placeholder: | - 1. Go to '...' - 2. Run '...' - 3. See error + 1. Run ... + 2. Observe ... validations: required: true @@ -133,9 +67,6 @@ body: id: expected_behavior attributes: label: Expected Behavior - description: >- - Describe what you expected to happen - placeholder: What should have happened... validations: required: true @@ -143,254 +74,70 @@ body: id: actual_behavior attributes: label: Actual Behavior - description: >- - Describe what actually happened - placeholder: What actually happened... validations: required: true - # NOTE: render: text is used here to prevent Markdown parsing from mangling stack traces, - # CLI output, and scripts. This ensures logs are displayed as plain text regardless of content. - # Prefer render: text for technical textareas as it's the safest default for mixed content - # (Python tracebacks, PowerShell errors, JSON, etc.). - type: textarea id: logs_output attributes: - label: Logs/Error Output - description: >- - Paste any error messages, stack traces, or relevant log output. - For longer outputs, attach log files by dragging and dropping. - ⚠️ Review logs for sensitive data (passwords, tokens, personal info) before submitting + label: Logs or Error Output + description: Review logs for secrets, tenant identifiers, recovery keys, tokens, and personal data before pasting. render: text - placeholder: | - Paste logs here... - - For very large logs, attach as a file by dragging and dropping below. validations: required: false - type: textarea - id: screenshots + id: environment attributes: - label: Screenshots - description: >- - If applicable, add screenshots to help explain the problem. - Drag and drop images directly into this field, or paste image URLs - placeholder: Drag and drop screenshots here... - validations: - required: false - - # CUSTOMIZE: Library/framework projects should consider making this required. - # Projects that don't accept external reproductions can remove this field. - - type: input - id: minimal_reproduction - attributes: - label: Minimal Reproduction URL - description: >- - Link to a minimal reproduction (GitHub repo, CodeSandbox, StackBlitz, etc.). - For complex bugs, this greatly speeds up investigation. - placeholder: e.g., https://github.com/username/repo-name or https://codesandbox.io/s/... - validations: - required: false - - - type: markdown - attributes: - value: "### Environment Information" - - # NOTE: GitHub issue forms do NOT support `pattern` validation on input fields. - # See: https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema#validations-for-input - # For input validation, consider using a dropdown with predefined options, - # or validate programmatically via GitHub Actions on issue creation. - - type: input - id: project_version - attributes: - label: Project Version - description: >- - The version of the project (e.g., v1.2.3 or commit SHA) - placeholder: e.g., v1.2.3 or commit SHA abc1234 - validations: - required: true - - - type: input - id: operating_system - attributes: - label: Operating System - description: >- - Your operating system name and version - placeholder: e.g., Windows 11, Ubuntu 24.04 LTS, macOS 15.x - validations: - required: true - - # CUSTOMIZE: Cross-platform projects should consider making this required. - # Single-platform projects can remove this field. - - type: dropdown - id: architecture - attributes: - label: Architecture - description: >- - Your CPU architecture, if relevant to the issue - options: - - x64 (AMD64 / Intel 64) - - ARM64 (AArch64 / Apple Silicon) - - x86 (32-bit) - - Other (specify in Additional Context) - validations: - required: false - - # CUSTOMIZE: Update placeholder examples to match your project's supported runtimes - # This template uses Python 3.13+ (current bugfix version) as example - # NOTE: render: text is used here because users paste multi-line version output. - # This prevents Markdown parsing from mangling version strings with special characters. - - type: textarea - id: runtime_version - attributes: - label: Language/Runtime Version(s) - description: >- - Relevant language or runtime versions for your environment. - List each on a separate line if multiple apply + label: Environment render: text placeholder: | - Python 3.13.1 (or your installed version) - PowerShell 7.4.6 or Windows PowerShell 5.1 - Markdown tooling/renderer (if relevant): e.g., Pandoc 3.1.2 + Host macOS version/build: + PowerShell version: + Provider: Parallels / UTM / Tart stub + Provider version: + Pester version, if relevant: validations: required: false - # CUSTOMIZE: CLI-focused projects should keep this field; others may remove it if not applicable - - type: input - id: shell_environment - attributes: - label: Shell/Terminal - description: >- - The shell or terminal you're using, if applicable - placeholder: e.g., PowerShell 7.4.x / Windows PowerShell 5.1, bash 5.2 (Linux/macOS), Windows Terminal / Command Prompt - validations: - required: false - - # CUSTOMIZE: Update placeholder examples to match your project's dependency management approach - # NOTE: render: text is used here because users paste exact commands with flags. - # This is especially important for Python and PowerShell where execution context varies widely. - type: textarea - id: how_ran + id: commands attributes: - label: How did you run it? (commands) - description: >- - Paste the exact commands you used to install and run the project (include flags). + label: Commands Used render: text placeholder: | - # Python (using pyproject.toml) - python -m venv .venv - source .venv/bin/activate # On Windows: .venv\Scripts\activate - pip install -e . - python -m your_package - - # Python (using requirements.txt) - pip install -r requirements.txt - python script.py --flag - - # PowerShell - pwsh -File .\script.ps1 -Param Value + pwsh -File ./scripts/Test-LabReadiness.ps1 + Invoke-Pester -Path tests/ -Output Detailed validations: required: false - - type: markdown - attributes: - value: "### Bug Characteristics" - - type: dropdown id: reproducibility attributes: label: Reproducibility - description: >- - How consistently can you reproduce this bug? options: - - Consistent (every time) - - Intermittent (sometimes) + - Consistent + - Intermittent - Unable to reproduce reliably - - Only happened once (haven't retried) + - Only happened once validations: required: true - # CUSTOMIZE: Remove this field if version history is not tracked or not relevant - - type: dropdown - id: regression - attributes: - label: Regression? - description: >- - Did this functionality work correctly in a previous version? - options: - - Unknown - I haven't used previous versions - - Yes - this worked before - - No - this never worked for me - validations: - required: false - - # CUSTOMIZE: Remove if regression tracking is not relevant to your project - - type: input - id: last_working_version - attributes: - label: Last Working Version (only if Regression = "Yes") - description: >- - Only fill this out if you selected "Yes - this worked before" above. - Specify the last version where it worked correctly. - placeholder: e.g., v1.1.0 - validations: - required: false - - # CUSTOMIZE: Adjust severity options to match your project's triage workflow - type: dropdown id: severity attributes: label: Severity - description: >- - Select the severity level that best describes the impact from your perspective. - Note: This is your self-assessment; maintainers may adjust during triage. - ⚠️ For security vulnerabilities, do NOT use this form—report privately via the Security tab or SECURITY.md options: - - Critical (system crash, data loss) - - High (major feature broken, no workaround) - - Medium (feature impaired, workaround exists) - - Low (minor inconvenience, cosmetic issue) + - Critical + - High + - Medium + - Low validations: required: true - - type: markdown - attributes: - value: "### Additional Information" - - - type: textarea - id: workaround - attributes: - label: Possible Solution or Workaround - description: >- - If you have ideas on how to fix this or a workaround you're using, - describe it here - placeholder: Describe any workarounds or potential fixes... - validations: - required: false - - # CUSTOMIZE: Update the cross-repo example in the placeholder to reference - # a related project in your ecosystem, or simplify to just `#123` - - type: textarea - id: related_issues - attributes: - label: Related Issues - description: >- - Link any related issues that may provide additional context - (e.g., #123 for same-repo references or owner/repo-name#456 for cross-repo references) - placeholder: | - #123 - owner/repo-name#456 - validations: - required: false - - type: textarea id: additional_context attributes: label: Additional Context - description: >- - Add any other context about the problem here, such as configuration - details or environment specifics - placeholder: Add any other context here... validations: required: false diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 6b66c64..4b8522f 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -1,100 +1,17 @@ -# ============================================================================= -# ISSUE TEMPLATE CONFIG -# ============================================================================= -# For detailed design decisions and customization options, see: -# .github/TEMPLATE_DESIGN_DECISIONS.md > Issue Template Design Decisions > config.yml -# -# CUSTOMIZE: markers below indicate actionable customization points. -# Look for `# CUSTOMIZE:` comments throughout this file. -# -# IMPORTANT: contact_links URLs MUST be absolute URLs (relative paths do not work). -# You MUST replace OWNER/REPO with your actual org/repo or links will 404. -# - GHES users: Replace github.com with your GHES host (e.g., github.company.com) -# ============================================================================= +# GitHub issue template chooser configuration. +# Contact link URLs must be absolute GitHub URLs. -# CUSTOMIZE: Set to `true` (default) to allow users flexibility in creating issues -# without a template. Set to `false` to require template usage—useful once you have -# comprehensive templates covering all issue types and want structured data collection. -# Consider setting to false once you have comprehensive templates and want structured intake. blank_issues_enabled: true contact_links: - # CUSTOMIZE: Update the URL below by replacing `OWNER/REPO` with your org/repo name. - # NOTE: blob/HEAD is used instead of blob/main to support repos with non-main default branches. - - name: 📝 Contributing Guide - url: https://github.com/OWNER/REPO/blob/HEAD/CONTRIBUTING.md - about: Read the contributing guide before opening an issue + - name: Contributing Guide + url: https://github.com/franklesniak/macOSLab/blob/HEAD/CONTRIBUTING.md + about: Read the contributing guide before opening an issue. - # ============================================================================= - # SECURITY LINK CONFIGURATION - # ============================================================================= - # CUSTOMIZE: Update the URL below by replacing `OWNER/REPO` with your org/repo name. - # - # IMPORTANT: Private vulnerability reporting is ONLY available for PUBLIC repositories. - # If your repository is private: - # - This link will NOT provide a "Report a vulnerability" option to external users - # - You MUST provide an email contact in SECURITY.md for security reports - # - Consider removing or commenting out this link if it would confuse reporters - # See .github/TEMPLATE_DESIGN_DECISIONS.md for detailed guidance on private vs. public repos. - # - # Trade-off notes on Security link target options: - # - Option A: /security (used here) - Broad compatibility; general entry point that - # usually exists. May not lead directly to submission flow if not configured. - # - Option B: /security/advisories/new - Best UX if enabled; direct private reporting UI. - # Can 404 if advisories/private reporting aren't enabled (common in new repos). - # - Option C: blob/HEAD/SECURITY.md - Always works if the file exists; communicates policy - # but is not a submission endpoint. - # - # Recommended: Keep /security as the default. After enabling private vulnerability - # reporting in your repository (public repos only), you can optionally change to - # /security/advisories/new. - # - # TO ENABLE PRIVATE VULNERABILITY REPORTING (PUBLIC REPOSITORIES ONLY): - # 1. Go to Settings → Security → Private vulnerability reporting - # 2. Enable the feature - # 3. Full documentation: - # https://docs.github.com/en/code-security/how-tos/report-and-fix-vulnerabilities/configure-vulnerability-reporting/configuring-private-vulnerability-reporting-for-a-repository - # ============================================================================= - - name: 🔒 Security Vulnerabilities - url: https://github.com/OWNER/REPO/security - about: Report security issues privately (do not open a public issue). Private vulnerability reporting is only available for public repositories. + - name: Security Vulnerabilities + url: https://github.com/franklesniak/macOSLab/security/advisories/new + about: Report security issues privately. Do not open a public issue. - # ============================================================================= - # DISCUSSIONS LINK (OPTIONAL) - # ============================================================================= - # Trade-off notes on Discussions link: - # - Kept commented out by default because many downstream repos don't enable Discussions. - # - If you know most downstream repos will enable Discussions, you can uncomment by default. - # - Enabling this redirects support questions away from issues, which improves triage. - # - # HOW TO ENABLE DISCUSSIONS: - # 1. Go to your repository Settings > General > Features - # 2. Check the "Discussions" checkbox to enable the feature - # See: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/enabling-or-disabling-github-discussions-for-a-repository - # 3. Uncomment the block below - # 4. Replace `OWNER/REPO` with your actual organization and repository name - # 5. Test the link in the issue template chooser to verify it works - # - # CUSTOMIZE: Uncomment the following block if your repository uses GitHub - # Discussions for Q&A. - # ============================================================================= - # - name: 💬 Questions & Discussions - # url: https://github.com/OWNER/REPO/discussions - # about: Ask questions and discuss ideas (not for bug reports) - - # ============================================================================= - # SUPPORT / FAQ LINK (OPTIONAL) - # ============================================================================= - # Use this link if your repository does NOT enable GitHub Discussions but you - # want to redirect support questions away from issues. - # - # ACTION ITEM: If you don't enable Discussions (see above), consider: - # 1. Adding a "## Support" section to your README.md - # 2. Uncommenting and updating the URL below to point to that section - # - # The URL assumes your README has a #support anchor. Update the URL if your - # support section has a different anchor or is in a different location. - # ============================================================================= - # - name: ❓ Support / FAQ - # url: https://github.com/OWNER/REPO#support - # about: Common questions, FAQs, and support guidance + - name: Questions and Discussions + url: https://github.com/franklesniak/macOSLab/discussions + about: Ask questions and discuss macOSLab ideas that are not bug reports. diff --git a/.github/ISSUE_TEMPLATE/documentation_issue.yml b/.github/ISSUE_TEMPLATE/documentation_issue.yml index c060032..8c7d9fe 100644 --- a/.github/ISSUE_TEMPLATE/documentation_issue.yml +++ b/.github/ISSUE_TEMPLATE/documentation_issue.yml @@ -1,94 +1,43 @@ -# ============================================================================= -# DOCUMENTATION ISSUE TEMPLATE -# ============================================================================= -# For detailed design decisions and customization options, see: -# .github/TEMPLATE_DESIGN_DECISIONS.md > Issue Template Design Decisions -# -# CUSTOMIZE: markers below indicate actionable customization points. -# Look for `# CUSTOMIZE:` comments throughout this file. -# -# NOTE: The `documentation` label is a GitHub DEFAULT LABEL that exists in all -# new repositories. If your org has renamed/deleted it, update accordingly. -# ============================================================================= - -name: 📚 Documentation Issue -description: Report a typo, unclear instructions, broken link, or suggest documentation improvements -# CUSTOMIZE: Adjust the title prefix to match your project's conventions +name: Documentation Issue +description: Report unclear, missing, outdated, or broken documentation. title: "[Docs] " -# CUSTOMIZE: Update these labels to match your repository's label taxonomy. -# Ensure these labels exist in your repository or create them before using this template. labels: - documentation - # - triage # ACTION ITEM: Uncomment after creating the `triage` label in your repository + - triage body: - - type: markdown - attributes: - value: | - Thanks for helping improve the documentation! - - type: checkboxes id: pre_flight attributes: label: Pre-flight Checklist - description: Please confirm the following before submitting options: - - label: I have searched existing issues for this documentation problem + - label: I searched existing issues for this documentation problem. required: true - - label: This is about documentation content, not a functional code bug + - label: This is about documentation content, not functional behavior. required: true + validations: + required: true - type: textarea id: description attributes: label: What's wrong or missing? - description: Describe the documentation issue clearly and concisely. + description: Describe the documentation issue clearly. validations: required: true - # NOTE: Placeholder uses simple file paths rather than full URLs with OWNER/REPO - # to avoid reporters pasting literal placeholders. Relative paths work for most - # documentation issues. - # ACTION ITEM: After setting up your repository, you may optionally update this - # placeholder to include a literal URL example from your repository (e.g., - # "e.g., https://github.com/your-org/your-repo/blob/HEAD/README.md#usage or docs/guide.md"). - type: input id: location attributes: label: Where is it? - description: >- - Link to the file/section/page (URL or path) where the issue exists. - Providing a location greatly speeds up investigation—please include if possible. - placeholder: e.g., README.md#usage or docs/guide.md - validations: - required: false - - - type: textarea - id: suggested_change - attributes: - label: Suggested change (optional) - description: If you know what should change, propose wording/structure here. + description: Link to the file, section, or page where the issue exists. + placeholder: e.g., README.md#validation or docs/planning/macOS-imaging-08c-repo-spec-final.md validations: required: false - # CUSTOMIZE: Remove if your project doesn't maintain versioned documentation - - type: input - id: doc_version - attributes: - label: Documentation Version (optional) - description: >- - If the documentation is versioned, which version are you viewing? - placeholder: e.g., v1.2.3, latest, main branch - validations: - required: false - - # CUSTOMIZE: Adjust options to match your project's documentation structure. - # This field is optional by default. Downstream repos can make it required if - # they do structured documentation triage. - type: dropdown id: doc_issue_type attributes: - label: Issue Type (optional) - description: What type of documentation issue is this? + label: Issue Type options: - Typo / Grammar - Unclear / Confusing @@ -100,3 +49,10 @@ body: - Other validations: required: false + + - type: textarea + id: suggested_change + attributes: + label: Suggested Change + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml index db6c8bf..09b6f8e 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.yml +++ b/.github/ISSUE_TEMPLATE/feature_request.yml @@ -1,76 +1,27 @@ -# ============================================================================= -# FEATURE REQUEST TEMPLATE -# ============================================================================= -# For detailed design decisions and customization options, see: -# .github/TEMPLATE_DESIGN_DECISIONS.md > Issue Template Design Decisions -# -# CUSTOMIZE: markers below indicate actionable customization points. -# Look for `# CUSTOMIZE:` comments throughout this file. -# ============================================================================= - -name: 💡 Feature Request -description: Suggest a new feature or enhancement -# CUSTOMIZE: Adjust the title prefix to match your project's conventions +name: Feature Request +description: Suggest a macOSLab feature or enhancement. title: "[Feature] " -# CUSTOMIZE: Update these labels to match your repository's label taxonomy. -# Ensure these labels exist in your repository or create them before using this template. -# Common patterns include: -# - Type labels: enhancement, feature, improvement -# - Status labels: triage, accepted, planned -# - Priority labels: priority:critical, priority:high, priority:medium, priority:low -# - Area labels: area:api, area:cli, area:docs labels: - enhancement - # - triage # ACTION ITEM: Uncomment after creating the `triage` label in your repository -# CUSTOMIZE: Uncomment and update to specify an issue type (defined at the organization level). -# See: https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms#top-level-syntax -# type: Feature -# CUSTOMIZE: Uncomment and update to pre-populate the assignees field when opening feature requests -# assignees: -# - maintainer-username -# CUSTOMIZE: Uncomment and update to auto-add feature requests to a GitHub Project (uses project number) -# projects: -# - org/1 + - triage body: - - type: markdown - attributes: - value: | - ## Thank you for your feature suggestion! - Please fill out the form below to help us understand and evaluate your request. - - type: checkboxes id: pre_flight attributes: label: Pre-flight Checklist - description: >- - Please confirm the following before submitting options: - - label: I have searched existing issues and this feature has not been requested + - label: I searched existing issues and discussions for this request. required: true - # CUSTOMIZE: Consider changing to `required: true` for projects with - # comprehensive documentation that users should consult before requesting features - - label: I have read the project documentation - required: false - # NOTE: This checkbox is worded generically to be portable across projects. - # Many downstream repos won't have a formal roadmap, so "existing issues/discussions/README" - # covers common places where planned work might be documented. - - label: I have checked existing issues, discussions, or README for planned work - required: false - # CUSTOMIZE: Remove if your project doesn't accept community contributions - - label: I am willing to submit a pull request to implement this feature + - label: I checked the current planning docs or TODO files for related deferred work. required: false + validations: + required: true - type: textarea id: problem_statement attributes: label: Problem Statement / Motivation - description: >- - Describe the problem you're trying to solve or the motivation for this feature. - A clear problem statement helps maintainers understand the need - placeholder: | - I'm frustrated when... - Currently, I have to... - This would help by... + description: Describe the problem this feature would solve. validations: required: true @@ -78,113 +29,45 @@ body: id: proposed_solution attributes: label: Proposed Solution - description: >- - Describe the solution you'd like to see implemented. - Be as specific as possible about the desired behavior - placeholder: | - I would like... - The feature should... - Expected behavior: + description: Describe the behavior you want. validations: required: true - - type: textarea - id: alternatives_considered - attributes: - label: Alternatives Considered - description: >- - Describe any alternative solutions or workarounds you've considered. - This helps maintainers understand the problem space - placeholder: | - I considered... - Current workaround: - Why alternatives don't work: - validations: - required: false - - - type: textarea - id: use_case - attributes: - label: Use Case / User Story - description: >- - Describe who benefits from this feature and how. - Use the format "As a [type of user], I want [goal] so that [benefit]" - placeholder: | - As a [developer/user/admin], I want [feature] - so that [benefit/outcome]. - - Example scenarios: - - Scenario 1: ... - - Scenario 2: ... - validations: - required: false - - - type: markdown - attributes: - value: "### Feature Classification" - - # Trade-off notes on area field being optional: - # - Optional (used here): Reduces friction for feature requesters. - # - Required: Improves categorization for repos that rely on area-based routing/automation. - # Recommended: Keep optional by default. Downstream repos can make it required if they - # use area-based routing or have automation that depends on this field. - # NOTE: Options match the bug report template for consistency. - type: dropdown id: area attributes: label: Area - description: What part of the project would this feature impact? - # CUSTOMIZE: Update these options to match your project's languages/components (remove unused entries). options: - - Python - - PowerShell + - PowerShell module + - PowerShell scripts + - Pester tests - Markdown / Documentation - GitHub Actions / CI - - Cross-language / Integration - - Cross-cutting / Repo-wide - - Other (describe/specify in Additional Context) + - JSON / YAML / schema validation + - Parallels provider + - UTM provider + - Tart stub / advanced path + - Evidence / redaction + - Other validations: required: false - # CUSTOMIZE: Adjust priority options to match your project's triage workflow - type: dropdown id: priority attributes: label: Priority - description: >- - Select the priority level that best describes the urgency from your perspective. - Note: This is your self-assessment; maintainers may adjust during triage options: - - Critical (blocking my adoption/usage) - - High (significant impact on my workflow) - - Medium (would improve my experience) - - Low (nice to have) + - Critical + - High + - Medium + - Low validations: required: false - # CUSTOMIZE: Adjust scope options to match your project's feature categories - - type: dropdown - id: scope - attributes: - label: Scope - description: >- - Select the scope that best describes the size of this feature - options: - - Major feature (new capability, significant change) - - Minor enhancement (improvement to existing feature) - - Quality of life (small improvement, polish) - validations: - required: false - - - type: markdown - attributes: - value: "### Additional Information" - - type: textarea id: acceptance_criteria attributes: - label: Acceptance Criteria (optional) - description: What needs to be true for this feature to be considered complete? + label: Acceptance Criteria placeholder: | - [ ] ... - [ ] ... @@ -192,29 +75,15 @@ body: required: false - type: textarea - id: additional_context + id: alternatives_considered attributes: - label: Additional Context - description: >- - Add any other context, mockups, screenshots, or examples about the feature request. - Drag and drop images directly into this field - placeholder: | - Additional context... - - For mockups or detailed designs, you can include images: - (drag and drop images here) + label: Alternatives Considered validations: required: false - type: textarea - id: related_issues + id: additional_context attributes: - label: Related Issues - description: >- - Link any related issues that may provide additional context - (e.g., #123 for same-repo references or owner/repo-name#456 for cross-repo references) - placeholder: | - #123 - owner/repo-name#456 + label: Additional Context validations: required: false diff --git a/.github/TEMPLATE_DESIGN_DECISIONS.md b/.github/TEMPLATE_DESIGN_DECISIONS.md index d7c2f1e..fc3d801 100644 --- a/.github/TEMPLATE_DESIGN_DECISIONS.md +++ b/.github/TEMPLATE_DESIGN_DECISIONS.md @@ -1,1451 +1,34 @@ -# Template Design Decisions + +# macOSLab Design Decisions -This document records design decisions made during the creation and maintenance of `franklesniak/copilot-repo-template`. It serves as institutional memory to prevent re-litigation of settled decisions during code review. +## Metadata -> **For consumers of this template:** This document is NOT an instruction guide. See [GETTING_STARTED_NEW_REPO.md](../GETTING_STARTED_NEW_REPO.md) or [GETTING_STARTED_EXISTING_REPO.md](../GETTING_STARTED_EXISTING_REPO.md) for setup instructions, and [OPTIONAL_CONFIGURATIONS.md](../OPTIONAL_CONFIGURATIONS.md) for customization options. +- **Status:** Phase 0 bootstrap +- **Owner:** Repository Maintainers +- **Last Updated:** 2026-05-05 +- **Scope:** Repository-local design decisions for the `franklesniak/macOSLab` bootstrap state. The planning ADRs remain the source of truth for product architecture decisions. +- **Related:** [macOSLab ADRs](../docs/planning/macOS-imaging-08e-ADRs.md), [macOSLab Repository Specification](../docs/planning/macOS-imaging-08c-repo-spec-final.md), [Phase 0 bootstrap instructions](../docs/phase-0-bootstrap-codex-instructions.md) ---- +This file replaces the inherited template design-decision record with macOSLab-specific bootstrap decisions. Protected instruction files still contain inherited links and language until the repository owner explicitly authorizes changes to those protected files. -## Table of Contents +## Phase 0 Bootstrap Decisions -- [File Placement](#file-placement) -- [Pull Request Template](#pull-request-template) -- [Pre-commit and Git Hooks](#pre-commit-and-git-hooks) -- [Coding Standards and Instructions](#coding-standards-and-instructions) - - [Current Provider Versions in Terraform Examples](#design-decision-current-provider-versions-in-terraform-examples) - - [Terraform Instructions Document Length Strategy](#design-decision-terraform-instructions-document-length-strategy) - - [Instruction Files Scope (Code Authoring, Not CI/CD)](#design-decision-instruction-files-scope-code-authoring-not-cicd) - - [Terraform Registry Reference URLs Use /latest/](#design-decision-terraform-registry-reference-urls-use-latest) -- [Agent Instruction Files](#agent-instruction-files) - - [Multi-Agent Instruction Files at Repository Root](#design-decision-multi-agent-instruction-files-at-repository-root) - - [Agent Files as Minimal Entry-Point Summaries (Not Canonical)](#design-decision-agent-files-as-minimal-entry-point-summaries-not-canonical) -- [Data File Standards (JSON/YAML)](#data-file-standards-jsonyaml) - - [Dedicated JSON and YAML Instruction Files](#design-decision-dedicated-json-and-yaml-instruction-files) - - [Baseline JSON/YAML Linting Stack](#design-decision-baseline-jsonyaml-linting-stack) - - [yamllint truthy.check-keys Default](#design-decision-yamllint-truthycheck-keys-default) - - [yamllint line-length Warning Level Default](#design-decision-yamllint-line-length-warning-level-default) - - [Prettier Deferral for Data Files](#design-decision-prettier-deferral-for-data-files) - - [Schema Location at Repository Root](#design-decision-schema-location-at-repository-root) - - [Schema Validation Tiers](#design-decision-schema-validation-tiers) - - [Built-in Schema Validation for Real Load-Bearing Configuration Files](#design-decision-built-in-schema-validation-for-real-load-bearing-configuration-files) - - [JSON5 Exclusion by Default](#design-decision-json5-exclusion-by-default) - - [`additionalProperties` Policy](#design-decision-additionalproperties-policy) - - [Testing Beyond Linting for JSON/YAML](#design-decision-testing-beyond-linting-for-jsonyaml) - - [.gitattributes JSON/YAML LF Pinning](#design-decision-gitattributes-jsonyaml-lf-pinning) -- [Node.js Package Configuration](#nodejs-package-configuration) -- [CI Workflow Configuration](#ci-workflow-configuration) -- [Python Configuration](#python-configuration) -- [Security and Vulnerability Reporting](#security-and-vulnerability-reporting) -- [License Configuration](#license-configuration) -- [Dependabot Configuration](#dependabot-configuration) - - [Workflow Version Pinning and Dependabot Coherence](#design-decision-workflow-version-pinning-and-dependabot-coherence) - - [Dependabot Enabled by Default](#design-decision-dependabot-enabled-by-default) -- [CODEOWNERS Configuration](#codeowners-configuration) -- [Issue Template Design Decisions](#issue-template-design-decisions) -- [Branch Ruleset Setup](#branch-ruleset-setup) +### PowerShell and Markdown Project Shape ---- +macOSLab is a PowerShell 7.4+ and Markdown repository. Python project source, Python package metadata, sample pytest tests, and HCL sample artifacts were removed during Phase 0. Python remains available only as development tooling where pre-commit hooks require it. -## File Placement +### Pre-commit Remains the Aggregate Hygiene Gate -### Design Decision: TEMPLATE_DESIGN_DECISIONS.md Location +The repository keeps pre-commit for trailing-whitespace, end-of-file, JSON syntax, YAML syntax/style, GitHub Actions linting, schema validation, large-file checks, and Markdown linting. Black, Ruff, and HCL hooks were removed because they do not match the repository language set. -This file is placed in `.github/` because it is GitHub-specific configuration guidance that relates directly to other files in this directory (pull_request_template.md, ISSUE_TEMPLATE/, workflows/, etc.). Keeping it here makes it discoverable alongside the files it documents and ensures template maintainers encounter it when exploring GitHub configuration. This is preferable to `docs/` (general project documentation) or the repo root (which should remain clean for end users). +### Worked Example Schema Temporarily Retained ---- +The worked-example schema remains in `schemas/` during Phase 0 so `check-jsonschema` and `check-metaschema` are still exercised. Phase 7 must replace it with the real evidence-bundle schema from the repository specification. -## Pull Request Template +### Private Vulnerability Reporting -### Design Decision: Contributing Guidelines Link +Security reporting points to GitHub private vulnerability reporting at `https://github.com/franklesniak/macOSLab/security/advisories/new`. Phase 0 does not add the optional project-specific `SECURITY.md` paragraph from ADR-0009; that remains deferred for owner review. -> **Status: Superseded** (2026-05-03). This decision has been replaced by the **Issue and PR templates** carve-out in [`.github/instructions/docs.instructions.md`](instructions/docs.instructions.md), which requires absolute `https://github.com/OWNER/REPO/...` URLs in `.github/ISSUE_TEMPLATE/*.yml` and `.github/pull_request_template.md` — `https://github.com/OWNER/REPO/blob/HEAD/` for repo-internal **file** targets and `https://github.com/OWNER/REPO/` for non-file targets such as `/security`, `/discussions`, or `/issues`. The original rationale (zero-friction adoption via relative links) was weakened once `.github/workflows/check-placeholders.yml` began enforcing `OWNER/REPO` replacement automatically, and the relative-link pattern is fragile across non-GitHub.com renderers, GitHub Mobile, and email notifications. The historical content is preserved below for context. +### Protected Instruction Files -The PR template uses a relative link for contributing guidelines: - -```md -[contributing guidelines](../blob/HEAD/CONTRIBUTING.md) -``` - -This relative link has been tested and confirmed to work correctly in rendered PR views on GitHub.com. It resolves to the repository's CONTRIBUTING.md file regardless of the default branch name (main, master, develop, etc.) due to the use of HEAD. - -**Why this approach:** - -1. **Clone works with minimal setup**: Template users don't need to find-and-replace OWNER/REPO placeholders—the link works immediately after cloning. - -2. **Reduces forgotten placeholder risk**: Absolute URLs with placeholders can lead to broken links if users forget to replace them. The relative link pattern eliminates this failure mode. - -3. **Tested and verified**: This link pattern is confirmed to work in GitHub.com PR views. - -**Trade-offs:** - -- The relative link may not resolve correctly in PR preview/draft mode before the branch is pushed, or in non-GitHub contexts (local Markdown preview, email notifications, etc.). -- GitHub Enterprise Server (GHES) compatibility varies by version. - -If you need the link to work in PR drafts, GHES, or external contexts, replace with: -`https://github.com///blob/HEAD/CONTRIBUTING.md` (remembering to replace `/` with your actual org/repo name). - -### Design Decision: Link Strategy in PR Template - -> **Status: Superseded** (2026-05-03). This decision has been replaced by the **Issue and PR templates** carve-out in [`.github/instructions/docs.instructions.md`](instructions/docs.instructions.md). The current rule is that Markdown links to repo-internal files inside `.github/ISSUE_TEMPLATE/*.yml` and `.github/pull_request_template.md` MUST use absolute `https://github.com/OWNER/REPO/blob/HEAD/` URLs; relative forms such as `../blob/HEAD/` and `blob/HEAD/` MUST NOT be used in those files. The historical content is preserved below for context. - -The PR template uses relative links for repository files (e.g., `../blob/HEAD/CONTRIBUTING.md`) rather than absolute URLs with `OWNER/REPO` placeholders. - -**Rationale:** - -- Template works immediately upon cloning (no placeholder replacement needed) -- Reduces forgotten placeholder risk (common failure mode) -- Proven to work for primary use case (GitHub.com PR body) -- Absolute links remain available as documented opt-in for GHES/email notifications - -**Alternative considered:** Use absolute `OWNER/REPO` placeholders as default - -**Rejected because:** Requires find-and-replace for all adopters, even when relative links work for their use case. Template portability prioritizes zero-friction adoption. - -### Design Decision: Type of Change Options - -The PR template includes "Dependencies update" as a standard change type. - -**Rationale:** - -- Dependency management is near-universal (npm, pip, cargo, Maven, etc.) -- Common workflow with automation tools (Dependabot, Renovate) -- Often requires different review standards than feature work -- Low cost, high applicability - -**Standard options:** - -- Bug fix -- New feature -- Breaking change -- Documentation update -- Dependencies update -- Configuration/tooling change - -### Design Decision: Checklist Item Links - -Checklist items that reference files/directories use inline code formatting (e.g., `.github/instructions/`) rather than hyperlinks. - -**Rationale:** - -- Checklists are reference documentation, not primary navigation -- Adding links to every path creates visual clutter -- Path references are unambiguous without links -- Maintains consistency across checklist items - -**Alternative considered:** Make all file/directory references clickable - -**Rejected because:** Minimal value for added noise. Contributors can navigate to commonly-referenced directories without hyperlinks. - ---- - -## Pre-commit and Git Hooks - -### Design Decision: Conditional Pre-commit Section - -The pre-commit section uses conditional language ("if this repository uses pre-commit") to maintain template portability. This is intentional: - -1. **Not all downstream repos use pre-commit**: Many projects use different linting/formatting approaches (IDE settings, CI-only checks, language-specific tools). - -2. **Reduces friction**: Contributors to repos without pre-commit won't be confused by irrelevant instructions. - -3. **Self-documenting**: The conditional phrasing makes it clear when the section applies. - -**Recommendation for repos using pre-commit:** - -If your repository uses pre-commit hooks, replace the conditional section with the more direct version for clearer contributor guidance. - -### Design Decision: Pre-commit as Sole Git Hook Manager - -This template uses pre-commit as the sole git hook manager. All hooks are configured in `.pre-commit-config.yaml`. - -**Why pre-commit only:** - -- **Single tool**: Unified configuration in one file -- **No conflicts**: Uses standard `.git/hooks/` location, no `core.hooksPath` issues -- **Python standard**: Pre-commit is the de facto standard in Python projects -- **Multi-language**: Also supports Markdown, YAML, JSON, and other file types -- **Isolated environments**: Manages its own tool installations per hook - -**For projects preferring Husky:** - -If you prefer Husky for git hooks: - -1. Remove `.pre-commit-config.yaml` -2. Run `npm install husky --save-dev` -3. Add `"prepare": "husky"` to `package.json` scripts -4. Create `.husky/pre-commit` with your hook commands -5. Do NOT run `pre-commit install` (the two tools conflict) - ---- - -## Coding Standards and Instructions - -### Design Decision: .github/instructions/ Reference - -The PR template reference to `.github/instructions/` assumes the directory structure will remain as provided in this template. The assumption is that downstream repos will: - -1. Keep the directory structure but ADD/REMOVE instruction files as appropriate for their project's languages/frameworks. - -2. NOT reorganize the directory to a different location. - -This allows the generic reference to work across all downstream repos without requiring customization. If you need to reorganize this directory, update this reference in the PR template accordingly. - -### Design Decision: Python Version Policy Reference Pattern - -CONTRIBUTING.md uses policy-based language ("Python version currently receiving bugfixes") rather than hardcoded version numbers throughout the document for consistency and maintainability. - -**Rationale:** - -1. **Reduces maintenance burden**: Version numbers don't need updates when Python releases new versions—the policy link is the single source of truth. - -2. **Consistency**: Aligning references with the established Python Version Requirements section prevents contradictory guidance. - -3. **Clear reference**: The anchor link (#python-version-requirements) helps contributors find the authoritative policy statement. - -**Trade-offs:** - -- Slightly more verbose than "Python 3.13+", but eliminates drift risk between sections. -- Template adopters who want specific version requirements can still customize the Python Version Requirements section as instructed. - -### Design Decision: Realistic Examples in Terraform Instructions (No REPLACE_ME_* Placeholders) - -The `.github/instructions/terraform.instructions.md` file uses realistic example values (e.g., `acme-corp-terraform-state`, `us-east-1`) instead of `REPLACE_ME_*` placeholder patterns. - -**Rationale:** - -1. **No adoption burden**: Consumers don't need to find-replace `REPLACE_ME_*` patterns throughout examples before the documentation is useful. - -2. **Cleaner examples**: Code looks like real Terraform, improving readability and comprehension for both humans and LLMs. - -3. **Consistency with other instructions**: PowerShell and Python instruction files in this repository use realistic examples without placeholders—Terraform instructions now follow the same pattern. - -4. **Reduced document length**: No need for a lengthy "Placeholder Convention" section with placeholder tables and usage rules. - -5. **Better LLM comprehension**: LLMs are trained on realistic code patterns, not `REPLACE_ME_*` conventions, making realistic examples more effective for AI-assisted coding. - -**What ensures examples aren't mistaken for prescriptive values:** - -- **"About Examples in This Document" section**: A clear statement at the beginning of the document explains that all code examples are illustrative. - -- **Inline comments on key examples**: Backend configuration blocks include comments like `# Use your state bucket name` to reinforce that values require customization. - -- **Self-documenting names**: Example names like `acme-corp-terraform-state` are obviously placeholder-like through their fictional organization prefix. - -**Alternative considered:** Use `REPLACE_ME_*` placeholder markers for all values requiring customization. - -**Rejected because:** - -- Adds friction for consumers who must do find-replace before examples are useful -- Looks unusual compared to typical Terraform documentation in the ecosystem -- Creates inconsistency with how PowerShell and Python instructions handle examples -- The only scenario where `REPLACE_ME_*` provides unique value is when someone might literally copy-paste a backend block into production without reading—but that's a user error, not a documentation problem, and the same risk exists with any example code in any documentation -- A clear "About Examples" note addresses the "examples are illustrative" concern sufficiently - -### Design Decision: Current Provider Versions in Terraform Examples - -The `.github/instructions/terraform.instructions.md` file uses the newest stable major versions in all provider version constraint examples. - -**Current versions as of 2026-02-04:** - -| Provider | Example Constraint | Current Stable | -| --- | --- | --- | -| AWS | `~> 6.0` | 6.31.0 | -| Azure | `~> 4.0` | 4.58.0 | -| GCP | `~> 7.0` | 7.18.0 | - -**Rationale:** - -1. **Best practice demonstration**: Examples should show current recommended practices, not outdated patterns. - -2. **Reduces adopter confusion**: Using current versions prevents questions like "why does the template use AWS provider 5.x when 6.x is current?" - -3. **Forward-compatible constraints**: The pessimistic constraint operator (`~>`) allows minor/patch updates within the major version, so examples remain valid until the next major version release. - -**Trade-offs:** - -- Requires periodic review to update when new major versions become the recommended stable release -- Examples may reference features not available in older provider versions (mitigated by the pessimistic constraint allowing updates) - -**When to update:** - -- When a new major version becomes the recommended stable release (not just released, but recommended for production use) -- When significant new features change best practices (e.g., a new authentication pattern becomes standard) -- As part of quarterly maintenance reviews - -### Design Decision: Terraform Instructions Document Length Strategy - -The `.github/instructions/terraform.instructions.md` file is intentionally comprehensive (~195KB) rather than split into multiple smaller files. - -**Rationale:** - -1. **Single source of truth**: All Terraform coding standards in one location eliminates questions about which file to consult. - -2. **LLM optimization**: The file is designed to be consumed by GitHub Copilot and similar LLMs. A single comprehensive file provides complete context without requiring the LLM to navigate between files or potentially miss relevant guidance. - -3. **Consistency with Terraform ecosystem**: HashiCorp's own Terraform documentation and style guides tend toward comprehensive single documents rather than fragmented collections. - -**Discoverability mitigations already in place:** - -- **Quick Reference Checklist**: A complete checklist near the top provides a scannable summary of all requirements -- **Scope tags**: Each checklist item is tagged with `[All]`, `[Module]`, `[Root]`, or `[Test]` for quick filtering -- **Comprehensive Table of Contents**: Full ToC with anchor links for navigation -- **Consistent heading structure**: Three-level hierarchy (section → topic → detail) for predictable navigation - -**Alternative considered:** Split into multiple files (e.g., `terraform-formatting.md`, `terraform-modules.md`, `terraform-testing.md`) - -**Rejected because:** - -- Creates navigation burden for both humans and LLMs -- Increases maintenance burden (cross-file consistency, duplicate content, broken links) -- The Quick Reference Checklist already provides the "summary view" that splitting would attempt to create -- No evidence that document length impairs usability given existing navigation aids - -### Design Decision: Instruction Files Scope (Code Authoring, Not CI/CD) - -Instruction files in `.github/instructions/` are scoped to **code authoring standards**, not CI/CD pipeline design or deployment workflows. - -**Rationale:** - -1. **Clear purpose**: Instruction files are for GitHub Copilot and developers writing code, not for DevOps engineers designing pipelines. - -2. **Separation of concerns**: CI/CD configuration lives in `.github/workflows/` where it can be versioned, tested, and maintained independently of coding standards. - -3. **Consistency across languages**: All instruction files (Python, PowerShell, Terraform, Markdown) follow this scope boundary, making the pattern predictable. - -4. **LLM context optimization**: Keeping instruction files focused on code authoring prevents context pollution with operational details that would distract from the primary task of writing code. - -**What instruction files cover:** - -- Syntax, formatting, and style rules -- Naming conventions -- File organization and structure -- Testing patterns (unit tests, integration tests, test file conventions) -- Security patterns in code (input validation, secret handling in code) -- Documentation standards (docstrings, comments, README patterns) - -**What instruction files do NOT cover:** - -- CI/CD pipeline design (workflow triggers, job dependencies, runner selection) -- Deployment workflows (approval gates, environment promotion, rollback procedures) -- Drift detection and remediation procedures -- Operational runbooks - -**Where CI/CD guidance lives:** - -- `.github/workflows/` — Workflow files with inline comments explaining design choices -- `.github/TEMPLATE_DESIGN_DECISIONS.md` — The "CI Workflow Configuration" section documents CI/CD design decisions -- `README.md` — High-level overview of available workflows and their purposes - -### Design Decision: Terraform Registry Reference URLs Use /latest/ - -Terraform Registry reference URLs that appear in **comments** in `.tf`, `.tftest.hcl`, `.tfvars`, `.tfbackend`, `.tftpl`, and other Terraform-related files **MUST** use the `latest` path segment instead of a pinned provider or module version. The same `/latest/` requirement applies to any Terraform Registry navigation links in instructional Markdown under this template (currently those in [`TEMPLATE_MAINTENANCE.md`](../TEMPLATE_MAINTENANCE.md); the same convention would apply to any future Registry links in other Markdown documentation under this template, including under `docs/`). The Markdown half of the rule is binding via this ADR rather than via an `applyTo`-scoped instruction file: the canonical instruction file [`terraform.instructions.md`](instructions/terraform.instructions.md) is scoped to Terraform extensions only, and the convention has not yet been mirrored into `docs.instructions.md` (which governs `**/*.md`). Because the repository's auto-apply mechanism is limited to `.github/instructions/*.instructions.md` files with `applyTo` front matter, the rule is *not* auto-surfaced to an agent editing a pure Markdown file; until the convention is codified into a Markdown-scoped instruction file, a maintainer or task description has to point at this ADR for the agent to apply it. - -The shape of the rule is: - -- Provider URLs use `latest` immediately after `/providers///`, with any trailing path, query string, or fragment, or none — for example, `https://registry.terraform.io/providers/hashicorp/aws/latest` or `.../latest/docs/resources/...`. -- Module URLs use `latest` immediately after `/modules////`, with any trailing path, query string, or fragment, or none — for example, `https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws/latest` or `.../latest/submodules/...`. -- Pinned examples such as `/azurerm/4.67.0/` or `/random/3.8.1/` **MUST NOT** appear in either Terraform-file documentation comments or instructional Markdown under this template, except as clearly labeled non-compliant examples that exist to document this rule. - -For Terraform-file comments, the full normative rule (with compliant and non-compliant examples) lives in [`.github/instructions/terraform.instructions.md`](instructions/terraform.instructions.md) under "Terraform Registry Documentation URLs"; that file is loaded automatically when an agent edits a Terraform file. For instructional Markdown, this ADR is the binding statement until the convention is mirrored into `docs.instructions.md` or a separate `applyTo`-scoped instruction file. This ADR records *why* the template adopts the rule and how it spans Terraform comments and Markdown today; the Terraform instruction file remains the single source of truth for *what* the rule says inside Terraform files. - -**Rationale:** - -1. **Registry URLs in comments are navigation aids, not version pins.** The provider and module versions that actually drive `terraform init`, `terraform plan`, and `terraform apply` are governed by `required_providers` blocks in `terraform.tf` / `versions.tf` and by `.terraform.lock.hcl` for *providers*, and by module `source` and `version` arguments for *modules* (`.terraform.lock.hcl` does not record module versions). Those files are the authoritative version sources. Comment URLs only help a reader (or an AI assistant) jump to relevant Registry documentation while reading the code. - -2. **Version updates do not rewrite arbitrary comment text.** When provider versions advance, maintainers (or, when configured, dependency automation such as Dependabot or Renovate) update `required_providers` constraints and `.terraform.lock.hcl`; when module versions advance, maintainers update the module `version` argument. None of these update paths parses Markdown prose or arbitrary `.tf` comments to find embedded Registry URLs, so a pinned URL such as `/providers////docs/...` stays at `` in the comment even after the underlying constraint, lockfile entry, or `version` argument has moved on. (At the time this ADR was recorded, this template's [`.github/dependabot.yml`](dependabot.yml) does not configure Terraform-ecosystem updates, so the entire update path here is currently manual; the rationale still applies if Terraform automation is added later.) - -3. **Pinned URLs in comments drift silently.** Once the executable configuration moves past the version embedded in a comment URL, the comment points at outdated documentation. Readers can be sent to deprecated argument descriptions, removed resources, or release-note pages that no longer reflect the configuration's actual behavior. Because nothing in CI mechanically reconciles comment URLs against the underlying version sources (`.terraform.lock.hcl` for providers, module `version` arguments for modules), the drift is invisible until a human notices it. - -4. **`latest` matches the navigation-aid intent.** Linking to `/providers///latest/docs/...` (or `/modules////latest` for modules) always resolves to the current Registry documentation, which is what a reader navigating from a comment generally wants. If a reader needs documentation that matches a specific pinned version, they can look up the version in `.terraform.lock.hcl` (for providers) or in the module `source` / `version` arguments (for modules) and adjust the URL by hand; comment URLs do not need to duplicate that capability. - -**Trade-offs:** - -- Readers who want documentation pinned to the exact version recorded in `.terraform.lock.hcl` (for providers) or in module `source` / `version` arguments (for modules) need to take an extra step (look up the lockfile or module block entry and adjust the URL by hand). The alternative — pinning every comment URL — was rejected because the silent-drift cost outweighs the convenience for that minority case. -- Clearly labeled non-compliant examples in the Terraform instruction file intentionally include pinned URLs so the rule itself can be illustrated. Tooling that enforces this rule in the future must distinguish those documentation examples from real violations. - -**Scope boundary:** - -This ADR governs only documentation-comment URLs and Markdown navigation links to the Terraform Registry. It does **not** change: - -- Provider version constraints in `required_providers` blocks. -- Module `source` addresses or `version` arguments. -- Entries in `.terraform.lock.hcl`. -- Any other intentionally pinned executable configuration. - -Those remain governed by their existing rules and by the relevant maintenance guidance in [`TEMPLATE_MAINTENANCE.md`](../TEMPLATE_MAINTENANCE.md). - -**Enforcement:** - -This template intentionally does not ship a pre-commit hook or custom lint rule for this convention at the time the decision was recorded. Adding mechanical enforcement would require carefully distinguishing real violations from the clearly labeled non-compliant examples in the Terraform instruction file (and any future similar examples in Markdown), and would need to handle both provider and module URL shapes. If enforcement is added later, it will be designed in a separate, dedicated change rather than rolled into the documentation/design-decision update that introduced this ADR. - ---- - -## Agent Instruction Files - -### Design Decision: Multi-Agent Instruction Files at Repository Root - -The template includes three agent-specific instruction files at the repository root: `CLAUDE.md` (for Claude Code), `AGENTS.md` (for OpenAI Codex CLI), and `GEMINI.md` (for Gemini Code Assist). - -**Rationale:** - -1. **Multi-agent coverage**: AI coding agents other than GitHub Copilot (Claude Code, OpenAI Codex CLI, Gemini Code Assist) use their own convention files and do not read `.github/copilot-instructions.md` or `.github/instructions/*.instructions.md`. - -2. **Enriches GitHub Copilot**: GitHub Copilot's coding agent reads `CLAUDE.md`, `AGENTS.md`, and `GEMINI.md` as supplemental agent instructions, so adding these files enriches Copilot as well. - -3. **Template mission alignment**: The template's mission is to provide coding standards for AI-assisted development—limiting to a single AI platform contradicts this mission. - -4. **Minimal entry-point summaries**: Agent files keep only a minimal inline summary of the highest-priority shared rules from `.github/copilot-instructions.md`, rather than restating the full canonical document. This avoids multiple sources of truth while preserving reliable first-read guidance. - -**Trade-offs:** - -- Pro: All major coding agents receive project-specific guidance -- Pro: GitHub Copilot coding agent receives enriched context from additional files -- Pro: Template adopters can delete files for platforms they don't use -- Con: Three additional files at the repository root -- Con: Manual alignment burden when high-priority shared guidance changes in `.github/copilot-instructions.md` -- Con: No CI enforcement for alignment between the canonical file and the agent entry points - -**Alternatives considered:** - -1. **Single `AGENTS.md` only:** Rejected because Claude Code reads only `CLAUDE.md` and Gemini reads only `GEMINI.md`—a single file does not cover all agents. - -2. **Symlinks from agent files to `.github/copilot-instructions.md`:** Rejected because agent files need a small amount of inline guidance for reliability, and symlinks may not work correctly on all platforms or in all agent runtimes. - -3. **No agent files (Copilot-only):** Rejected because it contradicts the template's mission of supporting AI-assisted development broadly. - -**Recommendation:** Keep all three files unless your project exclusively uses one AI coding agent. Delete files for platforms you do not use. When modifying high-priority shared guidance in `.github/copilot-instructions.md`, update the remaining agent files so their minimal summaries stay accurate. - -### Design Decision: Agent Files as Minimal Entry-Point Summaries (Not Canonical) - -Agent instruction files (`CLAUDE.md`, `AGENTS.md`, `GEMINI.md`) are minimal entry-point summaries of `.github/copilot-instructions.md`, not independent canonical sources. - -**Rationale:** - -1. **Single source of truth**: `.github/copilot-instructions.md` is the established single source of truth for repository coding standards. - -2. **Avoids divergence**: Making agent files canonical would create multiple sources of truth that could diverge. - -3. **Agent discoverability**: Agent files contain a minimal inline summary of key rules (safety, pre-commit, validation commands, language-instruction pointers) because some agents may only auto-read their convention file and may not reliably follow references to other files without explicit instruction. - -4. **DRY vs. discoverability trade-off**: The trade-off between DRY (Don't Repeat Yourself) and agent discoverability favors a small, intentional amount of duplication to ensure agents actually receive the rules. - -**Trade-offs:** - -- Pro: Single source of truth remains `.github/copilot-instructions.md` -- Pro: Agents that don't follow file references still receive critical rules -- Con: Rule changes require updating multiple files -- Con: No automated enforcement that alignment reviews happen after shared-rule changes - ---- - -## Data File Standards (JSON/YAML) - -These ten ADRs record the design decisions behind the JSON and YAML authoring guides, the baseline linting stack for data files, the schema-validation policy, and the surrounding tooling and portability trade-offs. They are policy-only prose; the implementations live in `.github/instructions/json.instructions.md`, `.github/instructions/yaml.instructions.md`, `.pre-commit-config.yaml`, `.yamllint.yml`, `.gitattributes`, and (for downstream adopters) the optional `schemas/` directory. - -### Design Decision: Dedicated JSON and YAML Instruction Files - -The template ships dedicated instruction files for JSON (`.github/instructions/json.instructions.md`) and YAML (`.github/instructions/yaml.instructions.md`) alongside the existing language instruction files for Python, PowerShell, Terraform, and Markdown. - -**Rationale:** - -1. **Data files are potentially load-bearing contracts.** JSON and YAML are commonly used as configuration files, lockfiles, fixtures, schemas, CI/CD definitions, and policy documents. Even when a project does not currently rely on a given JSON or YAML file as a contract, downstream consumers (CI, release tooling, deployment automation, schema validators) frequently turn data files into de facto contracts over time. Treating them as potentially load-bearing by default is safer than retrofitting rigor after a silent format break. - -2. **Authoring rules differ from prose rules.** Markdown, Python, PowerShell, and Terraform guides cover human-authored or executable content. JSON and YAML guides cover structured data where ordering, key naming, comment policy, schema discipline, and quoting rules matter independently of any host language. - -3. **Per-format guidance is non-trivial.** JSON forbids comments and trailing commas while JSONC permits both; YAML has a substantially larger surface area (anchors, tags, the YAML 1.1/1.2 truthy delta, multi-document streams). A single combined guide would either bloat or paper over these differences. - -4. **Consistency with the existing modular pattern.** The repo-wide constitution already mandates a modular instruction-file layout under `.github/instructions/`. Adding JSON and YAML files extends — rather than reshapes — the established pattern. - -**Trade-offs:** - -- Pro: Authors and AI agents get format-specific rules without scanning unrelated guides. -- Pro: Downstream repos can delete either file independently if they truly do not use that format. -- Con: Two more files to keep aligned when shared rules (e.g., schema policy) change. - -**Alternatives considered:** - -- **Combined `data.instructions.md`:** Rejected. JSON and YAML differ enough on comments, quoting, and truthy semantics that combining them obscured per-format rules in early drafts. -- **No dedicated guides; inline rules in language guides:** Rejected. Most JSON/YAML in this template is not co-located with a single host language (workflows, lockfiles, instruction front matter), so language-specific guides cannot carry the rules. - -### Design Decision: Baseline JSON/YAML Linting Stack - -The default pre-commit stack for JSON and YAML uses `check-json`, `check-yaml`, `yamllint`, `actionlint`, and — as of the worked-example schema rollout — `check-jsonschema` plus `check-metaschema` scoped to the worked-example schema (`schemas/example-config.schema.json`). - -**Selected hooks and their roles:** - -1. **`check-json` (pre-commit-hooks)** — Validates strict JSON syntax. Fast, dependency-free, fails on duplicate keys, trailing commas, and other strict-JSON violations. -2. **`check-yaml` (pre-commit-hooks)** — Fast YAML parse check. Catches structural errors before slower linters run. -3. **`yamllint`** — YAML style enforcement (indentation, line length, truthy values). Configured via `.yamllint.yml`. -4. **`actionlint`** — GitHub Actions workflow validation, including expression syntax, runner labels, and `shellcheck` integration over `run:` blocks. -5. **`check-jsonschema`** — JSON Schema validation. As of the worked-example schema rollout, this hook is wired by default with two scoped entries: one validates the valid example data files under `schemas/examples/example-config/valid/` against the worked-example schema, and one (`check-metaschema`) self-validates the schema against its declared JSON Schema Draft 2020-12 metaschema. Downstream repositories MAY add additional `check-jsonschema` hook entries for their own schema-backed file families. The "Avoid placeholder schema hooks" rule under the Schema Validation Tiers ADR is still in force: the wiring exists because a real schema and real example data ship in the template, not as a placeholder. - -**Notable scoping:** - -- **`check-json` validates `.json`, not `.jsonc`.** The hook is anchored with `files: \.json$` so JSONC files (which permit comments and trailing commas) are intentionally skipped. Strict-JSON syntax checks would reject any valid JSONC file, producing false positives. Repos that need strict JSONC enforcement should add **JSONC-aware tooling** (e.g., a JSONC parser or a schema validator that understands JSONC) rather than retrofitting `check-json` for files it cannot correctly parse. - -**Why `jsonlint` is not added by default:** - -- `jsonlint` overlaps with `check-json` for strict JSON syntax and adds a Node-only dependency. -- `check-json` already catches the high-value failure modes (parse errors, duplicate keys) and is part of the broader `pre-commit-hooks` repo already in use. -- Adding `jsonlint` would expand the toolchain for marginal benefit and would not validate JSONC either. - -**`actionlint` first-run-on-restricted-networks caveat:** - -The `actionlint` pre-commit hook builds the `actionlint` binary from source on first install. On networks that block Go module downloads (corporate proxies, air-gapped environments), the first-run install can fail. CI is the shared enforcement environment, so contributors who hit a network restriction locally can rely on CI to enforce the hook. The same caveat is surfaced in `.pre-commit-config.yaml` (inline comment on the `actionlint` repo block) and `CONTRIBUTING.md` so contributors encounter it where they look first. Keep these cross-references in sync if the hook is repinned, replaced, or removed. - -**Trade-offs:** - -- Pro: Coverage spans strict syntax, style, and Actions-specific semantics with widely-used, well-maintained hooks. -- Pro: Each hook is independent — downstream repos can disable any one without disturbing the rest. -- Con: First-run network requirements for `actionlint` can confuse new contributors; mitigated by inline comments and CI as the source of truth. - -**Alternatives considered:** - -- **`jsonlint` instead of `check-json`:** Rejected for the dependency and overlap reasons above. -- **`prettier` for JSON/YAML format enforcement:** Rejected as a default; see the Prettier deferral ADR below. -- **A single mega-linter image:** Rejected. Mega-linter aggregations obscure version pinning, slow first-run installs, and complicate per-hook overrides. - -### Design Decision: yamllint truthy.check-keys Default - -`yamllint` is configured with `truthy.check-keys: false` in `.yamllint.yml`. - -**Rationale:** - -- The most common GitHub Actions workflow idiom is the unquoted `on:` key (`on: push`). With `truthy.check-keys: true`, `yamllint` flags `on` as a truthy value, producing false positives on every workflow file. -- Disabling key checking preserves the idiomatic Actions style without weakening the value-side `truthy` rule, which still flags ambiguous values like `yes`, `no`, `on`, `off` outside of keys. - -**Stricter alternative:** - -Repos that prefer maximum YAML 1.2 hygiene can: - -1. Quote `"on":` (and any other reserved truthy keys) in workflow files, and -2. Set `truthy.check-keys: true` in `.yamllint.yml`. - -This is a deliberate opt-in because it requires editing every workflow file at adoption time. The default favors zero-friction Actions authoring. - -**Trade-offs:** - -- Pro: Default works out-of-the-box for GitHub Actions, the most common YAML use case in this template. -- Con: A small class of truly-ambiguous truthy keys (e.g., a literal `yes:` map key) would not be flagged. The risk is low and stylistic — quoting is still recommended in the YAML guide. - -### Design Decision: yamllint line-length Warning Level Default - -`yamllint` is configured with `line-length.level: warning` in `.yamllint.yml`, even though the rest of the rule baseline aligned with the full template recommendation uses default (error) severity. - -**Rationale:** - -- A large fraction of long YAML lines in this template are **non-breakable user-facing content**: URLs in `.github/ISSUE_TEMPLATE/config.yml`, descriptive prose in issue-form `body:` blocks, and `echo`/`gh` shell commands inside workflow `run:` blocks that print PR comments or summary lines. Forcing these into multi-line continuations either harms readability (URLs, single sentences of UI copy) or changes the literal output that downstream users see (PR comment bodies). -- `yamllint`'s `line-length` rule already enables `allow-non-breakable-words` and `allow-non-breakable-inline-mappings`, but these only suppress lines whose overflow is a single non-breakable token. Comment-prefixed URLs and prose with embedded spaces still trip the rule. -- Treating line length as a warning preserves the signal (long lines still surface in `yamllint` output and CI annotations) without blocking PRs on cosmetic wraps that would not improve the underlying file. -- All other style violations (indentation, trailing whitespace, end-of-file newline, truthy values, brace/bracket spacing, comment formatting, empty-line counts) remain at default severity and **do** fail CI. Line length is the single intentional softening. - -**Stricter alternative:** - -Repos that want every long line to fail can either: - -1. Remove the `level: warning` line from `.yamllint.yml` to inherit `yamllint`'s default error severity, and reformat any offending YAML files; or -2. Raise `line-length.max` if 120 columns is the friction point rather than the severity. - -Adopting alternative 1 requires reformatting issue templates and workflow `run:` blocks at adoption time, which is the friction this default is meant to avoid. - -**Trade-offs:** - -- Pro: Default keeps idiomatic GitHub Actions workflows and issue-form prose readable without per-file `# yamllint disable-line: line-length` exceptions. -- Pro: Keeps line-length signal visible in CI annotations without blocking PRs on cosmetic wraps. -- Con: Long lines do not gate merges; reviewers must rely on the warning surface to notice problematic cases. Mitigated by the fact that every other style rule still fails the build. - -### Design Decision: Prettier Deferral for Data Files - -Prettier is **not** part of the default JSON/YAML toolchain. - -**Rationale:** - -1. **Tooling sprawl.** Adopting Prettier for data files expands Node tooling from "Markdown linting only" into "data-file formatting policy." That is a scope increase several adopters will not want, especially non-Node projects. -2. **Limited semantic value for JSON.** Prettier formats JSON but does not sort keys, which is the highest-value JSON-stability transformation a formatter could provide. Stable key ordering still has to be enforced by hand or via a separate tool. -3. **Conflicts with `yamllint` for YAML.** Prettier's YAML output differs from idiomatic `yamllint` defaults (line wrapping, flow vs. block style, quoting). Running both without explicit reconciliation produces churn-only diffs. -4. **JSONC support is partial across tools.** Adopting Prettier as the JSONC formatter forces additional decisions about which tools in the chain understand JSONC. - -**Policy:** - -- **JSON/JSONC:** Prettier remains **opt-in**. Repos that already use Prettier for Markdown or JavaScript may extend it to JSON/JSONC at their discretion; the JSON instruction guide is compatible with that choice. -- **YAML:** Prettier is **discouraged-by-default** unless the project explicitly reconciles Prettier's output with `yamllint` rules and pins both tools. Otherwise the two will fight. - -**Trade-offs:** - -- Pro: Smaller default toolchain; no Node dependency required for non-Node projects. -- Pro: No `yamllint` ↔ Prettier conflict surface in the default configuration. -- Con: No automatic data-file reformatting; authors lean on `check-json`/`yamllint` to flag issues rather than auto-fix them. - -### Design Decision: Schema Location at Repository Root - -Project-owned JSON Schemas live under a top-level `schemas/` directory, not under `.github/schemas/`. - -**Rationale:** - -1. **Schemas are project assets, not GitHub configuration.** `.github/` is reserved for GitHub-specific configuration (workflows, issue templates, CODEOWNERS, instructions). Schemas describe project data contracts that are consumed by application code, CI tooling, IDEs, and external tools. They belong alongside other project assets at the repository root. -2. **Discoverability.** A root-level `schemas/` directory is the conventional location IDEs and schema validators look for project schemas. -3. **Easy opt-out.** Downstream repos that do not use schema-backed data files can delete the entire `schemas/` directory without touching `.github/`. - -**Trade-offs:** - -- Pro: Clear separation between GitHub configuration and project data contracts. -- Pro: Aligns with widespread community convention. -- Con: One more top-level directory in repos that adopt schemas. - -**Recommendation:** Repos adopting schema-backed validation should keep `schemas/` at the repository root and reference schemas by relative path from data files (`$schema`) or by configuring `check-jsonschema` in pre-commit. - -### Design Decision: Schema Validation Tiers - -Schema validation is required (RFC 2119 MUST/SHOULD/MAY) at three tiers based on the role each data file plays. - -1. **MUST — Production / load-bearing contracts.** Data files whose shape is consumed by production code, deployment pipelines, public APIs, or release tooling MUST be validated against an explicit schema. Examples: published API request/response payloads, release manifests, policy bundles consumed by enforcement tools, IaC variable contracts. -2. **SHOULD — Durable fixtures, examples, policy documents, and config contracts.** Data files that survive across releases and are referenced by humans or by non-production tooling SHOULD be schema-validated. Examples: long-lived test fixtures, documented example payloads, governance policy documents, internal config files with a stable shape. -3. **MAY — Tool-owned simple config.** Files whose shape is owned and validated by the consuming tool itself (e.g., `.markdownlint.jsonc`, `pyproject.toml` sections, `.yamllint.yml` itself) MAY rely on the owning tool's own validation rather than a separately-maintained schema. - -**Avoid placeholder schema hooks.** Do not wire `check-jsonschema` (or any other schema validator) into pre-commit until at least one schema actually exists and is referenced. Placeholder hooks produce no-op runs, drift silently, and obscure when real schema enforcement has been adopted. - -**Trade-offs:** - -- Pro: Tiered policy concentrates rigor where breakage is expensive and avoids over-engineering tool-owned config. -- Pro: Avoiding placeholder hooks keeps the pre-commit surface honest. -- Con: Authors must classify each new contract; the JSON/YAML guides provide the rubric. - -### Design Decision: Built-in Schema Validation for Real Load-Bearing Configuration Files - -This template wires `check-jsonschema --builtin-schema ...` validation for selected real, load-bearing repository configuration files, in addition to the worked-example schema-validation pattern under `schemas/`. - -**Context:** - -- The repository already ships a worked-example schema under `schemas/` ([`example-config.schema.json`](../schemas/example-config.schema.json)) whose primary purpose is to prove the schema-validation pipeline end to end. -- Real repository configuration files such as Dependabot, pre-commit, package metadata, markdownlint, and yamllint can also be load-bearing: they drive automation, dependency updates, formatting, or hook execution, and a malformed value can silently break those tools. -- The [Schema Validation Tiers](#design-decision-schema-validation-tiers) ADR requires (MUST) or recommends (SHOULD) schema validation for automation-driving files when validation is practical and low-noise. -- `check-jsonschema` ships a curated set of built-in vendor schemas (`--builtin-schema vendor.`) that do not require network access at hook runtime and that track upstream schema changes through pinned `check-jsonschema` releases (kept current via the Dependabot `pre-commit` ecosystem). - -**Decision:** - -1. **Use `check-jsonschema --builtin-schema ...` for selected real load-bearing configuration files** when a mature, low-noise built-in schema exists in the pinned `check-jsonschema` release. -2. **Keep project-owned schemas under root-level `schemas/`.** Built-in schemas are not vendored into `schemas/`. -3. **Do not vendor third-party schemas into `schemas/`** unless a strong reason emerges (for example, an upstream schema that is unmaintained but still needed). The default path is "use the built-in schema" or "rely on the owning tool's own validation." -4. **Keep GitHub Actions workflow validation with [`actionlint`](https://github.com/rhysd/actionlint), not a redundant generic schema hook.** `actionlint` understands Actions-specific semantics (expression syntax, runner labels, shell-script linting via shellcheck) that a generic JSON Schema check would miss. - -**Selected files (currently wired):** - -| File | Built-in schema identifier | Hook in `.pre-commit-config.yaml` | -| --- | --- | --- | -| `.github/dependabot.yml` | `vendor.dependabot` | `validate-dependabot-config` alias on the `check-jsonschema` hook | - -**Evaluated but deferred:** - -The following candidates were evaluated and intentionally **not** wired in this iteration. This subsection is the durable negative-space record: each entry exists so future authors do not re-litigate the same decision without new evidence. - -- **GitHub Actions workflow files (`.github/workflows/*.yml`).** Already validated by `actionlint` (pinned in [`.pre-commit-config.yaml`](../.pre-commit-config.yaml) and re-run by [`.github/workflows/data-ci.yml`](./workflows/data-ci.yml)). Adding a generic JSON Schema hook against the SchemaStore Actions schema would be **redundant** with `actionlint` and would not catch the Actions-specific issues `actionlint` is designed for. Decision: rely on `actionlint`. -- **`.pre-commit-config.yaml`.** The pinned `check-jsonschema` release does not ship a `vendor.pre-commit-config` (or equivalent) built-in schema. pre-commit itself validates the file's structure when it loads its configuration, so a separate schema check would have minimal marginal value even if one became available. Decision: rely on pre-commit's own configuration loader. -- **`package.json`.** The pinned `check-jsonschema` release does not ship a `vendor.package` (or equivalent) built-in schema. npm validates this file when it parses dependency manifests, so it is not unguarded. Decision: rely on the package manager's own validation; revisit if a stable built-in schema is added upstream. -- **`.markdownlint.jsonc`.** The pinned `check-jsonschema` release does not ship a `vendor.markdownlint` built-in schema. The file is intentionally JSONC (it contains comments) and **MUST NOT** be converted to strict JSON merely to satisfy a validator. markdownlint's own configuration loader remains the enforcement mechanism for this file. Decision: rely on markdownlint's own validation. -- **`.yamllint.yml`.** The pinned `check-jsonschema` release does not ship a `vendor.yamllint` built-in schema. The file **MUST NOT** be weakened to satisfy an incomplete external schema; yamllint itself enforces its configuration shape when it loads `.yamllint.yml`. Decision: rely on yamllint's own validation. - -If a future `check-jsonschema` release adds a mature built-in schema for any of the deferred candidates, add a narrowly scoped hook with an anchored `files:` pattern at that time. Do not vendor third-party schemas into this repository to fill the gap. - -**Alternatives considered:** - -- **Vendor external schemas under `schemas/external/`.** Rejected as the default. Vendoring couples the repository to a specific snapshot of an external contract and shifts maintenance burden onto template maintainers (re-syncing on every upstream change). Built-in schemas shipped with a pinned `check-jsonschema` release achieve the same effect with a single dependency-update touchpoint. -- **Leave real load-bearing repository configuration files without schema validation.** Rejected for files where a mature built-in schema exists and validation is low-noise. The Schema Validation Tiers ADR already classifies these as SHOULD-validate when practical; not wiring an available, low-cost validator would be inconsistent with that policy. -- **Validate every JSON/YAML file generically.** Rejected. The [Schema Validation Tiers](#design-decision-schema-validation-tiers) ADR explicitly forbids generic "validate every JSON/YAML file" sweeps. Schema validation is a contract check for specific file families, not a global sweep; `check-json` and `check-yaml` already cover syntax. -- **Add redundant schema validation for GitHub Actions workflows on top of `actionlint`.** Rejected. `actionlint` is the authoritative validator for Actions workflow files; adding a generic JSON Schema check would duplicate effort, increase noise, and miss Actions-specific issues. - -**Consequences:** - -- Stronger parity between JSON/YAML and other first-class file types: load-bearing configuration is checked by a real validator, not just by syntax-only `check-json` / `check-yaml`. -- Built-in schema names and the schemas they reference track `check-jsonschema` releases. Schema updates arrive through the normal `check-jsonschema` release cycle rather than through hand-vendored snapshots. -- Dependabot `pre-commit` ecosystem updates help keep schema support current; reviewers SHOULD treat `check-jsonschema` bumps as schema-content changes, not just dependency hygiene. -- Downstream adopters who delete a validated file family **MUST** also remove the corresponding `check-jsonschema` hook (and any matching `data-ci.yml` step), per the removal guidance below. -- External schemas can be stricter, looser, or differently timed than the consuming tool's actual behavior. Hooks that produce noisy or misleading failures **SHOULD** be evaluated carefully and either narrowed in scope or removed. - -**Downstream removal guidance:** - -To remove built-in schema validation for a single file family in a downstream repository: - -1. Delete or stop using the validated file (for example, remove `.github/dependabot.yml`). -2. Remove the corresponding `check-jsonschema` hook entry (or its `alias:`) from [`.pre-commit-config.yaml`](../.pre-commit-config.yaml). -3. If the granular `data-ci.yml` step list explicitly invokes the hook by ID or alias, remove the corresponding step from [`.github/workflows/data-ci.yml`](./workflows/data-ci.yml). Steps that invoke a hook by ID continue to work for any remaining files; only remove the step if the entire hook is being retired. -4. Update documentation that lists the active schema-validated files, including the table in this ADR, [`schemas/README.md`](../schemas/README.md), and [`OPTIONAL_CONFIGURATIONS.md`](../OPTIONAL_CONFIGURATIONS.md). - -**Trade-offs:** - -- Pro: Real load-bearing configuration files get cheap, fast schema feedback locally and in CI. -- Pro: Built-in schemas avoid vendoring and avoid runtime network access. -- Pro: The "Evaluated but deferred" subsection makes the negative space durable, so future authors do not silently re-add or re-evaluate already-rejected candidates without new evidence. -- Con: Built-in schema coverage tracks `check-jsonschema` release cadence rather than upstream-vendor cadence; new vendor schema fields may take a release cycle to be reflected. -- Con: External schemas can disagree with the consuming tool's actual behavior; noisy hooks must be evaluated carefully. - -### Design Decision: JSON5 Exclusion by Default - -JSON5 is **not** included in the default toolchain or instruction guidance. - -**Rationale:** - -1. **Limited ecosystem support.** Many common JSON consumers (the Go standard library `encoding/json`, .NET `System.Text.Json` in default mode, `jq` without flags, `check-json`) do not parse JSON5. -2. **JSONC already covers the most common use case.** The primary motivation for JSON5 in this codebase's context (comments and trailing commas) is satisfied by JSONC for tool-owned configuration files. -3. **Avoid format proliferation.** Three tiers of JSON-shaped formats (JSON, JSONC, JSON5) increase author confusion, expand the validator matrix, and create more places where strict-JSON tooling silently skips files. - -**Adoption requires an explicit project decision.** A downstream repo that intentionally adopts JSON5 should record that decision in its own design-decisions document, add a JSON5-capable parser and validator, and update the JSON instruction guide accordingly. The default template stays on JSON + JSONC. - -**Trade-offs:** - -- Pro: Smaller, more interoperable default surface. -- Con: Repos that already use JSON5 must opt in explicitly rather than inheriting it from the template. - -### Design Decision: `additionalProperties` Policy - -JSON Schemas owned by this template (and recommended for downstream adopters) use `additionalProperties: false` for closed contracts, with documented exceptions for ecosystem-mirroring schemas. - -**Rules:** - -1. **Project-owned closed contracts:** Use `additionalProperties: false`. Closing the schema makes typos and drift fail loudly rather than silently being accepted. -2. **Ecosystem-mirroring schemas:** Open schemas (`additionalProperties: true` or omitted) are allowed when the schema mirrors an external ecosystem that itself permits unknown keys (e.g., extension fields in OpenAPI `x-*`, vendor-specific keys in tool config that the upstream tool intentionally allows). Each open schema MUST include an inline rationale comment or `description` explaining why the schema is open. -3. **Mixed schemas:** Use `additionalProperties: false` at the closed level and a typed `patternProperties` entry (or a typed sub-object) for the controlled extension surface. - -**Rationale:** - -- Closed-by-default catches the highest-frequency real-world failure mode: a typoed key silently being ignored. -- A documented escape hatch prevents the closed-default from misrepresenting genuinely open ecosystems. - -**Trade-offs:** - -- Pro: Schema becomes a useful drift detector. -- Con: Authors must update the schema when intentionally adding new keys; this is the intended behavior. - -### Design Decision: Testing Beyond Linting for JSON/YAML - -Schema validation is the primary correctness strategy for static JSON and YAML. A separate JSON/YAML test framework is **not** added by default. - -**Rationale:** - -1. **Static data shape is best validated statically.** Schemas catch shape errors deterministically across every file in scope without authoring per-file tests. -2. **Behavioral correctness belongs to the consumer.** If a JSON or YAML file influences runtime behavior, the test that proves the behavior belongs in the **consuming language's** test framework (pytest for Python, Pester for PowerShell, Terraform Test for Terraform). That keeps the data file under test alongside the code that consumes it. -3. **Avoid orphan test infrastructure.** A standalone "JSON/YAML test framework" without consumers tends to either duplicate schema validation or drift into ad-hoc behavioral tests in the wrong layer. - -**Policy:** - -- Use schemas (with the tiered MUST/SHOULD/MAY policy above) for static shape correctness. -- Use the consumer's existing test framework for behavioral assertions about data files. -- Do not add a new top-level data-file test framework as part of this template. - -**Trade-offs:** - -- Pro: Single mental model — schemas for shape, language tests for behavior. -- Pro: No additional CI surface to maintain. -- Con: Authors must remember to write consumer-side tests for behaviorally-significant data; the language instruction guides reinforce this. - -### Design Decision: .gitattributes JSON/YAML LF Pinning - -`.gitattributes` is **left unchanged** when adding JSON/YAML support. The existing policy — pinning LF line endings only inside byte-exact text fixture locations (`tests/**/golden/**`, `tests/**/goldens/**`, `tests/**/snapshots/**`, `tests/**/__snapshots__/**`, `tests/**/fixtures/**`, `testdata/**`) — is retained. There is no blanket `*.json` / `*.yaml` / `*.yml` LF pin. - -This decision was recorded in the Issue 3 PR description (PR #423, "Add `.yamllint.yml` and extend pre-commit JSON/YAML hooks") and is transcribed here. - -**Rationale:** - -1. **Existing policy is intentionally narrow.** The current `.gitattributes` rules target locations where byte-exact comparison (hash equality, signature verification, snapshot diffing) is the actual correctness contract. JSON and YAML files outside those locations do not have a byte-exact contract — their consumers parse them, not hash them. -2. **Tooling already produces parse-equivalent output across line endings.** `check-json`, `check-yaml`, `yamllint`, `actionlint`, and downstream parsers do not depend on LF endings; CRLF-converted JSON/YAML still validates and still parses identically. -3. **Avoid forcing CRLF rewrites on Windows checkouts that do not need them.** A blanket `*.json text eol=lf` rule would override `core.autocrlf=true` on Windows hosts even for files that are not part of any byte-exact comparison. The cost would land on every Windows contributor for no contract benefit. -4. **Byte-exact JSON/YAML fixtures are already covered.** Any JSON or YAML file living under the existing fixture-location patterns is already pinned to LF by directory rule; no extra extension-level rule is required. - -**Downstream opt-in guidance:** - -Repos that genuinely need blanket LF pinning for JSON/YAML — for example, projects whose release pipeline hashes JSON manifests, projects publishing YAML policy documents whose checksums are signed, or projects with a strict cross-platform diff-noise policy — can opt in by adding lines such as: - -```gitattributes -*.json text eol=lf -*.jsonc text eol=lf -*.yaml text eol=lf -*.yml text eol=lf -``` - -These should be added in the project's own `.gitattributes`, with a brief comment explaining the byte-exact contract that motivates them. The template does not ship these rules so that adopters who do not need them are not silently opted in. - -**Trade-offs:** - -- Pro: Windows contributors are not forced into LF for JSON/YAML files that have no byte-exact contract. -- Pro: Existing byte-exact fixture protection is preserved unchanged. -- Pro: Adopters that need blanket LF can add it deliberately, with rationale documented locally. -- Con: Cross-repo diffs of JSON/YAML files may include CRLF↔LF noise on heterogeneous platforms; this is mitigated by `core.autocrlf` configuration and by editors that preserve the on-disk EOL. - -**Alternatives considered:** - -- **Blanket `*.json` / `*.yaml` / `*.yml` LF pin in the template `.gitattributes`:** Rejected because it imposes a byte-exact contract on every adopter regardless of whether they have one. The byte-exact-fixture-focused policy is preserved. -- **Conditional per-language opt-in via a commented-out block in the template `.gitattributes`:** Rejected for this iteration to avoid adding code that is dormant by default; the downstream-opt-in snippet above plus this ADR provides the same guidance without dormant rules in the file. - ---- - -## Node.js Package Configuration - -### Design Decision: package.json Minimal Configuration - -The template ships with minimal package.json configuration (no repository field, no engines field, generic metadata) to reduce template adoption friction. - -**Rationale:** - -1. **Reduces friction**: Most users only need dev tooling (markdownlint scripts) without Node.js runtime dependencies. - -2. **Prevents placeholder sprawl**: Unlike OWNER/REPO placeholders that break functionality if not replaced, missing optional fields don't affect usage. - -3. **Clear separation**: Dev tooling (present) vs. application code (user adds). - -4. **Private by default**: The `"private": true` flag means omitted fields like repository don't affect npm publishing. - -**Trade-offs:** - -- Users creating Node.js applications must manually add metadata fields -- No validation for Node.js version requirements -- Users must consult README for customization guidance - ---- - -## CI Workflow Configuration - -### Design Decision: Dedicated Data-File CI Workflow (`data-ci.yml`) - -The repository ships a dedicated `.github/workflows/data-ci.yml` workflow that runs JSON, YAML, and GitHub Actions workflow validation as a first-class CI gate, alongside `python-ci.yml`, `powershell-ci.yml`, `terraform-ci.yml`, and `markdownlint.yml`. - -**Why a dedicated workflow even though `python-ci.yml` already runs every pre-commit hook:** - -- `python-ci.yml` runs `pre-commit run --all-files`, which transitively enforces the JSON/YAML/actionlint hooks. That works functionally but makes data-file validation appear incidental to Python CI. -- A dedicated workflow gives JSON/YAML/Actions validation a distinct required-check identity, clearer ownership, and parity with other per-language workflows. Branch protection rules can require data-file validation independently from Python validation. -- Template adopters who do not use Python should still see data-file validation as a first-class concern; surfacing it only under Python CI would obscure that. - -**CI ownership decision:** `python-ci.yml` continues to run the full `pre-commit run --all-files` pipeline as the single canonical aggregate pre-commit gate. `data-ci.yml` runs the data-file hooks (`check-json`, `check-yaml`, `yamllint`, `actionlint`, `check-jsonschema`, and `check-metaschema`) explicitly. The hooks therefore execute in both workflows. - -**Trade-off accepted:** The duplication between `python-ci.yml` and `data-ci.yml` is intentional. Splitting responsibility — narrowing `python-ci.yml` to Python-only checks — was rejected because it would change existing CI topology and complicate downstream branch protection migrations for marginal benefit. Visibility and ownership were prioritized over avoiding the duplicated hook execution. - -**Distinction from `auto-fix-precommit.yml`:** `data-ci.yml` is a contract enforcement gate that runs on all PRs and pushes. `.github/workflows/auto-fix-precommit.yml` is intentionally NOT a peer of `data-ci.yml`: it is a fix-up workflow scoped to `copilot/**` branches that auto-applies pre-commit fixes and does not enforce results. The two workflows serve different purposes and must not be conflated. - -**Schema validation steps:** As of the worked-example schema rollout, `data-ci.yml` runs dedicated `Run check-jsonschema` and `Run check-metaschema` steps that validate the template's worked-example schema (`schemas/example-config.schema.json`), its valid example data files, and the schema itself against its declared Draft 2020-12 metaschema. The single `Run check-jsonschema` step also covers real load-bearing repository configuration validation against built-in vendor schemas shipped with `check-jsonschema` — currently `.github/dependabot.yml` validated against `vendor.dependabot`. The aggregate `pre-commit run check-jsonschema --all-files` step is intentionally kept as a single invocation (Option B in the rollout) because it transitively runs every hook sharing the `check-jsonschema` ID, including any additional hooks downstream consumers add to `.pre-commit-config.yaml`. The dedicated steps are kept alongside the standard data-file hooks so the schema-validation surface is independently visible in CI logs. - -### Design Decision: Non-Blocking Type Checking by Default - -The template uses `continue-on-error: true` for the mypy type checking job. This is a deliberate choice for template portability. - -**Why this is the right default for a template:** - -- Template adopters start with varying levels of type coverage (often zero) -- Blocking type errors on day one creates adoption friction -- Gradual type adoption is a well-established Python best practice -- Allows adopters to see type errors without blocking their workflow - -**Trade-offs:** - -Blocking type checking (strict): - -- Pros: Enforces type safety, prevents type debt accumulation -- Cons: High friction for new adopters, requires upfront investment - -Non-blocking type checking (current): - -- Pros: Zero friction adoption, gradual improvement path, visibility without blocking -- Cons: Type errors can accumulate if not addressed, requires discipline - -**Alternatives considered:** - -1. No type checking at all: Rejected because mypy provides value even when non-blocking (developers can see errors and fix them opportunistically) - -2. Strict by default with documentation to make it lenient: Rejected because this inverts the adoption experience—failing CI on first push is a poor template UX - -3. Conditional based on repository variable: Rejected as over-engineering for a simple toggle that adopters can easily change - -**When downstream repos should make this strict:** - -- After achieving reasonable type coverage (70%+ of public APIs) -- Before releasing stable versions (1.0+) -- When the team has committed to maintaining type annotations -- For libraries where type hints are part of the API contract - -### Design Decision: Placeholder Check Workflow Behavior - -The placeholder check workflow (`.github/workflows/check-placeholders.yml`) runs automatically in all repositories created from this template. It does NOT run in the template repository itself. - -**Implementation:** - -```yaml -if: github.repository != 'franklesniak/copilot-repo-template' -``` - -**This means:** - -- ✅ Zero configuration required for adopters -- ✅ Workflow activates automatically on first push/PR -- ✅ Template maintainers don't get spurious failures - -**Historical Context:** - -Previous versions of this template required setting a `TEMPLATE_INITIALIZED` repository variable to enable the workflow. This was changed because automatic behavior reduces adoption friction and eliminates the "forgot to configure" failure mode. The repository-name-based skip gate provides the same protection without requiring user action. - -### Design Decision: Documentation Strategy for Issue Templates - -Issue template design rationale is documented in this guide, not in extensive inline YAML comments. - -**Rationale:** - -- **Reduces duplication**: Design decisions apply across multiple templates; documenting once prevents inconsistency -- **Cleaner templates**: Makes YAML files easier to scan and edit -- **Centralized maintenance**: Updates to rationale don't require editing multiple files -- **Follows established pattern**: Consistent with PR template documentation approach - -**Alternative considered:** Keep all design rationale as inline YAML comments - -**Rejected because:** - -- Creates visual noise (30+ line comment blocks) -- Duplicates explanations across templates -- Makes it harder to find actionable customization markers -- Increases maintenance burden when rationale needs updating - -**Implementation:** - -- Each `.yml` file includes a brief header comment pointing to this guide -- `# CUSTOMIZE:` and `# ACTION ITEM:` markers remain inline for visibility -- Extended rationale, alternatives, and examples documented here - ---- - -## Python Configuration - -### Design Decision: Python Dependency Version Alignment - -The root `pyproject.toml` uses the same pytest version (>=8.0.0) as the template file (`templates/python/pyproject.toml`). This is a deliberate choice for template clarity and consistency. - -**Rationale:** - -1. **Single source of truth**: The root `pyproject.toml` serves both as CI configuration AND as a working example for template users. Using current best-practice versions demonstrates the intended configuration. - -2. **Reduces confusion**: When template consumers compare the root `pyproject.toml` to `templates/python/pyproject.toml`, consistent versions eliminate questions about which version to use. - -3. **Current stable version**: pytest 8.0+ is the current stable version as of January 2026, with significant improvements over pytest 7.x including better assertion introspection, improved output formatting, and enhanced plugin support. - -4. **Template portability**: Template adopters can use either file as reference without needing to reconcile version differences. - -**Trade-offs:** - -- Slightly higher minimum version requirement than strictly necessary for CI to pass -- May require newer Python environments (pytest 8.0 requires Python 3.8+) - -**Alternatives considered:** - -- Using pytest>=7.0 in root for minimal CI requirements: Rejected because it creates inconsistency between root and template files, leading to adopter confusion. - ---- - -## Security and Vulnerability Reporting - -### Design Decision: Private Vulnerability Reporting Availability - -Private vulnerability reporting via GitHub Security Advisories is ONLY available for PUBLIC repositories on GitHub.com. This is a GitHub platform limitation, not a template configuration choice. - -**Rationale:** - -- GitHub's private vulnerability reporting feature requires the repository to be publicly accessible so that external security researchers can submit reports -- Private repositories cannot receive external vulnerability reports because external users cannot access the Security tab -- This affects all GitHub.com repositories; GitHub Enterprise Server (GHES) may have different availability depending on version and licensing - -**Implications for template adopters:** - -1. If repository will remain private permanently: Remove GitHub Advisories option, use email-only approach -2. If repository is private now but will become public later: Keep both options, document that Advisories will work once public -3. If repository is already public: All options work as documented - -**Trade-offs:** - -- Pro: Template provides guidance for all repository visibility scenarios -- Pro: Prevents confusion when "Report a vulnerability" link doesn't work -- Con: Additional complexity in adoption documentation -- Con: Adopters must make visibility-dependent decisions during setup - ---- - -## License Configuration - -### Design Decision: MIT License as Template Default - -This template uses the MIT License as its default because: - -1. **Minimal friction**: MIT is one of the most permissive and widely-understood licenses -2. **Template portability**: Works for both open source and commercial projects (with modification) -3. **Simplicity**: Short, clear terms that don't require legal expertise to understand -4. **Compatibility**: Compatible with most other open source licenses - -**Trade-offs:** - -- Pro: Maximum adoption potential due to permissive terms -- Pro: Simple contributor agreement (no CLA needed for most cases) -- Pro: Widely recognized in enterprise and open source communities -- Con: Provides no patent protection (unlike Apache 2.0) -- Con: No copyleft protection (unlike GPL) -- Con: Proprietary projects must replace with appropriate license - -**Alternatives considered:** - -1. Apache 2.0 as default: Rejected because patent grant clause can be unfamiliar to some adopters and adds complexity for simple projects -2. No default license: Rejected because unlicensed code is legally unusable; providing a permissive default is better than no default -3. Dual licensing (MIT + Apache 2.0): Rejected as over-engineering for a template - -**Recommendation:** - -Most open source projects can keep MIT. Consider Apache 2.0 for projects involving patents. Proprietary projects MUST replace the license entirely. - ---- - -## Dependabot Configuration - -### Design Decision: Workflow Version Pinning and Dependabot Coherence - -The repository-wide rule for how third-party action versions and tool versions are referenced inside `.github/workflows/*.yml` lives in the **Workflow Version Pinning** section of [`.github/copilot-instructions.md`](copilot-instructions.md#workflow-version-pinning). This ADR records the rationale and the alternatives that were considered. - -**Context:** - -- Dependabot's `github-actions` ecosystem rewrites GitHub Actions `uses:` references when an action publishes a new version. It does **not** generally rewrite unrelated `with:` inputs, workflow-level `env:` values, cache keys, file paths, shell-script literals, manually constructed image tags, or comments — even when those locations contain a literal copy of the same action version string. -- Workflow files in this repository today contain repeated `uses:` references to the same action (for example, `actions/checkout@v6` and `actions/setup-python@v6` appear in multiple workflows and jobs). Each occurrence is a normal Dependabot-managed `uses:` reference, which is fine. -- Workflow files also contain manually maintained tool-version inputs that are **not** Dependabot-managed: `terraform_version: "1.14.4"` (passed to `hashicorp/setup-terraform@v4`) appears in multiple jobs in [`.github/workflows/terraform-ci.yml`](workflows/terraform-ci.yml), and `tflint_version: v0.51.1` is passed to `terraform-linters/setup-tflint@v6`. These are CLI versions installed by the setup actions, distinct from the setup action versions themselves. -- Without a written rule, future authors could reasonably assume that lifting an action version into a workflow-level `env:` variable is "DRY" or that a cache key embedding the action version is a useful invalidation hint. Both patterns silently break Dependabot coherence: Dependabot bumps the `uses:` line, the mirror remains stale, and the workflow runs with a mismatched literal until someone notices. - -**Decision:** - -1. **Action versions in `uses:` lines are the only authoritative source for the action version.** Repeated `uses:` references are acceptable; Dependabot updates each line. -2. **Action versions MUST NOT be mirrored** into workflow-level `env:` variables, comments-presented-as-authoritative-state, cache keys, file paths, shell literals, manually constructed image tags, or any other secondary location that Dependabot will not reliably rewrite. -3. **Secondary workflow behavior that needs to track a `uses:` change** must be derived from a stable source that naturally changes with the workflow or tool configuration — for example, cache keys scoped to the specific configuration file that governs the cached artifact (`hashFiles('.pre-commit-config.yaml')` for pre-commit caches, `hashFiles('.tflint.hcl')` for TFLint plugin caches, `hashFiles('**/.terraform.lock.hcl')` for Terraform provider caches, mirroring the pattern already used in this repository's workflows) — rather than embedding a duplicated literal version string. Avoid broad wildcard patterns like `hashFiles('.github/workflows/*.yml')` for cache keys: any unrelated workflow edit would invalidate every job's cache, and the goal is to track the configuration that drives the cached content, not the workflow definition that consumes it. -4. **Tool versions that are not Dependabot-managed** (for example, `terraform_version`, `tflint_version`) **SHOULD** still avoid unnecessary duplication. A workflow-level `env:` value is the encouraged single source of truth for these, precisely because Dependabot does not manage them and there is no desync risk. -5. **Wrapper actions and the tools they install are separate pins.** `hashicorp/setup-terraform@v4` is the setup action version (Dependabot-managed via `uses:`); `terraform_version: "1.14.4"` is the CLI version installed by that setup action (manually maintained). Both pins coexist in the same step; do not conflate them. -6. **The `.github/dependabot.yml` `ignore:` escape hatch** is appropriate only when a Dependabot-managed dependency genuinely cannot be represented without duplication and Dependabot would otherwise produce partial updates. When used, the `ignore:` entry **MUST** carry a YAML comment explaining why the dependency is intentionally not auto-updated. - -**Asymmetry around workflow-level `env:`:** - -- For **action versions**, a workflow-level `env:` mirror is forbidden because Dependabot will not rewrite the mirror. -- For **tool versions**, a workflow-level `env:` value is encouraged because Dependabot does not manage the value, so there is no desync risk and the `env:` value can serve as the source of truth across multiple steps. - -This asymmetry is the entire point of the rule: the choice of where a version literal lives depends on whether Dependabot owns updates for that literal, not on a generic preference for "DRY" or "single source of truth." - -**Alternatives considered:** - -- **Allow workflow-level `env:` mirrors of action versions, on the theory that a human reviewer will catch the desync.** Rejected. The whole purpose of Dependabot is to remove the human reviewer from routine version bumps; relying on review to catch silent drift defeats the automation. -- **Forbid repeated `uses:` references and require a single composite action wrapper.** Rejected. Repeated `uses:` references are explicitly compatible with Dependabot; introducing composite-action indirection trades a non-problem for new template complexity. -- **Bulk-refactor existing workflows now to centralize `terraform_version` and `tflint_version` under workflow-level `env:`.** Out of scope for this documentation-only change. The rule documents the preferred pattern; existing duplication is accepted until a separate refactor lands. -- **Add `.github/dependabot.yml` `ignore:` entries pre-emptively for the duplicated tool-version literals.** Rejected. Tool-version literals are not Dependabot-managed in the first place, so `ignore:` would be a no-op. `ignore:` is reserved for genuine partial-update problems on Dependabot-managed dependencies. - -**Trade-offs:** - -- Pro: Eliminates a class of silent partial-update bugs where a `uses:` line moves but a mirrored literal stays on the old version. -- Pro: Documents the wrapper-action vs. installed-tool distinction so future authors do not collapse them into a single "version" concept. -- Pro: Makes the `dependabot.yml` `ignore:` escape hatch a documented, narrow tool rather than a folk remedy. -- Con: Authors who reach for "DRY" instinctively when they see a repeated `uses:` value must learn the rule before refactoring. The asymmetry table in [`.github/copilot-instructions.md`](copilot-instructions.md#workflow-version-pinning) is the primary mitigation. -- Con: Existing duplicated tool-version literals (`terraform_version: "1.14.4"` in three places) remain; the rule documents the preferred pattern but does not by itself eliminate them. A follow-up refactor is the appropriate vehicle for that cleanup. - -**Consequences:** - -- New workflow code authored after this ADR **MUST** keep action versions only in `uses:` references and **SHOULD** keep tool versions in a single workflow-level `env:` value when the same tool version is needed in multiple steps. -- Reviewers **SHOULD** flag any new workflow change that introduces a mirrored action version (in `env:`, cache keys, comments, file paths, shell literals, or image tags) and request that it be removed in favor of relying on the `uses:` line plus a stable derivation source for any dependent behavior. -- Adding an `.github/dependabot.yml` `ignore:` entry **MUST** be justified inline with a YAML comment that explains the partial-update risk being mitigated; an `ignore:` without that comment is treated as incomplete. - -### Design Decision: Dependabot Enabled by Default - -Dependabot is enabled by default in this template repository to provide automated dependency security monitoring and update management. This configuration represents security-conscious defaults that align with best practices for modern software projects. - -**Rationale:** - -- Security vulnerabilities in dependencies are automatically detected and PRs created -- Reduces maintenance burden for keeping dependencies current with security patches -- Template repositories should model best practices; security-conscious defaults are appropriate -- Monitors all three dependency ecosystems used by this template: npm, pip, and GitHub Actions -- Weekly schedule with grouped minor/patch updates balances security with reduced PR noise - -**Trade-offs:** - -- Pro: Automated security vulnerability detection and remediation -- Pro: Keeps dependencies current without manual monitoring -- Pro: Reduces risk of using outdated, vulnerable packages -- Pro: Grouped updates reduce PR noise for minor/patch versions -- Con: Creates PR noise for minor updates that adopters may not want -- Con: Adopters who prefer manual dependency management must disable it -- Con: May suggest updates that require testing/validation before merging - -**Recommendation:** - -Keep Dependabot enabled unless you have a specific reason to manage dependencies manually or use an alternative tool (e.g., Renovate). If the PR volume is too high, consider adjusting the schedule from weekly to monthly, or customizing the grouping configuration. Delete `.github/dependabot.yml` to disable entirely. - ---- - -## CODEOWNERS Configuration - -### Design Decision: CODEOWNERS with Placeholder - -The template includes a CODEOWNERS file with `@OWNER` placeholders that template adopters must replace. This file enables automatic review request assignment for pull requests and documents code ownership. - -**Rationale:** - -- CODEOWNERS enables automatic review requests for PRs affecting specific paths -- Works well with branch rulesets requiring code owner approval -- Using `@OWNER` placeholder follows the existing `OWNER/REPO` pattern in this template -- Placeholder check workflow ensures adopters don't forget to customize -- Default rules cover repository root, workflows, and Copilot instructions - -**Trade-offs:** - -- Pro: Automatic PR review assignment reduces manual reviewer selection -- Pro: Documents code ownership explicitly in the repository -- Pro: Works with branch ruleset "required reviews from code owners" setting -- Pro: Placeholder check workflow ensures customization before use -- Con: Requires placeholder replacement during template adoption -- Con: Solo maintainers may not benefit from CODEOWNERS -- Con: Adds another file to the template adoption checklist - -**Recommendation:** - -Replace `@OWNER` with your GitHub username or team name (e.g., `@octocat` or `@my-org/maintainers`). For solo projects, you may delete the file if automatic review assignment is not needed. For team projects, CODEOWNERS is highly recommended to ensure consistent review practices. - ---- - -## Issue Template Design Decisions - -### Cross-Template Customization Patterns - -This section documents customization points that apply across all issue templates. - -#### Labels - -Update labels to match your repository's label taxonomy. Common patterns: - -- **Type labels**: `bug`, `enhancement`, `documentation` -- **Status labels**: `triage`, `confirmed`, `in-progress` -- **Priority labels**: `priority:critical`, `priority:high`, `priority:medium`, `priority:low` -- **Area labels**: `area:api`, `area:cli`, `area:docs` - -**ACTION ITEM:** If you want to use a `triage` label, you must first create it in your repository (it doesn't exist by default). The `triage` label cannot be auto-created when cloning a template repository—GitHub does not support this. - -#### Field IDs - -Templates use `snake_case` for all field IDs (e.g., `steps_to_reproduce`, `operating_system`). Maintain this convention when adding new fields for consistency. - -### bug_report.yml - -#### Security Notice URL Strategy - -> **Status: Superseded** (2026-05-03). This decision has been replaced by the **Issue and PR templates** carve-out in [`.github/instructions/docs.instructions.md`](instructions/docs.instructions.md). The relative links described below were not actually working: `[Security tab](security)` resolved to `/{owner}/{repo}/issues/security` (404), and `[SECURITY.md](blob/HEAD/SECURITY.md)` resolved to `/{owner}/{repo}/issues/blob/HEAD/SECURITY.md` (404), because issue-form `value:` blocks render at `/{owner}/{repo}/issues/new?...`. The current `bug_report.yml` uses absolute `https://github.com/OWNER/REPO/security` and `https://github.com/OWNER/REPO/blob/HEAD/SECURITY.md` URLs; the `OWNER/REPO` substitution is enforced by `.github/workflows/check-placeholders.yml` **only when that workflow is adopted and kept** (it is an optional adoption step that downstream repositories MAY skip or remove after initial setup, in which case no CI guardrail catches a missed substitution and adopters must verify it manually). The historical content is preserved below for context. - -The security notice uses relative links that work automatically after cloning: - -- `[Security tab](security)` - links to repository's Security tab -- `[SECURITY.md](blob/HEAD/SECURITY.md)` - links to security policy file - -**Tested and confirmed** to work in GitHub issue forms on GitHub.com. - -**Trade-offs:** - -- Relative links work immediately without OWNER/REPO replacement -- For GHES or external contexts, replace with absolute URLs - -#### runtime_version Placeholder Format - -The placeholder shows multiple runtime examples rather than a single language example, using currently-supported version numbers. - -**Rationale:** - -- Template supports Python, PowerShell, and Markdown-focused projects -- Multi-line examples help reporters provide complete information -- **Placeholder examples should use currently-supported versions** for consistency with project policy (e.g., Python 3.13+ aligns with template's Python version policy) -- Using exact version format (not vague `.x`) demonstrates correct format -- Downstream repos should customize to match their supported runtimes - -#### how_ran Placeholder Format - -The placeholder shows detailed, multi-step installation examples rather than brief one-liners, including both `pyproject.toml` and `requirements.txt` workflows. - -**Rationale:** - -- Shows both `pyproject.toml` and `requirements.txt` workflows (template portability) -- Demonstrates best practices (venv creation, activation) -- Helps reporters provide complete reproduction steps -- Supports diverse downstream adopter workflows -- Doesn't lock adopters into a single dependency management approach - -**Alternative considered:** Brief form with multiple options on same line - -**Rejected because:** - -- Compressed form is harder to parse visually -- Doesn't demonstrate best practices (venv setup, activation) -- Less helpful for users unfamiliar with Python dependency management - -#### Area Dropdown - No "I don't know" Option - -The Area dropdown does NOT include an "I don't know / not sure" option. - -**Rationale:** - -- "Other (describe/specify)" already handles uncertain cases -- Field is `required: false` by default (intentional for template portability) -- Projects needing required area should define clear, actionable categories -- "I don't know" encourages lazy reporting - -**Alternative considered:** Add "I don't know / not sure" option to enable making field required. - -**Rejected because:** - -- Defeats the purpose of requiring area-based routing -- If a project can't confidently categorize bugs, area shouldn't be required -- "Other" option already provides escape hatch for edge cases - -#### Redundant Security Warnings - -This template intentionally includes multiple security warnings (top-of-form notice, required checkbox, severity dropdown warning). - -**Why keep all three:** - -- **Different interaction patterns:** Some users skim headers (→ checkbox catches them), some focus on dropdowns (→ severity warning catches them). Multiple touchpoints maximize chance of catching accidental public disclosure. -- **High stakes, low cost:** Cost of redundancy is slightly longer form. Cost of failure is public disclosure of security vulnerability. Risk/reward strongly favors redundancy. -- **Template portability:** Downstream adopters can easily remove warnings if desired. Harder to add them back if not provided. Template should err on side of caution. -- **Audit trail:** Required checkbox provides explicit acknowledgment. - -**Alternative considered:** Remove severity dropdown warning text, keep top warning + checkbox. - -**Rejected because:** - -- Severity dropdown is where users actively interact (making selection) -- Warning at point of interaction provides contextual reminder -- Consistency with documented design decision (no compelling reason to change) - -### feature_request.yml - -#### Area Dropdown Consistency - -The Area dropdown options match `bug_report.yml` for consistency. Update both templates when modifying area categories. - -#### Priority vs Scope - -The template separates priority (urgency from reporter's perspective) and scope (size of the feature). Both are optional and self-assessed by reporters; maintainers may adjust during triage. - -### documentation_issue.yml - -#### No Area Dropdown by Default - -This template intentionally does NOT include an `area` dropdown field. - -**Rationale:** - -- Most consumers have simple documentation sets that don't require area routing -- Keeping the template lean encourages drive-by documentation reports -- Documentation issues are typically easy to locate via the `location` field - -#### Location Field Optional - -**Trade-off:** - -- Optional (current): Encourages drive-by typo reports with lower friction -- Required: More actionable for maintainers; may reduce submissions - -Recommendation: Keep optional but encourage providing location via description text. - -### config.yml - -#### blank_issues_enabled - -Set to `true` for flexibility (allows any issue format), or `false` to enforce template usage. Most projects benefit from `true` initially; consider `false` once you have comprehensive templates. - -#### contact_links URL Requirement - -**Critical:** Unlike issue-form markdown blocks (where relative links work), `contact_links` URLs MUST be absolute URLs. There is no way to use relative links here. - -- You MUST replace `OWNER/REPO` with your actual org/repo -- Use `blob/HEAD` instead of `blob/main` to support non-main default branches - -#### GitHub Enterprise Server (GHES) Portability - -GHES host replacement is documented in comments, not enforced via placeholders. - -**Rationale:** - -- GHES users universally know their host (appears in every URL) -- One-line note in "MUST READ" section is sufficient -- Avoids placeholder proliferation (simpler adoption) -- No CI validation needed for host placeholder - -#### Security Link Documentation - -Detailed setup instructions remain in comments, not in user-facing `about` text. - -**Rationale:** - -- `about` text appears in issue chooser UI (end-user-facing) -- Long docs URLs would clutter the chooser display -- Comment block is appropriate for template adoption guidance -- Quick setup steps (1-2-3) in comments reduce adopter friction - -#### Discussions Link - -Kept commented out by default because many downstream repos don't enable Discussions. - -To enable: - -1. Go to repository Settings > General > Features -2. Check "Discussions" checkbox -3. Uncomment the discussions contact link block -4. Replace `OWNER/REPO` with your actual org/repo - ---- - -## Branch Ruleset Setup - -This section documents how to configure a branch ruleset using the CI workflows provided by this template. Repository rulesets are the recommended approach for protecting branches, replacing classic branch protection rules. They offer more granular control and can be applied across multiple branches or repositories. - -### Design Decision: Branch Ruleset Documentation - -This template includes documentation for branch ruleset setup rather than attempting to configure it automatically. Branch rulesets are repository settings that cannot be included in template repositories, so documentation is the appropriate way to guide adopters. - -**Rationale:** - -- Helps adopters set up proper CI gates for their default branch -- Explains the intended use of CI workflows and how they relate to branch rulesets -- Documents which CI jobs are good candidates for required status checks -- Clarifies the relationship between `needs:` dependencies and branch rulesets - -**Trade-offs:** - -- Pro: Helps adopters set up proper CI gates quickly -- Pro: Explains intended use of CI workflows from this template -- Pro: Clarifies complementary nature of CI dependencies vs branch rulesets -- Con: GitHub UI may change over time, requiring documentation updates -- Con: Cannot be enforced via template (requires manual setup in each repository) -- Con: Adopters must manually configure settings in GitHub UI - -**Recommendation:** - -Configure a branch ruleset for your default branch after initial repository setup. At minimum, require the pre-commit check to pass before merging. For additional protection, also require downstream checks like tests and type checking. - -### CI Jobs Available as Required Status Checks - -The template provides these CI jobs that can be configured as required status checks: - -| Workflow | Job Name | Recommended as Required | Notes | -| --- | --- | --- | --- | -| `python-ci.yml` | **Pre-commit** | ✅ Yes | Foundational check—catches formatting and linting issues | -| `python-ci.yml` | **Type Check (mypy)** | Optional | Set to `continue-on-error: true` by default; make strict when ready | -| `python-ci.yml` | **Test** | ✅ Yes | Ensures tests pass on all platforms | -| `markdownlint.yml` | **Markdown Lint** | ✅ Yes | Ensures documentation quality | -| `powershell-ci.yml` | **Lint (PSScriptAnalyzer)** | Optional | Only if using PowerShell | -| `powershell-ci.yml` | **PowerShell Tests (Pester)** | Optional | Only if using PowerShell with tests | -| `check-placeholders.yml` | **Check for OWNER/REPO Placeholders** | Optional | Only runs in repos created from this template (skipped in template repo itself) | - -> **Note:** In the GitHub Actions UI and branch ruleset status-check picker, -> checks appear in the format **`Workflow name / Job name`** (for example, -> `Python CI / Pre-commit`). The **Job Name** column above lists only the -> job-level name; prepend the workflow name when searching for checks in the -> ruleset configuration. Status checks only appear for selection after the -> corresponding workflow has run at least once. -> -> The `auto-fix-precommit.yml` workflow is intentionally **not listed** above -> because it only triggers on pushes to `copilot/**` branches by the Copilot -> bot. Making it a required status check would block PRs from all other -> branches where the check never runs. - -### How to Configure a Branch Ruleset - -Complete this step **after** your CI workflows have run at least once so that status checks are available to select. - -1. Go to your repository on GitHub -2. Navigate to **Settings** > **Rules** > **Rulesets** -3. Click **New ruleset** → **New branch ruleset** -4. Configure the ruleset: - - **Ruleset name:** `main branch ruleset` - - **Enforcement status:** **Active** -5. Under **Target branches**, click **Add target** → **Include default branch** -6. Under **Branch rules**, enable the following: - - ✅ **Restrict deletions** - - ✅ **Require a pull request before merging** - - Required approvals: **1** - - ✅ Dismiss stale pull request approvals when new commits are pushed - - ✅ **Require status checks to pass** - - ✅ Require branches to be up to date before merging - - Click **Add checks** and search for the status checks you want to require (see the [CI Jobs table](#ci-jobs-available-as-required-status-checks) above for recommended checks) - - ✅ **Require conversation resolution before merging** (optional) - - ✅ **Block force pushes** -7. Under **Bypass list** (at the top of the ruleset): - - Leave empty if you want no one to bypass the rules - - Optionally click **Add bypass** → **Repository admin** if you want the ability to force-merge as an admin -8. Click **Create** - -> **Note:** Repository rulesets offer more granular control than classic -> branch protection rules and can be applied across multiple branches or -> repositories. See GitHub's -> [rulesets documentation](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets) -> for more details. - -### Understanding `needs:` vs Branch Rulesets - -The template CI workflows use `needs:` to create internal job dependencies: - -```yaml -test: - name: Test - needs: pre-commit # Test job waits for pre-commit to pass -``` - -**How `needs:` works (internal CI dependency):** - -- If `pre-commit` fails, the `test` job is automatically skipped -- This saves CI minutes by not running tests on poorly-formatted code -- The dependency is internal to the workflow—GitHub Actions manages it - -**How branch rulesets work (external gate):** - -- Branch rulesets are configured in repository settings, not in workflows -- They prevent PR merges until selected status checks pass -- They are an external enforcement mechanism that operates at the PR level - -**These are complementary:** - -- `needs:` optimizes CI execution (skip downstream jobs on early failure) -- Branch rulesets enforce quality gates (block merges until checks pass) -- Using both provides defense in depth - -**Recommendation:** Require **both** the `Pre-commit` job AND downstream jobs like `Test` in the branch ruleset. Even though `Test` won't run if `Pre-commit` fails (due to `needs:`), requiring both ensures that: - -1. Format/lint issues block the PR (Pre-commit requirement) -2. Test failures block the PR (Test requirement) -3. Skipped jobs (due to upstream failure) also block the PR - -### Example Branch Ruleset Configuration - -For a Python project using this template: - -**Required status checks:** - -- Pre-commit -- Test -- Markdown Lint - -**Optional but recommended:** - -- Type Check (mypy) — after making it strict by removing `continue-on-error: true` - -For a multi-language project (Python + PowerShell): - -**Required status checks:** - -- Pre-commit -- Test -- Markdown Lint -- Lint (PSScriptAnalyzer) -- PowerShell Tests (Pester) - ---- - -## Document Maintenance - -This document should be updated when: - -- New design decisions are made that affect template structure or behavior -- Existing design decisions are revised based on new information or feedback -- Design decisions are superseded by new approaches - -When adding new design decisions, follow the established format: - -1. Add a heading with the pattern `### Design Decision: [Topic]` -2. Explain the rationale and trade-offs -3. Document alternatives considered and why they were rejected -4. Include recommendations where appropriate +Protected instruction files were not edited during Phase 0. Any request to remove, de-link, or rewrite inherited protected guidance must be approved explicitly by the repository owner. diff --git a/.github/dependabot.yml b/.github/dependabot.yml index aa5ad34..f6a28f4 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -4,7 +4,6 @@ # # This configuration monitors: # - npm (package.json) - Node.js dependencies for dev tooling -# - pip (pyproject.toml) - Python dependencies # - GitHub Actions (workflows) - Action version updates # - pre-commit (.pre-commit-config.yaml) - Pre-commit hook updates # @@ -54,22 +53,6 @@ updates: prefix: "chore(deps)" open-pull-requests-limit: 10 - # Python dependencies (pyproject.toml) - - package-ecosystem: "pip" - directory: "/" - schedule: - interval: "weekly" - groups: - pip-minor-patch: - patterns: - - "*" - update-types: - - "minor" - - "patch" - commit-message: - prefix: "chore(deps)" - open-pull-requests-limit: 10 - # GitHub Actions (workflows) - package-ecosystem: "github-actions" directory: "/" diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 64d0d3d..399da47 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,130 +1,46 @@ - - ## Description - + ## Type of Change - - -- [ ] Bug fix (non-breaking change which fixes an issue) -- [ ] New feature (non-breaking change which adds functionality) -- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) +- [ ] Bug fix +- [ ] New feature +- [ ] Breaking change - [ ] Documentation update - [ ] Dependencies update - [ ] Configuration/tooling change ## Checklist - - ### General -- [ ] I have read the [contributing guidelines](https://github.com/OWNER/REPO/blob/HEAD/CONTRIBUTING.md) -- [ ] My code follows the coding standards in `.github/instructions/` -- [ ] My changes follow `.github/copilot-instructions.md` and any applicable `.github/instructions/*` -- [ ] I have performed a self-review of my own code -- [ ] I have commented my code where necessary, particularly in hard-to-understand areas -- [ ] I have made corresponding changes to the documentation -- [ ] My changes generate no new warnings -- [ ] I have added or updated tests where appropriate -- [ ] New and existing tests pass locally (where applicable) +- [ ] I have read the [contributing guidelines](https://github.com/franklesniak/macOSLab/blob/HEAD/CONTRIBUTING.md) +- [ ] My changes follow `.github/copilot-instructions.md` and the applicable `.github/instructions/*` files +- [ ] I did not modify protected instruction files without explicit owner authorization +- [ ] I reviewed the change for secrets, tenant identifiers, recovery keys, tokens, and personal data +- [ ] I added or updated tests where appropriate +- [ ] I updated documentation where behavior changed -### Pre-commit Verification (if configured) +### Pre-commit Verification -- [ ] If this repository uses pre-commit, I ran `pre-commit run --all-files` and all checks pass -- [ ] If pre-commit made auto-fixes, I reviewed and committed them +- [ ] I ran `pre-commit run --all-files` locally and all checks pass +- [ ] I reviewed and committed all auto-fixes made by pre-commit hooks -### Python-Specific (if applicable) +### PowerShell-Specific - +- [ ] Pester `5.7.1` passes locally with `Invoke-Pester -Path tests/ -Output Detailed` +- [ ] PSScriptAnalyzer is clean with `.github/linting/PSScriptAnalyzerSettings.psd1` +- [ ] `Test-LabReadiness.ps1` runs without parameter errors when applicable -- [ ] Minimum Python version complies with the bugfix support policy (see [Python Developer's Guide - Versions](https://devguide.python.org/versions/)) -- [ ] I have not defaulted to or required unsupported Python versions -- [ ] Type hints are included for public APIs (if using type checking) -- [ ] Tests have been added/updated for Python changes -- [ ] `pytest` passes locally -- [ ] `mypy` type checking passes (if applicable) +## Manual Actions Deferred to Owner -### PowerShell-Specific (if applicable) + - +## Open Questions / Requested Authorization -- [ ] PSScriptAnalyzer passes locally (`Invoke-ScriptAnalyzer -Path . -Settings .github/linting/PSScriptAnalyzerSettings.psd1`) -- [ ] Pester tests pass locally (`Invoke-Pester -Path tests/ -Output Detailed`) -- [ ] PowerShell formatting follows repository standards (OTBS, consistent line endings) - -## Additional Notes - - + ## Related Issues - - Closes # diff --git a/.github/scripts/lint-nested-markdown.js b/.github/scripts/lint-nested-markdown.js index b0256c8..22dbd74 100644 --- a/.github/scripts/lint-nested-markdown.js +++ b/.github/scripts/lint-nested-markdown.js @@ -12,7 +12,7 @@ * Lint specific files: node .github/scripts/lint-nested-markdown.js file1.md file2.md * * When file arguments are provided, only those files are linted (useful for pre-commit hooks). - * When no arguments are provided, all .md files are scanned via glob (excluding node_modules). + * When no arguments are provided, all .md files are scanned via glob (excluding dependency caches). * Both absolute and relative paths are supported; relative paths are resolved from cwd. */ @@ -256,9 +256,9 @@ async function main() { files = validFiles; console.log(`Linting ${files.length} specified file(s)\n`); } else { - // Find all markdown files (excluding node_modules) + // Find all markdown files, excluding dependency and tool caches. files = await glob('**/*.md', { - ignore: ['node_modules/**', '**/node_modules/**'], + ignore: ['node_modules/**', '**/node_modules/**', '.venv/**', '**/.venv/**'], cwd: REPO_ROOT, absolute: true }); diff --git a/.github/workflows/auto-fix-precommit.yml b/.github/workflows/auto-fix-precommit.yml index e349528..4ade91b 100644 --- a/.github/workflows/auto-fix-precommit.yml +++ b/.github/workflows/auto-fix-precommit.yml @@ -5,17 +5,16 @@ # created by the agent pass pre-commit CI checks without manual intervention. # # Hook coverage: This workflow runs the same `pre-commit run --all-files` -# pipeline as `python-ci.yml`, so it runs every active hook in +# pipeline as `precommit-ci.yml`, so it runs every active hook in # `.pre-commit-config.yaml` (auto-fixing where the hook supports it) — # including JSON (check-json), YAML (check-yaml, yamllint), GitHub Actions -# workflows (actionlint), Markdown (markdownlint-cli2), Terraform -# (terraform_fmt, terraform_validate, terraform_tflint), and general hygiene -# hooks. Note that hooks like yamllint, actionlint, terraform_validate, and -# terraform_tflint are validators that do not auto-fix; this workflow runs -# them but does not enforce their results (the `pre-commit` exit code is +# workflows (actionlint), Markdown (markdownlint-cli2), schema validation, and +# general hygiene hooks. Note that hooks like yamllint and actionlint are +# validators that do not auto-fix; this workflow runs them but does not +# enforce their results (the `pre-commit` exit code is # intentionally ignored via `|| true` so that auto-fixed files can still be # committed). Validator failures are enforced by the `Pre-commit` job in -# `python-ci.yml`, which runs the same hooks without `|| true`. +# `precommit-ci.yml`, which runs the same hooks without `|| true`. # # Design considerations: # - Only triggers on push to copilot/** branches to avoid interfering with human work @@ -64,16 +63,6 @@ jobs: with: python-version: "3.x" - # Install Terraform for pre-commit-terraform hooks (terraform_fmt, etc.) - # This is required because the pre-commit config includes Terraform hooks - # that need the Terraform binary to be available. - # Version Policy: Use the latest stable Terraform version for pre-commit. - # Review and update quarterly per TEMPLATE_MAINTENANCE.md. - - name: Setup Terraform - uses: hashicorp/setup-terraform@v4 - with: - terraform_version: "1.14.4" - - name: Install pre-commit run: pip install pre-commit diff --git a/.github/workflows/data-ci.yml b/.github/workflows/data-ci.yml index cb78f18..9dc6706 100644 --- a/.github/workflows/data-ci.yml +++ b/.github/workflows/data-ci.yml @@ -2,8 +2,7 @@ # # Purpose: Run JSON, YAML, and GitHub Actions workflow validation as a # first-class, dedicated CI workflow alongside the other per-language -# workflows (python-ci.yml, powershell-ci.yml, terraform-ci.yml, -# markdownlint.yml). +# workflows (precommit-ci.yml, powershell-ci.yml, markdownlint.yml). # # Hooks executed (all from .pre-commit-config.yaml): # - check-json — strict JSON syntax (.json only; .jsonc excluded) @@ -35,13 +34,13 @@ # # Design Decisions: # -# 1. CI Ownership / Relationship to python-ci.yml: -# `python-ci.yml` continues to run the full `pre-commit run --all-files` +# 1. CI Ownership / Relationship to precommit-ci.yml: +# `precommit-ci.yml` runs the full `pre-commit run --all-files` # pipeline as the single canonical aggregate pre-commit gate. This # workflow does NOT replace that — it adds dedicated visibility and a # distinct required-check identity for JSON/YAML/Actions validation so # branch protection rules and contributors can reason about data-file -# quality independently of Python CI. +# quality independently from the aggregate pre-commit gate. # # Trade-off: the JSON/YAML/actionlint hooks run in both workflows. This # duplication is intentional and accepted in exchange for clearer @@ -58,18 +57,15 @@ # # 3. Template Repository Trigger Policy: # This workflow does NOT use path-based filtering. This repository is a -# template, and existing per-language workflows (python-ci.yml, -# powershell-ci.yml, terraform-ci.yml) intentionally avoid path filters -# so the workflow itself is exercised on every change and copied -# accurately by downstream consumers. This workflow mirrors that -# convention by running on all pushes and pull requests, plus -# `workflow_dispatch` to match `python-ci.yml` and `terraform-ci.yml`. +# repository bootstrap, and active workflows intentionally avoid path filters +# so the workflow itself is exercised on every change. This workflow runs on +# all pushes and pull requests, plus `workflow_dispatch` to match +# `precommit-ci.yml`. # # 4. Job and Workflow Naming: # The workflow name (`Data file linting`) and job name (`Data file -# linting`) are deliberately distinct from `Python CI` / `Pre-commit` so -# that branch protection rules can require data-file validation -# independently from Python validation. +# linting`) are deliberately distinct from `Pre-commit` so branch +# protection rules can require data-file validation independently. name: Data file linting @@ -92,7 +88,7 @@ jobs: - name: Checkout repository uses: actions/checkout@v6 - - name: Setup Python + - name: Setup Python for pre-commit tooling uses: actions/setup-python@v6 with: python-version: '3.x' diff --git a/.github/workflows/powershell-ci.yml b/.github/workflows/powershell-ci.yml index 18b6a9e..97da35e 100644 --- a/.github/workflows/powershell-ci.yml +++ b/.github/workflows/powershell-ci.yml @@ -46,6 +46,13 @@ jobs: - name: Checkout repository uses: actions/checkout@v6 + - name: Confirm PowerShell 7.4 or newer + shell: pwsh + run: | + if ($PSVersionTable.PSVersion -lt [version]'7.4') { + throw "PowerShell 7.4 or newer is required. Found $($PSVersionTable.PSVersion)." + } + - name: Find PowerShell files id: find-ps1 run: | @@ -135,6 +142,13 @@ jobs: - name: Checkout repository uses: actions/checkout@v6 + - name: Confirm PowerShell 7.4 or newer + shell: pwsh + run: | + if ($PSVersionTable.PSVersion -lt [version]'7.4') { + throw "PowerShell 7.4 or newer is required. Found $($PSVersionTable.PSVersion)." + } + - name: Find Pester test files id: find-tests shell: pwsh @@ -154,7 +168,7 @@ jobs: if: steps.find-tests.outputs.found == 'true' shell: pwsh run: | - Install-Module -Name Pester -MinimumVersion 5.0 -Force -Scope CurrentUser + Install-Module -Name Pester -RequiredVersion 5.7.1 -Force -Scope CurrentUser Import-Module Pester Write-Host "Pester version: $((Get-Module Pester).Version)" diff --git a/.github/workflows/precommit-ci.yml b/.github/workflows/precommit-ci.yml new file mode 100644 index 0000000..aa90056 --- /dev/null +++ b/.github/workflows/precommit-ci.yml @@ -0,0 +1,45 @@ +# Pre-commit CI Workflow +# +# Purpose: Run the full pre-commit hook set as the aggregate repository hygiene +# gate. Python is installed only because pre-commit and retained hooks such as +# check-jsonschema are Python-based development tooling; macOSLab has no Python +# project source. + +name: Pre-commit + +on: + push: + branches: ["**"] + pull_request: + branches: ["**"] + workflow_dispatch: + +permissions: + contents: read + +jobs: + pre-commit: + name: Pre-commit + runs-on: ubuntu-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v6 + + - name: Setup Python for pre-commit tooling + uses: actions/setup-python@v6 + with: + python-version: '3.x' + cache: 'pip' + + - name: Cache pre-commit hooks + uses: actions/cache@v5 + with: + path: ~/.cache/pre-commit + key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }} + + - name: Install pre-commit + run: pip install pre-commit + + - name: Run pre-commit hooks + run: pre-commit run --all-files diff --git a/.github/workflows/python-ci.yml b/.github/workflows/python-ci.yml deleted file mode 100644 index af58c6e..0000000 --- a/.github/workflows/python-ci.yml +++ /dev/null @@ -1,241 +0,0 @@ -# Python CI Workflow -# -# Purpose: Run pre-commit hooks, type checking (mypy), and tests (pytest) for -# Python code in a single consolidated workflow. -# -# Despite the "Python CI" name, the `Pre-commit` job runs the full -# `pre-commit run --all-files` pipeline and therefore enforces every active -# hook in `.pre-commit-config.yaml`, including non-Python hooks: -# - JSON syntax (check-json, strict .json only) -# - YAML syntax and style (check-yaml, yamllint) -# - GitHub Actions workflow linting (actionlint) -# - Markdown linting (markdownlint-cli2) -# - Terraform formatting/validation (terraform_fmt, terraform_validate, -# terraform_tflint) -# - General hygiene (trailing-whitespace, end-of-file-fixer, -# check-added-large-files) -# This is the single canonical pre-commit enforcement workflow; downstream -# repos and other workflows should not duplicate `pre-commit run --all-files` -# as a required check. -# -# Design Decisions: -# -# 1. Single Workflow with Job Dependencies: -# This workflow combines pre-commit, type checking, and tests into a single -# workflow file using the `needs:` keyword for job dependencies. This design -# replaces the previous pattern of separate workflow files with `workflow_run` -# triggers. -# -# Benefits of this approach: -# - All jobs appear directly in the PR checks list (not hidden in Actions tab) -# - Easy to configure branch rulesets that require specific jobs -# - Tests don't run if pre-commit fails (saves CI minutes) -# - Simpler maintenance with a single workflow file -# -# 2. Job Dependencies: -# The `type-check` and `test` jobs both have `needs: pre-commit`, which means: -# - They only run AFTER the pre-commit job completes successfully -# - If pre-commit fails, downstream jobs are automatically skipped -# - This ensures tests don't run on poorly-formatted code -# -# 3. Template Repository Consideration: -# This workflow does NOT use path-based filtering (on.push.paths or -# on.pull_request.paths) because this is a template repository. Path -# filtering would prevent the workflow from being properly tested and -# copied by users of the template. Template repositories should run all -# workflows on all changes to ensure accuracy when the template is used. -# -# 4. Python Version Policy: -# Test only actively supported bugfix versions (not security-fix-only). -# Python versions in "security fix only" phase are NOT publicly installable -# with security updates—they require building from source. -# As of January 2026, Python 3.13 is the only version that qualifies. -# Update this when upstream support changes (typically annually around October). -# See: https://devguide.python.org/versions/ -# -# 5. Dependency Caching: -# This workflow uses caching to improve performance on cloud-hosted runners. -# Caching reduces CI execution time by reusing previously downloaded -# dependencies instead of re-downloading them on every run. -# -# What is cached: -# - pip dependencies: Cached via actions/setup-python's built-in cache -# parameter. Cache key is based on dependency specification files. -# - pre-commit hooks: Cached via actions/cache. Cache key is based on -# the hash of .pre-commit-config.yaml. -# -# Expected time savings: 60-150 seconds per CI run, depending on the -# number of dependencies and network conditions. -# -# Cache invalidation: -# - pip cache invalidates when dependency specification files change -# (e.g., pyproject.toml, requirements.txt, setup.py) -# - pre-commit cache invalidates when .pre-commit-config.yaml changes -# -# Note: Type checking (mypy) runs only on ubuntu-latest, not in a -# cross-platform matrix. mypy performs platform-agnostic static analysis, -# so cross-platform type checking would triple CI time for negligible -# benefit. Runtime tests already cover cross-platform behavior. -# -# Note: This workflow is optional. Remove if your project doesn't use Python. - -name: Python CI - -on: - push: - branches: ["**"] - pull_request: - branches: ["**"] - workflow_dispatch: - -permissions: - contents: read - -jobs: - pre-commit: - name: Pre-commit - runs-on: ubuntu-latest - - steps: - - name: Checkout repository - uses: actions/checkout@v6 - - - name: Setup Python - uses: actions/setup-python@v6 - with: - python-version: '3.x' - cache: 'pip' - - # Install Terraform for pre-commit-terraform hooks (terraform_fmt, etc.) - # This is required because the pre-commit config includes Terraform hooks - # that need the Terraform binary to be available. - # Version Policy: Use the latest stable Terraform version for pre-commit. - # Review and update quarterly per TEMPLATE_MAINTENANCE.md. - - name: Setup Terraform - uses: hashicorp/setup-terraform@v4 - with: - terraform_version: "1.14.4" - - - name: Cache pre-commit hooks - uses: actions/cache@v5 - with: - path: ~/.cache/pre-commit - key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }} - - - name: Install pre-commit - run: pip install pre-commit - - - name: Run pre-commit hooks - # Runs every hook in .pre-commit-config.yaml (Python formatters/linters, - # JSON/YAML checks, yamllint, actionlint, markdownlint-cli2, Terraform). - # Required tools are set up above (Python, Terraform); other hooks - # provision their own environments inside pre-commit's hook cache. - run: pre-commit run --all-files - - type-check: - name: Type Check (mypy) - needs: pre-commit - runs-on: ubuntu-latest - - steps: - - name: Checkout repository - uses: actions/checkout@v6 - - - name: Setup Python - uses: actions/setup-python@v6 - with: - # Python Version Policy: - # Always test with a Python version currently receiving bugfixes. - # As of January 2026, Python 3.13 is the latest bugfix-supported version. - # Update this when upstream support changes (typically annually around October). - # See: https://devguide.python.org/versions/ - python-version: '3.13' - cache: 'pip' - - - name: Install dependencies - run: | - python -m pip install --upgrade pip - # Install project with dev dependencies - # Adjust this if your project uses a different installation method - pip install -e ".[dev]" - - - name: Run mypy - # MYPY_PATHS Configuration: - # Set this to the directories/files containing Python source/tests you want mypy to check. - # - # Examples: - # - Flat layout (modules in root): MYPY_PATHS="." - # - src/ layout (recommended): MYPY_PATHS="src/ tests/" - # - Custom directories: MYPY_PATHS="foo/ bar/ baz.py" - # - # The command-line paths override any 'files' or 'exclude' settings in - # pyproject.toml or mypy.ini in terms of directory scope. - env: - MYPY_PATHS: "src/ tests/" - # MYPY_PATHS contains space-separated paths that must be word-split - # into multiple arguments; intentional unquoted expansion below. - run: | - # shellcheck disable=SC2086 - mypy $MYPY_PATHS - # Set continue-on-error initially; projects can make type checking strict later - continue-on-error: true - - test: - name: Test - needs: pre-commit - runs-on: ${{ matrix.os }} - - strategy: - matrix: - os: [ubuntu-latest, windows-latest, macos-latest] - # Python Version Policy: - # Test only actively supported bugfix versions (not security-fix-only). - # As of January 2026, Python 3.13 is the latest bugfix-supported version. - # Python 3.12 stopped receiving bugfixes in April 2025 (security-fix-only). - # Python versions in "security fix only" phase are NOT publicly installable - # with security updates—they require building from source. - # Update this when upstream support changes (typically annually around October). - # See: https://devguide.python.org/versions/ - python-version: ['3.13'] - - steps: - - name: Checkout repository - uses: actions/checkout@v6 - - - name: Setup Python ${{ matrix.python-version }} - uses: actions/setup-python@v6 - with: - python-version: ${{ matrix.python-version }} - cache: 'pip' - - - name: Install dependencies - run: | - python -m pip install --upgrade pip - # Install project with dev dependencies - # Adjust this if your project uses a different installation method - pip install -e ".[dev]" - - - name: Run tests with coverage - run: pytest tests/ -v --cov --cov-report=term-missing - - # Codecov Integration (Optional, Commented Out) - # - # To enable code coverage reporting with Codecov: - # 1. Uncomment the step below - # 2. For public repositories: No token required - # 3. For private repositories: Add CODECOV_TOKEN to repository secrets - # - Go to: https://codecov.io/ and sign in with GitHub - # - Find your repository and copy the token - # - Add to GitHub: Settings > Secrets and variables > Actions > New repository secret - # - Name: CODECOV_TOKEN, Value: (paste token) - # - # Documentation: https://docs.codecov.com/docs/github-action - # - # - name: Upload coverage to Codecov - # uses: codecov/codecov-action@v4 - # with: - # token: ${{ secrets.CODECOV_TOKEN }} # Required for private repos only - # files: ./coverage.xml - # flags: unittests - # name: codecov-umbrella - # fail_ci_if_error: false diff --git a/.github/workflows/terraform-ci.yml b/.github/workflows/terraform-ci.yml deleted file mode 100644 index a84fd69..0000000 --- a/.github/workflows/terraform-ci.yml +++ /dev/null @@ -1,229 +0,0 @@ -# Terraform CI Workflow -# -# Purpose: Run format checking, validation, linting, testing, and optional security -# scanning for Terraform code in a single consolidated workflow. -# -# Design Decisions: -# -# 1. Single Workflow with Job Dependencies: -# This workflow combines format, validate, lint, test, and security into a single -# workflow file using the `needs:` keyword for job dependencies. This design -# follows the pattern established in python-ci.yml. -# -# Benefits of this approach: -# - All jobs appear directly in the PR checks list (not hidden in Actions tab) -# - Easy to configure branch rulesets that require specific jobs -# - Later jobs don't run if earlier ones fail (saves CI minutes) -# - Simpler maintenance with a single workflow file -# -# 2. Job Dependencies: -# - validate needs format (don't validate poorly-formatted code) -# - lint needs validate (don't lint invalid code) -# - test needs validate (don't test invalid code) -# - security needs validate (runs in parallel with lint) -# This ensures we don't waste time on later checks if earlier ones fail. -# -# 3. Template Repository Consideration: -# This workflow does NOT use path-based filtering because this is a template -# repository. Path filtering would prevent the workflow from being properly tested -# and copied by users of the template. Template repositories should run all -# workflows on all changes to ensure accuracy when the template is used. -# -# 4. Terraform Version Policy: -# Pin to a specific Terraform version for reproducibility. All jobs use -# the latest stable Terraform version (1.14.4). The test framework requires -# Terraform 1.6.0+ and mock_provider requires 1.7.0+, both satisfied by -# the latest version. Review and update quarterly per TEMPLATE_MAINTENANCE.md. -# -# 5. TFLint Version Policy: -# Pin to a specific TFLint version (v0.51.1) for reproducibility. Update when -# new rules or fixes are needed. -# -# 6. Dependency Caching: -# This workflow uses caching to improve performance: -# - Terraform providers: Cached based on .terraform.lock.hcl hash -# - TFLint plugins: Cached based on .tflint.hcl hash -# Expected time savings: 30-60 seconds per CI run for provider downloads. -# -# Note: This workflow is optional. Remove if your project doesn't use Terraform. - -name: Terraform CI - -on: - push: - branches: ["**"] - pull_request: - branches: ["**"] - workflow_dispatch: - -permissions: - contents: read - -jobs: - format: - name: Format Check - runs-on: ubuntu-latest - - steps: - - name: Checkout repository - uses: actions/checkout@v6 - - - name: Setup Terraform - uses: hashicorp/setup-terraform@v4 - with: - terraform_version: "1.14.4" - - - name: Terraform Format Check - run: terraform fmt -check -recursive -diff - - validate: - name: Validate - needs: format - runs-on: ubuntu-latest - - steps: - - name: Checkout repository - uses: actions/checkout@v6 - - - name: Setup Terraform - uses: hashicorp/setup-terraform@v4 - with: - terraform_version: "1.14.4" - - - name: Cache Terraform Providers - uses: actions/cache@v5 - with: - path: ~/.terraform.d/plugin-cache - key: terraform-providers-${{ hashFiles('**/.terraform.lock.hcl') }} - restore-keys: | - terraform-providers- - - - name: Configure Provider Cache - run: | - mkdir -p ~/.terraform.d/plugin-cache - echo "plugin_cache_dir = \"$HOME/.terraform.d/plugin-cache\"" > ~/.terraformrc - - - name: Find Terraform directories - id: find-tf-dirs - run: | - # Find all directories containing .tf files - TF_DIRS=$(find . -name '*.tf' -exec dirname {} \; | sort -u | tr '\n' ' ') - echo "dirs=${TF_DIRS}" >> "$GITHUB_OUTPUT" - - - name: Terraform Init and Validate - run: | - # Validate each Terraform directory - for dir in ${{ steps.find-tf-dirs.outputs.dirs }}; do - echo "=== Validating $dir ===" - cd "$dir" - terraform init -backend=false - terraform validate - cd - > /dev/null - done - - lint: - name: Lint (TFLint) - needs: validate - runs-on: ubuntu-latest - - steps: - - name: Checkout repository - uses: actions/checkout@v6 - - - name: Cache TFLint Plugins - uses: actions/cache@v5 - with: - path: ~/.tflint.d/plugins - key: tflint-plugins-${{ hashFiles('.tflint.hcl') }} - restore-keys: | - tflint-plugins- - - - name: Setup TFLint - uses: terraform-linters/setup-tflint@v6 - with: - tflint_version: v0.51.1 - - - name: Init TFLint - run: tflint --init - - - name: Run TFLint - run: tflint --recursive --config "$(pwd)/.tflint.hcl" - - test: - name: Test - needs: validate - runs-on: ubuntu-latest - - steps: - - name: Checkout repository - uses: actions/checkout@v6 - - - name: Setup Terraform - uses: hashicorp/setup-terraform@v4 - with: - terraform_version: "1.14.4" - - - name: Cache Terraform Providers - uses: actions/cache@v5 - with: - path: ~/.terraform.d/plugin-cache - key: terraform-providers-${{ hashFiles('**/.terraform.lock.hcl') }} - restore-keys: | - terraform-providers- - - - name: Configure Provider Cache - run: | - mkdir -p ~/.terraform.d/plugin-cache - echo "plugin_cache_dir = \"$HOME/.terraform.d/plugin-cache\"" > ~/.terraformrc - - - name: Check for test files - id: check-tests - run: | - # Exclude templates/ directory - those are example files, not runnable tests - if find . -path './templates' -prune -o -name '*.tftest.hcl' -print | grep -q .; then - echo "has_tests=true" >> "$GITHUB_OUTPUT" - else - echo "has_tests=false" >> "$GITHUB_OUTPUT" - fi - - - name: Find Terraform directories with tests - if: steps.check-tests.outputs.has_tests == 'true' - id: find-test-dirs - run: | - # Find directories containing .tftest.hcl files, excluding templates/ - TEST_DIRS=$(find . -path './templates' -prune -o -name '*.tftest.hcl' -printf '%h\n' | sort -u | tr '\n' ' ') - echo "dirs=${TEST_DIRS}" >> "$GITHUB_OUTPUT" - - - name: Run Terraform Tests - if: steps.check-tests.outputs.has_tests == 'true' - run: | - for dir in ${{ steps.find-test-dirs.outputs.dirs }}; do - echo "=== Testing $dir ===" - cd "$dir" - terraform init - terraform test -verbose - cd - > /dev/null - done - - - name: Skip tests (no test files found) - if: steps.check-tests.outputs.has_tests == 'false' - run: echo "No Terraform test files (.tftest.hcl) found. Skipping tests." - - # Optional: Security scanning with tfsec - # This job runs in parallel with lint (both depend on validate). - # Set continue-on-error to true for advisory mode, false for blocking mode. - # Uncomment to enable security scanning. - # security: - # name: Security Scan - # needs: validate - # runs-on: ubuntu-latest - # continue-on-error: true # Advisory mode: report findings but don't fail workflow - # - # steps: - # - name: Checkout repository - # uses: actions/checkout@v6 - # - # - name: Run tfsec - # uses: aquasecurity/tfsec-action@v1.0.3 - # with: - # soft_fail: true # Set to false to fail on findings diff --git a/.gitignore b/.gitignore index 4b4240a..dd38d37 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,7 @@ # Dependencies node_modules/ -# Python +# Python tool environments and caches __pycache__/ *.py[cod] *$py.class @@ -22,9 +22,40 @@ htmlcov/ # Pre-commit .pre-commit-cache/ -# OS files +# macOS and local lab artifacts .DS_Store Thumbs.db +*.ipsw +*.dmg +*.iso +*.app/ +*.pvm/ +*.utm/ +evidence/ +Evidence/ +.maclab/ +media-cache/ +.media-cache/ +vm-cache/ +.vm-cache/ +screenshots/ +recordings/ +*.mov +*.mp4 +*.heic +*.png +*.jpg +*.jpeg +testResults.xml + +# Secrets and local credentials +.envrc +*.pem +*.p12 +*.pfx +*.key +id_rsa* +*.token # IDE .vscode/* diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 1998fb4..eb7bf3a 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -17,9 +17,9 @@ # Install: pip install pre-commit && pre-commit install # Run: pre-commit run --all-files # -# Note: This file configures pre-commit hooks for all projects. It handles -# Python formatting/linting and Markdown validation. Remove only if your -# project does not use pre-commit at all. +# Note: This file configures pre-commit hooks for macOSLab repository hygiene, +# Markdown validation, data-file validation, and schema validation. Remove only +# if the repository stops using pre-commit at all. repos: # General hooks @@ -53,20 +53,6 @@ repos: hooks: - id: actionlint - # Python formatting (remove if not using Python) - - repo: https://github.com/psf/black - rev: 26.3.1 - hooks: - - id: black - args: [--line-length=100] - - # Python linting (remove if not using Python) - - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.15.12 - hooks: - - id: ruff-check - args: [--fix, --line-length=100] - # JSON Schema validation # # The hooks below cover three distinct categories: @@ -81,10 +67,9 @@ repos: # Invalid example data files under # `schemas/examples/example-config/invalid/` are intentionally NOT # wired into a normal pre-commit hook because they are expected to - # fail validation. They are exercised by - # `tests/test_schema_examples.py`, which auto-discovers - # schema/example pairs and asserts that valid examples pass and - # invalid examples fail. + # fail validation. macOSLab does not keep Python project tests in Phase + # 0, so the invalid examples remain reference fixtures until the real + # evidence schema replaces the worked example in Phase 7. # # B. Schema self-validation # Validates the worked-example schema itself against its declared @@ -150,13 +135,3 @@ repos: rev: v0.22.1 hooks: - id: markdownlint-cli2 - - # Terraform formatting and validation (remove if not using Terraform) - - repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.105.0 - hooks: - - id: terraform_fmt - - id: terraform_validate - - id: terraform_tflint - args: - - --args=--config=__GIT_WORKING_DIR__/.tflint.hcl diff --git a/.tflint.hcl b/.tflint.hcl deleted file mode 100644 index 67689bd..0000000 --- a/.tflint.hcl +++ /dev/null @@ -1,97 +0,0 @@ -# TFLint Configuration -# https://github.com/terraform-linters/tflint - -config { - format = "compact" - - call_module_type = "local" - force = false - disabled_by_default = false -} - -# Terraform plugin for general linting -plugin "terraform" { - enabled = true - preset = "recommended" -} - -# AWS-specific rules (uncomment if using AWS) -# plugin "aws" { -# enabled = true -# version = "0.32.0" -# source = "github.com/terraform-linters/tflint-ruleset-aws" -# } - -# Azure-specific rules (uncomment if using Azure) -# plugin "azurerm" { -# enabled = true -# version = "0.27.0" -# source = "github.com/terraform-linters/tflint-ruleset-azurerm" -# } - -# Google Cloud-specific rules (uncomment if using GCP) -# plugin "google" { -# enabled = true -# version = "0.30.0" -# source = "github.com/terraform-linters/tflint-ruleset-google" -# } - -# ============================================================================ -# Rule Configuration -# ============================================================================ - -# Enforce snake_case naming convention for all Terraform identifiers -rule "terraform_naming_convention" { - enabled = true - format = "snake_case" -} - -# Require descriptions for all variables -rule "terraform_documented_variables" { - enabled = true -} - -# Require descriptions for all outputs -rule "terraform_documented_outputs" { - enabled = true -} - -# Require type declarations for all variables -rule "terraform_typed_variables" { - enabled = true -} - -# Warn on deprecated syntax -rule "terraform_deprecated_interpolation" { - enabled = true -} - -# Warn on unused declarations -rule "terraform_unused_declarations" { - enabled = true -} - -# Require version constraints for modules -rule "terraform_module_version" { - enabled = true -} - -# Require version constraints for providers -rule "terraform_required_providers" { - enabled = true -} - -# Require required_version for Terraform -rule "terraform_required_version" { - enabled = true -} - -# Enforce standard file structure -rule "terraform_standard_module_structure" { - enabled = true -} - -# Workspace usage warning -rule "terraform_workspace_remote" { - enabled = true -} diff --git a/.vscode/settings.json b/.vscode/settings.json index 748d1ff..1e75cbd 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -1,6 +1,6 @@ { - "window.title": "Go to .vscode/settings.json and make this the name of the repo", - "[powershell]": { - "files.encoding": "utf8" - } + "window.title": "macOSLab", + "[powershell]": { + "files.encoding": "utf8" + } } diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index def7517..9fca645 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -43,7 +43,7 @@ We agree to restrict the following behaviors in our community. Instances, threat Tensions can occur between community members even when they are trying their best to collaborate. Not every conflict represents a code of conduct violation, and this Code of Conduct reinforces encouraged behaviors and norms that can help avoid conflicts and minimize harm. -When an incident does occur, it is important to report it promptly. To report a possible violation, contact us via: [INSERT CONTACT METHOD] +When an incident does occur, it is important to report it promptly. To report a possible violation, contact the repository owners using the contact links on their GitHub profiles. Community Moderators take reports of violations seriously and will make every effort to respond in a timely manner. They will investigate all reports of code of conduct violations, reviewing messages, logs, and recordings, or interviewing witnesses and other participants. Community Moderators will keep investigation and enforcement actions as transparent as possible while prioritizing safety and confidentiality. In order to honor these values, enforcement actions are carried out in private with the involved parties, but communicating to the whole community may be part of a mutually agreed upon resolution. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7282129..c5e2f4d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,386 +1,69 @@ -# Contributing to This Project + +# Contributing to macOSLab - - -Thank you for your interest in contributing! This document provides guidelines for contributing to this repository. - -## Python Version Requirements - -**Important:** Contributors and maintainers must use a Python version that is **currently receiving bugfixes** from the Python core team. - -### Why This Matters - -- Python versions in "security fix only" phase are **not publicly installable** with security updates—they require building from source with manually applied patches -- This policy ensures all contributors can install and use the same Python version with current security updates -- It maintains consistency across development environments and CI - -### Current Requirements - -This project requires a Python version that is currently in "bugfix" status according to the Python core team. - -See the [Python Developer's Guide - Versions](https://devguide.python.org/versions/) page for current version status. - -> **Template adopters:** Customize this section based on your project's specific Python version requirements. You may specify a minimum version (e.g., "Python 3.11+") or reference your project's own support policy. - -### When to Update - -Check the [Python Developer's Guide - Versions](https://devguide.python.org/versions/) page annually (typically around October when new Python versions are released). Update the minimum required version when upstream support changes. - -**Do not default to or require unsupported Python versions in code, documentation, or configuration files.** +Thank you for contributing to `macOSLab`. This repository is a PowerShell 7.4+ and Markdown project for reproducible Apple-silicon macOS VM labs used in Intune policy testing. ## Development Setup -### 1. Clone the Repository - - - -```bash -git clone https://github.com/OWNER/REPO.git -cd REPO -``` - -### 2. Install Node.js Dependencies (for Markdown linting) +Clone the repository and install the Node.js tooling used for Markdown linting: ```bash +git clone https://github.com/franklesniak/macOSLab.git +cd macOSLab npm install ``` -This installs Node.js dependencies for markdown linting scripts. Git hooks are managed by pre-commit (see step 4 below). - -### Git Hooks - -This repository uses pre-commit for git hooks. Configured hooks include: - -- **Formatting**: Black (Python), trailing whitespace, end-of-file fixer -- **Linting**: Ruff (Python), markdownlint (Markdown) -- **Data-file validation**: `check-json` (strict `.json` files only — see note below), `check-yaml`, `yamllint` (configured by `.yamllint.yml`), `actionlint` (GitHub Actions workflows) -- **Schema validation**: `check-jsonschema` and `check-metaschema` for the template's worked-example schema (`schemas/example-config.schema.json`) and its valid example data under `schemas/examples/example-config/valid/`; downstream repositories MAY add additional `check-jsonschema` hook entries for their own schema-backed file families. See [`schemas/README.md`](schemas/README.md) for the worked example and the canonical downstream removal checklist. -- **Safety**: Large file detection - -> **`check-json` validates strict `.json` only.** It does **not** validate `.jsonc`. JSONC files are allowed only when the consuming tool supports JSONC; downstream repositories that need stricter `.jsonc` enforcement should add **JSONC-aware tooling** rather than retrofitting `check-json`. See [`.github/instructions/json.instructions.md`](.github/instructions/json.instructions.md) for the full JSON/JSONC dialect policy and [`.github/instructions/yaml.instructions.md`](.github/instructions/yaml.instructions.md) for YAML authoring standards. -> -> **`actionlint` first-run-on-restricted-networks caveat.** The `actionlint` pre-commit hook builds the `actionlint` binary from source on first install, which downloads a Go toolchain. On networks that block Go module downloads (corporate proxies, air-gapped environments), the first-run install can fail. CI is the shared enforcement environment, so contributors who hit a network restriction locally can rely on CI to enforce this hook. The same caveat is documented inline in `.pre-commit-config.yaml` and in `.github/TEMPLATE_DESIGN_DECISIONS.md`. - -If you need to bypass hooks temporarily (not recommended): - -```bash -git commit --no-verify -m "your message" -``` - -### Markdown Linting - -Run markdown linting manually: - -```bash -npm run lint:md # Lint all markdown files -npm run lint:md:nested # Lint nested markdown blocks in docs -``` - -### 3. Install Python (if working with Python code) - -Ensure you have a Python version that is currently receiving bugfixes (see [Python Version Requirements](#python-version-requirements) above): - -```bash -python --version # Should be a version currently receiving bugfixes -``` - -### 4. Install pre-commit (Globally) - -**Important:** `pre-commit` is intentionally **NOT** included as a project dev dependency. Install it globally: - -#### Option 1: Using pip (recommended for most users) - -```bash -pip install pre-commit -``` - -#### Option 2: Using pipx (recommended for tool isolation) - -```bash -pipx install pre-commit -``` - -#### Why Not a Dev Dependency? - -- `pre-commit` is a **development tool**, not a project runtime or test dependency -- It manages its own isolated environments for hooks (including Python tools like Black and Ruff) -- Installing it globally or via `pipx` keeps it separate from project dependencies -- This is the standard practice in the Python community -- CI workflows install `pre-commit` separately in their own steps - -### 5. Install Pre-commit Hooks - -After installing `pre-commit` globally, set up the hooks in your local repository: +Install `pre-commit` globally with your preferred tool. Python may be required by some pre-commit hook environments, but Python is development tooling here, not project source code. ```bash pre-commit install ``` -This configures Git to automatically run pre-commit hooks before each commit. - -### 6. Run Pre-commit Manually - -To run all pre-commit hooks on all files (recommended before submitting a PR): - -```bash -pre-commit run --all-files -``` - -To run pre-commit on staged files only: - -```bash -pre-commit run -``` - -## Code Quality Standards - -### Pre-commit Discipline - -**⚠️ CRITICAL: Always run pre-commit checks before committing code.** - -Pre-commit hooks are NOT optional. They enforce: - -- Code formatting (Black for Python, markdownlint for Markdown) -- Linting (Ruff for Python) -- Trailing whitespace removal -- End-of-file fixes -- Data-file validation (`check-json` for strict `.json`, `check-yaml`, `yamllint`, `actionlint` for GitHub Actions) -- Schema validation (`check-jsonschema`) for schema-backed file families where configured - -> **Network and dialect notes:** -> -> - First-run hook setup may require network access (for example, `actionlint` downloads a Go toolchain on first install; see the restricted-networks caveat above). -> - `check-json` validates strict `.json` files only and does **not** validate `.jsonc`. Use **JSONC-aware tooling** if `.jsonc` files in your project warrant stricter enforcement. -> - CI runs the same hooks and is the shared enforcement environment; auto-fixes produced by the hooks **must** be committed with the related change (do not push code that fails pre-commit and rely on a follow-up "fix formatting" commit). - -See `.pre-commit-config.yaml` for the complete list of configured hooks. See [`.github/instructions/json.instructions.md`](.github/instructions/json.instructions.md) and [`.github/instructions/yaml.instructions.md`](.github/instructions/yaml.instructions.md) for the JSON and YAML authoring policies that these hooks enforce. - -**Workflow:** - -1. Make your code changes -2. Run `pre-commit run --all-files` -3. Review and commit ALL auto-fixes as part of your change -4. Push to GitHub - -**If pre-commit CI fails:** - -1. Pull the latest branch -2. Run `pre-commit run --all-files` locally -3. Integrate the auto-fixes into the commit that introduced the change (for example, `git commit --amend` for the most recent commit, or `git commit --fixup=` followed by `git rebase -i --autosquash` for an earlier commit). Do **not** create a separate "Apply pre-commit auto-fixes" commit — auto-fixes belong in the same commit as the related change. -4. Push again (force-push is required when amending or rewriting history on a branch you have already pushed) - -**CI is a safety net, not a substitute for local checks.** - -### Language-Specific Guidelines - -This repository includes comprehensive coding standards for multiple languages and data file formats: - -- **Python:** `.github/instructions/python.instructions.md` -- **PowerShell:** `.github/instructions/powershell.instructions.md` -- **Terraform:** `.github/instructions/terraform.instructions.md` -- **Markdown/Documentation:** `.github/instructions/docs.instructions.md` -- **JSON/JSONC:** `.github/instructions/json.instructions.md` -- **YAML:** `.github/instructions/yaml.instructions.md` -- **Git attributes:** `.github/instructions/gitattributes.instructions.md` - -These standards apply to all contributions and are enforced by the repository's pre-commit hooks and CI workflows. AI coding agents (such as GitHub Copilot, Claude Code, Codex, and Gemini) are configured to read these instruction files when generating or editing code, but enforcement of the standards on every commit comes from the tooling listed above, not from any individual agent. - -### Data-File and Schema Validation - -JSON, YAML, and GitHub Actions workflow files are first-class file types in this template. Contributor expectations: - -- Run `pre-commit run --all-files` before opening a PR. -- JSON files **must** pass `check-json` (strict `.json` only — `.jsonc` is intentionally not validated; see the dialect note above). -- YAML files **must** pass `check-yaml` and `yamllint` (configured by `.yamllint.yml`). -- GitHub Actions workflow files **must** pass `actionlint`. -- Schema-backed files **must** pass `check-jsonschema` (and the schema itself **must** pass `check-metaschema` where wired). -- The schema example contracts under `schemas/examples//{valid,invalid}/` are tested by [`tests/test_schema_examples.py`](tests/test_schema_examples.py); when changing a schema or its example fixtures, run `pytest tests/test_schema_examples.py -v` to confirm the contract still holds. - -See [`schemas/README.md`](schemas/README.md) for schema conventions and the canonical downstream removal checklist for the worked example. - -### CI Workflows - -This repository includes several GitHub Actions workflows that run automatically: - -| Workflow | File | Purpose | -| --- | --- | --- | -| Python CI | `.github/workflows/python-ci.yml` | Runs pre-commit, mypy (type checking), and pytest on Python files | -| Auto-fix Pre-commit | `.github/workflows/auto-fix-precommit.yml` | Automatically commits pre-commit auto-fixes on pushes to `copilot/**` branches authored by `copilot-swe-agent[bot]` (optional) | -| Markdown Lint | `.github/workflows/markdownlint.yml` | Validates markdown formatting | -| PowerShell CI | `.github/workflows/powershell-ci.yml` | Runs PSScriptAnalyzer on PowerShell files | -| Data CI | `.github/workflows/data-ci.yml` | Runs `check-json`, `check-yaml`, `yamllint`, `actionlint`, `check-jsonschema`, and `check-metaschema` against JSON/YAML/GitHub Actions data files | - -The **Auto-fix Pre-commit** workflow is scoped specifically to the GitHub Copilot Coding Agent: it triggers only on pushes to `copilot/**` branches authored by `copilot-swe-agent[bot]`, and automatically commits any pre-commit auto-fixes back onto that branch. Human-authored PRs and PRs on non-`copilot/**` branches are not affected; their authors must run `pre-commit run --all-files` locally and integrate the fixes themselves before pushing. - -### Workflow Version Pinning - -When editing files under `.github/workflows/`, follow the [**Workflow Version Pinning**](.github/copilot-instructions.md#workflow-version-pinning) rule in the repo-wide constitution. In short: - -- Keep third-party action versions directly in `uses:` lines (for example, `actions/checkout@v6`) so Dependabot can update them. -- Do **not** mirror a `uses:` version into a workflow-level `env:` variable, comment, cache key, file path, or shell literal — Dependabot will not rewrite those mirrors, and they will silently drift. -- For tool versions that Dependabot does not manage (for example, `terraform_version` or `tflint_version` passed to setup actions), a workflow-level `env:` value is a fine single source of truth across multiple steps. -- If a Dependabot-managed dependency genuinely cannot be expressed without duplication, add a narrowly scoped `.github/dependabot.yml` `ignore:` entry with a YAML comment explaining why. - -See the deep link above for the full rule, including the wrapper-action vs. installed-tool distinction and concrete examples from this repository. - -## Making Changes - -### 1. Create a Branch - -```bash -git checkout -b your-feature-branch -``` - -### 2. Make Your Changes +## Required Checks -Follow the coding standards for the language(s) you're working with. - -### 3. Run Pre-commit Hooks +Run the repository checks before opening a pull request: ```bash +npm run lint:md +npm run lint:md:nested pre-commit run --all-files ``` -Fix any issues that are reported. - -### 4. Run Tests - -Before submitting a pull request, ensure all tests pass locally. - -#### Python Tests - -```bash -# Install dev dependencies -pip install -e ".[dev]" - -# Run tests with coverage -pytest tests/ -v --cov --cov-report=term-missing - -# Run type checks -mypy src/ tests/ -``` - -#### PowerShell Tests +For PowerShell changes, also run: ```powershell -# Install Pester if not already installed -Install-Module -Name Pester -MinimumVersion 5.0 -Force -Scope CurrentUser - -# Run all Pester tests +Invoke-ScriptAnalyzer -Path . -Settings .github/linting/PSScriptAnalyzerSettings.psd1 Invoke-Pester -Path tests/ -Output Detailed ``` -#### Test Requirements - -- **Python:** New functionality should include pytest tests in `tests/` -- **PowerShell:** New functions should include Pester tests in `tests/PowerShell/` -- All tests must pass on the CI matrix (Ubuntu, Windows, macOS) - -### 5. Commit Your Changes - -```bash -git add . -git commit -m "Your descriptive commit message" -``` - -Pre-commit hooks will run automatically. If they make changes, review them and commit again. - -### 6. Push Your Branch - -```bash -git push origin your-feature-branch -``` - -### 7. Open a Pull Request - -Open a PR on GitHub and fill out the PR template checklist. - -## Pull Request Guidelines - -When submitting a pull request: - -- [ ] Confirm minimum Python version (if applicable) complies with bugfix support policy -- [ ] Confirm `pre-commit run --all-files` passes locally -- [ ] Include tests for new functionality -- [ ] Update documentation as needed -- [ ] Ensure all CI checks pass - -## Questions or Issues? - -If you have questions or encounter issues: - - - -1. Check existing [Issues](https://github.com/OWNER/REPO/issues) -2. Review the documentation in `.github/instructions/` -3. Open a new issue with a clear description of the problem - -## License - - - -By contributing to this project, you agree that your contributions will be licensed under the same license as the project (MIT License). - -## For Template Users - - - -This repository is a template designed for projects that use GitHub Copilot for AI-assisted development. - -### Understanding the Instruction Files - -The `.github/instructions/` directory contains coding standards that guide GitHub Copilot's code generation: +Pre-commit auto-fixes must be committed with the related change. Do not land a standalone formatting-only or lint-only commit. -- **`docs.instructions.md`** - Documentation and Markdown writing standards -- **`powershell.instructions.md`** - PowerShell coding conventions (OTBS style, v1.0 compatibility patterns) -- **`python.instructions.md`** - Python coding standards (PEP 8, type hints, testing patterns) +## Project Standards -These instruction files are automatically applied by GitHub Copilot based on the file patterns specified in each file's front matter. +- PowerShell code targets PowerShell 7.4 or newer. +- Pester tests use Pester 5.7.1 syntax and conventions. +- Markdown follows the repository documentation style guide. +- JSON and YAML files must pass the data-file hooks configured in `.pre-commit-config.yaml`. +- Do not commit secrets, tenant identifiers, recovery keys, tokens, production policy names, screenshots, recordings, local VM bundles, or raw evidence output. +- Do not modify protected instruction files unless the repository owner explicitly authorizes that exact protected-file change in the current task. -### Customizing for Your Project +## Security Reports -You can customize these instruction files for your project's specific conventions: +Do not open public issues for security vulnerabilities. Use [GitHub private vulnerability reporting](https://github.com/franklesniak/macOSLab/security/advisories/new). -1. Edit the instruction files to match your team's coding standards -2. Remove instruction files for languages you don't use -3. Add new instruction files for additional languages as needed +## Pull Requests -### First-Time Setup Validation +Use the pull request template and include: -After creating a new repository from this template, see the [Validating Your New Repository](README.md#validating-your-new-repository) section in the README for guidance on verifying your setup. +- What changed and why. +- The validation commands you ran. +- Any skipped manual GitHub-side actions. +- Any requested owner authorization for protected instruction files or repository settings. diff --git a/COPILOT_CHAT_PROMPTS.md b/COPILOT_CHAT_PROMPTS.md deleted file mode 100644 index ae3598a..0000000 --- a/COPILOT_CHAT_PROMPTS.md +++ /dev/null @@ -1,84 +0,0 @@ -# Copilot Chat Prompts for Template Adoption - -This file contains ready-to-use prompts for GitHub Copilot Chat to help you analyze and adopt template features into an existing repository. - -## Prerequisites - -- You have an existing repository where you want to adopt template features -- You have access to GitHub Copilot Chat -- You have read through [GETTING_STARTED_EXISTING_REPO.md](GETTING_STARTED_EXISTING_REPO.md) to understand what the template provides - -## Model Recommendation - -**Use Claude Opus 4.5 or better for these prompts.** - -These prompts require the model to: - -- Read and synthesize multiple large documentation files -- Cross-reference configurations across two repositories -- Produce detailed, structured output with file-level specificity - -Claude Opus 4.5 handles these tasks reliably. Other models may produce incomplete or less accurate results. - -To select your model in GitHub Copilot Chat, click the model name in the chat interface and choose from the available options. - -## Usage - -Run these prompts in sequence in GitHub Copilot Chat. Replace `OWNER/REPO` with your actual organization/username and repository name. - ---- - -### Adding Repositories to Copilot Chat Context - -> Go to [GitHub Copilot Chat](https://github.com/copilot), then click `Add repositories, files, and spaces` > `Repositories...`. Search for `franklesniak/copilot-repo-template` and check its checkbox. Then, search for your destination repository (`OWNER/REPO`) and check its checkbox. Click outside the dialog (or press Escape) to return to the prompt input box. - ---- - -## Prompt 1: Gap Analysis - -Use this prompt to analyze your repository against the template and identify what's missing or incomplete. - -```markdown -I want to implement `franklesniak/copilot-repo-template` in the existing repository `OWNER/REPO`. - -Please review the following files in `franklesniak/copilot-repo-template`: - -- `GETTING_STARTED_EXISTING_REPO.md` (required configurations) -- `OPTIONAL_CONFIGURATIONS.md` (optional configurations) - -Then review the current state of `OWNER/REPO` and produce a gap analysis that: - -1. Lists every required configuration item and its completion status -2. Lists every optional configuration item and whether it's been customized or uses defaults -3. Identifies any gaps, errors, or incomplete implementations - -For each gap, indicate whether it's required or optional, and provide the specific file(s) and line(s) affected. -``` - ---- - -## Prompt 2: Actionable Checklist - -After reviewing the gap analysis from Prompt 1, use this prompt to generate a prioritized list of actions. - -```markdown -Based on the gap analysis, produce a prioritized checklist of configuration items I need to address or consider addressing. - -For each item, include: - -- Priority level (Required / Recommended / Optional) -- File(s) affected -- Current state -- Recommended change (with code snippets where helpful) -- Impact/rationale for making the change - -Group items by priority level. -``` - ---- - -## Tips - -- **Review the gap analysis before proceeding to Prompt 2.** You may have questions or context to add. -- **Provide context in follow-up prompts.** The more specific you are about your project's constraints (private vs. public repo, team size, languages used, etc.), the better the recommendations. -- **You don't need to address every optional item.** The template defaults are sensible for most projects. Focus on required items first, then consider optional items based on your team's needs. diff --git a/GETTING_STARTED_EXISTING_REPO.md b/GETTING_STARTED_EXISTING_REPO.md deleted file mode 100644 index f44fcf1..0000000 --- a/GETTING_STARTED_EXISTING_REPO.md +++ /dev/null @@ -1,2023 +0,0 @@ -# Getting Started: Adding Template Features to an Existing Repository - -This guide walks you through adopting features from `franklesniak/copilot-repo-template` into your **existing repository**. Unlike creating a new repository from a template, integrating template features into an existing project requires careful planning to avoid conflicts with your current configuration. - -> **Looking to create a new repository?** See [GETTING_STARTED_NEW_REPO.md](GETTING_STARTED_NEW_REPO.md) instead. - -**Recommendation:** Read through this entire guide before starting. This helps you plan which features to adopt and in what order. - -**Estimated time to complete:** 15-60 minutes (varies based on scope of adoption) - ---- - -## Table of Contents - -- [What This Template Provides](#what-this-template-provides) -- [Prerequisites](#prerequisites) - - [Existing Repository Requirements](#existing-repository-requirements) - - [Tools Needed](#tools-needed) -- [Planning Your Adoption](#planning-your-adoption) - - [Feature Decision Matrix](#feature-decision-matrix) - - [Recommended Adoption Order](#recommended-adoption-order) - - [Repo Layout Examples](#repo-layout-examples) -- [Getting the Template Files](#getting-the-template-files) - - [Files to Skip (Example/Demonstration Code)](#files-to-skip-exampledemonstration-code) -- [Adopting Simple Standalone Files](#adopting-simple-standalone-files) - - [CODEOWNERS](#codeowners) - - [Dependabot](#dependabot) - - [Security Policy](#security-policy) - - [LICENSE File](#license-file) - - [Code of Conduct](#code-of-conduct) - - [VS Code Settings](#vs-code-settings) -- [Adopting Issue Templates](#adopting-issue-templates) - - [Full Adoption](#full-adoption-recommended-if-you-have-none) - - [Partial Adoption](#partial-adoption-if-you-have-existing-templates) - - [Customizing Area Dropdowns](#customizing-area-dropdowns) - - [Creating Required Labels](#creating-required-labels) -- [Adopting PR Template](#adopting-pr-template) - - [Simple Adoption](#simple-adoption) - - [Customization Needed](#customization-needed) - - [Merging with Existing PR Template](#merging-with-existing-pr-template) -- [Adopting GitHub Copilot Instructions](#adopting-github-copilot-instructions) - - [Main Instructions File](#main-instructions-file) - - [Modular Instructions](#modular-instructions) - - [Merging with Existing Copilot Instructions](#merging-with-existing-copilot-instructions) - - [Creating Instructions for Other Languages](#creating-instructions-for-other-languages) - - [Agent Instruction Files (Multi-Platform Support)](#agent-instruction-files-multi-platform-support) -- [Adopting Markdown Linting](#adopting-markdown-linting) - - [If You Don't Have package.json](#if-you-dont-have-packagejson) - - [If You Already Have package.json](#if-you-already-have-packagejson) - - [Copying the Configuration](#copying-the-configuration) - - [Testing Markdown Linting](#testing-markdown-linting) -- [Adopting Pre-commit Hooks](#adopting-pre-commit-hooks) - - [If You Don't Have Pre-commit Configured](#if-you-dont-have-pre-commit-configured) - - [If You Already Have Pre-commit Configured](#if-you-already-have-pre-commit-configured) - - [Customizing Hooks](#customizing-hooks) -- [Adopting JSON/YAML Toolchain](#adopting-jsonyaml-toolchain) -- [Adopting CI Workflows](#adopting-ci-workflows) - - [Understanding Workflow Dependencies](#understanding-workflow-dependencies) - - [Markdown Lint Workflow](#markdown-lint-workflow) - - [Auto-fix Pre-commit Workflow](#auto-fix-pre-commit-workflow) - - [Placeholder Check Workflow](#placeholder-check-workflow) - - [Python CI Workflow](#python-ci-workflow) - - [If You Already Have pyproject.toml](#if-you-already-have-pyprojecttoml) - - [PowerShell CI Workflow](#powershell-ci-workflow) - - [Merging with Existing CI](#merging-with-existing-ci) -- [Adopting PSScriptAnalyzer Configuration](#adopting-psscriptanalyzer-configuration) - - [Copying the Configuration](#copying-the-configuration-1) - - [Using the Configuration](#using-the-configuration) - - [Customizing Rules](#customizing-rules) -- [Validation and Testing](#validation-and-testing) - - [Verify All Configurations Work](#verify-all-configurations-work) - - [Troubleshooting Common Issues](#troubleshooting-common-issues) -- [Cleanup and Documentation](#cleanup-and-documentation) - - [Files to Review After Adoption](#files-to-review-after-adoption) - - [Updating Your Project Documentation](#updating-your-project-documentation) -- [Migration Notes for Existing Terraform Repos](#migration-notes-for-existing-terraform-repos) -- [Next Steps](#next-steps) -- [Summary Checklist](#summary-checklist) - ---- - -## What This Template Provides - -This template repository includes several features you can adopt individually or together: - -| Feature | Description | -| --- | --- | -| **GitHub Copilot Instructions** | Comprehensive coding standards that guide AI-assisted development | -| **Issue Templates** | Structured templates for bug reports, feature requests, and documentation issues | -| **PR Template** | Checklist-based template for consistent pull request reviews | -| **CI Workflows** | GitHub Actions workflows for linting, testing, and validation | -| **Pre-commit Hooks** | Automated code quality checks before commits | -| **Linting Configurations** | Pre-configured settings for markdownlint and PSScriptAnalyzer | -| **Dependabot** | Automated dependency update monitoring | -| **CODEOWNERS** | Automatic reviewer assignment for pull requests | -| **Multi-Agent Support** | Instruction files for Claude Code, OpenAI Codex CLI, and Gemini Code Assist | - ---- - -## Prerequisites - -### Existing Repository Requirements - -Before adopting template features, ensure your repository meets these requirements: - -- [ ] **GitHub remote configured:** Your repository must be hosted on GitHub -- [ ] **Clean working tree:** All changes should be committed (`git status` shows no pending changes) -- [ ] **Working on a feature branch:** Recommended to avoid issues with your main branch - -**Create a feature branch for this work:** - -```bash -# Navigate to your repository -cd /path/to/your/repository - -# Ensure you're on your main branch and up to date -git checkout main -git pull origin main - -# Create a feature branch for template adoption -git checkout -b feature/adopt-template-features -``` - -### Tools Needed - -The tools you need depend on which features you plan to adopt: - -| Feature | Required Tools | -| --- | --- | -| Issue Templates, PR Template, CODEOWNERS, Dependabot | None (GitHub web interface only) | -| Copilot Instructions | None | -| Markdown Linting | Node.js | -| Pre-commit Hooks | Python, pre-commit | -| Python CI Workflow | Python | -| PowerShell CI Workflow | PowerShell | - -**Verify your installations:** - -**Windows (PowerShell):** - -```powershell -# Check Git version -git --version - -# Check Python version (if adopting Python features or pre-commit) -# Python 3.9+ is required for pre-commit hooks and CI workflows -python --version - -# Check pip version (if adopting Python features or pre-commit) -python -m pip --version - -# Check Node.js version (if adopting markdown linting) -node --version -``` - -**macOS/Linux/FreeBSD:** - -```bash -# Check Git version -git --version - -# Check Python version (if adopting Python features or pre-commit) -# Python 3.9+ is required for pre-commit hooks and CI workflows -python3 --version - -# Check pip version (if adopting Python features or pre-commit) -pip3 --version - -# Check Node.js version (if adopting markdown linting) -node --version -``` - -> **Need to install these tools?** See the [Prerequisites section in GETTING_STARTED_NEW_REPO.md](GETTING_STARTED_NEW_REPO.md#prerequisites) for detailed installation instructions. - ---- - -## Planning Your Adoption - -### Feature Decision Matrix - -Use this matrix to decide which features to adopt based on complexity and dependencies: - -| Feature | Files Involved | Dependencies | Complexity | -| --- | --- | --- | --- | -| Issue Templates | `.github/ISSUE_TEMPLATE/` | None | Low | -| PR Template | `.github/pull_request_template.md` | None | Low | -| Copilot Instructions | `.github/copilot-instructions.md`, `.github/instructions/` | None | Low | -| CODEOWNERS | `.github/CODEOWNERS` | None | Low | -| Dependabot | `.github/dependabot.yml` | None | Low | -| Security Policy | `SECURITY.md` | None | Low | -| VS Code Settings | `.vscode/settings.json` | None | Low | -| Markdown Linting | `.markdownlint.jsonc`, `package.json`, npm scripts | Node.js | Medium | -| Pre-commit Hooks | `.pre-commit-config.yaml` | Python, pre-commit | Medium | -| PowerShell CI Workflow | `.github/workflows/powershell-ci.yml` | PowerShell, Pester | Medium | -| PSScriptAnalyzer Config | `.github/linting/PSScriptAnalyzerSettings.psd1` | PowerShell | Low | -| Python CI Workflow | `.github/workflows/python-ci.yml` | Python project structure | High | -| Agent Instruction Files | `CLAUDE.md`, `AGENTS.md`, `GEMINI.md` | Adopt `.github/copilot-instructions.md` first | Low | - -### Recommended Adoption Order - -For the smoothest experience, adopt features in this order: - -1. **Simple standalone files first** — CODEOWNERS, Dependabot, Security Policy -2. **Issue/PR templates** — Low complexity, immediate usability improvements -3. **Copilot instructions** — Enhances AI-assisted development -4. **Linting configurations** — Establishes code quality standards -5. **CI workflows** — Most complex, most dependencies; adopt last - -> **Tip:** You don't need to adopt everything. Pick the features that provide the most value for your project. - -### Repo Layout Examples - -Before starting adoption, understand how your repository is structured. Here are two common patterns: - -**Root-Only Repo (Single Configuration):** - -A repository with a single Terraform or application configuration: - -```text -my-project/ -├── .github/ -│ ├── copilot-instructions.md -│ ├── instructions/ -│ └── workflows/ -├── main.tf # Primary configuration -├── variables.tf # Input variables -├── outputs.tf # Output values -├── versions.tf # Provider version constraints -├── .terraform.lock.hcl # Dependency lock file -└── README.md -``` - -**Module-Based Repo:** - -A repository containing reusable modules with examples and tests: - -```text -my-modules/ -├── .github/ -│ ├── copilot-instructions.md -│ ├── instructions/ -│ └── workflows/ -├── modules/ -│ └── vpc/ -│ ├── main.tf -│ ├── variables.tf -│ ├── outputs.tf -│ ├── versions.tf -│ └── README.md -├── examples/ -│ └── basic-vpc/ -│ ├── main.tf -│ └── README.md -├── tests/ -│ └── vpc.tftest.hcl -└── README.md -``` - -Choose your adoption approach based on your repository's structure. - ---- - -## Getting the Template Files - -Choose the method that works best for your situation: - -### Option A: Clone Template Separately (Recommended) - -This method gives you easy access to all template files for reference and copying. - -**Windows (PowerShell):** - -```powershell -# Create a temporary directory -mkdir $env:USERPROFILE\template-source -cd $env:USERPROFILE\template-source - -# Clone the template repository -git clone https://github.com/franklesniak/copilot-repo-template.git -``` - -**macOS/Linux/FreeBSD:** - -```bash -# Create a temporary directory -mkdir ~/template-source -cd ~/template-source - -# Clone the template repository -git clone https://github.com/franklesniak/copilot-repo-template.git -``` - -### Option B: Download as ZIP - -1. Navigate to -2. Click the green **Code** button -3. Select **Download ZIP** -4. Extract to a known location (e.g., `~/template-source/` or `C:\template-source\`) - -### Option C: Use GitHub's Web Interface - -Best for adopting just one or two files: - -1. Navigate to the file you want in the template repository -2. Click the file to view its contents -3. Click the **Raw** button to see the raw content -4. Copy the content and paste into a new file in your repository - -### Files to Skip (Example/Demonstration Code) - -The template repository includes example Python source code and tests that demonstrate coding standards. These files are intended for new repositories created from the template and should **NOT** be copied to existing repositories: - -| File/Directory | Purpose | Action | -| --- | --- | --- | -| `src/copilot_repo_template/` | Example Python package demonstrating coding standards | Do not copy | -| `tests/test_example.py` | Example pytest tests for the demo package | Do not copy | -| `tests/__init__.py` | Package marker with template-specific docstring | Do not copy | -| `pyproject.toml` | Configured for the template's example package | Copy only if you need a starting point, then heavily customize | - -If you already have Python tests in your existing repository, these template example files would conflict with your existing setup. - -**What you SHOULD copy:** - -- `.github/` directory contents (workflows, instructions, templates) -- Configuration files (`.markdownlint.jsonc`, `.pre-commit-config.yaml`) -- Community health files (`CONTRIBUTING.md`, `SECURITY.md`, `CODE_OF_CONDUCT.md`) -- `templates/` directory (reference templates for starting new test files) - -If your existing repository lacks Python project structure and you want to adopt the template's Python CI workflow, see the [Python CI Workflow](#python-ci-workflow) section below for guidance on setting up your own `pyproject.toml` and test structure. - ---- - -## Adopting Simple Standalone Files - -These files can be copied directly with minimal modifications. - -### CODEOWNERS - -The CODEOWNERS file automatically assigns reviewers to pull requests based on file paths. - -**Location:** `.github/CODEOWNERS` - -**Steps:** - -1. **If you don't have a CODEOWNERS file:** - - Copy `.github/CODEOWNERS` from the template to your `.github/` directory - - Replace `@OWNER` with your GitHub username or team name - -2. **If you already have a CODEOWNERS file:** - - Review the template's file for patterns you may want to add - - Merge entries manually, keeping your existing ownership rules - -**Example customization:** - -```text -# Default owners for everything in the repo -* @your-username - -# Workflow files require maintainer review -.github/workflows/ @your-username - -# Copilot instructions require maintainer review -.github/copilot-instructions.md @your-username -.github/instructions/ @your-username -``` - -### Dependabot - -Dependabot automatically creates pull requests to update your dependencies. - -**Location:** `.github/dependabot.yml` - -**Steps:** - -1. **If you don't have a dependabot.yml file:** - - Copy `.github/dependabot.yml` from the template - - Remove ecosystems you don't use: - - Remove the `npm` section if you don't use Node.js - - Remove the `pip` section if you don't use Python - - Keep the `github-actions` section (recommended for all repositories) - -2. **If you already have a dependabot.yml file:** - - Review the template's grouping strategy (groups minor/patch updates) - - Consider adopting the commit message prefix convention (`chore(deps)`) - - Merge any ecosystems you want to add - -**Example: Dependabot for a Python-only project:** - -```yaml -version: 2 -updates: - # Python dependencies (pyproject.toml) - - package-ecosystem: "pip" - directory: "/" - schedule: - interval: "weekly" - groups: - pip-minor-patch: - patterns: - - "*" - update-types: - - "minor" - - "patch" - commit-message: - prefix: "chore(deps)" - open-pull-requests-limit: 10 - - # GitHub Actions (workflows) - - package-ecosystem: "github-actions" - directory: "/" - schedule: - interval: "weekly" - groups: - actions-minor-patch: - patterns: - - "*" - update-types: - - "minor" - - "patch" - commit-message: - prefix: "chore(deps)" - open-pull-requests-limit: 10 -``` - -### Security Policy - -The security policy tells users how to report security vulnerabilities. - -**Location:** `SECURITY.md` (repository root) - -**Steps:** - -1. **If you don't have a SECURITY.md file:** - - Copy `SECURITY.md` from the template to your repository root - - Replace `[security contact email]` with your email address - - Or remove the email option and use only GitHub Security Advisories (for public repositories) - -2. **If you already have a SECURITY.md file:** - - Review the template's structure for ideas - - Consider adding sections you may be missing (response timeline, disclosure policy) - -> **Note:** Private vulnerability reporting via GitHub Security Advisories is only available for **public repositories**. If your repository is private, you must provide an email contact in SECURITY.md. Use a dedicated security email (e.g., `security@your-domain.com`) rather than a personal email when possible. - -### LICENSE File - -The template includes an MIT License file with the template author's name as the copyright holder. - -**Location:** `LICENSE` - -**Steps:** - -1. **If you want to use the MIT License:** - - Copy `LICENSE` from the template to your repository root - - Replace `Frank Lesniak` with your name or organization name (the copyright holder) - - Optionally update the copyright year to the current year or your project's start year - -2. **If you already have a LICENSE file:** - - Keep your existing license—no action needed - -3. **If you want a different license type:** - - See the [License Customization](OPTIONAL_CONFIGURATIONS.md#license-customization) section in `OPTIONAL_CONFIGURATIONS.md` for guidance on Apache 2.0, proprietary licenses, and updating all license references across your project - -### Code of Conduct - -The code of conduct defines community standards and expectations for behavior. - -**Location:** `CODE_OF_CONDUCT.md` (repository root) - -**Steps:** - -1. **If you don't have a CODE_OF_CONDUCT.md file:** - - Copy `CODE_OF_CONDUCT.md` from the template to your repository root - - Replace `[INSERT CONTACT METHOD]` with an email address, form URL, or other contact method for reporting violations - -2. **If you already have a CODE_OF_CONDUCT.md file:** - - Review the template's structure for ideas - - Consider adding sections you may be missing (enforcement ladder, scope definition) - -> **Note:** Small personal projects or projects that don't accept external contributions may choose to delete this file entirely. The placeholder check workflow treats the file as optional. - -### VS Code Settings - -The `.vscode/settings.json` file customizes VS Code behavior for your repository. The template includes a placeholder window title. - -**Location:** `.vscode/settings.json` - -**Steps:** - -1. **If you don't have a `.vscode/settings.json` file:** - - Copy `.vscode/settings.json` from the template to your `.vscode/` directory - - Replace the `window.title` value with your repository name - -2. **If you already have a `.vscode/settings.json` file:** - - Review the template's file for settings you may want to add - - Consider adding the `window.title` setting for easier workspace identification - -**Example customization:** - -```json -{ - "window.title": "my-awesome-project" -} -``` - ---- - -## Adopting Issue Templates - -The template includes three issue templates: bug reports, feature requests, and documentation issues. - -### Full Adoption (Recommended if You Have None) - -If your repository doesn't have issue templates, adopt the full set: - -1. Copy the entire `.github/ISSUE_TEMPLATE/` directory to your repository's `.github/` directory - -2. **Update `config.yml`:** Replace `OWNER/REPO` placeholders with your actual organization/username and repository name: - - ```yaml - # Before - url: https://github.com/OWNER/REPO/blob/HEAD/CONTRIBUTING.md - - # After - url: https://github.com/your-username/your-repo/blob/HEAD/CONTRIBUTING.md - ``` - -3. **Update `bug_report.yml`:** Replace `OWNER/REPO` placeholders in the security-notice URLs (the Security tab and `SECURITY.md` links). Both URLs use the same `https://github.com/OWNER/REPO/...` form as `config.yml`. If you also adopt and keep `.github/workflows/check-placeholders.yml` (an optional adoption step), CI will fail until this substitution is made; if you do not adopt that workflow or you remove it after initial setup, no CI guardrail catches a missed substitution and you must verify the replacement manually. - - **GHES adopters:** In both `config.yml` and `bug_report.yml`, also replace `github.com` with your GHES host (e.g., `github.company.com`). The host substitution is not validated by `.github/workflows/check-placeholders.yml`, but inline YAML comments above the affected blocks (in both files) note the requirement. - -4. **Review and customize each template** (see [Customizing Area Dropdowns](#customizing-area-dropdowns)) - -### Partial Adoption (If You Have Existing Templates) - -If you already have issue templates: - -1. Compare the template files with your existing templates -2. Copy specific templates you want to add (e.g., just `documentation_issue.yml`) -3. If you have a `config.yml`, merge the `contact_links` sections -4. **If you adopt `bug_report.yml`** (in whole or in part), remember to replace `OWNER/REPO` in the security-notice URLs — and, on GHES, the `github.com` host as well. - -### Customizing Area Dropdowns - -The issue templates include an "Area" dropdown with default options. Customize for your project: - -**In `bug_report.yml` and `feature_request.yml`:** - -```yaml -# Default options - modify for your project -options: - - Python - - PowerShell - - Markdown / Documentation - - GitHub Actions / CI - - Cross-language / Integration - - Cross-cutting / Repo-wide - - Other (describe/specify in Additional Context) -``` - -**Example for a JavaScript/TypeScript project:** - -```yaml -options: - - Frontend (React) - - Backend (Node.js) - - API - - Documentation - - CI/CD - - Other (describe in Additional Context) -``` - -> **Tip:** Update the Area dropdown in both template files to keep them consistent. - -### Creating Required Labels - -The issue templates use labels that should exist in your repository: - -**Default GitHub labels (already exist in most repositories):** - -- `bug` — Used by bug_report.yml -- `enhancement` — Used by feature_request.yml -- `documentation` — Used by documentation_issue.yml - -**Optional label to create:** - -The templates include a commented-out `triage` label. To use it: - -**Windows (PowerShell) / macOS / Linux:** - -```bash -# Using GitHub CLI -gh label create triage --description "Needs triage" --color "d4c5f9" -``` - -**Or via GitHub web UI:** - -1. Go to your repository -2. Click **Issues** or **Pull requests** -3. Above the list, click **Labels** -4. Click **New label** -5. Under "Label name", type `triage` -6. Under "Description", type `Needs triage` -7. Edit the color hexadecimal number to `d4c5f9` (light purple) -8. Click **Create label** - -After creating the label, uncomment the `- triage` line in each issue template. - ---- - -## Adopting PR Template - -The pull request template provides a checklist for contributors. - -### Simple Adoption - -If you don't have a PR template: - -1. Copy `.github/pull_request_template.md` to your `.github/` directory -2. Review the sections and remove any that don't apply to your project - -### Customization Needed - -Review these sections and modify as needed: - -**Language-specific sections:** - -- **Python-Specific:** Remove if your project doesn't use Python -- **PowerShell-Specific:** Remove if your project doesn't use PowerShell - -**Pre-commit section:** - -- Remove the "Pre-commit Verification" section if you're not adopting pre-commit hooks - -**Contributing guidelines link:** - -The template uses an absolute URL with the `OWNER/REPO` placeholder for the contributing guidelines link in the PR template: - -```markdown -[contributing guidelines](https://github.com/OWNER/REPO/blob/HEAD/CONTRIBUTING.md) -``` - -Replace `OWNER/REPO` with your actual organization and repository name. If you also adopt and keep `.github/workflows/check-placeholders.yml` (an optional adoption step), CI will fail until this substitution is made; if you do not adopt that workflow or you remove it after initial setup, no CI guardrail catches a missed substitution and you must verify the replacement manually. If your CONTRIBUTING.md is in a different location, update the path inside the URL accordingly. **GHES adopters** must additionally replace `github.com` with their GHES host (e.g., `github.company.com`); the host substitution is not validated by `.github/workflows/check-placeholders.yml` even when that workflow is kept. Absolute URLs are required in `.github/ISSUE_TEMPLATE/*.yml` and `.github/pull_request_template.md`; see the **Issue and PR templates** carve-out in `.github/instructions/docs.instructions.md` and the [Pull Request Template Customization](OPTIONAL_CONFIGURATIONS.md#contributing-guidelines-link) section in `OPTIONAL_CONFIGURATIONS.md`. - -### Merging with Existing PR Template - -If you already have a PR template: - -1. **Keep your existing structure** — Don't replace what's working -2. **Add relevant checklist items** from the template that you're missing -3. **Consider adopting:** - - The "Type of Change" section (if you don't have one) - - Language-specific checklists - - The pre-commit verification checkbox (if adopting pre-commit) - -**Example: Adding a "Type of Change" section to your existing template:** - -```markdown -## Type of Change - - - -- [ ] Bug fix (non-breaking change which fixes an issue) -- [ ] New feature (non-breaking change which adds functionality) -- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) -- [ ] Documentation update -- [ ] Dependencies update -- [ ] Configuration/tooling change -``` - ---- - -## Adopting GitHub Copilot Instructions - -GitHub Copilot Instructions guide AI-assisted development by providing project-specific coding standards and rules. The template includes both a main instructions file and language-specific instruction files. - -### Main Instructions File - -**Location:** `.github/copilot-instructions.md` - -This file serves as the "constitution" for all Copilot suggestions in your repository. It contains: - -- Safety and security rules (non-negotiable) -- Pre-commit discipline requirements -- Language-specific guideline references -- Linting and testing configurations - -**Steps:** - -1. Copy `.github/copilot-instructions.md` to your `.github/` directory - -2. **Customize the "Source of Truth" section** — Point to your project's authoritative documentation: - - ```markdown - ## Source of Truth - - > **Customize this section** for your project. Point to your authoritative specification or design document. Example: - > - > - Read **`docs/spec/requirements.md`** before making changes. - > - If any instruction here conflicts with the spec, **the spec wins**. - ``` - -3. **Update the modular instructions table** — Modify to reflect your project's languages and cross-cutting rules: - - ```markdown - | Scope | Instruction File | Applies To | - | --- | --- | --- | - | Git attributes | `.github/instructions/gitattributes.instructions.md` | `**/.gitattributes` | - | Markdown/Docs | `.github/instructions/docs.instructions.md` | `**/*.md` | - | Python | `.github/instructions/python.instructions.md` | `**/*.py` | - | PowerShell | `.github/instructions/powershell.instructions.md` | `**/*.ps1` | - ``` - -4. **Review and modify:** - - **Pre-commit section** — Update if using different tools or workflows - - **Testing section** — Update for your test frameworks and locations - -5. **Update linting and testing tables** — Modify to reflect your project's languages: - - **Linting Configurations table** — Remove the PSScriptAnalyzer row if not using PowerShell - - **Testing Tools table** — Remove rows for languages you're not using (Python row if not using Python, PowerShell row if not using PowerShell) - -### Modular Instructions - -**Location:** `.github/instructions/` - -These files use `applyTo` front matter to automatically apply to matching file patterns: - -```yaml ---- -applyTo: "**/*.py" -description: "Python coding standards for this repository" ---- -``` - -**Available instruction files:** - -| File | Purpose | Recommended For | -| --- | --- | --- | -| `gitattributes.instructions.md` | `.gitattributes` rules for byte-exact text artifacts | All projects | -| `docs.instructions.md` | Markdown/documentation standards | All projects | -| `python.instructions.md` | Python coding standards | Python projects | -| `powershell.instructions.md` | PowerShell coding standards | PowerShell projects | -| `terraform.instructions.md` | Terraform coding standards | Terraform/IaC projects | - -**Adoption options:** - -1. **Full adoption:** Copy the entire `.github/instructions/` directory -2. **Selective adoption:** Copy only the files relevant to your project's languages - -### Merging with Existing Copilot Instructions - -If you already have a `.github/copilot-instructions.md` file: - -1. **Review both files** — Compare your existing instructions with the template - -2. **Merge safety rules** — The template's non-negotiable safety rules are security-focused: - - No secrets in code or repo - - Treat all external input as untrusted - - Allowlisted file access only - - Consider adopting these if not already present. - -3. **Merge language guidelines** — Add references to language instruction files - -4. **Keep your project-specific guidance** — Preserve any custom rules specific to your project - -### Creating Instructions for Other Languages - -If your project uses languages not covered by the template (JavaScript, TypeScript, Go, Rust, etc.): - -1. Use an existing instruction file as a template - -2. Create a new file following the naming pattern: `{language}.instructions.md` - -3. Add the `applyTo` front matter: - - ```yaml - --- - applyTo: "**/*.ts" - description: "TypeScript coding standards for this repository" - --- - ``` - -4. Define your project's coding standards for that language - -5. Update the language table in `.github/copilot-instructions.md` - -### Agent Instruction Files (Multi-Platform Support) - -The template includes three agent instruction files at the repository root to support multi-platform AI coding agents: - -| File | Target Agent(s) | -| --- | --- | -| `CLAUDE.md` | Claude Code, GitHub Copilot coding agent | -| `AGENTS.md` | OpenAI Codex CLI, GitHub Copilot coding agent | -| `GEMINI.md` | Gemini Code Assist, GitHub Copilot coding agent | - -These files are thin entry points for their respective AI coding platforms. `.github/copilot-instructions.md` remains canonical, and the root agent files keep only a minimal inline summary of the highest-priority shared rules plus any platform-specific guidance. - -**Adoption steps:** - -1. **Copy agent files** — Copy the agent files you want from the template repository to your repository root -2. **Update to match your project** — If you customized `.github/copilot-instructions.md`, keep the copied agent files limited to a minimal inline summary of the highest-priority shared rules plus any platform-specific notes you need -3. **Remove unneeded files** — Delete agent files for platforms you do not use - ---- - -## Adopting Markdown Linting - -Markdown linting enforces consistent formatting across your documentation. The template uses markdownlint-cli2 with a configuration optimized for auto-fixable rules. - -**Required files:** - -- `.markdownlint.jsonc` — Linting rules configuration -- `package.json` — npm scripts and dependencies - -**Optional files:** - -- `.github/workflows/markdownlint.yml` — CI workflow -- `.github/scripts/lint-nested-markdown.js` — Lints markdown in code blocks - -### If You Don't Have package.json - -If your project doesn't have a `package.json`: - -1. Copy `package.json` from the template - -2. Update the metadata for your project: - - ```json - { - "name": "your-project-name", - "description": "Your project description", - "author": "Your Name" - } - ``` - -3. Install dependencies: - - **Windows (PowerShell) / macOS / Linux:** - - ```bash - npm install - ``` - -### If You Already Have package.json - -If your project already has a `package.json`: - -1. **Merge the scripts section** — Add these scripts: - - ```json - { - "scripts": { - "lint:md": "markdownlint-cli2 \"**/*.md\" \"#node_modules\"", - "lint:md:nested": "node .github/scripts/lint-nested-markdown.js" - } - } - ``` - -2. **Merge devDependencies** — Add these packages (check template for current versions): - - ```json - { - "devDependencies": { - "markdownlint": "^0.40.0", - "markdownlint-cli2": "^0.20.0" - } - } - ``` - - > **Note:** If adopting the nested markdown linting script, also add `glob`, `jsonc-parser`, and `markdown-it`. - -3. Run `npm install` to install the new dependencies - -### Copying the Configuration - -1. Copy `.markdownlint.jsonc` to your repository root - -2. Review the rules and adjust for your project's preferences. Key configurable rules: - - | Rule | Default | Purpose | - | --- | --- | --- | - | `MD003` | ATX style (`#`) | Heading style | - | `MD004` | Dashes | Unordered list marker | - | `MD029` | Ordered (1. 2. 3.) | Ordered list prefix | - | `MD035` | `---` | Horizontal rule style | - -3. **Optional:** Copy `.github/scripts/lint-nested-markdown.js` if you have markdown embedded in code blocks (common in documentation-heavy projects) - - > **Tip:** See [Nested Markdown Linting Configuration](OPTIONAL_CONFIGURATIONS.md#nested-markdown-linting-configuration) for details on using this script. - -### Testing Markdown Linting - -Run the linter to verify configuration: - -**Windows (PowerShell) / macOS / Linux:** - -```bash -npm run lint:md -``` - -If many errors appear, you have three options: - -1. **Fix the files** — Run with `--fix` to auto-correct: - - ```bash - npx markdownlint-cli2 "**/*.md" "#node_modules" --fix - ``` - -2. **Adjust rules** — Modify `.markdownlint.jsonc` to match your project's existing style - -3. **Disable specific rules** — Set rules to `false` in the configuration - ---- - -## Adopting Pre-commit Hooks - -Pre-commit hooks run automated checks before each commit, catching issues early in the development process. - -**Prerequisites:** - -- Python installed (3.9+) -- pre-commit installed (see installation steps below; pipx/Homebrew installs make `pre-commit` available via PATH, pip installs require module invocation) - -> **Why pipx is recommended:** -> -> When you install Python packages with CLI tools using `pip`, the executables are placed in a `Scripts` folder (Windows) or `bin` folder (macOS/Linux) that may not be in your system PATH. This can cause "command not found" errors. -> -> `pipx` addresses this by installing Python CLI tools in isolated environments and exposing their executables from a single, well-defined binary directory. To make that directory available on the command line, you must run `pipx ensurepath` once (and restart your shell); after that, new tools installed with `pipx` will typically be usable without additional PATH changes. This is the [official recommendation from the pre-commit project](https://pre-commit.com/#install). -> -> If the `pipx` command itself is not yet on your PATH (for example, just after installation on Windows), you can invoke it via the Python module instead, such as `python -m pipx ensurepath` on Windows or `python3 -m pipx ensurepath` on macOS/Linux/FreeBSD for the initial setup. -> -> If you prefer to use `pip`, you can invoke pre-commit as a Python module using `python -m pre_commit` (Windows) or `python3 -m pre_commit` (macOS/Linux/FreeBSD) instead of the `pre-commit` command directly. - -**Install pre-commit:** - -**Windows (PowerShell):** - -**Option 1: Using pipx (recommended):** - -```powershell -# First, upgrade pip to the latest version (recommended) -python -m pip install --upgrade pip - -# Install pipx -python -m pip install pipx - -# Configure PATH (use module invocation in case pipx isn't on PATH yet) -python -m pipx ensurepath -``` - -Then install pre-commit: - -```powershell -# Use module invocation to ensure it works even if pipx isn't on PATH -python -m pipx install pre-commit -``` - -> **Note:** You need to restart your PowerShell window (or open a new one) before running `pre-commit` directly by name, because PATH changes only apply to new shells. Using `python -m pipx` avoids needing `pipx` on PATH and lets you install packages in the same session, but `pipx run pre-commit` runs from a temporary environment and should not be used for `pre-commit install` (it can create hooks that reference a non-existent interpreter). - -**Option 2: Using pip:** - -```powershell -# First, upgrade pip to the latest version (recommended) -python -m pip install --upgrade pip - -# Then install pre-commit -python -m pip install pre-commit -``` - -> **Note:** When using pip, the `pre-commit` command may not be recognized because Python's `Scripts` folder is not always added to PATH. Use `python -m pre_commit` instead of `pre-commit` for all commands. - -**macOS/Linux/FreeBSD:** - -**Option 1: Using pipx (recommended):** - -> **Important (PEP 668 systems):** On newer Linux distributions (Ubuntu 23.04+, Fedora 38+) and some macOS configurations, `python3 -m pip install` commands fail with an `externally-managed-environment` error. If you're on one of these systems, **skip the pip commands below** and install pipx via your OS package manager instead: -> -> - Debian / Ubuntu: `sudo apt install pipx && pipx ensurepath` -> - Fedora: `sudo dnf install pipx && pipx ensurepath` -> - macOS (Homebrew): `brew install pipx && pipx ensurepath` -> -> After running `pipx ensurepath`, restart your terminal, then proceed to the "Then install pre-commit" step below. - -If pip works on your system: - -```bash -# First, upgrade pip to the latest version (recommended) -python3 -m pip install --upgrade pip - -# Install pipx -python3 -m pip install pipx - -# Configure PATH (use module invocation in case pipx isn't on PATH yet) -python3 -m pipx ensurepath -``` - -Then install pre-commit: - -```bash -# Use module invocation to ensure it works even if pipx isn't on PATH -python3 -m pipx install pre-commit -``` - -> **Note:** You need to restart your terminal (or open a new one) before running `pre-commit` directly by name, because PATH changes only apply to new shells. Using `python3 -m pipx` avoids needing `pipx` on PATH and lets you install packages in the same session, but `pipx run pre-commit` runs from a temporary environment and should not be used for `pre-commit install` (it can create hooks that reference a non-existent interpreter). - -**Option 2: Using Homebrew (macOS only):** - -```bash -brew install pre-commit -``` - -**Option 3: Using pip:** - -> **Important (PEP 668 systems):** On newer Linux distributions (Ubuntu 23.04+, Fedora 38+) and some macOS configurations, `python3 -m pip install` commands fail with an `externally-managed-environment` error. If you're on one of these systems, **do not use pip**—use Option 1 (pipx via OS package manager) instead: -> -> - Debian / Ubuntu: `sudo apt install pipx && pipx ensurepath` -> - Fedora: `sudo dnf install pipx && pipx ensurepath` -> - macOS (Homebrew): `brew install pipx && pipx ensurepath` -> -> After running `pipx ensurepath`, restart your terminal, then run `pipx install pre-commit`. - -If pip works on your system: - -```bash -# First, upgrade pip to the latest version (recommended) -python3 -m pip install --upgrade pip - -# Then install pre-commit -python3 -m pip install pre-commit -``` - -> **Note:** When using pip, the `pre-commit` command may not be recognized if Python's `bin` folder is not in your PATH. Use `python3 -m pre_commit` instead of `pre-commit` for all commands. - -### If You Don't Have Pre-commit Configured - -If your project doesn't have a `.pre-commit-config.yaml`: - -1. Copy `.pre-commit-config.yaml` to your repository root - -2. Review the hooks and remove those for languages you don't use: - - ```yaml - # Remove this section if not using Python - - repo: https://github.com/psf/black - rev: 26.1.0 - hooks: - - id: black - args: [--line-length=100] - - # Remove this section if not using Python - - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.14.14 - hooks: - - id: ruff-check - args: [--fix, --line-length=100] - ``` - -3. Install the hooks: - - **If you installed with pipx (Windows):** - - ```powershell - pre-commit install - ``` - - **If you installed with pipx or Homebrew (macOS/Linux/FreeBSD):** - - ```bash - pre-commit install - ``` - - **If you installed with pip (Windows):** - - ```powershell - python -m pre_commit install - ``` - - **If you installed with pip (macOS/Linux/FreeBSD):** - - ```bash - python3 -m pre_commit install - ``` - -4. Run all hooks to verify: - - **If you installed with pipx or Homebrew:** - - ```bash - pre-commit run --all-files - ``` - - **If you installed with pip (Windows):** - - ```powershell - python -m pre_commit run --all-files - ``` - - **If you installed with pip (macOS/Linux/FreeBSD):** - - ```bash - python3 -m pre_commit run --all-files - ``` - -### If You Already Have Pre-commit Configured - -If your project already uses pre-commit: - -1. Compare your `.pre-commit-config.yaml` with the template - -2. Consider adding hooks you may be missing: - - **General hooks (recommended for all projects):** - - `trailing-whitespace` - - `end-of-file-fixer` - - `check-yaml` - - `check-added-large-files` - - **Python hooks:** - - `black` (formatting) - - `ruff` (linting) - - **Markdown hooks:** - - `markdownlint-cli2` - -3. Update hook versions if the template has newer ones - -4. Run all hooks to verify: - - **If you installed with pipx or Homebrew (same command on all platforms/shells):** - - ```bash - pre-commit run --all-files - ``` - - **If you installed with pip (Windows):** - - ```powershell - python -m pre_commit run --all-files - ``` - - **If you installed with pip (macOS/Linux/FreeBSD):** - - ```bash - python3 -m pre_commit run --all-files - ``` - -### Customizing Hooks - -**Adjust line length for Black/Ruff:** - -```yaml -- repo: https://github.com/psf/black - rev: 26.1.0 - hooks: - - id: black - args: [--line-length=88] # Change from 100 to 88 (Black's default) -``` - -**Add hooks for other languages:** - -```yaml -# Example: Prettier for JavaScript/TypeScript -- repo: https://github.com/pre-commit/mirrors-prettier - rev: v3.1.0 - hooks: - - id: prettier - types_or: [javascript, typescript, json, yaml] -``` - -**Handling hook environment issues:** - -| Issue | Platform | Solution | -| --- | --- | --- | -| `pre-commit` not recognized | Windows | Use `python -m pre_commit` instead of `pre-commit`, or reinstall using `pipx` | -| `pre-commit` command not found | macOS/Linux | Use `python3 -m pre_commit` instead of `pre-commit`, or reinstall using `pipx` or Homebrew | -| `pip` not recognized | Windows | Use `python -m pip` instead of `pip` | -| `pip` not found | macOS/Linux | Use `python3 -m pip` instead of `pip` | -| `externally-managed-environment` error | Linux/macOS | Install pipx via OS package manager (`sudo apt install pipx`, `sudo dnf install pipx`, or `brew install pipx`) then run `pipx ensurepath` and use `pipx install pre-commit` (or `python3 -m pipx install pre-commit`) | -| Python not found | Windows | Reinstall Python and check "Add Python to PATH" | -| Hooks fail to initialize | All | See [Hook initialization troubleshooting](#hook-initialization-troubleshooting) below | - -### Hook initialization troubleshooting - -If hooks fail to initialize, follow these steps based on your installation method: - -**If `pre-commit` is on your PATH:** - -Run the following to clear the cache and reinstall hooks: - -```bash -pre-commit clean -pre-commit install -``` - -**If `pre-commit` is NOT on your PATH:** - -First, fix your PATH configuration or use module invocation: - -- **pipx users:** Run `pipx ensurepath` (or `python -m pipx ensurepath` / `python3 -m pipx ensurepath` if `pipx` is not on your PATH) and restart your terminal, then run the commands above. - - > **Note:** Do not use `pipx run pre-commit install` because it runs from a temporary environment and can create hooks that reference a non-existent interpreter. - -- **Homebrew users (macOS):** Ensure your Homebrew `bin` directory (e.g., `/opt/homebrew/bin` or `/usr/local/bin`) is on your PATH, then run the commands above. - -- **pip users:** Use module invocation: - - Windows (PowerShell): - - ```powershell - python -m pre_commit clean - python -m pre_commit install - ``` - - macOS/Linux: - - ```bash - python3 -m pre_commit clean - python3 -m pre_commit install - ``` - ---- - -## Adopting JSON/YAML Toolchain - -If you are adopting the template's JSON/YAML support into an existing repository, work through the following steps. Each step is independent — adopt only the pieces you need. - -> **Do not duplicate full JSON/YAML policy here.** The authoritative authoring rules live in [`.github/instructions/json.instructions.md`](.github/instructions/json.instructions.md) and [`.github/instructions/yaml.instructions.md`](.github/instructions/yaml.instructions.md). Link to those files from your own documentation rather than copying their contents. - -### 1. Start with the Instruction Files - -Copy the JSON and YAML authoring guides into your repository. They are small, self-contained, and apply repository-wide: - -- `.github/instructions/json.instructions.md` -- `.github/instructions/yaml.instructions.md` - -If your repository already has equivalents, merge the rules rather than overwriting. Pay particular attention to the dialect policy (strict `.json` vs. `.jsonc`, no JSON5 by default) and the schema-validation tier guidance. - -### 2. Add `.yamllint.yml` - -Copy `.yamllint.yml` to the repository root. It extends `default`, enforces 2-space indentation, sets the line-length warning at 120 characters, and disables `truthy.check-keys` so unquoted GitHub Actions `on:` keys are accepted. - -If you already have a yamllint configuration, reconcile its rules with the YAML authoring guide rather than replacing your file wholesale. - -### 3. Add or Merge Pre-commit Hooks - -Add the following hooks to your `.pre-commit-config.yaml` (or merge them with your existing configuration) so the JSON/YAML toolchain runs locally and in CI: - -- `check-json` — **must** be scoped to strict `.json` files only. Use `files: \.json$` so the hook does **not** run against `.jsonc`. -- `check-yaml` — basic YAML syntax check. -- `yamllint` — style and structural checks driven by `.yamllint.yml` (`args: [-c, .yamllint.yml]`). -- `actionlint` — GitHub Actions workflow linting (only needed if your repository contains workflow files; on networks that block Go module downloads, the hook's first-run install can fail — CI is the shared enforcement environment in that case). - -The repository's `.pre-commit-config.yaml` shows the current pinned versions and exact configuration; copy from there rather than retyping. - -### 4. Decide Whether `.jsonc` Needs Stricter Tooling - -The default pre-commit stack does **not** validate `.jsonc` syntax — `check-json` is intentionally limited to strict `.json`. Inspect the `.jsonc` files in your repository (for example, `.markdownlint.jsonc` or other tool configurations that ship with a `.jsonc` extension) and decide whether they warrant adding **JSONC-aware tooling** (a JSONC-aware parser, linter, or schema validator). If they are small, well-controlled, and consumed by tools that understand JSONC, no extra tooling is required. If `.jsonc` files are load-bearing, adopt JSONC-aware tooling rather than retrofitting `check-json`. The repository's JSON authoring guide reserves JSONC syntax for files that actually use the `.jsonc` extension; `.json` files **MUST** remain strict JSON, so they are out of scope for this step. - -### 5. Add `actionlint` for GitHub Actions - -If your repository has any GitHub Actions workflow files (`.github/workflows/*.yml`), keep or add the `actionlint` pre-commit hook. It validates workflow syntax, expression usage, and runner labels, and may also run ShellCheck over `run:` blocks when `shellcheck` is available on the contributor's `PATH`. The default pre-commit hook installs only `actionlint` itself, so ShellCheck coverage of `run:` blocks is conditional on a separate local `shellcheck` install. - -### 6. Adopt `schemas/` and `check-jsonschema` Gradually - -Schemas are opt-in and **should be added gradually**, only for **load-bearing** files (files whose shape is depended on by build, deploy, runtime, release automation, or downstream consumers). - -- Copy the `schemas/` directory (including `schemas/README.md`) only if you intend to define real schemas. The template ships `schemas/` with one clearly removable worked example (`example-config.schema.json` plus example data under `schemas/examples/example-config/`) wired into pre-commit and `data-ci.yml`. If you are not adopting schema-backed validation, follow the [downstream removal checklist](schemas/README.md#downstream-removal-checklist) in `schemas/README.md` to take the worked example out, or skip copying the directory entirely. -- When you add a real schema, add **one `check-jsonschema` hook per real schema-backed file family**, scoped to the files that family covers (for example, `^config/.*\.json$`). See [`schemas/README.md`](schemas/README.md) for an illustrative hook example. -- Do **not** add placeholder hooks for schemas that do not yet exist, and do **not** validate every JSON or YAML file by default. `check-json` and `check-yaml` already cover syntax; `check-jsonschema` is for contract checks against specific file families. - -> **No docs should imply that all JSON/YAML files require schemas.** Schemas are reserved for load-bearing contracts; most fixtures, examples, and ad-hoc configs do not need them. - -### 7. Avoid Ecosystem Validators Unless You Use Those Ecosystems - -Adopt ecosystem-specific validators (Kubernetes manifest validators, OpenAPI validators, Helm validators, Ansible validators, and so on) **only** when the repository actually uses those ecosystems. Generic YAML guidance does not require validators that are irrelevant to your stack, and adding them creates noise without enforcing anything useful. - -### Notes on Formatting - -- **Prettier is opt-in** and is not part of the default pre-commit toolchain. The default stack does not use Prettier on JSON or YAML, and it does **not** rely on Prettier (or any other tool) to sort JSON keys. The JSON authoring guide preserves intentional grouping and tool-managed key order. -- **JSON5 is not enabled by default.** The JSON authoring guide intentionally omits `.json5`. Do not introduce JSON5 without an explicit, documented project decision. - ---- - -## Adopting CI Workflows - -The template includes several GitHub Actions workflows. Adopt only the ones relevant to your project. - -### Understanding Workflow Dependencies - -Before adopting workflows, understand their requirements: - -| Workflow | Dependencies | Prerequisites | -| --- | --- | --- | -| `markdownlint.yml` | `package.json` with markdownlint-cli2 | Node.js | -| `auto-fix-precommit.yml` | `.pre-commit-config.yaml` | Python | -| `check-placeholders.yml` | None | Template placeholders in files | -| `python-ci.yml` | Python project structure, `pyproject.toml` | Python | -| `powershell-ci.yml` | PowerShell scripts, Pester tests | PowerShell | -| `data-ci.yml` | `.pre-commit-config.yaml`, `.yamllint.yml` (and, for schema validation, `schemas/`) | Python (for `pre-commit`) | - -### Markdown Lint Workflow - -**Location:** `.github/workflows/markdownlint.yml` - -**Purpose:** Enforces consistent Markdown formatting in CI. - -**Prerequisites:** - -- `markdownlint-cli2` in `package.json` -- `.markdownlint.jsonc` configuration file - -**Steps:** - -1. Copy `.github/workflows/markdownlint.yml` to your `.github/workflows/` directory -2. The workflow runs automatically on push and pull requests - -### Auto-fix Pre-commit Workflow - -**Location:** `.github/workflows/auto-fix-precommit.yml` - -**Purpose:** Automatically fixes pre-commit issues on `copilot/**` branches. This is useful for AI-assisted development where the Copilot Coding Agent may push code that doesn't pass pre-commit checks. - -**Prerequisites:** - -- `.pre-commit-config.yaml` configured - -**Steps:** - -1. Copy `.github/workflows/auto-fix-precommit.yml` to your `.github/workflows/` directory -2. The workflow triggers only on `copilot/**` branches when pushed by `copilot-swe-agent[bot]` - -> **Note:** This workflow is optional but recommended if you use GitHub Copilot Coding Agent. If you don't use the Copilot Coding Agent, you can skip adopting this workflow. If you've already adopted it but later decide to remove it, see [Auto-fix Pre-commit Workflow Configuration](OPTIONAL_CONFIGURATIONS.md#auto-fix-pre-commit-workflow-configuration) for removal instructions. - -### Placeholder Check Workflow - -**Location:** `.github/workflows/check-placeholders.yml` - -**Purpose:** Verifies that `OWNER/REPO` placeholders have been replaced after copying from the template. - -**No configuration required.** The workflow: - -- Runs automatically on push, pull request, and manual dispatch -- Is already configured to exclude only the original template repository (`franklesniak/copilot-repo-template`) -- Will check your repository for unreplaced placeholders - -**Adoption considerations:** - -1. **If you copied templates with placeholders:** The workflow will catch any unreplaced placeholders and fail CI until you fix them - -2. **After all placeholders are replaced:** You have two options: - - **Keep the workflow** — It serves as a safety net for any future template updates or additions - - **Remove the workflow** — Delete `.github/workflows/check-placeholders.yml` if you no longer need placeholder checking - -**What the workflow checks:** - -- `OWNER/REPO` in `.github/ISSUE_TEMPLATE/config.yml` (contact links URLs) -- `OWNER/REPO` in `CONTRIBUTING.md` (clone URL and issues link) -- `@OWNER` in `.github/CODEOWNERS` -- `[security contact email]` and `TODO: Replace` in `SECURITY.md` -- `https://github.com/OWNER/REPO` URLs in any file under `.github/` - -### Python CI Workflow - -**Location:** `.github/workflows/python-ci.yml` - -**Purpose:** Runs pre-commit hooks, type checking (mypy), and tests (pytest) for Python code. - -**Prerequisites:** - -- Python code in `src/` directory (or update `MYPY_PATHS`) -- Tests in `tests/` directory -- `pyproject.toml` with `[project.optional-dependencies] dev` section containing test dependencies - -**Steps:** - -1. Copy `.github/workflows/python-ci.yml` to your `.github/workflows/` directory - -2. **Update paths if needed:** - - If your Python code is in a different location, update the `MYPY_PATHS` environment variable: - - ```yaml - env: - MYPY_PATHS: "your_package/ tests/" # Change from "src/ tests/" - ``` - -3. **Update pytest path if needed:** - - ```yaml - run: pytest your_tests_directory/ -v --cov --cov-report=term-missing - ``` - -**If you have existing Python CI:** - -- Compare your workflow with the template -- Consider adding specific checks from the template as additional jobs -- The template's job dependency pattern (`needs: pre-commit`) ensures tests don't run on poorly-formatted code - -#### If You Already Have pyproject.toml - -If your Python project already has a `pyproject.toml` file, you'll need to ensure it includes the required configuration for the CI workflow to pass successfully. - -**Required Development Dependencies:** - -The CI workflow expects certain development dependencies to be installed. Add these to your `pyproject.toml` if not already present: - -| Dependency | Minimum Version | Purpose | -| --- | --- | --- | -| `pytest` | `>=8.0.0` | Running tests in the CI workflow | -| `pytest-cov` | `>=4.0` | Generating test coverage reports | -| `mypy` | `>=1.0` | Type checking Python code | -| `ruff` | `>=0.9.0` | Code linting and formatting (installed automatically by pre-commit, but useful for local development) | - -**Adding Development Dependencies:** - -Add or merge the `[project.optional-dependencies]` section in your existing `pyproject.toml`: - -```toml -[project.optional-dependencies] -dev = [ - # Required for CI workflow: - "pytest>=8.0.0", - "pytest-cov>=4.0", - "mypy>=1.0", - # Include your existing dev dependencies as well - # "your-existing-dependency>=1.0.0", -] -``` - -> **Note:** If you already have a `dev` extras section, merge these dependencies with your existing ones. - -**Adding mypy Configuration:** - -The CI workflow runs mypy for type checking. Add the `[tool.mypy]` section if not present: - -```toml -[tool.mypy] -python_version = "3.13" # Match your project's requires-python version -warn_return_any = true -warn_unused_configs = true -disallow_untyped_defs = false # Start permissive, tighten later as you add type hints -``` - -> **Tip:** Start with `disallow_untyped_defs = false` to avoid errors on untyped functions. You can tighten this requirement as your project matures and you add more type annotations. - -**Adding pytest Configuration:** - -Add the `[tool.pytest.ini_options]` section to configure test discovery: - -```toml -[tool.pytest.ini_options] -testpaths = ["tests"] # Adjust to match your test directory location -python_files = ["test_*.py"] -python_functions = ["test_*"] -``` - -> **Note:** See [Using the Python Template Files](OPTIONAL_CONFIGURATIONS.md#using-the-python-template-files) for more comprehensive Python configuration options, including Black and Ruff tool settings. - -### PowerShell CI Workflow - -**Location:** `.github/workflows/powershell-ci.yml` - -**Purpose:** Runs PSScriptAnalyzer linting and Pester tests for PowerShell scripts. - -**Prerequisites:** - -- PowerShell scripts (`.ps1` files) -- Pester tests (`.Tests.ps1` files) for the test job - -**Steps:** - -1. Copy `.github/workflows/powershell-ci.yml` to your `.github/workflows/` directory - -2. Copy `.github/linting/PSScriptAnalyzerSettings.psd1` for consistent linting - -3. **Update paths if needed:** - - If your PowerShell files are in different locations, update the find commands in the workflow. - - If your Pester tests aren't in `tests/`, update the configuration: - - ```powershell - $config.Run.Path = "your_tests_directory/" - ``` - -### Data CI Workflow - -**Location:** `.github/workflows/data-ci.yml` - -**Purpose:** Runs the data-file pre-commit hooks (`check-json`, `check-yaml`, `yamllint`, `actionlint`, and the worked-example `check-jsonschema` and `check-metaschema` hooks) as a dedicated check that can be required via branch protection independent of the Python CI job. - -**Prerequisites:** - -- `.pre-commit-config.yaml` with the data-file hooks configured -- `.yamllint.yml` at the repository root -- For schema validation: `schemas/.schema.json` plus matching `schemas/examples//{valid,invalid}/` fixtures - -**Steps:** - -1. Copy `.github/workflows/data-ci.yml` to your `.github/workflows/` directory. -2. Read the top-of-file comment in `.github/workflows/data-ci.yml` for how it differs from `.github/workflows/auto-fix-precommit.yml` (the auto-fix workflow only runs on `copilot/**` branches and commits fixes; the data CI workflow enforces the hooks on every push and PR without committing). -3. The workflow runs automatically on push and pull requests; no per-file configuration is required as long as the pre-commit hooks themselves are scoped correctly. - -**Caveat — `check-jsonschema` and `check-metaschema` steps run unconditionally.** `data-ci.yml` invokes `pre-commit run check-jsonschema --all-files` and `pre-commit run check-metaschema --all-files` as dedicated steps. If you adopt `data-ci.yml` but do **not** keep both hook IDs configured in `.pre-commit-config.yaml`, those steps will fail with `pre-commit: No hook with id ...`. Two safe paths forward: - -1. **Keep schema validation:** retain (or repurpose) the `check-jsonschema` and `check-metaschema` hooks in `.pre-commit-config.yaml`, and provide your own schema(s) and example fixture(s) at `schemas/.schema.json` plus `schemas/examples//{valid,invalid}/`. -2. **Drop schema validation entirely:** remove the `check-jsonschema` and `check-metaschema` steps from `data-ci.yml` (and the corresponding hooks from `.pre-commit-config.yaml`) so the workflow does not invoke missing hook IDs. See [`schemas/README.md`](schemas/README.md) for the canonical removal checklist for the worked example. - -`check-json`, `check-yaml`, `yamllint`, and `actionlint` are repository-agnostic and remain useful even without a worked-example schema, so the corresponding `data-ci.yml` steps can be kept regardless of which path above you take. - -### Merging with Existing CI - -If you already have CI workflows: - -1. **Don't blindly overwrite** — Review what your current CI does - -2. **Add template checks as additional jobs:** - - ```yaml - jobs: - existing-job: - # Your existing job - runs-on: ubuntu-latest - steps: [...] - - # Add this from the template - pre-commit: - runs-on: ubuntu-latest - steps: [...] - ``` - -3. **Consider matrix builds** — The template uses matrix builds for cross-platform testing. If not already doing this, consider adopting the pattern: - - ```yaml - strategy: - matrix: - os: [ubuntu-latest, windows-latest, macos-latest] - ``` - ---- - -## Adopting PSScriptAnalyzer Configuration - -PSScriptAnalyzer is a static code checker for PowerShell. The template includes a configuration file that enforces OTBS (One True Brace Style) formatting. - -### Copying the Configuration - -1. Create the directory if it doesn't exist: - - **Windows (PowerShell):** - - ```powershell - New-Item -ItemType Directory -Path .github\linting -Force - ``` - - **macOS/Linux:** - - ```bash - mkdir -p .github/linting - ``` - -2. Copy `PSScriptAnalyzerSettings.psd1` to `.github/linting/` - -### Using the Configuration - -**Check PowerShell files:** - -```powershell -Invoke-ScriptAnalyzer -Path .\your-script.ps1 -Settings .\.github\linting\PSScriptAnalyzerSettings.psd1 -``` - -**Auto-fix formatting issues:** - -```powershell -Invoke-ScriptAnalyzer -Path .\your-script.ps1 -Settings .\.github\linting\PSScriptAnalyzerSettings.psd1 -Fix -``` - -**Check all PowerShell files in a directory:** - -```powershell -Get-ChildItem -Path . -Filter "*.ps1" -Recurse | ForEach-Object { - Invoke-ScriptAnalyzer -Path $_.FullName -Settings .\.github\linting\PSScriptAnalyzerSettings.psd1 -} -``` - -### Customizing Rules - -Review the rules in `PSScriptAnalyzerSettings.psd1` and adjust as needed: - -**Key configurable rules:** - -| Rule | Default | Purpose | -| --- | --- | --- | -| `PSPlaceOpenBrace` | Same line | OTBS brace placement | -| `PSUseConsistentIndentation` | 4 spaces | Indentation style | -| `PSAvoidUsingPositionalParameters` | Enabled | Enforce named parameters | -| `PSProvideCommentHelp` | Enabled | Require help comments | - -**To disable a rule:** - -```powershell -PSAvoidUsingPositionalParameters = @{ - Enable = $false -} -``` - ---- - -## Validation and Testing - -Before considering adoption complete, verify that all configurations work correctly. - -### Verify All Configurations Work - -**1. Pre-commit (if adopted):** - -```bash -pre-commit run --all-files -``` - -Expected result: All hooks should pass, or you should understand and have addressed any failures. - -**2. Markdown linting (if adopted):** - -```bash -npm run lint:md -``` - -Expected result: No errors, or only warnings you've chosen to accept. - -**3. Push to feature branch:** - -```bash -git add . -git commit -m "feat: adopt template configurations from copilot-repo-template" -git push origin feature/adopt-template-features -``` - -**4. Verify CI workflows:** - -- Navigate to your repository's **Actions** tab -- Check that all adopted workflows run -- Fix any failures before merging - -**5. Test issue templates (if adopted):** - -- Navigate to **Issues** → **New Issue** -- Verify all templates appear correctly -- Open a test issue with each template to verify form fields work -- Verify links in the template chooser (`config.yml`) work -- Close test issues without saving (or delete after testing) - -**6. Test PR template (if adopted):** - -- Open a test pull request (can be against your feature branch) -- Verify the template renders correctly with all sections -- Close without merging - -### Troubleshooting Common Issues - -| Issue | Cause | Solution | -| --- | --- | --- | -| Workflow fails with "file not found" | Different project structure | Update paths in workflow file | -| Pre-commit downloads every time | Environment cache issue | Run `pre-commit clean && pre-commit install` | -| Markdown lint finds many errors | Stricter rules than before | Adjust `.markdownlint.jsonc` or fix files | -| Python CI fails on imports | Different package structure | Update `MYPY_PATHS` in workflow | -| PSScriptAnalyzer fails | Code doesn't match OTBS style | Run with `-Fix` or adjust settings | -| npm install fails | Node.js version mismatch | Update Node.js to v20+ (see `engines` in package.json) | -| Placeholder check fails | OWNER/REPO not replaced | Search and replace all placeholders | - ---- - -## Cleanup and Documentation - -### Files to Review After Adoption - -**Files you may want to delete after adoption:** - -- If you cloned the template separately, delete the clone directory -- `templates/` directory — Only useful if your project is also a template - -**Files you should keep:** - -- All configuration files you adopted (`.markdownlint.jsonc`, `.pre-commit-config.yaml`, etc.) -- All workflow files in `.github/workflows/` -- Copilot instructions (`.github/copilot-instructions.md` and `.github/instructions/`) -- `.github/TEMPLATE_DESIGN_DECISIONS.md` — Template design rationale (useful during setup to understand why configurations were made; can be deleted after review if not needed) - -### Updating Your Project Documentation - -**Update CONTRIBUTING.md:** - -If you adopted the template's `CONTRIBUTING.md`, you should: - -1. **Remove the "For Template Users" section** — This section (starting with `## For Template Users`) contains meta-instructions about the template itself. Delete it along with the HTML comment above it for non-template projects. - -2. **Replace `OWNER/REPO` placeholders** — Update with your actual organization and repository name in clone instructions and issue links. - -3. **Add pre-commit setup instructions** (if you adopted pre-commit): - -````markdown -## Development Setup - -Before making changes, install pre-commit hooks: - -**Option 1: Using pipx (recommended)** - -Windows (PowerShell): - -```powershell -python -m pip install pipx -python -m pipx ensurepath -python -m pipx install pre-commit -``` - -After running the above, restart your terminal, then run: - -```powershell -pre-commit install -``` - -macOS/Linux/FreeBSD: - -```bash -python3 -m pip install pipx -# Or use your OS package manager: sudo apt install pipx, brew install pipx, etc. -python3 -m pipx ensurepath -python3 -m pipx install pre-commit -``` - -After running the above, restart your terminal, then run: - -```bash -pre-commit install -``` - -**Option 2: Using pip (if you can't use pipx)** - -Windows (PowerShell): - -```powershell -python -m pip install pre-commit -# Use module invocation (avoids PATH issues): -python -m pre_commit install -``` - -macOS/Linux/FreeBSD: - -```bash -python3 -m pip install pre-commit -# Use module invocation (avoids PATH issues): -python3 -m pre_commit install -``` - -Pre-commit hooks will automatically run on each commit. You can also run them manually: - -**Windows (PowerShell):** - -```powershell -pre-commit run --all-files -``` - -Or if `pre-commit` is not on PATH: - -```powershell -python -m pre_commit run --all-files -``` - -**macOS/Linux/FreeBSD:** - -```bash -pre-commit run --all-files -``` - -Or if `pre-commit` is not on PATH: - -```bash -python3 -m pre_commit run --all-files -``` -```` - -See [OPTIONAL_CONFIGURATIONS.md](OPTIONAL_CONFIGURATIONS.md) for additional `CONTRIBUTING.md` customization guidance. - -**Update README.md:** - -> **Note:** Do NOT copy the template's `README.md` to your existing repository. The template's README is designed for new repositories created from the template and contains template-specific documentation that would not apply to your repository. Your existing README should remain intact. - -Instead, update your existing README to document any new development requirements you've adopted: - -````markdown -## Development - -### Prerequisites - -- Node.js 20+ (for markdown linting) -- Python 3.9+ (for pre-commit hooks) -- PowerShell (for PSScriptAnalyzer) - -### Setup - -Install markdown linting tools: - -```bash -npm install -``` - -Install pre-commit hooks: - -**Option 1: Using pipx (recommended)** - -Windows (PowerShell): - -```powershell -python -m pip install pipx -python -m pipx ensurepath -python -m pipx install pre-commit -``` - -After running the above, restart your terminal, then run: - -```powershell -pre-commit install -``` - -macOS/Linux/FreeBSD: - -```bash -python3 -m pip install pipx -# Or use your OS package manager: sudo apt install pipx, brew install pipx, etc. -python3 -m pipx ensurepath -python3 -m pipx install pre-commit -``` - -After running the above, restart your terminal, then run: - -```bash -pre-commit install -``` - -**Option 2: Using pip (if you can't use pipx)** - -Windows (PowerShell): - -```powershell -python -m pip install pre-commit -# Use module invocation (avoids PATH issues): -python -m pre_commit install -``` - -macOS/Linux/FreeBSD: - -```bash -python3 -m pip install pre-commit -# Use module invocation (avoids PATH issues): -python3 -m pre_commit install -``` -```` - -Consider adding sections for: - -- Pre-commit hook requirements (if adopted) -- Linting commands (e.g., `npm run lint:md`) -- CI workflow expectations - -**Notify your team:** - -Consider informing collaborators about: - -- New pre-commit requirements -- CI workflow changes -- New issue/PR templates - ---- - -## Migration Notes for Existing Terraform Repos - -If you are adopting this template into an existing Terraform repository, follow this step-by-step checklist to ensure a smooth migration: - -### Pre-Migration Checklist - -- [ ] **Baseline current state:** Run `terraform plan` and save the output. This gives you a baseline to verify no unintended changes occur after migration. -- [ ] **Commit any pending changes:** Ensure your working tree is clean before starting migration. - -### Alignment Checklist - -- [ ] **Align `versions.tf`:** Ensure your `versions.tf` follows the template's format with explicit `required_version` and `required_providers` blocks: - - ```hcl - terraform { - required_version = ">= 1.6.0" - - required_providers { - aws = { - source = "hashicorp/aws" - version = "~> 5.0" - } - } - } - ``` - -- [ ] **Update `.terraform.lock.hcl`:** Regenerate your lock file to include hashes for all platforms used in CI: - - ```bash - terraform providers lock \ - -platform=linux_amd64 \ - -platform=darwin_amd64 \ - -platform=darwin_arm64 - ``` - -- [ ] **Commit `.terraform.lock.hcl`:** Ensure the lock file is tracked in version control. - -### Formatting and Validation Checklist - -- [ ] **Run `terraform fmt -recursive`:** Format all Terraform files to match the template's style. -- [ ] **Run `terraform validate`:** Ensure all configurations are syntactically valid. -- [ ] **Run `tflint`:** Use the template's `.tflint.hcl` configuration to lint your code: - - ```bash - tflint --recursive - ``` - -- [ ] **Fix any issues:** Address formatting, validation, and linting errors before proceeding. - -### Refactoring Checklist - -If you need to rename resources or restructure your configuration: - -- [ ] **Use `moved` blocks for renames:** Instead of manual state manipulation, use declarative `moved` blocks: - - ```hcl - moved { - from = aws_instance.old_name - to = aws_instance.new_name - } - ``` - -- [ ] **Use `import` blocks for existing resources:** Bring unmanaged resources into Terraform using `import` blocks (Terraform 1.5+): - - ```hcl - import { - to = aws_instance.example - id = "i-1234567890abcdef0" - } - ``` - -- [ ] **Use `removed` blocks when appropriate:** Remove resources from state without destroying them using `removed` blocks (Terraform 1.7+): - - ```hcl - removed { - from = aws_instance.legacy_server - - lifecycle { - destroy = false # Remove from state without destroying - } - } - ``` - -### Documentation Checklist - -- [ ] **Document deviations:** If your repository deviates from the template's Terraform standards, document these in the **Scope Exceptions & Deviations from Standards** section of `.github/instructions/terraform.instructions.md`. -- [ ] **Update README:** Document any Terraform-specific setup requirements for your repository. - -### Post-Migration Verification - -- [ ] **Run `terraform plan`:** Compare against your pre-migration baseline. There should be no unexpected changes. -- [ ] **Run CI workflows:** Verify all GitHub Actions workflows pass. -- [ ] **Test in a non-production environment:** If possible, apply changes to a test environment before production. - ---- - -## Next Steps - -After adopting template features, you may want to explore additional customization options: - -- **[Optional Configurations](OPTIONAL_CONFIGURATIONS.md)**: Fine-tune your repository with optional settings like enabling GitHub Discussions, adjusting Dependabot frequency, customizing linting rules, and more. - ---- - -## Summary Checklist - -Before considering adoption complete, verify: - -### Files - -- [ ] All copied files have placeholders replaced (OWNER/REPO, @OWNER, `window.title` in `.vscode/settings.json`, etc.) -- [ ] Conflicting configurations have been merged, not overwritten -- [ ] Unused language files have been removed (e.g., PowerShell instructions if not using PowerShell) -- [ ] `.github/TEMPLATE_DESIGN_DECISIONS.md` reviewed (keep for reference or delete after review) - -### Functionality - -- [ ] Pre-commit runs successfully (`pre-commit run --all-files`) -- [ ] Markdown linting passes (`npm run lint:md`) -- [ ] All CI workflows pass in GitHub Actions -- [ ] Issue templates display correctly in GitHub -- [ ] PR template renders correctly when opening a PR -- [ ] PSScriptAnalyzer runs without unexpected errors (if using PowerShell) - -### Documentation - -- [ ] CONTRIBUTING.md updated with new development requirements -- [ ] README.md updated if setup steps changed -- [ ] Team notified of new tooling/workflows - ---- - -**Commit all your changes:** - -```bash -git add . -git commit -m "feat: adopt template configurations from copilot-repo-template" -git push origin feature/adopt-template-features -``` - -**Create a pull request:** - -1. Navigate to your repository on GitHub -2. Click **Compare & pull request** -3. Review the changes -4. Merge after CI passes - -Congratulations! You've successfully adopted features from the copilot-repo-template into your existing repository. diff --git a/GETTING_STARTED_NEW_REPO.md b/GETTING_STARTED_NEW_REPO.md deleted file mode 100644 index 57a9acf..0000000 --- a/GETTING_STARTED_NEW_REPO.md +++ /dev/null @@ -1,2276 +0,0 @@ -# Getting Started: Creating a New Repository from This Template - -This guide walks you through creating a brand-new repository using `franklesniak/copilot-repo-template`. It is designed for beginners who may not be familiar with Git, Python, Node.js, or pre-commit. If you are looking to merge this template into an existing repository, refer to the README.md instead. - -**Estimated time to complete:** 30-60 minutes (depending on your system and internet speed) - ---- - -## Table of Contents - -- [What This Template Provides](#what-this-template-provides) - - [Repo Layout Examples](#repo-layout-examples) -- [Prerequisites](#prerequisites) - - [Windows Setup](#windows-setup) - - [macOS Setup](#macos-setup) - - [Linux/FreeBSD Setup](#linuxfreebsd-setup) -- [Creating Your Repository on GitHub](#creating-your-repository-on-github) -- [Cloning Your New Repository](#cloning-your-new-repository) -- [Installing Dependencies](#installing-dependencies) -- [Initial Placeholder Replacement](#initial-placeholder-replacement) -- [Creating Optional Labels](#creating-optional-labels) -- [Installing and Configuring Pre-commit](#installing-and-configuring-pre-commit) -- [Language-Specific Customization](#language-specific-customization) - - [JSON/YAML-Heavy Repositories](#jsonyaml-heavy-repositories) -- [Updating package.json Metadata](#updating-packagejson-metadata) -- [Customizing the Pull Request Template](#customizing-the-pull-request-template) -- [Updating README.md](#updating-readmemd) -- [Customizing CONTRIBUTING.md](#customizing-contributingmd) -- [Customizing CODE_OF_CONDUCT.md](#customizing-code_of_conductmd) -- [Updating Copilot Instructions](#updating-copilot-instructions) -- [Additional Configuration (Optional)](#additional-configuration-optional) -- [Validation and Testing](#validation-and-testing) -- [Cleanup](#cleanup) -- [Troubleshooting](#troubleshooting) -- [Development Workflow](#development-workflow) -- [Next Steps](#next-steps) - ---- - -## What This Template Provides - -This template repository includes: - -- **GitHub Copilot Instructions:** Comprehensive coding standards that guide AI-assisted development -- **Language-Specific Guidelines:** Modular instruction files for Markdown, PowerShell, Python, and Terraform -- **Linting Configurations:** Pre-configured settings for markdownlint and PSScriptAnalyzer -- **Pre-commit Hooks:** Automated code quality checks before commits -- **Issue Templates:** Structured templates for bug reports, feature requests, and documentation issues -- **Pull Request Template:** Checklist-based template for consistent PR reviews -- **CI Workflows:** GitHub Actions workflows for linting, testing, and validation -- **Multi-Agent Support:** Instruction files for Claude Code, OpenAI Codex CLI, and Gemini Code Assist - -### Repo Layout Examples - -Depending on your project's needs, you may organize your repository in different ways. Here are two common patterns: - -**Root-Only Repo (Single Configuration):** - -A repository with a single Terraform or application configuration: - -```text -my-project/ -├── .github/ -│ ├── copilot-instructions.md -│ ├── instructions/ -│ └── workflows/ -├── main.tf # Primary configuration -├── variables.tf # Input variables -├── outputs.tf # Output values -├── versions.tf # Provider version constraints -├── .terraform.lock.hcl # Dependency lock file -└── README.md -``` - -**Module-Based Repo:** - -A repository containing reusable modules with examples and tests: - -```text -my-modules/ -├── .github/ -│ ├── copilot-instructions.md -│ ├── instructions/ -│ └── workflows/ -├── modules/ -│ └── vpc/ -│ ├── main.tf -│ ├── variables.tf -│ ├── outputs.tf -│ ├── versions.tf -│ └── README.md -├── examples/ -│ └── basic-vpc/ -│ ├── main.tf -│ └── README.md -├── tests/ -│ └── vpc.tftest.hcl -└── README.md -``` - -Choose the structure that best fits your project when customizing the template. - ---- - -## Prerequisites - -Before you can use this template, you need to install several tools on your computer. Follow the instructions for your operating system below. - -### Windows Setup - -#### 1. Install Git for Windows - -Git is the version control system used to track changes in your code. - -1. Download Git for Windows from [https://git-scm.com/download/win](https://git-scm.com/download/win) -2. Run the installer and use these recommended settings: - - **Select Components:** Keep defaults, ensure "Git Bash Here" is checked - - **Default editor:** Choose your preferred editor (VS Code recommended if installed) - - **Initial branch name:** Select "Let Git decide" (uses `master`) or "Override" and type `main` - - **PATH environment:** Select "Git from the command line and also from 3rd-party software" - - **SSH executable:** Select "Use bundled OpenSSH" - - **HTTPS transport backend:** Select "Use the native Windows Secure Channel library" - - **Line ending conversions:** Select "Checkout Windows-style, commit Unix-style line endings" (recommended) - - **Terminal emulator:** Select "Use MinTTY" - - **Default behavior of `git pull`:** Select "Fast-forward or merge" - - **Credential helper:** Select "Git Credential Manager" - - **Extra options:** Keep defaults -3. Click **Install** and wait for completion - -#### 2. Install Python - -Python is required for pre-commit hooks and Python-based linting tools. - -1. Download Python from [https://www.python.org/downloads/](https://www.python.org/downloads/) -2. Run the installer - - **IMPORTANT:** Check the box that says "Add Python to PATH" before clicking Install - - Click "Install Now" for the default installation - -> **Warning:** If you forget to check "Add Python to PATH," you will need to uninstall and reinstall Python, or manually add Python to your PATH environment variable. - -#### 3. Install Node.js - -Node.js is required for markdown linting scripts. - -1. Download the LTS (Long Term Support) version from [https://nodejs.org/](https://nodejs.org/) -2. Run the installer and accept the defaults -3. When prompted about "Automatically install the necessary tools," you can uncheck this option (not required for this template) - -#### 4. Verify Your Installations - -Open **PowerShell** (search for "PowerShell" in the Start menu) and run these commands: - -```powershell -# Check Git version -git --version - -# Check Python version -python --version - -# Check pip version -python -m pip --version - -# Check Node.js version -node --version - -# Check npm version (comes with Node.js) -npm --version -``` - -You should see version numbers for each command. If any command shows an error, revisit the installation steps for that tool. - -**Example output:** - -```text -git version 2.43.0.windows.1 -Python 3.12.1 -pip 23.3.2 from C:\Users\YourName\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip (python 3.12) -v20.10.0 -10.2.3 -``` - ---- - -### macOS Setup - -#### 1. Install Xcode Command Line Tools - -The Xcode Command Line Tools provide essential developer tools including Git. - -1. Open **Terminal** (press Cmd+Space, type "Terminal," and press Enter) -2. Run the following command: - - ```bash - xcode-select --install - ``` - -3. A dialog will appear asking you to install the tools. Click **Install** and wait for completion. - -> **Note:** This may take several minutes depending on your internet connection. - -#### 2. Install Homebrew (Recommended) - -Homebrew is a package manager that makes it easy to install and manage software on macOS. While optional, it simplifies installation of Python and Node.js. - -1. Open Terminal and run: - - ```bash - /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" - ``` - -2. Follow the on-screen instructions. You may be prompted to enter your password. -3. After installation, follow the instructions shown to add Homebrew to your PATH (the installer will display the exact commands). - -#### 3. Install Python - -**Option A: Using Homebrew (recommended if you installed Homebrew):** - -```bash -brew install python -``` - -**Option B: Using the official installer:** - -1. Download Python from [https://www.python.org/downloads/](https://www.python.org/downloads/) -2. Run the installer package -3. Follow the prompts to complete installation - -#### 4. Install Node.js - -**Option A: Using Homebrew (recommended):** - -```bash -brew install node -``` - -**Option B: Using the official installer:** - -1. Download the LTS version from [https://nodejs.org/](https://nodejs.org/) -2. Run the installer package -3. Follow the prompts to complete installation - -#### 5. Verify Your Installations - -Open **Terminal** and run these commands: - -```bash -# Check Git version -git --version - -# Check Python version -python3 --version - -# Check pip version -pip3 --version - -# Check Node.js version -node --version - -# Check npm version -npm --version -``` - -You should see version numbers for each command. - -**Example output:** - -```text -git version 2.39.3 (Apple Git-145) -Python 3.12.1 -pip 23.3.2 from /opt/homebrew/lib/python3.12/site-packages/pip (python 3.12) -v20.10.0 -10.2.3 -``` - -> **Note:** On macOS, use `python3` instead of `python` to ensure you're using Python 3. - ---- - -### Linux/FreeBSD Setup - -The commands below vary depending on your Linux distribution. Find your distribution and follow the appropriate instructions. - -#### Ubuntu/Debian - -```bash -# Update package lists -sudo apt update - -# Install Git -sudo apt install git - -# Install Python 3 and pip -sudo apt install python3 python3-pip python3-venv - -# Install Node.js (using NodeSource for LTS version) -curl -fsSL https://deb.nodesource.com/setup_lts.x | sudo -E bash - -sudo apt install nodejs -``` - -#### Fedora/RHEL/CentOS - -```bash -# Install Git -sudo dnf install git - -# Install Python 3 and pip -sudo dnf install python3 python3-pip - -# Install Node.js -sudo dnf install nodejs npm -``` - -#### Arch Linux - -```bash -# Install Git -sudo pacman -S git - -# Install Python 3 and pip -sudo pacman -S python python-pip - -# Install Node.js and npm -sudo pacman -S nodejs npm -``` - -#### FreeBSD - -```bash -# Install Git -sudo pkg install git - -# Install Python 3 and pip -sudo pkg install python3 py39-pip - -# Install Node.js and npm -sudo pkg install node npm -``` - -#### Alternative: Using nvm for Node.js - -If you prefer to manage multiple Node.js versions, you can use nvm (Node Version Manager): - -```bash -# Install nvm -curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash - -# Restart your terminal or run: -source ~/.bashrc # or ~/.zshrc if using zsh - -# Install the latest LTS version of Node.js -nvm install --lts - -# Verify installation -node --version -npm --version -``` - -#### Verify Your Installations - -Open a terminal and run these commands: - -```bash -# Check Git version -git --version - -# Check Python version -python3 --version - -# Check pip version -pip3 --version - -# Check Node.js version -node --version - -# Check npm version -npm --version -``` - -You should see version numbers for each command. - ---- - -## Creating Your Repository on GitHub - -Now that you have all the prerequisites installed, you can create your new repository from this template. - -### Step 1: Navigate to the Template Repository - -1. Open your web browser and go to [https://github.com/franklesniak/copilot-repo-template](https://github.com/franklesniak/copilot-repo-template) - -### Step 2: Create a New Repository from the Template - -1. Click the green **"Use this template"** button near the top of the page -2. Select **"Create a new repository"** from the dropdown menu - -### Step 3: Configure Your New Repository - -On the "Create a new repository" page: - -1. **Owner:** Select your GitHub username or an organization you belong to -2. **Repository name:** Enter a name for your new repository (e.g., `my-new-project`) -3. **Description (optional):** Enter a brief description of your project -4. **Visibility:** - - **Public:** Anyone can see your repository - - **Private:** Only you and people you invite can see your repository -5. **Include all branches:** Leave this **unchecked** unless you have a specific reason to include other branches. Most users only need the default branch. - -### Step 4: Create the Repository - -1. Click the green **"Create repository"** button -2. Wait a few seconds for GitHub to create your repository - -You will be redirected to your new repository's page. The URL will be something like `https://github.com/YOUR-USERNAME/your-repo-name`. - ---- - -## Cloning Your New Repository - -Now you need to download (clone) your new repository to your local computer. - -### Understanding SSH vs. HTTPS - -There are two main ways to connect to GitHub: - -- **HTTPS:** Easier to set up. You authenticate with your GitHub username and a personal access token (or GitHub CLI). -- **SSH:** More secure and convenient for frequent use. Requires setting up SSH keys once. - -For beginners, we recommend **HTTPS** because it's simpler to get started. Advanced users may prefer SSH. - -### Windows: Cloning with Git Bash or PowerShell - -1. Open **Git Bash** (right-click on your desktop or in a folder and select "Git Bash Here") or **PowerShell** -2. Navigate to the folder where you want to store your project: - - ```powershell - # Example: Navigate to your Documents folder - cd ~/Documents - ``` - -3. Clone your repository (replace `YOUR-USERNAME` and `your-repo-name` with your actual values): - - **Using HTTPS:** - - ```powershell - git clone https://github.com/YOUR-USERNAME/your-repo-name.git - ``` - - **Using SSH (if you've set up SSH keys):** - - ```powershell - git clone git@github.com:YOUR-USERNAME/your-repo-name.git - ``` - -4. Navigate into your cloned repository: - - ```powershell - cd your-repo-name - ``` - -### macOS/Linux/FreeBSD: Cloning with Terminal - -1. Open **Terminal** -2. Navigate to the folder where you want to store your project: - - ```bash - # Example: Navigate to your home directory's projects folder - cd ~/projects - # Or create one if it doesn't exist: - mkdir -p ~/projects && cd ~/projects - ``` - -3. Clone your repository (replace `YOUR-USERNAME` and `your-repo-name` with your actual values): - - **Using HTTPS:** - - ```bash - git clone https://github.com/YOUR-USERNAME/your-repo-name.git - ``` - - **Using SSH (if you've set up SSH keys):** - - ```bash - git clone git@github.com:YOUR-USERNAME/your-repo-name.git - ``` - -4. Navigate into your cloned repository: - - ```bash - cd your-repo-name - ``` - -> **Tip:** If you haven't set up SSH keys and want to use SSH, see [GitHub's SSH key documentation](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account). - ---- - -## Installing Dependencies - -After cloning, you need to install the project dependencies. - -### Step 1: Navigate to Your Repository Directory - -Make sure you're in your repository's root directory. You should see files like `package.json`, `README.md`, and the `.github` folder. - -**Windows (PowerShell):** - -```powershell -# If you're not already there: -cd C:\path\to\your-repo-name - -# Verify you're in the right place by listing files: -dir -``` - -**macOS/Linux/FreeBSD (Terminal):** - -```bash -# If you're not already there: -cd ~/projects/your-repo-name - -# Verify you're in the right place by listing files: -ls -la -``` - -### Step 2: Install Node.js Dependencies - -Run the following command to install the Node.js dependencies defined in `package.json`: - -**All platforms:** - -```bash -npm install -``` - -This command: - -- Reads the `package.json` file to determine which packages are needed -- Downloads and installs those packages into a `node_modules` folder -- Creates or updates `package-lock.json` to lock dependency versions - -**What gets installed:** The Node.js dependencies are primarily for **markdown linting** (markdownlint-cli2). This ensures your documentation follows consistent formatting rules. - -> **Note:** The `node_modules` folder is automatically excluded from Git (via `.gitignore`), so these files won't be committed to your repository. -> -> **Tip:** The repository also includes an optional nested markdown linting script that checks Markdown code blocks within your documentation. See [Nested Markdown Linting Configuration](OPTIONAL_CONFIGURATIONS.md#nested-markdown-linting-configuration) in the optional configurations guide for details. - ---- - -## Initial Placeholder Replacement - -This template uses placeholder values that you **must** replace with your actual repository information. If you keep `.github/workflows/check-placeholders.yml` (an optional adoption step — see *Verify Placeholder Check Workflow* below), CI will fail until you complete these replacements; if you do not adopt that workflow or you remove it after initial setup, no CI guardrail catches a missed substitution and you must verify the replacements manually. - -### Files That Need Placeholders Replaced - -| File | Placeholders to Replace | -| --- | --- | -| `.github/ISSUE_TEMPLATE/config.yml` | `OWNER/REPO` (appears in URLs twice) | -| `.github/ISSUE_TEMPLATE/bug_report.yml` | `OWNER/REPO` (appears in two security-notice URLs) | -| `.github/pull_request_template.md` | `OWNER/REPO` (appears in the contributing-guidelines link) | -| `.github/CODEOWNERS` | `@OWNER` (appears four times) | -| `CODE_OF_CONDUCT.md` | `[INSERT CONTACT METHOD]` (enforcement contact for code of conduct violations) | -| `CONTRIBUTING.md` | `OWNER/REPO` (appears in clone URL and issues URL) | -| `LICENSE` | `Frank Lesniak` (copyright holder name — replace with your name or organization) | -| `SECURITY.md` | `[security contact email]` | -| `.vscode/settings.json` | `window.title` value (replace with your repository name) | - -### What the Placeholders Mean - -- **`OWNER`:** Your GitHub username or organization name (e.g., `franklesniak`) -- **`REPO`:** Your repository name (e.g., `my-new-project`) -- **`OWNER/REPO`:** Combined format used in GitHub URLs (e.g., `franklesniak/my-new-project`) -- **`@OWNER`:** GitHub username with @ prefix for CODEOWNERS file (e.g., `@franklesniak`) -- **`Frank Lesniak`:** The template author's name in the LICENSE file. Replace with your name or organization name (the copyright holder for your project). -- **`[INSERT CONTACT METHOD]`:** A method for reporting code of conduct violations (e.g., an email address, a form URL, or instructions to contact maintainers). This should be actively monitored and capable of receiving sensitive reports. -- **`[security contact email]`:** An email address for receiving security vulnerability reports -- **`window.title` in `.vscode/settings.json`:** The VS Code window title that appears in the title bar when working in this repository. Replace the instruction text with your repository name for easy identification. - -> **GHES adopters:** The absolute URLs in `.github/ISSUE_TEMPLATE/config.yml`, `.github/ISSUE_TEMPLATE/bug_report.yml`, `.github/pull_request_template.md`, and `CONTRIBUTING.md` are all `https://github.com/OWNER/REPO`-prefixed (variants include `https://github.com/OWNER/REPO/blob/HEAD/` for file targets, `https://github.com/OWNER/REPO/security` and `https://github.com/OWNER/REPO/issues` for non-file repo targets, and `https://github.com/OWNER/REPO.git` for the clone URL in `CONTRIBUTING.md`). The `github.com` host is the assumed default and is **not** validated by `.github/workflows/check-placeholders.yml`. If your repository is hosted on GitHub Enterprise Server, you **MUST** replace `github.com` with your GHES host (e.g., `github.company.com`) in all four files in addition to substituting `OWNER/REPO`; otherwise the clone, issues, security, and contributing-guidelines links will point off-instance to GitHub.com. The PowerShell, GNU sed, and BSD sed scripts below include opt-in `github.com` → GHES-host replacement blocks (commented out by default) that you can uncomment when adopting the template on GHES. - -### Option A: Find and Replace Commands - -#### Windows (PowerShell) - -Open PowerShell in your repository directory and run these commands. Replace the placeholder values with your actual information: - -```powershell -# Define your values (replace these with your actual username/org, repo name, and email) -$Owner = "your-username" -$Repo = "your-repo-name" -$SecurityEmail = "security@example.com" - -# Replace OWNER/REPO in config.yml -(Get-Content ".github/ISSUE_TEMPLATE/config.yml" -Raw -Encoding UTF8).Replace('OWNER/REPO', "$Owner/$Repo") | Set-Content ".github/ISSUE_TEMPLATE/config.yml" -Encoding UTF8 - -# Replace OWNER/REPO in bug_report.yml -(Get-Content ".github/ISSUE_TEMPLATE/bug_report.yml" -Raw -Encoding UTF8).Replace('OWNER/REPO', "$Owner/$Repo") | Set-Content ".github/ISSUE_TEMPLATE/bug_report.yml" -Encoding UTF8 - -# Replace OWNER/REPO in pull_request_template.md -(Get-Content ".github/pull_request_template.md" -Raw -Encoding UTF8).Replace('OWNER/REPO', "$Owner/$Repo") | Set-Content ".github/pull_request_template.md" -Encoding UTF8 - -# Replace OWNER/REPO in CONTRIBUTING.md -(Get-Content "CONTRIBUTING.md" -Raw -Encoding UTF8).Replace('OWNER/REPO', "$Owner/$Repo") | Set-Content "CONTRIBUTING.md" -Encoding UTF8 - -# Replace @OWNER in CODEOWNERS (note the @ prefix) -(Get-Content ".github/CODEOWNERS" -Raw -Encoding UTF8).Replace('@OWNER', "@$Owner") | Set-Content ".github/CODEOWNERS" -Encoding UTF8 - -# Replace contact method placeholder in CODE_OF_CONDUCT.md (uses security email; change if different contact preferred) -(Get-Content "CODE_OF_CONDUCT.md" -Raw -Encoding UTF8).Replace('[INSERT CONTACT METHOD]', $SecurityEmail) | Set-Content "CODE_OF_CONDUCT.md" -Encoding UTF8 - -# Replace security email placeholder in SECURITY.md -(Get-Content "SECURITY.md" -Raw -Encoding UTF8).Replace('[security contact email]', $SecurityEmail) | Set-Content "SECURITY.md" -Encoding UTF8 - -# Replace window.title placeholder in VS Code settings -(Get-Content ".vscode\settings.json" -Raw -Encoding UTF8).Replace('Go to .vscode/settings.json and make this the name of the repo', $Repo) | Set-Content ".vscode\settings.json" -Encoding UTF8 - -# GHES adopters: uncomment the following block to also replace `github.com` with your GHES host -# in the four files that contain absolute https://github.com/OWNER/REPO/... URLs. The host -# substitution is not validated by check-placeholders.yml, so it MUST be done manually on GHES. -# $GHESHost = "github.company.com" -# foreach ($f in @(".github/ISSUE_TEMPLATE/config.yml", ".github/ISSUE_TEMPLATE/bug_report.yml", ".github/pull_request_template.md", "CONTRIBUTING.md")) { -# (Get-Content $f -Raw -Encoding UTF8).Replace('https://github.com/', "https://$GHESHost/") | Set-Content $f -Encoding UTF8 -# } -``` - -#### macOS/Linux/FreeBSD (Bash) - -Open Terminal in your repository directory and run the commands below. Replace the placeholder values with your actual information. - -> **Note:** The `LICENSE` file contains the template author's name (`Frank Lesniak`) in the copyright notice. This is not a pattern-based placeholder, so you'll need to manually update it with your own name or organization. Optionally update the copyright year to the current year or your project's start year. - -**Step 1: Define your values** (all platforms): - -```bash -# Define your values (replace these with your actual username/org, repo name, and email) -OWNER="your-username" -REPO="your-repo-name" -SECURITY_EMAIL="security@example.com" -``` - -**Step 2: Run the replacement commands** for your platform: - -##### Linux (GNU sed) - -```bash -# Replace OWNER/REPO in config.yml -sed -i "s|OWNER/REPO|$OWNER/$REPO|g" .github/ISSUE_TEMPLATE/config.yml - -# Replace OWNER/REPO in bug_report.yml -sed -i "s|OWNER/REPO|$OWNER/$REPO|g" .github/ISSUE_TEMPLATE/bug_report.yml - -# Replace OWNER/REPO in pull_request_template.md -sed -i "s|OWNER/REPO|$OWNER/$REPO|g" .github/pull_request_template.md - -# Replace OWNER/REPO in CONTRIBUTING.md -sed -i "s|OWNER/REPO|$OWNER/$REPO|g" CONTRIBUTING.md - -# Replace @OWNER in CODEOWNERS -sed -i "s|@OWNER|@$OWNER|g" .github/CODEOWNERS - -# Replace contact method placeholder in CODE_OF_CONDUCT.md -sed -i "s|\[INSERT CONTACT METHOD\]|$SECURITY_EMAIL|g" CODE_OF_CONDUCT.md - -# Replace security email placeholder in SECURITY.md -sed -i "s|\[security contact email\]|$SECURITY_EMAIL|g" SECURITY.md - -# Replace window.title placeholder in VS Code settings -sed -i 's|Go to \.vscode/settings\.json and make this the name of the repo|'"$REPO"'|g' .vscode/settings.json - -# GHES adopters: uncomment the following block to also replace `github.com` with your GHES host -# in the four files that contain absolute https://github.com/OWNER/REPO/... URLs. The host -# substitution is not validated by check-placeholders.yml, so it MUST be done manually on GHES. -# GHES_HOST="github.company.com" -# for f in .github/ISSUE_TEMPLATE/config.yml .github/ISSUE_TEMPLATE/bug_report.yml .github/pull_request_template.md CONTRIBUTING.md; do -# sed -i "s|https://github.com/|https://$GHES_HOST/|g" "$f" -# done -``` - -##### macOS / FreeBSD (BSD sed) - -```bash -# Replace OWNER/REPO in config.yml -sed -i '' "s|OWNER/REPO|$OWNER/$REPO|g" .github/ISSUE_TEMPLATE/config.yml - -# Replace OWNER/REPO in bug_report.yml -sed -i '' "s|OWNER/REPO|$OWNER/$REPO|g" .github/ISSUE_TEMPLATE/bug_report.yml - -# Replace OWNER/REPO in pull_request_template.md -sed -i '' "s|OWNER/REPO|$OWNER/$REPO|g" .github/pull_request_template.md - -# Replace OWNER/REPO in CONTRIBUTING.md -sed -i '' "s|OWNER/REPO|$OWNER/$REPO|g" CONTRIBUTING.md - -# Replace @OWNER in CODEOWNERS -sed -i '' "s|@OWNER|@$OWNER|g" .github/CODEOWNERS - -# Replace contact method placeholder in CODE_OF_CONDUCT.md -sed -i '' "s|\[INSERT CONTACT METHOD\]|$SECURITY_EMAIL|g" CODE_OF_CONDUCT.md - -# Replace security email placeholder in SECURITY.md -sed -i '' "s|\[security contact email\]|$SECURITY_EMAIL|g" SECURITY.md - -# Replace window.title placeholder in VS Code settings -sed -i '' 's|Go to \.vscode/settings\.json and make this the name of the repo|'"$REPO"'|g' .vscode/settings.json - -# GHES adopters: uncomment the following block to also replace `github.com` with your GHES host -# in the four files that contain absolute https://github.com/OWNER/REPO/... URLs. The host -# substitution is not validated by check-placeholders.yml, so it MUST be done manually on GHES. -# GHES_HOST="github.company.com" -# for f in .github/ISSUE_TEMPLATE/config.yml .github/ISSUE_TEMPLATE/bug_report.yml .github/pull_request_template.md CONTRIBUTING.md; do -# sed -i '' "s|https://github.com/|https://$GHES_HOST/|g" "$f" -# done -``` - -##### Windows (Git Bash or WSL) - -If using **Git Bash**, use the Linux (GNU sed) commands above. If using **WSL (Windows Subsystem for Linux)**, use the Linux commands. Alternatively, use the PowerShell commands in the previous section. - -> **Note on special characters:** If your email or contact method contains special `sed` characters (`&`, `\`, or `|`), escape them with a backslash or use the PowerShell commands instead, which handle special characters more reliably. - -### Option B: Manual Replacement - -If you prefer, you can open each file in a text editor and manually find and replace the placeholders: - -> **GHES adopters:** In addition to the `OWNER/REPO` substitutions below, also replace `github.com` with your GHES host (e.g., `github.company.com`) in items 1, 2, 3, and 6. The host substitution is not validated by `.github/workflows/check-placeholders.yml`, so it MUST be done manually. - -1. **`.github/ISSUE_TEMPLATE/config.yml`:** - - Find: `OWNER/REPO` - - Replace with: `your-username/your-repo-name` (appears in two URLs) - - **GHES only:** also replace `github.com` with your GHES host (in the same two URLs) - -2. **`.github/ISSUE_TEMPLATE/bug_report.yml`:** - - Find: `OWNER/REPO` - - Replace with: `your-username/your-repo-name` (appears in two security-notice URLs: `…/security` and `…/blob/HEAD/SECURITY.md`) - - **GHES only:** also replace `github.com` with your GHES host (in the same two URLs) - -3. **`.github/pull_request_template.md`:** - - Find: `OWNER/REPO` - - Replace with: `your-username/your-repo-name` (the live substitution - target is the contributing-guidelines link in the file body; the file - may also contain plain-text `OWNER/REPO` references inside the - top-of-file HTML comment block, which you can replace now or skip - because the entire HTML comment block is deleted in the - *Delete Template Comment* step under *Customizing the Pull Request - Template* later in this guide) - - **GHES only:** also replace `github.com` with your GHES host (in the same link) - -4. **`.github/CODEOWNERS`:** - - Find: `@OWNER` - - Replace with: `@your-username` (appears four times) - -5. **`CODE_OF_CONDUCT.md`:** - - Find: `[INSERT CONTACT METHOD]` - - Replace with: your contact method for code of conduct reports (e.g., email address) - -6. **`CONTRIBUTING.md`:** - - Find: `OWNER/REPO` - - Replace with: `your-username/your-repo-name` (appears in clone URL and issues link) - - **GHES only:** also replace `github.com` with your GHES host (in the same clone URL and issues link) - -7. **`SECURITY.md`:** - - Find: `[security contact email]` - - Replace with: your actual security contact email address - -8. **`.vscode/settings.json`:** - - Find: `Go to .vscode/settings.json and make this the name of the repo` - - Replace with: your repository name (e.g., `my-awesome-project`) - -9. **`LICENSE`:** - - Find: `Frank Lesniak` - - Replace with: your name or organization name (the copyright holder) - - Optionally update the copyright year to the current year or your project's start year - -### Understanding the CODEOWNERS File - -The `.github/CODEOWNERS` file defines who is automatically requested to review pull requests. The `@OWNER` placeholder should be replaced with: - -- Your GitHub username (e.g., `@octocat`) for personal repositories -- A team reference (e.g., `@my-org/maintainers`) for organization repositories - -For example, if your GitHub username is `janedoe`, replace `@OWNER` with `@janedoe`. - -### Understanding the Security Email Placeholder - -The `[security contact email]` placeholder in `SECURITY.md` should be replaced with an email address that: - -- Is actively monitored -- Can receive sensitive security reports -- Is not publicly visible (unlike GitHub issues) - -If you prefer not to use email, you can: - -1. Remove the email section entirely from `SECURITY.md` -2. Keep only the GitHub Security Advisories option (see [OPTIONAL_CONFIGURATIONS.md](OPTIONAL_CONFIGURATIONS.md) for details) - ---- - -## Creating Optional Labels - -The issue templates include labels that should exist in your repository. - -**Default GitHub labels (already exist in new repositories):** - -- `bug` — Used by bug_report.yml -- `enhancement` — Used by feature_request.yml -- `documentation` — Used by documentation_issue.yml - -**Optional label to create:** - -The templates include a commented-out `triage` label. To use it: - -**Using GitHub CLI (Windows PowerShell / macOS / Linux):** - -```bash -gh label create triage --description "Needs triage" --color "d4c5f9" -``` - -**Or via GitHub web UI:** - -1. Go to your repository -2. Click **Issues** or **Pull requests** -3. Above the list, click **Labels** -4. Click **New label** -5. Under "Label name", type `triage` -6. Under "Description", type `Needs triage` -7. Edit the color hexadecimal number to `d4c5f9` (light purple) -8. Click **Create label** - -After creating the label, uncomment the `- triage` line in each issue template (`bug_report.yml`, `feature_request.yml`, and `documentation_issue.yml`). - ---- - -## Installing and Configuring Pre-commit - -### Understanding Pre-commit - -[Pre-commit](https://pre-commit.com/) is a framework for managing git hooks. It automatically runs code quality checks (formatting, linting, validation) before each commit, catching issues early and ensuring consistent code quality across your team. - -**Why pre-commit is not a project dependency:** - -- Pre-commit is a **development tool**, not a runtime or test dependency -- It manages its own isolated environments for each hook (e.g., Black, Ruff) -- Installing it globally or via `pipx` keeps your project dependencies clean -- This is standard practice in the Python community -- CI workflows install pre-commit separately - -> **Why pipx is recommended:** -> -> When you install Python packages with CLI tools using `pip`, the executables are placed in a `Scripts` folder (Windows) or `bin` folder (macOS/Linux) that may not be in your system PATH. This can cause "command not found" errors. -> -> `pipx` addresses this by installing Python CLI tools in isolated environments and exposing their executables from a single, well-defined binary directory. To make that directory available on the command line, you must run `pipx ensurepath` once (and restart your shell); after that, new tools installed with `pipx` will typically be usable without additional PATH changes. This is the [official recommendation from the pre-commit project](https://pre-commit.com/#install). -> -> If the `pipx` command itself is not yet on your PATH (for example, just after installation on Windows), you can invoke it via the Python module instead, such as `python -m pipx ensurepath` on Windows or `python3 -m pipx ensurepath` on macOS/Linux/FreeBSD for the initial setup. -> -> If you prefer to use `pip`, you can invoke pre-commit as a Python module using `python -m pre_commit` (Windows) or `python3 -m pre_commit` (macOS/Linux/FreeBSD) instead of the `pre-commit` command directly. - -### Installation - Windows - -#### Option 1: Using pipx (recommended) - -[pipx](https://pipx.pypa.io/) installs Python applications in isolated environments and, after you run `pipx ensurepath` once, adds the pipx binaries directory to your PATH so you can run installed tools from the command line. First install pipx if you don't have it: - -```powershell -# First, upgrade pip to the latest version (recommended) -python -m pip install --upgrade pip - -# Install pipx -python -m pip install pipx - -# Configure PATH (use module invocation in case pipx isn't on PATH yet) -python -m pipx ensurepath -``` - -Then install pre-commit: - -```powershell -# Use module invocation to ensure it works even if pipx isn't on PATH -python -m pipx install pre-commit -``` - -> **Note:** You need to restart your PowerShell window (or open a new one) before running `pre-commit` directly by name, because PATH changes only apply to new shells. Using `python -m pipx` avoids needing `pipx` on PATH and lets you install packages in the same session, but `pipx run pre-commit` runs from a temporary environment and should not be used for `pre-commit install` (it can create hooks that reference a non-existent interpreter). - -#### Option 2: Using pip - -Open PowerShell and run: - -```powershell -# First, upgrade pip to the latest version (recommended) -python -m pip install --upgrade pip - -# Then install pre-commit -python -m pip install pre-commit -``` - -> **Note:** When using pip, the `pre-commit` command may not be recognized because Python's `Scripts` folder is not always added to PATH. Use `python -m pre_commit` instead of `pre-commit` for all commands. For example, use `python -m pre_commit --version` to verify installation. -> -> **Troubleshooting:** If you see `pip: The term 'pip' is not recognized`, ensure you checked "Add Python to PATH" during Python installation. Using `python -m pip` instead of `pip` directly is more reliable on Windows. - -#### Verifying installation - -**If you installed with pipx:** - -```powershell -pre-commit --version -``` - -> **Note:** If `pre-commit` is not found, you need to restart your PowerShell window so the PATH changes from `pipx ensurepath` take effect. Alternatively, you can verify pipx installed pre-commit by running `python -m pipx list` to see installed packages. - -**If you installed with pip:** - -```powershell -python -m pre_commit --version -``` - -You should see output like `pre-commit 4.0.1`. - -### Installation - macOS/Linux/FreeBSD - -#### Option 1: Using pipx (recommended) - -[pipx](https://pipx.pypa.io/) installs Python applications in isolated environments and, after you run `pipx ensurepath` once, adds the pipx binaries directory to your PATH so you can run installed tools from the command line. - -> **Important (PEP 668 systems):** On newer Linux distributions (Ubuntu 23.04+, Fedora 38+) and some macOS configurations, `python3 -m pip install` commands fail with an `externally-managed-environment` error. If you're on one of these systems, **skip the pip commands below** and install pipx via your OS package manager instead: -> -> - Debian / Ubuntu: `sudo apt install pipx && pipx ensurepath` -> - Fedora: `sudo dnf install pipx && pipx ensurepath` -> - macOS (Homebrew): `brew install pipx && pipx ensurepath` -> -> After running `pipx ensurepath`, restart your terminal, then proceed to the "Then install pre-commit" step below. - -If pip works on your system, first install pipx: - -```bash -# First, upgrade pip to the latest version (recommended) -python3 -m pip install --upgrade pip - -# Install pipx -python3 -m pip install pipx - -# Configure PATH (use module invocation in case pipx isn't on PATH yet) -python3 -m pipx ensurepath -``` - -Then install pre-commit: - -```bash -pipx install pre-commit -``` - -> **Note:** If you installed pipx via an OS package manager (Homebrew, apt, dnf), use `pipx install pre-commit` as shown above. If you installed pipx via pip and pipx isn't on your PATH yet, you can use `python3 -m pipx install pre-commit` instead. After installing pre-commit, you'll need to restart your terminal before running `pre-commit` directly by name. Do not use `pipx run pre-commit install`—it runs from a temporary environment and can create hooks referencing a non-existent interpreter. - -#### Option 2: Using Homebrew (macOS only) - -```bash -brew install pre-commit -``` - -#### Option 3: Using pip - -> **Important (PEP 668 systems):** On newer Linux distributions (Ubuntu 23.04+, Fedora 38+) and some macOS configurations, `python3 -m pip install` commands fail with an `externally-managed-environment` error. If you're on one of these systems, **do not use pip**—use Option 1 (pipx via OS package manager) instead: -> -> - Debian / Ubuntu: `sudo apt install pipx && pipx ensurepath` -> - Fedora: `sudo dnf install pipx && pipx ensurepath` -> - macOS (Homebrew): `brew install pipx && pipx ensurepath` -> -> After running `pipx ensurepath`, restart your terminal, then run `pipx install pre-commit`. - -If pip works on your system: - -```bash -# First, upgrade pip to the latest version (recommended) -python3 -m pip install --upgrade pip - -# Then install pre-commit -python3 -m pip install pre-commit -``` - -> **Note:** When using pip, the `pre-commit` command may not be recognized if Python's `bin` folder is not in your PATH. Use `python3 -m pre_commit` instead of `pre-commit` for all commands. For example, use `python3 -m pre_commit --version` to verify installation. - -#### Verifying installation - -**If you installed with pipx:** - -```bash -pre-commit --version -``` - -> **Note:** If `pre-commit` is not found, you need to restart your terminal so the PATH changes from `pipx ensurepath` take effect. Alternatively, you can verify pipx installed pre-commit by running `python3 -m pipx list` to see installed packages. - -**If you installed with Homebrew:** - -```bash -pre-commit --version -``` - -> **Note:** If `pre-commit` is not found, ensure Homebrew's `bin` directory (typically `/opt/homebrew/bin` on Apple Silicon or `/usr/local/bin` on Intel Macs) is in your PATH. You can add it by running `eval "$(/opt/homebrew/bin/brew shellenv)"` (Apple Silicon) or `eval "$(/usr/local/bin/brew shellenv)"` (Intel) in your shell configuration file. - -**If you installed with pip:** - -```bash -python3 -m pre_commit --version -``` - -You should see output like `pre-commit 4.0.1`. - -### Activating Hooks in Your Repository - -Navigate to your repository directory and run: - -**Windows (PowerShell):** - -**If you installed with pipx:** - -```powershell -cd C:\path\to\your-repo-name -pre-commit install -``` - -> **Note:** If `pre-commit` is not found, run `python -m pipx ensurepath` and restart your PowerShell window. Do not use `pipx run pre-commit install` because it runs from a temporary environment and can create hooks that reference a non-existent interpreter. - -**If you installed with pip:** - -```powershell -cd C:\path\to\your-repo-name -python -m pre_commit install -``` - -**macOS/Linux/FreeBSD:** - -**If you installed with pipx:** - -```bash -cd ~/projects/your-repo-name -pre-commit install -``` - -> **Note:** If `pre-commit` is not found, run `python3 -m pipx ensurepath` (or `pipx ensurepath` if `pipx` is already on your PATH) and restart your terminal. Do not use `pipx run pre-commit install` because it runs from a temporary environment and can create hooks that reference a non-existent interpreter. - -**If you installed with Homebrew:** - -```bash -cd ~/projects/your-repo-name -pre-commit install -``` - -> **Note:** If `pre-commit` is not found, ensure Homebrew's `bin` directory is in your PATH (see verification section above). - -**If you installed with pip:** - -```bash -cd ~/projects/your-repo-name -python3 -m pre_commit install -``` - -This command modifies `.git/hooks/pre-commit` to run pre-commit automatically before each commit. You only need to run this once per repository clone. - -**What happens:** Git will now automatically run all configured hooks every time you run `git commit`. If any hook fails, the commit is blocked until you fix the issues. - -### Running Pre-commit Manually - -Run pre-commit on all files: - -**If you installed with pipx:** - -```bash -pre-commit run --all-files -``` - -> **Note:** If `pre-commit` is not found, you may need to add the pipx binary location to your PATH. On **Windows**, run `python -m pipx ensurepath`. On **macOS/Linux/FreeBSD**, run `python3 -m pipx ensurepath`. In both cases, you can alternatively run `pipx ensurepath` if `pipx` itself is already on your PATH, then restart your terminal. `pipx run pre-commit` can be used for one-off commands but runs from a temporary environment (slower and doesn't validate your installed version). - -**If you installed with Homebrew:** - -```bash -pre-commit run --all-files -``` - -**If you installed with pip (Windows):** - -```powershell -python -m pre_commit run --all-files -``` - -**If you installed with pip (macOS/Linux/FreeBSD):** - -```bash -python3 -m pre_commit run --all-files -``` - -**First run behavior:** The first time you run pre-commit, it downloads and installs the hook environments (e.g., Black, Ruff). This may take a minute or two. Subsequent runs are much faster. - -**Interpreting output:** - -```text -Trim Trailing Whitespace......................................Passed -Fix End of Files..............................................Passed -Check Yaml....................................................Passed -Check for added large files...................................Passed -black.........................................................Passed -ruff..........................................................Passed -markdownlint-cli2.............................................Passed -``` - -- **Passed:** The check found no issues -- **Failed:** The check found issues (some hooks auto-fix, others require manual fixes) -- **Skipped:** The hook didn't apply to any files in this commit - -> **Tip:** If hooks auto-fix files (like trailing whitespace or formatting), review the changes and include them in your commit. Pre-commit will run again and should pass. - ---- - -## Language-Specific Customization - -This template includes support for Python, PowerShell, Terraform, and Markdown/documentation. You should remove support for languages you don't need and configure the ones you do use. - -### Decision Point: Which Languages Do You Need? - -Review your project requirements and decide which languages you'll be using: - -- **Python:** Server-side code, scripts, data processing, APIs -- **PowerShell:** Windows automation, cross-platform scripting, DevOps tasks -- **Markdown:** Documentation (always needed) - -### If Using Python - -#### 1. Rename the Package Directory - -The template includes an example Python package at `src/copilot_repo_template/`. Rename it to match your project name: - -**Windows (PowerShell):** - -```powershell -# Replace 'your_package_name' with your actual package name -# Use underscores, not hyphens (Python package naming convention) -Move-Item -Path "src\copilot_repo_template" -Destination "src\your_package_name" -``` - -**macOS/Linux/FreeBSD:** - -```bash -# Replace 'your_package_name' with your actual package name -mv src/copilot_repo_template src/your_package_name -``` - -#### 2. Update pyproject.toml - -Open `pyproject.toml` and update the following fields: - -```toml -[project] -name = "your-project-name" # Your project name (can use hyphens) -version = "0.1.0" -description = "Your project description" -authors = [ - { name = "Your Name" } -] -keywords = ["your", "keywords", "here"] - -# Add your runtime dependencies -dependencies = [ - # "requests>=2.28.0", - # "pydantic>=2.0.0", -] -``` - -#### 3. Update Test Imports - -Open `tests/test_example.py` and update the import statements to match your package name: - -```python -# Change from: -from copilot_repo_template.example import hello - -# To: -from your_package_name.example import hello -``` - -#### 4. Replace Example Code with Your Project Code - -The template includes example files demonstrating Python coding standards. Replace these with your actual project code: - -1. **Replace the example module:** - - Delete or replace `src/your_package_name/example.py` with your actual Python modules - - The example file contains `greet()` and `add_numbers()` functions for demonstration only - -2. **Update `__init__.py`:** - - Update `src/your_package_name/__init__.py` with your package's docstring and any exports you need - - Update the `__version__` variable if using versioning - -3. **Replace example tests:** - - Delete or replace `tests/test_example.py` with tests for your actual modules - - Update the docstring in `tests/__init__.py` to reflect your project name (replace "copilot-repo-template" with your project's name) - - Follow the pytest conventions demonstrated in the example test file - -> **Tip:** The example files demonstrate the coding standards defined in `.github/instructions/python.instructions.md`. Review this file when writing your own code to ensure consistency. - -#### 5. Verify Python Setup - -Install the package in development mode and run tests: - -**Windows (PowerShell):** - -```powershell -python -m pip install -e ".[dev]" -pytest tests/ -v -``` - -**macOS/Linux/FreeBSD:** - -```bash -pip3 install -e ".[dev]" -pytest tests/ -v -``` - -You should see output indicating tests passed. If you haven't yet replaced the example tests with your own, you'll see the template's example tests running. - -### If NOT Using Python - -Remove Python-related files and configurations: - -**Windows (PowerShell):** - -```powershell -# Remove Python package and test files -Remove-Item -Recurse -Force "src" -Remove-Item -Force "tests\test_example.py" -Remove-Item -Force "tests\__init__.py" -Remove-Item -Force "pyproject.toml" - -# Remove Python CI workflow -Remove-Item -Force ".github\workflows\python-ci.yml" - -# Remove Python instructions -Remove-Item -Force ".github\instructions\python.instructions.md" - -# Remove Python templates -Remove-Item -Recurse -Force "templates\python" -``` - -**macOS/Linux/FreeBSD:** - -```bash -# Remove Python package and test files -rm -rf src/ -rm -f tests/test_example.py tests/__init__.py -rm -f pyproject.toml - -# Remove Python CI workflow -rm -f .github/workflows/python-ci.yml - -# Remove Python instructions -rm -f .github/instructions/python.instructions.md - -# Remove Python templates -rm -rf templates/python/ -``` - -#### Update Pre-commit Configuration - -Edit `.pre-commit-config.yaml` to remove Python-specific hooks. Delete or comment out the Black and Ruff sections: - -```yaml -# Remove or comment out these sections: -# - repo: https://github.com/psf/black -# rev: 26.1.0 -# hooks: -# - id: black -# args: [--line-length=100] -# -# - repo: https://github.com/astral-sh/ruff-pre-commit -# rev: v0.14.14 -# hooks: -# - id: ruff-check -# args: [--fix, --line-length=100] -``` - -#### Update Issue Templates - -Edit `.github/ISSUE_TEMPLATE/bug_report.yml` and `.github/ISSUE_TEMPLATE/feature_request.yml` to remove "Python" from the Area dropdown options. - -#### Update Pull Request Template - -Edit `.github/pull_request_template.md` to remove the "Python-Specific (if applicable)" section. - -### If Using PowerShell - -#### 1. Review PSScriptAnalyzer Settings - -The template includes PSScriptAnalyzer configuration at `.github/linting/PSScriptAnalyzerSettings.psd1`. This enforces the OTBS (One True Brace Style) formatting style. - -Review the settings file to understand the rules being enforced. You can customize these settings based on your team's preferences. - -#### 2. Add PowerShell Scripts - -Add your PowerShell scripts to appropriate locations in your repository. The CI workflow will automatically lint any `.ps1` files. - -#### 3. Run PSScriptAnalyzer Locally - -**Install PSScriptAnalyzer (if not already installed):** - -```powershell -Install-Module -Name PSScriptAnalyzer -Force -Scope CurrentUser -``` - -**Run linting on a script:** - -```powershell -# Lint a single file -Invoke-ScriptAnalyzer -Path .\your-script.ps1 -Settings .\.github\linting\PSScriptAnalyzerSettings.psd1 - -# Auto-fix formatting issues -Invoke-ScriptAnalyzer -Path .\your-script.ps1 -Settings .\.github\linting\PSScriptAnalyzerSettings.psd1 -Fix -``` - -#### 4. Run Pester Tests - -**Install Pester 5.x (if not already installed):** - -```powershell -Install-Module -Name Pester -MinimumVersion 5.0 -Force -Scope CurrentUser -``` - -**Run tests:** - -```powershell -Invoke-Pester -Path tests/ -Output Detailed -``` - -### If NOT Using PowerShell - -Remove PowerShell-related files and configurations: - -**Windows (PowerShell):** - -```powershell -# Remove PowerShell CI workflow -Remove-Item -Force ".github\workflows\powershell-ci.yml" - -# Remove PowerShell instructions -Remove-Item -Force ".github\instructions\powershell.instructions.md" - -# Remove linting configuration -Remove-Item -Recurse -Force ".github\linting" - -# Remove PowerShell tests -Remove-Item -Recurse -Force "tests\PowerShell" - -# Remove PowerShell templates -Remove-Item -Recurse -Force "templates\powershell" -``` - -**macOS/Linux/FreeBSD:** - -```bash -# Remove PowerShell CI workflow -rm -f .github/workflows/powershell-ci.yml - -# Remove PowerShell instructions -rm -f .github/instructions/powershell.instructions.md - -# Remove linting configuration -rm -rf .github/linting/ - -# Remove PowerShell tests -rm -rf tests/PowerShell/ - -# Remove PowerShell templates -rm -rf templates/powershell/ -``` - -#### Update Issue Templates - -Edit `.github/ISSUE_TEMPLATE/bug_report.yml` and `.github/ISSUE_TEMPLATE/feature_request.yml` to remove "PowerShell" from the Area dropdown options. - -#### Update Pull Request Template - -Edit `.github/pull_request_template.md` to remove the "PowerShell-Specific (if applicable)" section. - -### JSON/YAML-Heavy Repositories - -The template ships with a default JSON/YAML toolchain that already covers most repositories — including JSON-config-only or YAML-heavy projects (for example, Kubernetes manifests, Helm charts, Ansible playbooks, GitHub Actions-only repos). Unlike the Python and PowerShell sections above, you typically do **not** need to add anything new for JSON/YAML; you mostly need to **keep** what is already there and decide how far to take optional schema validation. - -> **Do not duplicate full JSON/YAML policy here.** The authoritative authoring rules live in [`.github/instructions/json.instructions.md`](.github/instructions/json.instructions.md) and [`.github/instructions/yaml.instructions.md`](.github/instructions/yaml.instructions.md). Read those files when authoring JSON or YAML. - -#### What to Keep - -For any repository that contains JSON or YAML (which is essentially all of them), keep the following: - -- `.github/instructions/json.instructions.md` — JSON authoring standards. -- `.github/instructions/yaml.instructions.md` — YAML authoring standards. -- `.yamllint.yml` — YAML lint configuration consumed by the `yamllint` pre-commit hook (extends `default`, enforces 2-space indentation, sets the line-length warning at 120 characters, and disables `truthy.check-keys` so unquoted GitHub Actions `on:` keys are accepted; see the inline comment and the [yamllint truthy.check-keys ADR in `.github/TEMPLATE_DESIGN_DECISIONS.md`](.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-yamllint-truthycheck-keys-default) for the rationale). -- `.github/workflows/data-ci.yml` — the dedicated data-file CI workflow that re-runs `check-json`, `check-yaml`, `yamllint`, `actionlint`, `check-jsonschema`, and `check-metaschema` so JSON/YAML/Actions enforcement can be required via branch protection independent of the Python CI job. See the top-of-file comment in `.github/workflows/data-ci.yml` for how it differs from `auto-fix-precommit.yml`. -- The default pre-commit hooks for data files in `.pre-commit-config.yaml`: - - `check-json` (validates **strict `.json` only** — see below) - - `check-yaml` - - `yamllint` - - `actionlint` (GitHub Actions workflow validation) - - `check-jsonschema` and `check-metaschema` (worked-example schema validation; see [Schemas: Worked Example Plus Opt-In for Load-Bearing Contracts](#schemas-worked-example-plus-opt-in-for-load-bearing-contracts)) - -If you retain the template's pre-commit workflow (`.github/workflows/python-ci.yml` runs `pre-commit run --all-files`), CI will already enforce these hooks for every push and pull request — you do not need to wire up additional CI for JSON/YAML validation. - -#### `check-json` vs. `.jsonc` - -- The `check-json` hook validates **strict `.json`** files only. The hook is anchored with `files: \.json$`, so `.jsonc` files are intentionally skipped. -- `.jsonc` is allowed only when the consuming tool explicitly supports JSONC (for example, `markdownlint-cli2` reading `.markdownlint.jsonc`, or other tool configurations that ship with a `.jsonc` extension). `.json` files **MUST** remain strict JSON regardless of whether the consuming tool can also accept JSONC. -- The default pre-commit stack does **not** validate `.jsonc` syntax. Repositories that need stricter enforcement of `.jsonc` files should add **JSONC-aware tooling** (a JSONC-aware parser, linter, or schema validator) rather than retrofitting `check-json`. -- JSON5 is **not** enabled by default and **must not** be introduced without an explicit, documented project decision. - -#### Schemas: Worked Example Plus Opt-In for Load-Bearing Contracts - -The template ships a `schemas/` directory at the repository root for JSON Schemas that describe **load-bearing** JSON or YAML files (files whose shape is depended on by build, deploy, runtime, release automation, or downstream consumers). It is **not** scaffold-only: a worked example schema (`schemas/example-config.schema.json`), valid and invalid example fixtures under `schemas/examples/example-config/`, two pre-commit hooks (`check-jsonschema` for valid examples, `check-metaschema` for the schema itself), and a pytest contract at [`tests/test_schema_examples.py`](tests/test_schema_examples.py) ship enabled by default so the validation pipeline is exercised end to end out of the box. - -How schema-backed validation works in this repo: - -- `check-jsonschema` runs against `schemas/examples/example-config/valid/` to confirm valid examples pass. -- `check-metaschema` self-validates `schemas/example-config.schema.json` against its declared JSON Schema Draft 2020-12 metaschema. -- `tests/test_schema_examples.py` auto-discovers `schemas/*.schema.json` and the matching `schemas/examples//{valid,invalid}/` fixtures and asserts that **valid** fixtures exit with code `0` and **invalid** fixtures exit non-zero. Run it with `pytest tests/test_schema_examples.py -v` after any schema or fixture change. -- The dedicated [`.github/workflows/data-ci.yml`](.github/workflows/data-ci.yml) workflow re-runs the same data-file hooks so JSON/YAML/Actions enforcement can be required via branch protection. - -To **adapt the worked example** for your own contract, follow the schema authoring conventions in [`schemas/README.md`](schemas/README.md) (Draft 2020-12, `.schema.json` naming, `additionalProperties: false` for closed contracts, `schemas/examples//{valid,invalid}/` layout) rather than restating those conventions here. - -To **add a new schema-backed file family**, add **one `check-jsonschema` hook per real schema-backed file family**, scoped to the files that family covers (for example, `^config/.*\.json$`). Do not add placeholder hooks for schemas that do not yet exist, and do not validate every JSON or YAML file by default. See [`schemas/README.md`](schemas/README.md) for an illustrative hook configuration. - -To **remove the worked example** in a downstream repository (or to remove `schemas/` entirely if no schema-backed files are needed), follow the canonical [downstream removal checklist](schemas/README.md#downstream-removal-checklist) in `schemas/README.md` rather than improvising the steps here. - -#### Formatting: Prettier and JSON5 Are Not in the Default Toolchain - -- **Prettier is opt-in** and is not part of the default pre-commit toolchain. The default stack does not run Prettier on JSON or YAML, and does **not** rely on Prettier (or any other tool) to sort JSON keys. The JSON authoring guide intentionally preserves intentional grouping and tool-managed key order (for example, in `package.json` and lock files). See the [Prettier deferral ADR in `.github/TEMPLATE_DESIGN_DECISIONS.md`](.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-prettier-deferral-for-data-files) for the full rationale, and [Prettier for JSON/JSONC (Opt-in)](OPTIONAL_CONFIGURATIONS.md#prettier-for-jsonjsonc-opt-in) in `OPTIONAL_CONFIGURATIONS.md` for adoption guidance. -- **JSON5 is not enabled by default.** The JSON authoring guide does not target `.json5`. See the [JSON5 exclusion ADR in `.github/TEMPLATE_DESIGN_DECISIONS.md`](.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-json5-exclusion-by-default) for the rationale. -- If your project independently adopts Prettier or JSON5, document the decision and ensure the resulting configuration does not conflict with the JSON/YAML authoring guides linked above. - -#### Ecosystem Validators - -Adopt ecosystem-specific validators (Kubernetes manifest validators, OpenAPI validators, Helm validators, Ansible validators, etc.) only when the repository actually uses those ecosystems. Do not add validators that are not relevant to your stack. - ---- - -## Updating package.json Metadata - -**File:** `package.json` - -The `package.json` file contains template-specific values that should be updated to reflect your project. This applies to **all projects** using this template, not just Node.js projects, because `package.json` is used for the markdown linting tooling. - -### Fields to Update - -Open `package.json` and update the following fields: - -| Field | Template Value | Update To | -| --- | --- | --- | -| `name` | `"copilot-repo-template"` | Your project name (lowercase, no spaces) | -| `description` | `"Template repository with Copilot instructions and code quality tooling"` | Your project description | -| `author` | `"Frank Lesniak"` | Your name or organization | -| `keywords` | `["template", "copilot", "linting"]` | Keywords relevant to your project | - -### Example - -After updating, your `package.json` metadata might look like: - -```json -{ - "name": "my-awesome-project", - "version": "1.0.0", - "description": "A tool for automating cloud infrastructure deployments", - "private": true, - "keywords": [ - "automation", - "cloud", - "infrastructure" - ], - "author": "Jane Developer" -} -``` - -### Fields to Keep As-Is - -The following fields can typically remain unchanged: - -- `version` — Start at `1.0.0` unless you have a specific versioning scheme -- `private` — Keep as `true` unless you plan to publish to npm -- `scripts` — These are configured for markdown linting and should be kept -- `engines` — Specifies the minimum Node.js version required -- `devDependencies` — Required packages for markdown linting - -> **See Also:** [OPTIONAL_CONFIGURATIONS.md](OPTIONAL_CONFIGURATIONS.md#nodejs-package-configuration) for additional optional configurations like adding `repository`, `homepage`, and `bugs` fields. - ---- - -## Customizing the Pull Request Template - -**File:** `.github/pull_request_template.md` - -The pull request template provides a contributor checklist for consistent PR reviews. - -### Delete Template Comment - -Delete the HTML comment block at the top of the file — the entire `` -block immediately preceding the `## Description` heading. The block contains -template-user guidance (`LINK STYLE`, `CUSTOMIZE`, GHES host note, and a -`KNOWN LIMITATION` paragraph explaining the template-repo-only behavior of -the contributing-guidelines link) that downstream adopters do not need once -the placeholder substitution has been performed. Identify the block by the -opening `` (immediately preceding the `## Description` -heading) rather than by a fixed line range; both endpoints may shift as the -template comment is updated, or if other content is later added above the -block. - -Schematic of the block to delete (paragraph headers shown; full prose elided): - -```markdown - -``` - -### Language-Specific Sections - -Remove sections for languages you're not using: - -- **If NOT using Python:** Delete the entire "Python-Specific (if applicable)" section -- **If NOT using PowerShell:** Delete the entire "PowerShell-Specific (if applicable)" section - -### Pre-commit Verification Section - -- **If using pre-commit:** Keep the section. Optionally strengthen the conditional language to direct language (see [Pull Request Template Customization](OPTIONAL_CONFIGURATIONS.md#pull-request-template-customization) for details). -- **If NOT using pre-commit:** Delete the entire "Pre-commit Verification (if configured)" section. - -> **See Also:** [OPTIONAL_CONFIGURATIONS.md](OPTIONAL_CONFIGURATIONS.md#pull-request-template-customization) for additional customizations like adding language-specific sections for other languages (Node.js, .NET, Go, Rust, Java), customizing Type of Change options, or strengthening test requirements. - ---- - -## Updating README.md - -The template README includes both a project section (for your use) and a template documentation section (for template users). You should customize the project section and remove the template documentation. - -### Understanding the Current README Structure - -The README has two main parts: - -1. **Project section (lines 1-10):** Keep this—add your project name and description -2. **Template section (starting with "Readme for the Copilot Repository Template"):** Delete this - -### What to Keep - -Keep the top section and customize it: - -```markdown -# Your Project Name - -> **Note:** This repository was created from [`franklesniak/copilot-repo-template`](https://github.com/franklesniak/copilot-repo-template). - -## Description - -Your actual project description goes here. Explain what your project does, -why it exists, and who it's for. -``` - -### What to Delete - -Delete everything from the heading `## Readme for the Copilot Repository Template` down to the end of the file, including: - -- What This Template Provides -- Repository Structure -- How to Use This Template -- Validating Your New Repository -- Language Support -- Linting Tools -- Testing -- Code Quality -- Template Maintenance -- License (keep this, but update if needed) - -### Minimal README Template - -Here's a minimal template for your project README: - -```markdown -# Your Project Name - -> **Note:** This repository was created from [`franklesniak/copilot-repo-template`](https://github.com/franklesniak/copilot-repo-template). - -## Description - -Brief description of what your project does. - -## Installation - -Instructions for installing your project. - -## Usage - -Examples of how to use your project. - -## Contributing - -See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and guidelines. - -## License - -MIT License - See [LICENSE](LICENSE) for details. -``` - ---- - -## Customizing CONTRIBUTING.md - -**File:** `CONTRIBUTING.md` - -The `CONTRIBUTING.md` file includes a "For Template Users" section near the end that contains meta-instructions about the template itself. This section should be removed for most downstream projects. - -### Remove "For Template Users" Section - -This section (starting with `## For Template Users`) includes: - -- Information about understanding instruction files -- Guidance on customizing for your project -- First-time setup validation steps - -**For non-template projects (most common):** Delete the entire "For Template Users" section, including the HTML comment above it. - -**To delete:** - -Locate the section starting with: - -````markdown - - -## For Template Users -```` - -Delete from the HTML comment through the end of the file. - -**To keep (if your project is also a template):** Leave the section as-is and customize it for your template's specific guidance. - -### Other Customizations - -See [OPTIONAL_CONFIGURATIONS.md](OPTIONAL_CONFIGURATIONS.md) for additional `CONTRIBUTING.md` customization guidance, including: - -- Replacing `OWNER/REPO` placeholders with your organization and repository name -- Updating Python version requirements -- Removing language-specific sections you don't use - ---- - -## Customizing CODE_OF_CONDUCT.md - -**File:** `CODE_OF_CONDUCT.md` - -The `CODE_OF_CONDUCT.md` file defines community standards and expectations for behavior when participating in your project. It also establishes enforcement procedures for handling violations. - -### Understanding the Contact Method Placeholder - -The `[INSERT CONTACT METHOD]` placeholder in `CODE_OF_CONDUCT.md` should be replaced with a method for reporting code of conduct violations that: - -- Is actively monitored -- Can receive sensitive reports privately -- Is appropriate for your project's size and community - -**Examples of contact methods:** - -- An email address (e.g., `conduct@example.com`) -- A link to a reporting form -- Instructions to contact specific maintainers via direct message - -If you used the same email for both `CODE_OF_CONDUCT.md` and `SECURITY.md` during the [Initial Placeholder Replacement](#initial-placeholder-replacement) step, consider whether you want a separate contact method for code of conduct issues versus security vulnerabilities. - -### About the Contributor Covenant - -This template includes the [Contributor Covenant v3.0](https://www.contributor-covenant.org/version/3/0/code_of_conduct/), which is: - -- **Widely recognized:** Used by over 200,000 open source projects -- **GitHub-supported:** Listed as a recommended code of conduct when creating new repositories -- **Comprehensive:** Covers encouraged behaviors, restricted behaviors, enforcement procedures, and scope - -### Whether to Keep or Remove - -**Keep the file if your project:** - -- Accepts contributions from others -- Has or expects community interaction (issues, discussions, PRs) -- Is part of an organization that requires a code of conduct - -**Consider removing the file if your project:** - -- Is a personal project that doesn't accept external contributions -- Is internal/private with no external community interaction - -> **Note:** The placeholder check workflow treats `CODE_OF_CONDUCT.md` as optional—it will continue to pass if the file is missing. - -See [OPTIONAL_CONFIGURATIONS.md](OPTIONAL_CONFIGURATIONS.md#code-of-conduct-configuration) for advanced customization options including alternative code of conduct templates and customizing enforcement procedures. - ---- - -## Updating Copilot Instructions - -The `.github/copilot-instructions.md` file contains repository-wide coding standards that guide GitHub Copilot's code generation. You should customize this for your project. - -### Customizing the Source of Truth Section - -Edit `.github/copilot-instructions.md` and update the "Source of Truth" section to point to your project's specification or design documents: - -```markdown -## Source of Truth - -> **Important:** Read **`docs/spec/requirements.md`** before making changes. -> If any instruction here conflicts with the spec, **the spec wins**. -``` - -If you don't have a specification document yet, you can simplify this section or leave the placeholder guidance. - -### Updating the Modular Instructions Table - -If you removed Python or PowerShell support, update the modular instructions table to reflect your project's languages and cross-cutting rules: - -```markdown -## Modular Instructions - -This repository uses modular instruction files covering both language-specific standards and cross-cutting repository rules: - -| Scope | Instruction File | Applies To | -| --- | --- | --- | -| Git attributes | `.github/instructions/gitattributes.instructions.md` | `**/.gitattributes` | -| Markdown/Docs | `.github/instructions/docs.instructions.md` | `**/*.md` | -``` - -Remove rows for languages you're not using. - -### Updating Linting and Testing Tables - -If you removed Python or PowerShell support, also update the **Linting Configurations** and **Testing Tools** tables in `.github/copilot-instructions.md`. - -**Linting Configurations table** — Remove the PSScriptAnalyzer row if you removed PowerShell: - -```markdown -## Linting Configurations - -| Tool | Configuration File | Purpose | -| --- | --- | --- | -| markdownlint | `.markdownlint.jsonc` | Markdown linting | -``` - -**Testing Tools table** — Remove rows for languages you're not using: - -If you removed Python: - -```markdown -## Testing Tools - -| Language | Framework | Configuration | Test Location | -| --- | --- | --- | --- | -| PowerShell | Pester 5.x | Inline in `.github/workflows/powershell-ci.yml` | `tests/PowerShell/` | -``` - -If you removed PowerShell: - -```markdown -## Testing Tools - -| Language | Framework | Configuration | Test Location | -| --- | --- | --- | --- | -| Python | pytest | `pyproject.toml` (`[tool.pytest.ini_options]`) | `tests/` | -``` - -If you removed both Python and PowerShell, remove the entire Testing Tools section or update it for your project's languages. - -### Reviewing Instruction Files - -Review the files in `.github/instructions/` and remove or modify any that don't apply to your project: - -- `docs.instructions.md` - Documentation standards (keep for all projects) -- `powershell.instructions.md` - PowerShell standards (remove if not using PowerShell) -- `python.instructions.md` - Python standards (remove if not using Python) -- `terraform.instructions.md` - Terraform/IaC standards (remove if not using Terraform) - -### Reviewing Agent Instruction Files - -The template includes three agent instruction files at the repository root for multi-platform AI coding agent support: - -- `CLAUDE.md` — Claude Code, GitHub Copilot coding agent -- `AGENTS.md` — OpenAI Codex CLI, GitHub Copilot coding agent -- `GEMINI.md` — Gemini Code Assist, GitHub Copilot coding agent - -These files are thin entry points for their respective AI coding platforms. `.github/copilot-instructions.md` remains canonical, and the root agent files keep only a minimal inline summary of the highest-priority shared rules plus any platform-specific guidance. - -**To customize for your project:** - -1. **Remove unneeded files** — Delete agent files for platforms you do not use (e.g., if not using Claude Code, delete `CLAUDE.md`) -2. **Keep remaining files** — Keep all agent files for platforms you want to support, or accept that agents on those platforms receive no project-specific instructions -3. **Keep minimal summaries aligned** — If you modified high-priority shared guidance in `.github/copilot-instructions.md` (for example, safety rules, pre-commit requirements, validation commands, language instruction references, or canonical-file guidance), update any remaining agent files so their minimal summaries stay accurate - ---- - -## Additional Configuration (Optional) - -### Installing the GitHub CLI - -Several optional configuration steps use the GitHub CLI (`gh`). If you haven't installed it yet: - -**Windows:** - -Download from [cli.github.com](https://cli.github.com/) or use winget: - -```powershell -winget install --id GitHub.cli -``` - -**macOS:** - -```bash -brew install gh -``` - -**Linux (Debian/Ubuntu):** - -```bash -curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg -echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null -sudo apt update -sudo apt install gh -``` - -After installation, authenticate with GitHub: - -```bash -gh auth login -``` - -### Creating the `triage` Label - -The issue templates reference a `triage` label that doesn't exist by default. Creating this label enables better issue organization. - -**Why it's not created automatically:** GitHub doesn't support auto-creating labels when a repository is created from a template. This is a platform limitation. - -**Using GitHub CLI:** - -```bash -gh label create triage --description "Needs triage" --color "d4c5f9" -``` - -**Using the GitHub web interface:** - -1. Go to your repository on GitHub -2. Navigate to **Issues** > **Labels** (or go to Settings > Labels) -3. Click **New label** -4. Enter: - - **Label name:** `triage` - - **Description:** `Needs triage` - - **Color:** `d4c5f9` (lavender) -5. Click **Create label** - -**After creating the label:** Uncomment the `# - triage` line in each issue template where you want it applied: - -- `.github/ISSUE_TEMPLATE/bug_report.yml` -- `.github/ISSUE_TEMPLATE/documentation_issue.yml` -- `.github/ISSUE_TEMPLATE/feature_request.yml` - -### Configuring Dependabot - -The template includes Dependabot configuration at `.github/dependabot.yml` for automated dependency updates. - -**To customize update frequency:** - -Edit `.github/dependabot.yml` and modify the `schedule` section: - -```yaml -schedule: - interval: "weekly" # Options: daily, weekly, monthly -``` - -**To disable Dependabot:** - -Delete `.github/dependabot.yml` from your repository. - -### Branch Ruleset (Recommended) - -Repository rulesets help prevent accidental force pushes and ensure code review. Rulesets are the recommended replacement for classic branch protection rules and offer more granular control. See [GitHub's rulesets documentation](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets) for setup instructions. - -For detailed guidance on branch ruleset setup, see the "Branch Ruleset Setup" section in [`.github/TEMPLATE_DESIGN_DECISIONS.md`](.github/TEMPLATE_DESIGN_DECISIONS.md). - ---- - -## Validation and Testing - -Before committing your changes, validate that everything is configured correctly. - -### Run Pre-commit on All Files - -```bash -pre-commit run --all-files -``` - -**Expected output for a clean run:** - -```text -Trim Trailing Whitespace......................................Passed -Fix End of Files..............................................Passed -Check Yaml....................................................Passed -Check for added large files...................................Passed -black.........................................................Passed -ruff..........................................................Passed -markdownlint-cli2.............................................Passed -``` - -**Troubleshooting common failures:** - -- **Trailing whitespace / End of file:** These are auto-fixed. Re-run pre-commit and it should pass. -- **YAML errors:** Check for syntax errors in `.yml` files. -- **Markdown errors:** Run `npm run lint:md` to see specific issues. -- **Black/Ruff errors:** These may auto-fix. Re-run pre-commit. For persistent issues, check the specific error messages. - -### Run Tests (If Applicable) - -**Python:** - -```bash -pytest tests/ -v -``` - -**PowerShell:** - -```powershell -Invoke-Pester -Path tests/ -Output Detailed -``` - -### Commit and Push - -**Stage all changes:** - -```bash -git add . -``` - -**Review what will be committed:** - -```bash -git status -``` - -**Create your initial commit:** - -```bash -git commit -m "Initial project setup from template" -``` - -> **Note:** Pre-commit hooks will run automatically. If they modify files, review the changes, stage them (`git add .`), and commit again. - -**Push to GitHub:** - -```bash -git push origin main -``` - -### Verify CI Workflows Pass - -After pushing, go to your repository on GitHub and: - -1. Click the **Actions** tab -2. Verify that all workflows complete successfully (green checkmarks) - -### Verify Placeholder Check Workflow - -The `check-placeholders.yml` workflow verifies that you've replaced all `OWNER/REPO` placeholders. If this workflow fails: - -1. Read the error messages to identify which files still have placeholders -2. Replace the placeholders with your actual organization and repository name -3. Commit and push the fixes - -**What the workflow checks:** - -- `OWNER/REPO` in `.github/ISSUE_TEMPLATE/config.yml` -- `OWNER/REPO` in `.github/ISSUE_TEMPLATE/bug_report.yml` -- `OWNER/REPO` in `.github/pull_request_template.md` -- `OWNER/REPO` in `CONTRIBUTING.md` -- `@OWNER` in `.github/CODEOWNERS` -- `[security contact email]` in `SECURITY.md` -- `TODO: Replace` markers in `SECURITY.md` -- `OWNER/REPO` in any URL inside `SECURITY.md` (Option C direct-link variant) -- `[INSERT CONTACT METHOD]` in `CODE_OF_CONDUCT.md` -- `https://github.com/OWNER/REPO` URLs anywhere under `.github/` (recursive scan), excluding instructional/historical files (`.github/instructions/**`, `.github/copilot-instructions.md`, `.github/TEMPLATE_DESIGN_DECISIONS.md`) - -**After all placeholders are replaced:** - -Once you've replaced all placeholders and the workflow passes, you have two options: - -- **Keep the workflow** — It serves as a safety net for any future template updates or additions -- **Remove the workflow** — Delete `.github/workflows/check-placeholders.yml` if you no longer need placeholder checking - -See [Placeholder Check Workflow Configuration](OPTIONAL_CONFIGURATIONS.md#placeholder-check-workflow-configuration) for more details on customizing this workflow. - -### Post-clone Verification Plan - -After completing the setup checklist, perform the following quick verification: - -1. **Verify template functionality** in your newly created repository (template maintainers can create a separate test repository to verify template changes) -2. **Open each issue type** once and ensure required fields behave correctly -3. **Click key links** in the issue template chooser: - - Contributing Guide link - - Security Vulnerabilities link - - (If enabled) Discussions link -4. **Verify issue form rendering:** - - Open a bug report issue template - - Paste a Python traceback into the Logs/Error Output field - - Confirm it renders cleanly as plain text (not mangled by Markdown parsing) -5. **Verify security flow:** - - From your repository's main page, click the **Security** tab - - Confirm SECURITY.md is accessible -6. **Open a test PR** to verify the PR workflow: - - Create a trivial change (e.g., add a comment to a file) - - Open a pull request and confirm: - - PR template renders correctly - - Contributing guidelines link works - - CI workflows trigger as expected - - Close the test PR without merging - ---- - -## Cleanup - -After completing setup and verification, clean up template-specific files. - -### Review TEMPLATE_DESIGN_DECISIONS.md - -The template design decisions document (`.github/TEMPLATE_DESIGN_DECISIONS.md`) contains rationale for design choices made in the **template itself**, not your downstream project. Review it during setup to understand why the template is structured the way it is. - -**After review, choose one of these options:** - -1. **Keep it** if you want to preserve template rationale for future reference (e.g., onboarding new maintainers or understanding why configurations were made) -2. **Delete it** if the template rationale is not useful for your project going forward - -> **Note:** This file documents the *template's* design decisions. It is not about your project's design decisions. If you keep it, be aware that its content is template-specific and may reference context that does not apply to your repository. - -### Consider This Getting Started Guide - -You have three options for `GETTING_STARTED_NEW_REPO.md`: - -1. **Delete it** if your project won't be used as a template for others -2. **Keep it** if your project is also a template that others will clone -3. **Modify it** to be specific to your project's setup process - -**To delete:** - -**Windows (PowerShell):** - -```powershell -Remove-Item -Force "GETTING_STARTED_NEW_REPO.md" -``` - -**macOS/Linux/FreeBSD:** - -```bash -rm -f GETTING_STARTED_NEW_REPO.md -``` - -### Final Cleanup Commit - -```bash -git add . -git commit -m "Remove template documentation after initial setup" -git push origin main -``` - ---- - -## Troubleshooting - -### Pre-commit Hook Failures - -**Problem:** Pre-commit hooks fail with "command not found" errors. - -**Solution:** Ensure pre-commit is installed globally and in your PATH: - -```bash -pip install pre-commit -pre-commit --version -``` - -**Problem:** Hooks download every time you commit. - -**Solution:** This is normal on first run. The environments are cached in `~/.cache/pre-commit/`. If they keep downloading, check disk space and permissions. - -### Python ModuleNotFoundError - -**Problem:** `ModuleNotFoundError: No module named 'copilot_repo_template'` - -**Solution:** You need to either: - -1. Rename `src/copilot_repo_template/` to your package name and update imports -2. Install the package in development mode: `pip install -e ".[dev]"` -3. If not using Python, delete the `src/` directory - -### Node.js/npm Errors - -**Problem:** `npm install` fails with permission errors. - -**Solution:** - -- **Windows:** Run PowerShell as Administrator -- **macOS/Linux:** Don't use `sudo npm install`. Instead, fix npm permissions or use a version manager like nvm. - -**Problem:** `npm run lint:md` fails with "command not found". - -**Solution:** Run `npm install` first to install dependencies. - -### Placeholder Check CI Failures - -**Problem:** The `check-placeholders.yml` workflow fails. - -**Solution:** Read the error messages carefully. They tell you exactly which files and lines contain placeholders. Replace: - -- `OWNER/REPO` with `your-username/your-repo-name` -- `@OWNER` with `@your-username` -- `[security contact email]` with your actual email or remove the email option - -### Permission Errors - -**Windows:** - -- Run PowerShell as Administrator for system-wide installations -- Check that Python and Node.js are in your PATH -- Restart PowerShell after installing new tools - -**macOS/Linux/FreeBSD:** - -- Don't use `sudo` for pip or npm installations in your home directory -- Check file ownership: `ls -la` -- Fix ownership: `sudo chown -R $(whoami) ~/.npm ~/.cache` - -### Pre-commit Not Running on Commit - -**Problem:** Git commits succeed without running pre-commit hooks. - -**Solution:** Re-install the hooks: - -**Windows (PowerShell):** - -If you installed with pipx: - -```powershell -pre-commit install -``` - -> **Note:** If `pre-commit` is not found, run `python -m pipx ensurepath` and restart your PowerShell window. Do not use `pipx run pre-commit install` because it runs from a temporary environment and can create hooks that reference a non-existent interpreter. - -If you installed with pip: - -```powershell -python -m pre_commit install -``` - -**macOS/Linux/FreeBSD:** - -If you installed with pipx: - -```bash -pre-commit install -``` - -> **Note:** If `pre-commit` is not found, run `python3 -m pipx ensurepath` (or `pipx ensurepath` if `pipx` is already on your PATH) and restart your terminal. Do not use `pipx run pre-commit install` because it runs from a temporary environment and can create hooks that reference a non-existent interpreter. - -If you installed with Homebrew: - -```bash -pre-commit install -``` - -> **Note:** If `pre-commit` is not found, ensure Homebrew's `bin` directory (typically `/opt/homebrew/bin` on Apple Silicon or `/usr/local/bin` on Intel Macs) is in your PATH. - -If you installed with pip: - -```bash -python3 -m pre_commit install -``` - -Verify the hook exists: - -**Windows:** - -```powershell -Get-Content .git\hooks\pre-commit -``` - -**macOS/Linux/FreeBSD:** - -```bash -cat .git/hooks/pre-commit -``` - ---- - -## Development Workflow - -Now that your repository is set up, you're ready to start development! For the standard development workflow, including: - -- Creating branches -- Making changes -- Running tests -- Submitting pull requests -- Code review process - -See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed instructions. - -**Quick reference for daily development:** - -**Windows (PowerShell):** - -```powershell -# Create a feature branch -git checkout -b feature/your-feature-name - -# Make changes and run pre-commit -pre-commit run --all-files - -# Stage and commit -git add . -git commit -m "Add your feature" - -# Push and open a PR -git push origin feature/your-feature-name -``` - -If `pre-commit` is not on PATH: - -- **pipx users:** Run `python -m pipx ensurepath` (or `pipx ensurepath` if `pipx` is already on your PATH) and restart your PowerShell window, then run `pre-commit run --all-files`. -- **pip users:** Use module invocation: - - ```powershell - python -m pre_commit run --all-files - ``` - -**macOS/Linux/FreeBSD:** - -```bash -# Create a feature branch -git checkout -b feature/your-feature-name - -# Make changes and run pre-commit -pre-commit run --all-files - -# Stage and commit -git add . -git commit -m "Add your feature" - -# Push and open a PR -git push origin feature/your-feature-name -``` - -If `pre-commit` is not on PATH: - -- **pipx users:** Run `python3 -m pipx ensurepath` (or `pipx ensurepath` if `pipx` is already on your PATH) and restart your terminal, then run `pre-commit run --all-files`. -- **pip users:** Use module invocation: - - ```bash - python3 -m pre_commit run --all-files - ``` - ---- - -## Next Steps - -After completing the initial setup, you may want to explore additional customization options: - -- **[Optional Configurations](OPTIONAL_CONFIGURATIONS.md)**: Fine-tune your repository with optional settings like enabling GitHub Discussions, adjusting Dependabot frequency, customizing linting rules, and more. - ---- - -**Congratulations!** 🎉 Your repository is now fully configured and ready for development. Happy coding! diff --git a/OPTIONAL_CONFIGURATIONS.md b/OPTIONAL_CONFIGURATIONS.md deleted file mode 100644 index f360e08..0000000 --- a/OPTIONAL_CONFIGURATIONS.md +++ /dev/null @@ -1,3432 +0,0 @@ -# Optional Configurations - -This guide covers optional customizations you can make after completing the initial setup from either of the two getting started guides: - -- **[Creating a New Repository](GETTING_STARTED_NEW_REPO.md)**: For users creating new repositories from this template -- **[Adding Template Features to an Existing Repository](GETTING_STARTED_EXISTING_REPO.md)**: For users adopting template features into existing repositories - -> **Note:** None of these configurations are required. Your repository will work correctly with the default settings. These options allow you to fine-tune your setup based on your project's specific needs. - ---- - -## Table of Contents - -- [Issue Template Configuration](#issue-template-configuration) - - [Bug Report Template Customization](#bug-report-template-customization) - - [Feature Request Template Customization](#feature-request-template-customization) - - [Documentation Issue Template Customization](#documentation-issue-template-customization) -- [Security Configuration](#security-configuration) -- [Code of Conduct Configuration](#code-of-conduct-configuration) -- [Pull Request Template Customization](#pull-request-template-customization) -- [Dependabot Configuration](#dependabot-configuration) -- [Pre-commit Configuration](#pre-commit-configuration) -- [Schema Validation Configuration](#schema-validation-configuration) -- [Prettier for JSON/JSONC (Opt-in)](#prettier-for-jsonjsonc-opt-in) -- [JSONC Validation (Opt-in)](#jsonc-validation-opt-in) -- [JSON5 Support (Opt-in)](#json5-support-opt-in) -- [Ecosystem-Specific YAML Validators (Opt-in)](#ecosystem-specific-yaml-validators-opt-in) -- [Future SchemaStore Validation for Repository Configuration Files](#future-schemastore-validation-for-repository-configuration-files) -- [Removing the Worked Example Schema](#removing-the-worked-example-schema) -- [Markdown Linting Configuration](#markdown-linting-configuration) - - [Using the cli2-Specific Configuration Format](#using-the-cli2-specific-configuration-format) -- [Nested Markdown Linting Configuration](#nested-markdown-linting-configuration) -- [Markdown Lint Workflow Configuration](#markdown-lint-workflow-configuration) -- [Copilot Documentation Instructions Configuration](#copilot-documentation-instructions-configuration) -- [Copilot Python Instructions Configuration](#copilot-python-instructions-configuration) -- [Copilot PowerShell Instructions Configuration](#copilot-powershell-instructions-configuration) -- [Copilot Terraform Instructions Configuration](#copilot-terraform-instructions-configuration) -- [Copilot Main Instructions Configuration](#copilot-main-instructions-configuration) -- [Customizing Agent Instruction Files](#customizing-agent-instruction-files) -- [CI Workflow Configuration](#ci-workflow-configuration) -- [Auto-fix Pre-commit Workflow Configuration](#auto-fix-pre-commit-workflow-configuration) -- [Placeholder Check Workflow Configuration](#placeholder-check-workflow-configuration) -- [PowerShell CI Workflow Configuration](#powershell-ci-workflow-configuration) -- [Using the Python Template Files](#using-the-python-template-files) -- [Using the Pester Test Template](#using-the-pester-test-template) -- [PSScriptAnalyzer Configuration](#psscriptanalyzer-configuration) -- [CODEOWNERS Configuration](#codeowners-configuration) -- [Node.js Package Configuration](#nodejs-package-configuration) -- [Gitignore Configuration](#gitignore-configuration) -- [License Customization](#license-customization) -- [VS Code PowerShell File Encoding for Non-ASCII Characters](#vs-code-powershell-file-encoding-for-non-ascii-characters) -- [Ongoing Maintenance](#ongoing-maintenance) - - [Updating Pre-commit Hooks](#updating-pre-commit-hooks) - - [Reviewing Python Version Requirements](#reviewing-python-version-requirements) - ---- - -## Issue Template Configuration - -**File:** `.github/ISSUE_TEMPLATE/config.yml` - -### Requiring Issue Templates - -By default, `blank_issues_enabled` is set to `true`, which allows users to create issues without selecting a template. This provides flexibility for edge cases that don't fit your predefined templates. - -**To require template usage:** - -```yaml -blank_issues_enabled: false -``` - -> **Recommendation:** Set this to `false` once you have comprehensive templates covering all issue types and want structured data collection for better triage. - -### Enabling GitHub Discussions Link - -If your repository uses GitHub Discussions for Q&A and general discussions, you can add a link that redirects users away from the issue tracker. - -**Steps:** - -1. Enable GitHub Discussions in your repository: - - Go to **Settings** > **General** > **Features** - - Check the **Discussions** checkbox -2. Uncomment the Discussions contact link in `config.yml`: - - ```yaml - - name: 💬 Questions & Discussions - url: https://github.com/OWNER/REPO/discussions - about: Ask questions and discuss ideas (not for bug reports) - ``` - -3. Replace `OWNER/REPO` with your actual organization and repository name - -### Adding a Support/FAQ Link - -If you prefer not to enable Discussions but want to redirect support questions away from issues: - -**Steps:** - -1. Add a `## Support` section to your `README.md` with FAQs and support guidance -2. Uncomment the Support/FAQ contact link in `config.yml`: - - ```yaml - - name: ❓ Support / FAQ - url: https://github.com/OWNER/REPO#support - about: Common questions, FAQs, and support guidance - ``` - -3. Replace `OWNER/REPO` with your actual values -4. Update the URL anchor (`#support`) if your section has a different heading - -### Security Link URL Customization - -The `config.yml` file includes a security contact link that points to `/security` by default: - -```yaml -- name: 🔒 Security Vulnerabilities - url: https://github.com/OWNER/REPO/security - about: Report security issues privately (do not open a public issue). Private vulnerability reporting is only available for public repositories. -``` - -After enabling private vulnerability reporting in your repository, you can optionally update this URL to provide a more direct path to the vulnerability reporting form. - -> **Important:** Private vulnerability reporting is only available for **public repositories**. If your repository is private, security reporters must use email contact as specified in your `SECURITY.md` file. -> -> **See:** [Security Configuration](#security-configuration) for instructions on enabling private vulnerability reporting and updating this URL. - -### Bug Report Template Customization - -**File:** `.github/ISSUE_TEMPLATE/bug_report.yml` - -The bug report template includes numerous `# CUSTOMIZE:` comments indicating optional configuration points. This section documents each customization option. - -#### Top-Level Metadata - -##### Title Prefix - -Adjust the issue title prefix to match your project's conventions: - -```yaml -# Default: -title: "[Bug] " - -# Example alternatives: -title: "bug: " -title: "[BUG] " -title: "" # No prefix -``` - -##### Labels - -Update the labels to match your repository's label taxonomy. Ensure labels exist before using them: - -```yaml -labels: - - bug - # Add your project-specific labels: - # - priority:high - # - area:api -``` - -> **Note:** Labels must exist in your repository before they can be applied. Create them via **Settings** > **Labels** or use the GitHub CLI. - -##### Triage Label - -Uncomment the `triage` label after creating it in your repository: - -```yaml -labels: - - bug - - triage # Uncomment after creating the label -``` - -> **Cross-reference:** The [Getting Started Guides](GETTING_STARTED_NEW_REPO.md) provide instructions for creating labels in your repository. - -##### Issue Type (Organization-Level) - -For organizations using [GitHub issue types](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms#top-level-syntax), uncomment and configure the `type` field: - -```yaml -# Uncomment to enable: -type: Bug -``` - -##### Assignees - -Pre-populate the assignees field for bug reports: - -```yaml -# Uncomment and update: -assignees: - - maintainer-username - - your-github-handle -``` - -##### Projects - -Auto-add bug reports to a GitHub Project (uses project number): - -```yaml -# Uncomment and update with your org and project number: -projects: - - org/1 -``` - -#### Pre-flight Checklist Customization - -##### Making Documentation Check Required - -For projects with comprehensive documentation that users should consult before filing bugs: - -```yaml -# Change from: -- label: I have read the project documentation - required: false - -# To: -- label: I have read the project documentation - required: true -``` - -##### Removing PR Contribution Checkbox - -For projects that don't accept community contributions, remove this checkbox: - -```yaml -# Remove this block entirely: -- label: I am willing to submit a pull request to fix this issue - required: false -``` - -#### Environment Fields Customization - -##### Area Dropdown - -The Area dropdown is optional by default for template portability. Update the options to match your project's components: - -```yaml -options: - - Backend / API - - Frontend / UI - - CLI - - Documentation - - Other (describe/specify in Additional Context) -``` - -**Making it required:** For repos that rely on area-based routing, change `required: false` to `required: true`. - -##### Minimal Reproduction URL - -For library or framework projects, consider making this field required: - -```yaml -# Change from: -validations: - required: false - -# To: -validations: - required: true -``` - -Projects that don't accept external reproductions can remove this field entirely. - -##### Architecture Dropdown - -For cross-platform projects, consider making this required: - -```yaml -# Change from: -validations: - required: false - -# To: -validations: - required: true -``` - -Single-platform projects can remove this field entirely. - -##### Runtime Version Placeholders - -Update the placeholder examples to match your project's supported runtimes: - -```yaml -# Default (Python-focused): -placeholder: | - Python 3.13.1 (or your installed version) - PowerShell 7.4.6 or Windows PowerShell 5.1 - Markdown tooling/renderer (if relevant): e.g., Pandoc 3.1.2 - -# Node.js project example: -placeholder: | - Node.js 20.10.0 - npm 10.2.3 - -# .NET project example: -placeholder: | - .NET 8.0.1 - C# 12 -``` - -##### Shell/Terminal Field - -For non-CLI projects where shell environment isn't relevant, remove this field entirely. - -##### How Did You Run It? Placeholders - -Update the placeholder examples to match your project's dependency management approach: - -```yaml -# Default (Python-focused): -placeholder: | - # Python (using pyproject.toml) - python -m venv .venv - source .venv/bin/activate - pip install -e . - python -m your_package - -# Node.js example: -placeholder: | - npm install - npm run build - npm start - -# Docker example: -placeholder: | - docker build -t myapp . - docker run myapp -``` - -#### Bug Characteristics Customization - -##### Regression Fields - -For projects where version history or regression tracking is not relevant, remove these fields: - -- Remove the "Regression?" dropdown (`id: regression`) -- Remove the "Last Working Version" input (`id: last_working_version`) - -##### Severity Options - -Adjust the severity levels to match your project's triage workflow: - -```yaml -# Default: -options: - - Critical (system crash, data loss) - - High (major feature broken, no workaround) - - Medium (feature impaired, workaround exists) - - Low (minor inconvenience, cosmetic issue) - -# Alternative with P-levels: -options: - - P0 (production down) - - P1 (critical impact) - - P2 (moderate impact) - - P3 (low impact) -``` - -#### Additional Information Customization - -##### Related Issues Placeholder - -Update the cross-repo example to reference related projects in your ecosystem: - -```yaml -# Default: -placeholder: | - #123 - owner/repo-name#456 - -# Example for a monorepo or related projects: -placeholder: | - #123 - my-org/frontend#456 - my-org/backend#789 -``` - -### Feature Request Template Customization - -**File:** `.github/ISSUE_TEMPLATE/feature_request.yml` - -The feature request template shares many customization points with the bug report template. This section documents the unique or different options. - -#### Top-Level Metadata - -The same customizations apply as for the bug report template: - -- **Title Prefix:** Default is `"[Feature] "` -- **Labels:** Default is `enhancement` -- **Triage Label:** Uncomment after creating the label -- **Issue Type:** Use `type: Feature` for organization-level issue types -- **Assignees and Projects:** Same configuration as bug reports - -#### Shared Customizations with Bug Report - -The feature request template shares several customization points with the bug report template. See the [Bug Report Template Customization](#bug-report-template-customization) section for detailed instructions on: - -- **Making Documentation Check Required:** Change `required: false` to `required: true` for the "I have read the project documentation" checkbox -- **Area Dropdown Options:** Update the options to match your project's languages/components (e.g., Backend / API, Frontend / UI, CLI, etc.) - -> **Tip:** Keep the Area dropdown options consistent between bug report and feature request templates to maintain a unified user experience. - -#### Pre-flight Checklist - -##### Removing PR Contribution Checkbox - -Same as the bug report template—remove if your project doesn't accept community contributions: - -```yaml -# Remove this block: -- label: I am willing to submit a pull request to implement this feature - required: false -``` - -#### Feature Classification - -##### Priority Options - -Adjust priority levels to match your project's triage workflow: - -```yaml -# Default: -options: - - Critical (blocking my adoption/usage) - - High (significant impact on my workflow) - - Medium (would improve my experience) - - Low (nice to have) -``` - -##### Scope Options - -Adjust feature scope categories as needed: - -```yaml -# Default: -options: - - Major feature (new capability, significant change) - - Minor enhancement (improvement to existing feature) - - Quality of life (small improvement, polish) -``` - -### Documentation Issue Template Customization - -**File:** `.github/ISSUE_TEMPLATE/documentation_issue.yml` - -The documentation issue template is simpler than the other templates but still has customization points. - -#### Top-Level Metadata - -- **Title Prefix:** Default is `"[Docs] "` -- **Labels:** Default is `documentation` (a GitHub default label) -- **Triage Label:** Uncomment after creating the label - -> **Note:** The `documentation` label is a GitHub default label that exists in all new repositories. If your organization has renamed or deleted it, update accordingly. - -#### Location Placeholder URL - -The "Where is it?" field includes a generic placeholder that uses relative file paths: - -```yaml -placeholder: e.g., README.md#usage or docs/guide.md -``` - -For a more helpful user experience, you can update this placeholder to include a full URL example specific to your repository: - -```yaml -placeholder: e.g., https://github.com/your-org/your-repo/blob/HEAD/README.md#usage or docs/guide.md -``` - -Replace `your-org/your-repo` with your actual organization and repository name. - -> **Note:** The default uses simple file paths rather than full URLs to avoid reporters pasting literal placeholders. Updating this is optional and depends on your preference for guiding reporters. - -#### Documentation Version Field - -For projects that don't maintain versioned documentation, remove this field: - -```yaml -# Remove this entire block: -- type: input - id: doc_version - attributes: - label: Documentation Version (optional) - description: >- - If the documentation is versioned, which version are you viewing? - placeholder: e.g., v1.2.3, latest, main branch - validations: - required: false -``` - -#### Issue Type Dropdown - -Adjust the documentation issue type options to match your documentation structure: - -```yaml -# Default: -options: - - Typo / Grammar - - Unclear / Confusing - - Missing Information - - Broken Link - - Outdated Information - - Code Example Issue - - Formatting / Rendering - - Other - -# Simplified example: -options: - - Typo / Grammar - - Missing or Outdated Content - - Broken Link - - Other -``` - -To make this field required for structured documentation triage, change `required: false` to `required: true`. - ---- - -## Security Configuration - -**Files:** `SECURITY.md` and repository settings - -### Enabling Private Vulnerability Reporting - -> **Important:** Private vulnerability reporting is **only available for public repositories**. - -Private vulnerability reporting allows security researchers to report vulnerabilities directly to maintainers without public disclosure. - -**Steps:** - -1. Go to **Settings** > **Security** > **Private vulnerability reporting** -2. Click **Enable** - -**Optional:** After enabling, you can update the security link in `.github/ISSUE_TEMPLATE/config.yml` for a more direct path: - -```yaml -# Change from: -url: https://github.com/OWNER/REPO/security - -# To: -url: https://github.com/OWNER/REPO/security/advisories/new -``` - -### Customizing Supported Versions - -The default `SECURITY.md` includes a minimal supported versions table: - -```markdown -| Version | Supported | -| ------- | ------------------ | -| latest | :white_check_mark: | -``` - -**To customize for your versioning policy:** - -```markdown -| Version | Supported | -| ------- | ------------------ | -| 2.x | :white_check_mark: | -| 1.x | :white_check_mark: | -| < 1.0 | :x: | -``` - -Update this table to reflect which versions of your project receive security updates. - ---- - -## Code of Conduct Configuration - -**File:** `CODE_OF_CONDUCT.md` - -The template includes the Contributor Covenant v3.0, a widely-adopted code of conduct for open source projects. This section covers customization options and alternatives. - -### Alternative Code of Conduct Templates - -While the Contributor Covenant is the most widely used code of conduct for open source, you may choose a different template based on your project's needs: - -| Template | Description | Best For | -| --- | --- | --- | -| [Contributor Covenant v3.0](https://www.contributor-covenant.org/version/3/0/code_of_conduct/) | Comprehensive, widely recognized | Most open source projects (template default) | -| [Citizen Code of Conduct](https://github.com/stumpsyn/policies/blob/master/citizen_code_of_conduct.md) | Community-focused, detailed examples | Projects emphasizing community building | -| Organization-specific | Custom policies matching org standards | Enterprise or organizational projects | - -**To use a different template:** - -1. Replace the contents of `CODE_OF_CONDUCT.md` with your chosen template -2. Update any contact information or placeholders -3. Review enforcement procedures and adjust as needed - -### Customizing Enforcement Procedures - -The default enforcement section includes a four-tier ladder (Warning → Temporarily Limited Activities → Temporary Suspension → Permanent Ban). You may customize this based on your project's needs: - -#### Enforcement Contact Information - -Replace `[INSERT CONTACT METHOD]` with your preferred reporting method: - -```markdown -To report a possible violation, contact us via: conduct@your-project.org -``` - -**Contact method options:** - -- **Email address:** Simple, widely understood, but requires email monitoring -- **Web form:** Provides structured reporting, can integrate with issue tracking -- **Multiple channels:** List several options (email, form, direct message to maintainers) - -#### Response Timeline Commitments - -Consider adding explicit timeline commitments to the enforcement section: - -```markdown -Community Moderators will acknowledge receipt of reports within 48 hours and -aim to provide a resolution within 7 days for straightforward cases. Complex -cases may require additional time, and reporters will be updated on progress. -``` - -#### Scope Customization - -The default scope section covers community spaces and official representation. Customize based on your project's context: - -```markdown -## Scope - -This Code of Conduct applies within: - -- All repository spaces (issues, pull requests, discussions) -- Project communication channels (Slack, Discord, mailing lists) -- Project events (meetups, conferences, online gatherings) -- When representing the project in public spaces -``` - -### Removing the Code of Conduct - -Small personal projects or projects that don't accept external contributions may not need a code of conduct file. - -**To remove:** - -1. Delete `CODE_OF_CONDUCT.md` from your repository -2. The placeholder check workflow will continue to pass—`CODE_OF_CONDUCT.md` is treated as optional - -**Considerations before removing:** - -- Projects that grow to accept contributions later may want to add one -- Some organizations require a code of conduct for all projects -- Having a code of conduct signals that your project welcomes diverse contributors - ---- - -## Pull Request Template Customization - -**File:** `.github/pull_request_template.md` - -### Strengthening Pre-commit Language - -The default template uses conditional language for pre-commit verification: - -```markdown -### Pre-commit Verification (if configured) - -- [ ] If this repository uses pre-commit, I ran `pre-commit run --all-files` and all checks pass -- [ ] If pre-commit made auto-fixes, I reviewed and committed them -``` - -**If your repository uses pre-commit hooks**, replace with direct language: - -```markdown -### Pre-commit Verification - -- [ ] I have run `pre-commit run --all-files` locally and all checks pass -- [ ] I have reviewed and committed all auto-fixes made by pre-commit hooks -``` - -### Adding Language-Specific Sections - -Add checklist sections for your project's technology stack: - -**Node.js/TypeScript:** - -```markdown -### Node.js-Specific (if applicable) - -- [ ] `npm test` passes locally -- [ ] ESLint passes with no errors -- [ ] TypeScript compiles without errors -``` - -**.NET:** - -```markdown -### .NET-Specific (if applicable) - -- [ ] `dotnet test` passes locally -- [ ] No compiler warnings -- [ ] Code analysis passes -``` - -**Go:** - -```markdown -### Go-Specific (if applicable) - -- [ ] `go test ./...` passes locally -- [ ] `go vet ./...` passes -- [ ] `golint ./...` passes (if using golint) -``` - -**Rust:** - -```markdown -### Rust-Specific (if applicable) - -- [ ] `cargo test` passes locally -- [ ] `clippy` passes with no warnings -- [ ] `cargo fmt --check` shows no formatting issues -``` - -**Java:** - -```markdown -### Java-Specific (if applicable) - -- [ ] Maven/Gradle tests pass locally -- [ ] Checkstyle passes -- [ ] No compiler warnings -``` - -### Customizing Type of Change Options - -Add options relevant to your workflow: - -```markdown -- [ ] Refactoring (no functional changes) -- [ ] Security fix -- [ ] Performance improvement -``` - -Remove options that don't apply to your project. - -### Strengthening Test Requirements - -For mature projects with established test infrastructure, change: - -```markdown -- [ ] I have added or updated tests where appropriate -``` - -To: - -```markdown -- [ ] I have added tests for all new functionality -- [ ] I have updated tests for all modified functionality -``` - -### Contributing Guidelines Link - -The default PR template uses an absolute URL for the contributing guidelines link: - -```markdown -[contributing guidelines](https://github.com/OWNER/REPO/blob/HEAD/CONTRIBUTING.md) -``` - -`OWNER/REPO` follows this template's placeholder convention. Replace `OWNER/REPO` with your actual organization and repository name as part of template adoption. If you keep `.github/workflows/check-placeholders.yml` (an optional adoption step), CI will fail until this substitution is made; if you do not adopt that workflow or you remove it after initial setup, no CI guardrail catches a missed substitution and you must verify the replacement manually. - -**GHES adopters:** The `github.com` host is the assumed default and is **not** validated by `.github/workflows/check-placeholders.yml`. If your repository is hosted on GitHub Enterprise Server, you **MUST** also replace `github.com` with your GHES host (e.g., `github.company.com`); otherwise the link will point off-instance to GitHub.com. - -This is the canonical link form for files under `.github/` (issue forms and the PR template); see the **Issue and PR templates** carve-out in `.github/instructions/docs.instructions.md` for the full rule. Relative forms such as `../blob/HEAD/CONTRIBUTING.md` or `blob/HEAD/CONTRIBUTING.md` MUST NOT be used in `.github/ISSUE_TEMPLATE/*.yml` or `.github/pull_request_template.md`, because they either 404 in issue-form `value:` blocks or render unreliably across non-GitHub.com renderers, GitHub Mobile, and email notifications. - -**If your CONTRIBUTING.md is in a different location** (e.g., `docs/CONTRIBUTING.md`), update the path inside the absolute URL accordingly: - -```markdown -[contributing guidelines](https://github.com/OWNER/REPO/blob/HEAD/docs/CONTRIBUTING.md) -``` - -### Customizing Additional Notes Section - -The "Additional Notes" section provides PR authors a place to add context for reviewers that doesn't fit elsewhere. Consider adding prompts for common needs: - -```markdown -## Additional Notes - - - - - -``` - -### Customizing Related Issues Section - -The template uses `Closes #` for linking to issues. Update the syntax if your project uses different keywords: - -| Keyword | Effect | -| --- | --- | -| `Closes OWNER/REPO#123` | Closes the issue when PR is merged (default) | -| `Fixes OWNER/REPO#123` | Alternative keyword, same effect | -| `Resolves OWNER/REPO#123` | Alternative keyword, same effect | - -Choose one keyword and use it consistently across your project. - ---- - -## Dependabot Configuration - -**File:** `.github/dependabot.yml` - -### Adjusting Update Frequency - -The default configuration checks for updates weekly: - -```yaml -schedule: - interval: "weekly" -``` - -**Options:** - -- `"daily"`: More frequent updates, useful for security-critical projects -- `"weekly"`: Balanced approach (default) -- `"monthly"`: Less frequent updates, reduces PR volume - -### Adjusting PR Limits - -The `open-pull-requests-limit` controls how many Dependabot PRs can be open simultaneously: - -```yaml -open-pull-requests-limit: 10 -``` - -- **Increase** if you have a large dependency tree and want faster updates -- **Decrease** if Dependabot PRs are overwhelming your team - -### Adding Automatic Reviewers - -Automatically assign reviewers and assignees to Dependabot PRs: - -```yaml -- package-ecosystem: "npm" - directory: "/" - schedule: - interval: "weekly" - reviewers: - - "username1" - - "org/team-name" - assignees: - - "username2" -``` - -### Customizing Commit Message Prefixes - -The default prefix is `chore(deps)`: - -```yaml -commit-message: - prefix: "chore(deps)" -``` - -**To match your project's commit conventions:** - -```yaml -commit-message: - prefix: "deps" # Simpler prefix - prefix: "build(deps)" # For projects using build scope - prefix: "fix(deps)" # If you treat dependency updates as fixes -``` - ---- - -## Pre-commit Configuration - -**File:** `.pre-commit-config.yaml` - -### Adjusting Line Length - -The default line length is 100 characters for both Black and Ruff: - -```yaml -- repo: https://github.com/psf/black - rev: 26.1.0 - hooks: - - id: black - args: [--line-length=100] - -- repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.14.14 - hooks: - - id: ruff-check - args: [--fix, --line-length=100] -``` - -**To use Black's default (88 characters):** - -```yaml -args: [--line-length=88] -``` - -> **Note:** Ensure both Black and Ruff use the same line length to avoid conflicts. - -### Adding Hooks for Other Languages - -**Prettier (JavaScript/TypeScript/CSS):** - -```yaml -- repo: https://github.com/pre-commit/mirrors-prettier - rev: v3.1.0 - hooks: - - id: prettier - types_or: [javascript, jsx, ts, tsx, css, markdown] -``` - -> **JSON/JSONC and YAML are intentionally omitted from `types_or` above.** For JSON/JSONC, see [Prettier for JSON/JSONC (Opt-in)](#prettier-for-jsonjsonc-opt-in), which recommends a `repo: local` hook over `pre-commit/mirrors-prettier` to avoid pinning the Prettier version in two places. Prettier for YAML is discouraged by default in this template; see the YAML subsection of that same section for the reconciliation requirements if a project chooses to adopt it anyway. - -**ESLint:** - -```yaml -- repo: https://github.com/pre-commit/mirrors-eslint - rev: v8.56.0 - hooks: - - id: eslint - files: \.[jt]sx?$ - additional_dependencies: - - eslint@8.56.0 - - eslint-config-your-config -``` - -**ShellCheck (Bash/Shell scripts):** - -```yaml -- repo: https://github.com/shellcheck-py/shellcheck-py - rev: v0.9.0.6 - hooks: - - id: shellcheck -``` - -### Switching to Husky - -If your project prefers Husky for git hooks: - -1. Remove `.pre-commit-config.yaml` -2. Install Husky: - - ```bash - npm install husky --save-dev - ``` - -3. Add the prepare script to `package.json`: - - ```json - { - "scripts": { - "prepare": "husky" - } - } - ``` - -4. Create `.husky/pre-commit`: - - ```bash - #!/usr/bin/env sh - . "$(dirname -- "$0")/_/husky.sh" - - npm run lint - npm test - ``` - -> **Warning:** Pre-commit and Husky both manage `.git/hooks/pre-commit`. Do NOT run `pre-commit install` if using Husky, as the two tools conflict. - ---- - -## Schema Validation Configuration - -**Files:** `.pre-commit-config.yaml`, `schemas/`, project test directory (for example, `tests/`) - -This template ships `schemas/` with one clearly removable worked example (see [`schemas/README.md`](schemas/README.md)). **The default-enabled `check-jsonschema` configuration covers the worked example plus one real load-bearing repository file: `.github/dependabot.yml` is validated against the bundled `vendor.dependabot` schema.** Beyond that, schema validation is opt-in and SHOULD be added per real schema-backed file family. If your downstream project does not need the worked example, follow the canonical [downstream removal checklist](schemas/README.md#downstream-removal-checklist) in `schemas/README.md` to remove it. - -### When to Add `check-jsonschema` - -Add a [`check-jsonschema`](https://github.com/python-jsonschema/check-jsonschema) hook when you have: - -1. A real schema under `schemas/` (for example, `schemas/project-config.schema.json`), and -2. A real, identifiable file family that the schema validates (for example, every `config/*.json` in your project). - -Do **not** add `check-jsonschema` hooks for schemas that do not yet exist, and do not configure a generic "validate every JSON/YAML file" hook. Pre-commit already runs `check-json` and `check-yaml` for syntax; `check-jsonschema` is for contract validation against a specific schema. - -### Adding a Hook for One File Family - -Add one hook per `(schema, file family)` pair to `.pre-commit-config.yaml`: - -```yaml -- repo: https://github.com/python-jsonschema/check-jsonschema - rev: 0.33.3 - hooks: - - id: check-jsonschema - name: Validate project JSON config - files: ^config/.*\.json$ - args: - - --schemafile - - schemas/project-config.schema.json -``` - -> **Version pinning.** Implementers MUST verify and pin a current upstream version of `check-jsonschema` when enabling the hook, rather than copying the example `rev:` value above. Look up the latest tagged release at the upstream repository before adoption and update the pin via your normal dependency-update process (for example, Dependabot's `pre-commit` ecosystem). - -If you have multiple schema-backed file families, add one hook **per family**, each scoped with its own `files:` pattern. Do not combine unrelated families into a single hook. - -### Testing Valid Examples - -Sample data files that should validate cleanly can be placed in either of two locations, with different validation consequences: - -- **Inside the file family path covered by the family hook** (for example, `config/example.valid.json` for the hook above). The family hook will validate these automatically because they match its `files:` pattern. -- **Under `schemas/examples//{valid,invalid}/`** (for example, `schemas/examples/project-config/valid/minimal.json`). This `schemas/examples//{valid,invalid}/` layout is the convention used by `schemas/README.md` and the pytest tests referenced below, but it does **not** match the family hook's `files:` pattern, so these examples need a separate validation path. Choose one of: - - - Add a dedicated `check-jsonschema` hook scoped to **valid** fixtures under `schemas/examples/` only (for example, `files: ^schemas/examples/project-config/valid/.*\.json$`). Anchor the pattern under the `valid/` directory so the hook does not pick up `invalid/` fixtures (which MUST NOT be wired into a normal `check-jsonschema` hook — see the next subsection). This aligns with the `schemas/examples//{valid,invalid}/` layout used in `schemas/README.md` § Examples and exercised by both [`tests/test_schema_examples.py`](tests/test_schema_examples.py) and [`templates/python/tests/test_schema_examples.py`](templates/python/tests/test_schema_examples.py). - - Run `check-jsonschema` directly from a CI step or local script: - - ```bash - check-jsonschema \ - --schemafile schemas/project-config.schema.json \ - schemas/examples/project-config/valid/minimal.json - ``` - - - Wire the example into the opt-in pytest template described under [Testing Invalid Examples](#testing-invalid-examples) (it exercises both valid and invalid cases). - -Whichever placement you choose, valid examples MUST exit with code `0`; a non-zero exit indicates a broken example or a schema regression and MUST be fixed before merging. - -### Testing Invalid Examples - -Invalid examples (intentionally malformed fixtures used to prove that the schema rejects bad input) MUST NOT be wired into a normal `check-jsonschema` pre-commit hook, because the validator's non-zero exit would be reported as a hook failure on every run. Instead, write a test or script that **asserts validation fails**. - -A starter pytest template lives at [`templates/python/tests/test_schema_examples.py`](templates/python/tests/test_schema_examples.py); the active, canonical version that this repository runs in CI lives at [`tests/test_schema_examples.py`](tests/test_schema_examples.py). Both auto-discover `(schema, example, expected_to_pass)` triples from `schemas/*.schema.json` and `schemas/examples//{valid,invalid}/`. To use the starter: - -1. Copy the file into your project's real `tests/` directory. -2. Place schemas under `schemas/.schema.json` and examples under `schemas/examples//{valid,invalid}/`. Discovery is automatic — no per-case configuration is required. -3. Choose how to make `check-jsonschema` available to the test environment (see below). - -### Making `check-jsonschema` Available to Tests - -The active root test [`tests/test_schema_examples.py`](tests/test_schema_examples.py) requires `check-jsonschema`, which is declared in the root `pyproject.toml` `dev` dependency group and installed by `.github/workflows/python-ci.yml` via `pip install -e ".[dev]"`. The starter under `templates/python/tests/test_schema_examples.py` retains a `skipif` guard so that downstream projects can copy it before adding the dependency. If you want the starter to run unconditionally in your repository, do one of the following: - -- **Add it as a dev/test dependency.** For example, in `pyproject.toml`: - - ```toml - [project.optional-dependencies] - dev = [ - "pytest", - "check-jsonschema", - ] - ``` - - Then install with `pip install -e ".[dev]"` (or your preferred dev-install command) so `check-jsonschema` is on `PATH` for tests. - -- **Install it in CI only.** Add an explicit install step (for example, `pip install check-jsonschema`) in the relevant CI workflow before the test step. Local developers without the tool installed will see the test skip rather than fail. - -If you remove the `skipif` guard, you MUST ensure `check-jsonschema` is installed in every environment where the test runs; otherwise the test will fail with a `FileNotFoundError`-style error rather than a clear "tool missing" message. - -### Defaults Recap - -- The template ships **two** active `check-jsonschema` configurations by default: - 1. The worked-example schema (`schemas/example-config.schema.json`) and its valid example data files. A companion `check-metaschema` hook self-validates the schema against its declared JSON Schema Draft 2020-12 metaschema. - 2. A `check-jsonschema --builtin-schema vendor.dependabot` hook that validates `.github/dependabot.yml` against the Dependabot built-in schema bundled with `check-jsonschema`. See the [Built-in Schema Validation for Real Load-Bearing Configuration Files](.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-built-in-schema-validation-for-real-load-bearing-configuration-files) ADR for the policy, the explicit "Evaluated but deferred" negative-space record, and the downstream removal guidance. - - Downstream repositories MAY add additional `check-jsonschema` hook entries (project-owned `--schemafile` hooks for their own schema-backed file families, or additional `--builtin-schema` hooks for tool-owned configuration files), and MAY remove the worked example via the canonical [downstream removal checklist](schemas/README.md#downstream-removal-checklist). Hooks for files that a downstream repository deletes **MUST** be removed alongside the file. -- The template ships an active root test, [`tests/test_schema_examples.py`](tests/test_schema_examples.py), that depends on `check-jsonschema` (declared in the root `pyproject.toml` `dev` group). A starter version with the same essential pattern is provided at `templates/python/tests/test_schema_examples.py` for downstream adoption. -- Beyond the worked example and the wired built-in schema hooks, schema validation is opt-in; downstream repositories add real hooks and tests when they introduce additional real schemas, or when they want to enable additional `--builtin-schema` coverage. -- Project-owned schemas SHOULD continue to live under `schemas/`. Built-in schemas referenced via `--builtin-schema` are **not** vendored into `schemas/`; their content tracks `check-jsonschema` releases. -- JSONC, JSON5, ecosystem-specific YAML validators, and broader SchemaStore / catalog coverage remain opt-in unless a downstream repository explicitly ships them. See the dedicated subsections below for adoption guidance. - ---- - -## Prettier for JSON/JSONC (Opt-in) - -**Files:** `package.json`, `.prettierrc.json`, `.prettierignore`, `.pre-commit-config.yaml` - -[Prettier](https://prettier.io/) is **not** part of the default JSON/YAML toolchain in this template. Prettier MAY be adopted on a per-repository basis to enforce JSON and JSONC formatting beyond what `check-json` validates. **No Prettier configuration, dependency, or hook is added to the repo root by default.** Adoption is an explicit project decision. - -> **Background.** The default pre-commit stack uses `check-json` (strict `.json` syntax only), `check-yaml`, `yamllint`, and `actionlint`. See [Schema Validation Configuration](#schema-validation-configuration) for the schema-validation path, and the [Prettier deferral ADR in `.github/TEMPLATE_DESIGN_DECISIONS.md`](.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-prettier-deferral-for-data-files) for the rationale behind keeping Prettier opt-in. - -### When to Adopt Prettier for JSON/JSONC - -Adopt Prettier for JSON/JSONC when **at least one** of the following is true: - -- The project already uses Node tooling for other purposes (for example, `markdownlint-cli2`, JavaScript/TypeScript builds), so adding Prettier does not introduce a new ecosystem. -- The team wants editor format-on-save consistency across contributors. -- Many JSON/JSONC files in the project benefit from formatter enforcement (consistent indentation, trailing newline, key spacing) that `check-json` does not provide. -- The project has many `.jsonc` files and wants formatting enforcement, since `check-json` does not validate `.jsonc`. Prettier is **one option among several pieces of JSONC-aware tooling** that can fill this gap (other options include dedicated JSONC parsers and schema validators that understand JSONC). - -### When Not to Adopt Prettier for JSON/JSONC - -Do **not** adopt Prettier when: - -- Adding Node tooling would create friction (for example, the project is otherwise pure Python, PowerShell, or Terraform with no existing Node dependency). -- `check-json` syntax validation alone is sufficient for the project's JSON files. -- The project does not want formatter-controlled diffs on data files (Prettier rewrites files in-place; downstream consumers MUST be comfortable with formatter-driven churn). - -### Important Caveat: Prettier Does Not Sort JSON Keys - -Prettier formats JSON whitespace, indentation, and line endings, but it **does not** sort object keys. Stable key ordering — often the highest-value JSON-stability property — must still be enforced by hand or by a separate tool. Adopters who care about deterministic key order MUST treat Prettier as a formatting tool only and add a separate key-ordering process if needed. - -### Recommended `package.json` Scripts - -If Prettier is adopted, add `format:json` and `lint:json:format` scripts to `package.json` so that contributors and CI use the same commands: - -```json -{ - "scripts": { - "format:json": "prettier --write \"**/*.{json,jsonc}\" \"!node_modules/**\"", - "lint:json:format": "prettier --check \"**/*.{json,jsonc}\" \"!node_modules/**\"" - } -} -``` - -The `--check` variant is suitable for CI; the `--write` variant rewrites files in place locally. - -### Recommended `.prettierrc.json` - -A minimal Prettier configuration that aligns with the template's other formatting defaults (LF line endings, 2-space indent): - -```json -{ - "printWidth": 120, - "tabWidth": 2, - "useTabs": false, - "endOfLine": "lf", - "trailingComma": "none" -} -``` - -`trailingComma: "none"` is required for strict `.json`; for `.jsonc` consumers that accept trailing commas, this setting is still safe (Prettier will simply omit them). - -### Recommended `.prettierignore` - -Add a `.prettierignore` file that excludes generated files, dependencies, and any directories that should not be reformatted. A typical baseline: - -```text -node_modules/ -package-lock.json -dist/ -build/ -coverage/ -.venv/ -``` - -Adopters SHOULD review their repository for additional generated or vendored content (for example, lockfiles, cached schema bundles, fixtures whose formatting is asserted by tests) and add those paths as well. - -### Optional Local Pre-commit Hook - -If the project already runs Prettier from `package.json` scripts, a `repo: local` pre-commit hook keeps the version under project control rather than pinning a separate Prettier mirror: - -```yaml -- repo: local - hooks: - - id: prettier-json-check - name: Check JSON and JSONC formatting with Prettier - entry: npx --no-install prettier --check "**/*.{json,jsonc}" "!node_modules/**" - language: system - files: '\.(json|jsonc)$' - pass_filenames: false -``` - -The `files:` regex scopes the hook so pre-commit only invokes it on commits that touch `.json` or `.jsonc` files; without that filter, `pass_filenames: false` would cause the hook to run (and re-scan the repo's JSON/JSONC globs) on every commit. `pass_filenames: false` lets the glob in `entry:` decide which files Prettier inspects, which keeps the hook consistent with the `lint:json:format` script. The `--no-install` flag tells `npx` to fail rather than fetch an unpinned Prettier on demand, so the hook always uses the version recorded in `package.json` / `package-lock.json`. - -> **Node availability requirement.** Because this hook shells out to `npx`, every environment that runs `pre-commit` MUST have Node.js installed and the project's npm dependencies installed (typically via `npm ci`). The template's default CI workflows do not install Node, so adopters who add this hook MUST also add a Node setup step (for example, `actions/setup-node` followed by `npm ci`) to any workflow that runs `pre-commit run --all-files`, and ensure the same is true on contributor workstations. - - - -> **`pre-commit/mirrors-prettier` is not the recommended path.** The `pre-commit/mirrors-prettier` repository pins a Prettier version separately from the project's `package.json`, which creates two sources of truth for the Prettier version. Prefer the `repo: local` hook above so that `package.json`, CI scripts, and the pre-commit hook all use the same Prettier version. - -### YAML - -Prettier for YAML is **discouraged by default** in this template. Prettier's YAML output diverges from idiomatic `yamllint` defaults (line wrapping, flow vs. block style, quoting), and running both without explicit reconciliation produces churn-only diffs. - -If a project chooses to adopt Prettier for YAML anyway, it MUST reconcile `yamllint` and Prettier so they do not fight. At minimum: - -- Pin both tools. -- Adjust `.yamllint.yml` rules (for example, `line-length`, `quoted-strings`, `indentation`) to match Prettier's output, or configure Prettier to match `.yamllint.yml`. -- Run both tools on a representative sample and confirm a clean pass before enabling either in CI. - -### Docker Alternative - -Projects that do not want a local Node.js installation MAY run Prettier from a Docker image in CI (for example, the official `node:` images or community-maintained Prettier images). The `package.json` scripts and the `.prettierrc.json` configuration above remain unchanged; only the invocation differs. Local developers can still use `npx prettier` if they have Node available, or rely on CI to enforce formatting. - -### Editor Formatter Guidance - -The template's `.vscode/settings.json` only prescribes the PowerShell file encoding (see [VS Code PowerShell File Encoding for Non-ASCII Characters](#vs-code-powershell-file-encoding-for-non-ascii-characters)) and does **not** prescribe editor formatters for any language. Editor-formatter guidance for JSON, JSONC, and YAML is therefore intentionally omitted here; downstream consumers who adopt Prettier MAY add `[json]`, `[jsonc]`, and `[yaml]` formatter sections to their own `.vscode/settings.json` at their discretion. - ---- - -## JSONC Validation (Opt-in) - -**Files:** `.pre-commit-config.yaml`, JSONC-aware tooling of your choice - -The default `check-json` pre-commit hook is anchored with `files: \.json$`, so it deliberately does **not** validate `.jsonc` files. JSONC (JSON with comments) is a superset that strict JSON parsers reject, so `check-json` would mis-flag every JSONC file with comments or trailing commas as a syntax error. - -**Recommendation:** Repositories with load-bearing JSONC files (for example, configuration that downstream tools read with a JSONC-aware parser) SHOULD adopt **JSONC-aware tooling** rather than retrofitting `check-json` to accept JSONC. Candidate approaches include: - -- A JSONC-aware parser invoked from a `repo: local` pre-commit hook (for example, a small Python or Node.js script that uses a JSONC parser library). -- A JSONC-aware schema validator if the JSONC files are also schema-backed. -- Prettier with a JSONC parser (see [Prettier for JSON/JSONC (Opt-in)](#prettier-for-jsonjsonc-opt-in)) for formatting only — Prettier does not enforce strict syntax in the same way `check-json` does. - -**Why the default `check-json` regex is anchored.** The anchored regex is intentional and reflects an explicit design decision: `.json` and `.jsonc` are different dialects with different parsers, and validating both with the same strict-JSON tool would produce false positives on every JSONC comment or trailing comma. Retrofitting `check-json` is therefore **not** the recommended path; add separate JSONC-aware tooling instead. - ---- - -## JSON5 Support (Opt-in) - -**Files:** `.pre-commit-config.yaml`, JSON5-aware tooling of your choice - -JSON5 is **not** enabled by default in this template. The JSON authoring guide ([`.github/instructions/json.instructions.md`](.github/instructions/json.instructions.md)) intentionally omits `.json5`, and no pre-commit hook validates JSON5 syntax out of the box. - -**Recommendation:** Adopting JSON5 SHOULD be an explicit, documented project decision rather than an implicit drift from JSON. If a downstream project adopts JSON5: - -- Document the decision (which directories, which consumers, why JSON5 over strict JSON or JSONC). -- Add a JSON5-aware parser or validator (for example, a `repo: local` hook that uses a JSON5 parser library). -- Confirm that all consumers of the JSON5 files actually support JSON5 syntax — many tools that accept "loose JSON" only accept JSONC, not full JSON5. - -See the [JSON5 exclusion ADR in `.github/TEMPLATE_DESIGN_DECISIONS.md`](.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-json5-exclusion-by-default) for the rationale behind keeping JSON5 out of the default toolchain. - ---- - -## Ecosystem-Specific YAML Validators (Opt-in) - -**Files:** `.pre-commit-config.yaml`, `.github/workflows/*.yml`, ecosystem-specific configuration files - -The default YAML toolchain (`check-yaml`, `yamllint`, `actionlint`) covers syntax, style, and GitHub Actions workflow correctness. Ecosystem-specific validators add semantic checks for particular YAML dialects and SHOULD be adopted **only when the repository actually uses those ecosystems** — adding a validator for a stack the project does not use creates noise without enforcing anything useful. - -| Ecosystem | Recommended Validator | Notes | -| --- | --- | --- | -| Kubernetes manifests | [`kubeconform`](https://github.com/yannh/kubeconform) (or equivalent, e.g. `kubeval`, `kubectl --dry-run`) | Validates manifest schemas against the relevant Kubernetes API version. | -| OpenAPI / AsyncAPI | [`spectral`](https://github.com/stoplightio/spectral) | Lints API contracts against built-in or custom rulesets. | -| AWS CloudFormation | [`cfn-lint`](https://github.com/aws-cloudformation/cfn-lint) | Validates CloudFormation templates against AWS resource specifications. | -| Ansible playbooks/roles | [`ansible-lint`](https://github.com/ansible/ansible-lint) | Enforces Ansible best practices and detects common errors. | -| Helm charts | [`helm lint`](https://helm.sh/docs/helm/helm_lint/) (and optionally `kubeconform` post-render) | Validates chart structure and renders templates for further validation. | - -Adopters MAY add these as additional pre-commit hooks (or as separate CI jobs) when their repository contains the relevant file types. Do **not** add them speculatively. - ---- - -## Future SchemaStore Validation for Repository Configuration Files - -**Status:** Mixed — `.github/dependabot.yml` is validated by default (see [Schema Validation Configuration](#schema-validation-configuration) and the [Built-in Schema Validation for Real Load-Bearing Configuration Files ADR](.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-built-in-schema-validation-for-real-load-bearing-configuration-files)). Other candidates remain future work and are not enabled by default. - -Several common repository configuration files have public schemas published on [SchemaStore](https://www.schemastore.org/) or maintained by their respective ecosystems. Wiring schema validation for any of the **deferred** candidates below is out of scope for the worked example shipped under `schemas/` and requires an explicit downstream project decision. - -Candidate files (cross-referenced from the [Future Work section in `schemas/README.md`](schemas/README.md#future-work) and the [Built-in Schema Validation for Real Load-Bearing Configuration Files ADR](.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-built-in-schema-validation-for-real-load-bearing-configuration-files)): - -- **`package.json`** — schema available on SchemaStore. A downstream project MAY add a `check-jsonschema` hook scoped to `^package\.json$` that points at the SchemaStore-published schema (for example, via `check-jsonschema`'s built-in `vendor.package-json` schema selector if available). Useful when `package.json` carries non-trivial metadata that should be validated beyond basic JSON syntax. -- **Generated package-manager lockfiles** (for example, `package-lock.json`, `yarn.lock` in JSON form, `composer.lock`) — schema validation MAY be useful, but only if it does **not** conflict with the package manager's own validation. Most package managers already validate their lockfiles internally; an additional schema check is justified only when a stable schema-backed validation path adds value the package manager does not already provide. -- **GitHub Actions workflow files** — already covered by `actionlint`. An additional `check-jsonschema` hook against the SchemaStore Actions schema would be **redundant** with `actionlint` and is **not recommended**. See the "Evaluated but deferred" subsection of the [Built-in Schema Validation ADR](.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-built-in-schema-validation-for-real-load-bearing-configuration-files) for the durable rationale. - -For all of the above, follow the schema-validation guidance in [Schema Validation Configuration](#schema-validation-configuration): add **one `check-jsonschema` hook per real schema-backed file family**, scoped to the files that family covers, and keep the default toolchain free of placeholder hooks. See the [Future Work section in `schemas/README.md`](schemas/README.md#future-work) for additional context and avoid restating that future-work note here. - ---- - -## Removing the Worked Example Schema - -**Files:** `schemas/`, `.pre-commit-config.yaml`, `tests/test_schema_examples.py`, `.github/workflows/data-ci.yml`, related documentation - -The template ships a worked-example schema (`schemas/example-config.schema.json`), valid and invalid example fixtures under `schemas/examples/example-config/`, two pre-commit hooks (`check-jsonschema` for valid examples, `check-metaschema` for the schema itself), and the schema-example pytest contract at [`tests/test_schema_examples.py`](tests/test_schema_examples.py). The worked example is intentionally easy to remove for downstream repositories that do not use schema-backed validation. - -**To remove the worked example, follow the canonical [Downstream Removal Checklist](schemas/README.md#downstream-removal-checklist) in `schemas/README.md`.** That checklist is the single source of truth for the exact files, hooks, and references to remove; do not improvise the steps from memory and do not duplicate the checklist here. - ---- - -## Markdown Linting Configuration - -**File:** `.markdownlint.jsonc` - -### Customizing Style Rules - -The following rules can be configured to match your project's documentation style: - -| Rule | Description | Default | Options | -| --- | --- | --- | --- | -| MD003 | Heading style | `atx` | `atx`, `setext`, `consistent` | -| MD004 | Unordered list marker | `dash` | `dash`, `asterisk`, `plus`, `consistent` | -| MD012 | Maximum consecutive blank lines | `2` | Any positive integer | -| MD024 | Multiple headings with same content | `siblings_only: true` | `true`, `false`, or object with `siblings_only` | -| MD029 | Ordered list prefix | `ordered` | `ordered`, `one`, `one_or_ordered` | -| MD035 | Horizontal rule style | `---` | `---`, `***`, `___`, `consistent` | -| MD048 | Code fence style | `backtick` | `backtick`, `tilde`, `consistent` | -| MD049 | Emphasis style | `asterisk` | `asterisk`, `underscore`, `consistent` | -| MD050 | Strong style | `asterisk` | `asterisk`, `underscore`, `consistent` | - -**Example: Using asterisks for unordered lists:** - -```jsonc -{ - "MD004": { - "style": "asterisk" - } -} -``` - -**Example: Allowing up to 3 consecutive blank lines:** - -```jsonc -{ - "MD012": { - "maximum": 3 - } -} -``` - -**Example: Disallowing duplicate headings entirely:** - -```jsonc -{ - "MD024": { - "siblings_only": false - } -} -``` - -### Re-enabling Disabled Rules - -Several rules are disabled by default because they are not auto-fixable: - -| Rule | Why Disabled | When to Enable | -| --- | --- | --- | -| MD013 | Line length not auto-fixable | If you want to enforce line length limits | -| MD034 | Bare URLs may be intentional | If you want all URLs to use link syntax | -| MD036 | False positives with bold text | If you want to prevent emphasis as headings | -| MD041 | Prevents badges/metadata at start | If you require first line to be a heading | - -**To re-enable a rule:** - -```jsonc -{ - "MD013": { - "line_length": 120 - }, - "MD041": true -} -``` - -### Using the cli2-Specific Configuration Format - -The repository includes an alternative configuration template at `templates/markdown/.markdownlint-cli2.jsonc` that uses the `markdownlint-cli2` specific format where rules are nested under a `"config"` key. - -#### When to Use Each Format - -| File | Format | Use Case | -| --- | --- | --- | -| `.markdownlint.jsonc` | Standard (rules at root) | Default choice; works with both `markdownlint-cli` and `markdownlint-cli2` | -| `.markdownlint-cli2.jsonc` | cli2-specific (rules under `"config"`) | When you need cli2-specific features like `globs`, `ignores`, `customRules`, or `frontMatter` parser options | - -Both files contain **identical linting rules**; only the structure differs. - -#### Switching to the cli2-Specific Format - -If you want to use the cli2-specific format, copy the template to your repository root and remove the original configuration file. - -**Windows (PowerShell):** - -```powershell -Copy-Item -Path "templates/markdown/.markdownlint-cli2.jsonc" -Destination ".markdownlint-cli2.jsonc" -Remove-Item -Path ".markdownlint.jsonc" -Force -``` - -**macOS/Linux/FreeBSD:** - -```bash -cp templates/markdown/.markdownlint-cli2.jsonc .markdownlint-cli2.jsonc -rm .markdownlint.jsonc -``` - -> **Note:** When using `.markdownlint-cli2.jsonc`, any rule customizations must be made inside the `"config"` block, not at the root level of the JSON. - -**See also:** [markdownlint-cli2 documentation](https://github.com/DavidAnson/markdownlint-cli2) for additional cli2-specific configuration options. - ---- - -## Nested Markdown Linting Configuration - -**File:** `.github/scripts/lint-nested-markdown.js` - -This optional script lints Markdown content embedded within code fences (` ```markdown ` or ` ```md `) in Markdown files. This is useful for documentation-heavy projects that include Markdown examples, ensuring that nested Markdown content follows the same linting rules as the outer Markdown files. - -### How to Run - -**Scan all Markdown files in the repository:** - -```bash -npm run lint:md:nested -``` - -**Lint specific files:** - -```bash -node .github/scripts/lint-nested-markdown.js file1.md file2.md -``` - -When file arguments are provided, only those files are linted (useful for pre-commit hooks). When no arguments are provided, all `.md` files are scanned via glob (excluding `node_modules`). - -### Automatic Rule Adjustments - -The script automatically disables two rules for nested Markdown content: - -| Rule | Description | Why Disabled | -| --- | --- | --- | -| MD041 | First line in a file should be a top-level heading | Nested Markdown snippets may not start with a top-level heading | -| MD051 | Link fragments should be valid | Nested Markdown often contains example/placeholder links that reference anchors in other documents | - -### Pre-commit Integration (Optional) - -If you want to run this script as a pre-commit hook, you can add the following to your `.pre-commit-config.yaml`: - -```yaml -- repo: local - hooks: - - id: lint-nested-markdown - name: Lint nested Markdown - entry: node .github/scripts/lint-nested-markdown.js - language: node - files: \.md$ - pass_filenames: true -``` - -### When to Use This Feature - -This feature is most useful for: - -- **Documentation-heavy projects** with Markdown examples in code blocks -- **Template repositories** that include example Markdown snippets -- **Projects with contributing guides** that show Markdown formatting examples - -### Removing This Feature - -If you decide you don't need nested markdown linting, you can remove this optional feature to reduce your dependency footprint. The script and its dependencies are not required for the core functionality of this template. - -> **Note:** Removing this feature is optional. The script doesn't cause any problems if left in place—it simply won't be used if you don't invoke it. - -**Steps to remove:** - -1. **Delete the script file:** - - **Windows (PowerShell):** - - ```powershell - Remove-Item -Path ".github/scripts/lint-nested-markdown.js" -Force - ``` - - **macOS/Linux/FreeBSD:** - - ```bash - rm .github/scripts/lint-nested-markdown.js - ``` - -2. **Remove the npm script from `package.json`:** - - Open `package.json` and delete the `lint:md:nested` line from the `scripts` section. For example: - - ```json - { - "scripts": { - "lint:md": "markdownlint-cli2 \"**/*.md\" \"#node_modules\"", - "lint:md:nested": "node .github/scripts/lint-nested-markdown.js", ← Delete this line - ... - } - } - ``` - - > **Note:** Keep all other scripts in the section; only remove the `lint:md:nested` line. - -3. **Remove the npm dependencies only used by this script:** - - **Windows (PowerShell):** - - ```powershell - npm uninstall glob jsonc-parser markdown-it - ``` - - **macOS/Linux/FreeBSD:** - - ```bash - npm uninstall glob jsonc-parser markdown-it - ``` - -4. **Update the lock file:** - - Run npm install to update `package-lock.json` to reflect the removed dependencies: - - ```bash - npm install - ``` - -5. **If you added pre-commit integration, remove the hook:** - - Open `.pre-commit-config.yaml` and remove the `lint-nested-markdown` hook section: - - ```yaml - # Remove this entire block if present: - - repo: local - hooks: - - id: lint-nested-markdown - name: Lint nested Markdown - entry: node .github/scripts/lint-nested-markdown.js - language: node - files: \.md$ - pass_filenames: true - ``` - ---- - -## Markdown Lint Workflow Configuration - -**File:** `.github/workflows/markdownlint.yml` - -The Markdown Lint workflow enforces consistent Markdown formatting across your repository by running markdownlint on every push and pull request. While it works out-of-the-box, you can customize it to match your project's needs. - -### Restricting Branch Triggers - -The default configuration runs on all branches: - -```yaml -on: - push: - branches: ["**"] - pull_request: - branches: ["**"] -``` - -**To run only on the default branch:** - -```yaml -on: - push: - branches: ["main"] - pull_request: - branches: ["main"] -``` - -**To run on multiple specific branches:** - -```yaml -on: - push: - branches: ["main", "develop"] - pull_request: - branches: ["main", "develop"] -``` - -### Adding Path Filters - -By default, the workflow runs on every push and pull request regardless of which files changed. To only run the workflow when Markdown files or linting configuration changes: - -```yaml -on: - push: - branches: ["main"] - paths: - - '**/*.md' - - '.markdownlint.jsonc' - - 'package.json' - - 'package-lock.json' - pull_request: - branches: ["main"] - paths: - - '**/*.md' - - '.markdownlint.jsonc' - - 'package.json' - - 'package-lock.json' -``` - -> **Note:** Include configuration files in the path filter to ensure the workflow runs when linting rules change. - -### Changing Node.js Version - -The workflow uses Node.js 20 by default: - -```yaml -- name: Setup Node.js - uses: actions/setup-node@v6 - with: - node-version: '20' -``` - -**To use a different Node.js version:** - -```yaml -- name: Setup Node.js - uses: actions/setup-node@v6 - with: - node-version: '22' -``` - -> **Note:** Ensure the Node.js version you choose is compatible with your project's dependencies. Check the markdownlint-cli2 documentation for supported Node.js versions. - -### Disabling Nested Markdown Linting in CI - -The workflow runs two linting steps: one for outer Markdown files and one for nested Markdown code fences. If you want to keep the outer linting but disable the nested linting step in CI: - -**Option 1: Remove the step entirely** - -Delete or comment out the nested linting step: - -```yaml -# Remove or comment out this step: -# - name: Run markdownlint on nested Markdown code fences -# id: lint-nested -# continue-on-error: true -# run: npm run lint:md:nested -``` - -And update the final check step to only check the outer linting result: - -```yaml -- name: Check linting results - if: steps.lint-outer.outcome == 'failure' - run: | - echo "::error::Markdown linting failed. Check the logs above for details." - exit 1 -``` - -**Option 2: Skip the step conditionally** - -Add a condition to skip the nested linting step: - -```yaml -- name: Run markdownlint on nested Markdown code fences - id: lint-nested - if: false # Disabled - remove this line to re-enable - continue-on-error: true - run: npm run lint:md:nested -``` - -> **Note:** If you disable nested linting in CI, you may still want to run it locally using `npm run lint:md:nested` to catch issues before pushing. - -### Removing the Workflow - -If your project doesn't need Markdown linting in CI (for example, if you only use pre-commit hooks locally), you can remove the workflow file entirely. - -**Windows (PowerShell):** - -```powershell -Remove-Item -Path ".github/workflows/markdownlint.yml" -Force -``` - -**macOS/Linux/FreeBSD:** - -```bash -rm -f .github/workflows/markdownlint.yml -``` - -> **Note:** Removing the CI workflow does not affect local linting. You can still run `npm run lint:md` locally or use pre-commit hooks to lint Markdown files before committing. - ---- - -## Copilot Documentation Instructions Configuration - -**File:** `.github/instructions/docs.instructions.md` - -GitHub Copilot instruction files (`.github/instructions/*.md`) provide coding and documentation standards that Copilot applies when generating or editing code in your repository. The `docs.instructions.md` file specifically provides documentation standards that Copilot applies to all Markdown files, defining contract-first, traceable, and drift-resistant documentation practices. - -The file contains several customization points that should be updated to match your project's specific needs. - -### Customizing Documentation Taxonomy - -The default taxonomy suggests a folder structure for documentation: - -```markdown -- **Product spec:** `docs/spec/` (requirements + design; the source of truth) -- **Developer docs:** `docs/` (how to build, test, extend) -- **Operational docs / runbooks:** `docs/runbooks/` (diagnosis, remediation, safe operations) -- **Architecture Decision Records (ADRs):** `docs/adr/` (durable decisions) -``` - -**To customize for your project's structure:** - -Update the taxonomy section to reflect your actual documentation organization. For example, if your project uses a different structure: - -```markdown -- **API documentation:** `docs/api/` (API reference and usage guides) -- **User guides:** `docs/guides/` (end-user documentation) -- **Developer docs:** `docs/dev/` (how to build, test, extend) -- **Release notes:** `docs/releases/` (version history and changelogs) -``` - -> **Note:** If your project does not have a formal documentation structure, you can simplify this section to match your needs or remove categories that don't apply. - -### Customizing Canonical Source of Truth - -The file recommends defining a canonical specification document that serves as the authoritative reference for system behavior: - -```markdown -Projects SHOULD define a canonical specification document (e.g., `docs/spec/requirements.md`) -that serves as the authoritative reference for system behavior and requirements. -``` - -**To customize for your project:** - -- If you have a canonical spec, update the example path to match your actual location: - - ```markdown - Projects SHOULD define a canonical specification document (e.g., `docs/SPEC.md`) - that serves as the authoritative reference for system behavior and requirements. - ``` - -- If your project does not use a formal specification document, you can note this explicitly or remove the section entirely. - -### Customizing Requirements Documentation Standards - -The file provides a pattern for tracking formal requirements with identifiers: - -```markdown -- Each requirement SHOULD have a stable identifier (example pattern): - - `PROJ-REQ-001`, `PROJ-REQ-002`, ... -``` - -**To customize for your project:** - -1. **Update the requirement ID pattern** to match your project's naming convention: - - ```markdown - - Each requirement SHOULD have a stable identifier (example pattern): - - `MYPROJ-REQ-001`, `MYPROJ-REQ-002`, ... - ``` - -2. **Adjust the requirement entry format** if your project uses different metadata fields. The default includes Rationale, Acceptance Criteria, Priority, and Verification. You might customize this to: - - ```markdown - - Each requirement entry SHOULD include: - - **Rationale:** why it exists - - **Acceptance Criteria:** objective checks (bullets) - - **Owner:** responsible team or individual - - **Target Release:** version when this should be implemented - ``` - -3. **If your project does not track formal requirements**, you can simplify or remove this section entirely. Consider replacing it with guidance appropriate for your documentation needs. - ---- - -## Copilot Python Instructions Configuration - -**File:** `.github/instructions/python.instructions.md` - -The `python.instructions.md` file provides Python coding standards that GitHub Copilot applies when generating or editing `.py` files in your repository. These standards define style, structure, error handling, testing, and documentation requirements for Python code. The file supports two modes: **Baseline** (stdlib-first, portability-first) and **Modern-Advanced** (for projects using FastAPI, Pydantic, async frameworks, etc.). - -Teams may want to customize these standards to match their project's specific requirements and preferences. - -### Choosing Between Baseline and Modern-Advanced Mode - -The file defines two distinct coding modes: - -**Baseline Mode (Default):** - -- Minimal dependencies (stdlib-first) -- Type hints are optional/opportunistic -- Explicit control flow; avoids metaprogramming -- Uses plain datatypes (`dict`, `list`, `tuple`) for simple tasks -- Prioritizes portability and clarity - -**Modern-Advanced Mode:** - -- Type hints required pervasively -- Uses `pathlib.Path` over `os.path` -- Supports async/await patterns -- Uses structured logging -- Suited for FastAPI, Pydantic, and type-heavy APIs - -**To customize for your project:** - -1. **For modern-only projects (FastAPI, Pydantic, async stacks):** Update the "Executive Summary: Author Profile" section to indicate that Modern-Advanced mode is the default. You may also simplify or remove Baseline-specific guidance from sections like "Baseline vs Modern-Advanced Mode" and the Quick Reference Checklist items tagged `[Baseline]`. - -2. **For stdlib-first projects:** Keep Baseline mode as the default. Consider removing or de-emphasizing Modern-Advanced sections if your project never uses async frameworks or type-heavy APIs. - -3. **For mixed projects:** Keep both modes documented but add project-specific guidance on when each applies (e.g., "Use Baseline mode for CLI utilities, Modern-Advanced mode for API services"). - -> **Note:** The mode primarily affects type hint requirements and framework usage patterns. Core style rules (naming, formatting, error handling) apply to both modes. - -### Adjusting Line Length - -The Python instructions reference a line length target of **<= 100** characters: - -```markdown -- **[All]** **MUST** follow PEP 8/PEP 257; line length target **<= 100** -``` - -This setting should be consistent with your formatting tools in `.pre-commit-config.yaml`: - -```yaml -# .pre-commit-config.yaml -- repo: https://github.com/psf/black - hooks: - - id: black - args: [--line-length=100] - -- repo: https://github.com/astral-sh/ruff-pre-commit - hooks: - - id: ruff-check - args: [--fix, --line-length=100] -``` - -**To use Black's default of 88 characters:** - -1. Update `.pre-commit-config.yaml` to use `--line-length=88` for both Black and Ruff -2. Update the line length reference in `python.instructions.md`: - - ```markdown - - **[All]** **MUST** follow PEP 8/PEP 257; line length target **<= 88** - ``` - -> **Note:** Ensure Black, Ruff, and the instruction file all use the same line length to avoid conflicts between Copilot-generated code and formatting tools. - -### Customizing Type Hint Requirements - -The file has different type hint expectations based on mode: - -**Baseline Mode:** - -```markdown -- **[Baseline]** **MAY** use type hints opportunistically for public APIs and complex structures. -``` - -**Modern-Advanced Mode:** - -```markdown -- **[Modern]** Type hints are expected broadly; **MUST** run static checking (e.g., mypy/pyright) in CI. -``` - -**To customize for your project:** - -1. **To require type hints everywhere:** Update the Baseline guidance to match Modern-Advanced requirements. Change `MAY` to `MUST` and add static checking requirements: - - ```markdown - - **[All]** **MUST** use type hints for all function signatures and complex variables. - - **[All]** **MUST** run static checking (mypy/pyright) in CI. - ``` - -2. **To relax type hint requirements:** For projects where type hints are not a priority, update both mode sections to use `MAY` or `SHOULD`: - - ```markdown - - **[All]** **SHOULD** use type hints for public APIs. - - **[All]** **MAY** omit type hints for internal/private functions. - ``` - -3. **For gradual adoption:** Document a migration path based on project maturity: - - ```markdown - - New modules **MUST** include type hints for all public APIs. - - Legacy modules **SHOULD** add type hints when modified. - ``` - -> **Note:** If requiring strict type checking, ensure your CI workflow (`.github/workflows/python-ci.yml`) runs mypy without `continue-on-error: true`. See [CI Workflow Configuration](#ci-workflow-configuration) for details. - -### Adjusting Documentation Standards - -The file requires docstrings for all public modules, classes, and functions: - -```markdown -- **[All]** Every public module/class/function **MUST** have a docstring. -- **[All]** Docstrings **MUST** emphasize contract: inputs, outputs, errors, edge cases, examples. -``` - -The default docstring format includes: - -- Short summary line -- Longer description if needed -- Args, Returns, Raises sections -- Examples for tricky behavior - -**To customize for your project:** - -1. **To relax requirements for internal/private functions:** Add guidance that distinguishes between public and private documentation needs: - - ```markdown - - **[All]** Every public module/class/function **MUST** have a docstring. - - **[All]** Private functions (prefixed with `_`) **SHOULD** have a docstring but **MAY** use a brief one-line summary. - - **[All]** Internal helper functions **MAY** omit docstrings if their purpose is obvious from context. - ``` - -2. **To enforce a specific docstring style:** Add explicit style guidance such as Google style, NumPy style, or reStructuredText: - - ```markdown - - **[All]** Docstrings **MUST** use Google style format. - ``` - -3. **To require examples for all public functions:** Strengthen the example requirement: - - ```markdown - - **[All]** Public functions **MUST** include at least one example in the docstring. - ``` - -> **Note:** The instruction file uses a Google-style format (Args, Returns, Raises). If your project uses a different convention, update the example in the "Docstrings" section accordingly. - -### Customizing Testing Requirements - -The file specifies testing requirements for Python code: - -```markdown -- **[All]** Tests **MUST** exist for non-trivial logic; **SHOULD** use `pytest` unless repo standard differs. -``` - -**To customize for your project:** - -1. **To specify a different test framework:** If your project uses `unittest` or another framework, update the guidance: - - ```markdown - - **[All]** Tests **MUST** exist for non-trivial logic; **SHOULD** use `unittest`. - ``` - -2. **To add coverage requirements:** Specify minimum coverage thresholds: - - ```markdown - - **[All]** Tests **MUST** exist for non-trivial logic; **SHOULD** use `pytest`. - - **[All]** Test coverage **SHOULD** be >= 80% for new code. - ``` - -3. **To require specific test patterns:** Add guidance for test organization: - - ```markdown - - **[All]** Tests **SHOULD** be placed in `tests/` directory mirroring the `src/` structure. - - **[All]** Test files **MUST** use `test_*.py` naming convention. - - **[All]** **SHOULD** use table-driven tests for parsing/validation logic. - ``` - -4. **To relax testing requirements for prototypes:** Add context-dependent guidance: - - ```markdown - - **[All]** Production code **MUST** have tests for non-trivial logic. - - **[All]** Prototype/experimental code **SHOULD** have tests but **MAY** defer coverage. - ``` - -> **Note:** Testing configuration (pytest settings, coverage thresholds) is typically managed in `pyproject.toml`. See the "Testing" section in that file for related settings. - ---- - -## Copilot PowerShell Instructions Configuration - -**File:** `.github/instructions/powershell.instructions.md` - -The `powershell.instructions.md` file provides comprehensive PowerShell coding standards that GitHub Copilot applies when generating or editing `.ps1` files in your repository. These standards define naming conventions, documentation requirements, error handling patterns, and compatibility guidelines for both legacy (v1.0) and modern (v5.1+/v7.x+) PowerShell environments. - -Teams may want to customize these standards to match their project's specific requirements and preferences. - -### Customizing Variable Naming Conventions - -The file defaults to Hungarian-style type-prefixed variable naming for local variables, particularly in v1.0-targeted code: - -```powershell -$strMessage # String -$intCount # Integer -$boolResult # Boolean -$arrElements # Array -$objInstance # Object -``` - -Teams with modern codebases that have strong IDE support (IntelliSense, type inference) may prefer plain camelCase: - -```powershell -$message -$count -$result -$elements -$instance -``` - -**To change this preference, update the following sections:** - -1. **"Local Variable Naming: Type-Prefixed camelCase"** section - Modify the naming rules and examples -2. **"Options for Local Variable Prefixes: Analysis"** table - Update the recommendation based on your choice -3. **Quick Reference Checklist** - Update the item referencing variable naming conventions (the `[v1.0]` scoped item about local variables) - -> **Note:** If your project exclusively targets modern PowerShell (5.1+, 7.x), plain camelCase is generally preferred as IDEs provide type information. Type prefixes are most valuable in v1.0 environments or when editing without IDE support. - -### Choosing Between v1.0 and Modern Patterns - -The file distinguishes between two function architecture styles: - -**v1.0-targeted:** - -- Uses `trap` for error handling -- No `[CmdletBinding()]` attribute -- Explicit integer return codes (0=success, -1=failure) -- Reference parameters (`[ref]`) for outputs -- No pipeline input support - -**Modern (v2.0+):** - -- Uses `try/catch` for error handling -- Requires `[CmdletBinding()]` attribute -- Streaming output to pipeline -- `Write-Verbose` and `Write-Debug` for diagnostics -- Full pipeline support - -**To customize for your environment:** - -- **Modern-only projects (PowerShell 5.1+, 7.x):** Remove or de-emphasize the v1.0 sections. Update the Quick Reference Checklist to remove items tagged `[v1.0]` and make `[Modern]` items the default. - -- **Legacy compatibility projects:** Keep the v1.0 sections as primary guidance. Update the "Executive Summary: Author Profile" to emphasize v1.0 compatibility as the default. - -- **Mixed environments:** Keep both patterns but clarify when each applies based on your specific criteria (e.g., "Use v1.0 patterns for standalone utilities, Modern patterns for module functions"). - -### Adjusting Documentation Requirements - -The file requires comprehensive comment-based help for all functions, including: - -- `.SYNOPSIS`, `.DESCRIPTION`, `.PARAMETER`, `.EXAMPLE`, `.INPUTS`, `.OUTPUTS`, `.NOTES` -- Version number in `.NOTES` (format: `Major.Minor.YYYYMMDD.Revision`) -- Multiple examples with input, output, and explanation - -**Teams may want to customize these requirements:** - -1. **For internal/private helper functions:** Relax requirements in the "Comment-Based Help: Structure and Format" section to allow minimal documentation (e.g., `.SYNOPSIS` only) for private helper functions. - -2. **For versioning format:** Update the "Function and Script Versioning" section if your project uses a different versioning scheme (e.g., SemVer without date component): - - ```powershell - # Alternative format - # .NOTES - # Version: 1.2.3 - ``` - -3. **For example requirements:** Reduce the requirement for multiple examples in the "Help Content Quality: High Standards" section if this is too burdensome for your team. - -> **Note:** Even with relaxed requirements, maintaining at least `.SYNOPSIS` for all functions is strongly recommended for discoverability with `Get-Help`. - -### Customizing Brace Style Preference - -The file enforces OTBS (One True Brace Style) where opening braces are placed on the same line as the statement: - -```powershell -# OTBS (default) -if ($condition) { - # code -} else { - # code -} -``` - -Some teams prefer Allman style (braces on new lines): - -```powershell -# Allman style -if ($condition) -{ - # code -} -else -{ - # code -} -``` - -**To change brace style:** - -1. Update the "Brace Placement (OTBS)" section in this file to reflect your preferred style -2. Update `.github/linting/PSScriptAnalyzerSettings.psd1` to match (see [PSScriptAnalyzer Configuration](#psscriptanalyzer-configuration) for details) - -> **Note:** Brace style must be consistent between the instruction file and PSScriptAnalyzer settings. Inconsistent settings will cause conflicts between Copilot-generated code and linting rules. - -### Adjusting Error Handling Patterns - -The file documents specific return code conventions for v1.0-targeted functions: - -| Code | Meaning | -| --- | --- | -| `0` | Full success | -| `1-5` | Partial success with additional data | -| `-1` | Complete failure | - -**To customize for your project:** - -1. **Different return codes:** Update the "Return Semantics: Explicit Status Codes" section with your project's conventions. For example, some projects use positive integers for all error codes: - - ```powershell - # Alternative convention - # 0 = Success - # 1 = General error - # 2 = File not found - # 3 = Permission denied - ``` - -2. **Exception-based patterns:** For modern-only projects, you may prefer to rely entirely on exceptions rather than return codes. Update the "Modern catch Block Requirements" section to document your exception handling patterns. - -3. **Custom exception types:** If your project defines custom exception types, document them and update the error handling sections accordingly. - ---- - -## Copilot Terraform Instructions Configuration - -**File:** `.github/instructions/terraform.instructions.md` - -The `terraform.instructions.md` file provides Terraform coding standards that GitHub Copilot applies when generating or editing `.tf`, `.tfvars`, `.tftest.hcl`, `.tf.json`, `.tftpl`, and `.tfbackend` files in your repository. These standards cover style, formatting, naming conventions, file organization, variable and output design, resource configuration, module design, state management, security best practices, provider management, testing, and documentation requirements. - -### Adopting for Terraform Projects - -When adopting this template for a Terraform project, complete the following tasks: - -1. **Update the metadata** at the top of the file to reflect your project's ownership and versioning -2. **Review and adjust backend configuration** guidance for your project needs (S3, Azure Storage, GCS, Terraform Cloud, etc.) -3. **Verify required provider versions** and update the version constraint examples as needed -4. **Add organization-specific required tags** to the Required Tags section if your organization mandates specific tags -5. **Document any justified deviations** in the "Scope Exceptions & Deviations from Standards" section at the end of the file -6. **Remove non-relevant provider examples** — The file includes parallel examples for AWS, Azure, and GCP. Delete examples for providers your project does not use to reduce noise and confusion. Search for "AWS Example", "Azure Example", "GCP Example", and combined labels like "AWS/Azure Example" to identify provider-specific blocks. -7. **Replace all `REPLACE_ME_*` placeholders** with your organization's actual values. Run `grep -r "REPLACE_ME"` to find all placeholders requiring customization. - -### Customizing Provider Examples - -The file now includes parallel examples for AWS, Azure, and GCP throughout. Each example group is clearly labeled with "AWS Example", "Azure Example", "GCP Example", or combined labels like "AWS/Azure Example" (used when providers share the same pattern). To customize for your cloud provider: - -1. **For single-provider projects:** Remove examples for providers you don't use. Search for the provider labels (e.g., "Azure Example", "GCP Example") and delete those code blocks and their headers. - -2. **For multi-cloud projects:** Keep all examples or remove only those that don't apply to your specific environments. - -3. **For all projects:** Replace `REPLACE_ME_*` placeholders with your organization's actual values. See the "Placeholder Convention (`REPLACE_ME_*`)" section in `.github/instructions/terraform.instructions.md` for the complete list of standard placeholders. - -### Customizing for Terraform Cloud/Enterprise - -If your organization uses Terraform Cloud, Terraform Enterprise, Spacelift, or similar tools: - -1. **Update the State Management section** to reflect your workflow. The `backend.tf` file may not be applicable. - -2. **Add cloud block configuration** guidance as the primary example: - - ```hcl - terraform { - cloud { - organization = "REPLACE_ME_ORG" - workspaces { - name = "REPLACE_ME_WORKSPACE" - } - } - } - ``` - -3. **Document which sections do not apply** in the Scope Exceptions section. For example: - - Manual `backend.tf` configuration - - DynamoDB lock table configuration - - S3/GCS/Azure Storage bucket configuration for state - -### Customizing Required Tags - -The Required Tags section lists mandatory tags for all taggable resources. To customize for your organization: - -1. **Add organization-specific tags:** - - ```markdown - | Tag | Description | Example | - | --- | --- | --- | - | `CostCenter` | Budget/billing code | `CC-12345` | - | `Department` | Owning department | `Engineering` | - | `Compliance` | Compliance framework | `SOC2`, `HIPAA` | - ``` - -2. **Remove tags that don't apply** to your organization - -3. **Update the Default Tags Configuration** example to reflect your required tags - -### Scope Exceptions Section - -The file includes a "Scope Exceptions & Deviations from Standards" section at the end for documenting justified deviations. Use this section to record: - -- **Alternative backend workflows:** Using Terraform Cloud instead of `backend.tf` -- **Provider-specific requirements:** Organization policies that mandate specific provider configurations -- **Legacy compatibility:** Maintaining compatibility with older Terraform versions or modules -- **Organizational naming conventions:** Pre-existing naming conventions that differ from the template -- **Security policy overrides:** Stricter security requirements that go beyond the template defaults - ---- - -## Copilot Main Instructions Configuration - -**File:** `.github/copilot-instructions.md` - -The main `.github/copilot-instructions.md` file provides repository-wide instructions that GitHub Copilot applies when generating or editing code. It includes sections on pre-commit discipline, testing tools, and other project-wide standards. These sections should be customized to match your project's actual tools and workflows. - -### Customizing the Pre-commit Discipline Section - -The Pre-commit Discipline section (near the top of `.github/copilot-instructions.md`) tells Copilot how to run pre-commit checks, what commands to use, and how to handle CI failures. This ensures Copilot generates code that follows your project's code quality workflow. - -The default configuration assumes: - -- The [pre-commit](https://pre-commit.com/) framework with `pre-commit run --all-files` -- npm-based markdown linting with `npm run lint:md` -- A `copilot/**` branch pattern for automated fixes - -**To customize for your project:** - -1. **Different pre-commit tools:** If you use a different tool (e.g., Husky, lefthook, or custom scripts), update the workflow section: - - ```markdown - **Workflow:** - - 1. Make your code changes - 2. Run pre-commit checks locally (e.g., `npx husky run` or `make lint`) - 3. Review and commit ALL auto-fixes as part of your change - 4. Push to GitHub - ``` - -2. **Different linting commands:** Update command examples to match your project: - - ```markdown - **Workflow:** - - 1. Make your code changes - 2. Run pre-commit checks locally (e.g., `make lint` or `./scripts/lint.sh`) - 3. Review and commit ALL auto-fixes as part of your change - 4. Push to GitHub - ``` - -3. **No pre-commit framework:** If your project uses only CI-based checks without local pre-commit hooks, simplify the section: - - ```markdown - ## Pre-commit Discipline (CRITICAL) - - **⚠️ ALWAYS run linting checks before committing code.** - - **Workflow:** - - 1. Make your code changes - 2. Run linting locally: `npm run lint` (or your project's lint command) - 3. Review and fix all issues before committing - 4. Push to GitHub - - **If CI fails:** - - 1. Pull the latest branch - 2. Run linting locally and fix issues - 3. Commit fixes - 4. Push again - ``` - -4. **Different branch patterns for automated fixes:** If you use a different branch naming convention for AI-generated PRs, update the Auto-Fix Workflow section to match (and update the corresponding workflow file). - -> **Note:** The pre-commit section should accurately reflect your project's tooling. Incorrect instructions will cause Copilot to suggest wrong commands or skip necessary checks. - -### Customizing the Testing Tools Section - -The Testing Tools section (near the bottom of `.github/copilot-instructions.md`) tells Copilot what test frameworks your project uses, where tests are located, and how to run them. This ensures Copilot generates tests that match your project's conventions. - -The default configuration includes: - -| Language | Framework | Configuration | Test Location | -| --- | --- | --- | --- | -| Python | pytest | `pyproject.toml` (`[tool.pytest.ini_options]`) | `tests/` | -| PowerShell | Pester 5.x | Inline in `.github/workflows/powershell-ci.yml` | `tests/PowerShell/` | - -**To customize for your project:** - -1. **Different test frameworks:** Update the table to reflect your actual frameworks: - - ```markdown - ## Testing Tools - - This repository includes testing infrastructure for the following languages: - - | Language | Framework | Configuration | Test Location | - | --- | --- | --- | --- | - | Python | unittest | `setup.cfg` | `tests/` | - | JavaScript | Jest | `jest.config.js` | `__tests__/` | - | TypeScript | Vitest | `vitest.config.ts` | `src/**/*.test.ts` | - ``` - -2. **Different test locations:** Update the table and running instructions to match your directory structure: - - ```markdown - | Language | Framework | Configuration | Test Location | - | --- | --- | --- | --- | - | Python | pytest | `pytest.ini` | `spec/` | - | Ruby | RSpec | `.rspec` | `spec/` | - ``` - -3. **Update the "Running Tests" section:** Ensure the commands match your setup (this example shows commands for Jest and unittest, matching the frameworks in example 1): - - ````markdown - ### Running Tests - - **JavaScript:** - - ```bash - npm test - ``` - - **Python:** - - ```bash - python -m unittest discover -s tests - ``` - ```` - -4. **Single-language projects:** Remove rows for languages you don't use: - - ````markdown - ## Testing Tools - - This repository uses pytest for testing: - - | Language | Framework | Configuration | Test Location | - | --- | --- | --- | --- | - | Python | pytest | `pyproject.toml` | `tests/` | - - ### Running Tests - - ```bash - pytest tests/ -v - ``` - ```` - -5. **Additional test types:** If your project includes integration tests, end-to-end tests, or other test types, document them: - - ```markdown - ## Testing Tools - - | Test Type | Framework | Configuration | Location | - | --- | --- | --- | --- | - | Unit tests | pytest | `pyproject.toml` | `tests/unit/` | - | Integration tests | pytest | `pyproject.toml` | `tests/integration/` | - | E2E tests | Playwright | `playwright.config.ts` | `tests/e2e/` | - ``` - -> **Note:** Keep the Testing Tools section synchronized with your actual test configuration. Incorrect information will cause Copilot to generate tests in wrong locations or using wrong frameworks. - -### Updating Related Sections - -When customizing the Pre-commit or Testing sections, you may also need to update these related sections in `.github/copilot-instructions.md`: - -- **How to Work (Definition of Done):** Update test location references (e.g., `tests/` to `spec/`) -- **Modular Instructions table:** Ensure it matches your instruction files -- **Linting Configurations:** Update linting tool references if you use different linters - ---- - -## Customizing Agent Instruction Files - -**Files:** `CLAUDE.md`, `AGENTS.md`, `GEMINI.md` - -The template includes three agent instruction files at the repository root. `.github/copilot-instructions.md` remains the canonical source of truth, while these files act as thin entry points that keep a minimal inline summary of the highest-priority shared rules plus any platform-specific guidance. Each file targets a specific AI coding platform: - -| File | Target Agent(s) | -| --- | --- | -| `CLAUDE.md` | Claude Code, GitHub Copilot coding agent | -| `AGENTS.md` | OpenAI Codex CLI, GitHub Copilot coding agent | -| `GEMINI.md` | Gemini Code Assist, GitHub Copilot coding agent | - -### Removing Unneeded Files - -Delete agent files for platforms you do not use. For example, if your team does not use Claude Code, delete `CLAUDE.md`. If your team does not use OpenAI Codex CLI, delete `AGENTS.md`. Removing unused files reduces maintenance burden without affecting other platforms. - -### Keeping Minimal Summaries Aligned - -When high-priority shared guidance changes in `.github/copilot-instructions.md`, update the minimal summaries in any remaining agent files as needed. Common changes that require alignment include: - -- **Language table updates** — Adding or removing languages -- **Linting tool changes** — Different linting commands or configurations -- **Test command changes** — Different test frameworks or commands -- **Build and test commands** — Updated pre-commit or CI commands -- **Canonical-file guidance** — Changes to how agents should locate or interpret the canonical instructions - -> **Note:** No CI enforcement exists for agent-file alignment. Review the remaining agent files whenever you update `.github/copilot-instructions.md` so their minimal summaries stay accurate without regrowing into full duplicates. - -### Adding Platform-Specific Notes - -If an agent has unique behavioral requirements (e.g., different command syntax, specific tool limitations, or platform-specific workarounds), add agent-specific sections to the appropriate file. Platform-specific notes should supplement, not contradict, the canonical rules in `.github/copilot-instructions.md`. - ---- - -## CI Workflow Configuration - -**File:** `.github/workflows/python-ci.yml` - -### Enabling Codecov Integration - -Uncomment the Codecov step to enable code coverage reporting: - -```yaml -- name: Upload coverage to Codecov - uses: codecov/codecov-action@v4 - with: - token: ${{ secrets.CODECOV_TOKEN }} # Required for private repos only - files: ./coverage.xml - flags: unittests - name: codecov-umbrella - fail_ci_if_error: false -``` - -> **Note:** Public repositories do not require a token. For private repositories, add `CODECOV_TOKEN` to your repository secrets. - -### Making Type Checking Strict - -The mypy step includes `continue-on-error: true` by default: - -```yaml -- name: Run mypy - run: mypy $MYPY_PATHS - continue-on-error: true -``` - -**Once your project has full type coverage**, remove `continue-on-error: true` to make type checking a required check. - -### Customizing Python Version Matrix - -The test matrix runs on Python 3.13 by default: - -```yaml -strategy: - matrix: - python-version: ['3.13'] -``` - -**To test multiple versions:** - -```yaml -strategy: - matrix: - python-version: ['3.11', '3.12', '3.13'] -``` - -> **Note:** Only test Python versions currently receiving bugfix updates. Security-fix-only versions require building from source and are not recommended for CI. - -### Customizing Test Paths - -The `MYPY_PATHS` environment variable controls which directories mypy checks: - -```yaml -env: - MYPY_PATHS: "src/ tests/" -``` - -**Common configurations:** - -- Flat layout: `MYPY_PATHS: "."` -- src layout: `MYPY_PATHS: "src/ tests/"` -- Custom: `MYPY_PATHS: "mymodule/ tests/ scripts/"` - -### Customizing Dependency Installation - -The CI workflow installs the project with development dependencies using: - -```yaml -pip install -e ".[dev]" -``` - -This command appears in both the `type-check` and `test` jobs. - -**To use different dependency groups:** - -If your `pyproject.toml` uses a different optional dependency group name: - -```yaml -# For a [project.optional-dependencies] section named "test" -pip install -e ".[test]" - -# For multiple groups -pip install -e ".[dev,test]" -``` - -**To use requirements.txt instead:** - -If your project uses `requirements.txt` files instead of `pyproject.toml` optional dependencies: - -```yaml -- name: Install dependencies - run: | - python -m pip install --upgrade pip - pip install -r requirements.txt - pip install -r requirements-dev.txt # If you have separate dev requirements -``` - -> **Note:** Update both the `type-check` job (line ~131) and the `test` job (line ~183) to keep them consistent. - ---- - -## Auto-fix Pre-commit Workflow Configuration - -**File:** `.github/workflows/auto-fix-precommit.yml` - -The template includes an optional workflow that automatically runs pre-commit hooks and commits any auto-fixes (such as formatting corrections and trailing whitespace removal) for branches created by the GitHub Copilot Coding Agent. - -### Understanding the Workflow - -This workflow: - -- Triggers only on `copilot/**` branches when pushed by `copilot-swe-agent[bot]` -- Runs pre-commit hooks with auto-fix enabled -- Commits any changes back to the branch automatically -- Helps AI-assisted development pass pre-commit checks without manual intervention - -> **Recommendation:** Keep this workflow enabled if you use GitHub Copilot Coding Agent. The safety net significantly reduces the need for manual pre-commit fix commits. - -### When to Keep This Workflow - -Keep this workflow if: - -- You plan to use GitHub Copilot Coding Agent for automated PRs -- You want a safety net that auto-fixes pre-commit issues on `copilot/**` branches -- You prefer automated fixes over manual intervention - -### Removing This Workflow - -If you don't use GitHub Copilot Coding Agent or prefer to manually commit pre-commit fixes, you can safely remove this workflow. - -> **Note:** Removing this workflow is safe—the standard `python-ci.yml` workflow will still run pre-commit checks and report any issues that need to be fixed. - -**Steps to remove:** - -**Windows (PowerShell):** - -```powershell -Remove-Item -Path ".github/workflows/auto-fix-precommit.yml" -Force -``` - -**macOS/Linux/FreeBSD:** - -```bash -rm -f .github/workflows/auto-fix-precommit.yml -``` - ---- - -## Placeholder Check Workflow Configuration - -**File:** `.github/workflows/check-placeholders.yml` - -The placeholder check workflow verifies that template placeholders (`OWNER/REPO`, `@OWNER`, `[security contact email]`) have been replaced. It runs automatically in all repositories created from the template. - -### Understanding the Workflow - -The workflow uses automatic detection to determine whether to run: - -```yaml -if: github.repository != 'franklesniak/copilot-repo-template' -``` - -This means the workflow: - -- **Runs automatically** in your repository (no configuration needed) -- **Is disabled** only in the original template repository - -### When to Keep This Workflow - -Keep this workflow if you: - -- Plan to make future updates from the template that might introduce new placeholder files -- Want a safety net to catch accidental placeholder remnants -- Have contributors who might add files with placeholder patterns - -### Removing This Workflow - -If you have replaced all placeholders and don't anticipate needing this check: - -**Windows (PowerShell):** - -```powershell -Remove-Item -Force ".github\workflows\check-placeholders.yml" -``` - -**macOS/Linux/FreeBSD:** - -```bash -rm -f .github/workflows/check-placeholders.yml -``` - -### Adding Custom Placeholder Patterns - -If your project uses additional placeholder patterns that should be checked, edit `.github/workflows/check-placeholders.yml` and locate the "Additional placeholder patterns check" step. Find the `PATTERNS` array and add your custom patterns: - -```yaml -- name: Additional placeholder patterns check - run: | - # ... existing code ... - PATTERNS=( - "your-org" - "your-repo" - "YOUR_ORG" - "YOUR_REPO" - "YOUR_CUSTOM_PLACEHOLDER" # Add your custom patterns here - ) -``` - -### Converting Warnings to Hard Failures - -By default, some checks in the "Additional placeholder patterns check" step produce warnings rather than failures (e.g., `Project Name` in README.md, patterns in PR templates). To make these hard failures, edit `.github/workflows/check-placeholders.yml` and change the warning-only sections to set `FOUND_PLACEHOLDERS=true`: - -```yaml -# Before (warning only): -if grep -n "^# Project Name$" README.md; then - echo "::warning file=README.md::Found 'Project Name' placeholder..." - FOUND_WARNINGS=true -fi - -# After (hard failure): -if grep -n "^# Project Name$" README.md; then - echo "::error file=README.md::Found 'Project Name' placeholder..." - FOUND_PLACEHOLDERS=true # Changed from FOUND_WARNINGS -fi -``` - ---- - -## PowerShell CI Workflow Configuration - -**File:** `.github/workflows/powershell-ci.yml` - -The PowerShell CI workflow runs PSScriptAnalyzer linting and Pester tests for PowerShell scripts. It runs automatically on every push and pull request and automatically skips if no PowerShell files are found in the repository. - -### Understanding the Workflow - -The workflow consists of two jobs: - -1. **powershell-lint** (display name: "Lint (PSScriptAnalyzer)"): Runs PSScriptAnalyzer on all `.ps1` files (skips if no files found) -2. **test** (display name: "PowerShell Tests (Pester)"): Runs Pester tests on Windows, macOS, and Linux (skips if no `*.Tests.ps1` files found) - -The workflow uses automatic detection, so you don't need to configure anything if you have PowerShell files—it just works. - -### Customizing Pester Test Paths - -By default, Pester tests are run from the `tests/` directory. To use a different directory, modify the `$config.Run.Path` setting in the "Run Pester tests" step: - -```powershell -$config.Run.Path = "your_tests_directory/" # Change from default "tests/" -``` - -### Customizing Test Output Format - -The default test output format is `NUnitXml`. To use a different format: - -```powershell -$config.TestResult.OutputFormat = "JUnitXml" # Default is "NUnitXml" -``` - -Available formats include `NUnitXml`, `JUnitXml`, and `NUnit2.5`. - -### Customizing the OS Matrix - -By default, Pester tests run on Ubuntu, Windows, and macOS: - -```yaml -strategy: - matrix: - os: [ubuntu-latest, windows-latest, macos-latest] -``` - -**To run on fewer operating systems:** - -```yaml -strategy: - matrix: - os: [windows-latest] # Windows only -``` - -**To run on Windows and Linux only:** - -```yaml -strategy: - matrix: - os: [ubuntu-latest, windows-latest] -``` - -### Adding Path-Based Filtering - -The template deliberately does not use path-based filtering because this is a template repository where all workflows should run on all changes for testing purposes. However, consumers of the template can add path filtering for efficiency: - -```yaml -on: - push: - branches: ["**"] - paths: - - "**/*.ps1" - - ".github/workflows/powershell-ci.yml" - - ".github/linting/PSScriptAnalyzerSettings.psd1" - pull_request: - branches: ["**"] - paths: - - "**/*.ps1" - - ".github/workflows/powershell-ci.yml" - - ".github/linting/PSScriptAnalyzerSettings.psd1" -``` - -> **Note:** Include configuration files in the path filter to ensure the workflow runs when linting rules or the workflow itself changes. - -### Removing the Workflow - -If your project doesn't use PowerShell, you can remove the workflow: - -**Windows (PowerShell):** - -```powershell -Remove-Item -Path ".github/workflows/powershell-ci.yml" -Force -``` - -**macOS/Linux/FreeBSD:** - -```bash -rm -f .github/workflows/powershell-ci.yml -``` - -> **Note:** If you want to remove all PowerShell-related files from the repository (not just the workflow), see the "If NOT Using PowerShell" section in [GETTING_STARTED_NEW_REPO.md](GETTING_STARTED_NEW_REPO.md) for comprehensive removal instructions. - ---- - -## Using the Python Template Files - -**Directory:** `templates/python/` - -This template repository includes reference Python configuration files and scaffolding for projects adopting Python tooling. These files demonstrate how to configure Python tooling to align with the coding standards defined in [`.github/instructions/python.instructions.md`](.github/instructions/python.instructions.md). - -### Files Included - -- **`pyproject.toml`**: Sample configuration for Python project metadata, dependencies, and tooling (Black, Ruff, mypy, pytest) -- **`tests/__init__.py`**: Package marker for the test directory -- **`tests/test_placeholder.py`**: Placeholder test file that demonstrates pytest test structure -- **`tests/test_schema_examples.py`**: Starter pytest module that auto-discovers and validates schema example fixtures under `schemas/examples//{valid,invalid}/` with `check-jsonschema`. Skips cleanly when the tool is not installed or when no examples are present. Mirrors the active, canonical test at `tests/test_schema_examples.py` in the template repository root. See [Schema Validation Configuration](#schema-validation-configuration) for setup. -- **`README.md`**: Brief overview of the template files with links to external resources - -### How to Use the Template - -1. **Copy files to your project root** (or appropriate location based on your layout): - - **Windows (PowerShell):** - - ```powershell - Copy-Item -Path "templates/python/pyproject.toml" -Destination "pyproject.toml" - Copy-Item -Path "templates/python/tests" -Destination "tests" -Recurse - ``` - - **macOS/Linux/FreeBSD:** - - ```bash - cp templates/python/pyproject.toml pyproject.toml - cp -r templates/python/tests tests - ``` - -2. **Customize `pyproject.toml`**: - - Update the `[project]` section with your project's name, version, description, and authors - - Add your runtime dependencies to the `dependencies = []` list - - Adjust development dependencies as needed - - Update the `classifiers` list to reflect your project's maturity and supported Python versions (see below) - -3. **Create your source code** in either a flat layout (modules in project root) or `src/` layout (modules in `src/your_package/`). See the [Project Layout Options](#project-layout-options) section below for detailed layout options and directory structure examples. - -4. **Replace or delete `tests/test_placeholder.py`** once you have actual tests in place. - -### About the Placeholder File - -The file `templates/python/tests/test_placeholder.py` is a minimal placeholder that demonstrates pytest test structure. It contains a single test that always passes: - -```python -"""Placeholder test file for template demonstration. - -This is a template file. Delete or overwrite this with your actual tests. -""" - - -def test_placeholder(): - """Simple placeholder test that always passes. - - Replace this with your actual test cases. - """ - assert True -``` - -When you add real tests for your project: - -1. Create test files following the `test_*.py` naming convention -2. Delete `test_placeholder.py` once you have real tests in place -3. See the [Python Version Configuration](#python-version-configuration) and [mypy Path Configuration](#mypy-path-configuration) sections below for additional configuration details - -### Customizing Classifiers - -The `classifiers` field in `pyproject.toml` provides metadata for PyPI and other tools. Update these values to match your project: - -**Development Status:** Change based on project maturity: - -```toml -# Alpha - early development, unstable API -"Development Status :: 3 - Alpha" - -# Beta - feature complete, may have bugs -"Development Status :: 4 - Beta" - -# Production/Stable - ready for production use -"Development Status :: 5 - Production/Stable" -``` - -**Python Version Classifiers:** Update when your minimum Python version changes: - -```toml -classifiers = [ - "Development Status :: 4 - Beta", - "Programming Language :: Python :: 3", - "Programming Language :: Python :: 3.12", - "Programming Language :: Python :: 3.13", -] -``` - -> **Note:** The Python version classifiers should match the versions specified in `requires-python` and your CI test matrix. See [PyPI Classifiers](https://pypi.org/classifiers/) for the full list of available classifiers. - -### When to Use These Templates - -Use the Python template files when: - -- **Starting a new Python project from scratch**: These templates provide clean configuration files that you can customize for your project -- **Adding Python to an existing repository**: If your repository doesn't have Python tooling configured, these templates provide a complete starting point - -### Project Layout Options - -#### Option 1: Flat Layout - -Place your Python modules directly in the project root: - -```text -your-project/ -├── pyproject.toml -├── your_module.py -├── another_module.py -└── tests/ - └── test_your_module.py -``` - -For mypy in CI, use: - -```yaml -env: - MYPY_PATHS: "." -``` - -#### Option 2: src/ Layout (Recommended) - -Place your Python package(s) in a `src/` directory: - -```text -your-project/ -├── pyproject.toml -├── src/ -│ └── your_package/ -│ ├── __init__.py -│ └── module.py -└── tests/ - └── test_module.py -``` - -For mypy in CI, use: - -```yaml -env: - MYPY_PATHS: "src/ tests/" -``` - -The `src/` layout is recommended because it: - -- Prevents accidental imports of uninstalled code during development -- Makes it clear what code is part of the package vs. project tooling -- Aligns with modern Python packaging best practices - -### Python Version Configuration - -Different Python tools require different version format specifications. When updating the minimum Python version, you must update **all three** of these settings: - -#### 1. Project Metadata: `requires-python` - -```toml -[project] -requires-python = ">=3.13" # PEP 621 standard: ">=" operator with dotted version -``` - -#### 2. Black Configuration: `target-version` - -```toml -[tool.black] -target-version = ["py313"] # List of strings in "pyXYZ" format -``` - -#### 3. mypy Configuration: `python_version` - -```toml -[tool.mypy] -python_version = "3.13" # Dotted version string (no ">=" operator) -``` - -#### 4. Ruff Configuration: `target-version` (Optional) - -```toml -[tool.ruff] -# Ruff automatically infers target-version from [project].requires-python -# Only set this if you need to override: -# target-version = "py313" # Single string in "pyXYZ" format (not a list) -``` - -**Important:** If you set `[project].requires-python`, Ruff will automatically use that value. Setting `[tool.ruff].target-version` explicitly will override the inferred value. - -### Python Version Support Policy - -**Always use a Python version that is currently receiving bugfixes.** - -- Python versions in "security fix only" phase are **not publicly installable** with security updates—they require building from source with manually applied patches. -- Check the [Python Developer's Guide - Versions](https://devguide.python.org/versions/) page for current version status. - -> **Template adopters:** The template defaults to Python 3.13+. Customize the `requires-python` field in `pyproject.toml` based on your project's specific requirements. - -**When to update:** - -- Check the [Python Developer's Guide - Versions](https://devguide.python.org/versions/) page annually (typically around October when new Python versions are released) -- Update all version references in `pyproject.toml` when the minimum supported version changes -- Update the CI workflow's Python version matrix in `.github/workflows/python-ci.yml` - -### mypy Path Configuration - -The CI workflow (`.github/workflows/python-ci.yml`) uses the `MYPY_PATHS` environment variable to specify which directories/files mypy should check. - -**Default (for src/ layout):** - -```yaml -env: - MYPY_PATHS: "src/ tests/" -``` - -**For flat layout:** - -```yaml -env: - MYPY_PATHS: "." -``` - -**For custom directories:** - -```yaml -env: - MYPY_PATHS: "foo/ bar/ baz.py" -``` - -The command-line paths override any `files` or `exclude` settings in `pyproject.toml` or `mypy.ini` in terms of directory scope. However, per-file configuration options in those files still apply to the files that mypy discovers. - ---- - -## Using the Pester Test Template - -**File:** `templates/powershell/Example.Tests.ps1` - -This template repository includes a comprehensive Pester 5.x test template that demonstrates common testing patterns. Use this template as a starting point when creating tests for your PowerShell functions. - -### What the Template Demonstrates - -The template file includes working examples of: - -- **BeforeAll/BeforeEach** for test setup and dot-sourcing functions -- **Describe/Context/It** block structure for organizing tests -- **Arrange-Act-Assert (AAA)** pattern for clear test organization -- **Testing integer return codes** (0=success, -1=failure) for v1.0-style functions -- **Testing reference parameters** (`[ref]`) for functions that return data via references -- **Testing boolean returns** for `Test-*` functions -- **Basic mocking** with the `Mock` command to isolate tests from external dependencies - -### How to Use the Template - -1. **Copy the template file** to your tests directory: - - **Windows (PowerShell):** - - ```powershell - Copy-Item "templates/powershell/Example.Tests.ps1" "tests/PowerShell/MyFunction.Tests.ps1" - ``` - - **macOS/Linux/FreeBSD:** - - ```bash - cp templates/powershell/Example.Tests.ps1 tests/PowerShell/MyFunction.Tests.ps1 - ``` - -2. **Remove the inline example functions** from the `BeforeAll` block. These are demonstration functions (`Get-ExampleGreeting`, `Test-IsValidEmail`, `Get-ProcessedData`) that exist only to make the template runnable as-is. - -3. **Uncomment and update the dot-source line** to import your actual function file: - - ```powershell - BeforeAll { - # Update this path to your actual function file - . $PSScriptRoot/../../src/MyFunction.ps1 - } - ``` - -4. **Replace the example `Describe` blocks** with tests for your actual functions, following the patterns demonstrated for your specific return types. - -### Running the Template Tests - -You can run the template file directly to see the test patterns in action: - -```powershell -Invoke-Pester -Path templates/powershell/Example.Tests.ps1 -Output Detailed -``` - -### About the Placeholder File - -The file `tests/PowerShell/Placeholder.Tests.ps1` is a minimal placeholder that exists to ensure the PowerShell CI workflow passes with a valid test file. When you add real tests for your project: - -1. Copy the template file as described above -2. Customize it for your functions -3. Delete `Placeholder.Tests.ps1` once you have real tests in place - -The template file provides much more comprehensive examples than the placeholder and should be your primary reference when writing Pester tests. - ---- - -## PSScriptAnalyzer Configuration - -**File:** `.github/linting/PSScriptAnalyzerSettings.psd1` - -### Adjusting Severity Levels - -The default configuration enforces Error and Warning severity: - -```powershell -Severity = @('Error', 'Warning') -``` - -**To include informational messages:** - -```powershell -Severity = @('Error', 'Warning', 'Information') -``` - -**To only enforce errors:** - -```powershell -Severity = @('Error') -``` - -### Customizing Individual Rules - -Each rule can be individually enabled or disabled: - -```powershell -Rules = @{ - PSAvoidUsingCmdletAliases = @{ - Enable = $true # or $false to disable - } -} -``` - -For a complete list of available rules, see the [PSScriptAnalyzer documentation](https://github.com/PowerShell/PSScriptAnalyzer). - -### Relaxing Formatting Rules - -For teams preferring different brace styles, modify the `PSPlaceOpenBrace` rule: - -**Allman style (braces on new line):** - -```powershell -PSPlaceOpenBrace = @{ - Enable = $true - OnSameLine = $false - NewLineAfter = $true -} -``` - -**To disable brace checking entirely:** - -```powershell -PSPlaceOpenBrace = @{ - Enable = $false -} -PSPlaceCloseBrace = @{ - Enable = $false -} -``` - ---- - -## CODEOWNERS Configuration - -**File:** `.github/CODEOWNERS` - -### Adding Team-Based Ownership - -Use team references for organization repositories: - -```text -* @org/maintainers -.github/workflows/ @org/devops-team -docs/ @org/documentation-team -``` - -### Adding Path-Specific Ownership - -Assign different owners for different directories: - -```text -# Default owners -* @username - -# Documentation -docs/ @docs-team -*.md @docs-team - -# Tests -tests/ @qa-team - -# Specific modules -src/api/ @api-team -src/frontend/ @frontend-team -``` - -### Multiple Owners - -List multiple owners for the same pattern: - -```text -# Both users will be requested for review -src/critical/ @senior-dev @tech-lead - -# Team and individual -.github/ @org/maintainers @repo-admin -``` - ---- - -## Node.js Package Configuration - -**File:** `package.json` - -### Adding Application Metadata - -For projects using Node.js as a runtime (not just dev tooling), add these fields: - -```json -{ - "name": "your-package-name", - "version": "1.0.0", - "description": "Your project description", - "main": "dist/index.js", - "exports": { - ".": "./dist/index.js" - }, - "repository": { - "type": "git", - "url": "git+https://github.com/OWNER/REPO.git" - }, - "homepage": "https://github.com/OWNER/REPO#readme", - "bugs": { - "url": "https://github.com/OWNER/REPO/issues" - } -} -``` - -### Specifying Node.js Version Requirements - -The template includes an `engines` field: - -```json -{ - "engines": { - "node": ">=20.0.0" - } -} -``` - -Update this to match your project's Node.js version requirements. - ---- - -## Gitignore Configuration - -**File:** `.gitignore` - -The default `.gitignore` file includes standard exclusion patterns for Node.js, Python, pre-commit, OS-generated files, and IDE configurations. Most projects can use it without modification. - -### Excluding Lock Files - -The `.gitignore` includes a commented-out option to exclude `package-lock.json`: - -```text -# Lock files (optional - uncomment to exclude) -# package-lock.json -``` - -**To exclude lock files from version control:** - -Uncomment the `package-lock.json` line: - -```text -# Lock files (optional - uncomment to exclude) -package-lock.json -``` - -> **Note:** Most projects should keep `package-lock.json` committed to ensure reproducible builds. Only exclude it if you have a specific reason, such as always wanting fresh dependency resolution during installs. - -### Adding Project-Specific Exclusions - -To add custom exclusion patterns for your project: - -1. Open `.gitignore` in your editor -2. Add your patterns at the end of the file -3. Use comments to explain non-obvious exclusions - -**Example:** - -```text -# Project-specific exclusions -data/ -*.log -secrets/ -``` - ---- - -## License Customization - -**File:** `LICENSE` (and related files) - -This template uses the MIT License by default. If your project requires different license terms, you will need to update multiple files to ensure consistency. - -### Files That Reference the License - -When changing your project's license, update all of the following files: - -| File | What to Update | -| --- | --- | -| `LICENSE` | Replace entire file with your license text | -| `CONTRIBUTING.md` | Update the "License" section | -| `README.md` | Update the "License" section (near bottom of file) | -| `pyproject.toml` | Update `license = "MIT"` in `[project]` section | -| `package.json` | Update `"license": "MIT"` field | -| `templates/python/pyproject.toml` | Update `license = "MIT"` (only if keeping the templates directory) | - -> **Note:** The `package-lock.json` file contains `"license": "MIT"` entries, but these refer to the licenses of npm dependencies (not your project). These do NOT need to be changed when updating your project's license. - -### Keeping MIT License (Default) - -No changes required. The MIT License is suitable for most open source projects where you want to allow maximum reuse with minimal restrictions. - -### Changing to Apache 2.0 - -Replace MIT with Apache 2.0 if you need explicit patent protection. - -**Step 1:** Replace the `LICENSE` file content with the Apache 2.0 license text from [apache.org](https://www.apache.org/licenses/LICENSE-2.0.txt). - -**Step 2:** Update all references: - -```markdown - -By contributing to this project, you agree that your contributions will be licensed -under the same license as the project (Apache License 2.0). -``` - -```markdown - -Apache License 2.0 - See [LICENSE](LICENSE) for details. -``` - -```toml -# In pyproject.toml and templates/python/pyproject.toml -license = "Apache-2.0" -``` - -```json -// In package.json -"license": "Apache-2.0" -``` - -### Changing to a Proprietary License - -For closed-source or commercial projects, replace the MIT License with appropriate proprietary terms. - -**Step 1:** Replace the `LICENSE` file with your proprietary license text. - -**Step 2:** Update `CONTRIBUTING.md` to reflect contribution terms (replace `{{COMPANY}}` with your company name): - -```markdown -## License - -This project is proprietary software. By contributing to this project, you agree that: - -1. Your contributions become the property of {{COMPANY}} -2. You have the right to make the contribution -3. You grant {{COMPANY}} all rights to use your contribution - -Contributors may be required to sign a Contributor License Agreement (CLA) before -contributions can be accepted. -``` - -**Step 3:** Update `README.md` (replace `{{YEAR}}` with the copyright year or range, e.g., `2024` or `2020-2024`): - -```markdown -## License - -Proprietary - Copyright {{YEAR}} {{COMPANY}}. All rights reserved. -See [LICENSE](LICENSE) for details. -``` - -**Step 4:** Update package manifests: - -```toml -# In pyproject.toml -license = "Proprietary" -``` - -```json -// In package.json - choose based on your situation: -// Use "UNLICENSED" for internal/private projects with no license granted: -"license": "UNLICENSED" -// Use "SEE LICENSE IN LICENSE" when you have a custom license file: -"license": "SEE LICENSE IN LICENSE" -``` - -**Additional considerations:** - -- Consider requiring Contributor License Agreements (CLAs) for external contributions -- Ensure employment contracts or contributor agreements assign intellectual property rights appropriately -- Review all open source dependencies to ensure their licenses are compatible with proprietary use -- Have legal counsel review your license terms before public or customer distribution - -### Other Open Source Licenses - -For licenses not covered above (BSD, GPL, LGPL, MPL, etc.), follow the same pattern: - -1. Replace `LICENSE` file with the full license text -2. Update `CONTRIBUTING.md` contributor agreement -3. Update `README.md` license section -4. Update `pyproject.toml` with the appropriate SPDX identifier -5. Update `package.json` with the appropriate SPDX identifier - -**Common SPDX identifiers:** - -| License | SPDX Identifier | -| --- | --- | -| MIT | `MIT` | -| Apache 2.0 | `Apache-2.0` | -| BSD 2-Clause | `BSD-2-Clause` | -| BSD 3-Clause | `BSD-3-Clause` | -| GPL 3.0 | `GPL-3.0-only` | -| LGPL 3.0 | `LGPL-3.0-only` | -| MPL 2.0 | `MPL-2.0` | -| ISC | `ISC` | - -For the complete list of SPDX identifiers, see [spdx.org/licenses](https://spdx.org/licenses/). - -### Dual Licensing - -Some projects offer multiple license options (e.g., GPL for open source use, commercial license for proprietary use). If dual licensing: - -1. Include both license texts in `LICENSE` (or separate files like `LICENSE-MIT` and `LICENSE-APACHE`) -2. Clearly explain the licensing options in `README.md` -3. Document which license applies under which conditions - ---- - -## VS Code PowerShell File Encoding for Non-ASCII Characters - -**File:** `.vscode/settings.json` - -The template ships with this default encoding for PowerShell files: - -```json -{ - "[powershell]": { - "files.encoding": "utf8" - } -} -``` - -This is the correct default for the common case and should be kept when: - -- Your `.ps1` files contain **only ASCII characters**, or -- Your repository targets **PowerShell 7+ only** (which defaults to UTF-8). - -### When to Change the Encoding - -If your repository's PowerShell `.ps1` files: - -- contain **non-ASCII characters** (for example, accented characters, CJK text, or special symbols in strings or comments), **and** -- must run on **Windows PowerShell** (v5.1 or earlier), - -then you should change the VS Code workspace encoding setting for PowerShell files to UTF-8 **with** BOM. - -### What to Change - -In `.vscode/settings.json`, replace the `files.encoding` value for PowerShell files: - -```json -{ - "[powershell]": { - "files.encoding": "utf8bom" - } -} -``` - -`utf8bom` is VS Code's identifier for UTF-8 with BOM (Byte Order Mark, `U+FEFF`). - -### Why This May Be Necessary - -When a `.ps1` file is saved as UTF-8 **without** a BOM, older Windows PowerShell versions may not detect the encoding correctly. Instead, they may interpret the file using the system's ANSI code page, which can cause non-ASCII characters in strings, comments, or variable content to be read incorrectly. - -Adding a BOM lets Windows PowerShell detect that the file is UTF-8 and interpret it correctly. PowerShell 7+ defaults to UTF-8 regardless of BOM, so the BOM is unnecessary (but harmless) on modern PowerShell. - -### Down-Level Compatibility Note - -For very old Windows PowerShell compatibility scenarios, the safest portable strategy is often to **avoid non-ASCII source text entirely** when practical. If all characters in the script are ASCII, the file is interpreted identically regardless of whether the system reads it as UTF-8 or as ANSI, making the BOM question moot. - -Relying on "ANSI" or locale-specific encodings (for example, Windows-1252 or Shift_JIS) is generally **not recommended** because those encodings are non-portable and brittle across systems with different locale settings. - -> **Reference:** The full encoding rules are defined in the `### File Encoding` subsection of `.github/instructions/powershell.instructions.md`. - ---- - -## Ongoing Maintenance - -These are periodic maintenance tasks for repositories created from the template. - -### Updating Pre-commit Hooks - -Pre-commit hooks should be kept up-to-date for security and compatibility: - -```bash -# Check for and apply updates to pre-commit hooks -pre-commit autoupdate - -# Test that updated hooks work correctly -pre-commit run --all-files - -# Commit the updated configuration -git add .pre-commit-config.yaml -git commit -m "chore: update pre-commit hooks" -``` - -**Frequency:** Monthly or when security advisories are published for hook dependencies (Black, Ruff, etc.). - -### Reviewing Python Version Requirements - -If your project uses Python, periodically review your minimum Python version requirement: - -1. Visit the [Python Developer's Guide - Versions](https://devguide.python.org/versions/) page -2. Check which versions are in "bugfix" status -3. Update `pyproject.toml` `requires-python` field if needed -4. Update CI workflow Python version matrix if needed - ---- - -## Additional Resources - -- **[Creating a New Repository](GETTING_STARTED_NEW_REPO.md)**: Complete setup guide for new repositories -- **[Adding Template Features to an Existing Repository](GETTING_STARTED_EXISTING_REPO.md)**: Adoption guide for existing repositories -- **[Design Decisions](.github/TEMPLATE_DESIGN_DECISIONS.md)**: Rationale behind template design choices (for maintainers and code reviewers) - -> **Note:** The Design Decisions document (`.github/TEMPLATE_DESIGN_DECISIONS.md`) is internal documentation for understanding WHY the template was designed a certain way. It is NOT an instruction guide—use the getting started guides above for setup instructions. diff --git a/README.md b/README.md index eaaa179..5520570 100644 --- a/README.md +++ b/README.md @@ -1,313 +1,53 @@ -# Project Name + +# macOSLab -> **Note:** This repository was created from [`franklesniak/copilot-repo-template`](https://github.com/franklesniak/copilot-repo-template). +## Metadata -## Description +- **Status:** Phase 0 bootstrap +- **Owner:** Frank Lesniak +- **Last Updated:** 2026-05-05 +- **Scope:** Minimal bootstrap README for `franklesniak/macOSLab`. Full user-facing setup and operating documentation is planned for Phase 1. +- **Related:** [Phase 0 bootstrap instructions](docs/phase-0-bootstrap-codex-instructions.md), [macOSLab repository specification](docs/planning/macOS-imaging-08c-repo-spec-final.md), [macOSLab ADRs](docs/planning/macOS-imaging-08e-ADRs.md) -[Add your project description here] +`macOSLab` is a PowerShell 7.4+ starter kit for building reproducible Apple-silicon macOS VM labs for Intune policy testing. The repository is initialized from [`franklesniak/copilot-repo-template`](https://github.com/franklesniak/copilot-repo-template) and is being tailored for the `MacLab` PowerShell module. ---- +The project goal is to help Microsoft endpoint administrators test risky macOS policies before they reach production users. The supported local virtualization paths are Parallels Desktop Pro and UTM. Tart remains an advanced stub path until later owner approval. -## Table of Contents +## Current Bootstrap State -- [Readme for the Copilot Repository Template](#readme-for-the-copilot-repository-template) - - [What This Template Provides](#what-this-template-provides) - - [Getting Started](#getting-started) - - [Repository Structure](#repository-structure) - - [Language Support](#language-support) - - [Linting Tools](#linting-tools) - - [Testing](#testing) - - [Code Quality](#code-quality) - - [License](#license) +Phase 0 establishes repository identity, trims irrelevant template sample content, keeps governance and validation tooling intact, and records deferred verification work. The full module implementation, user docs, and evidence schema are not part of Phase 0. ---- +The repository languages are PowerShell and Markdown. Python may still appear as a tool runtime for pre-commit hooks such as `check-jsonschema`, but Python sample/source code is not part of this project. -## Readme for the Copilot Repository Template +## Validation -This is a template repository providing best-practice GitHub Copilot instructions and linting configurations for new projects. - -### What This Template Provides - -This template includes: - -- **GitHub Copilot Instructions:** Comprehensive coding standards that guide AI-assisted development -- **Multi-Agent Support:** Instruction files for Claude Code, OpenAI Codex CLI, and Gemini Code Assist (`CLAUDE.md`, `AGENTS.md`, `GEMINI.md`) -- **Language-Specific Guidelines:** Modular instruction files for Markdown, PowerShell, Python, Terraform, JSON/JSONC, and YAML -- **Linting Configurations:** Pre-configured settings for markdownlint, PSScriptAnalyzer, TFLint, and yamllint -- **Data-File Validation:** Pre-commit hooks for `check-json`, `check-yaml`, `yamllint`, `actionlint` (GitHub Actions workflows), `check-jsonschema` (validates the worked-example schema's valid example data under `schemas/examples/example-config/valid/`, plus selected real load-bearing configuration files against built-in vendor schemas), and `check-metaschema` (project-owned schema self-validation) -- **JSON Schemas:** Root-level `schemas/` directory convention for schema-backed JSON and YAML files -- **Pre-commit Hooks:** Automated code quality checks before commits - -### Getting Started - -Choose the guide that matches your situation: - -- **[Creating a New Repository](GETTING_STARTED_NEW_REPO.md)**: Step-by-step guide for creating a new repository from this template -- **[Adding to an Existing Repository](GETTING_STARTED_EXISTING_REPO.md)**: Guide for adopting template features into an existing repository -- **[Optional Configurations](OPTIONAL_CONFIGURATIONS.md)**: Advanced customization options after initial setup - -For template maintainers, see [TEMPLATE_MAINTENANCE.md](TEMPLATE_MAINTENANCE.md). - -### Repository Structure - -```text -.github/ -├── CODEOWNERS # Code ownership for automatic PR review requests -├── copilot-instructions.md # Repo-wide constitution for all changes -├── dependabot.yml # Automated dependency updates configuration -├── instructions/ # Language-specific coding standards -│ ├── docs.instructions.md # Markdown/documentation standards -│ ├── gitattributes.instructions.md # .gitattributes authoring standards -│ ├── json.instructions.md # JSON/JSONC authoring standards -│ ├── powershell.instructions.md # PowerShell coding standards -│ ├── python.instructions.md # Python coding standards -│ ├── terraform.instructions.md # Terraform coding standards -│ └── yaml.instructions.md # YAML authoring standards -├── linting/ # Linting tool configurations -│ └── PSScriptAnalyzerSettings.psd1 # PowerShell linting settings -├── scripts/ # Helper scripts for CI/tooling -└── workflows/ # GitHub Actions workflows - ├── auto-fix-precommit.yml # Auto-fix pre-commit on copilot/** pushes (optional) - ├── check-placeholders.yml # Verifies OWNER/REPO placeholders are replaced - ├── data-ci.yml # JSON/YAML/Actions data-file linting CI - ├── markdownlint.yml # Markdown linting CI (markdownlint) - ├── powershell-ci.yml # PowerShell linting and testing CI (optional) - ├── python-ci.yml # Python linting and testing CI (optional) - └── terraform-ci.yml # Terraform format, validate, lint, test, security CI (optional) - -src/ -└── copilot_repo_template/ # Example Python package (rename for your project) - ├── __init__.py - └── example.py - -tests/ # Test directory -├── __init__.py -├── test_example.py # Python pytest tests -└── PowerShell/ # PowerShell Pester tests - └── Placeholder.Tests.ps1 - -templates/ # Reference templates for project setup -├── python/ # Python project templates -└── powershell/ # PowerShell test templates - └── Example.Tests.ps1 # Comprehensive Pester test example - -schemas/ # JSON Schemas for load-bearing JSON/YAML files (Draft 2020-12) - -pyproject.toml # Python project configuration -.markdownlint.jsonc # Markdown linting configuration -.yamllint.yml # YAML linting configuration -.pre-commit-config.yaml # Pre-commit hooks (multi-language) -AGENTS.md # Agent instructions for OpenAI Codex CLI -CLAUDE.md # Agent instructions for Claude Code -GEMINI.md # Agent instructions for Gemini Code Assist -``` - -#### Key Files Explained - -| File | Purpose | -| --- | --- | -| `.github/CODEOWNERS` | Defines code ownership for automatic PR review requests - replace `@OWNER` placeholder | -| `.github/copilot-instructions.md` | The "constitution" for all code changes - defines safety rules, pre-commit discipline, and references language-specific instructions | -| `.github/dependabot.yml` | Dependabot configuration for automated dependency updates - enabled by default | -| `.github/instructions/*.md` | Language-specific coding standards applied based on file patterns | -| `.github/linting/PSScriptAnalyzerSettings.psd1` | PSScriptAnalyzer settings enforcing OTBS formatting for PowerShell | -| `.github/workflows/auto-fix-precommit.yml` | Automatically commits pre-commit auto-fixes on pushes to `copilot/**` branches by the Copilot coding agent (optional - remove if not using the Copilot coding agent) | -| `.github/workflows/check-placeholders.yml` | CI workflow to verify OWNER/REPO and @OWNER placeholders are replaced after cloning | -| `.github/workflows/data-ci.yml` | Data-file (JSON/YAML/GitHub Actions) linting CI workflow — runs `check-json`, `check-yaml`, `yamllint`, `actionlint`, `check-jsonschema`, and `check-metaschema` as a dedicated check that can be required via branch protection | -| `.github/workflows/markdownlint.yml` | Markdown linting CI workflow (uses [markdownlint](https://github.com/DavidAnson/markdownlint)) | -| `.github/workflows/powershell-ci.yml` | PowerShell linting and Pester testing CI workflow (optional - remove if not using PowerShell) | -| `.github/workflows/python-ci.yml` | Python linting and testing CI workflow (optional - remove if not using Python) | -| `.github/workflows/terraform-ci.yml` | Terraform format, validate, lint, test, and security CI workflow (optional - remove if not using Terraform) | -| `.markdownlint.jsonc` | Markdown linting rules prioritizing auto-fixable checks | -| `.yamllint.yml` | YAML linting configuration (2-space indentation, max line length 120 as warning, unquoted GitHub Actions `on:` allowed) | -| `.pre-commit-config.yaml` | Pre-commit hooks for all projects (multi-language) | -| `schemas/` | Root-level JSON Schemas (Draft 2020-12) describing load-bearing JSON and YAML files | -| `AGENTS.md` | Minimal agent entry point instructions for OpenAI Codex CLI and GitHub Copilot coding agent | -| `CLAUDE.md` | Minimal agent entry point instructions plus Claude-specific workflow guidance | -| `GEMINI.md` | Minimal agent entry point instructions for Gemini Code Assist and GitHub Copilot coding agent | -| `pyproject.toml` | Python project configuration with dev dependencies | -| `src/copilot_repo_template/` | Example Python package - rename for your project | -| `tests/` | Test directory with pytest tests (Python) and Pester tests (PowerShell) | -| `templates/powershell/Example.Tests.ps1` | Comprehensive Pester test template with examples | - -### Language Support - -| Language | Instruction File | File Pattern | CI Workflow | Description | -| --- | --- | --- | --- | --- | -| JSON/JSONC | `.github/instructions/json.instructions.md` | `**/*.json`, `**/*.jsonc` | `.github/workflows/data-ci.yml` (`check-json` on `.json`; `.jsonc` not validated) | JSON authoring standards (strict, schema-backed, deterministic) | -| Markdown/Docs | `.github/instructions/docs.instructions.md` | `**/*.md` | `.github/workflows/markdownlint.yml` | Documentation writing standards | -| PowerShell | `.github/instructions/powershell.instructions.md` | `**/*.ps1` | `.github/workflows/powershell-ci.yml` | PowerShell coding standards (OTBS, v1.0-v7.x) | -| Python | `.github/instructions/python.instructions.md` | `**/*.py` | `.github/workflows/python-ci.yml` | Python coding standards (PEP 8, typing) | -| Terraform | `.github/instructions/terraform.instructions.md` | `**/*.tf`, `**/*.tfvars`, `**/*.tftest.hcl`, etc. | `.github/workflows/terraform-ci.yml` | Terraform coding standards (HCL, modules) | -| YAML | `.github/instructions/yaml.instructions.md` | `**/*.yml`, `**/*.yaml` | `.github/workflows/data-ci.yml` (`check-yaml`, `yamllint`; `actionlint` for workflows only) | YAML authoring standards (explicit, conservative, schema-backed) | - -> **JSON note:** `check-json` validates strict `.json` only; it does **not** validate `.jsonc`. JSONC is allowed where the consuming tool supports it, and stricter enforcement requires JSONC-aware tooling. -> -> **Schemas:** JSON Schemas for load-bearing JSON and YAML files live at the repository root under `schemas/` (not `.github/schemas/`). See [`schemas/README.md`](schemas/README.md) for conventions. - -### Linting Tools - -This template organizes linting configurations in `.github/linting/` (for PSScriptAnalyzer) and the repository root (for markdownlint). Projects MAY reorganize these configurations to a different location (e.g., a project-specific `config/` directory) if preferred. If configurations are moved, update the paths referenced in CI workflows and `.github/copilot-instructions.md` accordingly. - -#### Markdown Linting - -Configuration: `.markdownlint.jsonc` +Use these commands before opening or updating a pull request: ```bash -# Check markdown files npm run lint:md - -# Auto-fix issues -npx markdownlint-cli2 "**/*.md" "#node_modules" --fix -``` - -#### PowerShell Linting (PSScriptAnalyzer) - -Configuration: `.github/linting/PSScriptAnalyzerSettings.psd1` - -```powershell -# Check PowerShell files -Invoke-ScriptAnalyzer -Path .\script.ps1 -Settings .\.github\linting\PSScriptAnalyzerSettings.psd1 - -# Auto-fix formatting issues -Invoke-ScriptAnalyzer -Path .\script.ps1 -Settings .\.github\linting\PSScriptAnalyzerSettings.psd1 -Fix -``` - -#### Python Linting - -Configuration: `.pre-commit-config.yaml` - -```bash -# Run all pre-commit hooks -pre-commit run --all-files - -# Run specific hooks -pre-commit run black --all-files -pre-commit run ruff-check --all-files -``` - -#### JSON, YAML, and GitHub Actions Linting - -JSON, YAML, and GitHub Actions workflow validation runs through pre-commit hooks. Configuration: `.pre-commit-config.yaml` (and `.yamllint.yml` for `yamllint`). - -- **`check-json`** — strict `.json` syntax validation. Does **not** validate `.jsonc`; use JSONC-aware tooling if you need stricter enforcement for `.jsonc` files. -- **`check-yaml`** — `.yml` / `.yaml` parse check. -- **`yamllint`** — YAML style enforcement per `.yamllint.yml`. -- **`actionlint`** — GitHub Actions workflow linting. -- **`check-jsonschema`** — JSON Schema validation, scoped to the worked-example schema's valid example data under `schemas/examples/example-config/valid/`. Downstream repositories MAY add additional `check-jsonschema` hook entries for their own schema-backed file families. -- **`check-metaschema`** — self-validates the worked-example schema (`schemas/example-config.schema.json`) against its declared JSON Schema Draft 2020-12 metaschema. - -Prettier is **opt-in** and is not part of the default data-file toolchain. - -> **Schema validation (worked example shipped).** `check-jsonschema` is wired into `.pre-commit-config.yaml` to validate the template's worked-example schema (`schemas/example-config.schema.json`) and its valid example data under `schemas/examples/example-config/valid/`, plus a `check-metaschema` self-validation hook for the schema itself. See [`schemas/README.md`](schemas/README.md) for the worked example, the canonical downstream removal checklist, and future-work candidates. Downstream repositories MAY add additional `check-jsonschema` hook entries for their own schema-backed file families. - -```bash -# Run all pre-commit hooks (includes data-file validators) +npm run lint:md:nested pre-commit run --all-files - -# Run a specific hook -pre-commit run check-json --all-files -pre-commit run check-yaml --all-files -pre-commit run yamllint --all-files -pre-commit run actionlint --all-files -pre-commit run check-jsonschema --all-files -pre-commit run check-metaschema --all-files -``` - -#### Terraform Linting - -This repository includes Terraform linting via: - -- **terraform fmt:** Format checking and auto-formatting -- **terraform validate:** Configuration validation -- **TFLint:** Best practice linting with cloud provider plugins - -Configuration: `.tflint.hcl` - -```bash -# Format check -terraform fmt -check -recursive - -# Format fix -terraform fmt -recursive - -# Validate (requires init) -terraform init -backend=false && terraform validate - -# Lint -tflint --init && tflint --recursive -``` - -### Testing - -#### Python Tests - -Python tests use pytest with coverage reporting. - -```bash -# Run all Python tests -pytest tests/ -v --cov --cov-report=term-missing - -# Run a specific test file -pytest tests/test_example.py -v - -# Run the schema-example contract test (validates schemas/examples//{valid,invalid}/ fixtures) -pytest tests/test_schema_examples.py -v ``` -The schema-example contract test ([`tests/test_schema_examples.py`](tests/test_schema_examples.py)) auto-discovers `schemas/*.schema.json` and the matching `schemas/examples//{valid,invalid}/` fixtures and asserts that valid fixtures pass and invalid fixtures fail. A starter version is also available at [`templates/python/tests/test_schema_examples.py`](templates/python/tests/test_schema_examples.py) for downstream adoption. - -> **Prerequisite — `check-jsonschema` must be on PATH.** This test shells out to the `check-jsonschema` CLI and uses `@pytest.mark.skipif(shutil.which("check-jsonschema") is None, ...)` to skip every parametrized case when the binary is not available. **A skipped test is not a passing test:** if `check-jsonschema` is missing, pytest still exits `0`, but no schema validation actually ran. Install it via `pip install -e ".[dev]"` (which pulls in `check-jsonschema` along with the other dev dependencies — see the `[project.optional-dependencies] dev` table in [`pyproject.toml`](pyproject.toml)) or `pip install check-jsonschema` if you only need this one tool; either path puts the binary on PATH so `shutil.which` finds it. **`pre-commit install --install-hooks` does NOT** add `check-jsonschema` to PATH — pre-commit installs hook dependencies into its own isolated per-hook environments under `~/.cache/pre-commit/`, which satisfies the pre-commit hook but not the `shutil.which` lookup this pytest test performs. To validate schemas through the pre-commit toolchain instead of through this pytest test, run `pre-commit run check-jsonschema --all-files`. Confirm the test reports collected cases (e.g., `N passed`) rather than `N skipped` to know it actually ran. - -#### PowerShell Tests - -PowerShell tests use Pester 5.x. +PowerShell validation is handled through PSScriptAnalyzer and Pester 5.7.1: ```powershell -# Install Pester if needed -Install-Module -Name Pester -MinimumVersion 5.0 -Force -Scope CurrentUser - -# Run all Pester tests +Invoke-ScriptAnalyzer -Path . -Settings .github/linting/PSScriptAnalyzerSettings.psd1 Invoke-Pester -Path tests/ -Output Detailed - -# Run a specific test file -Invoke-Pester -Path tests/PowerShell/Placeholder.Tests.ps1 ``` -CI runs PowerShell tests on Windows, macOS, and Linux to ensure cross-platform compatibility. - -See `templates/powershell/Example.Tests.ps1` for a comprehensive Pester test template. - -#### Terraform Tests - -Terraform tests use the native Terraform test framework (Terraform 1.6+). - -```bash -# Run all Terraform tests -terraform test -verbose - -# Run specific test file -terraform test -filter=tests/unit.tftest.hcl -``` - -Tests are located in `modules/*/tests/` directories. - -See `templates/terraform/Example.tftest.hcl` for a comprehensive Terraform test template. - -### Code Quality +## Deferred Work -This repository enforces code quality through: +Root TODO files track deferred verification and owner actions: -- **Markdown Linting:** Runs on pre-commit and in CI -- **JSON/YAML Validation:** `check-json` (strict `.json`), `check-yaml`, and `yamllint` run on pre-commit -- **GitHub Actions Linting:** `actionlint` runs on pre-commit -- **Schema Validation:** `check-jsonschema` validates the worked-example schema's example data and selected real load-bearing configuration files (e.g., `.github/dependabot.yml`) against built-in vendor schemas; `check-metaschema` self-validates project-owned schemas. See [`.github/TEMPLATE_DESIGN_DECISIONS.md`](.github/TEMPLATE_DESIGN_DECISIONS.md) (Built-in Schema Validation ADR) for the policy -- **JSON Schemas:** Root-level `schemas/` convention for schema-backed JSON/YAML files -- **GitHub Copilot Instructions:** Guides AI-assisted development -- **Pre-commit Hooks:** Catches issues before they reach CI -- **PSScriptAnalyzer:** PowerShell static analysis with OTBS formatting -- **TFLint:** Terraform linting with configurable rules and cloud provider plugins +- [TODO-Phase-00-Branch-Protection.md](TODO-Phase-00-Branch-Protection.md) +- [TODO-Phase-04-Media-Acquisition.md](TODO-Phase-04-Media-Acquisition.md) +- [TODO-Phase-05-Parallels-Provider.md](TODO-Phase-05-Parallels-Provider.md) +- [TODO-Phase-06-UTM-Provider.md](TODO-Phase-06-UTM-Provider.md) +- [TODO-Phase-07-Evidence-Pipeline.md](TODO-Phase-07-Evidence-Pipeline.md) +- [TODO-Phase-08-Validation-Loop.md](TODO-Phase-08-Validation-Loop.md) +- [TODO-Phase-10-Deferred-Work.md](TODO-Phase-10-Deferred-Work.md) -### License +## License -MIT License - See [LICENSE](LICENSE) for details. +This project is licensed under the MIT License. See [LICENSE](LICENSE). diff --git a/SECURITY.md b/SECURITY.md index 27009cf..4c068ca 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -16,12 +16,11 @@ If you discover a security vulnerability in this project, please report it priva This allows for private discussion and coordinated disclosure. -### Option 2: Email +### Option 2: GitHub Private Vulnerability Reporting Link -Contact the maintainers directly at: +Use the direct private vulnerability reporting link: - -- Email: [security contact email] +- https://github.com/franklesniak/macOSLab/security/advisories/new ### What to Include diff --git a/TEMPLATE_MAINTENANCE.md b/TEMPLATE_MAINTENANCE.md deleted file mode 100644 index 4d3619b..0000000 --- a/TEMPLATE_MAINTENANCE.md +++ /dev/null @@ -1,322 +0,0 @@ -# Template Maintenance Guide - -This guide is for **maintainers of the `franklesniak/copilot-repo-template` repository**. It documents periodic maintenance tasks to keep the template current and functional. - -> **Note:** If you created a repository FROM this template, see [OPTIONAL_CONFIGURATIONS.md](OPTIONAL_CONFIGURATIONS.md#ongoing-maintenance) for maintenance guidance relevant to your repository. - ---- - -## Table of Contents - -- [Recommended Review Cadence](#recommended-review-cadence) -- [Updating Pre-commit Hook Versions](#updating-pre-commit-hook-versions) -- [Reviewing the Worked-Example Schema and Data CI Workflow](#reviewing-the-worked-example-schema-and-data-ci-workflow) -- [Reviewing Python Version Requirements](#reviewing-python-version-requirements) -- [Reviewing Terraform Version Requirements](#reviewing-terraform-version-requirements) -- [Reviewing Terraform Provider Versions](#reviewing-terraform-provider-versions) -- [Reviewing Instruction File Versions](#reviewing-instruction-file-versions) -- [Reviewing Agent Instruction Files](#reviewing-agent-instruction-files) -- [Testing Template Changes](#testing-template-changes) - ---- - -## Recommended Review Cadence - -To keep the template current and functional, maintainers **SHOULD** review template documentation and workflows on a **quarterly basis**. - -**Quarterly Review Checklist:** - -- [ ] Review and update pre-commit hook versions -- [ ] Check for updates to GitHub Actions used in workflows -- [ ] Review and update Terraform version in CI workflows -- [ ] Review instruction files for accuracy and relevance -- [ ] Verify all CI workflows still pass with latest dependency versions -- [ ] Verify agent instruction files (`CLAUDE.md`, `AGENTS.md`, `GEMINI.md`) remain aligned with `.github/copilot-instructions.md` -- [ ] Review and address any open issues or feedback - -**Annual Review:** - -- [ ] Review Python version requirements (typically October) -- [ ] Review major version updates for key dependencies (Node.js, Terraform providers, etc.) -- [ ] Evaluate new GitHub features that could enhance the template - -> **Tip:** Set a calendar reminder for quarterly reviews to ensure consistent maintenance. - ---- - -## Updating Pre-commit Hook Versions - -Pre-commit hooks **SHOULD** be kept up-to-date for security and compatibility. - -### Maintenance Cadence - -This template uses a layered cadence for pre-commit hook maintenance: - -- **Routine (weekly, automated).** [Dependabot](.github/dependabot.yml)'s `pre-commit` ecosystem opens grouped pull requests for minor and patch updates to hooks pinned in `.pre-commit-config.yaml`. These PRs SHOULD be reviewed and merged routinely; CI exercises the updated hooks via `pre-commit run --all-files` in `.github/workflows/python-ci.yml`. -- **Major-version refreshes (manual).** Maintainers MAY run `pre-commit autoupdate` manually to refresh major versions, or for an explicit quarterly maintenance pass. Major version bumps are not handled by the grouped Dependabot configuration and require an intentional review. -- **Changelog review.** Before accepting any major hook update (whether from `pre-commit autoupdate` or a Dependabot PR), maintainers MUST review the upstream changelog or release notes for breaking changes. See [Breaking Change Considerations](#breaking-change-considerations) for tool-specific notes. -- **Validation.** After any update — Dependabot-driven or manual — maintainers MUST run `pre-commit run --all-files` locally and confirm a clean pass. -- **Commit hygiene.** Hook version bumps and any auto-fixes produced by the updated hooks MUST be committed together in the same change. Do not split formatting churn caused by an updated hook into a separate "fix formatting" commit. - -### Quick Update - -```bash -# Check for and apply updates to pre-commit hooks -pre-commit autoupdate - -# Test that updated hooks work correctly -pre-commit run --all-files -``` - -**Frequency:** Quarterly (or as needed for major-version refreshes), or sooner when security advisories are published for hook dependencies. Routine minor and patch updates are handled automatically by Dependabot — see [Maintenance Cadence](#maintenance-cadence) above for the full layered cadence. - -### Tools to Track - -The following pre-commit hooks are configured in this template. Check their repositories for the latest releases: - -| Tool | Repository | Purpose | -| --- | --- | --- | -| pre-commit-hooks | | General file checks (trailing whitespace, JSON validation via `check-json`, YAML parsing via `check-yaml`, etc.) | -| Black | | Python code formatting | -| Ruff | | Python linting and formatting | -| markdownlint-cli2 | | Markdown linting | -| pre-commit-terraform | | Terraform formatting, validation, and linting | -| yamllint | | YAML style enforcement (driven by `.yamllint.yml`) | -| actionlint | | GitHub Actions workflow linting | -| check-jsonschema | | JSON Schema validation (worked-example schema and any downstream-added schema-backed file families); also provides the `check-metaschema` hook | - -### Files Requiring Manual Updates - -After running `pre-commit autoupdate`, manually update version references in documentation files. The `pre-commit autoupdate` command only updates `.pre-commit-config.yaml`—version references in documentation examples require manual updates. - -#### Black (Python formatter) - -- `.pre-commit-config.yaml` (updated by `pre-commit autoupdate`) -- `OPTIONAL_CONFIGURATIONS.md` (Python pre-commit examples) -- `GETTING_STARTED_NEW_REPO.md` (commented example in pre-commit config) -- `GETTING_STARTED_EXISTING_REPO.md` (Python pre-commit examples) - -#### Ruff (Python linter) - -- `.pre-commit-config.yaml` (updated by `pre-commit autoupdate`) -- `OPTIONAL_CONFIGURATIONS.md` (Python pre-commit examples) -- `GETTING_STARTED_NEW_REPO.md` (commented example in pre-commit config) -- `GETTING_STARTED_EXISTING_REPO.md` (Python pre-commit examples) - -#### pre-commit-terraform (Terraform hooks) - -- `.pre-commit-config.yaml` (updated by `pre-commit autoupdate`) -- `.github/instructions/terraform.instructions.md` (pre-commit configuration examples) -- `docs/terraform/TERRAFORM_LINTING_GUIDE.md` (pre-commit configuration examples) -- `docs/terraform/TERRAFORM_COPILOT_INSTRUCTIONS_GUIDE.md` (pre-commit configuration examples) -- `docs/terraform/TERRAFORM_TESTING_GUIDE.md` (pre-commit configuration examples) - -#### Other Hooks (no documentation references) - -The following hooks are only referenced in `.pre-commit-config.yaml` and do not require manual documentation updates: - -- pre-commit-hooks -- markdownlint-cli2 - -### Verification - -After updating versions, use these commands to search for potentially stale version references: - -```bash -# Check for Black version references (update the version number as appropriate) -grep -rn "rev:.*26\.1\.0" --include="*.md" --include="*.yaml" . - -# Check for Ruff version references (update the version number as appropriate) -grep -rn "rev:.*v0\.14\.14" --include="*.md" --include="*.yaml" . - -# Check for pre-commit-terraform version references (update the version number as appropriate) -grep -rn "rev:.*v1\.105\.0" --include="*.md" --include="*.yaml" . - -# Generic search for any rev: patterns with version numbers -grep -rn "rev:.*v\?[0-9]\+\.[0-9]\+\.[0-9]\+" --include="*.md" --include="*.yaml" . -``` - -### Breaking Change Considerations - -When updating to new major versions, check the release notes for breaking changes: - -- **pre-commit-hooks:** May remove deprecated hooks or change Python version requirements. Review [pre-commit-hooks releases](https://github.com/pre-commit/pre-commit-hooks/releases). - -- **Black:** Major releases may introduce style changes that reformat existing code differently. Review [Black changelog](https://github.com/psf/black/blob/main/CHANGES.md). Consider running `black --check` on a representative codebase before upgrading. - -- **Ruff:** Frequently adds new rules that may flag previously-passing code. Review [Ruff changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md). New rules are typically disabled by default, but rule behavior changes can affect existing configurations. - -- **pre-commit-terraform:** May change hook IDs, arguments, or tool dependencies. Review [pre-commit-terraform releases](https://github.com/antonbabenko/pre-commit-terraform/releases). Ensure any referenced external tools (terraform, tflint, etc.) remain compatible. - -- **yamllint:** New rules or default tightening can flag previously-passing YAML. Review [yamllint releases](https://github.com/adrienverge/yamllint/releases). The repository's `.yamllint.yml` carries a documented `truthy.check-keys: false` exception; verify that exception still applies after major updates. - -- **actionlint:** Adds new checks as GitHub Actions evolves. Review [actionlint releases](https://github.com/rhysd/actionlint/releases). The hook builds the binary from a Go toolchain on first run; major version bumps may change ShellCheck integration or runner-label awareness. - -- **check-jsonschema:** Validator behavior, JSON Schema draft support, and bundled schema catalog versions can change. Review [check-jsonschema releases](https://github.com/python-jsonschema/check-jsonschema/releases). After bumping, re-run `pytest tests/test_schema_examples.py -v` to confirm the worked-example `valid/` and `invalid/` fixtures still produce the expected pass/fail outcomes. - ---- - -## Reviewing the Worked-Example Schema and Data CI Workflow - -The template ships a worked-example JSON Schema (`schemas/example-config.schema.json`), valid and invalid example fixtures under `schemas/examples/example-config/`, the schema-example pytest contract at `tests/test_schema_examples.py`, and the dedicated [`.github/workflows/data-ci.yml`](.github/workflows/data-ci.yml) workflow. These need periodic review to stay aligned with current JSON Schema and pre-commit hook versions. - -**When to review:** Quarterly, or whenever `check-jsonschema`, `pre-commit-hooks`, `yamllint`, or `actionlint` have a major version bump. - -**What to check:** - -1. The worked-example schema's declared `$schema` draft URI is still appropriate (the template uses JSON Schema Draft 2020-12). -2. The worked-example fixtures still demonstrate the contract clearly — at minimum, `valid/minimal.json`, `valid/full.json`, `invalid/missing-required.json`, `invalid/wrong-type.json`, and `invalid/extra-property.json` should each still illustrate a distinct schema behavior. -3. `pytest tests/test_schema_examples.py -v` still passes (every `valid/` fixture exits `0`; every `invalid/` fixture exits non-zero). -4. The `.github/workflows/data-ci.yml` workflow continues to invoke `check-json`, `check-yaml`, `yamllint`, `actionlint`, `check-jsonschema`, and `check-metaschema`, and its top-of-file comment still accurately describes how it differs from `auto-fix-precommit.yml`. -5. The canonical [downstream removal checklist](schemas/README.md#downstream-removal-checklist) in `schemas/README.md` still matches the actual files and hook IDs that ship with the worked example. - -**When to update:** - -- If a hook ID, hook scope, or schema file moves, update the schema, the fixtures, the pre-commit config, the data CI workflow, and the canonical removal checklist together in the same change. -- Do not duplicate the removal-checklist steps elsewhere; keep `schemas/README.md` as the single source of truth and link to it from any other documentation that mentions removal. - ---- - -## Reviewing Python Version Requirements - -This template requires Python versions that are currently receiving bugfix updates from the Python core team. - -**When to review:** Annually, typically around October when new Python versions are released. - -**What to check:** - -1. Visit the [Python Developer's Guide - Versions](https://devguide.python.org/versions/) page -2. Identify which versions are in "bugfix" status (not "security" or "end-of-life") -3. Update the following files if the minimum supported version changes: - - `.github/workflows/python-ci.yml` (Python version matrix) - - `pyproject.toml` (requires-python field) - - `templates/python/pyproject.toml` (requires-python field) - - `.github/instructions/python.instructions.md` (version references) - ---- - -## Reviewing Terraform Version Requirements - -This template uses a pinned Terraform version in CI workflows for reproducibility and pre-commit hook execution. - -**When to review:** Quarterly, or when a new stable Terraform release is available. - -**What to check:** - -1. Visit the [Terraform Releases](https://releases.hashicorp.com/terraform/) page or the [Terraform GitHub Releases](https://github.com/hashicorp/terraform/releases) -2. Identify the latest stable release (avoid alpha, beta, or RC versions) -3. Update the Terraform version in the following workflow files: - - `.github/workflows/terraform-ci.yml` (format, validate, and test jobs) - - `.github/workflows/python-ci.yml` (pre-commit job) - - `.github/workflows/auto-fix-precommit.yml` (auto-fix job) - -**Version considerations:** - -- **Pre-commit workflows:** Use the latest stable version for pre-commit hooks (terraform_fmt, terraform_validate, terraform_tflint) -- **Terraform CI tests:** The test framework requires Terraform 1.6.0+ and mock_provider requires 1.7.0+. The latest stable version satisfies both requirements. -- **Documentation:** After updating, verify that examples in documentation under `docs/terraform/` remain accurate. Note that these are illustrative examples and do not need to be updated unless the version syntax changes. - ---- - -## Reviewing Terraform Provider Versions - -The Terraform instructions file uses the newest stable major versions in provider version constraint examples. These should be reviewed periodically to ensure examples reflect current best practices. - -**When to review:** Quarterly, or when a new major version of a provider becomes the recommended stable release. - -**What to check:** - -1. Visit the provider registries: - - [AWS Provider](https://registry.terraform.io/providers/hashicorp/aws/latest) - - [Azure Provider](https://registry.terraform.io/providers/hashicorp/azurerm/latest) - - [GCP Provider](https://registry.terraform.io/providers/hashicorp/google/latest) - - > **Note:** Terraform Registry navigation links — including the provider links above — **MUST** use the `latest` path segment, not a pinned provider or module version. See the **Terraform Registry Reference URLs Use /latest/** ADR in [`.github/TEMPLATE_DESIGN_DECISIONS.md`](.github/TEMPLATE_DESIGN_DECISIONS.md) for the scope (Terraform-file comments and instructional Markdown), rationale, and authoritative version sources; the canonical, agent-loadable rule for Terraform-file comments lives in [`.github/instructions/terraform.instructions.md`](.github/instructions/terraform.instructions.md). -2. Identify current stable major versions for each provider -3. If a new major version is now the recommended stable release, update the following files: - - `.github/instructions/terraform.instructions.md` (version constraint examples throughout) - - `.github/TEMPLATE_DESIGN_DECISIONS.md` (current versions table in "Current Provider Versions in Terraform Examples" section) - -**Current versions (as of last update):** - -| Provider | Example Constraint | Current Stable | -| --- | --- | --- | -| AWS | `~> 6.0` | 6.31.0 | -| Azure | `~> 4.0` | 4.58.0 | -| GCP | `~> 7.0` | 7.18.0 | - -**How to update:** - -When updating provider versions in terraform.instructions.md, search for the version constraint patterns: - -```bash -# Search for AWS provider version references -grep -n "~> 5\.0\|~> 6\.0" .github/instructions/terraform.instructions.md - -# Search for Azure provider version references -grep -n "~> 3\.0\|~> 4\.0" .github/instructions/terraform.instructions.md - -# Search for GCP provider version references -grep -n "~> 6\.0\|~> 7\.0" .github/instructions/terraform.instructions.md -``` - -Update all occurrences to the new major version constraint (e.g., `~> 6.0` to `~> 7.0`). - ---- - -## Reviewing Instruction File Versions - -The instruction files in `.github/instructions/` include version numbers in the format `Major.Minor.YYYYMMDD.Revision`. - -**When to update:** - -- Major version: Breaking changes to coding standards -- Minor version: New guidance or significant clarifications -- Date/Revision: Bug fixes or minor wording changes - -**Files to review:** - -- `.github/instructions/docs.instructions.md` -- `.github/instructions/python.instructions.md` -- `.github/instructions/powershell.instructions.md` -- `.github/instructions/terraform.instructions.md` - ---- - -## Reviewing Agent Instruction Files - -Agent instruction files (`CLAUDE.md`, `AGENTS.md`, `GEMINI.md`) are thin entry points that **MUST** stay aligned with `.github/copilot-instructions.md`. The canonical file holds the full shared rule set; the root agent files keep only a minimal inline summary of the highest-priority shared rules plus any platform-specific guidance. - -**What to check during review:** - -- Canonical-source wording is accurate in `.github/copilot-instructions.md` and the root agent files -- Minimal shared guidance in the agent files still matches the canonical file for safety, pre-commit, validation commands, and language-instruction references -- Platform-specific sections still apply and do not contradict the canonical file -- Removed duplication does not creep back into the agent entry points unnecessarily - -**Verification command:** - -```bash -# Compare recent commit history to identify changes that may require alignment review -# If copilot-instructions.md was modified more recently than agent files, -# manually verify the minimal summaries and any platform-specific sections -git log --oneline -5 .github/copilot-instructions.md -git log --oneline -5 CLAUDE.md AGENTS.md GEMINI.md -``` - -**When to update:** Whenever high-priority shared guidance in `.github/copilot-instructions.md` changes, whenever platform-specific guidance changes, or when new agent platforms emerge that require their own convention files. - ---- - -## Testing Template Changes - -Before merging significant template changes: - -1. Create a test repository from the template -2. Complete the setup process following `GETTING_STARTED_NEW_REPO.md` -3. Verify all CI workflows pass -4. Test the placeholder check workflow triggers and passes after placeholder replacement -5. Verify issue templates render correctly -6. Open and close a test PR to verify the PR template - -Delete the test repository after verification. diff --git a/TODO-Phase-00-Branch-Protection.md b/TODO-Phase-00-Branch-Protection.md new file mode 100644 index 0000000..1d7c809 --- /dev/null +++ b/TODO-Phase-00-Branch-Protection.md @@ -0,0 +1,13 @@ + +# TODO Phase 00 Branch Protection + +## Metadata + +- **Status:** Active +- **Owner:** Repository owner +- **Last Updated:** 2026-05-05 +- **Scope:** Manual GitHub-side branch-protection setup deferred from Phase 0 bootstrap. + +## Checklist + +- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 0 closeout; Why it matters: branch protection cannot be configured from the local workspace; Action: configure protection or a ruleset for `main` after this bootstrap PR is ready; Acceptance condition: `main` requires the agreed CI checks and review policy before direct changes are allowed; Source: `docs/phase-0-bootstrap-codex-instructions.md`. diff --git a/TODO-Phase-04-Media-Acquisition.md b/TODO-Phase-04-Media-Acquisition.md new file mode 100644 index 0000000..b71eb66 --- /dev/null +++ b/TODO-Phase-04-Media-Acquisition.md @@ -0,0 +1,13 @@ + +# TODO Phase 04 Media Acquisition + +## Metadata + +- **Status:** Active +- **Owner:** Repository owner +- **Last Updated:** 2026-05-05 +- **Scope:** Deferred verification for media discovery and download tooling before implementing Phase 4. + +## Checklist + +- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 4 media acquisition; Why it matters: `mist-cli` syntax and output can change across releases; Action: verify current `mist-cli` list/download syntax and record exact commands for discovering and downloading pinned macOS restore images or provider-appropriate install artifacts; Acceptance condition: the verified commands, version output, and expected metadata fields are documented before `Get-MacLabMedia` implementation; Source: ADR-0005 and spec Section 9.6. diff --git a/TODO-Phase-05-Parallels-Provider.md b/TODO-Phase-05-Parallels-Provider.md new file mode 100644 index 0000000..ec36813 --- /dev/null +++ b/TODO-Phase-05-Parallels-Provider.md @@ -0,0 +1,13 @@ + +# TODO Phase 05 Parallels Provider + +## Metadata + +- **Status:** Active +- **Owner:** Repository owner +- **Last Updated:** 2026-05-05 +- **Scope:** Deferred verification for Parallels Desktop command-line automation before implementing Phase 5. + +## Checklist + +- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 5 Parallels provider; Why it matters: provider automation must match the installed Parallels Desktop Pro command surface; Action: verify current `prlctl` syntax, version output, and edition detection on the owner/demo host; Acceptance condition: exact commands and expected outputs are documented and provider tests can be written without guessing; Source: ADR-0005 and spec Section 9.6. diff --git a/TODO-Phase-06-UTM-Provider.md b/TODO-Phase-06-UTM-Provider.md new file mode 100644 index 0000000..b257560 --- /dev/null +++ b/TODO-Phase-06-UTM-Provider.md @@ -0,0 +1,13 @@ + +# TODO Phase 06 UTM Provider + +## Metadata + +- **Status:** Active +- **Owner:** Repository owner +- **Last Updated:** 2026-05-05 +- **Scope:** Deferred verification for UTM automation and manual-step boundaries before implementing Phase 6. + +## Checklist + +- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 6 UTM provider; Why it matters: UTM automation may not mirror Parallels CLI parity and manual gaps must be explicit; Action: verify current UTM and `utmctl` automation surface, including VM creation, launch, stop, snapshot, import/export, and any manual-step requirements; Acceptance condition: UTM capability matrix and manual-step gaps are documented before provider implementation; Source: ADR-0005 and spec Section 9.6. diff --git a/TODO-Phase-07-Evidence-Pipeline.md b/TODO-Phase-07-Evidence-Pipeline.md new file mode 100644 index 0000000..79573b3 --- /dev/null +++ b/TODO-Phase-07-Evidence-Pipeline.md @@ -0,0 +1,13 @@ + +# TODO Phase 07 Evidence Pipeline + +## Metadata + +- **Status:** Active +- **Owner:** Repository owner +- **Last Updated:** 2026-05-05 +- **Scope:** Deferred replacement of the temporary worked-example schema with the real macOSLab evidence-bundle schema. + +## Checklist + +- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 7 evidence pipeline; Why it matters: Phase 0 keeps the worked-example schema only to exercise validation tooling; Action: replace `schemas/example-config.schema.json` and its example fixtures with the real evidence-bundle schema from spec Section 25, then update `.pre-commit-config.yaml`, data-file CI, schema docs, and examples together; Acceptance condition: valid evidence examples pass schema validation, invalid examples are rejected by an explicit test or validation script, and the worked example is fully removed; Source: `docs/phase-0-bootstrap-codex-instructions.md` and spec Section 25. diff --git a/TODO-Phase-08-Validation-Loop.md b/TODO-Phase-08-Validation-Loop.md new file mode 100644 index 0000000..4ba029d --- /dev/null +++ b/TODO-Phase-08-Validation-Loop.md @@ -0,0 +1,13 @@ + +# TODO Phase 08 Validation Loop + +## Metadata + +- **Status:** Active +- **Owner:** Repository owner +- **Last Updated:** 2026-05-05 +- **Scope:** Deferred verification for Defender health evidence before implementing Phase 8. + +## Checklist + +- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 8 validation loop; Why it matters: Defender `mdatp health` output can change and evidence fixtures must not leak real tenant or device data; Action: verify the current Defender `mdatp health` output shape and create sanitized fixtures for evidence tests; Acceptance condition: fixture fields needed by validation code are documented, sanitized, and covered by tests before Defender validation ships; Source: ADR-0005 and spec Section 9.6. diff --git a/TODO-Phase-10-Deferred-Work.md b/TODO-Phase-10-Deferred-Work.md new file mode 100644 index 0000000..978e14e --- /dev/null +++ b/TODO-Phase-10-Deferred-Work.md @@ -0,0 +1,15 @@ + +# TODO Phase 10 Deferred Work + +## Metadata + +- **Status:** Active +- **Owner:** Repository owner +- **Last Updated:** 2026-05-05 +- **Scope:** Deferred optional or owner-approval-dependent work that is outside Phase 0 and not required for the first working local lab path. + +## Checklist + +- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 10 optional providers and CI; Why it matters: Tart is useful for advanced CLI and CI paths but is not a v1 parity provider; Action: decide whether to implement full Tart provider parity beyond the documented/stubbed path; Acceptance condition: owner approves scope, license posture is re-verified, and provider tests define supported and unsupported capabilities; Source: ADR-0006. +- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 10 cloud cleanup; Why it matters: local VM rollback does not rewind Intune, Entra, Defender portal state, audit logs, or reporting history, and unsafe matching could mutate the wrong cloud record; Action: decide whether `Reset-IntuneMacLabDevice.ps1` may move beyond report-only mode into cloud mutation; Acceptance condition: owner approves the change and tests cover scoping, confirmation, redaction, soft-delete/retire behavior, and failure paths; Source: ADR-0010. +- [ ] Owner: Repository owner; Status: Open; Phase gate affected: Phase 1 or later security documentation; Why it matters: ADR-0009 approves only a narrow optional project-specific paragraph for `SECURITY.md`; Action: decide whether to add the ADR-0009 paragraph after owner review; Acceptance condition: owner explicitly approves the exact paragraph and the inherited responsible-disclosure process remains intact; Source: ADR-0009. diff --git a/docs/phase-0-bootstrap-pr-description.md b/docs/phase-0-bootstrap-pr-description.md new file mode 100644 index 0000000..1648e30 --- /dev/null +++ b/docs/phase-0-bootstrap-pr-description.md @@ -0,0 +1,93 @@ + +# Phase 0 Bootstrap PR Description Draft + +## Metadata + +- **Status:** Draft +- **Owner:** Repository Maintainers +- **Last Updated:** 2026-05-05 +- **Scope:** PR-description content for the Phase 0 bootstrap branch. This records applied steps, skipped steps, manual owner actions, validation, and requested authorization. +- **Related:** [Phase 0 bootstrap instructions](phase-0-bootstrap-codex-instructions.md), [macOSLab repository specification](planning/macOS-imaging-08c-repo-spec-final.md), [macOSLab ADRs](planning/macOS-imaging-08e-ADRs.md) + +## Summary + +This Phase 0 bootstrap customizes the template repository for `franklesniak/macOSLab`, a PowerShell 7.4+ and Markdown repository for reproducible Intune-managed macOS VM labs. + +## Applied Changes + +- Replaced repository identity placeholders with `franklesniak/macOSLab`, `@franklesniak`, `macOSLab`, and `Frank Lesniak` / `2026` where applicable. +- Updated contact guidance so conduct reports go to repository owners through GitHub profile contact links. +- Updated security reporting to GitHub private vulnerability reporting at `https://github.com/franklesniak/macOSLab/security/advisories/new`. +- Kept `SECURITY.md` changes narrow and did not add the deferred project-specific paragraph from ADR-0009. +- Customized README, CONTRIBUTING, issue templates, PR template, CODEOWNERS, package metadata, VS Code title, pre-commit, Dependabot, workflows, and `.gitignore`. +- Removed non-protected Python sample/source files and Python package metadata. +- Removed non-protected HCL sample files, HCL workflow/configuration, and HCL-specific docs. +- Kept Markdown linting, nested Markdown linting, data-file validation, pre-commit, PowerShell/Pester CI, PSScriptAnalyzer settings, issue templates, PR template, CODEOWNERS, and governance files. +- Kept the worked-example schema and `check-jsonschema` / `check-metaschema` hooks for Phase 0 validation. +- Added required root phase TODO files, including branch-protection and Phase 7 evidence-schema replacement tracking. + +## GETTING_STARTED_NEW_REPO.md Checklist + +- Repository creation: skipped because `franklesniak/macOSLab` already exists. +- Clone/local prerequisite setup: skipped because this work is already in the provided workspace. +- Dependency install: skipped as a manual local-machine prerequisite, except existing Node tooling is retained and lockfile is regenerated. +- Initial placeholder replacement: applied. +- Creating optional labels: skipped in GitHub because the owner is creating `triage`; issue templates assume it will exist and include `triage`. +- Pre-commit configuration: applied; `pre-commit install` skipped because hook installation is a local-machine action. +- Language-specific customization: applied for PowerShell/Markdown; non-protected Python and HCL template artifacts removed. +- Package metadata: applied. +- PR template customization: applied with unconditional pre-commit language and PowerShell-specific checks. +- README customization: applied as a minimal Phase 0 README. +- CONTRIBUTING customization: applied for PowerShell/Markdown and retained tooling. +- CODE_OF_CONDUCT customization: applied. +- Copilot/protected instruction updates: skipped because protected instruction files require explicit owner authorization. +- Validation/testing: run locally as noted in this PR after edits. +- Cleanup: applied for non-protected template artifacts that do not match macOSLab. + +## OPTIONAL_CONFIGURATIONS.md Checklist + +- Issue template configuration: applied; blank issues remain enabled, Discussions link enabled, and security link points to private vulnerability reporting. +- Support/FAQ link: skipped because no concrete support or FAQ page exists yet. +- Pull request template: applied. +- Dependabot configuration: applied; weekly cadence kept, npm, GitHub Actions, and pre-commit ecosystems kept, pip removed. +- Pre-commit configuration: applied; Black, Ruff, and HCL hooks removed, retained data-file and Markdown hooks kept. +- Schema validation configuration: applied; worked example retained and Phase 7 replacement TODO added. +- Markdown linting/workflow: kept. +- Copilot instruction configuration: skipped because protected instruction files require explicit owner authorization. +- CI workflow configuration: applied; Python and HCL CI removed, narrow aggregate pre-commit workflow added for retained tooling hooks. +- Auto-fix pre-commit workflow: kept and updated to match retained hook set. +- Placeholder check workflow: kept. +- PowerShell CI workflow: kept, PowerShell 7.4 floor check added, Pester pinned to `5.7.1`. +- Python template files: removed. +- Pester test template: kept. +- PSScriptAnalyzer configuration: kept. +- CODEOWNERS configuration: applied. +- Node.js package configuration: kept only for Markdown linting tooling. +- Gitignore configuration: applied for macOS, PowerShell/lab artifacts, Parallels, UTM, local evidence/cache, and sensitive file patterns. +- License customization: verified as `Copyright (c) 2026 Frank Lesniak`. +- VS Code PowerShell encoding: preserved while setting `window.title` to `macOSLab`. +- Ongoing maintenance: inherited template maintenance docs removed because they described the old template rather than this repository. + +## Manual Actions Deferred to Owner + +- Create/configure repository from template: already done before this branch. +- Local-machine prerequisites: Git, Node.js, Python for pre-commit tooling, `npm install`, and `pre-commit install`. +- Create the `triage` label before PR review or before issue templates are used. +- Configure branch protection or rulesets for `main`; see `TODO-Phase-00-Branch-Protection.md`. +- Set repository About description and topics. +- Verify GitHub Actions are green after the branch is pushed. + +## Open Questions / Requested Authorization + +- Authorization requested: remove or de-link `.github/instructions/terraform.instructions.md` and remove inherited HCL references from protected instruction entry points. +- Authorization requested: remove or de-link `.github/instructions/python.instructions.md` and remove inherited Python references from protected instruction entry points. +- Authorization requested: update `.github/copilot-instructions.md`, `AGENTS.md`, `CLAUDE.md`, and `GEMINI.md` so their language tables, validation commands, and summaries match PowerShell/Markdown macOSLab. +- Owner decision requested: whether to add the exact ADR-0009 `SECURITY.md` paragraph in Phase 1. + +## Validation + +- `npm run lint:md` - passed. +- `npm run lint:md:nested` - passed. +- `pre-commit run --all-files` - passed. +- `Invoke-ScriptAnalyzer` against repository PowerShell files, excluding ignored dependency/tool caches - passed. +- `Invoke-Pester -Path tests/ -Output Detailed` with Pester `5.7.1` - passed, 2 tests. diff --git a/docs/terraform/TERRAFORM_COPILOT_INSTRUCTIONS_GUIDE.md b/docs/terraform/TERRAFORM_COPILOT_INSTRUCTIONS_GUIDE.md deleted file mode 100644 index be4b005..0000000 --- a/docs/terraform/TERRAFORM_COPILOT_INSTRUCTIONS_GUIDE.md +++ /dev/null @@ -1,1855 +0,0 @@ -# Guide: Writing Terraform Copilot Instructions - -**Version:** 1.0.20260124.0 - -## Metadata - -- **Status:** Active -- **Owner:** Repository Maintainers -- **Last Updated:** 2026-01-24 -- **Scope:** This document provides comprehensive guidance for creating a `.github/instructions/terraform.instructions.md` file that matches the depth, structure, and quality of the PowerShell instructions file (~143KB). It covers best practices, recommendations, rationale, and implementation guidance for Terraform-specific Copilot instructions. -- **Related:** [Repository Copilot Instructions](../../.github/copilot-instructions.md) - -## Purpose and Scope - -This guide serves as a **reference document** for writing a comprehensive `terraform.instructions.md` file. The goal is to enable the creation of an instruction file that: - -- Matches the **comprehensiveness and detail** of the existing PowerShell instructions file -- Follows the **established patterns and conventions** of this repository -- Provides **actionable, specific guidance** for Terraform development -- Integrates seamlessly with the **repository constitution** (`.github/copilot-instructions.md`) -- Includes **embedded testing guidance** for Terraform's native test framework - -This guide is **prescriptive**—it uses RFC 2119 keywords (**MUST**, **SHOULD**, **MAY**, etc.) to indicate requirement levels for the resulting instruction file. - ---- - -## Table of Contents - -- [Document Structure Recommendations](#document-structure-recommendations) -- [YAML Front Matter Specification](#yaml-front-matter-specification) -- [Required Sections and Content](#required-sections-and-content) -- [Terraform Best Practices Research](#terraform-best-practices-research) -- [Coding Standards Recommendations](#coding-standards-recommendations) -- [Mode Distinctions: Scope Tags](#mode-distinctions-scope-tags) -- [Security Considerations](#security-considerations) -- [Testing with Terraform Test](#testing-with-terraform-test) -- [Documentation Standards](#documentation-standards) -- [Integration with Repository Constitution](#integration-with-repository-constitution) -- [Tooling and Pre-commit Configuration](#tooling-and-pre-commit-configuration) -- [Implementation Checklist](#implementation-checklist) - ---- - -## Document Structure Recommendations - -The `terraform.instructions.md` file **MUST** follow the structural pattern established by the PowerShell and Python instruction files. This section provides the recommended document skeleton. - -### Required Document Skeleton - -The instruction file **MUST** include the following sections in this order: - -````markdown ---- -applyTo: "**/*.tf,**/*.tfvars,**/*.tftest.hcl" -description: "Terraform coding standards: secure, modular, and well-documented infrastructure as code." ---- - -# Terraform Writing Style - -**Version:** 1.0.YYYYMMDD.0 - -## Metadata - -- **Status:** Active -- **Owner:** Repository Maintainers -- **Last Updated:** YYYY-MM-DD -- **Scope:** [Define scope] -- **Related:** [Repository Copilot Instructions](../copilot-instructions.md) - -## Table of Contents - -[Full navigation links to all major sections] - -## Keywords - -[RFC 2119 keyword definitions] - -## Quick Reference Checklist - -[Comprehensive checklist with scope tags and internal links] - -## Executive Summary: Philosophy and Approach - -[Overall Terraform philosophy for this repository] - -## [Detailed Coding Standards Sections] - -[Multiple deep-dive sections organized by topic] - -## Testing with Terraform Test - -[Embedded testing guidance - NOT just a reference] - -## "Done" Definition for Terraform Changes - -[What constitutes a complete change] -```` - -### Section Count and Depth - -Based on the PowerShell instructions (~143KB, ~2800 lines), the Terraform instruction file **SHOULD** include: - -- **At least 12-15 major sections** (H2 headers) -- **40-60 subsections** (H3 headers) -- **Comprehensive code examples** throughout -- **Tables for structured information** (naming conventions, options, comparisons) -- **A Quick Reference Checklist** with **40+ items** organized by category - ---- - -## YAML Front Matter Specification - -### Required YAML Front Matter - -The instruction file **MUST** begin with YAML front matter specifying file patterns and description: - -```yaml ---- -applyTo: "**/*.tf,**/*.tfvars,**/*.tftest.hcl" -description: "Terraform coding standards: secure, modular, and well-documented infrastructure as code." ---- -``` - -### Complete File Pattern Coverage - -The `applyTo` field **SHOULD** cover all Terraform-related file types. Consider the following patterns: - -| File Type | Pattern | Description | -| --- | --- | --- | -| Configuration files | `**/*.tf` | Main Terraform configuration files | -| Variable values | `**/*.tfvars` | Variable definition files | -| Auto-loaded variables | `**/*.auto.tfvars` | Auto-loaded variable files (subset of tfvars) | -| Test files | `**/*.tftest.hcl` | Native Terraform test files | -| JSON configuration | `**/*.tf.json` | JSON-format Terraform configuration | -| Template files | `**/*.tftpl` | Terraform template files | -| Backend config | `**/*.tfbackend` | Backend configuration files | - -**Recommended comprehensive pattern:** - -```yaml -applyTo: "**/*.tf,**/*.tfvars,**/*.tftest.hcl,**/*.tf.json,**/*.tftpl,**/*.tfbackend" -``` - -**Note:** The pattern syntax uses comma separation without spaces. Verify the pattern syntax is valid for the Copilot instruction file loader. - -### Alternative: Multiple Instruction Files - -For complex repositories, consider whether separate instruction files for different Terraform contexts are warranted: - -| File | Pattern | Use Case | -| --- | --- | --- | -| `terraform.instructions.md` | `**/*.tf,**/*.tfvars` | Core Terraform configuration | -| `terraform-test.instructions.md` | `**/*.tftest.hcl` | Testing-specific guidance | -| `terraform-modules.instructions.md` | `modules/**/*.tf` | Module development | - -**Recommendation:** Start with a single comprehensive file. Split only if the file becomes unwieldy (>200KB) or if distinct contexts require significantly different guidance. - ---- - -## Required Sections and Content - -This section details what **MUST** be included in each major section of the instruction file. - -### 1. Title and Version - -```text -# Terraform Writing Style - -**Version:** 1.0.20260124.0 -``` - -**Version format:** `Major.Minor.YYYYMMDD.Revision` - -- **Major:** Increment for breaking changes to coding standards -- **Minor:** Increment for non-breaking additions/enhancements -- **YYYYMMDD:** Current date of modification -- **Revision:** Increment for same-day modifications - -### 2. Metadata Section - -**MUST** include: - -```text -## Metadata - -- **Status:** Active -- **Owner:** Repository Maintainers -- **Last Updated:** 2026-01-24 -- **Scope:** Defines Terraform coding standards for all `.tf`, `.tfvars`, `.tftest.hcl`, and related files in this repository. Covers style, formatting, naming conventions, module design, security, testing, and documentation requirements. -- **Related:** [Repository Copilot Instructions](../copilot-instructions.md) -``` - -### 3. Table of Contents - -**MUST** include a comprehensive table of contents with anchor links to all major sections. Example: - -```text -## Table of Contents - -- [Keywords](#keywords) -- [Quick Reference Checklist](#quick-reference-checklist) -- [Executive Summary: Terraform Philosophy](#executive-summary-terraform-philosophy) -- [Formatting and Style](#formatting-and-style) -- [Naming Conventions](#naming-conventions) -- [File Organization](#file-organization) -- [Variable and Output Design](#variable-and-output-design) -- [Resource Configuration](#resource-configuration) -- [Module Design](#module-design) -- [State Management](#state-management) -- [Security Best Practices](#security-best-practices) -- [Provider Management](#provider-management) -- [Testing with Terraform Test](#testing-with-terraform-test) -- [Documentation Standards](#documentation-standards) -- ["Done" Definition for Terraform Changes](#done-definition-for-terraform-changes) -``` - -### 4. Keywords Section - -**MUST** copy the RFC 2119 keyword definitions pattern from the PowerShell instructions: - -```text -## Keywords - -The key words "**MUST**", "**MUST NOT**", "**REQUIRED**", "**SHALL**", "**SHALL NOT**", "**SHOULD**", "**SHOULD NOT**", "**RECOMMENDED**", "**MAY**", and "**OPTIONAL**" in this document are to be interpreted as described in [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119). - -- **MUST** / **REQUIRED** / **SHALL** — Absolute requirement. Non-negotiable. -- **MUST NOT** / **SHALL NOT** — Absolute prohibition. -- **SHOULD** / **RECOMMENDED** — Strong recommendation. Valid reasons may exist to deviate, but implications must be understood. -- **SHOULD NOT** / **NOT RECOMMENDED** — Strong discouragement. Valid reasons may exist to do otherwise, but implications must be understood. -- **MAY** / **OPTIONAL** — Truly optional. Implementations can choose to include or omit. -``` - -### 5. Quick Reference Checklist - -**MUST** include a comprehensive checklist following the PowerShell pattern. Each item **MUST** include: - -1. **Scope tag** in bold brackets (e.g., `[All]`, `[Module]`, `[Root]`, `[Test]`) -2. **RFC 2119 keyword** in bold (e.g., **MUST**, **SHOULD**) -3. **Concise requirement statement** -4. **Link to detailed section** using `→ [Section Name](#anchor)` - -Example structure: - -```text -## Quick Reference Checklist - -This checklist provides a quick reference for both human developers and LLMs (like GitHub Copilot) to follow the Terraform style guidelines. Each item includes a scope tag indicating applicability: - -- **[All]** — Applies to all Terraform files -- **[Module]** — Applies when developing reusable modules -- **[Root]** — Applies to root configurations (deployments) -- **[Test]** — Applies to test files (`.tftest.hcl`) - -### Formatting and Style - -- **[All]** Code **MUST** pass `terraform fmt` without modifications → [Formatting Standards](#formatting-standards) -- **[All]** Code **MUST** use 2 spaces for indentation, never tabs → [Indentation Rules](#indentation-rules) -- **[All]** Files **MUST** use UTF-8 encoding → [File Encoding](#file-encoding) -- **[All]** Files **MUST** end with a single newline → [File Endings](#file-endings) -- **[All]** Lines **SHOULD NOT** exceed 120 characters → [Line Length](#line-length) - -### Naming Conventions - -- **[All]** Resources **MUST** use snake_case names → [Resource Naming](#resource-naming) -- **[All]** Variables **MUST** use snake_case with descriptive names → [Variable Naming](#variable-naming) -- **[All]** Outputs **MUST** use snake_case matching resource attribute patterns → [Output Naming](#output-naming) -- **[Module]** Module names **MUST** use hyphen-separated lowercase words → [Module Naming](#module-naming) -- **[All]** Data sources **MUST** be prefixed with purpose when multiple exist → [Data Source Naming](#data-source-naming) - -[Continue for all categories...] -``` - -### 6. Executive Summary - -**MUST** describe the overall Terraform philosophy and approach for this repository: - -```text -## Executive Summary: Terraform Philosophy - -This repository approaches Terraform as **infrastructure as code** with the same rigor applied to application code: - -- **Deterministic and reproducible:** Infrastructure changes **MUST** produce predictable, repeatable results -- **Security-first:** Secrets **MUST NEVER** appear in code or state; least-privilege **MUST** be the default -- **Modular and reusable:** Common patterns **SHOULD** be extracted into versioned modules -- **Well-documented:** Every variable, output, and module **MUST** be documented -- **Testable:** Infrastructure **SHOULD** be validated with automated tests before deployment - -The coding standards in this document enforce these principles through specific, actionable requirements. -``` - ---- - -## Terraform Best Practices Research - -This section documents the Terraform best practices that **SHOULD** be incorporated into the instruction file. These are based on widely accepted HashiCorp conventions and industry standards as of Terraform 1.6+. - -### HashiCorp Style Conventions - -HashiCorp's official style conventions include: - -1. **Indentation:** 2 spaces (not tabs) -2. **Alignment:** Use `terraform fmt` for consistent alignment -3. **Ordering:** Within resource blocks, follow `meta-arguments → arguments → nested blocks` -4. **Comments:** Use `#` for single-line comments; `/* */` for multi-line (rarely needed) - -### File Organization Standards - -Standard file organization for Terraform projects: - -| File | Purpose | Required | -| --- | --- | --- | -| `main.tf` | Primary resource definitions | Yes | -| `variables.tf` | Input variable declarations | Yes | -| `outputs.tf` | Output value declarations | Yes | -| `providers.tf` | Provider configuration | Yes (root modules) | -| `versions.tf` or `terraform.tf` | Version constraints | Yes | -| `locals.tf` | Local value definitions | When needed | -| `data.tf` | Data source definitions | When needed | -| `backend.tf` | Backend configuration | Root modules only | - -### Module Structure - -Standard module directory structure: - -```text -modules/ -└── / - ├── main.tf # Primary resources - ├── variables.tf # Input variables (REQUIRED) - ├── outputs.tf # Output values (REQUIRED) - ├── versions.tf # Version constraints (REQUIRED) - ├── README.md # Module documentation (REQUIRED) - ├── examples/ # Usage examples (RECOMMENDED) - │ └── basic/ - │ ├── main.tf - │ └── outputs.tf - └── tests/ # Test files (RECOMMENDED) - └── basic.tftest.hcl -``` - -### Version Constraint Best Practices - -Provider version constraints **SHOULD** follow these patterns: - -| Pattern | Example | Use Case | -| --- | --- | --- | -| Pessimistic constraint | `~> 5.0` | Allow minor version updates only | -| Exact version | `= 5.31.0` | Strict reproducibility required | -| Range constraint | `>= 5.0, < 6.0` | Explicit major version bounds | - -**Recommended approach:** - -```hcl -terraform { - required_version = ">= 1.6.0" - - required_providers { - aws = { - source = "hashicorp/aws" - version = "~> 5.0" - } - } -} -``` - -### Lock File Management - -The `.terraform.lock.hcl` file: - -- **MUST** be committed to version control -- **SHOULD** be updated explicitly using `terraform providers lock` -- **MUST** be updated when provider versions change -- **SHOULD** include hashes for all platforms in CI (`-platform=linux_amd64 -platform=darwin_amd64`) - ---- - -## Coding Standards Recommendations - -This section provides detailed recommendations for each coding standard area. - -### Formatting Standards - -#### terraform fmt Compliance - -```text -### Formatting Standards - -#### terraform fmt Compliance - -All Terraform code **MUST** pass `terraform fmt` without modifications. This is non-negotiable. - -**Verification command:** - -\`\`\`bash -terraform fmt -check -recursive -\`\`\` - -**Auto-format command:** - -\`\`\`bash -terraform fmt -recursive -\`\`\` - -**Pre-commit integration:** - -\`\`\`yaml -- repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.105.0 - hooks: - - id: terraform_fmt -\`\`\` -``` - -#### Indentation Rules - -```text -#### Indentation Rules - -- Code **MUST** use 2 spaces for indentation -- Tabs **MUST NOT** be used -- Nested blocks **MUST** maintain consistent indentation - -**Compliant:** - -\`\`\`hcl -resource "aws_instance" "example" { - ami = var.ami_id - instance_type = var.instance_type - - tags = { - Name = var.instance_name - Environment = var.environment - } -} -\`\`\` -``` - -### Naming Conventions - -The instruction file **MUST** include comprehensive naming convention tables similar to PowerShell's approved verbs. - -#### Resource Naming - -```text -### Resource Naming Conventions - -Resources **MUST** use `snake_case` for names. Names **SHOULD** be descriptive and indicate purpose. - -| Resource Type | Naming Pattern | Example | -| --- | --- | --- | -| Primary/main resource | `main` or descriptive name | `aws_instance.main` | -| Multiple of same type | Purpose-based suffix | `aws_instance.web_server` | -| Associated resources | Parent reference | `aws_security_group.web_server` | - -**Anti-patterns to avoid:** - -| Bad | Good | Reason | -| --- | --- | --- | -| `aws_instance.this` | `aws_instance.main` | "this" is not descriptive | -| `aws_instance.instance1` | `aws_instance.primary` | Numeric suffixes are meaningless | -| `aws_instance.MyInstance` | `aws_instance.my_instance` | Must be snake_case | -``` - -#### Variable Naming - -```text -### Variable Naming Conventions - -Variables **MUST** use `snake_case` and **MUST** be descriptive. - -| Category | Pattern | Example | -| --- | --- | --- | -| Simple values | `` or `_` | `instance_type`, `environment` | -| Lists/Sets | Plural nouns | `subnet_ids`, `security_group_ids` | -| Maps | `_map` or descriptive | `tags`, `instance_settings` | -| Booleans | `enable_*`, `is_*`, `has_*` | `enable_monitoring`, `is_public` | -| Resource references | `_id` or `_arn` | `vpc_id`, `role_arn` | - -**Compliant variable names:** - -\`\`\`hcl -variable "environment" { - description = "Deployment environment (dev, staging, prod)" - type = string -} - -variable "enable_monitoring" { - description = "Enable CloudWatch detailed monitoring" - type = bool - default = false -} - -variable "subnet_ids" { - description = "List of subnet IDs for deployment" - type = list(string) -} -\`\`\` -``` - -#### Output Naming - -```text -### Output Naming Conventions - -Outputs **MUST** use `snake_case` and **SHOULD** follow the pattern of the attribute being exposed. - -| Output Type | Pattern | Example | -| --- | --- | --- | -| Resource ID | `_id` | `instance_id`, `vpc_id` | -| Resource ARN | `_arn` | `role_arn`, `bucket_arn` | -| Resource name | `_name` | `bucket_name`, `cluster_name` | -| Endpoints/URLs | `_endpoint` | `rds_endpoint`, `api_endpoint` | -| Collections | Plural form | `instance_ids`, `subnet_ids` | -``` - -### Variable and Output Design - -#### Variable Documentation Requirements - -```text -### Variable Documentation Requirements - -Every variable **MUST** include: - -1. **description** (REQUIRED) — Human-readable explanation -2. **type** (REQUIRED) — Explicit type constraint -3. **default** (CONDITIONAL) — Required for optional variables -4. **validation** (RECOMMENDED) — For constrained inputs -5. **sensitive** (CONDITIONAL) — Required for secrets - -**Minimal compliant variable:** - -\`\`\`hcl -variable "environment" { - description = "Deployment environment. Valid values: dev, staging, prod." - type = string - - validation { - condition = contains(["dev", "staging", "prod"], var.environment) - error_message = "Environment must be dev, staging, or prod." - } -} -\`\`\` - -**Variable with default:** - -\`\`\`hcl -variable "instance_type" { - description = "EC2 instance type for the application server." - type = string - default = "t3.micro" -} -\`\`\` - -**Sensitive variable:** - -\`\`\`hcl -variable "database_password" { - description = "Password for the RDS database. Must be provided via environment variable or tfvars." - type = string - sensitive = true -} -\`\`\` -``` - -#### Output Documentation Requirements - -```text -### Output Documentation Requirements - -Every output **MUST** include a description. Sensitive outputs **MUST** be marked. - -\`\`\`hcl -output "instance_id" { - description = "The ID of the created EC2 instance." - value = aws_instance.main.id -} - -output "instance_private_ip" { - description = "The private IP address of the EC2 instance." - value = aws_instance.main.private_ip - sensitive = true # If this should be protected -} -\`\`\` -``` - -### Resource Configuration - -#### Meta-Argument Ordering - -```text -### Meta-Argument Ordering - -Within resource blocks, arguments **MUST** follow this order: - -1. **Meta-arguments first:** `count`, `for_each`, `provider`, `depends_on`, `lifecycle` -2. **Required arguments:** Arguments without defaults -3. **Optional arguments:** Arguments with defaults -4. **Nested blocks last:** Dynamic blocks, inline blocks - -**Compliant example:** - -\`\`\`hcl -resource "aws_instance" "web_server" { - # Meta-arguments first - count = var.instance_count - provider = aws.primary - - # Required arguments - ami = data.aws_ami.amazon_linux.id - instance_type = var.instance_type - subnet_id = var.subnet_id - - # Optional arguments - associate_public_ip_address = var.is_public - monitoring = var.enable_monitoring - - # Nested blocks last - root_block_device { - volume_size = var.root_volume_size - encrypted = true - } - - tags = local.common_tags - - # Lifecycle block at the end - lifecycle { - create_before_destroy = true - } -} -\`\`\` -``` - -#### Resource Tagging Standards - -```text -### Resource Tagging Standards - -#### Required Tags - -All taggable resources **MUST** include these tags: - -| Tag | Description | Example | -| --- | --- | --- | -| `Name` | Human-readable resource name | `prod-web-server-1` | -| `Environment` | Deployment environment | `prod`, `staging`, `dev` | -| `Project` | Project or application name | `my-application` | -| `ManagedBy` | Management method | `terraform` | -| `Owner` | Team or individual owner | `platform-team` | - -#### Default Tags Configuration - -Root modules **SHOULD** use provider-level default tags: - -\`\`\`hcl -provider "aws" { - region = var.aws_region - - default_tags { - tags = { - Environment = var.environment - Project = var.project_name - ManagedBy = "terraform" - Owner = var.owner_team - } - } -} -\`\`\` - -#### Local Tags Pattern - -Use locals for computed or merged tags: - -\`\`\`hcl -locals { - common_tags = { - Name = "${var.project_name}-${var.environment}" - Environment = var.environment - Project = var.project_name - ManagedBy = "terraform" - } -} -\`\`\` -``` - -### Module Design - -```text -### Module Design Principles - -#### Reusability Guidelines - -Modules **SHOULD** be designed for reuse: - -- **Single responsibility:** Each module **SHOULD** manage one logical component -- **Minimal coupling:** Modules **SHOULD** have minimal dependencies on each other -- **Explicit interface:** All inputs and outputs **MUST** be documented -- **Sensible defaults:** Optional variables **SHOULD** have reasonable defaults -- **Version constraints:** Modules **MUST** specify required Terraform and provider versions - -#### Module Interface Design - -**Inputs:** - -- Variable names **MUST** be consistent across modules (e.g., always `environment`, not sometimes `env`) -- Required variables **SHOULD** be minimized to essential values -- Complex inputs **SHOULD** use object types with documented structure - -**Outputs:** - -- Expose only values needed by calling modules -- Use consistent naming patterns across modules -- Document output types and formats - -#### Module Versioning - -For published modules: - -- Use semantic versioning (e.g., `v1.2.3`) -- Document breaking changes in CHANGELOG -- Pin module versions in root configurations - -\`\`\`hcl -module "vpc" { - source = "terraform-aws-modules/vpc/aws" - version = "~> 5.0" - - # ... -} -\`\`\` -``` - -### State Management - -```text -### State Management - -#### Backend Configuration - -Root modules **MUST** configure a remote backend for team environments: - -\`\`\`hcl -terraform { - backend "s3" { - bucket = "my-terraform-state" - key = "environments/prod/terraform.tfstate" - region = "us-east-1" - encrypt = true - dynamodb_table = "terraform-state-lock" - } -} -\`\`\` - -**Backend requirements:** - -- State files **MUST** be encrypted at rest -- State access **MUST** be controlled via IAM -- State locking **MUST** be enabled to prevent concurrent modifications -- State files **MUST NOT** be committed to version control - -#### State File Organization - -Organize state files by environment and component: - -\`\`\`text -state-bucket/ -├── environments/ -│ ├── dev/ -│ │ └── terraform.tfstate -│ ├── staging/ -│ │ └── terraform.tfstate -│ └── prod/ -│ └── terraform.tfstate -└── shared/ - ├── networking/ - │ └── terraform.tfstate - └── iam/ - └── terraform.tfstate -\`\`\` - -#### Workspace Usage - -Workspaces **MAY** be used for environment separation in simple cases: - -\`\`\`bash -terraform workspace select prod -terraform apply -\`\`\` - -**Caution:** For complex environments, separate state files per environment are often clearer than workspaces. -``` - ---- - -## Mode Distinctions: Scope Tags - -The instruction file **SHOULD** define distinct scope tags to indicate when rules apply. Based on Terraform's structure, the following tags are recommended: - -### Recommended Scope Tags - -| Tag | Applies To | Description | -| --- | --- | --- | -| `[All]` | All Terraform files | Universal rules applicable everywhere | -| `[Module]` | Reusable modules | Rules for developing modules in `modules/` | -| `[Root]` | Root configurations | Rules for deployment configurations | -| `[Test]` | Test files | Rules for `.tftest.hcl` files | - -### Tag Usage Examples - -```text -### Quick Reference Checklist - -#### File Organization - -- **[All]** Every Terraform directory **MUST** have a `versions.tf` file → [Version Constraints](#version-constraints) -- **[Module]** Modules **MUST** include a `README.md` → [Module Documentation](#module-documentation) -- **[Root]** Root modules **MUST** configure a remote backend → [Backend Configuration](#backend-configuration) -- **[Test]** Test files **MUST** use the `.tftest.hcl` extension → [Test File Naming](#test-file-naming) - -#### Security - -- **[All]** Secrets **MUST NOT** appear in `.tf` files → [Secret Management](#secret-management) -- **[Root]** State backends **MUST** enable encryption → [State Security](#state-security) -- **[Module]** Sensitive outputs **MUST** be marked with `sensitive = true` → [Sensitive Data](#sensitive-data) -``` - -### Rationale for Tag Selection - -- **[All]** vs specific tags: Use `[All]` for universal formatting, naming, and documentation rules that apply regardless of context. -- **[Module]** vs **[Root]**: Modules have different requirements (e.g., no backend config) than root deployments (e.g., must have backend config). -- **[Test]**: Test files have unique syntax and patterns that warrant separate guidance. - ---- - -## Security Considerations - -This section outlines the security content that **MUST** be included in the instruction file. - -### Secrets and Sensitive Variable Handling - -```text -## Security Best Practices - -### Secret Management - -Secrets **MUST NEVER** appear in Terraform code or version control. - -#### Prohibited Patterns - -The following patterns are **PROHIBITED**: - -\`\`\`hcl -# NEVER DO THIS -variable "db_password" { - default = "SuperSecretPassword123!" # PROHIBITED -} - -resource "aws_db_instance" "main" { - password = "hardcoded-password" # PROHIBITED -} -\`\`\` - -#### Approved Secret Patterns - -**Pattern 1: Environment Variables** - -\`\`\`hcl -variable "db_password" { - description = "Database password. Set via TF_VAR_db_password environment variable." - type = string - sensitive = true -} -\`\`\` - -\`\`\`bash -export TF_VAR_db_password="$(aws secretsmanager get-secret-value ...)" -terraform apply -\`\`\` - -**Pattern 2: Secret Manager Integration** - -\`\`\`hcl -data "aws_secretsmanager_secret_version" "db_password" { - secret_id = "prod/database/password" -} - -resource "aws_db_instance" "main" { - password = data.aws_secretsmanager_secret_version.db_password.secret_string -} -\`\`\` - -**Pattern 3: HashiCorp Vault** - -\`\`\`hcl -data "vault_generic_secret" "db_creds" { - path = "secret/data/database" -} - -resource "aws_db_instance" "main" { - username = data.vault_generic_secret.db_creds.data["username"] - password = data.vault_generic_secret.db_creds.data["password"] -} -\`\`\` - -### Sensitive Variable Marking - -Variables containing sensitive data **MUST** be marked: - -\`\`\`hcl -variable "api_key" { - description = "API key for external service" - type = string - sensitive = true # REQUIRED for secrets -} - -output "connection_string" { - description = "Database connection string (contains credentials)" - value = local.connection_string - sensitive = true # REQUIRED if contains secrets -} -\`\`\` -``` - -### State File Security - -```text -### State File Security - -State files contain sensitive data and **MUST** be protected. - -#### Requirements - -1. **Encryption at rest:** State backends **MUST** enable encryption -2. **Access control:** State files **MUST** be accessible only to authorized users/roles -3. **No local state in production:** Local state files **MUST NOT** be used for production -4. **State locking:** Backends **MUST** support state locking - -#### S3 Backend Security Configuration - -\`\`\`hcl -terraform { - backend "s3" { - bucket = "my-terraform-state" - key = "prod/terraform.tfstate" - region = "us-east-1" - encrypt = true # REQUIRED - dynamodb_table = "terraform-state-lock" # REQUIRED for locking - # Use IAM role or credentials from environment - } -} -\`\`\` - -#### State Bucket Policy Example - -\`\`\`json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Deny", - "Principal": "*", - "Action": "s3:*", - "Resource": [ - "arn:aws:s3:::my-terraform-state", - "arn:aws:s3:::my-terraform-state/*" - ], - "Condition": { - "Bool": { - "aws:SecureTransport": "false" - } - } - } - ] -} -\`\`\` -``` - -### Least-Privilege Resource Policies - -```text -### Least-Privilege Principles - -IAM policies and resource permissions **MUST** follow least-privilege: - -#### IAM Policy Guidelines - -- Grant only required permissions -- Use resource-level restrictions when possible -- Avoid wildcard actions (`*`) except when truly needed -- Use conditions to further restrict access - -\`\`\`hcl -# GOOD: Specific permissions -resource "aws_iam_policy" "s3_reader" { - name = "s3-reader" - - policy = jsonencode({ - Version = "2012-10-17" - Statement = [ - { - Effect = "Allow" - Action = [ - "s3:GetObject", - "s3:ListBucket" - ] - Resource = [ - aws_s3_bucket.data.arn, - "${aws_s3_bucket.data.arn}/*" - ] - } - ] - }) -} - -# BAD: Overly permissive -resource "aws_iam_policy" "bad_example" { - policy = jsonencode({ - Statement = [{ - Effect = "Allow" - Action = ["s3:*"] # TOO BROAD - Resource = ["*"] # TOO BROAD - }] - }) -} -\`\`\` -``` - -### Security Scanning Integration - -```text -### Security Scanning - -Security scanning tools **SHOULD** be integrated into the development workflow. - -#### Recommended Tools - -| Tool | Purpose | Integration | -| --- | --- | --- | -| `tfsec` | Static security analysis | Pre-commit, CI | -| `checkov` | Policy-as-code scanning | Pre-commit, CI | -| `terrascan` | Security and compliance | CI | -| `trivy` | Misconfiguration scanning | CI | - -#### Pre-commit Integration Example - -\`\`\`yaml -- repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.105.0 - hooks: - - id: terraform_tfsec - - id: terraform_checkov -\`\`\` - -**Note:** Security scanning is **RECOMMENDED** but specific tool selection depends on project requirements. The instruction file **SHOULD** document which tools are expected for the repository. -``` - ---- - -## Testing with Terraform Test - -Following the PowerShell pattern, testing guidance **MUST** be embedded directly in the instruction file. This section provides the content that **SHOULD** be included. - -### Overview - -```text -## Testing with Terraform Test - -Terraform's native test framework (introduced in Terraform 1.6) provides a way to validate configurations without external testing tools. This section documents testing conventions that integrate with the coding standards in this guide. - -> **Note:** Terraform tests require Terraform 1.6.0 or later. For older Terraform versions, consider Terratest or other external testing frameworks. -``` - -### Test File Naming and Location - -```text -### Test File Naming and Location - -Test files **MUST** follow consistent naming conventions: - -- **File extension:** Test files **MUST** use `.tftest.hcl` extension -- **Location:** Test files **SHOULD** be placed in a `tests/` directory within the module -- **Naming:** Test files **SHOULD** be named descriptively based on what they test - -**Module with tests:** - -\`\`\`text -modules/ -└── vpc/ - ├── main.tf - ├── variables.tf - ├── outputs.tf - ├── versions.tf - ├── README.md - └── tests/ - ├── basic.tftest.hcl - ├── validation.tftest.hcl - └── integration.tftest.hcl -\`\`\` - -**Alternative: Tests alongside configuration:** - -\`\`\`text -modules/ -└── vpc/ - ├── main.tf - ├── variables.tf - ├── outputs.tf - ├── vpc.tftest.hcl - └── vpc_validation.tftest.hcl -\`\`\` -``` - -### Test File Structure - -```text -### Test File Structure - -Terraform test files use HCL syntax with specific blocks. - -#### Basic Test Structure - -\`\`\`hcl -# tests/basic.tftest.hcl - -# Variables block (optional) - Set values for tests -variables { - environment = "test" - vpc_cidr = "10.0.0.0/16" -} - -# Run block - Defines a test scenario -run "creates_vpc_with_correct_cidr" { - command = plan # or apply - - assert { - condition = aws_vpc.main.cidr_block == "10.0.0.0/16" - error_message = "VPC CIDR block does not match expected value" - } -} - -run "creates_required_subnets" { - command = plan - - assert { - condition = length(aws_subnet.private) == 3 - error_message = "Expected 3 private subnets" - } -} -\`\`\` - -#### Test Block Reference - -| Block | Purpose | Required | -| --- | --- | --- | -| `variables {}` | Set input variable values for tests | Optional | -| `provider {}` | Configure provider for tests (e.g., mock) | Optional | -| `run "name" {}` | Define a test scenario | Required (at least one) | -| `assert {}` | Define a test assertion within a run | Required (at least one per run) | -| `expect_failures` | Expect specific resources/outputs to fail | Optional | -``` - -### Test Patterns - -```text -### Test Patterns - -#### Arrange-Act-Assert in Terraform Tests - -Tests **SHOULD** follow a logical structure similar to AAA: - -\`\`\`hcl -# tests/instance.tftest.hcl - -# Arrange: Set up test inputs -variables { - instance_type = "t3.micro" - environment = "test" -} - -# Act & Assert: Run terraform and check results -run "instance_has_correct_type" { - command = plan - - # Assert - assert { - condition = aws_instance.main.instance_type == "t3.micro" - error_message = "Instance type should be t3.micro" - } -} - -run "instance_has_required_tags" { - command = plan - - assert { - condition = aws_instance.main.tags["Environment"] == "test" - error_message = "Instance must have Environment tag" - } - - assert { - condition = aws_instance.main.tags["ManagedBy"] == "terraform" - error_message = "Instance must have ManagedBy tag" - } -} -\`\`\` - -#### Testing Variable Validation - -\`\`\`hcl -# tests/validation.tftest.hcl - -run "rejects_invalid_environment" { - command = plan - - variables { - environment = "invalid" # Should fail validation - } - - expect_failures = [ - var.environment - ] -} - -run "accepts_valid_environment" { - command = plan - - variables { - environment = "prod" - } - - # No expect_failures - should succeed - assert { - condition = var.environment == "prod" - error_message = "Environment should be prod" - } -} -\`\`\` - -#### Testing Outputs - -\`\`\`hcl -# tests/outputs.tftest.hcl - -run "outputs_vpc_id" { - command = apply - - assert { - condition = output.vpc_id != null && output.vpc_id != "" - error_message = "vpc_id output must not be empty" - } -} - -run "outputs_correct_subnet_count" { - command = apply - - assert { - condition = length(output.subnet_ids) == 3 - error_message = "Expected 3 subnet IDs in output" - } -} -\`\`\` -``` - -### Mock Providers - -```text -### Mock Providers - -For unit testing without real infrastructure, use mock providers: - -\`\`\`hcl -# tests/unit.tftest.hcl - -mock_provider "aws" { - mock_data "aws_availability_zones" { - defaults = { - names = ["us-east-1a", "us-east-1b", "us-east-1c"] - } - } -} - -run "uses_all_availability_zones" { - command = plan - - assert { - condition = length(aws_subnet.private) == 3 - error_message = "Should create subnet in each AZ" - } -} -\`\`\` -``` - -### Unit vs Integration Tests - -```text -### Unit vs Integration Tests - -#### Unit Tests (command = plan) - -- Use `command = plan` (default) -- Do NOT create real resources -- Fast execution -- Test configuration logic, not cloud provider behavior - -\`\`\`hcl -run "unit_test_example" { - command = plan # Explicit, though it's the default - - assert { - condition = aws_instance.main.instance_type == var.instance_type - error_message = "Instance type mismatch" - } -} -\`\`\` - -#### Integration Tests (command = apply) - -- Use `command = apply` -- Create and destroy real resources -- Slower execution -- Test actual infrastructure behavior -- Require valid provider credentials - -\`\`\`hcl -run "integration_test_example" { - command = apply # Creates real resources - - assert { - condition = aws_instance.main.id != "" - error_message = "Instance should be created" - } -} -\`\`\` - -**Best Practice:** Run unit tests (`plan`) frequently during development. Run integration tests (`apply`) in CI or before releases. -``` - -### Running Tests - -```text -### Running Tests - -#### Basic Test Execution - -\`\`\`bash -# Run all tests in current directory -terraform test - -# Run tests with verbose output -terraform test -verbose - -# Run specific test file -terraform test -filter=tests/basic.tftest.hcl -\`\`\` - -#### CI Integration Example - -\`\`\`yaml -# .github/workflows/terraform-ci.yml -jobs: - test: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: hashicorp/setup-terraform@v3 - with: - terraform_version: "1.6.0" - - - name: Terraform Init - run: terraform init - - - name: Terraform Test - run: terraform test -verbose -\`\`\` -``` - -### What to Test - -```text -### What to Test - -Tests **SHOULD** cover: - -| Category | What to Test | Example | -| --- | --- | --- | -| **Variable validation** | Custom validation rules work correctly | Invalid environment rejected | -| **Computed values** | Locals and expressions compute correctly | CIDR calculations | -| **Resource configuration** | Resources have expected attributes | Instance type, tags | -| **Output values** | Outputs contain expected values | VPC ID not empty | -| **Module integration** | Modules work together | VPC + subnets + security groups | -| **Edge cases** | Boundary conditions | Zero instances, empty lists | - -**Not to test:** - -- Cloud provider behavior (e.g., "does AWS actually create an EC2?") -- Terraform core functionality -- External service availability -``` - ---- - -## Documentation Standards - -This section defines the documentation requirements for the instruction file. - -### README Requirements for Modules - -```text -### Module Documentation - -Every module **MUST** include a `README.md` with: - -1. **Description** — What the module does -2. **Usage example** — Minimal working example -3. **Requirements** — Terraform and provider versions -4. **Inputs** — All variables with descriptions -5. **Outputs** — All outputs with descriptions - -#### README Template - -\`\`\`markdown -# Module Name - -Brief description of what this module creates. - -## Usage - -\`\`\`hcl -module "example" { - source = "./modules/example" - - required_variable = "value" -} -\`\`\` - -## Requirements - -| Name | Version | -| --- | --- | -| terraform | >= 1.6.0 | -| aws | ~> 5.0 | - -## Inputs - -| Name | Description | Type | Default | Required | -| --- | --- | --- | --- | --- | -| required_variable | Description here | `string` | n/a | yes | -| optional_variable | Description here | `string` | `"default"` | no | - -## Outputs - -| Name | Description | -| --- | --- | -| output_name | Description of output | -\`\`\` - -**Automation:** Consider using `terraform-docs` to generate input/output tables automatically. -``` - -### Inline Comment Conventions - -```text -### Inline Comments - -Comments **SHOULD** explain "why," not "what." - -#### Single-Line Comments - -Use `#` for single-line comments: - -\`\`\`hcl -# Enable encryption to meet compliance requirements (SOC2) -resource "aws_s3_bucket_server_side_encryption_configuration" "main" { - bucket = aws_s3_bucket.main.id - # ... -} -\`\`\` - -#### Block Comments - -Use `/* */` sparingly for multi-line explanations: - -\`\`\`hcl -/* - * This security group allows inbound traffic from the corporate VPN. - * CIDR ranges are managed by the network team and should not be - * modified without their approval. - */ -resource "aws_security_group" "vpn_access" { - # ... -} -\`\`\` - -#### TODO Comments - -Use standardized TODO format: - -\`\`\`hcl -# TODO(username): Migrate to new VPC module after v2.0 release -\`\`\` -``` - ---- - -## Integration with Repository Constitution - -The instruction file **MUST** integrate with the repository's `.github/copilot-instructions.md`. - -### Referencing the Constitution - -```text -## Related - -- **[Repository Copilot Instructions](../copilot-instructions.md)** — Repo-wide constitution. If any instruction in this file conflicts with the constitution, **the constitution wins**. -``` - -### Pre-commit Discipline - -The instruction file **MUST** emphasize pre-commit discipline consistent with the constitution: - -```text -### Pre-commit Discipline for Terraform - -**⚠️ ALWAYS run pre-commit checks before committing Terraform code.** - -Pre-commit hooks for Terraform **SHOULD** include: - -1. `terraform fmt` — Format check -2. `terraform validate` — Syntax validation -3. `tflint` — Linting -4. Security scanning (optional but recommended) - -**Workflow:** - -1. Make Terraform changes -2. Run `terraform fmt -recursive` -3. Run `terraform validate` -4. Run pre-commit hooks: `pre-commit run --all-files` -5. Review and commit ALL auto-fixes as part of your change -6. Push to GitHub - -**CI is a safety net, not a substitute for local checks.** -``` - -### Updating the Modular Instructions Table - -When the `terraform.instructions.md` file is created, the table in `.github/copilot-instructions.md` **MUST** be updated: - -```text -| Scope | Instruction File | Applies To | -| --- | --- | --- | -| Git attributes | `.github/instructions/gitattributes.instructions.md` | `**/.gitattributes` | -| Markdown/Docs | `.github/instructions/docs.instructions.md` | `**/*.md` | -| PowerShell | `.github/instructions/powershell.instructions.md` | `**/*.ps1` | -| Python | `.github/instructions/python.instructions.md` | `**/*.py` | -| Terraform | `.github/instructions/terraform.instructions.md` | `**/*.tf`, `**/*.tfvars`, `**/*.tftest.hcl`, `**/*.tf.json`, `**/*.tftpl`, `**/*.tfbackend` | -``` - -The Linting Configurations table **SHOULD** also be updated if Terraform-specific linting configurations are added: - -```text -| Tool | Configuration File | Purpose | -| --- | --- | --- | -| PSScriptAnalyzer | `.github/linting/PSScriptAnalyzerSettings.psd1` | PowerShell formatting/linting (OTBS style) | -| markdownlint | `.markdownlint.jsonc` | Markdown linting | -| TFLint | `.tflint.hcl` | Terraform linting | -``` - -The Testing Tools table **SHOULD** be updated: - -```text -| Language | Framework | Configuration | Test Location | -| --- | --- | --- | --- | -| Python | pytest | `pyproject.toml` (`[tool.pytest.ini_options]`) | `tests/` | -| PowerShell | Pester 5.x | Inline in `.github/workflows/powershell-ci.yml` | `tests/PowerShell/` | -| Terraform | Terraform Test (requires Terraform 1.6+) | Built-in | `modules/*/tests/` or `tests/` | -``` - ---- - -## Tooling and Pre-commit Configuration - -### Recommended Pre-commit Hooks - -```text -### Terraform Pre-commit Hooks - -The following pre-commit configuration is **RECOMMENDED** for Terraform: - -\`\`\`yaml -# .pre-commit-config.yaml (Terraform section) -repos: - - repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.105.0 # Pinned version – update periodically - hooks: - # Required hooks - - id: terraform_fmt - - id: terraform_validate - - # Recommended hooks - - id: terraform_tflint - args: - - --args=--config=__GIT_WORKING_DIR__/.tflint.hcl - - # Optional security scanning - - id: terraform_tfsec - - id: terraform_checkov - args: - - --args=--quiet - - --args=--compact -\`\`\` -``` - -### TFLint Configuration - -```text -### TFLint Configuration - -A `.tflint.hcl` configuration file **SHOULD** be created at the repository root: - -\`\`\`hcl -# .tflint.hcl - -config { - format = "compact" - plugin_dir = "~/.tflint.d/plugins" - - call_module_type = "local" - force = false - disabled_by_default = false -} - -plugin "terraform" { - enabled = true - preset = "recommended" -} - -# AWS-specific rules (if using AWS) -plugin "aws" { - enabled = true - version = "0.32.0" - source = "github.com/terraform-linters/tflint-ruleset-aws" -} - -# Rule overrides -rule "terraform_naming_convention" { - enabled = true - format = "snake_case" -} - -rule "terraform_documented_variables" { - enabled = true -} - -rule "terraform_documented_outputs" { - enabled = true -} -\`\`\` -``` - -### CI Workflow Example - -```text -### Terraform CI Workflow - -A GitHub Actions workflow for Terraform **SHOULD** include: - -\`\`\`yaml -# .github/workflows/terraform-ci.yml -name: Terraform CI - -on: - push: - paths: - - '**/*.tf' - - '**/*.tfvars' - - '**/*.tftest.hcl' - - '.tflint.hcl' - pull_request: - paths: - - '**/*.tf' - - '**/*.tfvars' - - '**/*.tftest.hcl' - - '.tflint.hcl' - -jobs: - validate: - name: Validate - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - - uses: hashicorp/setup-terraform@v3 - with: - terraform_version: "1.6.0" - - - name: Terraform Format Check - run: terraform fmt -check -recursive -diff - - - name: Terraform Init - run: terraform init -backend=false - - - name: Terraform Validate - run: terraform validate - - lint: - name: Lint - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - - uses: terraform-linters/setup-tflint@v4 - - - name: Init TFLint - run: tflint --init - - - name: Run TFLint - run: tflint --recursive - - test: - name: Test - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - - uses: hashicorp/setup-terraform@v3 - with: - terraform_version: "1.6.0" - - - name: Terraform Init - run: terraform init - - - name: Terraform Test - run: terraform test -verbose - - security: - name: Security Scan - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - - name: Run tfsec - uses: aquasecurity/tfsec-action@v1.0.3 - with: - soft_fail: true # Set to false to fail on findings -\`\`\` -``` - ---- - -## Implementation Checklist - -Use this checklist when creating the `terraform.instructions.md` file: - -### Document Structure - -- [ ] YAML front matter with `applyTo` and `description` -- [ ] Title with version number (`1.0.YYYYMMDD.0` format) -- [ ] Metadata section (Status, Owner, Last Updated, Scope, Related) -- [ ] Comprehensive Table of Contents with anchor links -- [ ] Keywords section (RFC 2119 definitions) -- [ ] Quick Reference Checklist (40+ items with scope tags and links) -- [ ] Executive Summary describing Terraform philosophy - -### Coding Standards Coverage - -- [ ] Formatting standards (`terraform fmt`, indentation) -- [ ] Naming conventions (resources, variables, outputs, modules) -- [ ] File organization patterns (standard files, when to split) -- [ ] Variable documentation requirements (description, type, validation) -- [ ] Output documentation requirements -- [ ] Resource configuration (meta-argument ordering, tagging) -- [ ] Module design principles -- [ ] State management guidance -- [ ] Provider management and version constraints - -### Security Content - -- [ ] Secret management (prohibited patterns, approved patterns) -- [ ] Sensitive variable marking -- [ ] State file security (encryption, access control, locking) -- [ ] Least-privilege resource policies -- [ ] Security scanning recommendations - -### Testing Content - -- [ ] Test file naming and location -- [ ] Test file structure (blocks, syntax) -- [ ] Test patterns (AAA, validation, outputs) -- [ ] Mock providers -- [ ] Unit vs integration tests -- [ ] Running tests (commands, CI integration) -- [ ] What to test guidance - -### Documentation Standards - -- [ ] Module README requirements -- [ ] Inline comment conventions -- [ ] Example usage documentation - -### Integration - -- [ ] Reference to repository constitution -- [ ] Pre-commit discipline guidance -- [ ] "Done" definition section -- [ ] Instructions for updating `.github/copilot-instructions.md` tables - -### Quality Checks - -- [ ] All checklist items link to detailed sections -- [ ] Comprehensive code examples throughout -- [ ] Tables for structured information -- [ ] RFC 2119 keywords used consistently -- [ ] No contradictions with repository constitution -- [ ] Document is comprehensive (~100KB+ target for depth matching PowerShell) - ---- - -## Recommended Next Steps - -1. **Create the instruction file:** Use this guide to write `.github/instructions/terraform.instructions.md` - -2. **Add tooling configuration:** - - Create `.tflint.hcl` at repository root - - Update `.pre-commit-config.yaml` with Terraform hooks - - Create `.github/workflows/terraform-ci.yml` - -3. **Update the repository constitution:** - - Add Terraform to the Modular Instructions table - - Add TFLint to the Linting Configurations table - - Add Terraform Test to the Testing Tools table - -4. **Validate the instruction file:** - - Verify all anchor links work - - Ensure no contradictions with `.github/copilot-instructions.md` - - Test that the applyTo pattern works correctly - -5. **Iterate based on usage:** - - Gather feedback from Copilot-generated Terraform code - - Refine rules that produce undesirable outputs - - Add examples for common patterns - ---- - -## References - -This guide is based on widely accepted Terraform conventions and best practices: - -- HashiCorp Terraform Style Conventions -- HashiCorp Terraform Best Practices -- Terraform Module Registry conventions -- terraform-docs documentation standards -- pre-commit-terraform hook collection -- Community patterns from terraform-aws-modules and similar projects - -**Note:** Specific URLs and external links are intentionally omitted as they may change. The patterns documented here represent stable, long-standing conventions in the Terraform ecosystem as of Terraform 1.6+. diff --git a/docs/terraform/TERRAFORM_LINTING_GUIDE.md b/docs/terraform/TERRAFORM_LINTING_GUIDE.md deleted file mode 100644 index ba0b625..0000000 --- a/docs/terraform/TERRAFORM_LINTING_GUIDE.md +++ /dev/null @@ -1,1650 +0,0 @@ -# Terraform Linting Implementation Guide - -**Version:** 1.0.20260124.0 - -## Metadata - -- **Status:** Active -- **Owner:** Repository Maintainers -- **Last Updated:** 2026-01-24 -- **Scope:** This document provides comprehensive guidance for implementing Terraform linting in CI for the `franklesniak/copilot-repo-template` repository. It covers tool selection, workflow design, configuration, pre-commit integration, and best practices. This is a **guidance-only** document—it does not modify workflows or configurations directly. -- **Related:** [Repository Copilot Instructions](../../.github/copilot-instructions.md), [Terraform Instructions](../../.github/instructions/terraform.instructions.md) - ---- - -## Table of Contents - -- [Introduction](#introduction) -- [Linting Tool Analysis](#linting-tool-analysis) - - [terraform fmt](#terraform-fmt) - - [terraform validate](#terraform-validate) - - [TFLint](#tflint) - - [Checkov](#checkov) - - [tfsec](#tfsec) - - [Terrascan](#terrascan) - - [Tool Recommendation Summary](#tool-recommendation-summary) -- [GitHub Actions Workflow Recommendations](#github-actions-workflow-recommendations) - - [Workflow Structure](#workflow-structure) - - [Job Organization](#job-organization) - - [Job Configuration Details](#job-configuration-details) - - [Matrix Strategy Considerations](#matrix-strategy-considerations) - - [Caching Strategies](#caching-strategies) - - [Security Scanning Guidance](#security-scanning-guidance) -- [Configuration File Recommendations](#configuration-file-recommendations) - - [TFLint Configuration Structure](#tflint-configuration-structure) - - [Configuration File Placement](#configuration-file-placement) - - [Recommended Rule Sets and Plugins](#recommended-rule-sets-and-plugins) - - [Rule Customization](#rule-customization) -- [Pre-commit Integration](#pre-commit-integration) - - [Recommended Hooks](#recommended-hooks) - - [Updating pre-commit-config.yaml](#updating-pre-commit-configyaml) - - [Local Workflow](#local-workflow) - - [Troubleshooting Pre-commit Hooks](#troubleshooting-pre-commit-hooks) -- [CI Workflow Design Decisions](#ci-workflow-design-decisions) - - [Header Comment Documentation](#header-comment-documentation) - - [Job Dependencies](#job-dependencies) - - [Continue-on-error Usage](#continue-on-error-usage) - - [Multiple Terraform Roots](#multiple-terraform-roots) - - [Path Filtering](#path-filtering) - - [Failure Handling](#failure-handling) -- [Integration with Existing Infrastructure](#integration-with-existing-infrastructure) - - [Auto-fix Workflow Integration](#auto-fix-workflow-integration) - - [Updates to .github/copilot-instructions.md](#updates-to-githubcopilot-instructionsmd) - - [README Update Recommendations](#readme-update-recommendations) -- [Versioning Policy Guidance](#versioning-policy-guidance) -- [Summary of Recommended Steps](#summary-of-recommended-steps) - ---- - -## Introduction - -This guide provides comprehensive recommendations for implementing Terraform linting in CI for this repository. The goal is to establish a consistent, secure, and maintainable Terraform development workflow that: - -- **Catches formatting issues** before code review -- **Validates configuration syntax** before apply operations -- **Enforces coding standards** via configurable linting rules -- **Identifies security vulnerabilities** through static analysis -- **Integrates seamlessly** with the existing pre-commit and CI infrastructure - -All recommendations in this guide follow the patterns established in this repository's existing Python and PowerShell CI workflows. This guide uses RFC 2119 keywords (**MUST**, **SHOULD**, **MAY**, etc.) to indicate requirement levels. - ---- - -## Linting Tool Analysis - -This section provides a comprehensive comparison of Terraform linting tools to inform tool selection. - -### terraform fmt - -**Official Documentation:** [terraform fmt Command](https://developer.hashicorp.com/terraform/cli/commands/fmt) - -#### What It Does - -`terraform fmt` reformats Terraform configuration files to a canonical format and style. It applies HashiCorp's style conventions consistently across all `.tf` files. - -#### How to Run - -```bash -# Check mode (returns exit code 1 if changes needed) -terraform fmt -check -recursive -diff - -# Write mode (modifies files in-place) -terraform fmt -recursive -``` - -#### Modes - -| Mode | Flag | Behavior | CI Use | -| --- | --- | --- | --- | -| Check | `-check` | Returns exit code 1 if changes needed, does not modify files | **Required** in CI | -| Write | (default) | Modifies files in-place | Pre-commit only | -| Diff | `-diff` | Shows differences when used with `-check` | Recommended for debugging | -| Recursive | `-recursive` | Processes subdirectories | Always use | - -#### Pros - -- **Built-in:** No additional installation required (bundled with Terraform) -- **Deterministic:** Produces identical output regardless of initial formatting -- **Fast:** Minimal overhead for format checking -- **Auto-fixable:** Can automatically correct issues in pre-commit - -#### Cons - -- **Limited scope:** Only checks formatting, not validity or best practices -- **No customization:** Style rules are not configurable - -#### When to Use - -`terraform fmt` **MUST** be the first check in any Terraform CI pipeline. It ensures consistent formatting before any other validation occurs. - ---- - -### terraform validate - -**Official Documentation:** [terraform validate Command](https://developer.hashicorp.com/terraform/cli/commands/validate) - -#### What It Validates - -`terraform validate` checks configuration files for syntax errors and internal consistency. It validates: - -- HCL syntax correctness -- Attribute types and values -- Required arguments are present -- Module source paths exist -- Variable and output declarations are valid - -#### Initialization Requirements - -`terraform validate` **requires** `terraform init` to be run first to: - -- Download provider plugins -- Initialize backend configuration -- Download module sources - -For CI validation, use `-backend=false` to skip backend initialization: - -```bash -terraform init -backend=false -terraform validate -``` - -#### How to Run - -```bash -# Basic validation (requires init first) -terraform validate - -# With JSON output for parsing -terraform validate -json -``` - -#### Pros - -- **Built-in:** No additional installation required -- **Comprehensive:** Catches syntax errors and type mismatches -- **Provider-aware:** Validates against actual provider schemas - -#### Cons - -- **Requires init:** Must download providers before validation -- **No best practices:** Does not enforce coding standards -- **No security scanning:** Does not identify security issues -- **Slower:** Network calls to download providers - -#### When to Use - -`terraform validate` **SHOULD** run after `terraform fmt` to catch configuration errors before linting. It validates that the configuration is syntactically correct and internally consistent. - ---- - -### TFLint - -**Official Documentation:** [TFLint](https://github.com/terraform-linters/tflint) - -#### Purpose and Capabilities - -TFLint is a pluggable Terraform linter that enforces best practices and coding standards. Key capabilities: - -- **Syntax linting:** Variable naming, deprecated syntax, unused declarations -- **Best practice enforcement:** Module versioning, provider versioning, required blocks -- **Provider-specific rules:** AWS, Azure, GCP plugins for cloud-specific linting -- **Extensibility:** Custom rules via plugins - -#### Plugin Ecosystem - -| Plugin | Source | Purpose | -| --- | --- | --- | -| terraform | Built-in | Core Terraform rules | -| aws | `terraform-linters/tflint-ruleset-aws` | AWS resource validation | -| azurerm | `terraform-linters/tflint-ruleset-azurerm` | Azure resource validation | -| google | `terraform-linters/tflint-ruleset-google` | GCP resource validation | - -#### Configuration Options - -TFLint is configured via `.tflint.hcl`: - -```hcl -# .tflint.hcl - Basic configuration -config { - format = "compact" # Output format: default, json, checkstyle, compact - call_module_type = "local" # Which modules to inspect: local, all, none - force = false # Continue on errors - disabled_by_default = false # Start with all rules enabled -} - -# Enable Terraform plugin with recommended preset -plugin "terraform" { - enabled = true - preset = "recommended" # all, recommended, none -} -``` - -#### How to Run - -```bash -# Initialize plugins (downloads ruleset plugins) -tflint --init - -# Run linting (current directory) -tflint - -# Recursive mode (all subdirectories) -tflint --recursive - -# With specific config file -tflint --config /path/to/.tflint.hcl -``` - -#### Pros - -- **Configurable:** Fine-grained control over which rules to enable -- **Extensible:** Plugin ecosystem for cloud providers -- **Fast:** Efficient static analysis -- **Active development:** Well-maintained with regular updates - -#### Cons - -- **Additional tool:** Requires separate installation -- **Plugin management:** Cloud provider plugins require initialization -- **No security focus:** Not designed for security scanning - -#### Recommended Use Cases - -TFLint **SHOULD** be the primary linting tool for enforcing coding standards. Use the `recommended` preset and enable cloud provider plugins as needed. - ---- - -### Checkov - -**Official Documentation:** [Checkov](https://www.checkov.io/) - -#### Purpose and Capabilities - -Checkov is a static code analysis tool for infrastructure as code that scans for security and compliance misconfigurations. Key capabilities: - -- **Security scanning:** Identifies insecure configurations (open security groups, unencrypted storage, etc.) -- **Compliance frameworks:** Supports CIS, SOC2, HIPAA, PCI-DSS, and more -- **Multi-IaC support:** Terraform, CloudFormation, Kubernetes, ARM templates, etc. -- **Custom policies:** Write custom policies in Python or YAML - -#### Policy Coverage - -Checkov includes 1000+ built-in policies covering: - -- Network security (security groups, NACLs, firewall rules) -- Encryption at rest and in transit -- IAM and access control -- Logging and monitoring -- Data protection -- Container security - -#### Custom Policy Support - -Custom policies can be defined in: - -- **Python:** Full programmatic control -- **YAML:** Declarative policy definitions - -```yaml -# Example custom policy (custom_policy.yaml) -metadata: - name: "Ensure S3 bucket has versioning enabled" - id: "CUSTOM_S3_001" - category: "general" -definition: - and: - - cond_type: "attribute" - attribute: "versioning.enabled" - operator: "equals" - value: "true" -``` - -#### How to Run - -```bash -# Basic scan -checkov -d /path/to/terraform - -# Specific framework -checkov -d /path/to/terraform --framework terraform - -# With output file -checkov -d . -o sarif > checkov-results.sarif - -# Soft fail (exit 0 even with findings) -checkov -d . --soft-fail -``` - -#### Pros - -- **Comprehensive:** Extensive policy coverage out of the box -- **Multi-framework:** Supports many IaC formats -- **Compliance-focused:** Built-in compliance framework mappings -- **Active community:** Regular policy updates - -#### Cons - -- **Python dependency:** Requires Python runtime -- **Slower:** More comprehensive scans take longer -- **Noisy:** Can produce many findings on existing codebases -- **Overlap with tfsec:** Some redundancy in security scanning - -#### When to Use - -Checkov is **RECOMMENDED** when compliance framework mappings are required or when scanning multiple IaC formats. For pure Terraform security scanning, consider tfsec as a lighter alternative. - ---- - -### tfsec - -**Official Documentation:** [tfsec](https://github.com/aquasecurity/tfsec) (now part of Trivy) - -> **Note:** tfsec has been integrated into Trivy. While tfsec remains available as a standalone tool, Aqua Security recommends using Trivy for new implementations. - -#### Purpose and Capabilities - -tfsec is a security scanner for Terraform that identifies potential security issues. Key capabilities: - -- **Security-focused:** Purpose-built for Terraform security scanning -- **Fast:** Written in Go, optimized for speed -- **Terraform-native:** Deep understanding of Terraform semantics -- **GitHub integration:** Native SARIF output for code scanning - -#### Differences vs Checkov - -| Aspect | tfsec | Checkov | -| --- | --- | --- | -| Focus | Security only | Security + compliance | -| Speed | Faster | Slower | -| Language | Go | Python | -| IaC support | Terraform only | Multi-framework | -| Policy format | Go | Python/YAML | -| Compliance | Limited | Extensive | - -#### How to Run - -```bash -# Basic scan -tfsec . - -# With SARIF output (for GitHub code scanning) -tfsec . --format sarif > tfsec-results.sarif - -# Soft fail -tfsec . --soft-fail - -# Minimum severity -tfsec . --minimum-severity HIGH -``` - -#### Pros - -- **Fast:** Efficient Go implementation -- **Focused:** Security-only means less noise -- **Easy setup:** Single binary, no dependencies -- **GitHub native:** Excellent CI integration - -#### Cons - -- **Deprecated path:** Migrating to Trivy -- **Terraform only:** No multi-framework support -- **Less compliance:** Limited compliance framework support - -#### When to Use - -tfsec is **RECOMMENDED** as the primary security scanner for Terraform-only repositories that don't require extensive compliance framework mappings. - ---- - -### Terrascan - -**Official Documentation:** [Terrascan](https://github.com/tenable/terrascan) - -#### Purpose and Capabilities - -Terrascan is a static code analyzer for Infrastructure as Code that detects compliance and security violations. Key capabilities: - -- **Policy as Code:** Uses Rego (Open Policy Agent) for policy definitions -- **Multi-IaC support:** Terraform, Kubernetes, AWS CloudFormation, Azure ARM, GCP Deployment Manager -- **CI/CD integration:** Supports various CI platforms -- **Extensibility:** Custom policies via Rego - -#### Policy as Code Support - -Terrascan uses Open Policy Agent (OPA) Rego for policies: - -```rego -# Example Rego policy -package accurics - -# Ensure S3 bucket is encrypted -encryptedS3Bucket[resource.id] { - resource := input.aws_s3_bucket[_] - not resource.config.server_side_encryption_configuration -} -``` - -#### How to Run - -```bash -# Basic scan -terrascan scan - -# Specify IaC type -terrascan scan -i terraform - -# With output format -terrascan scan -o sarif - -# Custom policy path -terrascan scan -p /path/to/policies -``` - -#### Pros - -- **OPA/Rego policies:** Industry-standard policy language -- **Multi-IaC:** Broad infrastructure support -- **Active development:** Maintained by Tenable - -#### Cons - -- **Rego learning curve:** Custom policies require Rego knowledge -- **Less Terraform-specific:** Generalist approach -- **Smaller community:** Fewer resources than Checkov/tfsec - -#### When to Use - -Terrascan is **RECOMMENDED** when OPA/Rego policy standardization is required across multiple IaC types or when integrating with existing OPA infrastructure. - ---- - -### Tool Recommendation Summary - -Based on the analysis above, here are the recommended tools for this repository: - -#### Recommended Toolchain - -| Tool | Purpose | Priority | Rationale | -| --- | --- | --- | --- | -| `terraform fmt` | Format checking | **Required** | Built-in, fast, deterministic | -| `terraform validate` | Syntax validation | **Required** | Built-in, catches configuration errors | -| TFLint | Linting/best practices | **Required** | Configurable, extensible, well-maintained | -| tfsec (or Trivy) | Security scanning | **Recommended** | Fast, focused, Terraform-native | - -#### Why This Combination - -1. **No redundancy:** Each tool has a distinct purpose -2. **Fast feedback:** Format and validation checks run quickly -3. **Configurable:** TFLint provides fine-grained control over linting rules -4. **Security coverage:** tfsec provides security scanning without compliance overhead -5. **Existing integration:** TFLint is already configured in this repository's `.tflint.hcl` and pre-commit - -#### When to Add Additional Tools - -- **Add Checkov:** When compliance framework mappings (CIS, SOC2, HIPAA) are required -- **Add Terrascan:** When OPA/Rego policy standardization is required -- **Use Trivy instead of tfsec:** For new implementations (tfsec is deprecated) - -#### Avoiding Redundant Scans - -Running multiple security scanners (Checkov + tfsec + Terrascan) is **NOT RECOMMENDED** because: - -- Increased CI time with diminishing returns -- Conflicting/duplicate findings create noise -- Policy management becomes complex - -**SHOULD** choose one primary security scanner and use others only for advisory purposes (with `continue-on-error: true`). - ---- - -## GitHub Actions Workflow Recommendations - -This section provides detailed guidance on implementing Terraform linting in GitHub Actions, following the patterns established in this repository. - -### Workflow Structure - -#### Consolidated Single Workflow Approach - -Following the pattern in `python-ci.yml`, Terraform CI **SHOULD** use a single consolidated workflow with job dependencies: - -```yaml -# .github/workflows/terraform-ci.yml -# Single workflow with job dependencies - -name: Terraform CI - -on: - push: - branches: ["**"] - pull_request: - branches: ["**"] - workflow_dispatch: - -permissions: - contents: read - -jobs: - format: - # Format checking job - # ... - - validate: - needs: format - # Validation job - # ... - - lint: - needs: validate - # TFLint job - # ... - - security: - needs: validate - # Security scanning (optional) - # ... -``` - -#### Benefits of This Approach - -- **All jobs appear in PR checks:** Not hidden in Actions tab -- **Easy branch ruleset setup:** Configure required jobs directly -- **Efficient CI minutes:** Failed jobs skip downstream work -- **Simpler maintenance:** Single file to manage - -#### Header Comment Documentation - -Following the established pattern, the workflow file **MUST** include a comprehensive header comment: - -```yaml -# Terraform CI Workflow -# -# Purpose: Run format checking, validation, linting, and optional security -# scanning for Terraform code in a single consolidated workflow. -# -# Design Decisions: -# -# 1. Single Workflow with Job Dependencies: -# This workflow combines format, validate, lint, and security into a single -# workflow file using the `needs:` keyword for job dependencies. -# -# 2. Job Dependencies: -# - validate needs format (don't validate poorly-formatted code) -# - lint needs validate (don't lint invalid code) -# - security needs validate (runs in parallel with lint) -# -# 3. Template Repository Consideration: -# This workflow does NOT use path-based filtering because this is a template -# repository. Template repositories should run all workflows on all changes. -# -# 4. Terraform Version Policy: -# Pin to a specific Terraform version for reproducibility. Update when -# upstream support changes. -# -# Note: This workflow is optional. Remove if your project doesn't use Terraform. -``` - ---- - -### Job Organization - -#### Recommended Job Structure - -```yaml -jobs: - format: - name: Format Check - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: hashicorp/setup-terraform@v3 - with: - terraform_version: "1.6.0" - - name: Terraform Format Check - run: terraform fmt -check -recursive -diff - - validate: - name: Validate - needs: format - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: hashicorp/setup-terraform@v3 - with: - terraform_version: "1.6.0" - - name: Find Terraform Directories - id: find-dirs - run: | - TF_DIRS=$(find . -name '*.tf' -exec dirname {} \; | sort -u | tr '\n' ' ') - echo "dirs=${TF_DIRS}" >> $GITHUB_OUTPUT - - name: Terraform Init and Validate - run: | - for dir in ${{ steps.find-dirs.outputs.dirs }}; do - echo "=== Validating $dir ===" - cd "$dir" - terraform init -backend=false - terraform validate - cd - > /dev/null - done - - lint: - name: Lint (TFLint) - needs: validate - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: terraform-linters/setup-tflint@v4 - with: - tflint_version: v0.51.1 # Pin version for reproducibility - - name: Init TFLint - run: tflint --init - - name: Run TFLint - run: tflint --recursive --config "$(pwd)/.tflint.hcl" - - security: - name: Security Scan (Optional) - needs: validate - runs-on: ubuntu-latest - continue-on-error: true # Advisory, not blocking - steps: - - uses: actions/checkout@v4 - - uses: aquasecurity/tfsec-action@v1.0.3 - with: - soft_fail: true -``` - -#### Job Dependencies Diagram - -```text -format - │ - ▼ -validate - │ - ├───────────┐ - ▼ ▼ - lint security -``` - -#### Rationale for Dependencies - -- **format → validate:** Don't validate poorly-formatted code; formatting issues should be fixed first -- **validate → lint:** Don't lint invalid configuration; syntax errors should be fixed first -- **validate → security:** Security scanning can run in parallel with linting (both depend on valid code) - ---- - -### Job Configuration Details - -#### Runner Selection - -| Job | Recommended Runner | Rationale | -| --- | --- | --- | -| format | `ubuntu-latest` | Terraform is cross-platform; Linux is fastest | -| validate | `ubuntu-latest` | Provider downloads faster on Linux | -| lint | `ubuntu-latest` | TFLint is available on all platforms | -| security | `ubuntu-latest` | Security tools optimized for Linux | - -#### Terraform Version Setup - -Use `hashicorp/setup-terraform` for consistent Terraform installation: - -```yaml -- uses: hashicorp/setup-terraform@v3 - with: - terraform_version: "1.6.0" # Pin specific version - terraform_wrapper: true # Enables output capturing -``` - -**Official Documentation:** [hashicorp/setup-terraform](https://github.com/hashicorp/setup-terraform) - -#### Permissions Requirements - -Follow least-privilege principles: - -```yaml -permissions: - contents: read # Minimum required for checkout -``` - -For security scanning with SARIF upload: - -```yaml -permissions: - contents: read - security-events: write # Required for code scanning upload -``` - ---- - -### Matrix Strategy Considerations - -#### When Matrix Testing Is Useful - -Matrix testing across Terraform versions is useful when: - -- Publishing reusable modules that must support multiple Terraform versions -- Migrating from an older Terraform version -- Testing provider compatibility across versions - -#### When Matrix Testing Is Overkill - -Matrix testing is **NOT RECOMMENDED** for: - -- Internal infrastructure code (pin to one version) -- Template repositories (test with recommended version only) -- Simple configurations without version-sensitive features - -#### Example Matrix Configuration - -```yaml -# Only use when testing module compatibility -strategy: - matrix: - terraform-version: ['1.5.0', '1.6.0', '1.7.0'] - -steps: - - uses: hashicorp/setup-terraform@v3 - with: - terraform_version: ${{ matrix.terraform-version }} -``` - ---- - -### Caching Strategies - -#### Provider Caching - -Cache the Terraform plugin directory to avoid repeated provider downloads: - -```yaml -- name: Cache Terraform Providers - uses: actions/cache@v4 - with: - path: ~/.terraform.d/plugin-cache - key: terraform-providers-${{ hashFiles('**/.terraform.lock.hcl') }} - restore-keys: | - terraform-providers- - -- name: Configure Provider Cache - run: | - mkdir -p ~/.terraform.d/plugin-cache - echo "plugin_cache_dir = \"$HOME/.terraform.d/plugin-cache\"" > ~/.terraformrc -``` - -#### TFLint Plugin Caching - -Cache TFLint plugins to avoid repeated downloads: - -```yaml -- name: Cache TFLint Plugins - uses: actions/cache@v4 - with: - path: ~/.tflint.d/plugins - key: tflint-plugins-${{ hashFiles('.tflint.hcl') }} - restore-keys: | - tflint-plugins- -``` - -#### Cache Key Strategy - -| Cache Type | Key Strategy | Invalidation Trigger | -| --- | --- | --- | -| Terraform providers | Hash of `.terraform.lock.hcl` | Provider version changes | -| TFLint plugins | Hash of `.tflint.hcl` | Plugin configuration changes | -| Pre-commit hooks | Hash of `.pre-commit-config.yaml` | Hook version changes | - ---- - -### Security Scanning Guidance - -#### Recommended Primary Scanner - -**SHOULD** use one primary security scanner and treat others as optional/advisory: - -| Scanner | Recommendation | Configuration | -| --- | --- | --- | -| tfsec/Trivy | Primary | `continue-on-error: false` | -| Checkov | Optional | `continue-on-error: true` | -| Terrascan | Optional | `continue-on-error: true` | - -#### Example Configuration - -```yaml -security: - name: Security Scan - needs: validate - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - # Primary scanner (required) - - name: Run tfsec - uses: aquasecurity/tfsec-action@v1.0.3 - with: - soft_fail: false # Fail on findings - - # Advisory scanner (optional) - - name: Run Checkov (Advisory) - uses: bridgecrewio/checkov-action@v12 - continue-on-error: true # Don't fail workflow - with: - directory: . - framework: terraform - soft_fail: true -``` - ---- - -## Configuration File Recommendations - -### TFLint Configuration Structure - -The repository already includes a `.tflint.hcl` configuration file. This section documents the recommended structure and options. - -#### Basic Structure - -```hcl -# .tflint.hcl - -# Global configuration -config { - format = "compact" # Output format - call_module_type = "local" # Module inspection scope - force = false # Continue on errors - disabled_by_default = false # Start with rules enabled -} - -# Core Terraform plugin -plugin "terraform" { - enabled = true - preset = "recommended" # Preset: all, recommended, none -} - -# Provider-specific plugins (enable as needed) -# plugin "aws" { ... } -# plugin "azurerm" { ... } -# plugin "google" { ... } - -# Rule-specific configuration -rule "terraform_naming_convention" { - enabled = true - format = "snake_case" -} -``` - -#### Configuration Options Reference - -| Option | Values | Description | -| --- | --- | --- | -| `format` | `default`, `json`, `checkstyle`, `compact`, `sarif` | Output format | -| `call_module_type` | `local`, `all`, `none` | Which modules to inspect | -| `force` | `true`, `false` | Continue on errors | -| `disabled_by_default` | `true`, `false` | Start with rules disabled | - ---- - -### Configuration File Placement - -#### Options Comparison - -| Location | Pros | Cons | -| --- | --- | --- | -| `.tflint.hcl` (repo root) | Standard location, easy to find | Clutters repo root | -| `.github/linting/.tflint.hcl` | Centralized with other linting configs | Non-standard, requires config path | - -#### Recommendation - -**SHOULD** place `.tflint.hcl` in the repository root for the following reasons: - -1. **Standard location:** TFLint looks for `.tflint.hcl` in the working directory by default -2. **Tool compatibility:** IDE plugins and pre-commit hooks expect this location -3. **Consistency:** Matches Terraform's other config files (`.terraform.lock.hcl`) - -The existing repository configuration follows this pattern correctly. - ---- - -### Recommended Rule Sets and Plugins - -#### Core Rules (Recommended) - -The following rules are enabled in the repository's `.tflint.hcl` and **SHOULD** remain enabled: - -```hcl -# Naming conventions -rule "terraform_naming_convention" { - enabled = true - format = "snake_case" -} - -# Documentation requirements -rule "terraform_documented_variables" { - enabled = true -} - -rule "terraform_documented_outputs" { - enabled = true -} - -# Type safety -rule "terraform_typed_variables" { - enabled = true -} - -# Code quality -rule "terraform_deprecated_interpolation" { - enabled = true -} - -rule "terraform_unused_declarations" { - enabled = true -} - -# Version constraints -rule "terraform_module_version" { - enabled = true -} - -rule "terraform_required_providers" { - enabled = true -} - -rule "terraform_required_version" { - enabled = true -} - -# Structure -rule "terraform_standard_module_structure" { - enabled = true -} - -rule "terraform_workspace_remote" { - enabled = true -} -``` - -#### Provider Plugins - -Enable provider plugins based on your cloud provider usage: - -```hcl -# AWS (uncomment if using AWS) -plugin "aws" { - enabled = true - version = "0.32.0" - source = "github.com/terraform-linters/tflint-ruleset-aws" -} - -# Azure (uncomment if using Azure) -plugin "azurerm" { - enabled = true - version = "0.27.0" - source = "github.com/terraform-linters/tflint-ruleset-azurerm" -} - -# Google Cloud (uncomment if using GCP) -plugin "google" { - enabled = true - version = "0.30.0" - source = "github.com/terraform-linters/tflint-ruleset-google" -} -``` - -**Official Documentation:** - -- [tflint-ruleset-aws](https://github.com/terraform-linters/tflint-ruleset-aws) -- [tflint-ruleset-azurerm](https://github.com/terraform-linters/tflint-ruleset-azurerm) -- [tflint-ruleset-google](https://github.com/terraform-linters/tflint-ruleset-google) - ---- - -### Rule Customization - -#### Disabling Specific Rules - -To disable a rule, set `enabled = false`: - -```hcl -rule "terraform_standard_module_structure" { - enabled = false # Disabled for flat directory structure -} -``` - -#### Adjusting Rule Severity - -Some rules support severity configuration: - -```hcl -rule "terraform_deprecated_interpolation" { - enabled = true - # Note: severity is controlled by the rule itself, not configurable -} -``` - -#### Organization-Specific Customizations - -For organization-specific rules, consider: - -1. **Custom naming patterns:** - - ```hcl - rule "terraform_naming_convention" { - enabled = true - format = "snake_case" - - # Custom format for specific resources (example) - custom = "^[a-z][a-z0-9_]*$" - } - ``` - -2. **Module inspection depth:** - - ```hcl - config { - call_module_type = "all" # Inspect all modules, not just local - } - ``` - ---- - -## Pre-commit Integration - -This repository already has Terraform hooks configured in `.pre-commit-config.yaml`. This section documents the configuration and local workflow. - -### Recommended Hooks - -The following hooks are **RECOMMENDED** and are currently configured: - -| Hook | Purpose | Auto-fix | -| --- | --- | --- | -| `terraform_fmt` | Format checking | Yes | -| `terraform_validate` | Syntax validation | No | -| `terraform_tflint` | Linting | No | - -#### Additional Optional Hooks - -| Hook | Purpose | When to Use | -| --- | --- | --- | -| `terraform_docs` | Auto-generate documentation | When maintaining module READMEs | -| `terraform_tfsec` | Security scanning | When security scanning in pre-commit is required | -| `terraform_checkov` | Compliance scanning | When compliance checks are required | - ---- - -### Updating pre-commit-config.yaml - -The repository's current configuration: - -```yaml -# Terraform formatting and validation (remove if not using Terraform) -- repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.105.0 # Pin to specific version - hooks: - - id: terraform_fmt - - id: terraform_validate - - id: terraform_tflint - args: - - --args=--config=__GIT_WORKING_DIR__/.tflint.hcl -``` - -**Official Documentation:** [pre-commit-terraform](https://github.com/antonbabenko/pre-commit-terraform) - -#### Hook Ordering - -Hooks execute in the order listed. The recommended order is: - -1. `terraform_fmt` — Fix formatting first -2. `terraform_validate` — Validate syntax -3. `terraform_tflint` — Lint for best practices - -This order ensures that formatting is fixed before validation, and validation passes before linting. - -#### Adding terraform-docs Hook - -To add automatic documentation generation: - -```yaml -- id: terraform_docs - args: - - --hook-config=--path-to-file=README.md - - --hook-config=--add-to-existing-file=true - - --hook-config=--create-file-if-not-exist=true -``` - ---- - -### Local Workflow - -#### Installation - -```bash -# Install pre-commit globally -pip install pre-commit - -# Install hooks in repository -pre-commit install -``` - -#### Running Hooks Manually - -```bash -# Run all hooks on all files -pre-commit run --all-files - -# Run specific hook -pre-commit run terraform_fmt --all-files -pre-commit run terraform_tflint --all-files - -# Run on staged files only -pre-commit run -``` - -#### Updating Hook Versions - -```bash -# Update all hooks to latest versions -pre-commit autoupdate - -# Review changes to .pre-commit-config.yaml -git diff .pre-commit-config.yaml -``` - ---- - -### Troubleshooting Pre-commit Hooks - -#### Common Issues - -| Issue | Cause | Solution | -| --- | --- | --- | -| `terraform_validate` fails | Terraform not initialized | Run `terraform init -backend=false` in each directory | -| `terraform_tflint` fails | TFLint plugins not initialized | Run `tflint --init` | -| Hook times out | Large repository | Increase timeout with `--hook-timeout` | -| Hook skipped | No matching files | Verify file patterns in hook config | - -#### Debugging Hook Failures - -```bash -# Run with verbose output -pre-commit run --verbose terraform_tflint --all-files - -# Show hook configuration -pre-commit show-config -``` - -#### Bypassing Hooks (Emergency Only) - -```bash -# Skip all hooks (not recommended) -git commit --no-verify -m "Emergency commit" -``` - -> **Warning:** Bypassing hooks should be reserved for emergencies. CI will still enforce these checks. - ---- - -## CI Workflow Design Decisions - -This section explains the design decisions that **SHOULD** be documented in the workflow header comment. - -### Header Comment Documentation - -Every workflow file **SHOULD** include a comprehensive header comment explaining: - -1. **Purpose:** What the workflow does -2. **Design Decisions:** Why key choices were made -3. **Template Considerations:** Special handling for template repos -4. **Version Policy:** How tool versions are managed - -#### Example Header - -```yaml -# Terraform CI Workflow -# -# Purpose: Run format checking, validation, linting, and testing for -# Terraform code in a single consolidated workflow. -# -# Design Decisions: -# -# 1. Single Workflow with Job Dependencies: -# Combines format, validate, lint, and test into one workflow file using -# `needs:` for job dependencies. Benefits: all jobs visible in PR checks, -# simpler branch ruleset config, efficient CI minutes. -# -# 2. Job Dependencies: -# format → validate → lint (sequential) -# validate → test (if tests exist) -# This ensures we don't waste time on later checks if earlier ones fail. -# -# 3. Template Repository Consideration: -# No path-based filtering. Template repos must test all workflows. -# -# 4. Terraform Version Policy: -# Pin to specific version (1.6.0) for reproducibility. Update when -# upstream support changes or new features are needed. -``` - ---- - -### Job Dependencies - -#### Why Format Before Validate - -- **Consistency:** Ensure all code has consistent formatting before validation -- **Cleaner diffs:** Formatting issues don't obscure validation errors -- **Faster feedback:** Formatting check is fast, fails early if needed - -#### Why Validate Before Lint - -- **Prerequisite:** TFLint assumes valid Terraform syntax -- **Cleaner output:** Validation errors are more fundamental than lint errors -- **Logical progression:** Fix syntax before style - ---- - -### Continue-on-error Usage - -#### When to Use `continue-on-error: true` - -| Scenario | Recommendation | -| --- | --- | -| Advisory security scans | Use `continue-on-error: true` | -| Secondary scanners | Use `continue-on-error: true` | -| Experimental checks | Use `continue-on-error: true` | -| Core linting/validation | **Never** use `continue-on-error` | - -#### Example - -```yaml -# Primary security scan - MUST pass -- name: Run tfsec - uses: aquasecurity/tfsec-action@v1.0.3 - -# Advisory scan - informational only -- name: Run Checkov (Advisory) - uses: bridgecrewio/checkov-action@v12 - continue-on-error: true -``` - ---- - -### Multiple Terraform Roots - -#### Monorepo Considerations - -When a repository contains multiple Terraform configurations (monorepo pattern): - -1. **Discovery approach:** Find all directories containing `.tf` files -2. **Iterate over directories:** Run init/validate in each -3. **Parallel execution:** Use matrix strategy for large repos - -#### Example Discovery Script - -```yaml -- name: Find Terraform Directories - id: find-tf-dirs - run: | - # Find all directories containing .tf files - TF_DIRS=$(find . -name '*.tf' -exec dirname {} \; | sort -u | tr '\n' ' ') - echo "dirs=${TF_DIRS}" >> $GITHUB_OUTPUT - -- name: Validate Each Directory - run: | - for dir in ${{ steps.find-tf-dirs.outputs.dirs }}; do - echo "=== Validating $dir ===" - cd "$dir" - terraform init -backend=false - terraform validate - cd - > /dev/null - done -``` - -#### Matrix Strategy for Large Repos - -```yaml -strategy: - matrix: - directory: ['modules/vpc', 'modules/ec2', 'environments/prod'] - -steps: - - name: Validate ${{ matrix.directory }} - working-directory: ${{ matrix.directory }} - run: | - terraform init -backend=false - terraform validate -``` - ---- - -### Path Filtering - -#### Why This Template Avoids Path Filtering - -The current `terraform-ci.yml` uses path filtering: - -```yaml -on: - push: - paths: - - '**/*.tf' - - '**/*.tfvars' - - '.tflint.hcl' -``` - -However, for template repositories, path filtering **SHOULD NOT** be used because: - -1. **Template testing:** All workflows should run to verify template works -2. **Hidden failures:** Path filtering can hide workflow issues -3. **User confusion:** Template users may not understand path filtering behavior - -#### Guidance for Non-Template Repos - -For non-template repositories, path filtering is **RECOMMENDED** to: - -- Reduce unnecessary CI runs -- Speed up feedback for non-Terraform changes -- Conserve CI minutes - -```yaml -# Recommended for non-template repos -on: - push: - paths: - - '**/*.tf' - - '**/*.tfvars' - - '**/*.tftest.hcl' - - '.tflint.hcl' - - '.github/workflows/terraform-ci.yml' -``` - ---- - -### Failure Handling - -#### Which Jobs Should Fail the Workflow - -| Job | Should Fail Workflow | Rationale | -| --- | --- | --- | -| format | Yes | Formatting is required | -| validate | Yes | Valid syntax is required | -| lint | Yes | Best practices are required | -| test | Yes (if tests exist) | Tests must pass | -| security (primary) | Configurable | Depends on security policy | -| security (advisory) | No | Informational only | - -#### Configuring Required Jobs in Branch Ruleset - -In GitHub branch ruleset settings, configure these jobs as required status checks: - -1. `Terraform CI / Format Check` -2. `Terraform CI / Validate` -3. `Terraform CI / Lint (TFLint)` -4. `Terraform CI / Test` (if tests exist) - -Security scans can be optional based on team preference. - ---- - -## Integration with Existing Infrastructure - -### Auto-fix Workflow Integration - -The repository's `auto-fix-precommit.yml` workflow could be extended to support Terraform auto-fixes. - -#### Which Checks Can Auto-fix - -| Check | Auto-fixable | Tool | -| --- | --- | --- | -| Formatting | Yes | `terraform fmt` | -| Validation errors | No | Manual fix required | -| Lint errors | Some | TFLint can fix some issues | -| Security findings | No | Manual fix required | - -#### Potential Auto-fix Addition - -The existing auto-fix workflow runs `pre-commit run --all-files`, which includes `terraform_fmt`. No additional configuration is needed for format auto-fixes. - -For repositories where TFLint can auto-fix issues, consider: - -```yaml -# In auto-fix workflow (if TFLint fixes are desired) -- name: Run TFLint with fixes - run: tflint --fix --recursive -``` - -> **Note:** TFLint's `--fix` flag is limited. Most lint issues require manual fixes. - ---- - -### Updates to .github/copilot-instructions.md - -When implementing Terraform linting, the following sections in `.github/copilot-instructions.md` should be updated: - -#### Linting Configurations Table - -Add TFLint to the existing table: - -```markdown -| Tool | Configuration File | Purpose | -| --- | --- | --- | -| PSScriptAnalyzer | `.github/linting/PSScriptAnalyzerSettings.psd1` | PowerShell formatting/linting (OTBS style) | -| markdownlint | `.markdownlint.jsonc` | Markdown linting | -| TFLint | `.tflint.hcl` | Terraform linting | -``` - -> **Note:** This is already present in the current `.github/copilot-instructions.md`. - -#### Running Linters Section - -Add Terraform commands: - -**Example Terraform linter commands:** - -```bash -terraform fmt -check -recursive -tflint --recursive -``` - -> **Note:** This is already present in the current `.github/copilot-instructions.md`. - ---- - -### README Update Recommendations - -When fully implementing Terraform support, consider updating the README with: - -#### Linting Tools Entry - -Add a Terraform section to the README: - -> **Terraform** -> -> This repository includes Terraform linting via: -> -> - **terraform fmt:** Format checking and auto-formatting -> - **terraform validate:** Configuration validation -> - **TFLint:** Best practice linting with cloud provider plugins -> -> Configuration: `.tflint.hcl` - -#### Command Examples Entry - -Add Terraform commands to the linters section: - -```bash -# Format check -terraform fmt -check -recursive - -# Format fix -terraform fmt -recursive - -# Validate (requires init) -terraform init -backend=false && terraform validate - -# Lint -tflint --init && tflint --recursive -``` - ---- - -## Versioning Policy Guidance - -### Terraform Version Pinning - -#### Recommendation - -**SHOULD** pin Terraform version for reproducibility: - -```yaml -- uses: hashicorp/setup-terraform@v3 - with: - terraform_version: "1.6.0" # Pin specific version -``` - -#### Version Constraint Options - -| Pattern | Example | Use Case | -| --- | --- | --- | -| Exact version | `1.6.0` | Maximum reproducibility | -| Minor range | `~> 1.6.0` | Accept patch updates | -| Major range | `~> 1.6` | Accept minor updates | -| Latest | `latest` | Always newest (not recommended) | - -#### When "latest" Is Acceptable - -Using `latest` is acceptable **ONLY** for: - -- Personal development environments -- Non-critical testing -- Intentionally tracking latest features - -Using `latest` is **NOT** acceptable for: - -- Production CI pipelines -- Reproducible builds -- Module publishing - -### TFLint Version Pinning - -The current configuration uses `latest`: - -```yaml -- uses: terraform-linters/setup-tflint@v4 - with: - tflint_version: latest -``` - -**Recommendation:** Pin to specific version for reproducibility: - -```yaml -- uses: terraform-linters/setup-tflint@v4 - with: - tflint_version: v0.51.1 -``` - -### Pre-commit Hook Version Pinning - -The repository follows best practice by pinning pre-commit hook versions: - -```yaml -- repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.105.0 # Pinned version -``` - -**MUST** pin versions in pre-commit configuration for: - -- Reproducible behavior -- Controlled updates -- CI consistency - ---- - -## Summary of Recommended Steps - -This section provides a clear action plan for implementing Terraform linting in this repository. - -### Immediate Actions (Already Complete) - -- [x] TFLint configuration file (`.tflint.hcl`) in repository root -- [x] Pre-commit hooks configured for Terraform -- [x] Basic Terraform CI workflow exists - -### Recommended Improvements - -1. **Update Terraform CI workflow header comments** - - Add comprehensive design decision documentation - - Document template repository considerations - -2. **Add job dependencies to Terraform CI** - - Configure `needs:` between format, validate, and lint jobs - - Ensure format runs before validate, validate before lint - -3. **Remove path filtering for template repository** - - Remove `paths:` filter from `on.push` and `on.pull_request` - - Run workflow on all changes for template accuracy - -4. **Add caching to Terraform CI** - - Cache Terraform providers - - Cache TFLint plugins - -5. **Consider security scanning** - - Add tfsec as optional security scan - - Configure with `continue-on-error: true` initially - -6. **Pin tool versions** - - Pin Terraform version in CI - - Pin TFLint version in CI - -### Implementation Checklist - -For reference, here is a checklist of all changes that would fully implement Terraform linting: - -- [ ] Update `terraform-ci.yml` with comprehensive header comments -- [ ] Add job dependencies (`needs:`) to workflow -- [ ] Remove path filtering from template workflow -- [ ] Add provider and TFLint plugin caching -- [ ] Pin Terraform and TFLint versions -- [ ] Add optional tfsec security scanning -- [ ] Verify pre-commit hooks work locally -- [ ] Test workflow on sample Terraform code -- [ ] Update README with Terraform linting documentation - -### Documentation Maintenance - -This guide **SHOULD** be updated when: - -- New Terraform linting tools become available -- Tool versions or configurations change significantly -- Repository CI patterns evolve -- Best practices change in the Terraform ecosystem - ---- - -## References - -### Official Documentation Links - -| Tool | Documentation URL | -| --- | --- | -| Terraform CLI | | -| terraform fmt | | -| terraform validate | | -| TFLint | | -| setup-terraform Action | | -| setup-tflint Action | | -| pre-commit-terraform | | -| Checkov | | -| tfsec | | -| Terrascan | | -| Trivy | | - -### Related Repository Documentation - -| Document | Path | -| --- | --- | -| Repository Constitution | `.github/copilot-instructions.md` | -| Terraform Instructions | `.github/instructions/terraform.instructions.md` | -| TFLint Configuration | `.tflint.hcl` | -| Pre-commit Configuration | `.pre-commit-config.yaml` | -| Terraform CI Workflow | `.github/workflows/terraform-ci.yml` | -| Auto-fix Workflow | `.github/workflows/auto-fix-precommit.yml` | diff --git a/docs/terraform/TERRAFORM_TESTING_GUIDE.md b/docs/terraform/TERRAFORM_TESTING_GUIDE.md deleted file mode 100644 index 37418af..0000000 --- a/docs/terraform/TERRAFORM_TESTING_GUIDE.md +++ /dev/null @@ -1,1845 +0,0 @@ -# Terraform Unit Testing Implementation Guide - -**Version:** 1.0.20260124.0 - -## Metadata - -- **Status:** Active -- **Owner:** Repository Maintainers -- **Last Updated:** 2026-01-24 -- **Scope:** This document provides comprehensive guidance for implementing Terraform unit testing in CI for the `franklesniak/copilot-repo-template` repository. It serves two purposes: (1) CI/Infrastructure Implementation Guide for setting up Terraform testing in GitHub Actions, and (2) Content Specification for what testing guidance should be embedded in `terraform.instructions.md`. This is a **guidance-only** document—it does not modify workflows or configurations directly. -- **Related:** [Repository Copilot Instructions](../../.github/copilot-instructions.md), [Terraform Instructions](../../.github/instructions/terraform.instructions.md), [Terraform Linting Guide](./TERRAFORM_LINTING_GUIDE.md) - ---- - -## Table of Contents - -- [Introduction](#introduction) -- [Part 1: Testing Framework Analysis](#part-1-testing-framework-analysis) -- [Part 2: Native Terraform Test Framework Deep Dive](#part-2-native-terraform-test-framework-deep-dive) -- [Part 3: Test Organization Best Practices](#part-3-test-organization-best-practices) -- [Part 4: CI Integration Recommendations](#part-4-ci-integration-recommendations) -- [Part 5: Test Writing Guidelines](#part-5-test-writing-guidelines) -- [Part 6: Content Specification for terraform.instructions.md](#part-6-content-specification-for-terraforminstructionsmd) -- [Part 7: Integration with Repository Patterns](#part-7-integration-with-repository-patterns) -- [Part 8: Advanced Topics](#part-8-advanced-topics) -- [Assumptions and Validation Notes](#assumptions-and-validation-notes) -- [Summary of Recommended Implementation Steps](#summary-of-recommended-implementation-steps) - ---- - -## Introduction - -This guide provides comprehensive recommendations for implementing Terraform unit testing in CI for this repository. The goal is to establish a consistent, reliable, and maintainable Terraform testing workflow that: - -- **Validates configuration logic** before deployment -- **Catches errors early** in the development cycle -- **Ensures module interfaces** work as designed -- **Verifies variable validation rules** function correctly -- **Integrates seamlessly** with the existing CI infrastructure - -All recommendations in this guide follow the patterns established in this repository's existing Python and PowerShell CI workflows. This guide uses RFC 2119 keywords (**MUST**, **SHOULD**, **MAY**, etc.) to indicate requirement levels. - -### Dual Purpose of This Document - -This document serves **TWO critical purposes**: - -1. **CI/Infrastructure Implementation Guide:** Detailed guidance on setting up Terraform testing in GitHub Actions workflows, test directory structure, CI configuration, and operational patterns. - -2. **Content Specification for terraform.instructions.md:** Explicit recommendations for what testing guidance should be **embedded directly in the Terraform Copilot instructions file**, following the pattern established by the PowerShell instructions file's "Testing with Pester" section. - -The distinction between these purposes is highlighted throughout the document where applicable. - ---- - -## Part 1: Testing Framework Analysis - -This section provides a comprehensive comparison of Terraform testing approaches to inform framework selection. - -### Native Terraform Testing (`terraform test`) - -**Official Documentation:** [Terraform Test Command](https://developer.hashicorp.com/terraform/cli/commands/test) - -#### How It Works - -The native Terraform test framework, introduced in Terraform 1.6.0, allows you to write tests in HCL using `.tftest.hcl` files. Tests are executed using the `terraform test` command, which: - -1. Discovers all `.tftest.hcl` files in the current directory and `tests/` subdirectory -2. Runs each test file's `run` blocks in sequence -3. Evaluates `assert` blocks to verify expected conditions -4. Reports pass/fail status with detailed error messages - -#### .tftest.hcl File Structure - -Test files use HCL syntax with specialized blocks: - -```hcl -# tests/basic.tftest.hcl - -# Global variables for all run blocks -variables { - environment = "test" - instance_type = "t3.micro" -} - -# Optional: Mock provider configuration -mock_provider "aws" { - mock_data "aws_availability_zones" { - defaults = { - names = ["us-east-1a", "us-east-1b"] - } - } -} - -# Test scenario 1 -run "validates_instance_type" { - command = plan - - assert { - condition = aws_instance.main.instance_type == "t3.micro" - error_message = "Instance type mismatch" - } -} - -# Test scenario 2 with overridden variables -run "rejects_invalid_environment" { - command = plan - - variables { - environment = "invalid" - } - - expect_failures = [ - var.environment - ] -} -``` - -#### Capabilities and Limitations - -| Capability | Description | -| --- | --- | -| Unit testing (`plan`) | Validates configuration without creating resources | -| Integration testing (`apply`) | Creates and destroys real resources | -| Mock providers | Simulates provider behavior for isolated testing | -| Variable validation testing | Tests custom validation rules via `expect_failures` | -| Output verification | Validates output values in assertions | -| Module testing | Tests modules in isolation or as compositions | - -| Limitation | Description | -| --- | --- | -| Terraform 1.6.0+ required | Not available in older Terraform versions | -| Limited assertion operators | No built-in regex or complex matchers | -| Sequential execution | Test files run sequentially, not in parallel | -| Mock provider limitations | Not all provider behaviors can be mocked | - -#### Pros - -- **Native integration:** Built into Terraform CLI, no additional tools required -- **HCL-based:** Uses familiar Terraform syntax for test definitions -- **Mock providers:** Enables true unit testing without cloud access -- **Fast unit tests:** `command = plan` tests execute quickly -- **No language barrier:** No need to learn Go, Ruby, or Python - -#### Cons - -- **Version requirement:** Requires Terraform 1.6.0 or later -- **Limited matchers:** Fewer assertion options than full testing frameworks -- **Newer framework:** Less community adoption and examples than Terratest -- **Mock limitations:** Complex provider interactions may not be mockable - -#### When to Use - -Use native Terraform testing as the **primary testing approach** for: - -- Variable validation rule testing -- Output value verification -- Configuration logic validation -- Module interface testing -- Any unit testing that doesn't require real infrastructure - ---- - -### Terratest (Go-based Integration Testing) - -**Official Documentation:** [Terratest on GitHub](https://github.com/gruntwork-io/terratest) - -#### How It Works - -Terratest is a Go library that provides helper functions for testing Terraform code. Tests are written in Go and executed using the standard Go testing framework: - -```go -// test/vpc_test.go -package test - -import ( - "testing" - "github.com/gruntwork-io/terratest/modules/terraform" - "github.com/stretchr/testify/assert" -) - -func TestVpcCreation(t *testing.T) { - terraformOptions := &terraform.Options{ - TerraformDir: "../modules/vpc", - Vars: map[string]interface{}{ - "cidr_block": "10.0.0.0/16", - }, - } - - defer terraform.Destroy(t, terraformOptions) - terraform.InitAndApply(t, terraformOptions) - - vpcId := terraform.Output(t, terraformOptions, "vpc_id") - assert.NotEmpty(t, vpcId) -} -``` - -#### Capabilities - -- **Real infrastructure testing:** Creates and destroys actual cloud resources -- **Rich assertions:** Full power of Go testing and assertion libraries -- **HTTP/SSH testing:** Built-in helpers for testing deployed applications -- **Retry logic:** Handles eventual consistency with configurable retries -- **Parallel testing:** Go test parallelization support - -#### Pros - -- **Comprehensive testing:** Tests actual infrastructure behavior -- **Mature ecosystem:** Large community and extensive documentation -- **Powerful assertions:** Full Go testing capabilities -- **Cross-platform:** Tests AWS, Azure, GCP, Kubernetes, and more - -#### Cons - -- **Requires Go knowledge:** Learning curve for non-Go developers -- **Slower execution:** Creates real resources, incurring time and cost -- **Complex setup:** Requires Go installation and module management -- **Cost implications:** Real resource creation generates cloud costs - -#### When to Use - -Use Terratest for: - -- End-to-end integration testing of complete infrastructure -- Testing actual cloud provider behavior -- Verifying deployed application functionality -- Complex multi-resource dependency testing - ---- - -### Kitchen-Terraform (Ruby-based Testing) - -**Official Documentation:** [Kitchen-Terraform on GitHub](https://github.com/newcontext-oss/kitchen-terraform) - -#### How It Works - -Kitchen-Terraform integrates Terraform with Test Kitchen, a Ruby-based testing framework. Tests are defined using InSpec profiles: - -```ruby -# test/integration/default/controls/vpc_test.rb -control 'vpc-1.0' do - impact 1.0 - title 'VPC Configuration' - desc 'Verify VPC is created correctly' - - describe aws_vpc(vpc_id: input('vpc_id')) do - it { should exist } - its('cidr_block') { should eq '10.0.0.0/16' } - end -end -``` - -#### Pros - -- **Compliance-focused:** InSpec profiles align with security/compliance requirements -- **Readable tests:** Human-readable test definitions -- **Platform maturity:** Test Kitchen is a mature testing framework - -#### Cons - -- **Ruby dependency:** Requires Ruby runtime and gem management -- **Additional tooling:** Requires Test Kitchen and InSpec setup -- **Smaller community:** Less Terraform-specific adoption than Terratest -- **Complex configuration:** Multiple configuration files required - -#### When to Use - -Use Kitchen-Terraform for: - -- Compliance and security testing scenarios -- Organizations already using Chef/InSpec -- Teams with Ruby expertise - ---- - -### pytest-terraform (Python-based Testing) - -**Official Documentation:** [pytest-terraform on GitHub](https://github.com/cloud-custodian/pytest-terraform) - -#### How It Works - -pytest-terraform provides pytest fixtures and helpers for testing Terraform: - -```python -# test_vpc.py -import pytest - -@pytest.mark.terraform('modules/vpc') -def test_vpc_creation(terraform): - terraform.init() - terraform.apply() - outputs = terraform.output() - assert outputs['vpc_id'] is not None -``` - -#### Pros - -- **Python ecosystem:** Integrates with pytest and Python tooling -- **Familiar syntax:** Uses standard pytest patterns -- **Existing infrastructure:** Leverages Python CI/CD pipelines - -#### Cons - -- **Less mature:** Smaller community than Terratest -- **Limited features:** Fewer Terraform-specific helpers -- **Still requires apply:** Tests create real resources - -#### When to Use - -Use pytest-terraform for: - -- Python-centric organizations -- Integration with existing pytest suites -- Teams with strong Python expertise - ---- - -### Recommendation Summary - -Based on the analysis above, this guide recommends the following testing approach: - -| Priority | Framework | Use Case | -| --- | --- | --- | -| **Primary** | Native Terraform Test (`terraform test`) | Unit testing, variable validation, output verification, module interfaces | -| **Secondary** | Terratest | Integration testing requiring real infrastructure when native tests are insufficient | -| **Not Recommended** | Kitchen-Terraform, pytest-terraform | Unless organization has specific Ruby/Python requirements | - -**Rationale for Native Terraform Test as Primary:** - -1. **No additional tooling:** Uses Terraform CLI already required for development -2. **HCL familiarity:** Developers write tests in the same language as configurations -3. **Fast execution:** Unit tests with `command = plan` are quick and cost-free -4. **Mock providers:** Enables true isolation for unit testing -5. **Built-in integration:** Aligns with HashiCorp's product direction -6. **Lower barrier to entry:** No Go or Ruby knowledge required - -**When to Opt into Terratest:** - -- Tests require verification of actual cloud provider behavior -- Complex multi-resource dependencies need end-to-end validation -- HTTP/SSH connectivity testing is required for deployed applications -- Native mocking is insufficient for the testing scenario - ---- - -## Part 2: Native Terraform Test Framework Deep Dive - -Since native Terraform testing is the recommended primary approach, this section provides detailed coverage of its features and patterns. - -### .tftest.hcl File Structure and Syntax - -#### File Format Overview - -Terraform test files use HCL syntax with specialized blocks. The file extension **MUST** be `.tftest.hcl`. - -#### Complete Block Reference - -| Block | Purpose | Cardinality | -| --- | --- | --- | -| `variables {}` | Set global variable values for all run blocks | 0..1 per file | -| `provider "name" {}` | Configure provider for tests | 0..n per file | -| `mock_provider "name" {}` | Mock a provider for unit testing | 0..n per file | -| `run "name" {}` | Define a test scenario | 1..n per file (at least one required) | -| `assert {}` | Define an assertion within a run block | 1..n per run (at least one required) | -| `expect_failures` | List of resources/variables expected to fail | 0..1 per run | - -#### Complete Syntax Reference - -```hcl -# tests/comprehensive.tftest.hcl - -# Global variables (optional) - Apply to all run blocks -variables { - environment = "test" - instance_type = "t3.micro" - enable_logging = true -} - -# Mock provider (optional) - For unit testing without real infrastructure -mock_provider "aws" { - # Mock data source responses - mock_data "aws_availability_zones" { - defaults = { - names = ["us-east-1a", "us-east-1b", "us-east-1c"] - } - } - - # Mock resource behavior - mock_resource "aws_instance" { - defaults = { - id = "i-mock12345" - arn = "arn:aws:ec2:us-east-1:123456789012:instance/i-mock12345" - } - } -} - -# Run block - Defines a test scenario -run "test_name_describing_scenario" { - # Command type: plan (default) or apply - command = plan - - # Override variables for this specific run (optional) - variables { - instance_type = "t3.small" - } - - # Module reference (optional) - Test a specific module - module { - source = "./modules/example" - } - - # Assertion blocks (at least one required) - assert { - condition = aws_instance.main.instance_type == "t3.small" - error_message = "Instance type should be t3.small" - } - - assert { - condition = length(aws_subnet.private) == 3 - error_message = "Expected 3 private subnets" - } -} - -# Negative testing with expect_failures -run "rejects_invalid_input" { - command = plan - - variables { - environment = "invalid" # This should fail validation - } - - # List of objects expected to fail - expect_failures = [ - var.environment - ] -} -``` - -### Test File Organization Patterns - -#### Directory Structure - -Test files **SHOULD** be placed in a `tests/` directory within each module: - -```text -modules/ -└── vpc/ - ├── main.tf - ├── variables.tf - ├── outputs.tf - ├── versions.tf - ├── README.md - └── tests/ - ├── unit.tftest.hcl # Unit tests with mock providers - ├── validation.tftest.hcl # Variable validation tests - └── integration.tftest.hcl # Integration tests (apply) -``` - -#### Alternative: Tests Alongside Configuration - -For simple modules, tests **MAY** be placed alongside configuration files: - -```text -modules/ -└── simple-module/ - ├── main.tf - ├── variables.tf - ├── outputs.tf - ├── simple-module.tftest.hcl # Primary test file - └── simple-module-edge.tftest.hcl # Edge case tests -``` - -#### Naming Conventions - -| Pattern | Purpose | Example | -| --- | --- | --- | -| `unit.tftest.hcl` | Unit tests with mocks | `tests/unit.tftest.hcl` | -| `validation.tftest.hcl` | Variable validation tests | `tests/validation.tftest.hcl` | -| `integration.tftest.hcl` | Apply-based tests | `tests/integration.tftest.hcl` | -| `.tftest.hcl` | Feature-specific tests | `tests/networking.tftest.hcl` | -| `.tftest.hcl` | Module-named tests | `vpc.tftest.hcl` | - -### Writing Unit Tests vs Integration Tests - -#### Unit Tests (command = plan) - -Unit tests use `command = plan` (the default) and **SHOULD**: - -- Use mock providers to avoid real infrastructure -- Execute quickly (seconds, not minutes) -- Test configuration logic, not provider behavior -- Validate variable values and computed attributes -- Verify resource configuration before creation - -```hcl -# tests/unit.tftest.hcl - -mock_provider "aws" { - mock_data "aws_caller_identity" { - defaults = { - account_id = "123456789012" - } - } -} - -run "validates_tag_configuration" { - command = plan # Explicit but optional (plan is default) - - assert { - condition = aws_instance.main.tags["Environment"] == var.environment - error_message = "Environment tag not set correctly" - } -} -``` - -#### Integration Tests (command = apply) - -Integration tests use `command = apply` and **SHOULD**: - -- Create real resources for full validation -- Be run sparingly due to time and cost -- Include proper cleanup (automatic with `terraform test`) -- Be separated from unit tests for selective execution -- Require valid provider credentials - -```hcl -# tests/integration.tftest.hcl - -run "creates_vpc_successfully" { - command = apply # Creates real resources - - assert { - condition = aws_vpc.main.id != "" - error_message = "VPC should be created" - } - - assert { - condition = aws_vpc.main.enable_dns_hostnames == true - error_message = "DNS hostnames should be enabled" - } -} -``` - -#### Separation Recommendation - -| Test Type | Command | Speed | Cost | When to Run | -| --- | --- | --- | --- | --- | -| Unit | `plan` | Fast (seconds) | Free | Every commit, PR | -| Integration | `apply` | Slow (minutes) | Paid | Pre-release, scheduled | - -### Mock Providers and Mock Data - -#### Mock Provider Configuration - -Mock providers replace real provider interactions for unit testing: - -```hcl -mock_provider "aws" { - # Mock data sources - mock_data "aws_availability_zones" { - defaults = { - names = ["us-east-1a", "us-east-1b", "us-east-1c"] - zone_ids = ["use1-az1", "use1-az2", "use1-az3"] - } - } - - mock_data "aws_ami" { - defaults = { - id = "ami-12345678" - name = "mock-ami" - } - } - - # Mock resource computed attributes - mock_resource "aws_instance" { - defaults = { - id = "i-mock12345" - arn = "arn:aws:ec2:us-east-1:123456789012:instance/i-mock12345" - private_ip = "10.0.1.100" - public_ip = "52.1.2.3" - } - } - - mock_resource "aws_vpc" { - defaults = { - id = "vpc-mock12345" - arn = "arn:aws:ec2:us-east-1:123456789012:vpc/vpc-mock12345" - default_network_acl_id = "acl-mock12345" - } - } -} -``` - -#### Override Mock Values Per Run - -```hcl -run "tests_with_custom_mock" { - command = plan - - override_data { - target = data.aws_availability_zones.available - values = { - names = ["us-west-2a", "us-west-2b"] - } - } - - override_resource { - target = aws_instance.main - values = { - private_ip = "10.0.2.50" - } - } - - assert { - condition = length(aws_subnet.private) == 2 - error_message = "Should create 2 subnets for 2 AZs" - } -} -``` - -### Test Assertions and Expected Outcomes - -#### Assertion Block Structure - -```hcl -assert { - condition = - error_message = "" -} -``` - -#### Common Assertion Patterns - -```hcl -# Equality check -assert { - condition = aws_instance.main.instance_type == var.instance_type - error_message = "Instance type mismatch" -} - -# Not empty check -assert { - condition = aws_vpc.main.id != "" - error_message = "VPC ID should not be empty" -} - -# Length check -assert { - condition = length(aws_subnet.private) == 3 - error_message = "Expected 3 private subnets" -} - -# Contains check -assert { - condition = contains(keys(aws_instance.main.tags), "Environment") - error_message = "Instance must have Environment tag" -} - -# Regex match (using can() and regex()) -assert { - condition = can(regex("^vpc-[a-z0-9]+$", aws_vpc.main.id)) - error_message = "VPC ID format is invalid" -} - -# Null check -assert { - condition = aws_instance.main.public_ip != null - error_message = "Instance should have a public IP" -} - -# Boolean attribute -assert { - condition = aws_vpc.main.enable_dns_hostnames == true - error_message = "DNS hostnames should be enabled" -} - -# Output verification -assert { - condition = output.vpc_id != "" && output.vpc_id != null - error_message = "vpc_id output must not be empty" -} -``` - -### Variables and Variable Files in Tests - -#### Global Variables Block - -Set variables that apply to all run blocks: - -```hcl -variables { - environment = "test" - project_name = "testing" - instance_type = "t3.micro" -} -``` - -#### Per-Run Variable Overrides - -Override specific variables for individual test scenarios: - -```hcl -run "tests_production_config" { - variables { - environment = "prod" - instance_type = "t3.large" - } - - assert { - condition = aws_instance.main.instance_type == "t3.large" - error_message = "Production should use t3.large" - } -} -``` - -### Run Blocks and Their Configuration - -#### Run Block Reference - -```hcl -run "descriptive_test_name" { - # Command type (optional, defaults to plan) - command = plan # or apply - - # Override variables (optional) - variables { - key = "value" - } - - # Test a specific module (optional) - module { - source = "./modules/submodule" - } - - # Override mock data/resources (optional) - override_data { - target = data.aws_ami.latest - values = { id = "ami-custom" } - } - - # Assertions (at least one required) - assert { - condition = - error_message = "Message" - } - - # Expected failures for negative testing (optional) - expect_failures = [ - var.environment, - aws_instance.web - ] -} -``` - -#### expect_failures for Negative Testing - -Test that invalid inputs are properly rejected: - -```hcl -run "rejects_invalid_environment" { - command = plan - - variables { - environment = "invalid" # Not in allowed list - } - - expect_failures = [ - var.environment # Expects variable validation to fail - ] -} - -run "rejects_invalid_cidr" { - command = plan - - variables { - vpc_cidr = "not-a-cidr" - } - - expect_failures = [ - var.vpc_cidr - ] -} -``` - ---- - -## Part 3: Test Organization Best Practices - -### Directory Structure Recommendations - -#### Recommended Module Test Structure - -```text -repository/ -├── modules/ -│ ├── vpc/ -│ │ ├── main.tf -│ │ ├── variables.tf -│ │ ├── outputs.tf -│ │ ├── versions.tf -│ │ ├── README.md -│ │ └── tests/ -│ │ ├── unit.tftest.hcl -│ │ ├── validation.tftest.hcl -│ │ └── integration.tftest.hcl -│ ├── ec2-instance/ -│ │ ├── main.tf -│ │ ├── variables.tf -│ │ ├── outputs.tf -│ │ ├── versions.tf -│ │ ├── README.md -│ │ └── tests/ -│ │ ├── unit.tftest.hcl -│ │ └── validation.tftest.hcl -│ └── s3-bucket/ -│ ├── main.tf -│ ├── variables.tf -│ ├── outputs.tf -│ ├── versions.tf -│ ├── README.md -│ └── tests/ -│ └── basic.tftest.hcl -├── environments/ -│ ├── dev/ -│ │ ├── main.tf -│ │ ├── variables.tf -│ │ └── tests/ -│ │ └── environment.tftest.hcl -│ └── prod/ -│ ├── main.tf -│ └── variables.tf -└── templates/ - └── terraform/ - └── Example.tftest.hcl # Recommended template (to be created) -``` - -> **Note:** The `templates/terraform/Example.tftest.hcl` file is a recommended template that **SHOULD** be created following the pattern established by `templates/powershell/Example.Tests.ps1`. See [Part 7: Integration with Repository Patterns](#part-7-integration-with-repository-patterns) for the recommended template content. - -### Naming Conventions for Test Files - -| Convention | Pattern | Description | -| --- | --- | --- | -| Functional | `unit.tftest.hcl`, `integration.tftest.hcl` | Grouped by test type | -| Feature | `networking.tftest.hcl`, `security.tftest.hcl` | Grouped by feature area | -| Scenario | `basic.tftest.hcl`, `complex.tftest.hcl` | Grouped by complexity | -| Module-named | `vpc.tftest.hcl` | Named after the module | - -**Recommendation:** Use functional naming (`unit`, `validation`, `integration`) for consistency across modules. - -### Separating Unit Tests from Integration Tests - -Unit and integration tests **SHOULD** be in separate files: - -| File | Purpose | CI Behavior | -| --- | --- | --- | -| `tests/unit.tftest.hcl` | Fast plan-based tests with mocks | Run on every commit | -| `tests/validation.tftest.hcl` | Variable validation testing | Run on every commit | -| `tests/integration.tftest.hcl` | Apply-based tests with real resources | Run on release/scheduled | - -**Filter by File:** - -```bash -# Run only unit tests -terraform test -filter=tests/unit.tftest.hcl - -# Run only validation tests -terraform test -filter=tests/validation.tftest.hcl -``` - -### Module Testing vs Root Module Testing - -#### Testing Modules in Isolation - -Each module **SHOULD** have its own test suite: - -```hcl -# modules/vpc/tests/unit.tftest.hcl - -variables { - cidr_block = "10.0.0.0/16" - environment = "test" -} - -mock_provider "aws" { - mock_data "aws_availability_zones" { - defaults = { names = ["us-east-1a", "us-east-1b"] } - } -} - -run "creates_vpc_with_specified_cidr" { - assert { - condition = aws_vpc.main.cidr_block == "10.0.0.0/16" - error_message = "VPC CIDR mismatch" - } -} -``` - -#### Testing Root Module Configurations - -Root modules (environments) **MAY** have tests verifying composition: - -```hcl -# environments/dev/tests/environment.tftest.hcl - -run "dev_environment_uses_small_instances" { - assert { - condition = module.compute.instance_type == "t3.micro" - error_message = "Dev should use t3.micro instances" - } -} -``` - -### Test Fixtures and Helper Modules - -#### Creating Reusable Test Fixtures - -For complex test scenarios, create fixture modules: - -```text -modules/ -└── vpc/ - └── tests/ - ├── unit.tftest.hcl - └── fixtures/ - └── mock-vpc/ - ├── main.tf - └── outputs.tf -``` - -```hcl -# tests/fixtures/mock-vpc/main.tf -output "vpc_id" { - value = "vpc-fixture12345" -} - -output "subnet_ids" { - value = ["subnet-1", "subnet-2", "subnet-3"] -} -``` - ---- - -## Part 4: CI Integration Recommendations - -### GitHub Actions Workflow Design - -Following the patterns established in the existing `terraform-ci.yml` workflow, test execution **SHOULD** be integrated as a dedicated job. - -#### Workflow Structure - -The existing `terraform-ci.yml` already includes a `test` job with appropriate design. Key elements: - -```yaml -test: - name: Test - needs: validate - runs-on: ubuntu-latest - - steps: - - name: Checkout repository - uses: actions/checkout@v4 - - - name: Setup Terraform - uses: hashicorp/setup-terraform@v3 - with: - terraform_version: "1.6.0" - - - name: Check for test files - id: check-tests - run: | - if find . -name '*.tftest.hcl' | grep -q .; then - echo "has_tests=true" >> $GITHUB_OUTPUT - else - echo "has_tests=false" >> $GITHUB_OUTPUT - fi - - - name: Run Terraform Tests - if: steps.check-tests.outputs.has_tests == 'true' - run: terraform test -verbose -``` - -### Job Organization and Dependencies - -The recommended job dependency chain: - -```text -format → validate → lint (parallel with test) → security (optional) - ↘ ↗ - ----→ test --------------- -``` - -| Job | Depends On | Purpose | -| --- | --- | --- | -| format | none | Check formatting first | -| validate | format | Validate after formatting passes | -| lint | validate | Lint valid configurations | -| test | validate | Test valid configurations | -| security | validate | Security scan (optional, parallel with lint) | - -### Matrix Testing Across Terraform Versions - -For modules supporting multiple Terraform versions: - -```yaml -test: - name: Test (Terraform ${{ matrix.terraform }}) - runs-on: ubuntu-latest - strategy: - matrix: - terraform: ['1.6.0', '1.7.0', '1.8.0'] - fail-fast: false - - steps: - - uses: actions/checkout@v4 - - - uses: hashicorp/setup-terraform@v3 - with: - terraform_version: ${{ matrix.terraform }} - - - name: Run Tests - run: terraform test -verbose -``` - -**Recommendation:** For this template repository, pin to a single version (1.6.0) for simplicity. Users can add matrix testing as needed. - -### Provider Authentication for Integration Tests - -#### Mock-First Approach (Recommended for Unit Tests) - -Use mock providers to avoid authentication requirements: - -```hcl -# tests/unit.tftest.hcl -mock_provider "aws" { - # No credentials needed -} -``` - -#### Real Provider Authentication (Integration Tests) - -When integration tests are required: - -```yaml -- name: Configure AWS Credentials - uses: aws-actions/configure-aws-credentials@v4 - with: - role-to-assume: ${{ secrets.AWS_ROLE_ARN }} - aws-region: us-east-1 - -- name: Run Integration Tests - run: terraform test -filter=tests/integration.tftest.hcl -``` - -**Security Recommendations:** - -- Use OIDC authentication (role-to-assume) over static credentials -- Limit IAM role permissions to minimum required for tests -- Use separate AWS accounts for testing -- Never store credentials in code or logs - -### Secrets and Environment Variable Handling - -```yaml -env: - TF_VAR_environment: "ci-test" - TF_IN_AUTOMATION: "true" - -steps: - - name: Run Tests with Variables - env: - TF_VAR_api_key: ${{ secrets.TEST_API_KEY }} - run: terraform test -verbose -``` - -**Best Practices:** - -- Use `TF_VAR_` prefix for Terraform variables -- Set `TF_IN_AUTOMATION=true` to suppress interactive prompts -- Use GitHub Secrets for sensitive values -- Never echo or log secret values - -### Test Result Reporting - -```yaml -- name: Run Terraform Tests - id: test - run: | - terraform test -verbose 2>&1 | tee test-output.txt - echo "exit_code=${PIPESTATUS[0]}" >> $GITHUB_OUTPUT - -- name: Upload Test Results - if: always() - uses: actions/upload-artifact@v4 - with: - name: terraform-test-results - path: test-output.txt -``` - ---- - -## Part 5: Test Writing Guidelines - -### What to Test in Terraform - -| Category | What to Test | Example | -| --- | --- | --- | -| **Variable validation** | Custom validation rules work correctly | Invalid environment rejected | -| **Computed values** | Locals and expressions compute correctly | CIDR calculations, name generation | -| **Resource configuration** | Resources have expected attributes | Instance type, tags, security settings | -| **Output values** | Outputs contain expected values | VPC ID not empty, ARN format valid | -| **Conditional logic** | Count/for_each behave correctly | Zero instances when disabled | -| **Module interfaces** | Modules accept inputs and produce outputs | Module composition works | -| **Edge cases** | Boundary conditions handled | Empty lists, null values | -| **Negative scenarios** | Invalid inputs are rejected | Bad CIDR, invalid region | - -### What NOT to Test - -- **Cloud provider behavior:** Don't test if AWS actually creates an EC2 instance -- **Terraform core functionality:** Don't test if `count` works -- **External service availability:** Don't test if the AWS API is up -- **Provider bugs:** Report to provider maintainers instead - -### Test Isolation and Idempotency - -Tests **MUST** be isolated and idempotent: - -```hcl -# Good: Uses unique names via variables or locals -variables { - name_prefix = "test" -} - -run "creates_unique_resources" { - variables { - # Note: Using timestamp() can affect test reproducibility. - # For CI, consider using a static prefix or run-specific identifier. - name_prefix = "ci-test-unique" - } - - # ... -} -``` - -**Principles:** - -- Each test should be independent of others -- Tests should produce same results when run multiple times -- Use unique resource names to avoid conflicts -- Clean up resources after tests (automatic with `terraform test`) - -### Handling Stateful Resources in Tests - -For unit tests, use mocks to avoid state issues: - -```hcl -mock_provider "aws" { - mock_resource "aws_s3_bucket" { - defaults = { - id = "mock-bucket" - arn = "arn:aws:s3:::mock-bucket" - } - } -} -``` - -For integration tests, rely on `terraform test` automatic cleanup: - -- Resources are destroyed after each test file -- Use `command = apply` only when necessary -- Design tests to minimize resource creation - -### Cleanup and Resource Destruction - -`terraform test` automatically handles cleanup: - -1. After each `run` block with `command = apply`, resources are preserved -2. After all runs in a file complete, resources are destroyed -3. If a test fails, cleanup still occurs - -**For manual cleanup:** - -```bash -# If automatic cleanup fails -cd modules/vpc -terraform destroy -auto-approve -``` - -### Testing Modules Independently - -Each module **SHOULD** have tests that validate its interface: - -```hcl -# modules/ec2-instance/tests/unit.tftest.hcl - -variables { - instance_type = "t3.micro" - ami_id = "ami-12345678" - subnet_id = "subnet-mock" -} - -mock_provider "aws" {} - -run "uses_specified_instance_type" { - assert { - condition = aws_instance.main.instance_type == var.instance_type - error_message = "Instance type should match input variable" - } -} - -run "accepts_custom_tags" { - variables { - additional_tags = { - Team = "Engineering" - } - } - - assert { - condition = aws_instance.main.tags["Team"] == "Engineering" - error_message = "Custom tags should be applied" - } -} -``` - ---- - -## Part 6: Content Specification for terraform.instructions.md - -This section specifies what testing guidance should be **embedded directly in the Terraform Copilot instructions file**, following the pattern established by the PowerShell instructions file's "Testing with Pester" section. - -### Section Structure for "Testing with Terraform Test" - -The current `terraform.instructions.md` already includes a "Testing with Terraform Test" section (starting at line 1356). This section provides the specification for its content. - -#### Recommended Table of Contents Entry - -```markdown -- [Testing with Terraform Test](#testing-with-terraform-test) -``` - -#### Recommended Subsections - -The "Testing with Terraform Test" section **SHOULD** include these subsections (most already exist): - -1. Test File Naming -2. Test File Location -3. Test Structure -4. Test Assertions -5. Testing Variable Validation -6. Testing Outputs -7. Mock Providers -8. Unit Tests -9. Integration Tests -10. Running Tests -11. What to Test - -### Quick Reference Checklist Entries for Testing - -The following entries **SHOULD** appear in the Quick Reference Checklist under a "### Testing" heading. These entries are currently present in the terraform.instructions.md: - -```markdown -### Testing - -- **[Test]** Test files **MUST** use `.tftest.hcl` extension → [Test File Naming](#test-file-naming) -- **[Test]** Test files **SHOULD** be in a `tests/` directory → [Test File Location](#test-file-location) -- **[Test]** Tests **MUST** include at least one `run` block → [Test Structure](#test-structure) -- **[Test]** Each `run` block **MUST** include at least one `assert` → [Test Assertions](#test-assertions) -- **[Test]** Variable validation **SHOULD** be tested → [Testing Variable Validation](#testing-variable-validation) -- **[Test]** Unit tests **SHOULD** use `command = plan` → [Unit Tests](#unit-tests) -- **[Test]** Integration tests **MAY** use `command = apply` → [Integration Tests](#integration-tests) -``` - -#### Additional Checklist Entries to Consider - -```markdown -- **[Module]** Modules **SHOULD** include corresponding Terraform tests → [Module Tests](#module-tests) -- **[Test]** Mock providers **SHOULD** be used for unit tests → [Mock Providers](#mock-providers) -- **[Test]** Negative test cases **SHOULD** use `expect_failures` → [Testing Variable Validation](#testing-variable-validation) -``` - -### RFC 2119 Keyword Usage - -The testing section **SHOULD** use RFC 2119 keywords consistently: - -| Keyword | Testing Requirement | -| --- | --- | -| **MUST** | Test file extension (`.tftest.hcl`), run block requirement, assert requirement | -| **SHOULD** | Test file location, unit test command, mock provider usage, variable validation testing | -| **MAY** | Integration test usage, test organization alternatives | - -### Detailed Section Content Specification - -The "Testing with Terraform Test" section **SHOULD** include the following content (adapting the patterns from the PowerShell testing section): - -#### 1. Introduction Paragraph - -```markdown -Terraform's native test framework (introduced in Terraform 1.6) provides a way to validate configurations without external testing tools. This section documents testing conventions that integrate with the coding standards in this guide. - -> **Note:** Terraform tests require Terraform 1.6.0 or later. For older Terraform versions, consider Terratest or other external testing frameworks. -``` - -#### 2. Test File Naming - -Document the required file extension and naming patterns. - -#### 3. Test File Location - -Include directory structure diagrams showing both preferred and alternative locations. - -#### 4. Test Structure - -Provide the HCL block reference table and basic test structure example. - -#### 5. Test Assertions - -Show assertion patterns with examples for common scenarios. - -#### 6. Testing Variable Validation - -Demonstrate `expect_failures` pattern for negative testing. - -#### 7. Mock Providers - -Show how to configure mock providers for unit testing. - -#### 8. Unit Tests vs Integration Tests - -Clearly distinguish command types and when to use each. - -#### 9. Running Tests - -Provide CLI commands for local and CI execution. - -#### 10. What to Test - -Include a table of what to test and what not to test. - -### Example Content Structure - -The following demonstrates the style and structure that **SHOULD** be followed (already present in terraform.instructions.md): - -```markdown -## Testing with Terraform Test - -Terraform's native test framework (introduced in Terraform 1.6) provides a way to validate configurations without external testing tools. - -### Test File Naming - -Test files **MUST** use the `.tftest.hcl` extension: - -- `basic.tftest.hcl` -- `validation.tftest.hcl` -- `integration.tftest.hcl` - -### Test File Location - -Test files **SHOULD** be placed in a `tests/` directory within the module: - -```text -modules/ -└── vpc/ - ├── main.tf - ├── variables.tf - └── tests/ - └── basic.tftest.hcl -``` - -[Continue with remaining sections...] - ---- - -## Part 7: Integration with Repository Patterns - -### Updating the Testing Tools Table in .github/copilot-instructions.md - -The `.github/copilot-instructions.md` file includes a Testing Tools table that **SHOULD** include Terraform. The current table already includes Terraform: - -```markdown -| Language | Framework | Configuration | Test Location | -| --- | --- | --- | --- | -| Python | pytest | `pyproject.toml` (`[tool.pytest.ini_options]`) | `tests/` | -| PowerShell | Pester 5.x | Inline in `.github/workflows/powershell-ci.yml` | `tests/PowerShell/` | -| Terraform | Terraform Test (requires Terraform 1.6+) | Built-in | `modules/*/tests/` or `tests/` | -``` - -**Verification:** The Terraform entry is already present in the current `.github/copilot-instructions.md`. - -### Template Test File Creation - -A template test file **SHOULD** be created at `templates/terraform/Example.tftest.hcl` following the pattern established by `templates/powershell/Example.Tests.ps1`. - -#### What the Template Should Demonstrate - -1. **File header comments:** Purpose, prerequisites, usage instructions -2. **Global variables block:** Setting test variables -3. **Mock provider configuration:** AWS mock with data sources and resources -4. **Basic run block:** Simple assertion example -5. **Variable validation testing:** Using `expect_failures` -6. **Output verification:** Testing output values -7. **Multiple assertions:** Showing assertion patterns -8. **Unit vs integration examples:** Both `plan` and `apply` commands - -#### Recommended Template Structure - -> **Implementation Note:** This template **SHOULD** be created at `templates/terraform/Example.tftest.hcl` as part of the Terraform testing implementation. The `templates/terraform/` directory does not currently exist and would need to be created. - -```hcl -# Example.tftest.hcl -# -# TEMPLATE FILE: Copy this file to modules//tests/ and customize. -# -# This file demonstrates Terraform test patterns including: -# - Global variables for test configuration -# - Mock providers for unit testing -# - Basic assertions with condition/error_message -# - Variable validation testing with expect_failures -# - Output verification -# - Unit tests (command = plan) vs integration tests (command = apply) -# -# Prerequisites: -# - Terraform 1.6.0 or later -# -# Usage (from module directory): -# terraform test -# terraform test -verbose -# terraform test -filter=tests/Example.tftest.hcl -# -# For complete Terraform test documentation: -# https://developer.hashicorp.com/terraform/cli/commands/test - -# ============================================================================== -# Global Variables -# ============================================================================== -# Variables defined here apply to all run blocks unless overridden. - -variables { - environment = "test" - instance_type = "t3.micro" - # Add your module's required variables here -} - -# ============================================================================== -# Mock Provider Configuration -# ============================================================================== -# Mock providers enable unit testing without real cloud credentials. - -mock_provider "aws" { - # Mock data sources - mock_data "aws_availability_zones" { - defaults = { - names = ["us-east-1a", "us-east-1b", "us-east-1c"] - } - } - - mock_data "aws_caller_identity" { - defaults = { - account_id = "123456789012" - } - } - - # Mock resource computed attributes - mock_resource "aws_instance" { - defaults = { - id = "i-mock12345" - arn = "arn:aws:ec2:us-east-1:123456789012:instance/i-mock12345" - private_ip = "10.0.1.100" - } - } -} - -# ============================================================================== -# Unit Tests (command = plan) -# ============================================================================== -# Unit tests validate configuration logic without creating resources. - -run "validates_instance_type" { - command = plan - - assert { - condition = aws_instance.main.instance_type == var.instance_type - error_message = "Instance type should match input variable" - } -} - -run "applies_environment_tag" { - command = plan - - assert { - condition = aws_instance.main.tags["Environment"] == var.environment - error_message = "Environment tag should be set" - } -} - -# ============================================================================== -# Variable Validation Tests -# ============================================================================== -# Test that invalid inputs are properly rejected. - -run "rejects_invalid_environment" { - command = plan - - variables { - environment = "invalid" # Should fail validation - } - - expect_failures = [ - var.environment - ] -} - -# ============================================================================== -# Output Verification -# ============================================================================== -# Verify outputs contain expected values. - -run "outputs_instance_id" { - command = plan - - assert { - condition = output.instance_id != null - error_message = "instance_id output must not be null" - } -} - -# ============================================================================== -# Integration Tests (command = apply) -# ============================================================================== -# Integration tests create real resources. Use sparingly. -# Uncomment when you have valid provider credentials. -# -# run "creates_instance_successfully" { -# command = apply -# -# assert { -# condition = aws_instance.main.id != "" -# error_message = "Instance should be created" -# } -# } -``` - -### Pre-commit Hooks for Test Validation - -Consider adding test-related pre-commit hooks: - -```yaml -# .pre-commit-config.yaml additions (recommendations only) -- repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.105.0 - hooks: - # Existing hooks... - - id: terraform_validate - # Validates test files as part of validation -``` - -**Note:** `terraform fmt` already handles `.tftest.hcl` files. No additional hooks are required for test file formatting. - -### README.md Updates - -The repository README **SHOULD** include a testing section: - -```markdown -## Testing - -### Terraform - -Terraform modules include unit tests using the native Terraform test framework: - -```bash -# Run all tests -terraform test -verbose - -# Run specific test file -terraform test -filter=tests/unit.tftest.hcl -``` - -Tests are located in `modules/*/tests/` directories. - ---- - -## Part 8: Advanced Topics - -### Testing with Multiple Providers - -#### Multi-Provider Test Configuration - -```hcl -# tests/multi-provider.tftest.hcl - -mock_provider "aws" { - alias = "primary" - mock_data "aws_region" { - defaults = { name = "us-east-1" } - } -} - -mock_provider "aws" { - alias = "secondary" - mock_data "aws_region" { - defaults = { name = "us-west-2" } - } -} - -run "uses_correct_regions" { - assert { - condition = aws_vpc.primary.tags["Region"] == "us-east-1" - error_message = "Primary VPC should be in us-east-1" - } - - assert { - condition = aws_vpc.secondary.tags["Region"] == "us-west-2" - error_message = "Secondary VPC should be in us-west-2" - } -} -``` - -### Testing Provider-Specific Resources - -#### AWS-Specific Testing Considerations - -```hcl -mock_provider "aws" { - # Mock common AWS data sources - mock_data "aws_caller_identity" { - defaults = { account_id = "123456789012" } - } - - mock_data "aws_partition" { - defaults = { partition = "aws" } - } - - mock_data "aws_region" { - defaults = { name = "us-east-1" } - } -} -``` - -#### Azure-Specific Testing Considerations - -```hcl -mock_provider "azurerm" { - mock_data "azurerm_client_config" { - defaults = { - tenant_id = "00000000-0000-0000-0000-000000000000" - subscription_id = "00000000-0000-0000-0000-000000000000" - } - } -} -``` - -#### GCP-Specific Testing Considerations - -```hcl -mock_provider "google" { - mock_data "google_project" { - defaults = { - project_id = "mock-project-id" - number = "123456789012" - } - } -} -``` - -#### Generic Patterns - -When writing provider-agnostic tests: - -1. Mock all data sources your configuration uses -2. Provide realistic default values for computed attributes -3. Test resource configurations, not provider behavior -4. Use variables for provider-specific values (regions, zones, etc.) - -### Continuous Testing Strategies - -#### When to Run Full vs Partial Test Suites - -| Trigger | Test Scope | Rationale | -| --- | --- | --- | -| PR/Commit | Unit tests only | Fast feedback, no cost | -| Merge to main | Unit + validation | Ensure quality gate | -| Pre-release | Full suite including integration | Comprehensive validation | -| Scheduled (weekly) | Full integration | Catch drift, provider changes | - -#### Scheduled Testing for Drift Detection - -```yaml -# Scheduled workflow for comprehensive testing -on: - schedule: - - cron: '0 0 * * 0' # Weekly on Sunday - -jobs: - integration-tests: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: hashicorp/setup-terraform@v3 - - run: terraform test # Full suite -``` - -### Test Coverage Considerations - -#### What "Coverage" Means for Terraform - -Unlike code coverage for programming languages, Terraform test coverage refers to: - -- **Configuration coverage:** Percentage of resources/modules with tests -- **Validation coverage:** Percentage of validation rules tested -- **Output coverage:** Percentage of outputs verified -- **Scenario coverage:** Percentage of use cases tested - -#### Measuring Test Effectiveness - -| Metric | How to Measure | -| --- | --- | -| Resource coverage | Count resources with tests / total resources | -| Validation coverage | Count validation rules tested / total validation rules | -| Output coverage | Count outputs verified / total outputs | -| Branch coverage | Count conditional paths tested / total conditional paths | - -#### Critical Paths to Cover - -Tests **SHOULD** prioritize: - -1. **Security configurations:** Encryption, IAM policies, network security -2. **Variable validation rules:** All custom validations should be tested -3. **Complex expressions:** Locals with calculations or conditionals -4. **Module interfaces:** All inputs and outputs -5. **Edge cases:** Empty lists, null values, boundary conditions - -### Performance Optimization for Test Suites - -#### Parallelization Strategies - -While `terraform test` runs files sequentially, you can parallelize at the CI level: - -```yaml -strategy: - matrix: - module: [vpc, ec2-instance, s3-bucket] - -steps: - - run: | - cd modules/${{ matrix.module }} - terraform test -verbose -``` - -#### Caching for Faster Tests - -Provider caching reduces initialization time: - -```yaml -- name: Cache Terraform Providers - uses: actions/cache@v4 - with: - path: ~/.terraform.d/plugin-cache - key: terraform-providers-${{ hashFiles('**/.terraform.lock.hcl') }} - -- name: Configure Provider Cache - run: | - mkdir -p ~/.terraform.d/plugin-cache - echo "plugin_cache_dir = \"$HOME/.terraform.d/plugin-cache\"" > ~/.terraformrc -``` - -#### Minimizing Provider Initialization Overhead - -1. **Use mock providers** for unit tests to skip provider download -2. **Cache provider plugins** across CI runs -3. **Share initialization** across tests in the same directory -4. **Limit integration tests** to essential scenarios - ---- - -## Assumptions and Validation Notes - -This section lists Terraform features and behaviors that **SHOULD** be verified against current official documentation before implementation. - -### Features to Verify - -| Feature | Documentation Source | Notes | -| --- | --- | --- | -| `terraform test` command | [Terraform Test Command](https://developer.hashicorp.com/terraform/cli/commands/test) | Verify current syntax and options | -| `.tftest.hcl` syntax | [Test File Reference](https://developer.hashicorp.com/terraform/language/tests) | Verify block types and attributes | -| `mock_provider` block | [Mock Providers](https://developer.hashicorp.com/terraform/language/tests#mocking-providers) | Verify mock capabilities | -| `expect_failures` | [Expecting Failures](https://developer.hashicorp.com/terraform/language/tests#expecting-failures) | Verify syntax for negative tests | -| `override_data`/`override_resource` | Mock override documentation | Verify per-run override syntax | -| CI action versions | GitHub Marketplace | Verify `hashicorp/setup-terraform@v3` is current | - -### Version Compatibility - -| Terraform Version | Test Framework Support | -| --- | --- | -| < 1.6.0 | Not supported (use Terratest) | -| 1.6.0 | Initial release of test framework | -| 1.7.0+ | Enhanced mock provider capabilities | - -### Known Limitations - -1. **Mock provider completeness:** Not all provider behaviors can be mocked -2. **Parallel execution:** Test files run sequentially -3. **State handling:** Integration tests manage state internally -4. **Provider credential requirements:** Integration tests require real credentials - ---- - -## Summary of Recommended Implementation Steps - -### CI/Infrastructure Implementation - -1. **Verify existing workflow:** The `terraform-ci.yml` already includes a `test` job with proper configuration -2. **Create template test file:** Add `templates/terraform/Example.tftest.hcl` following the structure in Part 7 -3. **Add sample module tests:** Create example `.tftest.hcl` files in any sample modules -4. **Update documentation:** Ensure README includes testing section -5. **Configure caching:** Verify provider caching is configured for test jobs - -### terraform.instructions.md Content - -1. **Verify existing section:** The "Testing with Terraform Test" section exists and covers most requirements -2. **Review Quick Reference Checklist:** Ensure all testing checklist items are present (currently they are) -3. **Add module test guidance:** Consider adding a checklist item for module testing -4. **Verify examples are complete:** Ensure all code examples are current and correct - -### Template File Creation - -1. **Create `templates/terraform/` directory** if it doesn't exist -2. **Create `Example.tftest.hcl`** following the template in Part 7 -3. **Include comprehensive examples:** Mock providers, assertions, validation tests, output tests -4. **Add usage documentation:** Header comments explaining how to use the template - -### Pre-commit and Tooling - -1. **Verify `.tftest.hcl` formatting:** Confirm `terraform fmt` handles test files (it does) -2. **No additional hooks required:** Existing pre-commit configuration is sufficient -3. **Consider CI enhancements:** Optional matrix testing for multiple Terraform versions - -### Documentation Updates - -1. **Update README.md:** Add testing section with commands -2. **Verify `.github/copilot-instructions.md`:** Terraform is already in the Testing Tools table -3. **Cross-reference guides:** Link this guide from terraform.instructions.md if desired - -### Validation Steps - -1. **Review official Terraform documentation** for any syntax changes -2. **Test example configurations** locally before committing -3. **Verify CI workflow** passes with test job enabled -4. **Run pre-commit checks** on all new files diff --git a/package-lock.json b/package-lock.json index 6d764ae..7385cd5 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,11 +1,11 @@ { - "name": "copilot-repo-template", + "name": "macoslab", "version": "1.0.0", "lockfileVersion": 3, "requires": true, "packages": { "": { - "name": "copilot-repo-template", + "name": "macoslab", "version": "1.0.0", "license": "MIT", "devDependencies": { diff --git a/package.json b/package.json index 4012693..3039ad9 100644 --- a/package.json +++ b/package.json @@ -1,16 +1,19 @@ { - "name": "copilot-repo-template", + "name": "macoslab", "version": "1.0.0", - "description": "Template repository with Copilot instructions and code quality tooling", + "description": "PowerShell tooling and Markdown docs for reproducible Intune-managed macOS VM labs", "private": true, "scripts": { - "lint:md": "markdownlint-cli2 \"**/*.md\" \"#node_modules\"", + "lint:md": "markdownlint-cli2 \"**/*.md\" \"#node_modules\" \"#.venv\" \"#**/.venv\"", "lint:md:nested": "node .github/scripts/lint-nested-markdown.js" }, "keywords": [ - "template", - "copilot", - "linting" + "macos", + "intune", + "powershell", + "parallels", + "utm", + "pester" ], "author": "Frank Lesniak", "license": "MIT", diff --git a/pyproject.toml b/pyproject.toml deleted file mode 100644 index 5f3b6bf..0000000 --- a/pyproject.toml +++ /dev/null @@ -1,69 +0,0 @@ -# pyproject.toml for the copilot-repo-template repository -# -# This file provides a minimal Python project configuration that allows the CI -# workflows (.github/workflows/python-ci.yml) to run successfully. -# -# This is a TEMPLATE REPOSITORY. When you create a new repository from this -# template, you should customize this file for your project. See -# templates/python/pyproject.toml for a more comprehensive example with -# additional tooling configuration (Black, Ruff, etc.). - -[build-system] -requires = ["setuptools>=82.0.1", "wheel"] -build-backend = "setuptools.build_meta" - -[project] -name = "copilot-repo-template" -version = "0.1.0" -description = "Template repository providing GitHub Copilot instructions and CI configurations" -readme = "README.md" -# Python Version Policy: -# Always require a Python version that is currently receiving bugfixes. -# Check https://devguide.python.org/versions/ for current version status. -# Update this when upstream support changes (typically annually around October). -requires-python = ">=3.13" -license = "MIT" -authors = [ - { name = "Frank Lesniak" } -] -keywords = ["template", "copilot", "github"] -classifiers = [ - "Development Status :: 3 - Alpha", - "Programming Language :: Python :: 3", - "Programming Language :: Python :: 3.13", -] - -# Runtime dependencies -dependencies = [] - -# Development dependencies -# These are the minimum required for CI workflows to pass. -# The template in templates/python/pyproject.toml includes additional -# tools like Black and Ruff for comprehensive development setups. -[project.optional-dependencies] -dev = [ - "pytest>=9.0.3", - "pytest-cov>=7.1.0", - "mypy>=1.20.0", - "ruff>=0.15.10", - # check-jsonschema is required by tests/test_schema_examples.py to - # validate files under schemas/examples//{valid,invalid}/. - "check-jsonschema>=0.33.0", -] - -[tool.setuptools.packages.find] -where = ["src"] - -[tool.mypy] -# Update python_version when minimum Python version changes. -# mypy uses a dotted version string (e.g., "3.13" for Python 3.13). -python_version = "3.13" -warn_return_any = true -warn_unused_configs = true -# Start permissive for template; projects can tighten as they mature -disallow_untyped_defs = false - -[tool.pytest.ini_options] -testpaths = ["tests"] -python_files = ["test_*.py"] -python_functions = ["test_*"] diff --git a/schemas/README.md b/schemas/README.md index 7d3e20d..92357a1 100644 --- a/schemas/README.md +++ b/schemas/README.md @@ -6,222 +6,41 @@ - **Status:** Active - **Owner:** Repository Maintainers -- **Last Updated:** 2026-05-03 -- **Scope:** Conventions for JSON Schemas that describe load-bearing JSON and YAML files in this repository, plus a clearly removable worked example (`example-config.schema.json` with valid and invalid example data) wired into pre-commit and data CI to demonstrate the schema-validation pipeline end to end. -- **Related:** [JSON Authoring Standards](../.github/instructions/json.instructions.md), [YAML Authoring Standards](../.github/instructions/yaml.instructions.md), [Repository Copilot Instructions](../.github/copilot-instructions.md), [Template Design Decisions — Schema Location at Repository Root](../.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-schema-location-at-repository-root), [Template Design Decisions — Schema Validation Tiers](../.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-schema-validation-tiers), [Template Design Decisions — Built-in Schema Validation for Real Load-Bearing Configuration Files](../.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-built-in-schema-validation-for-real-load-bearing-configuration-files), [Template Design Decisions — `additionalProperties` Policy](../.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-additionalproperties-policy), [Template Design Decisions — Testing Beyond Linting for JSON/YAML](../.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-testing-beyond-linting-for-jsonyaml) +- **Last Updated:** 2026-05-05 +- **Scope:** Schema conventions and the temporary worked-example schema retained during Phase 0 bootstrap. The real macOSLab evidence-bundle schema is deferred to Phase 7. +- **Related:** [JSON authoring standards](../.github/instructions/json.instructions.md), [YAML authoring standards](../.github/instructions/yaml.instructions.md), [Phase 7 TODO](../TODO-Phase-07-Evidence-Pipeline.md) -## Purpose +This directory contains JSON Schemas for load-bearing JSON and YAML contracts. -This directory contains JSON Schemas for load-bearing JSON and YAML files in this repository. A "load-bearing" file is one whose shape is depended on by build, deploy, runtime, release automation, or downstream consumers, such that a malformed value would cause incorrect behavior. +During Phase 0, macOSLab keeps the template worked example so the schema-validation toolchain remains exercised: -Schemas live at the repository root (under `schemas/`, not `.github/schemas/`) so they are discoverable to IDEs, schema validators, and downstream consumers, and so projects that do not use schema-backed data files can opt out by deleting this directory. +- Schema: [example-config.schema.json](example-config.schema.json) +- Valid examples: [examples/example-config/valid/](examples/example-config/valid/) +- Invalid reference fixtures: [examples/example-config/invalid/](examples/example-config/invalid/) -## Template Portability +The valid examples are checked by the `Validate example-config valid examples` pre-commit hook. The schema is checked by the `Self-validate example-config schema` pre-commit hook. The invalid fixtures are retained only as reference material during Phase 0; they are not wired into a normal pre-commit hook because they are expected to fail validation. -This template provides `schemas/` as a convention for repositories that adopt schema-backed JSON or YAML contracts. Downstream repositories MAY delete `schemas/` (including this `README.md`) if they do not use schema-backed data files. +## Future Evidence Schema -## Conventions +The repository specification defines the real evidence-bundle schema in Section 25 of [macOSLab Repository Specification](../docs/planning/macOS-imaging-08c-repo-spec-final.md#25-evidence-bundle-schema). Phase 7 must replace this worked example with that real schema and update `.pre-commit-config.yaml`, data-file CI, examples, and documentation in the same change. -### Draft +That work is tracked in [TODO-Phase-07-Evidence-Pipeline.md](../TODO-Phase-07-Evidence-Pipeline.md). -- Schemas SHOULD use [JSON Schema Draft 2020-12](https://json-schema.org/draft/2020-12/schema) unless a specific consumer requires another draft (for example, an OpenAPI version pinned to Draft-07). -- The chosen draft SHOULD be stated in the schema's `$schema` field, and any deviation from Draft 2020-12 SHOULD be called out in the schema's `description` or in this `README.md`. +## Authoring Rules -### File Naming - -- Schema files SHOULD use the suffix `.schema.json` (for example, `schemas/feature-flags.schema.json`). -- Filenames SHOULD use lowercase `kebab-case`. - -### Required Schema Metadata - -Every schema SHOULD include the following top-level keywords: - -- `$schema` — the JSON Schema draft URI. -- `$id` — a stable, absolute URI that identifies the schema. -- `title` — a short, human-readable name for the contract. -- `description` — a concise explanation of what the schema describes and which files it applies to. - -### Object Schemas - -Schemas whose root type is `object` SHOULD define: - -- `type: "object"` -- `required` — the list of properties that MUST be present. -- `properties` — the typed shape of each known property. - -### Open vs. Closed Contracts - -- Project-owned closed contracts SHOULD set `"additionalProperties": false` so that unknown keys are caught early. -- Ecosystem-mirroring schemas (schemas that describe an external format the project does not own, for example a third-party config) MAY leave additional properties open and SHOULD document why in the schema's `description` or in this `README.md`. +- Schemas SHOULD use JSON Schema Draft 2020-12 unless a concrete consumer requires another draft. +- Schema files SHOULD use the `.schema.json` suffix and lowercase kebab-case names. +- Project-owned object schemas SHOULD set `"additionalProperties": false` unless the schema intentionally mirrors an external open contract. +- Examples MUST use synthetic data and MUST NOT contain secrets, tenant identifiers, tokens, recovery keys, production policy names, device IDs, or personal data. +- New schema-backed file families SHOULD add narrowly scoped `check-jsonschema` hooks. Do not add placeholder hooks for schemas that do not exist. ## Validation -Schema-backed files are validated by pre-commit and the dedicated data-file CI workflow ([`.github/workflows/data-ci.yml`](../.github/workflows/data-ci.yml)). This template ships a worked example (see [Worked Example](#worked-example) below) so the validation pipeline is exercised end to end out of the box. Downstream repositories that do not use schema-backed data files SHOULD remove the worked example using the [Downstream Removal Checklist](#downstream-removal-checklist). See the JSON authoring standards for the schema-validation policy and tier guidance. - -### Schema Categories - -This repository distinguishes two schema categories. The distinction matters for where schemas live, how they are tested, and how they are wired into pre-commit and CI. - -1. **Project-owned schemas.** - - Stored under `schemas/` in this repository. - - MAY include valid and invalid example fixtures under `schemas/examples//{valid,invalid}/`. - - Tested by [`tests/test_schema_examples.py`](../tests/test_schema_examples.py), which auto-discovers schema/example pairs and asserts that valid examples pass and invalid examples fail. - - Wired into pre-commit by adding a `check-jsonschema` hook that points at the schema with `--schemafile schemas/.schema.json` and an anchored `files:` pattern matching the file family the schema covers. - - The [Worked Example](#worked-example) below is the canonical illustration of this category. - -2. **External built-in schemas.** - - Referenced through `check-jsonschema --builtin-schema vendor.` against schemas that ship inside the pinned `check-jsonschema` release. - - **Not vendored** into this repository. Schema content tracks `check-jsonschema` upstream releases and is updated through the Dependabot `pre-commit` ecosystem. - - Used for selected real, load-bearing repository configuration files where the external schema is mature and validation is low-noise. - - See the [Built-in Schema Validation for Real Load-Bearing Configuration Files](../.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-built-in-schema-validation-for-real-load-bearing-configuration-files) ADR for the policy, the full list of selected files, and the explicit "Evaluated but deferred" negative-space record. - -The two categories are complementary. A downstream repository MAY use either, both, or neither. - -### Real Repository Configuration Files Validated Through Built-in Schemas - -The following real, load-bearing repository configuration files are validated by default through `check-jsonschema --builtin-schema ...` hooks in [`.pre-commit-config.yaml`](../.pre-commit-config.yaml): - -| File | Built-in schema identifier | -| --- | --- | -| [`.github/dependabot.yml`](../.github/dependabot.yml) | `vendor.dependabot` | - -If a downstream repository deletes one of these files, it **MUST** also remove the corresponding `check-jsonschema` hook (and any matching `data-ci.yml` step) per the [downstream removal guidance in the ADR](../.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-built-in-schema-validation-for-real-load-bearing-configuration-files). - -### File-Family Hooks - -When real schemas exist, validation SHOULD be wired in per **file family**: - -- Add **one `check-jsonschema` hook per real schema-backed file family**, scoped to the files that family covers (for example, `^config/.*\.json$`). -- **Do not add placeholder hooks** for schemas that do not yet exist. An empty or speculative hook adds noise without enforcing anything. -- **Do not validate every JSON or YAML file by default.** Generic `check-jsonschema --check-metaschema` style sweeps are out of scope; pre-commit already runs `check-json` and `check-yaml` for syntax. Schema validation is a contract check for specific file families, not a global sweep. - -Example hook pattern (illustrative — do not copy verbatim without re-verifying the version): - -```yaml -- repo: https://github.com/python-jsonschema/check-jsonschema - rev: 0.33.3 - hooks: - - id: check-jsonschema - name: Validate project JSON config - files: ^config/.*\.json$ - args: - - --schemafile - - schemas/project-config.schema.json -``` - -> **Version pinning.** Implementers MUST verify and pin a current upstream version of `check-jsonschema` when enabling the hook, rather than copying the example `rev:` value above. Look up the latest tagged release at the upstream repository ([python-jsonschema/check-jsonschema](https://github.com/python-jsonschema/check-jsonschema)) before adoption, and update the pin via your normal dependency-update process. - -## Examples - -Example pairs (a sample data file plus the schema it validates against) MAY live under: - -```text -schemas/examples/ -``` - -Examples MUST NOT contain real secrets or credentials. Example values MUST be obviously fake (for example, `"REPLACE_ME"`, `"example-token-not-real"`). - -### Testing Valid Examples - -Valid examples can be validated directly with `check-jsonschema` from the command line or from a pre-commit hook: +Run the configured schema hooks through pre-commit: ```bash -check-jsonschema \ - --schemafile schemas/project-config.schema.json \ - schemas/examples/project-config/valid/minimal.json +pre-commit run check-jsonschema --all-files +pre-commit run check-metaschema --all-files ``` -A valid example MUST produce exit code `0`. A non-zero exit indicates either a broken example or a schema regression and MUST be fixed before merging. - -### Testing Invalid Examples - -Invalid examples (intentionally malformed fixtures used to prove the schema rejects bad input) MUST NOT be wired directly into a normal pre-commit hook, because `check-jsonschema` would treat their failure as a hook failure. - -Instead, invalid examples SHOULD be exercised by a test or script that asserts validation **fails**. For example, using `pytest` and a subprocess invocation: - -```python -import shutil -import subprocess -import pytest - -CHECK_JSONSCHEMA = shutil.which("check-jsonschema") - - -@pytest.mark.skipif( - CHECK_JSONSCHEMA is None, - reason="check-jsonschema is not installed in this environment", -) -def test_invalid_example_is_rejected(): - result = subprocess.run( - [ - CHECK_JSONSCHEMA, - "--schemafile", - "schemas/project-config.schema.json", - "schemas/examples/project-config/invalid/missing-required.json", - ], - capture_output=True, - text=True, - ) - assert result.returncode != 0, ( - "Invalid example was unexpectedly accepted by the schema; " - "either the schema is too permissive or the example is no longer invalid." - ) -``` - -The same shape applies in PowerShell, Bash, or any CI step: invoke the validator on the invalid fixture and assert a non-zero exit. - -A starter version of this pattern lives at [`templates/python/tests/test_schema_examples.py`](../templates/python/tests/test_schema_examples.py); the active, canonical version that this repository runs in CI lives at [`tests/test_schema_examples.py`](../tests/test_schema_examples.py). Both auto-discover schema/example pairs under `schemas/`. The starter retains a `skipif` guard so it remains safe to copy into downstream projects that have not yet added `check-jsonschema` to their dev/test dependencies. - -## Worked Example - -This template ships a worked example so the schema-validation pipeline works out of the box. The worked example is **template starter content**, not a production contract for downstream repositories. - -- Schema: [`example-config.schema.json`](./example-config.schema.json) -- Valid example data: [`examples/example-config/valid/`](./examples/example-config/valid/) - - [`minimal.json`](./examples/example-config/valid/minimal.json) — only the required properties. - - [`full.json`](./examples/example-config/valid/full.json) — every optional property exercised. -- Invalid example data: [`examples/example-config/invalid/`](./examples/example-config/invalid/) - - [`missing-required.json`](./examples/example-config/invalid/missing-required.json) — required property omitted. - - [`wrong-type.json`](./examples/example-config/invalid/wrong-type.json) — required property has the wrong JSON type. - - [`extra-property.json`](./examples/example-config/invalid/extra-property.json) — unknown property rejected by `additionalProperties: false`. - -How the worked example is validated: - -- The `valid/` example data files are validated by the `Validate example-config valid examples` `check-jsonschema` hook in [`.pre-commit-config.yaml`](../.pre-commit-config.yaml) and by [`.github/workflows/data-ci.yml`](../.github/workflows/data-ci.yml). -- The schema itself is self-validated against its declared JSON Schema Draft 2020-12 metaschema by the `Self-validate example-config schema` `check-metaschema` hook in [`.pre-commit-config.yaml`](../.pre-commit-config.yaml), also executed by [`.github/workflows/data-ci.yml`](../.github/workflows/data-ci.yml). -- The `invalid/` example data files are exercised by [`tests/test_schema_examples.py`](../tests/test_schema_examples.py), which uses `check-jsonschema` to assert that each invalid example causes a non-zero exit code (and that each valid example exits cleanly). A starter version of this pattern, with the same discovery and assertion logic but with project-root resolution suitable for downstream repositories, is also available at [`templates/python/tests/test_schema_examples.py`](../templates/python/tests/test_schema_examples.py). -- Invalid example data files MUST NOT be wired into a normal pre-commit hook because `check-jsonschema` would treat their (expected) failure as a hook failure. - -### Downstream Removal Checklist - -The worked example is intentionally easy to remove. To take it out of a downstream repository: - -1. Delete [`schemas/example-config.schema.json`](./example-config.schema.json). -2. Delete the [`schemas/examples/example-config/`](./examples/example-config/) directory and all of its contents. -3. Remove the `Validate example-config valid examples` and `Self-validate example-config schema` hooks (and the surrounding `python-jsonschema/check-jsonschema` repo block, if no other hooks from that repo remain) from [`.pre-commit-config.yaml`](../.pre-commit-config.yaml). -4. If you adopted the optional schema-example tests (for example, by copying [`templates/python/tests/test_schema_examples.py`](../templates/python/tests/test_schema_examples.py) into your repository's `tests/` directory), remove or adjust the corresponding test cases there if no schemas remain in the downstream repository. -5. Update any documentation that mentions the example schema, including this `README.md` and any references in [`.github/workflows/data-ci.yml`](../.github/workflows/data-ci.yml). - -## Future Work - -Candidate load-bearing repository configuration files that could later be schema-validated against [SchemaStore](https://www.schemastore.org/)-published schemas, `check-jsonschema` built-in schemas, or other stable schema sources include: - -- `package.json` — schema available on SchemaStore. Not currently shipped as a `check-jsonschema` `--builtin-schema`; would require pinning an external schema URL or a future builtin. -- Generated package-manager lockfiles — only if a stable schema-backed validation path is useful and does not conflict with the package manager's own validation. -- `pyproject.toml` — TOML rather than JSON, but conceptually parallel; would require a TOML-aware validator rather than `check-jsonschema`. -- `.pre-commit-config.yaml` — not currently shipped as a `check-jsonschema` `--builtin-schema`. pre-commit itself validates the file's structure when it loads its configuration. -- `.markdownlint.jsonc` — intentionally JSONC (contains comments). MUST NOT be converted to strict JSON merely to satisfy a validator. markdownlint's own config loader remains the enforcement mechanism for this file. -- `.yamllint.yml` — not currently shipped as a `check-jsonschema` `--builtin-schema`. MUST NOT be weakened to satisfy an incomplete external schema; yamllint itself enforces its configuration shape when it loads `.yamllint.yml`. -- GitHub Actions workflow files — already covered by `actionlint`, so an additional schema check would primarily be redundant. - -`.github/dependabot.yml` is already validated by default; see the [Real Repository Configuration Files Validated Through Built-in Schemas](#real-repository-configuration-files-validated-through-built-in-schemas) section above. The candidates above remain out of scope until a verified, mature builtin schema (or an explicitly pinned stable schema source) becomes available; downstream repositories MAY adopt them as additional `check-jsonschema` hooks at their discretion. See the [Built-in Schema Validation for Real Load-Bearing Configuration Files](../.github/TEMPLATE_DESIGN_DECISIONS.md#design-decision-built-in-schema-validation-for-real-load-bearing-configuration-files) ADR for the durable "Evaluated but deferred" record covering each candidate. - -## Out of Scope for This Worked Example - -This directory ships exactly one worked example schema and its example data so the validation pipeline is observable end to end. It does not introduce: - -- Any production schema for this repository's own load-bearing files. -- Additional SchemaStore-backed validation hooks beyond the wired `vendor.dependabot` validation of `.github/dependabot.yml` (which lives in `.pre-commit-config.yaml`, not in this directory). -- Any JSONC, JSON5, or TOML schema validation tooling. - -Those will be added in follow-up changes when concrete schema-backed file families are introduced or when downstream consumers decide to adopt them. +The dedicated data-file workflow also runs these hooks for pull requests and pushes. diff --git a/src/copilot_repo_template/__init__.py b/src/copilot_repo_template/__init__.py deleted file mode 100644 index cf98d1e..0000000 --- a/src/copilot_repo_template/__init__.py +++ /dev/null @@ -1,25 +0,0 @@ -"""Copilot Repo Template - Example Python Package. - -This package is part of the copilot-repo-template repository, which provides -GitHub Copilot instructions and CI configurations for new projects. - -This is a TEMPLATE REPOSITORY. When you create a new repository from this -template: - -1. Rename this package directory to match your project name - (e.g., src/your_project_name/) -2. Update pyproject.toml with your project's metadata -3. Replace these example modules with your actual code -4. Update tests/ with your actual test cases - -The example code here demonstrates: -- Proper package structure with src/ layout -- Type hints for function signatures -- Docstrings following project conventions -- Basic test structure - -See .github/instructions/python.instructions.md for the full Python coding -standards used in this template. -""" - -__version__ = "0.1.0" diff --git a/src/copilot_repo_template/example.py b/src/copilot_repo_template/example.py deleted file mode 100644 index b3167f7..0000000 --- a/src/copilot_repo_template/example.py +++ /dev/null @@ -1,58 +0,0 @@ -"""Example module demonstrating Python coding standards. - -This module provides simple example functions that demonstrate: -- Type hints for function parameters and return values -- Proper docstrings with Args, Returns, and Raises sections -- Basic error handling patterns - -This is a TEMPLATE FILE. Replace this with your actual project code when -using this template repository. -""" - - -def greet(name: str) -> str: - """Generate a greeting message for the given name. - - This is an example function demonstrating proper type hints and docstrings. - Replace this with your actual project functionality. - - Args: - name: The name of the person to greet. - - Returns: - A greeting string in the format "Hello, {name}!". - - Raises: - ValueError: If name is empty or contains only whitespace. - - Examples: - >>> greet("World") - 'Hello, World!' - >>> greet("Copilot") - 'Hello, Copilot!' - """ - if not name or not name.strip(): - raise ValueError("Name cannot be empty or whitespace") - return f"Hello, {name}!" - - -def add_numbers(a: int | float, b: int | float) -> int | float: - """Add two numbers together. - - This is another example function demonstrating type hints with union types. - Replace this with your actual project functionality. - - Args: - a: The first number. - b: The second number. - - Returns: - The sum of a and b. - - Examples: - >>> add_numbers(2, 3) - 5 - >>> add_numbers(2.5, 3.5) - 6.0 - """ - return a + b diff --git a/templates/python/README.md b/templates/python/README.md deleted file mode 100644 index 14d34a7..0000000 --- a/templates/python/README.md +++ /dev/null @@ -1,37 +0,0 @@ -# Python Template Files - -This directory contains template Python configuration files and scaffolding for projects using Python in this repository template. - -## Purpose - -These template files demonstrate how to configure Python tooling to align with the coding standards defined in `.github/instructions/python.instructions.md`. - -## Files Included - -- **`pyproject.toml`**: Sample configuration for Python project metadata, dependencies, and tooling (Black, Ruff, mypy, pytest) -- **`tests/`**: Sample test directory containing: - - `__init__.py`: Package marker for the test directory. - - `test_placeholder.py`: Placeholder test file that demonstrates pytest test structure. - - `test_schema_examples.py`: Starter pytest module that auto-discovers and validates schema example fixtures under `schemas/examples//{valid,invalid}/` with `check-jsonschema`; skips cleanly when the tool is not installed. Mirrors the active, canonical test at `tests/test_schema_examples.py` in the template repository root. See the **"Schema Validation Configuration"** section in [`OPTIONAL_CONFIGURATIONS.md`](../../OPTIONAL_CONFIGURATIONS.md#schema-validation-configuration). -- **`README.md`**: This file - -## How to Use - -For detailed setup instructions including: - -- How to copy and customize these template files -- Project layout options (flat vs. `src/` layout) -- Python version configuration across different tools -- Python version support policy -- mypy path configuration - -See the **"Using the Python Template Files"** section in [`OPTIONAL_CONFIGURATIONS.md`](../../OPTIONAL_CONFIGURATIONS.md#using-the-python-template-files). - -## Additional Resources - -- [Python Developer's Guide - Versions](https://devguide.python.org/versions/) -- [PEP 621 - Python Project Metadata](https://peps.python.org/pep-0621/) -- [Black Documentation](https://black.readthedocs.io/) -- [Ruff Documentation](https://docs.astral.sh/ruff/) -- [mypy Documentation](https://mypy.readthedocs.io/) -- [pytest Documentation](https://docs.pytest.org/) diff --git a/templates/python/pyproject.toml b/templates/python/pyproject.toml deleted file mode 100644 index 4898515..0000000 --- a/templates/python/pyproject.toml +++ /dev/null @@ -1,85 +0,0 @@ -# Sample pyproject.toml for Python projects -# This file demonstrates how to configure Python tooling to align with the coding standards -# in .github/instructions/python.instructions.md -# -# Copy this file to your project root and customize the [project] section. - -[build-system] -requires = ["setuptools>=68.0", "wheel"] -build-backend = "setuptools.build_meta" - -[project] -name = "your-project-name" -version = "0.1.0" -description = "Project description" -readme = "README.md" -# Python Version Policy: -# Always require a Python version that is currently receiving bugfixes. -# Check https://devguide.python.org/versions/ for current version status. -# Update this when upstream support changes (typically annually around October). -requires-python = ">=3.13" -license = "MIT" -authors = [ - { name = "Your Name", email = "your.email@example.com" } -] -keywords = [] -classifiers = [ - "Development Status :: 3 - Alpha", - "Programming Language :: Python :: 3", - "Programming Language :: Python :: 3.13", -] - -# Runtime dependencies -dependencies = [] - -# Development dependencies -# Note: pre-commit is intentionally NOT included here. -# Install pre-commit globally via: pip install pre-commit -# See CONTRIBUTING.md for details. -[project.optional-dependencies] -dev = [ - "pytest>=8.0.0", - "pytest-cov>=4.1.0", - "ruff>=0.9.0", - "black>=24.0.0", - "mypy>=1.8.0", - # check-jsonschema is required by tests/test_schema_examples.py - # (the starter that mirrors the active test in the template - # repository) to validate files under - # schemas/examples//{valid,invalid}/. Remove this line if - # your downstream project does not use schema example tests. - "check-jsonschema>=0.33.0", -] - -[tool.black] -line-length = 100 -# Update target-version when minimum Python version changes. -# Black uses a list of "pyXYZ" strings (e.g., ["py313"] for Python 3.13). -target-version = ["py313"] - -[tool.ruff] -line-length = 100 -# Note: Ruff automatically infers target-version from [project].requires-python -# Only set target-version here if you need to override the inferred value. -# If set, use the format: target-version = "py313" (single string, not a list). - -[tool.ruff.lint] -select = ["E", "W", "F", "I", "B", "C4", "UP"] -ignore = ["E501"] # line too long handled by black - -[tool.ruff.lint.per-file-ignores] -"__init__.py" = ["F401"] - -[tool.mypy] -# Update python_version when minimum Python version changes. -# mypy uses a dotted version string (e.g., "3.13" for Python 3.13). -python_version = "3.13" -warn_return_any = true -warn_unused_configs = true -# Start permissive, tighten later as project matures -disallow_untyped_defs = false - -[tool.pytest.ini_options] -testpaths = ["tests"] -python_files = ["test_*.py"] -python_functions = ["test_*"] diff --git a/templates/python/tests/__init__.py b/templates/python/tests/__init__.py deleted file mode 100644 index 0bc8e11..0000000 --- a/templates/python/tests/__init__.py +++ /dev/null @@ -1,2 +0,0 @@ -# This file marks the tests directory as a Python package. -# This is a template file - you can modify or remove it as needed for your project. diff --git a/templates/python/tests/test_placeholder.py b/templates/python/tests/test_placeholder.py deleted file mode 100644 index 4a7e9f8..0000000 --- a/templates/python/tests/test_placeholder.py +++ /dev/null @@ -1,12 +0,0 @@ -"""Placeholder test file for template demonstration. - -This is a template file. Delete or overwrite this with your actual tests. -""" - - -def test_placeholder(): - """Simple placeholder test that always passes. - - Replace this with your actual test cases. - """ - assert True diff --git a/templates/python/tests/test_schema_examples.py b/templates/python/tests/test_schema_examples.py deleted file mode 100644 index 2fd0274..0000000 --- a/templates/python/tests/test_schema_examples.py +++ /dev/null @@ -1,348 +0,0 @@ -"""Starter template: validate schema example files with ``check-jsonschema``. - -This file is **starter content** that downstream Python projects copy -into their own ``tests/`` directory. The active, canonical version -that this starter is derived from lives at -``tests/test_schema_examples.py`` in the ``copilot-repo-template`` -repository — see that file for the source of truth on discovery, -invocation, and assertion logic. - -The starter and the active test share the same essential pattern: - -- Auto-discover schemas via ``schemas/*.schema.json``. -- For each schema, derive the example directory by stripping - ``.schema.json`` from the file name (``schemas/.schema.json`` - maps to ``schemas/examples//``). -- Recursively walk ``schemas/examples//valid/`` (must validate) - and ``schemas/examples//invalid/`` (must be rejected). -- Invoke ``check-jsonschema`` via ``subprocess.run`` with - ``check=False``, ``capture_output=True``, ``text=True``. - -Intentional differences from the active test: - -- Path resolution: this starter resolves the project root by walking - up from this file's location until it finds a ``pyproject.toml``, - ``setup.cfg``, or ``pytest.ini`` marker. The active test in the - template repository hardcodes its location as - ``/tests/test_schema_examples.py``; downstream projects may - place this file at a different depth, so root discovery is dynamic - here. See ``.github/instructions/python.instructions.md`` - (Filesystem and Paths): paths SHOULD be resolved from a clear root - rather than the process CWD. -- The ``check-jsonschema`` skipif guard is retained so the starter - remains safe to copy even into projects that have not yet added - ``check-jsonschema`` to their dev/test dependencies. Downstream - projects SHOULD add ``check-jsonschema`` to their dev/test - dependency group (see ``templates/python/pyproject.toml``) so the - test always runs. - -How to use: - -1. Copy this file into your project's real ``tests/`` directory. -2. Add ``check-jsonschema`` to your project's dev/test dependencies - (already declared in ``templates/python/pyproject.toml``). -3. Place schemas under ``schemas/.schema.json`` and matching - examples under ``schemas/examples//{valid,invalid}/``. No - per-case configuration is required — discovery is automatic. - -Both valid and invalid examples are exercised here: - -- Valid examples MUST validate cleanly (exit code ``0``). -- Invalid examples MUST be rejected (non-zero exit code). - -Invalid examples are intentionally NOT wired into a normal -``check-jsonschema`` pre-commit hook, because a failing exit code from -the validator would be reported as a hook failure. Use this test (or -an equivalent script) to prove that the schema actually rejects them. -""" - -from __future__ import annotations - -import os -import shutil -import subprocess -from pathlib import Path - -import pytest - -CHECK_JSONSCHEMA = shutil.which("check-jsonschema") - -SCHEMA_SUFFIX = ".schema.json" -_ROOT_MARKERS = ("pyproject.toml", "setup.cfg", "pytest.ini") - - -def _find_project_root(start: Path) -> Path: - """Walk up from ``start`` until a project-root marker is found. - - Args: - start: A directory inside the downstream project (typically - the directory containing this test file). Callers MUST - pass a directory; file paths should be normalized at the - call site (e.g., ``Path(__file__).resolve().parent``) so - the first iteration does not check a nonexistent - ``/`` path. - - Returns: - The first ancestor directory containing ``pyproject.toml``, - ``setup.cfg``, or ``pytest.ini``. Falls back to ``start`` - itself when no marker is found, which keeps the starter - usable in unusual layouts without raising at import time. - """ - for candidate in (start, *start.parents): - if any((candidate / marker).is_file() for marker in _ROOT_MARKERS): - return candidate - return start - - -PROJECT_ROOT = _find_project_root(Path(__file__).resolve().parent) -SCHEMAS_DIR = PROJECT_ROOT / "schemas" -EXAMPLES_DIR = SCHEMAS_DIR / "examples" - - -def _is_within_root(candidate: Path, root: Path) -> bool: - """Return ``True`` iff ``candidate`` is reachable from ``root`` with no symlink ancestors. - - Walks upward from ``candidate`` to ``root`` (inclusive), checking - ``Path.is_symlink()`` at each step. Any symlink found on that path - causes a refusal, which closes the symlink-rooted-discovery - escape: a malicious symlink at, for example, - ``schemas/examples/``, ``schemas/examples/``, - ``schemas/``, or even ``schemas/.schema.json`` itself - would otherwise survive a downstream ``os.walk(followlinks=False)`` - or a yielded-path-only ``Path.resolve().relative_to(base)`` check, - because those checks examine paths *under* ``candidate`` rather - than the symlink chain *to* ``candidate``. - - Also returns ``False`` if ``candidate.resolve()`` does not stay - inside ``root.resolve()``, or if any ``OSError`` / ``ValueError`` - is raised while resolving (callers should treat such errors as a - hard refusal rather than a recoverable condition, which is why - they are caught here and surfaced as a boolean). - - This helper does **not** enforce a mount-point or device boundary - (``Path.is_symlink()`` does not detect bind mounts). Downstream - projects that require that level of isolation should layer an - explicit ``os.stat().st_dev`` (or mount-table) check on top of - this helper. - - Args: - candidate: Path that the caller wants to use as a discovery - root or as a discovered fixture. May be a file or a - directory; need not exist (``is_symlink`` is well-defined - on dangling symlinks). - root: Trusted ancestor that ``candidate`` must live inside. - Typically the downstream-derived ``PROJECT_ROOT`` or - ``SCHEMAS_DIR``. - - Returns: - ``True`` only when ``candidate`` resolves under ``root`` *and* - every path component from ``candidate`` up to and including - ``root`` is a regular (non-symlink) entry; ``False`` otherwise. - """ - try: - candidate.resolve().relative_to(root.resolve()) - cur = candidate - while True: - if cur.is_symlink(): - return False - if cur == root: - return True - parent = cur.parent - if parent == cur: - return False - cur = parent - except (OSError, ValueError): - return False - - -def _iter_safe_files(directory: Path, root: Path) -> list[Path]: - """List regular files under ``directory``, refusing symlink escapes. - - Caller's contract: ``directory`` must be intended to live inside - ``root``. This helper enforces that contract before walking by - calling :func:`_is_within_root`, which rejects discovery if - ``directory`` (or any ancestor up to ``root``) is itself a - symlink, or if ``directory.resolve()`` escapes ``root.resolve()``. - Without this pre-walk check, an attacker who controls a path - component (e.g., a symlink at ``schemas/examples/`` - pointing outside the project) could coerce the test suite into - walking and validating files outside ``root`` even though - ``os.walk(followlinks=False)`` and the per-yielded-path - boundary check would each, in isolation, appear to "succeed". - - Once the discovery root is accepted, walks ``directory`` with - ``os.walk(followlinks=False)`` so that symlinked subdirectories - are never traversed, drops any yielded entry that is itself a - symlink, and re-checks each remaining path with - ``Path.resolve()`` and ``Path.relative_to`` to guarantee its - fully-resolved path string still lives inside ``directory``'s - resolved location, which catches symlink and ``..``-style - traversals that survive the initial walk-time skip. - - Downstream projects that copy this starter inherit the same - policy: example-fixture discovery refuses any symlink-rooted - escape (a symlink at any level from ``root`` down to the discovery - directory), never follows symlinks during traversal, and never - returns files outside the resolved fixture tree, so a malicious - or accidentally-introduced symlink at any level of the - ``schemas/`` tree cannot coerce the test into validating files - outside the project. - - This helper does **not** enforce a mount-point or device boundary. - A bind mount (or any other mount) located *under* ``directory`` - will still expose its target content because ``Path.resolve()`` - returns a path that, from the filesystem's perspective, remains - inside ``directory``. Downstream projects that require mount- - boundary isolation should layer an explicit - ``os.stat().st_dev`` (or mount-table) check on top of this - helper. - - Args: - directory: Directory whose regular-file descendants should be - returned. May not exist; if absent, an empty list is - returned. - root: Trusted ancestor of ``directory`` (typically - ``PROJECT_ROOT``). Discovery is refused if ``directory`` - or any ancestor up to ``root`` is a symlink, or if - ``directory`` resolves outside ``root``. - - Returns: - A list of regular-file ``Path`` objects below ``directory``, - in the order produced by ``os.walk`` (callers that need a - deterministic order should sort the result). Returns ``[]`` - when ``directory`` is missing, fails the symlink-ancestor / - containment check against ``root``, or contains no eligible - regular files. - """ - if not directory.is_dir(): - return [] - if not _is_within_root(directory, root): - return [] - base = directory.resolve() - discovered: list[Path] = [] - for current_root, _dir_names, file_names in os.walk(directory, followlinks=False): - for file_name in file_names: - file_path = Path(current_root) / file_name - if file_path.is_symlink(): - continue - try: - file_path.resolve().relative_to(base) - except (OSError, ValueError): - continue - discovered.append(file_path) - return discovered - - -def _discover_cases() -> list[tuple[Path, Path, bool]]: - """Discover ``(schema, example, expected_to_pass)`` triples. - - Returns: - A list of ``(schema_path, example_path, expected_to_pass)`` - tuples for every regular file under - ``schemas/examples//valid/`` (expected to pass) and - ``schemas/examples//invalid/`` (expected to fail). - Schema files and example directories whose path chain from - ``PROJECT_ROOT`` includes a symlink (or that resolve outside - ``PROJECT_ROOT``) are rejected via :func:`_is_within_root` so - a malicious or accidentally-introduced symlink at any level - cannot coerce the suite into validating files outside the - project. Returns an empty list when no schemas or examples - are present so the test suite degrades gracefully rather - than hard-failing on a count assertion. - """ - cases: list[tuple[Path, Path, bool]] = [] - if not SCHEMAS_DIR.is_dir(): - return cases - for schema_path in sorted(SCHEMAS_DIR.glob(f"*{SCHEMA_SUFFIX}")): - if not _is_within_root(schema_path, PROJECT_ROOT): - continue - schema_name = schema_path.name[: -len(SCHEMA_SUFFIX)] - schema_examples_dir = EXAMPLES_DIR / schema_name - for kind, expected_to_pass in (("valid", True), ("invalid", False)): - kind_dir = schema_examples_dir / kind - if not kind_dir.is_dir(): - continue - for example_path in sorted(_iter_safe_files(kind_dir, PROJECT_ROOT)): - if not example_path.is_file(): - continue - cases.append((schema_path, example_path, expected_to_pass)) - return cases - - -def _case_id(case: tuple[Path, Path, bool]) -> str: - """Build a readable parametrize ID for a discovered case.""" - schema_path, example_path, expected_to_pass = case - try: - schema_rel = schema_path.relative_to(PROJECT_ROOT).as_posix() - example_rel = example_path.relative_to(PROJECT_ROOT).as_posix() - except ValueError: - schema_rel = schema_path.as_posix() - example_rel = example_path.as_posix() - outcome = "valid" if expected_to_pass else "invalid" - return f"{schema_rel}::{outcome}::{example_rel}" - - -_CASES = _discover_cases() - - -@pytest.mark.skipif( - CHECK_JSONSCHEMA is None, - reason="check-jsonschema is not installed in this environment", -) -@pytest.mark.skipif( - not _CASES, - reason="No schema example files found under schemas/examples/", -) -@pytest.mark.parametrize( - ("schema_path", "example_path", "expected_to_pass"), - _CASES, - ids=[_case_id(c) for c in _CASES], -) -def test_schema_example( - schema_path: Path, - example_path: Path, - expected_to_pass: bool, -) -> None: - """Validate one ``(schema, example)`` pair against its labeled outcome. - - Args: - schema_path: Absolute path to a ``*.schema.json`` file under - ``schemas/``. - example_path: Absolute path to an example file under either - ``schemas/examples//valid/`` or - ``schemas/examples//invalid/``. - expected_to_pass: ``True`` for ``valid/`` examples (must - validate cleanly), ``False`` for ``invalid/`` examples - (must be rejected). - - Raises: - AssertionError: If a valid example is rejected, or an invalid - example is accepted, by ``check-jsonschema``. - """ - # CHECK_JSONSCHEMA is non-None here because of the skipif guard - # above; assert for type-checkers and as a defensive runtime check. - assert CHECK_JSONSCHEMA is not None - - result = subprocess.run( - [ - CHECK_JSONSCHEMA, - "--schemafile", - str(schema_path), - str(example_path), - ], - check=False, - capture_output=True, - text=True, - ) - - if expected_to_pass: - assert result.returncode == 0, ( - f"Valid example {example_path} was unexpectedly rejected by " - f"{schema_path}.\nstdout:\n{result.stdout}\nstderr:\n{result.stderr}" - ) - else: - assert result.returncode != 0, ( - f"Invalid example {example_path} was unexpectedly accepted by " - f"{schema_path}; the schema may be too permissive or the example " - f"is no longer invalid.\nstdout:\n{result.stdout}\nstderr:\n{result.stderr}" - ) diff --git a/templates/terraform/Example.tftest.hcl b/templates/terraform/Example.tftest.hcl deleted file mode 100644 index 7bb15b6..0000000 --- a/templates/terraform/Example.tftest.hcl +++ /dev/null @@ -1,131 +0,0 @@ -# Example.tftest.hcl -# -# TEMPLATE FILE: Copy this file to modules//tests/ and customize. -# -# This file demonstrates Terraform test patterns including: -# - Global variables for test configuration -# - Mock providers for unit testing -# - Basic assertions with condition/error_message -# - Variable validation testing with expect_failures -# - Output verification -# - Unit tests (command = plan) vs integration tests (command = apply) -# -# Prerequisites: -# - Terraform 1.7.0 or later (mock_provider requires 1.7.0+) -# -# Usage (from module directory): -# terraform test -# terraform test -verbose -# terraform test -filter=tests/Example.tftest.hcl -# -# For complete Terraform test documentation: -# https://developer.hashicorp.com/terraform/cli/commands/test - -# ============================================================================== -# Global Variables -# ============================================================================== -# Variables defined here apply to all run blocks unless overridden. - -variables { - environment = "test" - instance_type = "t3.micro" - # Add your module's required variables here -} - -# ============================================================================== -# Mock Provider Configuration -# ============================================================================== -# Mock providers enable unit testing without real cloud credentials. - -mock_provider "aws" { - # Mock data sources - mock_data "aws_availability_zones" { - defaults = { - names = ["us-east-1a", "us-east-1b", "us-east-1c"] - } - } - - mock_data "aws_caller_identity" { - defaults = { - account_id = "123456789012" - } - } - - # Mock resource computed attributes - mock_resource "aws_instance" { - defaults = { - id = "i-mock12345" - arn = "arn:aws:ec2:us-east-1:123456789012:instance/i-mock12345" - private_ip = "10.0.1.100" - } - } -} - -# ============================================================================== -# Unit Tests (command = plan) -# ============================================================================== -# Unit tests validate configuration logic without creating resources. - -run "validates_instance_type" { - command = plan - - assert { - condition = aws_instance.main.instance_type == var.instance_type - error_message = "Instance type should match input variable" - } -} - -run "applies_environment_tag" { - command = plan - - assert { - condition = aws_instance.main.tags["Environment"] == var.environment - error_message = "Environment tag should be set" - } -} - -# ============================================================================== -# Variable Validation Tests -# ============================================================================== -# Test that invalid inputs are properly rejected. - -run "rejects_invalid_environment" { - command = plan - - variables { - environment = "invalid" # Should fail validation - } - - expect_failures = [ - var.environment - ] -} - -# ============================================================================== -# Output Verification -# ============================================================================== -# Verify outputs contain expected values. - -run "outputs_instance_id" { - command = plan - - assert { - condition = output.instance_id != null - error_message = "instance_id output must not be null" - } -} - -# ============================================================================== -# Integration Tests (command = apply) -# ============================================================================== -# Integration tests create real resources. Use sparingly. -# Uncomment when you have valid provider credentials. -# -# run "creates_instance_successfully" { -# command = apply -# -# assert { -# condition = aws_instance.main.id != "" -# error_message = "Instance should be created" -# } -# } diff --git a/tests/__init__.py b/tests/__init__.py deleted file mode 100644 index 167a271..0000000 --- a/tests/__init__.py +++ /dev/null @@ -1,10 +0,0 @@ -"""Test package for copilot-repo-template. - -This directory contains tests for the example Python package. When using this -template repository, replace these example tests with your actual test cases. - -The test structure follows pytest conventions: -- Test files are named test_*.py -- Test functions are named test_* -- Tests are organized to mirror the source code structure -""" diff --git a/tests/test_example.py b/tests/test_example.py deleted file mode 100644 index 8ac9e14..0000000 --- a/tests/test_example.py +++ /dev/null @@ -1,62 +0,0 @@ -"""Tests for the example module. - -This test file demonstrates: -- Basic pytest test structure -- Testing functions with different input scenarios -- Testing error conditions with pytest.raises - -This is a TEMPLATE FILE. Replace these tests with your actual test cases -when using this template repository. -""" - -import pytest - -from copilot_repo_template.example import add_numbers, greet - - -class TestGreet: - """Tests for the greet function.""" - - def test_greet_returns_greeting(self) -> None: - """Test that greet returns a proper greeting message.""" - result = greet("World") - assert result == "Hello, World!" - - def test_greet_with_different_name(self) -> None: - """Test greet with a different name.""" - result = greet("Copilot") - assert result == "Hello, Copilot!" - - def test_greet_raises_on_empty_string(self) -> None: - """Test that greet raises ValueError for empty string.""" - with pytest.raises(ValueError, match="cannot be empty or whitespace"): - greet("") - - def test_greet_raises_on_whitespace(self) -> None: - """Test that greet raises ValueError for whitespace-only string.""" - with pytest.raises(ValueError, match="cannot be empty or whitespace"): - greet(" ") - - -class TestAddNumbers: - """Tests for the add_numbers function.""" - - def test_add_integers(self) -> None: - """Test adding two integers.""" - result = add_numbers(2, 3) - assert result == 5 - - def test_add_floats(self) -> None: - """Test adding two floats.""" - result = add_numbers(2.5, 3.5) - assert result == 6.0 - - def test_add_mixed_types(self) -> None: - """Test adding an integer and a float.""" - result = add_numbers(2, 3.5) - assert result == 5.5 - - def test_add_negative_numbers(self) -> None: - """Test adding negative numbers.""" - result = add_numbers(-5, 3) - assert result == -2 diff --git a/tests/test_schema_examples.py b/tests/test_schema_examples.py deleted file mode 100644 index 48e2b6e..0000000 --- a/tests/test_schema_examples.py +++ /dev/null @@ -1,302 +0,0 @@ -"""Validate schema example files with ``check-jsonschema``. - -This is the **active, canonical** schema-example test for this -repository. It auto-discovers schema/example pairs under ``schemas/`` -and verifies that: - -- Every file under ``schemas/examples//valid/`` validates - successfully against ``schemas/.schema.json`` - (``check-jsonschema`` exits ``0``). -- Every file under ``schemas/examples//invalid/`` is - rejected (``check-jsonschema`` exits non-zero). - -Discovery rules: - -- Schemas are found via the glob ``schemas/*.schema.json``. -- For each schema, the example directory name is derived by removing - the trailing ``.schema.json`` from the file name. For example, - ``schemas/example-config.schema.json`` maps to - ``schemas/examples/example-config/``. -- Only regular files under ``valid/`` and ``invalid/`` are exercised; - directories and other non-file entries are ignored. - -Paths are resolved from the repository root, which is derived from -this file's known location at ``/tests/test_schema_examples.py`` -(via ``Path(__file__).resolve().parent.parent``) rather than the -process current working directory, so the test behaves the same -regardless of where ``pytest`` is invoked from. - -Invalid examples are intentionally NOT wired into a normal -``check-jsonschema`` pre-commit hook, because a failing exit code from -the validator would be reported as a hook failure. This test exists in -part to prove that the schema actually rejects them. - -A starter version of this test is available at -``templates/python/tests/test_schema_examples.py`` for downstream -consumers of the template; the two files share the same essential -validation pattern. -""" - -from __future__ import annotations - -import os -import shutil -import subprocess -from pathlib import Path - -import pytest - -CHECK_JSONSCHEMA = shutil.which("check-jsonschema") - -# Repository root, relative to this file: ``/tests/test_schema_examples.py``. -REPO_ROOT = Path(__file__).resolve().parent.parent -SCHEMAS_DIR = REPO_ROOT / "schemas" -EXAMPLES_DIR = SCHEMAS_DIR / "examples" -SCHEMA_SUFFIX = ".schema.json" - - -def _is_within_root(candidate: Path, root: Path) -> bool: - """Return ``True`` iff ``candidate`` is reachable from ``root`` with no symlink ancestors. - - Walks upward from ``candidate`` to ``root`` (inclusive), checking - ``Path.is_symlink()`` at each step. Any symlink found on that path - causes a refusal, which closes the symlink-rooted-discovery - escape: a malicious symlink at, for example, - ``schemas/examples/``, ``schemas/examples/``, - ``schemas/``, or even ``schemas/.schema.json`` itself - would otherwise survive a downstream ``os.walk(followlinks=False)`` - or a yielded-path-only ``Path.resolve().relative_to(base)`` check, - because those checks examine paths *under* ``candidate`` rather - than the symlink chain *to* ``candidate``. - - Also returns ``False`` if ``candidate.resolve()`` does not stay - inside ``root.resolve()``, or if any ``OSError`` / ``ValueError`` - is raised while resolving (callers should treat such errors as a - hard refusal rather than a recoverable condition, which is why - they are caught here and surfaced as a boolean). - - This helper does **not** enforce a mount-point or device boundary - (``Path.is_symlink()`` does not detect bind mounts). Callers that - require that level of isolation should layer an explicit - ``os.stat().st_dev`` (or mount-table) check on top of this helper. - - Args: - candidate: Path that the caller wants to use as a discovery - root or as a discovered fixture. May be a file or a - directory; need not exist (``is_symlink`` is well-defined - on dangling symlinks). - root: Trusted ancestor that ``candidate`` must live inside. - Typically a constant such as ``REPO_ROOT`` or - ``SCHEMAS_DIR`` derived from ``Path(__file__)`` and - therefore not user-controlled. - - Returns: - ``True`` only when ``candidate`` resolves under ``root`` *and* - every path component from ``candidate`` up to and including - ``root`` is a regular (non-symlink) entry; ``False`` otherwise. - """ - try: - candidate.resolve().relative_to(root.resolve()) - cur = candidate - while True: - if cur.is_symlink(): - return False - if cur == root: - return True - parent = cur.parent - if parent == cur: - return False - cur = parent - except (OSError, ValueError): - return False - - -def _iter_safe_files(directory: Path, root: Path) -> list[Path]: - """List regular files under ``directory``, refusing symlink escapes. - - Caller's contract: ``directory`` must be intended to live inside - ``root``. This helper enforces that contract before walking by - calling :func:`_is_within_root`, which rejects discovery if - ``directory`` (or any ancestor up to ``root``) is itself a - symlink, or if ``directory.resolve()`` escapes ``root.resolve()``. - Without this pre-walk check, an attacker who controls a path - component (e.g., a symlink at ``schemas/examples/`` - pointing outside the repository) could coerce the test suite into - walking and validating files outside ``root`` even though - ``os.walk(followlinks=False)`` and the per-yielded-path - boundary check would each, in isolation, appear to "succeed". - - Once the discovery root is accepted, walks ``directory`` with - ``os.walk(followlinks=False)`` so that symlinked subdirectories - are never traversed, drops any yielded entry that is itself a - symlink, and re-checks each remaining path with - ``Path.resolve()`` and ``Path.relative_to`` to guarantee its - fully-resolved path string still lives inside ``directory``'s - resolved location, which catches symlink and ``..``-style - traversals that survive the initial walk-time skip. - - The repository constitution requires that file-discovery callers - "reject path traversal and symlink escapes"; this helper - centralizes that policy for example-fixture discovery so the test - suite cannot be coerced into validating files outside the - schemas/examples tree by a malicious or accidentally-introduced - symlink at any level. - - This helper does **not** enforce a mount-point or device boundary. - A bind mount (or any other mount) located *under* ``directory`` - will still expose its target content because ``Path.resolve()`` - returns a path that, from the filesystem's perspective, remains - inside ``directory``. Callers that require mount-boundary - isolation should layer an explicit ``os.stat().st_dev`` (or - mount-table) check on top of this helper. - - Args: - directory: Directory whose regular-file descendants should be - returned. May not exist; if absent, an empty list is - returned. - root: Trusted ancestor of ``directory`` (typically - ``REPO_ROOT``). Discovery is refused if ``directory`` or - any ancestor up to ``root`` is a symlink, or if - ``directory`` resolves outside ``root``. - - Returns: - A list of regular-file ``Path`` objects below ``directory``, - in the order produced by ``os.walk`` (callers that need a - deterministic order should sort the result). Returns ``[]`` - when ``directory`` is missing, fails the symlink-ancestor / - containment check against ``root``, or contains no eligible - regular files. - """ - if not directory.is_dir(): - return [] - if not _is_within_root(directory, root): - return [] - base = directory.resolve() - discovered: list[Path] = [] - for current_root, _dir_names, file_names in os.walk(directory, followlinks=False): - for file_name in file_names: - file_path = Path(current_root) / file_name - if file_path.is_symlink(): - continue - try: - file_path.resolve().relative_to(base) - except (OSError, ValueError): - continue - discovered.append(file_path) - return discovered - - -def _discover_cases() -> list[tuple[Path, Path, bool]]: - """Discover ``(schema, example, expected_to_pass)`` triples. - - Returns: - A list of ``(schema_path, example_path, expected_to_pass)`` - tuples covering every regular file under - ``schemas/examples//valid/`` (expected to pass) - and ``schemas/examples//invalid/`` (expected to - fail). Schema files and example directories whose path chain - from ``REPO_ROOT`` includes a symlink (or that resolve outside - ``REPO_ROOT``) are rejected via :func:`_is_within_root` so a - malicious or accidentally-introduced symlink at any level - cannot coerce the suite into validating files outside the - repository. Returns an empty list if no schemas or examples - are present, so the test suite degrades gracefully rather - than hard-failing on a count assertion. - """ - cases: list[tuple[Path, Path, bool]] = [] - if not SCHEMAS_DIR.is_dir(): - return cases - for schema_path in sorted(SCHEMAS_DIR.glob(f"*{SCHEMA_SUFFIX}")): - if not _is_within_root(schema_path, REPO_ROOT): - continue - schema_name = schema_path.name[: -len(SCHEMA_SUFFIX)] - schema_examples_dir = EXAMPLES_DIR / schema_name - for kind, expected_to_pass in (("valid", True), ("invalid", False)): - kind_dir = schema_examples_dir / kind - if not kind_dir.is_dir(): - continue - for example_path in sorted(_iter_safe_files(kind_dir, REPO_ROOT)): - if not example_path.is_file(): - continue - cases.append((schema_path, example_path, expected_to_pass)) - return cases - - -def _case_id(case: tuple[Path, Path, bool]) -> str: - """Build a readable parametrize ID for a discovered case. - - The ID identifies both the schema and the example path relative to - the repository root, plus the expected outcome, so failing cases - are easy to locate from pytest output. - """ - schema_path, example_path, expected_to_pass = case - schema_rel = schema_path.relative_to(REPO_ROOT).as_posix() - example_rel = example_path.relative_to(REPO_ROOT).as_posix() - outcome = "valid" if expected_to_pass else "invalid" - return f"{schema_rel}::{outcome}::{example_rel}" - - -_CASES = _discover_cases() - - -@pytest.mark.skipif( - CHECK_JSONSCHEMA is None, - reason="check-jsonschema is not installed in this environment", -) -@pytest.mark.skipif( - not _CASES, - reason="No schema example files found under schemas/examples/", -) -@pytest.mark.parametrize( - ("schema_path", "example_path", "expected_to_pass"), - _CASES, - ids=[_case_id(c) for c in _CASES], -) -def test_schema_example( - schema_path: Path, - example_path: Path, - expected_to_pass: bool, -) -> None: - """Validate one ``(schema, example)`` pair against its labeled outcome. - - Args: - schema_path: Absolute path to a ``*.schema.json`` file under - ``schemas/``. - example_path: Absolute path to an example file under either - ``schemas/examples//valid/`` or - ``schemas/examples//invalid/``. - expected_to_pass: ``True`` when the example lives under - ``valid/`` (must validate cleanly), ``False`` when it - lives under ``invalid/`` (must be rejected). - - Raises: - AssertionError: If a valid example is rejected, or an invalid - example is accepted, by ``check-jsonschema``. - """ - # CHECK_JSONSCHEMA is non-None here because of the skipif guard - # above; assert for type-checkers and as a defensive runtime check. - assert CHECK_JSONSCHEMA is not None - - result = subprocess.run( - [ - CHECK_JSONSCHEMA, - "--schemafile", - str(schema_path), - str(example_path), - ], - check=False, - capture_output=True, - text=True, - ) - - if expected_to_pass: - assert result.returncode == 0, ( - f"Valid example {example_path} was unexpectedly rejected by " - f"{schema_path}.\nstdout:\n{result.stdout}\nstderr:\n{result.stderr}" - ) - else: - assert result.returncode != 0, ( - f"Invalid example {example_path} was unexpectedly accepted by " - f"{schema_path}; the schema may be too permissive or the example " - f"is no longer invalid.\nstdout:\n{result.stdout}\nstderr:\n{result.stderr}" - ) From c263accc5d364a21d7087104393e18ba7fd8ea57 Mon Sep 17 00:00:00 2001 From: Frank Lesniak Date: Tue, 5 May 2026 00:47:02 +0000 Subject: [PATCH 06/10] feat: add devcontainer lock file with feature configurations --- .devcontainer/devcontainer-lock.json | 29 ++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 .devcontainer/devcontainer-lock.json diff --git a/.devcontainer/devcontainer-lock.json b/.devcontainer/devcontainer-lock.json new file mode 100644 index 0000000..34ee7fe --- /dev/null +++ b/.devcontainer/devcontainer-lock.json @@ -0,0 +1,29 @@ +{ + "features": { + "ghcr.io/devcontainers/features/common-utils:2": { + "version": "2.5.7", + "resolved": "ghcr.io/devcontainers/features/common-utils@sha256:dbf431d6b42d55cde50fa1df75c7f7c3999a90cde6d73f7a7071174b3c3d0cc4", + "integrity": "sha256:dbf431d6b42d55cde50fa1df75c7f7c3999a90cde6d73f7a7071174b3c3d0cc4" + }, + "ghcr.io/devcontainers/features/git-lfs:1": { + "version": "1.2.5", + "resolved": "ghcr.io/devcontainers/features/git-lfs@sha256:71c2b371cf12ab7fcec47cf17369c6f59156100dad9abf9e4c593049d789de72", + "integrity": "sha256:71c2b371cf12ab7fcec47cf17369c6f59156100dad9abf9e4c593049d789de72" + }, + "ghcr.io/devcontainers/features/node:1": { + "version": "1.7.1", + "resolved": "ghcr.io/devcontainers/features/node@sha256:8c0de46939b61958041700ee89e3493f3b2e4131a06dc46b4d9423427d06e5f6", + "integrity": "sha256:8c0de46939b61958041700ee89e3493f3b2e4131a06dc46b4d9423427d06e5f6" + }, + "ghcr.io/devcontainers/features/powershell:1": { + "version": "1.5.1", + "resolved": "ghcr.io/devcontainers/features/powershell@sha256:df7baa89598c93bfd15808641d9ec9eb03e0ccdf52e5de4cbbce9ab2d9755d18", + "integrity": "sha256:df7baa89598c93bfd15808641d9ec9eb03e0ccdf52e5de4cbbce9ab2d9755d18" + }, + "ghcr.io/devcontainers/features/python:1": { + "version": "1.8.0", + "resolved": "ghcr.io/devcontainers/features/python@sha256:fbcad6955caeecc5ad3f7886baf652e25cba5225a6c4c2287c536de2e5607511", + "integrity": "sha256:fbcad6955caeecc5ad3f7886baf652e25cba5225a6c4c2287c536de2e5607511" + } + } +} From 4482f9d21cb8b7e89cf038c375cb2e5ee9d6ef98 Mon Sep 17 00:00:00 2001 From: Frank Lesniak Date: Tue, 5 May 2026 01:00:10 +0000 Subject: [PATCH 07/10] chore: remove pip cache from Python setup in CI workflows --- .github/workflows/data-ci.yml | 1 - .github/workflows/precommit-ci.yml | 1 - 2 files changed, 2 deletions(-) diff --git a/.github/workflows/data-ci.yml b/.github/workflows/data-ci.yml index 9dc6706..fee1f66 100644 --- a/.github/workflows/data-ci.yml +++ b/.github/workflows/data-ci.yml @@ -92,7 +92,6 @@ jobs: uses: actions/setup-python@v6 with: python-version: '3.x' - cache: 'pip' - name: Cache pre-commit hooks uses: actions/cache@v5 diff --git a/.github/workflows/precommit-ci.yml b/.github/workflows/precommit-ci.yml index aa90056..f291cb3 100644 --- a/.github/workflows/precommit-ci.yml +++ b/.github/workflows/precommit-ci.yml @@ -30,7 +30,6 @@ jobs: uses: actions/setup-python@v6 with: python-version: '3.x' - cache: 'pip' - name: Cache pre-commit hooks uses: actions/cache@v5 From ae1acbc1e2cbaf3c53ce04e19b11404fdc548dd2 Mon Sep 17 00:00:00 2001 From: Frank Lesniak Date: Tue, 5 May 2026 01:05:27 +0000 Subject: [PATCH 08/10] fix: simplify pre-commit installation in post-create script --- .devcontainer/post-create.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.devcontainer/post-create.sh b/.devcontainer/post-create.sh index 9ccc632..44b9a05 100644 --- a/.devcontainer/post-create.sh +++ b/.devcontainer/post-create.sh @@ -19,7 +19,7 @@ sudo -H env "PATH=$PATH" "$NPM_BIN" install -g "@openai/codex@latest" python -m venv .venv . .venv/bin/activate python -m pip install --upgrade pip -python -m pip install -e ".[dev]" pre-commit +python -m pip install pre-commit pre-commit install # Reclaim ownership in case a prior run created ~/.codex as root before the From d4d62af938b1c4c86259c6acf77fa32945722e82 Mon Sep 17 00:00:00 2001 From: Frank Lesniak Date: Mon, 4 May 2026 20:10:47 -0500 Subject: [PATCH 09/10] fix: update security vulnerability reporting instructions in bug report template --- .github/ISSUE_TEMPLATE/bug_report.yml | 6 +- .github/workflows/check-placeholders.yml | 425 ----------------------- .github/workflows/powershell-ci.yml | 12 +- 3 files changed, 14 insertions(+), 429 deletions(-) delete mode 100644 .github/workflows/check-placeholders.yml diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index 17389a5..fe770a1 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -10,7 +10,11 @@ body: value: | Thank you for reporting a bug. - Security vulnerabilities must be reported privately through [GitHub private vulnerability reporting](https://github.com/franklesniak/macOSLab/security/advisories/new), not through this form. + Security vulnerabilities must be reported privately through + [GitHub private vulnerability reporting][private-vulnerability-reporting], + not through this form. + + [private-vulnerability-reporting]: https://github.com/franklesniak/macOSLab/security/advisories/new - type: checkboxes id: pre_flight diff --git a/.github/workflows/check-placeholders.yml b/.github/workflows/check-placeholders.yml deleted file mode 100644 index f1aad36..0000000 --- a/.github/workflows/check-placeholders.yml +++ /dev/null @@ -1,425 +0,0 @@ -# Placeholder Check Workflow -# -# Purpose: Verify that OWNER/REPO placeholders have been replaced in issue -# templates and configuration files after cloning from the template repository. -# -# ============================================================================= -# IMPORTANT - WHY THIS WORKFLOW EXISTS -# ============================================================================= -# When you create a repository from this template, several files contain -# `OWNER/REPO` placeholders that MUST be replaced with your actual org/repo -# name. If forgotten, users will encounter broken links in the issue chooser -# and documentation. -# -# This workflow checks for these placeholders and fails if they are found, -# ensuring you don't accidentally ship broken links to your users. -# -# ============================================================================= -# HOW THIS WORKFLOW OPERATES -# ============================================================================= -# This workflow runs AUTOMATICALLY in all repositories created from this -# template. It is automatically DISABLED in the template repository itself -# to avoid spurious failures. -# -# Implementation: -# if: github.repository != 'franklesniak/copilot-repo-template' -# -# This means: -# - ✅ Zero configuration required for adopters -# - ✅ Workflow activates automatically on first push/PR -# - ✅ Template maintainers don't get spurious failures -# -# NO CONFIGURATION REQUIRED - the workflow runs automatically on: -# - Pull requests to any branch -# - Pushes to any branch -# - Manual workflow_dispatch triggers -# -# ============================================================================= -# HISTORICAL CONTEXT -# ============================================================================= -# Previous versions of this template required setting a `TEMPLATE_INITIALIZED` -# repository variable to enable this workflow. That approach was replaced with -# automatic detection to: -# - Reduce adoption friction (no configuration step to remember) -# - Eliminate the "forgot to configure" failure mode -# - Provide better out-of-box experience for template adopters -# ============================================================================= - -name: Check Placeholders - -on: - push: - pull_request: - workflow_dispatch: - -permissions: - contents: read - -jobs: - check-placeholders: - name: Check for OWNER/REPO Placeholders - runs-on: ubuntu-latest - - # Run automatically in all repos EXCEPT the template repository itself - # This removes the need for adopters to set any configuration variable - if: github.repository != 'franklesniak/copilot-repo-template' - - steps: - - name: Checkout repository - uses: actions/checkout@v6 - - - name: Check for OWNER/REPO placeholders - run: | - echo "Checking for OWNER/REPO placeholders in repository files..." - echo "" - - # ============================================================================= - # PLACEHOLDER CHECK STRATEGY - # ============================================================================= - # This check enforces different severity levels based on the impact of - # unreplaced placeholders: - # - # HARD FAIL (exit 1) - when file exists and contains placeholders: - # 1. config.yml - Active contact_links URLs containing OWNER/REPO - # 2. CONTRIBUTING.md - OWNER/REPO occurrences (clone instructions, issue links) - # 3. SECURITY.md - "[security contact email]" or "TODO: Replace" placeholders - # 4. CODE_OF_CONDUCT.md - "[INSERT CONTACT METHOD]" placeholder - # 5. Issue templates (.yml/.yaml) - OWNER/REPO in lines containing "http" - # 6. .github/ directory - recursive scan for https://github.com/OWNER/REPO links - # (excluding this workflow file itself) - # 7. .github/CODEOWNERS - @OWNER placeholder - # - # Note: Files 1-4 and 7 are optional. If a file doesn't exist, its check is - # skipped with a "file not found (optional)" message. - # - # WARNING ONLY (no exit 1): - # - README.md containing OWNER/REPO in link contexts (emits ::warning but - # does not fail CI, as README may contain legitimate examples) - # - # FILE-TYPE-AWARE COMMENT FILTERING: - # - YAML files: Lines starting with # are comments (ignored) - # - Markdown files: Single-line HTML comments are ignored - # (multi-line HTML comments are NOT filtered; see filter_single_line_html_comments) - # - Markdown #: These are HEADINGS, not comments (checked!) - # - CODEOWNERS: Lines starting with # are comments (ignored) - # - # This approach balances strictness (broken links = fail) with flexibility - # (warning-only patterns = nudge without blocking CI). - # ============================================================================= - - FOUND_PLACEHOLDERS=false - - # Helper function to filter YAML comments (lines starting with #) - filter_yaml_comments() { - grep -v "^[0-9]*:[[:space:]]*#" || true - } - - # Helper function to filter lines where the content (after the grep -n line number prefix) - # is itself a single-line HTML comment. Lines that contain HTML comments *and* other content - # are preserved so that placeholders outside the comment are still detected. - # Multi-line HTML comment blocks (where are on different lines) are NOT handled. - # Rationale: A proper multi-line filter while preserving grep -n line numbers is complex, - # and multi-line comments containing placeholders are rare in practice. - filter_single_line_html_comments() { - grep -v "^[0-9]*:[[:space:]]*[[:space:]]*$" || true - } - - echo "=== [1] Checking config.yml (contact_links URLs) ===" - CONFIG_FILE=".github/ISSUE_TEMPLATE/config.yml" - if [ -f "$CONFIG_FILE" ]; then - # Check for OWNER/REPO in URL lines (lines containing 'url:') - # Exclude commented-out lines (YAML comments start with #) - matches=$(grep -n "url:.*OWNER/REPO" "$CONFIG_FILE" 2>/dev/null | filter_yaml_comments || true) - if [ -n "$matches" ]; then - echo "❌ Found in $CONFIG_FILE:" - echo "$matches" - echo "::error file=$CONFIG_FILE::Found OWNER/REPO placeholder in active URL. Replace with your org/repo name." - FOUND_PLACEHOLDERS=true - else - echo "✅ No active OWNER/REPO URLs in config.yml" - fi - else - echo "⚠️ config.yml not found (optional)" - fi - - echo "" - echo "=== [2] Checking CONTRIBUTING.md for OWNER/REPO placeholders ===" - CONTRIB_FILE="CONTRIBUTING.md" - if [ -f "$CONTRIB_FILE" ]; then - # Check for OWNER/REPO, excluding HTML comments () - # Note: In Markdown, # is a heading, NOT a comment, so we don't filter it - matches=$(grep -n "OWNER/REPO" "$CONTRIB_FILE" 2>/dev/null | filter_single_line_html_comments || true) - if [ -n "$matches" ]; then - echo "❌ Found in $CONTRIB_FILE:" - echo "$matches" - echo "::error file=$CONTRIB_FILE::Found OWNER/REPO placeholder(s). Replace with your org/repo name." - FOUND_PLACEHOLDERS=true - else - echo "✅ No active OWNER/REPO placeholders in CONTRIBUTING.md" - fi - else - echo "⚠️ CONTRIBUTING.md not found (optional)" - fi - - echo "" - echo "=== [3] Checking SECURITY.md for placeholder markers ===" - SECURITY_FILE="SECURITY.md" - if [ -f "$SECURITY_FILE" ]; then - found_security_issues=false - - # Check for "[security contact email]" placeholder - email_matches=$(grep -n "\[security contact email\]" "$SECURITY_FILE" 2>/dev/null || true) - if [ -n "$email_matches" ]; then - echo "❌ Found placeholder email in $SECURITY_FILE:" - echo "$email_matches" - echo "::error file=$SECURITY_FILE::Found [security contact email] placeholder. Replace with your actual security contact." - found_security_issues=true - fi - - # Check for "TODO: Replace" marker - todo_matches=$(grep -n "TODO: Replace" "$SECURITY_FILE" 2>/dev/null || true) - if [ -n "$todo_matches" ]; then - echo "❌ Found TODO marker in $SECURITY_FILE:" - echo "$todo_matches" - echo "::error file=$SECURITY_FILE::Found TODO marker. Complete the security contact setup." - found_security_issues=true - fi - - # Check for OWNER/REPO in URLs (for Option C with direct links) - url_matches=$(grep -n "github\.com/OWNER/REPO" "$SECURITY_FILE" 2>/dev/null || true) - if [ -n "$url_matches" ]; then - echo "❌ Found OWNER/REPO placeholder in URL in $SECURITY_FILE:" - echo "$url_matches" - echo "::error file=$SECURITY_FILE::Found OWNER/REPO placeholder in URL. Replace with your actual repository details (e.g., octocat/hello-world)." - found_security_issues=true - fi - - if [ "$found_security_issues" = true ]; then - FOUND_PLACEHOLDERS=true - else - echo "✅ No placeholder markers in SECURITY.md" - fi - else - echo "⚠️ SECURITY.md not found (optional)" - fi - - echo "" - echo "=== [4] Checking CODE_OF_CONDUCT.md for placeholder markers ===" - COC_FILE="CODE_OF_CONDUCT.md" - if [ -f "$COC_FILE" ]; then - # Check for "[INSERT CONTACT METHOD]" placeholder, excluding HTML comments - contact_matches=$(grep -n "\[INSERT CONTACT METHOD\]" "$COC_FILE" 2>/dev/null | filter_single_line_html_comments || true) - if [ -n "$contact_matches" ]; then - echo "❌ Found placeholder in $COC_FILE:" - echo "$contact_matches" - echo "::error file=$COC_FILE::Found [INSERT CONTACT METHOD] placeholder. Replace with your actual contact method." - FOUND_PLACEHOLDERS=true - else - echo "✅ No placeholder markers in CODE_OF_CONDUCT.md" - fi - else - echo "⚠️ CODE_OF_CONDUCT.md not found (optional)" - fi - - echo "" - echo "=== [5] Scanning issue templates for OWNER/REPO in URLs ===" - # Use find to avoid issues with unmatched glob patterns - # Check both .yml and .yaml extensions - while IFS= read -r -d '' file; do - # Skip config.yml which we already checked more specifically - if [ "$file" = "$CONFIG_FILE" ]; then - continue - fi - - # Grep for OWNER/REPO, exclude YAML comment lines - # Only fail if line contains "http" (actual URLs, not examples in comments) - matches=$(grep -n "OWNER/REPO" "$file" | filter_yaml_comments | grep -v "placeholder:" || true) - if [ -n "$matches" ]; then - # Check if any matches contain http (actual URLs vs just text) - if echo "$matches" | grep -q "http"; then - echo "❌ Found URL with OWNER/REPO in $file:" - echo "$matches" - echo "::error file=$file::Found OWNER/REPO placeholder in URL. Replace with your org/repo name." - FOUND_PLACEHOLDERS=true - fi - fi - done < <(find .github/ISSUE_TEMPLATE -type f \( -name "*.yml" -o -name "*.yaml" \) -print0 2>/dev/null || true) - echo "✅ Issue template URL check complete" - - echo "" - echo "=== [6] Scanning .github/ directory for OWNER/REPO URLs ===" - # Recursive scan of .github/ for https://github.com/OWNER/REPO links - # This provides comprehensive coverage and catches any OWNER/REPO URLs - # that might be added to new files in the future. - # Note: Some files may be checked in earlier sections as well; this is - # intentional to ensure comprehensive coverage with the full URL pattern. - # Exclude this workflow file itself, plus instructional and historical - # files (style guides under .github/instructions/, the canonical - # constitution at .github/copilot-instructions.md, and the design- - # decision record at .github/TEMPLATE_DESIGN_DECISIONS.md), where the - # literal URL appears as didactic example text rather than as a live - # template placeholder. - while IFS= read -r -d '' file; do - # Skip this workflow file to avoid self-referential matches - if [ "$file" = ".github/workflows/check-placeholders.yml" ]; then - continue - fi - - # Skip purely instructional or historical files where the literal - # `https://github.com/OWNER/REPO` substring appears as didactic - # example text (style guides describing the required URL pattern, - # and design-decision records preserving historical rationale) - # rather than as a live template placeholder that adopters must - # replace. These files do not produce broken links for end users. - # Note: in shell `case` patterns, `*` matches any string including - # `/`, so `.github/instructions/*` covers files at any depth under - # that directory. - case "$file" in - .github/instructions/*|.github/TEMPLATE_DESIGN_DECISIONS.md|.github/copilot-instructions.md) - continue - ;; - esac - - # Determine file type for comment filtering - case "$file" in - *.yml|*.yaml) - matches=$(grep -n "https://github.com/OWNER/REPO" "$file" 2>/dev/null | filter_yaml_comments || true) - ;; - *.md) - matches=$(grep -n "https://github.com/OWNER/REPO" "$file" 2>/dev/null | filter_single_line_html_comments || true) - ;; - *) - matches=$(grep -n "https://github.com/OWNER/REPO" "$file" 2>/dev/null || true) - ;; - esac - - if [ -n "$matches" ]; then - echo "❌ Found OWNER/REPO URL in $file:" - echo "$matches" - echo "::error file=$file::Found https://github.com/OWNER/REPO placeholder. Replace with your org/repo name." - FOUND_PLACEHOLDERS=true - fi - done < <(find .github -type f \( -name "*.yml" -o -name "*.yaml" -o -name "*.md" \) -print0 2>/dev/null) - echo "✅ .github/ directory scan complete" - - echo "" - echo "=== [Warning] Optional: Checking README.md for OWNER/REPO in links ===" - README_FILE="README.md" - if [ -f "$README_FILE" ]; then - # Only fail if OWNER/REPO appears in active link contexts (contains http or markdown link syntax) - matches=$(grep -n "OWNER/REPO" "$README_FILE" | filter_single_line_html_comments | grep -E "(http|\]\()" || true) - if [ -n "$matches" ]; then - echo "⚠️ Found OWNER/REPO in link context in $README_FILE:" - echo "$matches" - echo "::warning file=$README_FILE::Found OWNER/REPO in link context. Consider replacing with your org/repo name." - # Note: This is a warning only, not a hard fail, as README may have legitimate examples - else - echo "✅ No OWNER/REPO in link contexts in README.md" - fi - fi - - echo "" - echo "=== [7] Checking CODEOWNERS for @OWNER placeholder ===" - CODEOWNERS_FILE=".github/CODEOWNERS" - if [ -f "$CODEOWNERS_FILE" ]; then - # Check for @OWNER placeholder (the pattern template adopters must replace) - # Exclude comment lines (lines starting with #) - matches=$(grep -n "@OWNER" "$CODEOWNERS_FILE" | grep -v "^[0-9]*:[[:space:]]*#" || true) - if [ -n "$matches" ]; then - echo "❌ Found @OWNER placeholder in $CODEOWNERS_FILE:" - echo "$matches" - echo "::error file=$CODEOWNERS_FILE::Found @OWNER placeholder. Replace with your GitHub username or team (e.g., @octocat or @my-org/my-team), or delete the file if CODEOWNERS is not needed." - FOUND_PLACEHOLDERS=true - else - echo "✅ No @OWNER placeholder in CODEOWNERS" - fi - else - echo "⚠️ CODEOWNERS file not found (optional)" - fi - - echo "" - echo "===========================================" - - if [ "$FOUND_PLACEHOLDERS" = true ]; then - echo "❌ Placeholders found that require replacement!" - echo "" - echo "Please replace placeholders with your actual values:" - echo " - OWNER/REPO: Replace with your organization/repository name" - echo " - @OWNER: Replace with your GitHub username or team" - echo " - [INSERT CONTACT METHOD]: Replace with your Code of Conduct contact info" - echo "See the Template Setup Checklist in README.md for details." - exit 1 - else - echo "✅ All placeholder checks passed." - fi - - - name: Additional placeholder patterns check - run: | - echo "Checking for other common placeholder patterns..." - echo "" - - # ============================================================================= - # WARNING-ONLY CHECKS - # ============================================================================= - # These checks produce warnings, not errors. They help nudge adopters to - # customize templates without blocking CI. This is intentional because: - # - # 1. Some projects may intentionally keep certain placeholders temporarily - # 2. "Project Name" in README may be acceptable for early-stage projects - # 3. PR template placeholders don't break user-facing functionality - # - # The goal is visibility without friction—adopters see the warnings and can - # decide whether to address them. - # ============================================================================= - - # Check for other patterns that might indicate uncustomized templates - # These are warnings, not errors - - PATTERNS=( - "your-org" - "your-repo" - "YOUR_ORG" - "YOUR_REPO" - ) - - FOUND_WARNINGS=false - - echo "=== Checking issue templates for placeholder patterns ===" - for pattern in "${PATTERNS[@]}"; do - if grep -rn "$pattern" .github/ISSUE_TEMPLATE/ 2>/dev/null | grep -v "^[[:space:]]*#"; then - echo "::warning::Found potential placeholder pattern '$pattern' in issue templates" - FOUND_WARNINGS=true - fi - done - - echo "" - echo "=== Checking README for 'Project Name' placeholder ===" - if [ -f "README.md" ]; then - # Check for the default "# Project Name" header that should be customized - if grep -n "^# Project Name$" README.md; then - echo "::warning file=README.md::Found 'Project Name' placeholder in README.md - consider updating with your actual project name" - FOUND_WARNINGS=true - fi - fi - - echo "" - echo "=== Checking PR template for potential placeholders ===" - PR_TEMPLATE=".github/pull_request_template.md" - if [ -f "$PR_TEMPLATE" ]; then - for pattern in "${PATTERNS[@]}"; do - if grep -n "$pattern" "$PR_TEMPLATE" | grep -v "^[0-9]*:[[:space:]]*` immediately after any YAML front matter (or at the very top of the file if there is no front matter), and **before any other content**, including badges, links, the H1 heading, and any prose. A single optional blank line MAY appear between the front matter terminator (`---`) and the directive for readability; blank lines are not "content" for this rule. - Placement matters: markdownlint's inline `` directive only suppresses the rule for content that follows it. Placing the directive after badges or other long lines leaves those lines unprotected when the file is processed with default markdownlint settings outside this repo. @@ -137,12 +137,12 @@ This is an explicit carve-out from the "Prefer relative links" rule above. Issue For the issue-form `value:` and PR-template cases, relative forms are additionally unreliable across non-GitHub.com renderers, GitHub Mobile, email notifications, and copied/quoted content. -- Markdown links inside `.github/ISSUE_TEMPLATE/*.yml` issue-form `value:` blocks (for example, in `bug_report.yml`) and inside `.github/pull_request_template.md`, as well as the `url:` values inside `.github/ISSUE_TEMPLATE/config.yml`'s `contact_links` entries, that point to repo-internal files (for example, `SECURITY.md`, `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `README.md`) **MUST** use full absolute URLs of the form `https://github.com/OWNER/REPO/blob/HEAD/`. The `OWNER/REPO` placeholder follows this template's placeholder convention (see the comment block at the top of `CONTRIBUTING.md`) and is enforced by `.github/workflows/check-placeholders.yml` **if the downstream repository keeps that workflow** — `.github/workflows/check-placeholders.yml` is itself an optional adoption step (see `OPTIONAL_CONFIGURATIONS.md` and `GETTING_STARTED_*` guides) and may be removed once placeholders are substituted, so authors and adopters MUST NOT rely on the workflow as an unconditional CI guardrail. The `github.com` host is the assumed default; **GHES adopters MUST replace `github.com` with their GHES host** (e.g., `github.company.com`). The host substitution is not enforced by CI today (even when `.github/workflows/check-placeholders.yml` is kept, it only validates `OWNER/REPO`), so each affected file SHOULD include a brief inline comment reminding adopters of the host substitution, mirroring the convention already used in `.github/ISSUE_TEMPLATE/config.yml`. -- Repo-internal references that are not file paths (for example, the GitHub Security tab) **MUST** likewise use absolute URLs, such as `https://github.com/OWNER/REPO/security`. The same `github.com` host assumption applies. +- Markdown links inside `.github/ISSUE_TEMPLATE/*.yml` issue-form `value:` blocks (for example, in `bug_report.yml`) and inside `.github/pull_request_template.md`, as well as the `url:` values inside `.github/ISSUE_TEMPLATE/config.yml`'s `contact_links` entries, that point to repo-internal files (for example, `SECURITY.md`, `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `README.md`) **MUST** use full absolute URLs of the form `https://github.com/franklesniak/macOSLab/blob/HEAD/`. The old template placeholder workflow has been removed; authors MUST NOT introduce live `OWNER/REPO` placeholder URLs into this initialized repository. If the repository is ever migrated to a different host or owner/name, update these absolute URLs as part of that migration. +- Repo-internal references that are not file paths (for example, the GitHub Security tab) **MUST** likewise use absolute URLs, such as `https://github.com/franklesniak/macOSLab/security`. - Relative paths such as `../blob/HEAD/`, `blob/HEAD/`, `./`, or bare relative refs such as `(security)` **MUST NOT** be used in those files. Rationale: in issue-form `value:` blocks rendered at `/{owner}/{repo}/issues/new?...`, a link like `[SECURITY.md](blob/HEAD/SECURITY.md)` resolves to `/{owner}/{repo}/issues/blob/HEAD/SECURITY.md` (404), and `[Security tab](security)` resolves to `/{owner}/{repo}/issues/security` (404). In `config.yml` `contact_links` `url:` fields, GitHub itself rejects relative values at form-load time because the field is parsed as a URL rather than as Markdown. In `.github/pull_request_template.md` rendered at `/{owner}/{repo}/pull/`, a parent-relative form such as `[contributing guidelines](../blob/HEAD/CONTRIBUTING.md)` does resolve correctly on GitHub.com PR pages, but a sibling-relative form such as `[contributing guidelines](blob/HEAD/CONTRIBUTING.md)` resolves to `/{owner}/{repo}/pull/blob/HEAD/CONTRIBUTING.md` (404), and even working relative forms remain unreliable across non-GitHub.com renderers, GitHub Mobile, email notifications, and copied/quoted content; requiring absolute URLs avoids both pitfalls. - Use `blob/HEAD` rather than `blob/main` so the URL works regardless of the repository's default branch name. - This rule applies only to the files listed above. Tree-rendered Markdown such as `README.md`, `CONTRIBUTING.md`, and files under `docs/**` continue to follow the default "prefer relative links" guidance. -- The literal `https://github.com/OWNER/REPO/...` example URL is permitted to appear in didactic prose inside style-guide and design-decision files (`.github/instructions/**`, `.github/copilot-instructions.md`, and `.github/TEMPLATE_DESIGN_DECISIONS.md`); section [6] of `.github/workflows/check-placeholders.yml` skips those files specifically so that adopters are not forced to edit instructional/historical prose to satisfy placeholder CI. Section [6] also skips the workflow file itself (`.github/workflows/check-placeholders.yml`) to avoid self-referential matches against the literal URL embedded in its own grep patterns. The recursive scan in section [6] only enumerates `*.md`, `*.yml`, and `*.yaml` files (`find .github -type f \( -name "*.yml" -o -name "*.yaml" -o -name "*.md" \)`); other file types under `.github/` (for example, `.github/CODEOWNERS` or scripts) are not scanned by section [6] and are covered, where applicable, by other dedicated phases of the workflow rather than the recursive URL scan. Any other Markdown or YAML file under `.github/` (i.e., not `.github/instructions/**`, not `.github/copilot-instructions.md`, not `.github/TEMPLATE_DESIGN_DECISIONS.md`, and not `.github/workflows/check-placeholders.yml`) that contains the literal `https://github.com/OWNER/REPO` substring outside a single-line HTML comment (in Markdown) or a YAML comment line (in YAML) is treated as a live template placeholder and **MUST** be customized by adopters. (Section [6] of the placeholder workflow filters single-line HTML comments in Markdown and YAML comment lines in YAML before matching, so a single-line HTML comment such as `` will not by itself cause CI to fail; authors **SHOULD NOT** rely on that filter to ship live template URLs disguised as comments.) +- Live GitHub.com placeholder URLs that use generic owner/repo tokens **MUST NOT** appear in this initialized repository. Didactic examples that need to discuss generic repository URLs SHOULD use non-live placeholders such as `https://github.example/OWNER/REPO/...` or explain the pattern in prose rather than embedding a live GitHub URL. - The rule for issue-form YAML files (`.github/ISSUE_TEMPLATE/*.yml`) is mirrored in [`.github/instructions/yaml.instructions.md`](./yaml.instructions.md) so that contributors editing those YAML files receive the same guidance. The two instruction files are intentionally self-contained: each may be removed independently in downstream repositories that do not need it, and each restates the rule rather than relying on the other. ## ADR Standards diff --git a/.github/instructions/gitattributes.instructions.md b/.github/instructions/gitattributes.instructions.md index 1bbfa9d..6d3b0a7 100644 --- a/.github/instructions/gitattributes.instructions.md +++ b/.github/instructions/gitattributes.instructions.md @@ -14,12 +14,12 @@ description: "Rules for .gitattributes entries, including line-ending pinning fo - **Status:** Active - **Owner:** Repository Maintainers - **Last Updated:** 2026-04-21 -- **Scope:** Applies to any `.gitattributes` file in repositories that adopt these instructions, independent of programming language. Governs how committed text artifacts are protected against platform-dependent checkout rewriting. +- **Scope:** Applies to any `.gitattributes` file in this repository, independent of programming language. Governs how committed text artifacts are protected against platform-dependent checkout rewriting. - **Related:** [Repository Copilot Instructions](../copilot-instructions.md) ## Purpose and Scope -This file defines the normative rule for entries in `.gitattributes` that protect byte-exact text artifacts from platform-dependent line-ending rewriting. It applies to every `.gitattributes` file in any repository that adopts these instructions, regardless of the programming languages used in the repository. +This file defines the normative rule for entries in `.gitattributes` that protect byte-exact text artifacts from platform-dependent line-ending rewriting. It applies to every `.gitattributes` file in this repository, regardless of the programming languages used in the repository. > **Note:** This document uses [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119) keywords (**MUST**, **MUST NOT**, **SHOULD**, **SHOULD NOT**, **MAY**) to indicate requirement levels. @@ -48,9 +48,9 @@ tests/**/golden/*.json text eol=lf A blanket rule such as `* text=auto` **MUST NOT** be treated as a substitute for per-path `eol=lf` pinning. `text=auto` lets Git auto-detect text files and normalize to LF in the repository, but it does **not** force LF on Windows checkouts when `core.autocrlf=true` is in effect: Git still applies the working-tree line-ending conversion configured on the host and can rewrite LF to CRLF on checkout. Only explicit `eol=lf` on an artifact path guarantees that the bytes in the working tree match the bytes in the repository on every platform. -## Defaults Shipped by This Template +## Repository Defaults -This template ships a repo-root `.gitattributes` file with LF-pinning defaults for common byte-exact fixture locations: +This repository ships a repo-root `.gitattributes` file with LF-pinning defaults for common byte-exact fixture locations: - `tests/**/golden/**` - `tests/**/goldens/**` @@ -61,7 +61,7 @@ This template ships a repo-root `.gitattributes` file with LF-pinning defaults f These paths are assumed to contain **text** fixtures. To keep the defaults safe when binary assets are committed under the same directories (for example, `.png` screenshots under `__snapshots__/`), the shipped `.gitattributes` also declassifies a curated list of common binary extensions (images, documents and archives, compiled artifacts, audio and video, fonts) using the `binary` macro so that Git does not apply line-ending conversion to them. -Downstream adopters **MUST** extend these entries whenever they introduce a new byte-exact fixture location that is not already covered (for example, a project-specific `expected/` directory, a `golden_files/` tree, or signed payloads under a custom path). New entries **SHOULD** follow the "as narrow as practical" guidance above. Existing template entries **SHOULD NOT** be removed unless the maintainer has confirmed that no byte-exact comparison exists in the repository that depends on those paths. +Maintainers **MUST** extend these entries whenever they introduce a new byte-exact fixture location that is not already covered (for example, a project-specific `expected/` directory, a `golden_files/` tree, or signed payloads under a custom path). New entries **SHOULD** follow the "as narrow as practical" guidance above. Existing entries **SHOULD NOT** be removed unless the maintainer has confirmed that no byte-exact comparison exists in the repository that depends on those paths. ### Excluding Binary Files Under Fixture Paths @@ -83,7 +83,7 @@ The Git-layer rule defined here is necessary but not sufficient for byte-exact s - Tools that **compare** fixtures **SHOULD** read bytes in a mode that does not perform its own newline translation (for example, binary mode) when the comparison is byte-exact. - Hashing and signing tools **SHOULD** operate on raw bytes and **MUST NOT** depend on on-disk text normalization. -These language-specific concerns are out of scope for this instructions file; they are addressed in the relevant language instructions (for example, `python.instructions.md`, `powershell.instructions.md`). The Git-layer rule and the producer/consumer rules are complementary: each alone is insufficient, and both are needed for stable byte-exact artifacts across platforms. +These language-specific concerns are out of scope for this instructions file; they are addressed in the relevant language instructions when this repository has matching source files (for example, `powershell.instructions.md` for PowerShell code). The Git-layer rule and the producer/consumer rules are complementary: each alone is insufficient, and both are needed for stable byte-exact artifacts across platforms. ## Rationale diff --git a/.github/instructions/python.instructions.md b/.github/instructions/python.instructions.md deleted file mode 100644 index aa3d738..0000000 --- a/.github/instructions/python.instructions.md +++ /dev/null @@ -1,234 +0,0 @@ ---- -applyTo: "**/*.py" -description: "Python coding standards: portability-first by default, modern-advanced when the stack requires it." ---- - - - -# Python Writing Style - -**Version:** 1.2.20260503.4 - -## Metadata - -- **Status:** Active -- **Owner:** Repository Maintainers -- **Last Updated:** 2026-05-03 -- **Scope:** Defines Python coding standards for all Python files in this repository, including modules, scripts, tests, and tooling. Covers style, structure, error handling, testing, and documentation requirements. -- **Related:** [Repository Copilot Instructions](../copilot-instructions.md) - -## Purpose and Scope - -This guide establishes the Python coding standards for the repository. Code **MUST** be maintainable, deterministic, and security-conscious. The style adapts to project constraints: **stdlib-first, portability-first** by default, shifting to **modern-advanced** when required by the technology stack (async frameworks, type-heavy APIs, etc.). - -> **Note:** This document uses [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119) keywords (**MUST**, **MUST NOT**, **SHOULD**, **SHOULD NOT**, **MAY**) to indicate requirement levels. - -## Executive Summary: Author Profile - -The default style is a highly disciplined **stdlib-first, portability-first** approach. When code can reasonably run on a minimal, widely available Python baseline, it **SHOULD**: minimize dependencies, avoid unnecessary metaprogramming, and prefer clarity over cleverness. - -This baseline is not dogma. When external constraints require modern Python (e.g., `typing`-heavy APIs, async I/O, Pydantic models, FastAPI), the style intentionally shifts to a **modern-advanced** posture. - -## Quick Reference Checklist - -### Layout and Formatting - -- **[All]** **MUST** use 4 spaces; never tabs. -- **[All]** **MUST** follow PEP 8/PEP 257; line length target **<= 100** (**MAY** exceed for URLs/long strings when readability wins). -- **[All]** **MUST** keep formatting tool-friendly: **MUST NOT** hand-align with extra whitespace. -- **[All]** **SHOULD** use f-strings for interpolation; **SHOULD NOT** use `%` or `.format()` formatting except when required. -- **[All]** **MUST NOT** include trailing whitespace; files **MUST** end with a single newline. - -### Naming and Structure - -- **[All]** **MUST** use `snake_case` for functions/variables; `PascalCase` for classes; `UPPER_SNAKE_CASE` for constants. -- **[All]** Modules **SHOULD** be nouns; functions **SHOULD** be verbs. -- **[All]** **SHOULD NOT** use abbreviations; names **SHOULD** be explicit and descriptive. -- **[All]** **SHOULD** keep functions small and single-purpose; **SHOULD** avoid deep nesting. - -### Documentation - -- **[All]** Every public module/class/function **MUST** have a docstring. -- **[All]** Docstrings **MUST** emphasize contract: inputs, outputs, errors, edge cases, examples. -- **[All]** Inline comments **SHOULD** explain "why," not "what." - -### Error Handling - -- **[All]** **SHOULD** use exceptions over sentinel values unless explicitly modeling "no result." -- **[All]** **MUST** catch narrowly; **MUST NOT** use bare `except:` unless re-raising immediately. -- **[All]** Errors **MUST** be actionable: **MUST** include context, **MUST** preserve original exceptions (`raise ... from e`). - -### Types, Testing, and Tooling - -- **[Baseline]** **MAY** use type hints opportunistically for public APIs and complex structures. -- **[Modern]** Type hints are expected broadly; **MUST** run static checking (e.g., mypy/pyright) in CI. -- **[All]** Tests **MUST** exist for non-trivial logic; **SHOULD** use `pytest` unless repo standard differs. -- **[All]** **SHOULD** use Black for formatting and Ruff for linting (configured via `pyproject.toml` and pre-commit hooks). - -## Baseline vs Modern-Advanced Mode - -### Baseline Mode (Default) - -Use this mode unless the project's constraints clearly require modern-only features. - -**Goals:** - -- Minimal dependencies (stdlib-first). -- Deterministic behavior; explicit control flow. -- Easy to read without specialized tooling. - -**Rules:** - -- **SHOULD** avoid metaprogramming, magic decorators, and clever one-liners. -- **SHOULD** use explicit loops over heavily nested comprehensions when clarity improves. -- **SHOULD** use plain datatypes (`dict`, `list`, `tuple`) over heavy frameworks for simple tasks. -- **SHOULD** keep I/O boundaries obvious (pure functions where possible; isolate side effects). - -### Modern-Advanced Mode (When Required) - -Use when: - -- The repo depends on modern frameworks (FastAPI, Pydantic, async stacks, etc.). -- The domain benefits from stronger contracts (rich types, schemas, validation). -- Performance or concurrency requirements demand modern primitives. - -**Rules:** - -- **MUST** use type hints pervasively (inputs/outputs, key variables in complex logic). -- **SHOULD** use `pathlib.Path` over `os.path` for paths. -- **SHOULD** use structured logging when available. -- For async: **SHOULD** use `async`/`await`, `anyio`/`asyncio` patterns; **MUST** keep sync/async boundaries explicit. -- **MAY** raise domain-specific exceptions where helpful. - -## Code Layout and Formatting - -- Indentation: **MUST** use 4 spaces, no tabs. -- Imports: - - **MUST** group as: standard library, third-party, local. - - Within each group: **SHOULD** sort alphabetically, one import per line when reasonable. - - **MUST NOT** use wildcard imports. - -Example: - -```python -from __future__ import annotations - -import json -import logging -from dataclasses import dataclass -from pathlib import Path - -# import third_party_package # Third-party imports go here - -from myproject.core.models import Requirement -``` - -## Naming Conventions - -- Functions: **SHOULD** use `verb_noun` (e.g., `parse_markdown`, `build_index`, `validate_input`). -- Classes: **SHOULD** use `NounPhrase` (e.g., `RequirementGraph`, `DesignAnalyzer`). -- Variables: **MUST** be explicit, non-abbreviated (e.g., `requirements_text`, not `req_txt`). -- Boolean variables: **SHOULD** use `is_`, `has_`, `should_` prefixes (e.g., `is_valid`, `has_errors`). - -## Documentation and Comments - -### Docstrings - -**MUST** use docstrings for all public interfaces. **SHOULD** use a consistent, readable style: - -- Short summary line. -- Longer description if needed. -- Arguments, returns, raises. -- Examples for tricky behavior. - -Example: - -```python -def parse_requirements_markdown(markdown_text: str) -> list[str]: - """ - Parse requirements from a markdown document. - - Args: - markdown_text: Raw markdown content. - - Returns: - A list of normalized requirement strings. - - Raises: - ValueError: If the markdown is empty or cannot be parsed. - """ -``` - -### Inline Comments - -- **SHOULD** explain rationale and invariants. -- **SHOULD NOT** narrate obvious code. - -## Error Handling - -- **MUST** raise specific exceptions (`ValueError`, `KeyError`, `TypeError`, or custom domain exceptions). -- **MUST** always add context. **MUST** preserve the original exception when wrapping: - -```python -try: - parsed = json.loads(text) -except json.JSONDecodeError as error: - raise ValueError(f"Invalid JSON in requirements file: {path}") from error -``` - -- **MUST NOT** swallow errors silently. If you must continue, **MUST** log at `debug` or `warning` with rationale. - -## Logging and Output - -- **MUST** use `logging` for non-test code. **MUST NOT** use `print` except in CLI entrypoints. -- Logging messages **MUST** be human-actionable and **SHOULD** include identifiers/paths when relevant. - -## Data Modeling - -### Baseline - -- **SHOULD** use `dataclasses` for lightweight models. -- **SHOULD** avoid "magic" model behavior unless it clearly reduces complexity. - -### Modern - -- If using Pydantic or similar, **SHOULD** enforce schema boundaries at the edges (I/O) and keep the core logic mostly framework-agnostic. - -## Filesystem and Paths - -- **SHOULD** use `pathlib.Path`. -- *Note on Python version callouts in this section:* the bullets below intentionally cite minimum Python versions for specific APIs (for example `Path.is_relative_to()` on 3.9+ and `Path.walk(...)` on 3.12+) even though this repository's own `pyproject.toml` currently sets `requires-python = ">=3.13"`. These callouts exist for **portability**, consistent with this guide's "portability-first by default" posture (see *Executive Summary*), so that downstream projects which adopt these instructions on a lower Python baseline still receive correct guidance. Projects whose own baseline is 3.13+ **MAY** simplify call sites to the 3.13+ form (for example, always using `is_relative_to(...)` and `Path.walk(follow_symlinks=False)`), but **MUST NOT** weaken the underlying symlink-rejection or containment requirements. -- **MUST NOT** follow symbolic links during recursive directory discovery in code that processes untrusted, repo-supplied, or otherwise externally influenced fixtures, configuration, or input files. Prefer explicit traversal with `os.walk(directory, followlinks=False)` or `base_path.walk(follow_symlinks=False)` on Python 3.12+ (where `base_path` is a concrete `pathlib.Path` instance). Prune or skip entries that are symlinks — for `os.walk` and `pathlib.Path.walk`, which both yield directory and file *names* as strings relative to the per-iteration directory, test by joining `name` to that per-iteration directory: `os.path.islink(os.path.join(dirpath, name))` for `os.walk`, or `(dirpath / name).is_symlink()` for `Path.walk` (where `dirpath` is the `Path` yielded by the current iteration); when using `os.scandir`, test `entry.is_symlink()` on the `os.DirEntry`. Verify each yielded path remains under the declared discovery root by comparing resolved paths on the candidate path instance, e.g. `candidate.resolve().relative_to(base_path.resolve())` (which raises `ValueError` when `candidate` is outside `base_path` — callers **MUST** handle that as a containment failure), or use an equivalent safe boundary check. -- **MUST** validate the discovery root (`base_path`) itself before walking. In addition to the per-yielded-path containment check, callers **MUST** verify that `base_path.resolve()` is contained within `trusted_root.resolve()`, where `trusted_root` is an independently determined allowlisted root, such as `REPO_ROOT`, `PROJECT_ROOT`, or a configured root, and is not derived from `base_path`. Use a concrete, unambiguous containment check on the resolved paths — for example `base_path.resolve().is_relative_to(trusted_root.resolve())` on Python 3.9+, or `base_path.resolve().relative_to(trusted_root.resolve())` (catching `ValueError` and treating it as a containment failure) — and **MUST NOT** fall back to string-prefix comparisons such as `str(base_path).startswith(str(trusted_root))`, which are vulnerable to sibling-directory false positives (for example `/srv/data-evil` vs. `/srv/data`) and to separator/normalization mismatches. This discovery-root validation **MUST NOT** be anchored against `base_path` itself (for example `base_path.resolve().is_relative_to(base_path.resolve())`, which is trivially true), because `base_path.resolve()` follows any symlink at or above the discovery root and a self-anchored check can silently accept a `base_path` that re-anchors to an external target. Note that this prohibition is specifically about the **discovery-root check**: the per-yielded-entry containment check in the previous bullet *is* expressed relative to `base_path` (for example `candidate.resolve().relative_to(base_path.resolve())`), and remains correct **once `base_path` itself has been independently validated against `trusted_root`** as required here. `os.walk(..., followlinks=False)` and `base_path.walk(follow_symlinks=False)` on Python 3.12+ only refuse to descend into symlinked subdirectories; they do not detect the case where `base_path` is itself a symlink, or has a symlinked ancestor, pointing outside the trusted area. Refuse discovery if the resolved-root containment check fails. -- **SHOULD** additionally reject discovery roots whose ancestor chain from `trusted_root` down to `base_path` contains symlinks, as a stricter defense-in-depth posture for high-sensitivity inputs. **SHOULD**, not **MUST**, is used here because legitimate environments can have symlinked ancestors, such as macOS `/tmp` → `/private/tmp`, container bind-mount layouts, or developer workspaces under symlinked home directories. -- **MUST** apply the same symlink and containment guidance to non-walk discovery APIs, not only recursive directory discovery, and to *every* discovered entry regardless of whether it is a file or a directory. Entries returned by `base_path.glob(...)`, `base_path.rglob(...)`, `os.scandir(base_path)`, or `os.listdir(base_path)` in untrusted, repo-supplied, or externally influenced discovery contexts **MUST** be rejected by a per-entry symlink check such as `Path.is_symlink()`, `os.path.islink(...)`, or `os.DirEntry.is_symlink()`, and the resolved candidate **MUST** remain contained within the same `trusted_root` introduced above using the same concrete check (for example `candidate.resolve().is_relative_to(trusted_root.resolve())` on Python 3.9+, or `candidate.resolve().relative_to(trusted_root.resolve())` with `ValueError` handled as a containment failure). Note that `os.listdir(base_path)` yields bare entry *names* and that `os.scandir` yields `os.DirEntry` objects whose `name` attribute is also a bare name; **if you are operating on the bare name** (the result of `os.listdir(...)` or `DirEntry.name`), callers **MUST** join the name back to `base_path` (for example `Path(base_path) / name` or `os.path.join(base_path, name)`) before any `os.path.islink(...)`, `Path.resolve()`, or containment check, mirroring the per-iteration join required for `os.walk` and `pathlib.Path.walk` above. No additional join is required when using `os.DirEntry.is_symlink()` or `os.DirEntry.path` directly, because the `DirEntry` object already carries its full path and those APIs are safe to use as-is. -- **SHOULD NOT** use `base_path.rglob("*")` or `base_path.glob("**/*")` (where `base_path` is a concrete `pathlib.Path` instance) for fixture, config, or input discovery unless the implementation also makes symlink handling and root-containment checks explicit. Prefer the safer `os.walk` / `pathlib.Path.walk` patterns above for untrusted or externally influenced discovery roots. -- This guidance implements the repo-wide rule "Refuse path traversal and symlink escapes" in [`.github/copilot-instructions.md`](../copilot-instructions.md) § "Non-negotiable Safety and Security Rules" item 3, "Allowlisted file access only". -- **SHOULD NOT** rely on current working directory; **SHOULD** resolve paths from a clear root (e.g., repo root or config). - -## Tests - -- Non-trivial logic **MUST** have tests. -- Tests **MUST** be deterministic and **MUST NOT** depend on network unless explicitly marked/integration-only. -- **SHOULD** use table-driven tests for parsing/validation logic. - -## Performance and Safety - -- **SHOULD** prefer clarity first; optimize only when needed and measured. -- **SHOULD** avoid quadratic algorithms in obvious hot paths (parsers, matchers, large loops). -- **MUST** validate untrusted input at boundaries; **MUST NOT** use `eval`. - -## "Done" Definition for Python Changes - -A PR/commit is considered complete when: - -- **All pre-commit hooks MUST pass** (Black formatting, Ruff linting, trailing whitespace, etc.). - - **MUST** run `pre-commit run --all-files` locally before pushing. - - Pre-commit hooks **MAY** modify files (auto-fix formatting/linting). **MUST** always review and commit these changes before pushing. - - **If pre-commit CI fails:** **MUST** pull the branch, run `pre-commit run --all-files`, commit fixes, and push again. -- Code **MUST** conform to this style guide. -- Public APIs **MUST** have docstrings. -- Errors **MUST** be explicit and actionable. -- Tests **MUST** exist for core logic and pass locally/CI. -- **MUST NOT** include debug prints; logging **SHOULD** be appropriate for runtime visibility. -- All CI checks **MUST** pass. diff --git a/.github/instructions/terraform.instructions.md b/.github/instructions/terraform.instructions.md deleted file mode 100644 index 0c0dd5d..0000000 --- a/.github/instructions/terraform.instructions.md +++ /dev/null @@ -1,4083 +0,0 @@ ---- -applyTo: "**/*.tf,**/*.tfvars,**/*.tftest.hcl,**/*.tf.json,**/*.tftpl,**/*.tfbackend" -description: "Terraform coding standards: secure, modular, and well-documented infrastructure as code." ---- - -# Terraform Writing Style - -**Version:** 2.3.20260503.0 - -## Keywords - -The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**, **SHOULD**, **SHOULD NOT**, **RECOMMENDED**, **MAY**, and **OPTIONAL** are defined in [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119). Requirements apply when their scope is present (e.g., module rules apply only when modules exist). - -## Quick Reference Checklist - -This checklist provides a quick reference for both human developers and LLMs (like GitHub Copilot) to follow the Terraform style guidelines. Each item includes a scope tag indicating applicability: - -- **[All]** — Applies to all Terraform files -- **[Module]** — Applies when developing reusable modules -- **[Root]** — Applies to root configurations (deployments) -- **[Test]** — Applies to test files (`.tftest.hcl`) - -> **Scope reminder:** Items tagged **[Module]**, **[Root]**, or **[Test]** are mandatory **when those constructs are present**. If the construct does not exist in your repo, the requirement is not yet applicable. - -### Formatting and Style (Quick Reference) - -- **[All]** Code **MUST** pass `terraform fmt` without modifications -- **[All]** Code **MUST** use 2 spaces for indentation, never tabs -- **[All]** Files **MUST** use UTF-8 encoding -- **[All]** Files **MUST** end with a single newline -- **[All]** Lines **SHOULD NOT** exceed 120 characters except for long strings or URLs -- **[All]** Blank lines **MUST** be completely empty (no whitespace) -- **[All]** Comments **MUST** use `#` for single-line; `/* */` **MAY** be used for multi-line - -### Naming Conventions (Quick Reference) - -- **[All]** Resources **MUST** use `snake_case` names -- **[All]** Variables **MUST** use `snake_case` with descriptive names -- **[All]** Outputs **MUST** use `snake_case` matching resource attribute patterns -- **[Module]** Module directory names **MUST** use hyphen-separated lowercase words -- **[All]** Data sources **MUST** be prefixed with purpose when multiple exist -- **[All]** Locals **MUST** use `snake_case` with descriptive names -- **[All]** Boolean variables **SHOULD** use `enable_*`, `is_*`, or `has_*` prefixes -- **[All]** Globally unique resource names **SHOULD** include random suffixes or organization prefixes - -### File Organization (Quick Reference) - -- **[All]** Every Terraform directory **MUST** have a `versions.tf` file -- **[All]** Input variables **MUST** be in `variables.tf` -- **[All]** Outputs **MUST** be in `outputs.tf` -- **[Root]** Root modules **MUST** have a `providers.tf` file -- **[Root]** Root modules **MUST** have a `backend.tf` or backend configuration -- **[Module]** Modules **MUST** include a `README.md` -- **[Module]** Modules **SHOULD** include `examples/` directory -- **[Module]** Modules **SHOULD** include `tests/` directory -- **[All]** Template files **SHOULD** use `.tftpl` extension -- **[All]** Template files **SHOULD** be placed in a `templates/` subdirectory -- **[Root]** Large root modules **MAY** split resources into domain-specific files - -### Variable and Output Design (Quick Reference) - -- **[All]** Variables **MUST** include a `description` -- **[All]** Variables **MUST** include explicit `type` constraint -- **[All]** Optional variables **MUST** have a `default` value -- **[All]** Sensitive variables **MUST** be marked with `sensitive = true` -- **[All]** Variables with constrained values **SHOULD** use `validation` blocks -- **[All]** Outputs **MUST** include a `description` -- **[All]** Sensitive outputs **MUST** be marked with `sensitive = true` -- **[All]** Variables **SHOULD** explicitly set `nullable` to document null-handling behavior -- **[All]** `try()` **SHOULD** be used for defensive access to attributes that may not exist - -### Continuous Validation (Quick Reference) - -- **[All]** `check` blocks **MAY** be used for continuous validation - -### Resource Configuration (Quick Reference) - -- **[All]** Meta-arguments **MUST** appear first in resource blocks -- **[All]** Required arguments **MUST** appear before optional arguments -- **[All]** Nested blocks **MUST** appear last in resource blocks -- **[All]** Resources **MUST** include required tags -- **[Root]** Provider-level default tags **SHOULD** be configured -- **[All]** Local values **SHOULD** be used for computed or merged tags -- **[All]** `precondition` blocks **SHOULD** validate assumptions before resource creation -- **[All]** `postcondition` blocks **SHOULD** validate resource state after creation -- **[All]** `depends_on` **SHOULD** be avoided unless dependencies are not inferable -- **[All]** `for_each` **SHOULD** be preferred over `count` for collections -- **[All]** Dynamic blocks **SHOULD** be used sparingly -- **[All]** `prevent_destroy` **SHOULD** be used for critical resources -- **[All]** `ignore_changes` **SHOULD** be used for attributes managed outside Terraform -- **[All]** Custom `timeouts` blocks **MAY** be used for long-running resource operations - -### Module Design (Quick Reference) - -- **[Module]** Modules **MUST** have a single, well-defined responsibility -- **[Module]** Modules **MUST** specify required Terraform and provider versions -- **[Module]** Module inputs **MUST** use consistent naming across modules -- **[Module]** Required module variables **SHOULD** be minimized -- **[Module]** Complex inputs **SHOULD** use object types with documented structure -- **[Module]** Modules **SHOULD** expose only necessary outputs -- **[Module]** Published modules **MUST** use semantic versioning -- **[Module]** Modules accepting multiple provider configurations **MUST** use `configuration_aliases` -- **[All]** Module-level `depends_on` **SHOULD** be avoided unless implicit dependencies are insufficient - -### Refactoring (Quick Reference) - -- **[All]** Resource renames **MUST** use `moved` blocks instead of manual state commands -- **[All]** Existing infrastructure imports **SHOULD** use `import` blocks instead of CLI commands -- **[All]** Resources removed from management **SHOULD** use `removed` blocks -- **[All]** Direct state manipulation commands **SHOULD** be avoided - -### State Management (Quick Reference) - -- **[Root]** Root modules **MUST** configure a remote backend -- **[Root]** State files **MUST** be encrypted at rest -- **[Root]** State locking **MUST** be enabled -- **[All]** Local state files **MUST NOT** be used in production -- **[All]** State files **MUST NOT** be committed to version control -- **[All]** `terraform apply -target` **SHOULD NOT** be used in normal workflows -- **[Root]** New state storage infrastructure **SHOULD** follow the bootstrap workflow -- **[All]** Plan output **MUST** be reviewed for unexpected destroys or replacements before applying -- **[Root]** State storage buckets **MUST** have versioning enabled for production use -- **[All]** Manual state backups **SHOULD** be created before risky operations -- **[All]** State files **MUST NOT** be manually edited - -### Cross-Stack Data Sharing (Quick Reference) - -- **[Root]** Cross-stack data **SHOULD** be shared via cloud-native parameter stores -- **[Root]** `terraform_remote_state` **MAY** be used with documented coupling implications -- **[All]** Secrets **MUST NOT** be shared via parameter stores or remote state - -### Provider Management (Quick Reference) - -- **[All]** Provider versions **MUST** be constrained -- **[All]** `.terraform.lock.hcl` **MUST** be committed to version control -- **[All]** Pessimistic constraint operator (`~>`) **SHOULD** be used for providers -- **[All]** Multi-region or multi-account deployments **MUST** use provider aliases - -### Security (Quick Reference) - -- **[All]** Secrets **MUST NOT** appear in `.tf` files -- **[All]** Secrets **MUST NOT** have default values -- **[All]** Secrets **MUST** be provided via environment variables or secret managers -- **[Root]** State backends **MUST** enable encryption -- **[All]** IAM policies **MUST** follow least-privilege principles -- **[All]** Wildcard actions **SHOULD NOT** be used in IAM policies -- **[All]** Sensitive values **MUST NOT** be used in `for_each` or `count` expressions -- **[All]** Sensitive outputs **MUST NOT** be logged in CI/CD pipelines - -### Testing (Quick Reference) - -- **[Test]** Test files **MUST** use `.tftest.hcl` extension -- **[Test]** Test files **SHOULD** be in a `tests/` directory -- **[Test]** Tests **MUST** include at least one `run` block -- **[Test]** Each `run` block **MUST** include at least one `assert` -- **[Test]** Variable validation **SHOULD** be tested -- **[Test]** Unit tests **SHOULD** use `command = plan` -- **[Test]** Integration tests **MAY** use `command = apply` -- **[Module]** Modules **SHOULD** include corresponding Terraform tests -- **[Test]** Mock providers **SHOULD** be used for unit tests -- **[Test]** Negative test cases **SHOULD** use `expect_failures` - -### Documentation (Quick Reference) - -- **[Module]** Modules **MUST** have a `README.md` with usage examples -- **[All]** Inline comments **SHOULD** explain "why," not "what" -- **[All]** TODO comments **SHOULD** include username and context -- **[All]** Terraform Registry reference URLs in comments **MUST** use the `latest` path segment, not a pinned provider or module version -- **[All]** Error messages in validation blocks **SHOULD** be actionable and reference valid options or acceptable ranges - -### Code Authoring Guidelines (Quick Reference) - -The following guidelines apply to all code authors, including human developers and AI assistants such as GitHub Copilot: - -- **[All]** Authors **MUST NOT** invent providers, modules, or placeholder values without explicit confirmation of requirements -- **[All]** Authors **MUST** ask for or verify missing required information (e.g., `bucket`, `region`, `project_id`) rather than inserting assumptions -- **[All]** Authors **MUST NOT** include secrets, API keys, tokens, or hardcoded sensitive information in code -- **[All]** Authors **SHOULD** default to minimal, reproducible, and well-documented code -- **[All]** Authors **MUST** only modify backend configuration when explicitly required -- **[All]** Authors **SHOULD NOT** assume a default cloud provider; when the provider is not specified, authors **SHOULD** use provider-agnostic examples and document that provider selection is required -- **[All]** Authors **SHOULD** include `description` for all variables and outputs, and use `sensitive = true` as appropriate -- **[All]** Authors **MUST NOT** modify lock files (`.terraform.lock.hcl`) or commit state unless explicitly required - ---- - -## Terraform Version Requirements - -The following table summarizes Terraform version requirements for features referenced in this document: - -| Feature | Minimum Terraform Version | -| --- | --- | -| `moved` blocks | 1.1.0 | -| `nullable` variable attribute | 1.1.0 | -| `precondition` / `postcondition` blocks | 1.2.0 | -| `replace_triggered_by` lifecycle argument | 1.2.0 | -| `optional()` type constraint modifier | 1.3.0 | -| `terraform_data` resource | 1.4.0 | -| `check` blocks | 1.5.0 | -| `import` blocks | 1.5.0 | -| Native test framework (`terraform test`) | 1.6.0 | -| `removed` blocks | 1.7.0 | -| `mock_provider` in tests | 1.7.0 | -| `ephemeral` values | 1.10.0 | - -> **Note:** Examples in this document assume Terraform 1.10.0 or later unless otherwise noted. Users on older Terraform versions should verify feature availability before adopting specific patterns. - ---- - -### Version Upgrade Requirements - -The following requirements apply to Terraform version upgrades: - -- Version upgrades **MUST** be tested in non-production environments first -- State **SHOULD** be backed up before any version upgrade -- `.terraform.lock.hcl` **MUST** be updated and committed after version changes -- Major version upgrades **MUST** include review of the official upgrade guide - ---- - -## Formatting and Style - -### terraform fmt Compliance - -All Terraform code **MUST** pass `terraform fmt` without modifications. This is non-negotiable. - -**Verification command:** - -```bash -terraform fmt -check -recursive -``` - -**Auto-format command:** - -```bash -terraform fmt -recursive -``` - -**Pre-commit integration:** - -```yaml -- repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.105.0 - hooks: - - id: terraform_fmt -``` - -### Indentation Rules - -- Code **MUST** use 2 spaces for indentation -- Tabs **MUST NOT** be used -- Nested blocks **MUST** maintain consistent indentation -- Alignment of `=` signs is handled automatically by `terraform fmt` - -**Compliant:** - -```hcl -resource "aws_instance" "example" { - ami = var.ami_id - instance_type = var.instance_type - - tags = { - Name = var.instance_name - Environment = var.environment - } -} -``` - -### File Encoding - -All Terraform files **MUST** use UTF-8 encoding without BOM (Byte Order Mark). - -### File Endings - -All files **MUST** end with a single newline character. Trailing blank lines **MUST NOT** be present. - -### Line Length - -Lines **SHOULD NOT** exceed 120 characters. Exceptions are permitted for: - -- Long strings that cannot be reasonably split -- URLs in comments or string values -- Complex expressions where splitting reduces readability - -### Blank Lines - -Blank lines **MUST** be completely empty—they **MUST NOT** contain any whitespace characters (spaces or tabs). - -Use blank lines to: - -- Separate logical sections within a file -- Separate resource blocks -- Separate groups of related arguments within a block - -### Comment Style - -**Single-line comments:** - -Use `#` for single-line comments. Comments **SHOULD** be placed on their own line above the code they describe. - -```hcl -# Enable encryption to meet compliance requirements -resource "aws_s3_bucket_server_side_encryption_configuration" "main" { - bucket = aws_s3_bucket.main.id - # ... -} -``` - -**Multi-line comments:** - -Use `/* */` sparingly for multi-line explanations when a single `#` comment is insufficient. - -```hcl -/* - * This security group allows inbound traffic from the corporate VPN. - * CIDR ranges are managed by the network team and should not be - * modified without their approval. - */ -resource "aws_security_group" "vpn_access" { - # ... -} -``` - ---- - -## Naming Conventions - -### Resource Naming - -Resources **MUST** use `snake_case` for names. Names **SHOULD** be descriptive and indicate purpose. - -| Resource Type | Naming Pattern | Example | -| --- | --- | --- | -| Primary/main resource | `main` or descriptive name | `aws_instance.main` | -| Multiple of same type | Purpose-based suffix | `aws_instance.web_server` | -| Associated resources | Parent reference | `aws_security_group.web_server` | - -**Anti-patterns to avoid:** - -| Bad | Good | Reason | -| --- | --- | --- | -| `aws_instance.this` | `aws_instance.main` | "this" is not descriptive | -| `aws_instance.instance1` | `aws_instance.primary` | Numeric suffixes are meaningless | -| `aws_instance.MyInstance` | `aws_instance.my_instance` | Must be snake_case | -| `aws_instance.i` | `aws_instance.web_server` | Single-letter names lack meaning | - -### Variable Naming - -Variables **MUST** use `snake_case` and **MUST** be descriptive. - -| Category | Pattern | Example | -| --- | --- | --- | -| Simple values | `` or `_` | `instance_type`, `environment` | -| Lists/Sets | Plural nouns | `subnet_ids`, `security_group_ids` | -| Maps | `_map` or descriptive | `tags`, `instance_settings` | -| Booleans | `enable_*`, `is_*`, `has_*` | `enable_monitoring`, `is_public` | -| Resource references | `_id` or `_arn` | `vpc_id`, `role_arn` | - -**Compliant variable names:** - -```hcl -variable "environment" { - description = "Deployment environment (dev, staging, prod)" - type = string -} - -variable "enable_monitoring" { - description = "Enable CloudWatch detailed monitoring" - type = bool - default = false -} - -variable "subnet_ids" { - description = "List of subnet IDs for deployment" - type = list(string) -} -``` - -### Output Naming - -Outputs **MUST** use `snake_case` and **SHOULD** follow the pattern of the attribute being exposed. - -| Output Type | Pattern | Example | -| --- | --- | --- | -| Resource ID | `_id` | `instance_id`, `vpc_id` | -| Resource ARN | `_arn` | `role_arn`, `bucket_arn` | -| Resource name | `_name` | `bucket_name`, `cluster_name` | -| Endpoints/URLs | `_endpoint` | `rds_endpoint`, `api_endpoint` | -| Collections | Plural form | `instance_ids`, `subnet_ids` | - -### Module Naming - -Module directory names **MUST** use hyphen-separated lowercase words. - -**Compliant:** - -```text -modules/ -├── vpc-network/ -├── ec2-instance/ -├── rds-database/ -└── s3-bucket/ -``` - -**Non-compliant:** - -```text -modules/ -├── VpcNetwork/ # PascalCase -├── ec2_instance/ # snake_case -└── rdsdatabase/ # No separation -``` - -### Data Source Naming - -When multiple data sources of the same type exist, they **MUST** be prefixed with their purpose. - -**Single data source:** - -```hcl -data "aws_ami" "amazon_linux" { - # ... -} -``` - -**Multiple data sources:** - -```hcl -data "aws_ami" "web_server" { - # ... -} - -data "aws_ami" "database_server" { - # ... -} -``` - -### Local Value Naming - -Local values **MUST** use `snake_case` with descriptive names. - -```hcl -locals { - common_tags = { - Environment = var.environment - Project = var.project_name - ManagedBy = "terraform" - } - - instance_name = "${var.project_name}-${var.environment}" -} -``` - -### Boolean Naming Patterns - -Boolean variables and locals **SHOULD** use these prefixes: - -| Prefix | Use Case | Example | -| --- | --- | --- | -| `enable_*` | Feature flags | `enable_monitoring`, `enable_encryption` | -| `is_*` | State checks | `is_public`, `is_production` | -| `has_*` | Presence checks | `has_custom_domain`, `has_ssl_certificate` | - -### Globally Unique Resource Names - -Some cloud resources require globally unique names across all customers. Attempting to create these resources with simple, predictable names often results in `409 Conflict`, `BucketAlreadyExists`, or similar errors. - -#### Resources Requiring Globally Unique Names - -| Provider | Resources | -| --- | --- | -| **AWS** | S3 buckets, some IAM resources (roles with path-based ARNs) | -| **Azure** | Storage Accounts, Key Vaults, App Services, Cosmos DB accounts | -| **GCP** | GCS buckets, project IDs, Cloud SQL instances | - -#### Recommended Patterns - -**Pattern 1: Random suffix using `random_id`:** - -Use the `random_id` resource to generate a unique suffix that remains stable across applies: - -**AWS Example:** - -```hcl -resource "random_id" "bucket_suffix" { - byte_length = 4 -} - -resource "aws_s3_bucket" "main" { - bucket = "${var.project_name}-${var.environment}-${random_id.bucket_suffix.hex}" -} -``` - -> **Note:** If you are not using AWS, replace `aws_s3_bucket` and the `bucket` argument with the equivalent resource type and naming attribute for your provider. - -### Provider Configuration File - -Root modules **MUST** have a `providers.tf` file with provider configuration: - -**AWS Example:** - -```hcl -# providers.tf - -provider "aws" { - region = var.aws_region - - default_tags { - tags = { - Environment = var.environment - Project = var.project_name - ManagedBy = "terraform" - } - } -} -``` - -> **Note:** Azure does not support provider-level default tags; define common tags in a `locals` block. GCP supports `default_labels` at the provider level (Google provider 4.x+), but label keys must be lowercase. - -### Backend Configuration - -Root modules **MUST** configure a remote backend, either in `backend.tf` or within the `terraform` block: - -**AWS Example:** - -```hcl -# backend.tf - Example configuration - -terraform { - backend "s3" { - bucket = "acme-corp-terraform-state" # Use your state bucket name - key = "environments/prod/terraform.tfstate" - region = "us-east-1" # Use your preferred region - encrypt = true - dynamodb_table = "terraform-locks" - } -} -``` - -> **Note:** Azure uses the `azurerm` backend with a Storage Account, and GCP uses the `gcs` backend with a Cloud Storage bucket. -> -> **Important:** Unlike resource blocks, backend blocks do not support variable interpolation. Example values in backend configuration (such as bucket names and regions) must be replaced with your organization's actual values before running `terraform init`. Alternatively, use [partial backend configuration](#partial-backend-configuration) to provide values at runtime. - -#### Partial Backend Configuration - -As an alternative to placeholder values, Terraform supports **partial backend configuration**. This pattern separates static configuration (committed to version control) from dynamic values (provided at runtime): - -**AWS Backend file (committed):** - -```hcl -# backend.tf - partial configuration - -terraform { - backend "s3" { - key = "environments/prod/terraform.tfstate" - encrypt = true - # bucket, region, and dynamodb_table provided via -backend-config - } -} -``` - -**AWS Backend config file (environment-specific):** - -```hcl -# config/prod.s3.tfbackend - -bucket = "acme-corp-terraform-state" -region = "us-east-1" -dynamodb_table = "terraform-locks" -``` - -**Usage:** - -```bash -terraform init -backend-config=config/prod.s3.tfbackend # AWS -terraform init -backend-config=config/prod.azurerm.tfbackend # Azure -terraform init -backend-config=config/prod.gcs.tfbackend # GCP -``` - -This pattern is useful when: - -- Backend values vary by environment but the state key structure is consistent -- Teams prefer runtime configuration over placeholder replacement -- CI/CD pipelines inject backend configuration dynamically - -Both the inline example values and partial configuration pattern are valid approaches. Choose the pattern that best fits your team's workflow. - -### Variable Files (.tfvars) - -Terraform variable files (`.tfvars`) provide environment-specific or deployment-specific values. This section defines conventions for organizing and managing these files. - -#### Variable File Naming Conventions - -| Pattern | Use Case | Example | -| --- | --- | --- | -| `terraform.tfvars` | Default values loaded automatically | `terraform.tfvars` | -| `.tfvars` | Environment-specific values | `prod.tfvars`, `dev.tfvars` | -| `.auto.tfvars` | Auto-loaded environment values | `prod.auto.tfvars` | - -#### Content Guidelines - -**What belongs in `.tfvars` files:** - -- Environment-specific values (e.g., instance sizes, replica counts) -- Non-sensitive configuration overrides -- Deployment-specific settings - -**What does NOT belong in `.tfvars` files:** - -- Secrets, API keys, or passwords — see [Security Best Practices](#security-best-practices) -- Hardcoded credentials or tokens -- Any value that should not be committed to version control - -**Relationship to variable defaults:** - -Values in `.tfvars` files override `default` values in variable declarations. The loading order is: - -1. `default` value in `variables.tf` -2. Environment variables (`TF_VAR_name`) -3. `terraform.tfvars` and `terraform.tfvars.json` (if present) -4. `*.auto.tfvars` and `*.auto.tfvars.json` files (in alphabetical order) -5. `-var-file` command-line arguments (in order specified) -6. `-var` command-line arguments (later flags override earlier ones) - -#### Version Control Guidelines - -- Non-sensitive `.tfvars` files **MAY** be committed to version control -- Sensitive `.tfvars` files **MUST NOT** be committed; use `.tfvars.example` templates instead -- Template files **SHOULD** use the `.tfvars.example` extension to indicate they require customization - -**Example `.gitignore` patterns for sensitive tfvars:** - -```gitignore -# Sensitive variable files (use *.tfvars.example as templates) -*.sensitive.tfvars -*-secrets.tfvars -``` - -#### Example File Structure - -```text -environments/ -├── prod/ -│ ├── main.tf -│ ├── variables.tf -│ ├── terraform.tfvars # Committed: non-sensitive defaults -│ ├── secrets.tfvars.example # Committed: template for secrets -│ └── secrets.tfvars # NOT committed: actual secrets -└── dev/ - ├── main.tf - ├── variables.tf - └── terraform.tfvars -``` - -### Terraform Cloud Variable Precedence - -When using Terraform Cloud or Terraform Enterprise, variables can be set at multiple levels. The precedence order (highest to lowest) is: - -1. `-var` and `-var-file` flags in CLI-driven runs -2. `*.auto.tfvars` files (in alphabetical order) -3. `terraform.tfvars` (if present) -4. Workspace-specific variables (set in Terraform Cloud UI/API) -5. Variable sets (shared across workspaces) -6. Environment variables (`TF_VAR_*`) -7. `default` values in variable declarations - -**Note:** In Terraform Cloud, workspace variables and variable sets take precedence over environment variables (`TF_VAR_*`). This differs from local Terraform execution where environment variables have higher precedence. - -Document which variable management approach your organization uses in the [Scope Exceptions](#scope-exceptions--deviations-from-standards) section to ensure consistency across team members. - -### Module Directory Structure - -Standard module directory structure: - -```text -modules/ -└── / - ├── main.tf # Primary resources - ├── variables.tf # Input variables (REQUIRED) - ├── outputs.tf # Output values (REQUIRED) - ├── versions.tf # Version constraints (REQUIRED) - ├── README.md # Module documentation (REQUIRED) - ├── locals.tf # Local values (when needed) - ├── data.tf # Data sources (when needed) - ├── examples/ # Usage examples (RECOMMENDED) - │ └── basic/ - │ ├── main.tf - │ ├── variables.tf - │ └── outputs.tf - └── tests/ # Test files (RECOMMENDED) - └── basic.tftest.hcl -``` - -### Module Examples - -Modules **SHOULD** include an `examples/` directory with working examples: - -- Each example **MUST** be a complete, runnable configuration -- Examples **SHOULD** demonstrate common use cases -- Examples **SHOULD** include a `README.md` explaining the example - -### Module Tests - -Modules **SHOULD** include a `tests/` directory with Terraform test files: - -- Tests **MUST** use the `.tftest.hcl` extension -- Tests **SHOULD** cover both valid and invalid inputs -- Tests **SHOULD** validate critical outputs - -### Template Files (.tftpl) - -Template files are used with the `templatefile()` function to generate dynamic content such as configuration files, scripts, or policy documents. Template files **SHOULD** follow these conventions: - -- Template files **SHOULD** use the `.tftpl` extension for clear identification -- Template files **SHOULD** be placed in a `templates/` subdirectory within the module or root configuration -- Template file variables **SHOULD** be documented at the top of the template file using comments - -**Directory structure:** - -```text -modules/ -└── / - ├── main.tf - ├── variables.tf - ├── outputs.tf - └── templates/ - ├── user_data.sh.tftpl - └── policy.json.tftpl -``` - -**Template file example with documentation:** - -```tftpl -#!/bin/bash -# Template: user_data.sh.tftpl -# Variables: -# - environment: Deployment environment (string) -# - app_name: Application name (string) -# - enable_monitoring: Whether to enable monitoring (bool) - -echo "Deploying ${app_name} to ${environment}" - -%{ if enable_monitoring ~} -echo "Monitoring enabled" -%{ endif ~} -``` - -**Using templatefile() function:** - -```hcl -resource "aws_instance" "main" { - ami = var.ami_id - instance_type = var.instance_type - - user_data = templatefile("${path.module}/templates/user_data.sh.tftpl", { - environment = var.environment - app_name = var.app_name - enable_monitoring = var.enable_monitoring - }) -} -``` - ---- - -## Variable and Output Design - -### Variable Documentation Requirements - -Every variable **MUST** include a `description`. The description **SHOULD** explain: - -- What the variable is for -- Valid values or constraints -- Any special considerations - -```hcl -variable "environment" { - description = "Deployment environment. Valid values: dev, staging, prod." - type = string - - validation { - condition = contains(["dev", "staging", "prod"], var.environment) - error_message = "Environment must be dev, staging, or prod." - } -} -``` - -### Variable Type Constraints - -Variables **MUST** include explicit `type` constraints: - -```hcl -# String variable -variable "instance_type" { - description = "EC2 instance type for the application server." - type = string - default = "t3.micro" -} - -# List variable -variable "subnet_ids" { - description = "List of subnet IDs for deployment." - type = list(string) -} - -# Map variable -variable "tags" { - description = "Additional tags to apply to resources." - type = map(string) - default = {} -} - -# Object variable -variable "instance_config" { - description = "Configuration for the EC2 instance." - type = object({ - instance_type = string - ami_id = string - volume_size = optional(number, 20) - }) -} -``` - -### Variable Defaults - -Optional variables **MUST** have a `default` value. Required variables **MUST NOT** have a `default`. - -```hcl -# Required variable (no default) -variable "vpc_id" { - description = "VPC ID where resources will be created." - type = string -} - -# Optional variable (has default) -variable "enable_monitoring" { - description = "Enable CloudWatch detailed monitoring." - type = bool - default = false -} -``` - -### Variable Validation - -Variables with constrained values **SHOULD** use `validation` blocks: - -```hcl -variable "environment" { - description = "Deployment environment. Valid values: dev, staging, prod." - type = string - - validation { - condition = contains(["dev", "staging", "prod"], var.environment) - error_message = "Environment must be one of: dev, staging, prod." - } -} - -variable "instance_count" { - description = "Number of instances to create. Must be between 1 and 10." - type = number - - validation { - condition = var.instance_count >= 1 && var.instance_count <= 10 - error_message = "Instance count must be between 1 and 10." - } -} - -variable "cidr_block" { - description = "CIDR block for the VPC." - type = string - - validation { - condition = can(cidrhost(var.cidr_block, 0)) - error_message = "Must be a valid CIDR block." - } -} -``` - -### Sensitive Variable Marking - -Variables containing sensitive data **MUST** be marked: - -```hcl -variable "database_password" { - description = "Password for the RDS database. Must be provided via environment variable or tfvars." - type = string - sensitive = true -} - -variable "api_key" { - description = "API key for external service" - type = string - sensitive = true -} -``` - -### Output Documentation Requirements - -Every output **MUST** include a `description`: - -```hcl -output "instance_id" { - description = "The ID of the created EC2 instance." - value = aws_instance.main.id -} - -output "instance_public_ip" { - description = "The public IP address of the EC2 instance." - value = aws_instance.main.public_ip -} -``` - -### Sensitive Output Marking - -Outputs containing sensitive data **MUST** be marked: - -```hcl -output "database_connection_string" { - description = "Database connection string (contains credentials)." - value = local.connection_string - sensitive = true -} - -output "instance_private_ip" { - description = "The private IP address of the EC2 instance." - value = aws_instance.main.private_ip - sensitive = true -} -``` - -### Nullable Variables - -The `nullable` attribute (Terraform 1.1+) controls whether a variable can accept `null` as a valid value. By default, all variables have `nullable = true`, meaning they can accept `null` values. - -**Purpose:** - -- Explicitly allow or disallow `null` as a valid value -- Distinguish between "not provided" and "explicitly set to null" -- Improve input validation for required values - -**When to use explicit `nullable`:** - -- Use `nullable = true` when `null` is a meaningful value distinct from the default -- Use `nullable = false` when `null` values **MUST** be rejected - -**Example:** - -```hcl -variable "optional_cidr" { - description = "Optional secondary CIDR block. Set to null to skip configuration." - type = string - default = null - nullable = true # Explicitly allow null values -} - -variable "required_name" { - description = "Required resource name. Cannot be null." - type = string - nullable = false # Null values will be rejected -} -``` - -**Usage pattern:** - -```hcl -resource "aws_vpc_ipv4_cidr_block_association" "secondary" { - count = var.optional_cidr != null ? 1 : 0 - - vpc_id = aws_vpc.main.id - cidr_block = var.optional_cidr -} -``` - -### Null Value Patterns - -Terraform provides several functions and patterns for handling null values effectively. This section documents common patterns for working with potentially null values. - -**Using `coalesce()` for fallback values:** - -```hcl -locals { - # Use provided value or fall back to default - instance_type = coalesce(var.instance_type_override, "t3.micro") - - # Chain multiple fallbacks - region = coalesce(var.region, data.aws_region.current.name, "us-east-1") -} -``` - -**Handling null in conditional expressions:** - -```hcl -resource "aws_instance" "main" { - ami = var.ami_id - instance_type = var.instance_type - - # Only set user_data if provided, otherwise use default script - user_data = var.custom_user_data != null ? var.custom_user_data : file("${path.module}/default_user_data.sh") -} -``` - -**Null handling in `for` expressions:** - -```hcl -locals { - # Filter out null values from a list - valid_subnets = [for s in var.subnet_ids : s if s != null] - - # Filter out entries with null values from a map - valid_tags = { for k, v in var.tags : k => v if v != null } -} -``` - -**Using `optional()` with defaults (Terraform 1.3+):** - -```hcl -variable "instance_config" { - type = object({ - instance_type = string - volume_size = optional(number, 20) # Defaults to 20 when attribute is not set - monitoring = optional(bool, false) # Defaults to false when attribute is not set - }) -} -``` - -### Defensive Attribute Access with try() - -The `try()` function attempts to evaluate expressions and returns a fallback value if any expression fails. Use it for defensive access to attributes that may not exist. - -**Syntax:** `try(expression, fallback)` - -**Use cases:** - -```hcl -locals { - # Safely access nested attributes that may not exist - instance_ip = try(aws_instance.main.private_ip, "unknown") - - # Handle optional nested blocks - root_volume_size = try(aws_instance.main.root_block_device[0].volume_size, 8) - - # Chain multiple attempts - endpoint = try( - aws_db_instance.main.endpoint, - aws_rds_cluster.main.endpoint, - "no-database-configured" - ) -} -``` - -**Difference between `try()` and `can()`:** - -| Function | Returns | Use Case | -| --- | --- | --- | -| `try(expr, fallback)` | The value of `expr` if successful, otherwise `fallback` | Getting a value with a default | -| `can(expr)` | `true` if `expr` evaluates without error, `false` otherwise | Validation conditions | - -```hcl -# Use can() in validation blocks -validation { - condition = can(regex("^[a-z][a-z0-9-]*$", var.name)) - error_message = "Name must start with a letter and contain only lowercase alphanumeric characters and hyphens." -} - -# Use try() when you need the actual value -locals { - parsed_json = try(jsondecode(var.config_json), {}) -} -``` - ---- - -## Continuous Validation with check Blocks - -The `check` block (Terraform 1.5+) provides a mechanism for continuous validation assertions that run on every `plan` and `apply` operation. Unlike `precondition` and `postcondition` blocks, `check` blocks produce **warning diagnostics** that do **not** halt execution when assertions fail. - -### When to Use check Blocks - -`check` blocks **MAY** be used for deterministic validations that rely only on Terraform configuration, state, and resource attributes, such as: - -- **Policy conformance:** Ensure resources comply with tagging, encryption, and sizing standards -- **Configuration invariants:** Assert relationships between resources (for example, capacity, counts, or naming patterns) -- **Drift and safety nets:** Highlight situations where the actual infrastructure shape no longer matches declared expectations - -In this repository, `check` block `assert` conditions **MUST** be deterministic and **MUST NOT** introduce network calls or depend on live external service reachability. Use dedicated observability/monitoring tooling for runtime health checks and external dependency validation. - -### check Block Syntax - -```hcl -check "alb_has_listeners" { - assert { - condition = length(aws_lb_listener.app) > 0 - error_message = "Application load balancer must have at least one listener configured." - } -} -``` - -### Comparison with precondition/postcondition - -| Feature | `check` blocks | `precondition`/`postcondition` | -| --- | --- | --- | -| **Causes failure** | No (warning only) | Yes (error) | -| **Runs during** | Every plan/apply | Resource creation/update | -| **Scope** | Configuration-wide | Resource-specific | -| **Use case** | Ongoing invariant checks | Input/output validation | - -### Best Practices - -- **Bounded usage:** Each module **SHOULD** define at most 3 `check` blocks and **SHOULD** reserve them for critical invariants (for example, cross-resource consistency or security posture), not routine validation that can be handled with `precondition`/`postcondition`. -- **Deterministic and cheap assertions:** `condition` expressions **MUST** be deterministic (no dependence on current time, randomness, or external services) and **MUST NOT** introduce additional network calls (for example, avoid `http` or other remote data sources inside a `check` block). They **SHOULD** only reference values already available during planning (resource attributes, variables, locals, and data sources the plan already needs). -- **Constrained complexity:** Assertions **SHOULD** be expressed as a small number of boolean expressions combined with `&&`/`||`, and **SHOULD NOT** iterate over large collections or use deeply nested conditionals that materially increase plan/apply latency. -- **Document purpose:** `error_message` values **MUST** clearly state which invariant failed and what the operator **SHOULD** do next (for example, "rotate API token X" or "scale service Y"). - ---- - -## Resource Configuration - -### Meta-Argument Ordering - -Within resource blocks, arguments **MUST** follow this order: - -1. **Meta-arguments first:** `count`, `for_each`, `provider`, `depends_on`, `lifecycle` -2. **Required arguments:** Arguments without defaults -3. **Optional arguments:** Arguments with defaults -4. **Nested blocks last:** Dynamic blocks, inline blocks - -**Compliant example:** - -```hcl -resource "aws_instance" "web_server" { - # Meta-arguments first - count = var.instance_count - provider = aws.primary - - # Required arguments - ami = data.aws_ami.amazon_linux.id - instance_type = var.instance_type - subnet_id = var.subnet_id - - # Optional arguments - associate_public_ip_address = var.is_public - monitoring = var.enable_monitoring - - # Nested blocks last - root_block_device { - volume_size = var.root_volume_size - encrypted = true - } - - tags = local.common_tags - - # Lifecycle block at the end - lifecycle { - create_before_destroy = true - } -} -``` - -### Argument Ordering - -Within the arguments section: - -1. Required arguments appear before optional arguments -2. Related arguments are grouped together -3. `tags` typically appears last before nested blocks - -### Nested Block Placement - -Nested blocks **MUST** appear after all simple arguments: - -```hcl -resource "aws_security_group" "web" { - # Simple arguments first - name = "${var.project_name}-web-sg" - description = "Security group for web servers" - vpc_id = var.vpc_id - - # Nested blocks last - ingress { - from_port = 443 - to_port = 443 - protocol = "tcp" - cidr_blocks = ["0.0.0.0/0"] - } - - egress { - from_port = 0 - to_port = 0 - protocol = "-1" - cidr_blocks = ["0.0.0.0/0"] - } - - tags = local.common_tags -} -``` - -### Required Tags - -All taggable resources **MUST** include these tags: - -| Tag | Description | Example | -| --- | --- | --- | -| `Name` | Human-readable resource name | `prod-web-server-1` | -| `Environment` | Deployment environment | `prod`, `staging`, `dev` | -| `Project` | Project or application name | `my-application` | -| `ManagedBy` | Management method | `terraform` | -| `Owner` | Team or individual owner | `platform-team` | - -### Default Tags Configuration - -Root modules **SHOULD** use provider-level default tags to ensure consistent tagging: - -**AWS Example:** - -```hcl -provider "aws" { - region = var.aws_region - - default_tags { - tags = { - Environment = var.environment - Project = var.project_name - ManagedBy = "terraform" - Owner = var.owner_team - } - } -} -``` - -> **Note:** Azure does not support provider-level default tags. GCP supports `default_labels` at the provider level (Google provider 4.x+), but label keys must be lowercase. For Azure, use a `locals` block to define common tags. For GCP, you may use `default_labels` or a `locals` block if you need consistent lowercase key enforcement. See the [Local Tags Pattern](#local-tags-pattern) section below. - -### Local Tags Pattern - -Use locals for computed or merged tags. This pattern is **REQUIRED** for Azure (no provider-level support) and **RECOMMENDED** for GCP when consistent lowercase label key enforcement is needed: - -**AWS/Azure Example (Tags):** - -```hcl -locals { - common_tags = { - Name = "${var.project_name}-${var.environment}" - Environment = var.environment - Project = var.project_name - ManagedBy = "terraform" - } - - # Merge common tags with resource-specific tags - instance_tags = merge(local.common_tags, { - Role = "web-server" - }) -} -``` - -> **Note:** GCP label keys must be lowercase and can only contain lowercase letters, numeric characters, underscores, and dashes. AWS and Azure tags support mixed-case keys. - -### Lifecycle Validation - -Terraform 1.2+ introduced `precondition` and `postcondition` blocks within the `lifecycle` block, enabling resource-level validation of assumptions and outcomes. - -#### Resource Preconditions - -`precondition` blocks **SHOULD** validate assumptions before resource creation. Use them to ensure that related resources or variables meet requirements before Terraform attempts to create or modify a resource. - -```hcl -resource "aws_instance" "main" { - ami = var.ami_id - instance_type = var.instance_type - - lifecycle { - precondition { - condition = var.environment != "prod" || var.enable_monitoring - error_message = "Production instances must have monitoring enabled." - } - } -} -``` - -**Use cases for preconditions:** - -- Validating cross-resource dependencies -- Enforcing business rules that span multiple variables -- Ensuring prerequisites are met before expensive operations - -#### Resource Postconditions - -`postcondition` blocks **SHOULD** validate resource state after creation. Use them to ensure that computed values meet expectations after Terraform creates or modifies a resource. - -```hcl -resource "aws_instance" "main" { - ami = var.ami_id - instance_type = var.instance_type - - lifecycle { - postcondition { - condition = self.public_ip != null - error_message = "Instance must have a public IP assigned." - } - } -} -``` - -**Use cases for postconditions:** - -- Validating computed attributes (IPs, ARNs, generated names) -- Ensuring provider-side defaults meet expectations -- Catching unexpected resource configurations - -### Lifecycle Block Options - -The `lifecycle` block supports several meta-argument options beyond `precondition` and `postcondition` for controlling resource behavior. The following table summarizes these lifecycle meta-arguments: - -| Option | Purpose | Use Case | -| --- | --- | --- | -| `create_before_destroy` | Create replacement before destroying original | Zero-downtime replacements | -| `prevent_destroy` | Prevent accidental resource deletion | Protect critical production resources | -| `ignore_changes` | Ignore changes to specific attributes | Attributes managed outside Terraform | -| `replace_triggered_by` | Force replacement when dependencies change | Trigger replacement on related resource changes (Terraform 1.2+) | - -#### When to Use Each Option - -**`create_before_destroy`:** Use when replacing a resource must not cause downtime. The new resource is created first, then the old one is destroyed. - -**`prevent_destroy`:** Use for critical resources that **MUST NOT** be accidentally deleted, such as production databases, state storage buckets, or encryption keys. - -**`ignore_changes`:** Use when an attribute is intentionally managed outside Terraform (e.g., auto-scaling group desired capacity, tags managed by external tools). - -**`replace_triggered_by`:** Use when a resource **SHOULD** be replaced whenever a related resource changes, even if no direct attributes are affected. - -#### Example: Protecting a Critical Resource - -```hcl -# Protect production database from accidental deletion -resource "aws_db_instance" "production" { - identifier = "prod-database" - engine = "postgres" - instance_class = var.db_instance_class - # ... other configuration - - lifecycle { - prevent_destroy = true - } -} - -# Protect Terraform state bucket -resource "aws_s3_bucket" "terraform_state" { - bucket = "acme-corp-terraform-state" - - lifecycle { - prevent_destroy = true - } -} -``` - -#### Example: Ignoring External Changes - -```hcl -# Ignore changes to tags managed by AWS Config or other external tools -resource "aws_instance" "main" { - ami = var.ami_id - instance_type = var.instance_type - - lifecycle { - ignore_changes = [ - tags["LastScannedBy"], - tags["ComplianceStatus"], - ] - } -} -``` - -### Resource Timeouts - -Some Terraform resources support custom timeout configurations for create, update, and delete operations via a `timeouts` block. Timeouts are provider-specific—not all resources support them, and available timeout options vary by resource type. - -#### Timeout Block Structure - -```hcl -resource "aws_db_instance" "main" { - identifier = "production-database" - engine = "postgres" - instance_class = var.db_instance_class - # ... other configuration - - timeouts { - create = "60m" - update = "90m" - delete = "30m" - } -} -``` - -#### Common Use Cases - -Custom timeouts are commonly needed for: - -- **RDS/database instances:** Database creation and modification can take 30-60+ minutes -- **Large EKS/AKS/GKE clusters:** Kubernetes cluster operations may exceed default timeouts -- **Complex networking resources:** VPN gateways, Transit Gateway attachments, and peering connections -- **Large-scale storage operations:** Creating or resizing large storage volumes - -> **Note:** Default timeouts are usually sufficient for most operations. Custom timeouts **SHOULD** only be set when operations consistently exceed default values or when specific SLAs require longer wait times. - -### Explicit Dependencies - -Terraform automatically infers dependencies from resource references. The `depends_on` meta-argument **SHOULD** be avoided unless dependencies are not inferable from the configuration. - -#### When depends_on is Appropriate - -`depends_on` **SHOULD** only be used for: - -- **Hidden dependencies:** When a resource depends on another resource's side effects (e.g., IAM policy propagation delays) -- **Module dependencies:** When a module depends on another module's resources without direct references -- **Timing issues:** When the order of operations matters but isn't reflected in resource attributes - -#### Anti-pattern: Redundant depends_on - -```hcl -# BAD: Unnecessary depends_on - dependency is already inferred from the reference -resource "aws_instance" "main" { - subnet_id = aws_subnet.main.id - depends_on = [aws_subnet.main] # Redundant -} - -# GOOD: Dependency is implicit from the reference -resource "aws_instance" "main" { - subnet_id = aws_subnet.main.id -} -``` - -#### Legitimate Use Case - -```hcl -# GOOD: Hidden dependency on IAM role policy attachment -resource "aws_instance" "main" { - ami = data.aws_ami.amazon_linux.id - instance_type = "t3.micro" - - iam_instance_profile = aws_iam_instance_profile.main.name - - # The instance profile references the role, but doesn't reference the policy attachment. - # Without depends_on, the instance may launch before the policy is attached. - depends_on = [aws_iam_role_policy_attachment.main] -} -``` - -### Module-Level depends_on - -The `depends_on` meta-argument can also be used at the module level (Terraform 0.13+). However, module-level `depends_on` has different implications than resource-level `depends_on` and **SHOULD** be used sparingly. - -**Key differences from resource-level depends_on:** - -- When a module block uses `depends_on` with a **module address** (for example, `depends_on = [module.networking]`), it creates a dependency on **all resources in that referenced module** -- When a module block uses `depends_on` with a **resource address**, it creates a dependency only on that specific resource (not all resources in its module) -- This can cause unnecessary serialization and slower apply times when used with module addresses -- It is a blunt instrument that should only be used when finer-grained dependencies cannot be expressed - -**When module-level depends_on is appropriate:** - -- When a module depends on side effects from another module (e.g., IAM propagation, DNS resolution delays) -- When there is no data to pass between modules but ordering is required -- When debugging timing issues during development (should be removed after identifying root cause) - -**Example: Legitimate use case:** - -```hcl -# Module-level depends_on - use sparingly -module "application" { - source = "./modules/application" - - vpc_id = module.networking.vpc_id - - # Only use when implicit dependencies are insufficient - # (e.g., module.networking creates IAM roles needed by the application) - depends_on = [module.networking] -} -``` - -**Anti-pattern: Redundant module depends_on:** - -```hcl -# AVOID: Unnecessary module depends_on when data is already passed -module "application" { - source = "./modules/application" - - vpc_id = module.networking.vpc_id # This creates implicit dependency - subnet_ids = module.networking.subnet_ids # This too - - depends_on = [module.networking] # REDUNDANT - remove this -} -``` - -**Best practice:** Prefer explicit data passing between modules over `depends_on`. When you pass outputs from one module as inputs to another, Terraform automatically understands the dependency relationship. - -### for_each vs count - -When creating multiple instances of a resource, `for_each` **SHOULD** be preferred over `count` for collections where resources have unique identifiers. - -#### Comparison Table - -| Use `for_each` when... | Use `count` when... | -| --- | --- | -| Resources have unique identifiers | Simple on/off toggle (`count = var.enabled ? 1 : 0`) | -| Order doesn't matter | Resources are truly identical and ordered | -| Items may be added/removed from middle of collection | Only adding/removing from the end | -| Each resource is addressable by key | Index-based addressing is acceptable | - -#### Why for_each is Preferred - - - -```hcl -# BAD: Using count with a list - removing any item shifts all subsequent indices -resource "aws_instance" "servers" { - count = length(var.server_names) - ami = data.aws_ami.amazon_linux.id - instance_type = "t3.micro" - tags = { - Name = var.server_names[count.index] - } -} - -# GOOD: Using for_each with a set - removing "server-b" only affects that resource -resource "aws_instance" "servers" { - for_each = toset(var.server_names) - ami = data.aws_ami.amazon_linux.id - instance_type = "t3.micro" - tags = { - Name = each.value - } -} -``` - -#### When count is Appropriate - -```hcl -# GOOD: count for conditional resource creation -resource "aws_cloudwatch_log_group" "main" { - count = var.enable_logging ? 1 : 0 - name = "/app/${var.environment}" -} -``` - -#### Conditional Resource Creation - -The `count` meta-argument is appropriate for conditional resource creation using a boolean toggle: - -```hcl -variable "enable_logging" { - description = "Whether to create the CloudWatch log group" - type = bool - default = false -} - -resource "aws_cloudwatch_log_group" "main" { - count = var.enable_logging ? 1 : 0 - name = "/app/${var.environment}" -} -``` - -**Referencing conditional resources:** - -When referencing a conditionally created resource, use index `[0]` and handle the case where it doesn't exist: - -```hcl -# Safe reference to conditional resource -output "log_group_arn" { - description = "ARN of the log group, if created" - value = var.enable_logging ? aws_cloudwatch_log_group.main[0].arn : null -} - -# Alternative using try() for readability -# try() returns the first argument when the resource exists, or null when it does not -output "log_group_name" { - description = "Name of the log group, if created" - value = try(aws_cloudwatch_log_group.main[0].name, null) -} -``` - -### Dynamic Blocks - -Dynamic blocks allow generating multiple nested blocks from a collection. They **SHOULD** be used sparingly because they reduce readability and complicate debugging. - -#### When to Use Dynamic Blocks - -Use dynamic blocks when: - -- The number of nested blocks is variable and determined by configuration -- The block structure is repetitive and follows a consistent pattern -- Manual repetition would create maintenance burden - -```hcl -# GOOD: Variable number of ingress rules from configuration -resource "aws_security_group" "main" { - name = "${var.project_name}-sg" - description = "Security group for ${var.project_name}" - vpc_id = var.vpc_id - - dynamic "ingress" { - for_each = var.ingress_rules - content { - from_port = ingress.value.from_port - to_port = ingress.value.to_port - protocol = ingress.value.protocol - cidr_blocks = ingress.value.cidr_blocks - } - } -} -``` - -#### When to Avoid Dynamic Blocks - -Avoid dynamic blocks when the number of blocks is fixed and small. Writing explicit blocks improves readability: - -```hcl -# GOOD: Fixed, small number of rules - explicit is clearer -resource "aws_security_group" "web" { - name = "${var.project_name}-web-sg" - description = "Web security group" - vpc_id = var.vpc_id - - ingress { - from_port = 443 - to_port = 443 - protocol = "tcp" - cidr_blocks = ["0.0.0.0/0"] - } - - ingress { - from_port = 80 - to_port = 80 - protocol = "tcp" - cidr_blocks = ["0.0.0.0/0"] - } - - egress { - from_port = 0 - to_port = 0 - protocol = "-1" - cidr_blocks = ["0.0.0.0/0"] - } -} - -# AVOID: Using dynamic for only 2-3 fixed rules adds unnecessary complexity -# dynamic "ingress" { ... } -``` - -### The terraform_data Resource - -The `terraform_data` resource (Terraform 1.4+) is a built-in managed resource that provides trigger-based replacement and data passing without requiring any provider. It is the **preferred replacement** for `null_resource` in modern Terraform configurations. - -#### When to Use terraform_data - -- **Trigger-based replacement:** Force resource replacement when specific values change -- **Data passing:** Store and pass values between resources or modules -- **Provisioner execution:** Run local-exec or remote-exec provisioners (same as null_resource) - - - -#### Basic Usage Pattern - -```hcl -resource "terraform_data" "replacement" { - input = var.revision - - # Force replacement when any of these values change - triggers_replace = [ - aws_instance.main.id, - var.force_replacement, - ] -} - -# Reference the trigger in other resources -resource "aws_instance" "dependent" { - # ... - - lifecycle { - replace_triggered_by = [terraform_data.replacement] - } -} -``` - -#### Using input and output Attributes - -The `input` attribute accepts any value and makes it available as `output`: - -```hcl -resource "terraform_data" "config" { - input = { - environment = var.environment - version = var.app_version - timestamp = timestamp() - } -} - -# Access the stored values -output "deployment_config" { - description = "Configuration used for this deployment." - value = terraform_data.config.output -} -``` - -#### Migration from null_resource - -When migrating from `null_resource` to `terraform_data`: - -```hcl -# Before (null_resource) -resource "null_resource" "trigger" { - triggers = { - instance_id = aws_instance.main.id - } -} - -# After (terraform_data) - preferred -resource "terraform_data" "trigger" { - triggers_replace = [aws_instance.main.id] -} -``` - -> **Note:** `terraform_data` requires Terraform 1.4.0 or later. For configurations that must support older versions, `null_resource` remains available via the `hashicorp/null` provider. - ---- - -## Module Design - -### Single Responsibility - -Modules **MUST** have a single, well-defined responsibility: - -- Each module **SHOULD** manage one logical component -- Modules **SHOULD NOT** try to do too much -- Complex infrastructure **SHOULD** be composed of multiple modules - -**Good:** A VPC module that creates VPC, subnets, route tables, and internet gateway. - -**Bad:** A "full-stack" module that creates VPC, EC2, RDS, and S3 all together. - -### Module Version Constraints - -Modules **MUST** specify required Terraform and provider versions: - -```hcl -# versions.tf in module directory - -terraform { - required_version = ">= 1.7.0" - - required_providers { - aws = { - source = "hashicorp/aws" - version = ">= 6.0.0" - } - } -} -``` - -### Module Interface Design - -**Inputs:** - -- Variable names **MUST** be consistent across modules (e.g., always `environment`, not sometimes `env`) -- Required variables **SHOULD** be minimized to essential values -- Complex inputs **SHOULD** use object types with documented structure - -**Outputs:** - -- Expose only values needed by calling modules -- Use consistent naming patterns across modules -- Document output types and formats - -### Minimal Required Inputs - -Required variables **SHOULD** be minimized. Provide sensible defaults where possible: - -```hcl -# Good: Only truly required inputs are mandatory -variable "vpc_id" { - description = "VPC ID where resources will be created." - type = string - # No default - this is genuinely required -} - -variable "instance_type" { - description = "EC2 instance type." - type = string - default = "t3.micro" # Sensible default -} -``` - -### Complex Input Types - -For complex inputs, use object types with clear documentation: - -```hcl -variable "instance_config" { - description = <<-EOT - Configuration for the EC2 instance. - - Attributes: - - instance_type: EC2 instance type (e.g., "t3.micro") - - ami_id: AMI ID to use for the instance - - volume_size: Root volume size in GB (default: 20) - - enable_monitoring: Enable detailed monitoring (default: false) - EOT - type = object({ - instance_type = string - ami_id = string - volume_size = optional(number, 20) - enable_monitoring = optional(bool, false) - }) -} -``` - -### Module Output Design - -Outputs **SHOULD** expose only values needed by calling modules: - -```hcl -# Good: Specific, useful outputs -output "instance_id" { - description = "The ID of the created EC2 instance." - value = aws_instance.main.id -} - -output "instance_private_ip" { - description = "The private IP address of the EC2 instance." - value = aws_instance.main.private_ip -} - -# Avoid: Exposing entire resource -output "instance" { - description = "The entire instance resource." - value = aws_instance.main # Too broad -} -``` - -### Module Versioning - -For published modules, use semantic versioning: - -- **MAJOR:** Breaking changes (removed inputs, changed behavior) -- **MINOR:** New features, backward-compatible changes -- **PATCH:** Bug fixes, documentation updates - -Pin module versions in root configurations: - -```hcl -module "vpc" { - source = "terraform-aws-modules/vpc/aws" - version = "~> 5.0" - - # ... -} - -module "internal_module" { - source = "./modules/my-module" - # Local modules don't use version constraint -} -``` - ---- - -## Refactoring - -Terraform provides declarative blocks for safely refactoring infrastructure without manual state manipulation. These blocks enable auditable, version-controlled changes that can be reviewed and tested before applying. - -### Refactoring with Moved Blocks - -The `moved` block (Terraform 1.1+) enables renaming or restructuring resources without destroying and recreating them. This is essential for: - -- Renaming resources to follow updated naming conventions -- Restructuring modules -- Moving resources between modules - -**Syntax and Example:** - -```hcl -moved { - from = aws_instance.web - to = aws_instance.web_server -} - -resource "aws_instance" "web_server" { - # Previously aws_instance.web - ami = var.ami_id - instance_type = var.instance_type - # ... -} -``` - -**Requirements:** - -- You **MUST** run `terraform plan` before applying the `moved` block changes and again after applying/refactoring to verify that no unexpected changes remain -- The `moved` block **SHOULD** remain in configuration until all environments have been updated -- After all state has been migrated, `moved` blocks **MAY** be removed - -### Importing Resources with Import Blocks - -The `import` block (Terraform 1.5+) brings existing infrastructure under Terraform management. This is preferred over the `terraform import` CLI command because: - -- Import configuration is version-controlled and reviewable -- Multiple imports can be planned and applied together -- Import intentions are visible in code review - -**Syntax and Example:** - -```hcl -import { - to = aws_instance.example - id = "i-0abc123def456789" -} - -resource "aws_instance" "example" { - # Configuration matching the imported resource - ami = "ami-0abcdef1234567890" - instance_type = "t3.micro" - # ... -} -``` - -**Requirements:** - -- The resource configuration **MUST** match the existing infrastructure -- You **MUST** run `terraform plan` before applying imports to verify the import will not cause unintended changes -- After successful import and apply, you **SHOULD** run `terraform plan` again to verify no further changes are pending, and `import` blocks **SHOULD** then be removed from configuration - -### Removing Resources from State with Removed Blocks - -The `removed` block (Terraform 1.7+) removes resources from Terraform management without destroying the underlying infrastructure. Use cases include: - -- Transitioning resource ownership to another team or configuration -- Removing resources that are now managed outside Terraform -- Cleaning up state without affecting production infrastructure - -**Syntax and Example:** - -```hcl -removed { - from = aws_instance.legacy - - lifecycle { - destroy = false # Remove from state without destroying - } -} -``` - -**The `lifecycle.destroy` Option:** - -| Value | Behavior | -| --- | --- | -| `true` (default) | Resource is destroyed when removed from configuration | -| `false` | Resource is removed from state but preserved in the cloud | - -**Requirements:** - -- Use `destroy = false` when the resource **MUST** continue to exist -- The reason for removal **MUST** be documented in comments or commit messages -- After successful apply, `removed` blocks **MAY** be removed from configuration - -### State Manipulation Commands - -Direct state manipulation commands (`terraform state mv`, `terraform state rm`, `terraform import`) **SHOULD** be avoided in favor of the declarative blocks above. The declarative approach provides: - -- Version control and audit trail -- Code review before changes -- Consistent behavior across team members -- Reduced risk of human error - -**When state commands are necessary, they MUST be:** - -- Documented in commit messages with clear rationale -- Performed only after creating state backups -- Reviewed by a second team member when possible -- Followed by verification that state is consistent - -**Backup command before state manipulation:** - -```bash -terraform state pull > terraform.tfstate.backup -``` - ---- - -## State Management - -> **Note:** For state modification best practices including resource renaming, importing existing infrastructure, and removing resources from management, see the [Refactoring](#refactoring) section. - -### Remote Backend Configuration - -Root modules **MUST** configure a remote backend for team environments: - -**AWS Example:** - -```hcl -terraform { - backend "s3" { - bucket = "acme-corp-terraform-state" - key = "environments/prod/terraform.tfstate" - region = "us-east-1" - encrypt = true - dynamodb_table = "terraform-locks" - } -} -``` - -> **Note:** Azure uses `azurerm` backend and GCP uses `gcs` backend. -**Backend requirements:** - -- State files **MUST** be encrypted at rest -- State access **MUST** be controlled via appropriate access controls (e.g., IAM for AWS, RBAC for Azure, IAM for GCP) -- State locking **MUST** be enabled to prevent concurrent modifications -- State files **MUST NOT** be committed to version control - -### Terraform Cloud, Enterprise, and Alternative Backends - -Organizations using **Terraform Cloud**, **Terraform Enterprise**, **Spacelift**, or similar workflow tools **MAY** use alternative state management approaches. In these cases: - -- The `backend.tf` file **MAY** be omitted if state is managed by the orchestration platform. -- The `cloud` block **MAY** replace the `backend` block for Terraform Cloud/Enterprise integrations. -- Document your backend approach in the [Scope Exceptions](#scope-exceptions--deviations-from-standards) section. - -**Example Terraform Cloud configuration:** - -```hcl -terraform { - cloud { - organization = "acme-corp" - workspaces { - name = "prod-infrastructure" - } - } -} -``` - -When using alternative backends, the following sections still apply: - -- State encryption requirements (handled by the platform) -- State locking requirements (typically automatic with cloud backends) -- Version control exclusion of state files - -The following sections **MAY** not apply when using Terraform Cloud/Enterprise: - -- Manual `backend.tf` configuration -- DynamoDB lock table configuration -- S3/GCS/Azure Storage bucket configuration - -### Terraform Cloud Workspace Configuration - -Terraform Cloud supports two patterns for workspace selection: explicit naming and tag-based selection. - -**Explicit workspace selection:** - -The `name` attribute selects a specific, named workspace: - -```hcl -terraform { - cloud { - organization = "acme-corp" - workspaces { - name = "prod-infrastructure" - } - } -} -``` - -**Tag-based workspace selection:** - -The `tags` attribute enables dynamic workspace selection based on workspace tags. Terraform will operate on all workspaces that have **all** of the specified tags: - -```hcl -terraform { - cloud { - organization = "acme-corp" - workspaces { - tags = ["app:my-application", "env:production"] - } - } -} -``` - -**Key differences:** - -| Attribute | Behavior | Use Case | -| --- | --- | --- | -| `name` | Selects a single, specific workspace | Standard deployments with known workspace names | -| `tags` | Selects all workspaces matching the specified tags | Multi-workspace operations, batch deployments | - -**Common patterns with tags:** - -- Use `tags` for operations that should apply to multiple environments (e.g., deploying a fix to all production workspaces) -- Combine application and environment tags for precise targeting -- Tags are defined in Terraform Cloud, not in the configuration - -> **Note:** You cannot use both `name` and `tags` in the same `workspaces` block. Choose one pattern based on your workflow requirements. - -### Bootstrapping State Infrastructure - -When creating a new Terraform configuration, you face a chicken-and-egg problem: the remote backend (e.g., S3 bucket, Azure Storage Account, GCS bucket) must exist before Terraform can use it to store state, but you want Terraform to manage that backend infrastructure. - -#### Bootstrap Workflow - -**Step 1: Create bootstrap configuration with backend commented out:** - -Create your state storage resources in a dedicated bootstrap configuration, initially without a backend block (or with the backend block commented out). This allows Terraform to use local state temporarily. - -**AWS Example (`bootstrap/versions.tf`):** - -```hcl -terraform { - required_version = ">= 1.7.0" - - required_providers { - aws = { - source = "hashicorp/aws" - version = "~> 6.0" - } - } - - # Backend will be added after bootstrap - # backend "s3" { - # bucket = "acme-corp-terraform-state" - # key = "bootstrap/terraform.tfstate" - # region = "us-east-1" - # encrypt = true - # dynamodb_table = "terraform-locks" - # } -} - -# Create the S3 bucket for state storage -# Note: S3 bucket names must be globally unique. See "Globally Unique Resource Names" -# section for patterns using random_id or organization prefixes. -resource "aws_s3_bucket" "terraform_state" { - bucket = "acme-corp-terraform-state" - - lifecycle { - prevent_destroy = true - } -} - -resource "aws_s3_bucket_versioning" "terraform_state" { - bucket = aws_s3_bucket.terraform_state.id - versioning_configuration { - status = "Enabled" - } -} - -resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" { - bucket = aws_s3_bucket.terraform_state.id - rule { - apply_server_side_encryption_by_default { - sse_algorithm = "AES256" - } - } -} - -# Create DynamoDB table for state locking -resource "aws_dynamodb_table" "terraform_locks" { - name = "terraform-locks" - billing_mode = "PAY_PER_REQUEST" - hash_key = "LockID" - - attribute { - name = "LockID" - type = "S" - } -} -``` - -**Step 2: Apply locally to create state storage:** - -```bash -cd bootstrap -terraform init -terraform apply -``` - -This creates the state storage infrastructure using local state (stored in `terraform.tfstate`). - -**Step 3: Configure backend and migrate state:** - -Uncomment or add the backend block in your configuration, then run: - -```bash -terraform init -migrate-state -``` - -Terraform will prompt you to confirm migration of the existing local state to the remote backend. - -**Step 4: Clean up local state file:** - -After successful migration, the local state file is no longer needed: - -```bash -rm terraform.tfstate terraform.tfstate.backup -``` - -> **Warning:** Only delete local state files after confirming the remote state is properly configured and accessible. Run `terraform plan` to verify that Terraform can read from the remote backend before deleting local files. - -#### Alternative: Partial Backend Configuration - -As documented in [Backend Configuration](#backend-configuration), you can use partial backend configuration to separate the bootstrap workflow from the backend values. This approach allows you to: - -1. Commit a partial backend configuration without sensitive values -2. Provide backend values via `-backend-config` during `terraform init` -3. Manage state storage creation in a separate, pre-provisioned step - -#### Bootstrap Best Practices - -- **Separate bootstrap configuration:** Keep state infrastructure in a dedicated directory (`bootstrap/` or `infrastructure/state/`) separate from application infrastructure. -- **Use `prevent_destroy`:** Protect state storage resources from accidental deletion. -- **Enable versioning:** Always enable versioning on state storage buckets to allow recovery from corruption. -- **Document the process:** Include bootstrap instructions in your repository's README or contributing guide. - -### State Encryption - -State backends **MUST** enable encryption: - -**AWS Example:** - -```hcl -# S3 backend with encryption -terraform { - backend "s3" { - bucket = "acme-corp-terraform-state" - key = "prod/terraform.tfstate" - region = "us-east-1" - encrypt = true # REQUIRED - } -} -``` - -> **Note:** Azure Storage and GCS backends encrypt state by default. For additional security, configure customer-managed keys. - -### State Locking - -State backends **MUST** support and enable state locking: - -**AWS Example:** - -```hcl -# S3 backend with DynamoDB locking -terraform { - backend "s3" { - bucket = "acme-corp-terraform-state" - key = "prod/terraform.tfstate" - region = "us-east-1" - encrypt = true - dynamodb_table = "terraform-locks" # REQUIRED for locking - } -} -``` - -> **Note:** Azure Storage backend uses blob leases for locking automatically. GCS backend supports state locking natively. No additional configuration required for either. - -### No Local State in Production - -Local state files **MUST NOT** be used for production environments. Local state: - -- Cannot be shared across team members -- Has no locking mechanism -- Has no encryption at rest -- Is easily lost or corrupted - -### State File Exclusion - -State files and related artifacts **MUST NOT** be committed to version control. Add to `.gitignore`: - -```gitignore -# Terraform state files -*.tfstate -*.tfstate.* - -# Crash log files -crash.log -crash.*.log - -# .terraform directories -**/.terraform/* - -# Override files -override.tf -override.tf.json -*_override.tf -*_override.tf.json - -# CLI configuration -.terraformrc -terraform.rc -``` - -### State File Organization - -Organize state files by environment and component: - -```text -state-bucket/ -├── environments/ -│ ├── dev/ -│ │ └── terraform.tfstate -│ ├── staging/ -│ │ └── terraform.tfstate -│ └── prod/ -│ └── terraform.tfstate -└── shared/ - ├── networking/ - │ └── terraform.tfstate - └── iam/ - └── terraform.tfstate -``` - -> **Note:** This diagram represents the **key/path structure in your remote backend** (S3 object keys, Azure blob paths, GCS object prefixes), not a local filesystem directory structure. These paths are configured via the `key` or `prefix` attribute in your backend configuration. Your local repository structure is separate and typically organized by environment directories containing Terraform configuration files. - -### State Backup and Recovery - -State files are critical infrastructure artifacts. Losing or corrupting state can result in significant operational challenges. This section documents backup strategies, manual backup procedures, and recovery approaches for common state-related problems. - -#### State Versioning Requirements - -State storage backends **MUST** have versioning enabled for production use: - -- Versioning enables recovery from accidental state corruption or deletion -- Version retention period **SHOULD** be defined based on organizational requirements (typically 30-90 days minimum) -- Versioned state serves as a backup mechanism without additional tooling - -#### State Backup Strategies - -Each cloud provider offers native mechanisms for state versioning and backup. - -**AWS S3 Backend:** - -Enable versioning on the S3 bucket used for state storage: - -```hcl -resource "aws_s3_bucket" "terraform_state" { - bucket = "acme-corp-terraform-state" - - # Protect critical state storage from accidental deletion - lifecycle { - prevent_destroy = true - } -} - -resource "aws_s3_bucket_versioning" "terraform_state" { - bucket = aws_s3_bucket.terraform_state.id - versioning_configuration { - status = "Enabled" - } -} - -# Optional: Configure lifecycle policy for version retention -# Uses time-based retention (90 days). Adjust based on your recovery requirements. -resource "aws_s3_bucket_lifecycle_configuration" "terraform_state" { - bucket = aws_s3_bucket.terraform_state.id - - rule { - id = "retain-old-versions" - status = "Enabled" - - noncurrent_version_expiration { - noncurrent_days = 90 - } - } -} -``` - -To recover a previous state version from S3: - -```bash -# List available versions -aws s3api list-object-versions \ - --bucket acme-corp-terraform-state \ - --prefix environments/prod/terraform.tfstate - -# Download a specific version -aws s3api get-object \ - --bucket acme-corp-terraform-state \ - --key environments/prod/terraform.tfstate \ - --version-id \ - terraform.tfstate.recovered -``` - -> **Note:** Azure Storage supports blob versioning with soft delete. GCS supports object versioning. Terraform Cloud provides automatic state versioning. - -#### Manual State Backup - -Before performing risky operations, create a manual state backup: - -```bash -# Create timestamped backup (Unix/Linux/macOS) -terraform state pull > terraform.tfstate.backup.$(date +%Y%m%d_%H%M%S) - -# Windows PowerShell equivalent -# terraform state pull > "terraform.tfstate.backup.$(Get-Date -Format 'yyyyMMdd_HHmmss')" -``` - -**When manual backups are recommended:** - -- Before running `terraform state rm` to remove resources from state -- Before running `terraform state mv` to move or rename resources -- Before major refactoring with `moved` blocks -- Before upgrading Terraform to a new major version -- Before running `terraform force-unlock` -- Before any operation that modifies state outside normal `apply` workflows - -**To restore from a manual backup:** - -```bash -# Review the backup contents first (local state file) -terraform show -json terraform.tfstate.backup.20260202_120000 | head -50 - -# Push the backup to the remote backend (use with caution) -terraform state push terraform.tfstate.backup.20260202_120000 -``` - -> **Warning:** `terraform state push` overwrites the remote state. Ensure no other operations are in progress and that you have verified the backup contents before pushing. - -### Workspace Usage - -Workspaces **MAY** be used for environment separation in simple cases: - -```bash -terraform workspace select prod -terraform apply -``` - -**Caution:** For complex environments, separate state files per environment are often clearer than workspaces. - -### Environment Separation Strategies - -Organizations need a consistent strategy for managing multiple environments (dev, staging, prod). Two primary approaches exist: **workspaces** and **directory-based separation**. This section provides guidance on when to use each approach. - - - -#### Workspace-Based Approach - -Workspaces create isolated state files within a single configuration. Each workspace shares the same backend configuration but maintains separate state. - -**How workspaces work:** - -```bash -# Create a new workspace -terraform workspace new staging - -# List available workspaces -terraform workspace list - -# Switch to an existing workspace -terraform workspace select prod - -# Show current workspace -terraform workspace show -``` - -**Using `terraform.workspace` for environment-specific values:** - -```hcl -locals { - environment_config = { - dev = { - instance_type = "t3.micro" - instance_count = 1 - } - staging = { - instance_type = "t3.small" - instance_count = 2 - } - prod = { - instance_type = "t3.medium" - instance_count = 3 - } - } - - config = local.environment_config[terraform.workspace] -} - -resource "aws_instance" "app" { - count = local.config.instance_count - instance_type = local.config.instance_type - # ... -} -``` - -**When workspaces are appropriate:** - -- Infrastructure is identical across environments except for variable values -- Small team with clear communication about which workspace is active -- Non-production environments where accidental applies have limited impact -- Rapid prototyping or development scenarios - - - -#### Directory-Based Approach - -Directory-based separation uses distinct directories for each environment, each with its own configuration files and backend configuration. Shared logic is extracted into reusable modules. - -**Recommended directory structure:** - -```text -. -├── modules/ # Shared reusable modules -│ ├── vpc/ -│ │ ├── main.tf -│ │ ├── variables.tf -│ │ └── outputs.tf -│ └── application/ -│ ├── main.tf -│ ├── variables.tf -│ └── outputs.tf -├── environments/ -│ ├── dev/ -│ │ ├── main.tf # Calls modules with dev-specific values -│ │ ├── variables.tf -│ │ ├── outputs.tf -│ │ ├── providers.tf -│ │ ├── backend.tf # Dev-specific backend configuration -│ │ └── terraform.tfvars # Dev-specific variable values -│ ├── staging/ -│ │ ├── main.tf -│ │ ├── variables.tf -│ │ ├── outputs.tf -│ │ ├── providers.tf -│ │ ├── backend.tf -│ │ └── terraform.tfvars -│ └── prod/ -│ ├── main.tf -│ ├── variables.tf -│ ├── outputs.tf -│ ├── providers.tf -│ ├── backend.tf -│ └── terraform.tfvars -└── README.md -``` - -**Example environment configuration (`environments/prod/main.tf`):** - -```hcl -module "vpc" { - source = "../../modules/vpc" - - environment = "prod" - cidr_block = "10.0.0.0/16" -} - -module "application" { - source = "../../modules/application" - - environment = "prod" - vpc_id = module.vpc.vpc_id - instance_type = "t3.medium" - instance_count = 3 -} -``` - -**When directory separation is preferred:** - -- Different configurations per environment (not just variable values) -- Production environments require explicit review and approval -- Team isolation—different teams manage different environments -- Compliance requirements mandate separation of concerns -- Environment-specific features or integrations - -**Sharing modules across environment directories:** - -- Extract common infrastructure patterns into modules under `modules/` -- Each environment directory calls these modules with environment-specific values -- Modules **SHOULD** be versioned when used across repositories -- Use relative paths (`../../modules/vpc`) for repository-local modules - -#### Recommendation - -For production use, directory-based separation is **RECOMMENDED** as the default approach because: - - - -### Resource Targeting - -`terraform apply -target` **SHOULD NOT** be used in normal workflows. Resource targeting: - -- Creates state drift between targeted and non-targeted resources -- Can leave infrastructure in inconsistent states -- Bypasses dependency validation - -Resource targeting is intended **only** for exceptional recovery scenarios where a specific resource must be modified in isolation. - -**If targeting is needed regularly**, this indicates the configuration is too large. Split the configuration into smaller, independent modules that can be applied separately. - -### Reviewing Plan Output - -Before running `terraform apply`, review the plan output carefully. Understanding the plan symbols and identifying warning signs helps prevent unintended infrastructure changes. - -**Plan output symbols:** - -| Symbol | Meaning | Action | -| --- | --- | --- | -| `+` (green) | Resource will be created | Verify this is intentional | -| `-` (red) | Resource will be destroyed | Verify this is intentional; check for data loss | -| `~` (yellow) | Resource will be updated in-place | Review which attributes are changing | -| `-/+` | Resource will be destroyed and recreated | Investigate why; consider `lifecycle` rules | -| `<=` | Data source will be read | Normal behavior | - -**Warning signs to investigate:** - -- Unexpected destroys, especially for stateful resources (databases, storage) -- Resources being replaced (`-/+`) when you expected an in-place update -- Changes to resources you didn't modify (may indicate drift or upstream changes) -- Large numbers of changes from a small code modification (may indicate a variable or module change with wide impact) - -**Best practice:** Use `terraform plan -out=tfplan` to save the plan, then `terraform apply tfplan` to ensure the exact reviewed plan is applied. - ---- - -## Cross-Stack Data Sharing - -When infrastructure is split across multiple independent Terraform configurations (often called "stacks"), you need a mechanism to share data between them. Common scenarios include: - -- **Network and application separation:** A networking team manages VPCs/VNets, and application teams need to reference VPC IDs and subnet IDs. -- **Shared services:** Central services (e.g., logging, monitoring) need to be referenced by multiple application stacks. -- **Multi-team ownership:** Different teams own different parts of infrastructure but need to integrate. - -### Approaches Comparison - - - -### Preferred: Cloud-Native Parameter Stores - -Cloud-native parameter stores provide a **loosely coupled** mechanism for sharing configuration values between stacks. Values are explicitly published and consumed, providing clear contracts between teams. - -#### AWS: SSM Parameter Store - -**Publishing values (network stack):** - -```hcl -# outputs.tf - Network stack -output "vpc_id" { - description = "VPC ID for application stacks" - value = aws_vpc.main.id -} - -# main.tf - Publish to SSM -resource "aws_ssm_parameter" "vpc_id" { - name = "/${var.environment}/network/vpc_id" - type = "String" - value = aws_vpc.main.id - - tags = local.common_tags -} - -resource "aws_ssm_parameter" "private_subnet_ids" { - name = "/${var.environment}/network/private_subnet_ids" - type = "StringList" - value = join(",", aws_subnet.private[*].id) - - tags = local.common_tags -} -``` - -**Consuming values (application stack):** - -```hcl -# data.tf - Application stack -data "aws_ssm_parameter" "vpc_id" { - name = "/${var.environment}/network/vpc_id" -} - -data "aws_ssm_parameter" "private_subnet_ids" { - name = "/${var.environment}/network/private_subnet_ids" -} - -locals { - vpc_id = data.aws_ssm_parameter.vpc_id.value - private_subnet_ids = split(",", data.aws_ssm_parameter.private_subnet_ids.value) -} - -# Use in resources -resource "aws_instance" "app" { - subnet_id = local.private_subnet_ids[0] - # ... -} -``` - -> **Note:** Azure App Configuration and GCP Cloud Storage with JSON config files provide equivalent cross-stack sharing. - -### Acceptable: terraform_remote_state Data Source - -The `terraform_remote_state` data source reads outputs from another Terraform state file. While functional, it creates **tight coupling** between configurations. - -```hcl -data "terraform_remote_state" "network" { - backend = "s3" - config = { - bucket = "acme-corp-terraform-state" - key = "network/terraform.tfstate" - region = "us-east-1" - } -} - -# Access outputs from the network stack -locals { - vpc_id = data.terraform_remote_state.network.outputs.vpc_id - subnet_ids = data.terraform_remote_state.network.outputs.private_subnet_ids -} -``` - - - -### What to Share (and What Not to Share) - -| Share | Examples | Notes | -| --- | --- | --- | -| Resource IDs | VPC ID, Subnet IDs, Security Group IDs | Primary use case | -| ARNs/Resource Names | Role ARNs, Bucket names, Queue ARNs | For cross-account references | -| Endpoints | RDS endpoints, API Gateway URLs | For service discovery | -| Non-sensitive configuration | CIDR blocks, region, environment name | Shared context | - -| Do NOT Share | Reason | -| --- | --- | -| Secrets/credentials | Use dedicated secret managers with proper access controls | -| Full resource objects | Exposes unnecessary implementation details | -| Internal implementation details | Creates coupling to internal structure | - -### Cross-Stack Data Sharing Best Practices - -- **Define explicit contracts:** Document what values are published and their format. -- **Use consistent naming conventions:** Establish a parameter naming scheme (e.g., `/${environment}/${team}/${resource_type}`). -- **Version your contracts:** When changing published values, consider backward compatibility. -- **Prefer loose coupling:** Parameter stores allow consuming stacks to be applied independently of producer stacks (after initial setup). - ---- - -## Provider Management - -### Provider Version Constraints - -Provider versions **MUST** be constrained in `versions.tf`: - -| Pattern | Example | Use Case | -| --- | --- | --- | -| Pessimistic constraint | `~> 6.0` | Allow minor and patch version updates within major 6 (any 6.x.y, but not 7.0.0) | -| Exact version | `= 6.31.0` | Strict reproducibility required | -| Range constraint | `>= 6.0, < 7.0` | Explicit major version bounds (any 6.x, but not 7.0.0+) | - -In general, for a major version **M**, use a range of the form `>= M.0, < (M+1).0` to constrain to that major version while allowing all patch and minor updates within it (for example, for **M = 6**: `>= 6.0, < 7.0`). - -**Recommended approach:** - -```hcl -terraform { - required_version = ">= 1.7.0" - - required_providers { - aws = { - source = "hashicorp/aws" - version = "~> 6.0" - } - random = { - source = "hashicorp/random" - version = "~> 3.0" - } - } -} -``` - -### Lock File Management - -The `.terraform.lock.hcl` file: - -- **MUST** be committed to version control -- **SHOULD** be updated explicitly using `terraform providers lock` -- **MUST** be updated when provider versions change -- **SHOULD** include hashes for all platforms used in CI - -```bash -# Update lock file with hashes for multiple platforms -terraform providers lock \ - -platform=linux_amd64 \ - -platform=linux_arm64 \ - -platform=darwin_amd64 \ - -platform=darwin_arm64 \ - -platform=windows_amd64 \ - -platform=windows_arm64 -``` - -> **When to regenerate `.terraform.lock.hcl`:** -> -> Regenerate the lock file **only** when you intentionally change provider versions, add/remove providers, or add new execution platforms (e.g., new CI OS/architecture). Avoid running `terraform init -upgrade` unless you intend to update providers. After regeneration, review diffs and commit the updated lock file. - -### Pessimistic Constraints - -Use the pessimistic constraint operator (`~>`) for providers to allow patch updates while preventing breaking changes: - -```hcl -# Good: Allows 6.x updates but not 7.0 -version = "~> 6.0" - -# Good: Allows 6.31.x updates but not 6.32.0 -version = "~> 6.31.0" -``` - -### Provider Aliasing - -Provider aliasing enables multiple instances of the same provider for multi-region or multi-account deployments. Use aliases when resources need to be created in different regions, accounts, or with different configurations. - -**Common use cases:** - -- Disaster recovery across regions -- Cross-region replication (S3, RDS, etc.) -- Multi-account architectures -- Resources requiring different provider configurations - -**AWS Example - Defining aliased providers:** - -```hcl -# providers.tf - -provider "aws" { - region = "us-east-1" # e.g., us-east-1 - # Default provider (no alias) -} - -provider "aws" { - alias = "west" - region = "us-west-2" # e.g., us-west-2 -} - -provider "aws" { - alias = "eu" - region = "eu-west-1" # e.g., eu-west-1 -} -``` - -**AWS Example - Using aliased providers in resources:** - -```hcl -# Use default provider -resource "aws_s3_bucket" "primary" { - bucket = "acme-corp-primary-data" -} - -# Use aliased provider -resource "aws_s3_bucket" "replica" { - provider = aws.west - bucket = "acme-corp-replica-data" -} -``` - -> **Note:** Azure uses `subscription_id` and GCP uses `project`/`region` for aliased providers. - -**Passing providers to modules:** - -```hcl -module "vpc_west" { - source = "./modules/vpc" - - providers = { - aws = aws.west - } - - cidr_block = "10.1.0.0/16" -} -``` - -### Module Provider Configuration with configuration_aliases - -When a module needs to accept multiple provider configurations (e.g., for multi-region deployments), it **MUST** declare the expected provider configurations using `configuration_aliases` in the `required_providers` block. - - - -**Module declaration example:** - -```hcl -# modules/multi-region-storage/versions.tf - -terraform { - required_version = ">= 1.7.0" - - required_providers { - aws = { - source = "hashicorp/aws" - version = ">= 5.0" - configuration_aliases = [aws.primary, aws.secondary] - } - } -} -``` - -**Using aliased providers in module resources:** - -```hcl -# modules/multi-region-storage/main.tf - -resource "aws_s3_bucket" "primary" { - provider = aws.primary - bucket = "${var.name}-primary" -} - -resource "aws_s3_bucket" "secondary" { - provider = aws.secondary - bucket = "${var.name}-secondary" -} -``` - -**Calling module with provider mappings:** - -```hcl -# Root module - -module "multi_region_storage" { - source = "./modules/multi-region-storage" - - providers = { - aws.primary = aws.us_east - aws.secondary = aws.eu_west - } - - name = "my-storage" -} -``` - -**Key requirements:** - -- The `configuration_aliases` list **MUST** include all provider aliases used within the module -- Calling modules **MUST** map their provider configurations to the expected aliases -- Provider alias names in the module do not need to match the caller's alias names—the `providers` argument handles the mapping - -### Cross-Account and Service Account Patterns - -#### AWS: Assume Role Configuration - -The `assume_role` block enables Terraform to assume an IAM role in a different AWS account, providing temporary credentials for cross-account access. This pattern **SHOULD** be used instead of static credentials for cross-account AWS access. - -**Basic assume_role configuration:** - -```hcl -provider "aws" { - region = "us-east-1" - - assume_role { - role_arn = "arn:aws:iam::ACCOUNT_ID:role/TerraformRole" - session_name = "terraform-session" - external_id = var.external_id # Optional, for third-party access - } -} -``` - -**When to use assume_role:** - -- Cross-account resource management from a central deployment account -- Implementing least-privilege access patterns -- Enabling CI/CD pipelines to access multiple accounts with a single identity -- Third-party access scenarios (using `external_id` for additional security) - -**IAM trust relationship requirement:** - -The target role must have a trust policy that allows the source principal to assume it. The trust policy should specify the source account, role, or user that is permitted to assume the role. - -**Multi-account pattern with provider aliases:** - -```hcl -provider "aws" { - alias = "production" - region = "us-east-1" - - assume_role { - role_arn = "arn:aws:iam::PROD_ACCOUNT_ID:role/TerraformRole" - session_name = "terraform-prod" - } -} - -provider "aws" { - alias = "staging" - region = "us-east-1" - - assume_role { - role_arn = "arn:aws:iam::STAGING_ACCOUNT_ID:role/TerraformRole" - session_name = "terraform-staging" - } -} - -resource "aws_s3_bucket" "prod_bucket" { - provider = aws.production - bucket = "my-prod-bucket" -} - -resource "aws_s3_bucket" "staging_bucket" { - provider = aws.staging - bucket = "my-staging-bucket" -} -``` - -> **Note:** Azure uses multi-subscription with `subscription_id`/`tenant_id` and GCP uses `impersonate_service_account`. - -#### Cross-Account Pattern Summary - -| Provider | Pattern | Use Case | -| --- | --- | --- | -| AWS | `assume_role` | Access resources in different AWS accounts | -| Azure | Multiple providers with different `subscription_id` | Manage resources across subscriptions | -| GCP | `impersonate_service_account` or project-per-provider | Access resources with elevated permissions | - -#### Security Considerations for Cross-Account Access - -Cross-account patterns **MUST** follow security best practices: - -- **Prefer short-lived credentials:** `assume_role` and `impersonate_service_account` **SHOULD** be used instead of static access keys or service account key files -- **Document requirements:** Cross-account access patterns **MUST** be documented in module README files when modules require cross-account permissions -- **Use external_id for third parties:** When granting cross-account access to third parties in AWS, use `external_id` to prevent confused deputy attacks -- **Apply least-privilege:** Cross-account roles and service accounts **SHOULD** have only the permissions required for their specific tasks -- **Audit cross-account access:** Enable logging to track which identities are assuming roles or impersonating service accounts - ---- - -## Security Best Practices - -### Secret Management - -Secrets **MUST NEVER** appear in Terraform code or version control. - -#### Prohibited Patterns - -The following patterns are **PROHIBITED**: - -```hcl -# NEVER DO THIS -variable "db_password" { - default = "SuperSecretPassword123!" # PROHIBITED -} - -resource "aws_db_instance" "main" { - password = "hardcoded-password" # PROHIBITED -} -``` - -### No Secret Defaults - -Sensitive variables **MUST NOT** have default values: - -```hcl -# Correct: No default for secrets -variable "database_password" { - description = "Database password. Set via TF_VAR_database_password." - type = string - sensitive = true - # No default! -} -``` - -### Approved Secret Patterns - -#### Pattern 1: Environment Variables - -```hcl -variable "db_password" { - description = "Database password. Set via TF_VAR_db_password environment variable." - type = string - sensitive = true -} -``` - -```bash -# Choose one of the following exports based on your cloud provider: - -# AWS -export TF_VAR_db_password="$(aws secretsmanager get-secret-value --secret-id database-password --query SecretString --output text)" - -# Azure -export TF_VAR_db_password="$(az keyvault secret show --vault-name kv-acme-prod --name database-password --query value -o tsv)" - -# GCP -export TF_VAR_db_password="$(gcloud secrets versions access latest --secret=database-password)" - -# Then run terraform apply -terraform apply -``` - -#### Pattern 2: Cloud Provider Secret Managers - -**AWS Example - Secrets Manager:** - -```hcl -data "aws_secretsmanager_secret_version" "db_password" { - secret_id = "database-password" -} - -resource "aws_db_instance" "main" { - password = data.aws_secretsmanager_secret_version.db_password.secret_string -} -``` - -> **Note:** Azure Key Vault and GCP Secret Manager follow the same data source pattern. - -#### Pattern 3: HashiCorp Vault (Provider-Agnostic) - -> **Note:** This example uses `vault_generic_secret` which works with both KV v1 and KV v2 secrets engines. For KV v2, include `/data/` in the path (e.g., `secret/data/database`). The data is accessed via `.data["key"]` regardless of KV version. - -```hcl -data "vault_generic_secret" "db_creds" { - # Path MUST be customized for your Vault secret location. - path = "secret/data/production/database" -} - -# AWS Example -resource "aws_db_instance" "main" { - username = data.vault_generic_secret.db_creds.data["username"] - password = data.vault_generic_secret.db_creds.data["password"] -} - -``` - -### Sensitive Marking Requirements - -Variables containing sensitive data **MUST** be marked: - -```hcl -variable "api_key" { - description = "API key for external service" - type = string - sensitive = true # REQUIRED for secrets -} - -output "connection_string" { - description = "Database connection string (contains credentials)" - value = local.connection_string - sensitive = true # REQUIRED if contains secrets -} -``` - -### State Security - -State files contain sensitive data and **MUST** be protected. - -#### Requirements - -1. **Encryption at rest:** State backends **MUST** enable encryption -2. **Access control:** State files **MUST** be accessible only to authorized users/roles -3. **No local state in production:** Local state files **MUST NOT** be used for production -4. **State locking:** Backends **MUST** support state locking - -#### Backend Security Configuration - -**AWS Example:** - -```hcl -terraform { - backend "s3" { - bucket = "acme-corp-terraform-state" - key = "prod/terraform.tfstate" - region = "us-east-1" - encrypt = true # REQUIRED - dynamodb_table = "terraform-locks" # REQUIRED for locking - # Use IAM role or credentials from environment - } -} -``` - -**Azure Example:** - -```hcl -terraform { - backend "azurerm" { - resource_group_name = "rg-terraform-state" - storage_account_name = "stacmeterraform" - container_name = "tfstate" - key = "prod/terraform.tfstate" - # Encryption and locking are enabled by default on Azure Storage - # Use managed identity or service principal for authentication - } -} -``` - -**GCP Example:** - -```hcl -terraform { - backend "gcs" { - bucket = "acme-corp-terraform-state" - prefix = "prod" - # Encryption and locking are enabled by default on GCS - # Use service account for authentication - } -} -``` - -### Least-Privilege Principles - -IAM policies and resource permissions **MUST** follow least-privilege: - -#### IAM Policy Guidelines - -- Grant only required permissions -- Use resource-level restrictions when possible -- Avoid wildcard actions (`*`) except when truly needed -- Use conditions to further restrict access - -**AWS Example - Specific permissions (GOOD):** - -```hcl -resource "aws_iam_policy" "s3_reader" { - name = "s3-reader" - - policy = jsonencode({ - Version = "2012-10-17" - Statement = [ - { - Effect = "Allow" - Action = [ - "s3:GetObject", - "s3:ListBucket" - ] - Resource = [ - aws_s3_bucket.data.arn, - "${aws_s3_bucket.data.arn}/*" - ] - } - ] - }) -} -``` - -**AWS Example - Overly permissive (BAD):** - -```hcl -resource "aws_iam_policy" "bad_example" { - policy = jsonencode({ - Statement = [{ - Effect = "Allow" - Action = ["s3:*"] # TOO BROAD - Resource = ["*"] # TOO BROAD - }] - }) -} -``` - -**Azure Example - Specific permissions (GOOD):** - -```hcl -resource "azurerm_role_assignment" "storage_reader" { - scope = azurerm_storage_account.data.id - role_definition_name = "Storage Blob Data Reader" - principal_id = azurerm_user_assigned_identity.app.principal_id -} -``` - -**Azure Example - Overly permissive (BAD):** - -```hcl -resource "azurerm_role_assignment" "bad_example" { - scope = data.azurerm_subscription.current.id - role_definition_name = "Contributor" # TOO BROAD - grants write access to entire subscription - principal_id = azurerm_user_assigned_identity.app.principal_id -} -``` - -**GCP Example - Specific permissions (GOOD):** - -```hcl -resource "google_storage_bucket_iam_member" "viewer" { - bucket = google_storage_bucket.data.name - role = "roles/storage.objectViewer" - member = "serviceAccount:${google_service_account.app.email}" -} -``` - -**GCP Example - Overly permissive (BAD):** - -```hcl -resource "google_project_iam_member" "bad_example" { - project = var.project_id - role = "roles/editor" # TOO BROAD - grants write access to entire project - member = "serviceAccount:${google_service_account.app.email}" -} -``` - -### Sensitive Values in Meta-Arguments - -Values used in `for_each` keys and `count` expressions appear in Terraform state and plan output, even if the source variable is marked `sensitive = true`. This creates a security risk where sensitive data can be exposed. - -**Security requirements:** - -- Sensitive values **MUST NOT** be used as `for_each` keys -- Sensitive values **MUST NOT** be used in `count` conditions where the value itself is exposed -- Use non-sensitive identifiers as keys and reference sensitive values only in resource attributes - -**Anti-pattern (sensitive values exposed in state):** - -```hcl -# BAD: Sensitive value used as for_each key - names will appear in state -variable "secret_names" { - type = list(string) - sensitive = true -} - -resource "aws_secretsmanager_secret" "secrets" { - for_each = toset(var.secret_names) # Keys leak to state! - name = each.key -} -``` - -**Correct pattern (non-sensitive keys):** - -```hcl -# GOOD: Use non-sensitive identifiers as keys -variable "secrets" { - type = map(object({ - name = string - value = string - })) - sensitive = true -} - -resource "aws_secretsmanager_secret" "secrets" { - for_each = { for k, v in var.secrets : k => v.name } - name = each.value # each.key is the non-sensitive identifier -} -``` - -### Ephemeral Values - -Terraform 1.10 introduced `ephemeral` values—a mechanism for handling sensitive data that should never be persisted to state. Unlike `sensitive` values (which are stored in state but redacted in output), ephemeral values are never written to state at all. - -#### When to Use Ephemeral vs Sensitive - -| Attribute | Behavior | Use Case | -| --- | --- | --- | -| `sensitive = true` | Value is stored in state but redacted in plan/apply output | Secrets that Terraform needs to track for drift detection | -| `ephemeral = true` | Value is never written to state | Temporary tokens, session credentials, values that should not persist | - -**Key constraint:** Ephemeral values **MUST NOT** be used in resource arguments that persist to state. They are intended for use in providers, provisioners, and other contexts where the value is consumed but not stored. - -#### Ephemeral Variable Declaration - -```hcl -variable "temporary_token" { - description = "Short-lived authentication token for provider configuration." - type = string - ephemeral = true # Value is never written to state -} - -variable "session_credentials" { - description = "Temporary session credentials that should not persist." - type = object({ - access_key = string - secret_key = string - session_token = string - }) - ephemeral = true -} -``` - -#### Ephemeral Output Declaration - -```hcl -output "generated_token" { - description = "Token generated during apply (not persisted to state)." - value = local.computed_token - ephemeral = true -} -``` - -#### Use Cases for Ephemeral Values - -- **Provider authentication:** Temporary tokens used to authenticate providers that should not persist in state -- **Session credentials:** Short-lived credentials from assume-role operations -- **One-time values:** Tokens or secrets used during provisioning but not needed afterward -- **Compliance requirements:** Values that organizational policy prohibits from being stored - -> **Note:** Ephemeral values require Terraform 1.10.0 or later. For older versions, use `sensitive = true` and implement additional state protection measures. - -### Sensitive Output Exposure in CLI - -Marking outputs as `sensitive = true` prevents values from appearing in `terraform plan` and `terraform apply` output, but it **does not** prevent programmatic access via certain CLI commands: - -- `terraform output -json` — Returns all outputs, and sensitive values are present in the JSON payload -- `terraform output -raw ` — Prints the raw value of the named output, including sensitive values, in plaintext -- `terraform output ` — Prints non-sensitive outputs in plaintext; sensitive outputs are redacted unless you use `-raw` or `-json` -- `terraform show -json` — When used to show **state** (current or historical), the JSON output can include sensitive values from that state; when used to show a **plan** file, Terraform intentionally redacts/omits sensitive values from the plan JSON - -This behavior is intentional—it allows programmatic access to sensitive values when needed (primarily via state and outputs). However, it creates significant security risks in CI/CD pipelines. - -**CI/CD Security Implications:** - -When accessing sensitive outputs in CI/CD pipelines: - -- **DO NOT** log the output of `terraform output -json` directly -- Use targeted output access: `terraform output -raw ` for single values -- Pipe sensitive values directly to consumers without intermediate logging -- Consider using external secret managers instead of Terraform outputs for highly sensitive values - -**Anti-pattern:** - -```bash -# BAD: Logs all outputs including sensitive values -terraform output -json | tee outputs.json - -# BAD: May appear in CI logs -echo "Database password: $(terraform output -raw database_password)" -``` - -**Recommended patterns:** - -```bash -# GOOD: Targeted access without logging -DB_PASSWORD=$(terraform output -raw database_password) - -# GOOD: Direct pipe to consumer without intermediate storage -terraform output -raw rendered_manifest | kubectl apply -f - - -# GOOD: Mask in CI systems that support it (GitHub Actions example) -echo "::add-mask::$(terraform output -raw api_key)" -API_KEY=$(terraform output -raw api_key) -``` - -**Best practices:** - -- Use CI platform secret masking features when available -- Prefer external secret managers (AWS Secrets Manager, Azure Key Vault, HashiCorp Vault) for highly sensitive values -- Audit CI pipeline logs to ensure sensitive values are not exposed -- Consider using `nonsensitive()` function only when you explicitly need to expose a value and understand the security implications - -### Security Scanning - -Security scanning tools **SHOULD** be integrated into the development workflow. - -#### Recommended Tools - -| Tool | Purpose | Integration | Notes | -| --- | --- | --- | --- | -| `trivy` | Comprehensive security scanning | Pre-commit, CI | Recommended; successor to tfsec | -| `checkov` | Policy-as-code scanning | Pre-commit, CI | | -| `terrascan` | Security and compliance | CI | | -| `tfsec` | Static security analysis | Pre-commit, CI | Legacy; now part of Trivy | - -> **Note:** `tfsec` has been absorbed into Aqua Security's `trivy` and is now in maintenance mode. New projects **SHOULD** use `trivy` for Terraform security scanning. Existing workflows using `tfsec` will continue to work but **SHOULD** plan migration to `trivy`. - -#### Pre-commit Integration Example - -```yaml -- repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.105.0 - hooks: - - id: terraform_trivy # Recommended for new projects - - id: terraform_checkov - # - id: terraform_tfsec # Legacy; use terraform_trivy instead -``` - ---- - -## Testing with Terraform Test - -Terraform's native test framework (introduced in Terraform 1.6) provides a way to validate configurations without external testing tools. This section documents testing conventions that integrate with the coding standards in this guide. - -> **Note:** Terraform tests require Terraform 1.6.0 or later. Mock providers require Terraform 1.7.0 or later. For older Terraform versions, consider Terratest or other external testing frameworks. - -### Test File Naming - -Test files **MUST** use the `.tftest.hcl` extension: - -- `basic.tftest.hcl` -- `validation.tftest.hcl` -- `integration.tftest.hcl` - -### Test File Location - -Test files **SHOULD** be placed in a `tests/` directory within the module: - -```text -modules/ -└── vpc/ - ├── main.tf - ├── variables.tf - ├── outputs.tf - ├── versions.tf - ├── README.md - └── tests/ - ├── basic.tftest.hcl - ├── validation.tftest.hcl - └── integration.tftest.hcl -``` - -**Alternative:** Tests alongside configuration: - -```text -modules/ -└── vpc/ - ├── main.tf - ├── variables.tf - ├── outputs.tf - ├── vpc.tftest.hcl - └── vpc_validation.tftest.hcl -``` - -### Test Structure - -Terraform test files use HCL syntax with specific blocks. - -#### Basic Test Structure - -```hcl -# tests/basic.tftest.hcl - -# Variables block (optional) - Set values for tests -variables { - environment = "test" - vpc_cidr = "10.0.0.0/16" -} - -# Run block - Defines a test scenario -run "creates_vpc_with_correct_cidr" { - command = plan # or apply - - assert { - condition = aws_vpc.main.cidr_block == "10.0.0.0/16" - error_message = "VPC CIDR block does not match expected value" - } -} - -run "creates_required_subnets" { - command = plan - - assert { - condition = length(aws_subnet.private) == 3 - error_message = "Expected 3 private subnets" - } -} -``` - -#### Test Block Reference - -| Block | Purpose | Requirement | -| --- | --- | --- | -| `variables {}` | Set input variable values for tests | MAY | -| `provider {}` | Configure provider for tests (e.g., mock) | MAY | -| `run "name" {}` | Define a test scenario | MUST (at least one) | -| `assert {}` | Define a test assertion within a run | MUST (at least one per run) | -| `expect_failures` | Expect specific resources/outputs to fail | MAY | - -### Test Assertions - -Each `run` block **MUST** include at least one `assert`: - -```hcl -run "instance_has_correct_type" { - command = plan - - assert { - condition = aws_instance.main.instance_type == var.instance_type - error_message = "Instance type mismatch" - } - - assert { - condition = aws_instance.main.tags["Environment"] == "test" - error_message = "Instance must have Environment tag" - } -} -``` - -### Testing Variable Validation - -Test validation rules with `expect_failures`: - -```hcl -run "rejects_invalid_environment" { - command = plan - - variables { - environment = "invalid" # Should fail validation - } - - expect_failures = [ - var.environment - ] -} - -run "accepts_valid_environment" { - command = plan - - variables { - environment = "prod" - } - - assert { - condition = var.environment == "prod" - error_message = "Environment should be prod" - } -} -``` - -### Testing Outputs - -```hcl -run "outputs_vpc_id" { - command = apply - - assert { - condition = output.vpc_id != null && output.vpc_id != "" - error_message = "vpc_id output must not be empty" - } -} - -run "outputs_correct_subnet_count" { - command = apply - - assert { - condition = length(output.subnet_ids) == 3 - error_message = "Expected 3 subnet IDs in output" - } -} -``` - -### Mock Providers - -For unit testing without real infrastructure, use mock providers: - -> **Note:** The `mock_provider` block requires Terraform 1.7.0 or later. For earlier versions of the test framework (Terraform 1.6.x), tests must use real provider credentials or skip provider-dependent tests. - -**AWS Example:** - -```hcl -# tests/unit.tftest.hcl - -mock_provider "aws" { - mock_data "aws_availability_zones" { - defaults = { - names = ["us-east-1a", "us-east-1b", "us-east-1c"] - } - } -} - -run "uses_all_availability_zones" { - command = plan - - assert { - condition = length(aws_subnet.private) == 3 - error_message = "Should create subnet in each AZ" - } -} -``` - -**Azure Example:** - -```hcl -# tests/unit.tftest.hcl - -mock_provider "azurerm" { - mock_data "azurerm_client_config" { - defaults = { - tenant_id = "00000000-0000-0000-0000-000000000002" - subscription_id = "00000000-0000-0000-0000-000000000001" - } - } -} - -run "resource_group_has_correct_location" { - command = plan - - assert { - condition = azurerm_resource_group.main.location == "eastus" - error_message = "Resource group should be in eastus" - } -} -``` - -**GCP Example:** - -```hcl -# tests/unit.tftest.hcl - -mock_provider "google" { - mock_data "google_compute_zones" { - defaults = { - names = ["us-central1-a", "us-central1-b", "us-central1-c"] - } - } -} - -run "uses_all_compute_zones" { - command = plan - - assert { - condition = length(google_compute_instance.main) == 3 - error_message = "Should create instance in each zone" - } -} -``` - -### Unit Tests - -Unit tests use `command = plan` (the default): - -- Do NOT create real resources -- Fast execution -- Test configuration logic, not cloud provider behavior - -```hcl -run "unit_test_example" { - command = plan # Explicit, though it's the default - - assert { - condition = aws_instance.main.instance_type == var.instance_type - error_message = "Instance type mismatch" - } -} -``` - -### Integration Tests - -Integration tests use `command = apply`: - -- Create and destroy real resources -- Slower execution -- Test actual infrastructure behavior -- Require valid provider credentials - -```hcl -run "integration_test_example" { - command = apply # Creates real resources - - assert { - condition = aws_instance.main.id != "" - error_message = "Instance should be created" - } -} -``` - -**Best Practice:** Run unit tests (`plan`) frequently during development. Run integration tests (`apply`) in CI or before releases. - -### Running Tests - -#### Basic Test Execution - -```bash -# Run all tests in current directory -terraform test - -# Run tests with verbose output -terraform test -verbose - -# Run specific test file -terraform test -filter=tests/basic.tftest.hcl -``` - -#### CI Integration - -```yaml -# .github/workflows/terraform-ci.yml (test job) -- name: Terraform Init - run: terraform init - -- name: Terraform Test - run: terraform test -verbose -``` - -### What to Test - -Tests **SHOULD** cover: - -| Category | What to Test | Example | -| --- | --- | --- | -| **Variable validation** | Custom validation rules work correctly | Invalid environment rejected | -| **Computed values** | Locals and expressions compute correctly | CIDR calculations | -| **Resource configuration** | Resources have expected attributes | Instance type, tags | -| **Output values** | Outputs contain expected values | VPC ID not empty | -| **Module integration** | Modules work together | VPC + subnets + security groups | -| **Edge cases** | Boundary conditions | Zero instances, empty lists | - -**Not to test:** - -- Cloud provider behavior (e.g., "does AWS actually create an EC2?") -- Terraform core functionality -- External service availability - ---- - -## Documentation Standards - -### Module README Requirements - -Every module **MUST** include a `README.md` with: - -1. **Description** — What the module does -2. **Usage example** — Minimal working example -3. **Requirements** — Terraform and provider versions -4. **Inputs** — All variables with descriptions -5. **Outputs** — All outputs with descriptions - -#### README Template - -````markdown -# Module Name - -Brief description of what this module creates. - -## Usage - -```hcl -module "example" { - source = "./modules/example" - - required_variable = "value" -} -``` - -## Requirements - -| Name | Version | -| --- | --- | -| terraform | >= 1.6.0 | -| aws | ~> 6.0 | - -## Inputs - -| Name | Description | Type | Default | Required | -| --- | --- | --- | --- | --- | -| required_variable | Description here | `string` | n/a | yes | -| optional_variable | Description here | `string` | `"default"` | no | - -## Outputs - -| Name | Description | -| --- | --- | -| output_name | Description of output | -```` - -**Note:** Consider using `terraform-docs` to generate input/output tables automatically. - -### Inline Comment Conventions - -Comments **SHOULD** explain "why," not "what." - -#### Single-Line Comments - -Use `#` for single-line comments: - -```hcl -# Enable encryption to meet compliance requirements (SOC2) -resource "aws_s3_bucket_server_side_encryption_configuration" "main" { - bucket = aws_s3_bucket.main.id - # ... -} -``` - -#### Block Comments - -Use `/* */` sparingly for multi-line explanations: - -```hcl -/* - * This security group allows inbound traffic from the corporate VPN. - * CIDR ranges are managed by the network team and should not be - * modified without their approval. - */ -resource "aws_security_group" "vpn_access" { - # ... -} -``` - -### TODO Comment Format - -Use standardized TODO format: - -```hcl -# TODO(username): Migrate to new VPC module after v2.0 release -# TODO(team-name): Add support for IPv6 when available -``` - -### Terraform Registry Documentation URLs - -Terraform Registry reference URLs embedded in comments in `.tf`, `.tftest.hcl`, `.tfvars`, `.tfbackend`, `.tftpl`, and other Terraform-related files that support comments **MUST** use the `latest` path segment, not a pinned provider or module version. - -- Provider documentation URLs **MUST** use `latest` as the version segment immediately after `/providers///` (for example, `https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/storage_account`). Any path, query string, or fragment after the `latest` segment is permitted. -- Module documentation URLs **MUST** use `latest` as the version segment immediately after `/modules////` (for example, `https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws/latest`). Any path (such as `/submodules/...`), query string, or fragment after the `latest` segment is permitted. -- Pinned Terraform Registry documentation URLs (for example, `/azurerm/4.67.0/` or `/random/3.8.1/`) **MUST NOT** appear in comments, except in clearly labeled non-compliant examples that document this rule. - -This rule applies **only** to documentation/navigation comments. It **MUST NOT** affect provider constraints, module `source` addresses, module `version` arguments, `.terraform.lock.hcl`, selected module `source` references, or any other intentionally pinned executable configuration. - -**Compliant:** - -```hcl -# See https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/storage_account -resource "azurerm_storage_account" "main" { - # ... -} - -# Module reference: https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws/latest -module "vpc" { - source = "terraform-aws-modules/vpc/aws" - version = "~> 5.0" - # ... -} -``` - -**Non-compliant:** - -```hcl -# See https://registry.terraform.io/providers/hashicorp/azurerm/4.67.0/docs/resources/storage_account -resource "azurerm_storage_account" "main" { - # ... -} - -# Module reference: https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws/5.21.0 -module "vpc" { - source = "terraform-aws-modules/vpc/aws" - version = "~> 5.0" - # ... -} -``` - - - -### Error Message Best Practices - -Error messages in `validation`, `precondition`, `postcondition`, and `assert` blocks **MUST** be actionable and help users understand how to fix the problem. - -**Requirements:** - -- Error messages **MUST** be actionable and tell the user how to fix the problem -- Error messages **SHOULD** include the actual value that failed validation when possible -- Error messages **SHOULD** reference valid options or acceptable ranges -- Error messages **MUST NOT** expose sensitive values - -**Validation block examples:** - -```hcl -# Poor: Not actionable -validation { - condition = contains(["dev", "staging", "prod"], var.environment) - error_message = "Invalid environment." -} - -# Good: Actionable with valid options -validation { - condition = contains(["dev", "staging", "prod"], var.environment) - error_message = "Environment must be one of: dev, staging, prod. Received: ${var.environment}" -} -``` - -**Postcondition examples:** - -```hcl -# Poor: No context -postcondition { - condition = self.public_ip != null - error_message = "No public IP." -} - -# Good: Explains what to check -postcondition { - condition = self.public_ip != null - error_message = "Instance must have a public IP assigned. Verify that the subnet has map_public_ip_on_launch enabled or associate an Elastic IP." -} -``` - ---- - -## Common Anti-Patterns to Avoid - -This section consolidates common anti-patterns to help developers avoid frequent mistakes. Each anti-pattern links to detailed guidance elsewhere in this document. - -| Anti-Pattern | Why It's Problematic | Correct Approach | -| --- | --- | --- | -| Hardcoded secrets in `.tf` files | Secrets exposed in version control and state | Use environment variables or secret managers | -| Using `terraform apply -target` regularly | Creates state drift and inconsistent infrastructure | Split into smaller, independent configurations | -| Redundant `depends_on` for inferable dependencies | Adds noise and can mask actual dependency issues | Let Terraform infer dependencies from references | -| Using `count` with collections that may reorder | Index shifts cause unintended resource recreation | Use `for_each` with stable keys | -| Wildcard IAM actions (`"*"`) | Violates least-privilege; security risk | Specify exact required actions | -| Local state in team/production environments | No locking, no encryption, easily lost | Configure remote backend | -| Committing `.tfstate` files to version control | Exposes sensitive data; causes merge conflicts | Add to `.gitignore`; use remote backend | -| Using `try()` without understanding failure modes | Silently masks errors that should be investigated | Document expected failures; consider explicit null checks | -| Overly complex expressions in resource blocks | Reduces readability; hard to debug | Extract to `locals` with descriptive names | -| Default values for sensitive variables | Encourages insecure deployments | Require explicit values; no defaults for secrets | - ---- - -## Pre-commit Discipline for Terraform - -**⚠️ ALWAYS run pre-commit checks before committing Terraform code.** - -Pre-commit hooks for Terraform **SHOULD** include: - -1. `terraform fmt` — Format check -2. `terraform validate` — Syntax validation -3. `tflint` — Linting -4. Security scanning (optional but recommended) - -### Workflow - -1. Make Terraform changes -2. Run `terraform fmt -recursive` -3. Run `terraform validate` -4. Run pre-commit hooks: `pre-commit run --all-files` -5. Review and commit ALL auto-fixes as part of your change -6. Push to GitHub - -**CI is a safety net, not a substitute for local checks.** - -### Pre-commit Configuration - -```yaml -# .pre-commit-config.yaml (Terraform section) -repos: - - repo: https://github.com/antonbabenko/pre-commit-terraform - rev: v1.105.0 - hooks: - - id: terraform_fmt - - id: terraform_validate - - id: terraform_tflint - args: - - --args=--config=__GIT_WORKING_DIR__/.tflint.hcl -``` - -### TFLint Configuration - -This repository's TFLint configuration is defined in `.tflint.hcl` at the repository root. The configuration: - -- Enables the `terraform` plugin with the `recommended` preset -- Enforces `snake_case` naming conventions -- Requires documentation for variables and outputs -- Requires type declarations for variables -- Includes commented provider-specific plugins (AWS, Azure, GCP) that **SHOULD** be uncommented based on your cloud provider - -When adopting this template, review and customize `.tflint.hcl` for your project's provider requirements. - -> **Note:** When enabling provider-specific TFLint plugins (AWS, Azure, GCP), verify that you are using the latest stable versions. The plugin versions in the template may become outdated. Check the [TFLint Ruleset Registry](https://github.com/terraform-linters) for current versions. - -### CI Workflow Integration - -This repository includes a comprehensive Terraform CI workflow at `.github/workflows/terraform-ci.yml` that enforces these standards automatically: - -- **Format Check:** Runs `terraform fmt -check -recursive` -- **Validate:** Runs `terraform init` and `terraform validate` for all Terraform directories -- **Lint:** Runs TFLint with the repository's `.tflint.hcl` configuration -- **Test:** Runs `terraform test` for directories containing `.tftest.hcl` files - -The CI workflow uses job dependencies to fail fast: format issues block validation, and validation issues block linting and testing. - -For workflow customization options (such as enabling security scanning), see the comments in `.github/workflows/terraform-ci.yml`. - ---- - -## "Done" Definition for Terraform Changes - -A Terraform change is considered complete when: - -### Code Quality - -- [ ] All code passes `terraform fmt -check -recursive` -- [ ] All code passes `terraform validate` -- [ ] All code passes `tflint` without errors -- [ ] Pre-commit hooks pass - -### Documentation - -- [ ] All variables have `description` attributes -- [ ] All outputs have `description` attributes -- [ ] Sensitive variables/outputs are marked with `sensitive = true` -- [ ] Module `README.md` is updated (if applicable) - -### Security - -- [ ] No secrets appear in code or tfvars files -- [ ] State backend has encryption enabled -- [ ] IAM policies follow least-privilege -- [ ] Security scanning passes (if configured) - -### Testing - -- [ ] Variable validation rules are tested -- [ ] Critical outputs are tested -- [ ] Tests pass: `terraform test` - -### Version Control - -- [ ] `.terraform.lock.hcl` is committed -- [ ] State files are NOT committed -- [ ] All changes are committed with descriptive messages - -### CI/CD - -- [ ] CI pipeline passes -- [ ] Plan output reviewed (no unexpected changes) - ---- - -## Scope Exceptions & Deviations from Standards - -Document justified deviations from these standards. No deviations are currently recorded. diff --git a/.github/instructions/yaml.instructions.md b/.github/instructions/yaml.instructions.md index 0809c8a..61b66e5 100644 --- a/.github/instructions/yaml.instructions.md +++ b/.github/instructions/yaml.instructions.md @@ -39,7 +39,7 @@ To keep YAML safe to edit, easy to diff, and portable across parsers, this repos - **[Actions]** **MUST** apply least-privilege `permissions:` on GitHub Actions workflows. - **[Schemas]** Schema-backed YAML **MUST** pass any schema validator wired into pre-commit or CI; where no validator is wired up for a particular file family, authors **SHOULD** run the appropriate validator locally before committing. - **[Naming]** YAML filenames **SHOULD** be lowercase kebab-case; GitHub Actions workflows **MUST** use the `.yml` extension; project-owned YAML **MUST** choose `.yml` or `.yaml` and use it consistently. -- **[IssueForms]** In `.github/ISSUE_TEMPLATE/*.yml`, repo-internal targets in both issue-form `value:` Markdown links (e.g., `bug_report.yml`) and `config.yml` `contact_links` `url:` fields **MUST** use absolute `https://github.com/OWNER/REPO/...` URLs (with `blob/HEAD` for file links); relative paths **MUST NOT** be used. The two file types fail for different reasons: `value:` Markdown blocks render at `/{owner}/{repo}/issues/new?...` so relative paths resolve against that URL and 404, while `contact_links` `url:` fields are not Markdown at all — GitHub validates them as absolute URLs at form-load time and rejects relative values outright. +- **[IssueForms]** In `.github/ISSUE_TEMPLATE/*.yml`, repo-internal targets in both issue-form `value:` Markdown links (e.g., `bug_report.yml`) and `config.yml` `contact_links` `url:` fields **MUST** use absolute `https://github.com/franklesniak/macOSLab/...` URLs (with `blob/HEAD` for file links); relative paths **MUST NOT** be used. The two file types fail for different reasons: `value:` Markdown blocks render at `/{owner}/{repo}/issues/new?...` so relative paths resolve against that URL and 404, while `contact_links` `url:` fields are not Markdown at all — GitHub validates them as absolute URLs at form-load time and rejects relative values outright. ## Dialect and Consumer Policy @@ -106,12 +106,12 @@ GitHub issue forms render their `value:` Markdown blocks at the URL `/{owner}/{r To make these links robust across non-GitHub.com renderers, GitHub Mobile, email notifications, and copied/quoted content, the following rules apply to all files matching `.github/ISSUE_TEMPLATE/*.yml` (including `config.yml`): -- Markdown links to repo-internal files (for example, `SECURITY.md`, `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `README.md`) **MUST** use full absolute URLs of the form `https://github.com/OWNER/REPO/blob/HEAD/`. The `OWNER/REPO` placeholder follows this template's placeholder convention (see the comment block at the top of `CONTRIBUTING.md`) and is enforced by `.github/workflows/check-placeholders.yml` **if the downstream repository keeps that workflow** — `.github/workflows/check-placeholders.yml` is itself an optional adoption step (see `OPTIONAL_CONFIGURATIONS.md` and `GETTING_STARTED_*` guides) and may be removed once placeholders are substituted, so authors and adopters MUST NOT rely on the workflow as an unconditional CI guardrail. -- Repo-internal references that are not file paths (for example, the GitHub Security tab) **MUST** likewise use absolute URLs, such as `https://github.com/OWNER/REPO/security`. +- Markdown links to repo-internal files (for example, `SECURITY.md`, `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `README.md`) **MUST** use full absolute URLs of the form `https://github.com/franklesniak/macOSLab/blob/HEAD/`. The old template placeholder workflow has been removed; authors MUST NOT introduce live `OWNER/REPO` placeholder URLs into this initialized repository. +- Repo-internal references that are not file paths (for example, the GitHub Security tab) **MUST** likewise use absolute URLs, such as `https://github.com/franklesniak/macOSLab/security`. - Relative paths such as `../blob/HEAD/`, `blob/HEAD/`, `./`, or bare relative refs such as `(security)` **MUST NOT** be used in issue-form `value:` Markdown blocks or in `contact_links` URLs. - Use `blob/HEAD` rather than `blob/main` so the URL works regardless of the repository's default branch name. -- The `github.com` host is the assumed default; **GHES adopters MUST replace `github.com` with their GHES host** (e.g., `github.company.com`). The host substitution is not enforced by CI today (even when `.github/workflows/check-placeholders.yml` is kept, it only validates `OWNER/REPO`), so each affected file SHOULD include a brief inline YAML comment reminding adopters of the host substitution, mirroring the convention already used in `.github/ISSUE_TEMPLATE/config.yml`. -- The literal `https://github.com/OWNER/REPO/...` example URL is permitted to appear in didactic prose inside the style-guide files under `.github/instructions/**`; section [6] of `.github/workflows/check-placeholders.yml` skips those files specifically so adopters are not forced to edit instructional prose to satisfy placeholder CI. Section [6] also skips the workflow file itself (`.github/workflows/check-placeholders.yml`) to avoid self-referential matches against the literal URL embedded in its own grep patterns. Any other YAML file under `.github/` (i.e., not `.github/instructions/**` and not `.github/workflows/check-placeholders.yml`) that contains the literal `https://github.com/OWNER/REPO` substring outside a YAML comment line is treated as a live template placeholder and **MUST** be customized by adopters. (Sections [5] and [6] of the placeholder workflow filter YAML comment lines before matching, so a commented example such as `# https://github.com/OWNER/REPO/...` will not by itself cause CI to fail; authors **SHOULD NOT** rely on that filter to ship live template URLs disguised as comments.) +- If the repository is ever migrated to a different host or owner/name, update these absolute URLs as part of that migration. +- Live GitHub.com placeholder URLs that use generic owner/repo tokens **MUST NOT** appear in YAML files in this initialized repository. Didactic examples that need to discuss generic repository URLs SHOULD use non-live placeholders such as `https://github.example/OWNER/REPO/...` or explain the pattern in prose rather than embedding a live GitHub URL. This rule is mirrored in [`.github/instructions/docs.instructions.md`](./docs.instructions.md) (which governs `.github/pull_request_template.md` and applies to `**/*.md`). The two instruction files are intentionally self-contained: each restates the rule rather than relying on the other so that downstream repositories may remove either file independently without losing the guidance. diff --git a/AGENTS.md b/AGENTS.md index 24646e5..2ed8d81 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -38,12 +38,10 @@ If a style-guide update appears warranted but has not been explicitly authorized - Do not push code when pre-commit or required validation checks are failing; fix issues and re-run until the checks pass. - Use the repository's existing validation commands as needed: - `npm run lint:md` - - `pytest tests/ -v --cov --cov-report=term-missing` - - `pytest tests/test_schema_examples.py -v` (after any schema or schema-example change) + - `npm run lint:md:nested` + - `pre-commit run --all-files` + - `Invoke-ScriptAnalyzer -Path . -Settings .\.github\linting\PSScriptAnalyzerSettings.psd1` - `Invoke-Pester -Path tests/ -Output Detailed` - - `terraform fmt -check -recursive` - - `tflint --recursive` - - `terraform test -verbose` - The `pre-commit run --all-files` command exercises the data-file hooks configured in [`.pre-commit-config.yaml`](.pre-commit-config.yaml) (the authoritative list of active hooks), covering categories such as strict JSON syntax (`check-json`), YAML parsing (`check-yaml`) and style (`yamllint`), GitHub Actions linting (`actionlint`), JSON Schema validation (`check-jsonschema`), and schema self-validation (`check-metaschema`). The dedicated [`.github/workflows/data-ci.yml`](.github/workflows/data-ci.yml) workflow re-runs these so JSON/YAML/Actions enforcement can be required via branch protection. - JSON, YAML, and schema authoring guidance lives in: - [`.github/instructions/json.instructions.md`](.github/instructions/json.instructions.md) @@ -57,8 +55,6 @@ If a style-guide update appears warranted but has not been explicitly authorized - JSON: `.github/instructions/json.instructions.md` - Markdown/Docs: `.github/instructions/docs.instructions.md` - PowerShell: `.github/instructions/powershell.instructions.md` - - Python: `.github/instructions/python.instructions.md` - - Terraform: `.github/instructions/terraform.instructions.md` - YAML: `.github/instructions/yaml.instructions.md` - **Do not** @@ -68,7 +64,3 @@ If a style-guide update appears warranted but has not been explicitly authorized - Add new major dependencies without clear justification. - Invent behavior when requirements are ambiguous; use an explicit Open Question. - Create separate formatting-only or lint-only commits. - ---- - -> This file is part of the `franklesniak/copilot-repo-template` template. Customize or remove agent instruction files for platforms you do not use. See `OPTIONAL_CONFIGURATIONS.md` for details. diff --git a/CLAUDE.md b/CLAUDE.md index 190515b..7363b62 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -38,12 +38,10 @@ If a style-guide update appears warranted but has not been explicitly authorized - Do not push code when pre-commit or required validation checks are failing; fix issues and re-run until the checks pass. - Use the repository's existing validation commands as needed: - `npm run lint:md` - - `pytest tests/ -v --cov --cov-report=term-missing` - - `pytest tests/test_schema_examples.py -v` (after any schema or schema-example change) + - `npm run lint:md:nested` + - `pre-commit run --all-files` + - `Invoke-ScriptAnalyzer -Path . -Settings .\.github\linting\PSScriptAnalyzerSettings.psd1` - `Invoke-Pester -Path tests/ -Output Detailed` - - `terraform fmt -check -recursive` - - `tflint --recursive` - - `terraform test -verbose` - The `pre-commit run --all-files` command exercises the data-file hooks configured in [`.pre-commit-config.yaml`](.pre-commit-config.yaml) (the authoritative list of active hooks), covering categories such as strict JSON syntax (`check-json`), YAML parsing (`check-yaml`) and style (`yamllint`), GitHub Actions linting (`actionlint`), JSON Schema validation (`check-jsonschema`), and schema self-validation (`check-metaschema`). The dedicated [`.github/workflows/data-ci.yml`](.github/workflows/data-ci.yml) workflow re-runs these so JSON/YAML/Actions enforcement can be required via branch protection. - JSON, YAML, and schema authoring guidance lives in: - [`.github/instructions/json.instructions.md`](.github/instructions/json.instructions.md) @@ -57,8 +55,6 @@ If a style-guide update appears warranted but has not been explicitly authorized - JSON: `.github/instructions/json.instructions.md` - Markdown/Docs: `.github/instructions/docs.instructions.md` - PowerShell: `.github/instructions/powershell.instructions.md` - - Python: `.github/instructions/python.instructions.md` - - Terraform: `.github/instructions/terraform.instructions.md` - YAML: `.github/instructions/yaml.instructions.md` - **Do not** @@ -175,7 +171,3 @@ When a pull request is created or when the owner posts a PR comment containing ` ### Resuming a paused loop When the PR owner posts a comment containing `@claude resume review loop`, resume the loop from step 1 (request a fresh Copilot review). The round counter and timeout reset on resume. - ---- - -> This file is part of the `franklesniak/copilot-repo-template` template. Customize or remove agent instruction files for platforms you do not use. See `OPTIONAL_CONFIGURATIONS.md` for details. diff --git a/GEMINI.md b/GEMINI.md index 6e9aa80..7f57a87 100644 --- a/GEMINI.md +++ b/GEMINI.md @@ -38,12 +38,10 @@ If a style-guide update appears warranted but has not been explicitly authorized - Do not push code when pre-commit or required validation checks are failing; fix issues and re-run until the checks pass. - Use the repository's existing validation commands as needed: - `npm run lint:md` - - `pytest tests/ -v --cov --cov-report=term-missing` - - `pytest tests/test_schema_examples.py -v` (after any schema or schema-example change) + - `npm run lint:md:nested` + - `pre-commit run --all-files` + - `Invoke-ScriptAnalyzer -Path . -Settings .\.github\linting\PSScriptAnalyzerSettings.psd1` - `Invoke-Pester -Path tests/ -Output Detailed` - - `terraform fmt -check -recursive` - - `tflint --recursive` - - `terraform test -verbose` - The `pre-commit run --all-files` command exercises the data-file hooks configured in [`.pre-commit-config.yaml`](.pre-commit-config.yaml) (the authoritative list of active hooks), covering categories such as strict JSON syntax (`check-json`), YAML parsing (`check-yaml`) and style (`yamllint`), GitHub Actions linting (`actionlint`), JSON Schema validation (`check-jsonschema`), and schema self-validation (`check-metaschema`). The dedicated [`.github/workflows/data-ci.yml`](.github/workflows/data-ci.yml) workflow re-runs these so JSON/YAML/Actions enforcement can be required via branch protection. - JSON, YAML, and schema authoring guidance lives in: - [`.github/instructions/json.instructions.md`](.github/instructions/json.instructions.md) @@ -57,8 +55,6 @@ If a style-guide update appears warranted but has not been explicitly authorized - JSON: `.github/instructions/json.instructions.md` - Markdown/Docs: `.github/instructions/docs.instructions.md` - PowerShell: `.github/instructions/powershell.instructions.md` - - Python: `.github/instructions/python.instructions.md` - - Terraform: `.github/instructions/terraform.instructions.md` - YAML: `.github/instructions/yaml.instructions.md` - **Do not** @@ -68,7 +64,3 @@ If a style-guide update appears warranted but has not been explicitly authorized - Add new major dependencies without clear justification. - Invent behavior when requirements are ambiguous; use an explicit Open Question. - Create separate formatting-only or lint-only commits. - ---- - -> This file is part of the `franklesniak/copilot-repo-template` template. Customize or remove agent instruction files for platforms you do not use. See `OPTIONAL_CONFIGURATIONS.md` for details. diff --git a/docs/phase-0-bootstrap-codex-instructions.md b/docs/phase-0-bootstrap-codex-instructions.md index ae61f21..d54c12e 100644 --- a/docs/phase-0-bootstrap-codex-instructions.md +++ b/docs/phase-0-bootstrap-codex-instructions.md @@ -4,15 +4,17 @@ ## Metadata -- **Status:** Active +- **Status:** Deprecated - **Owner:** Repository Maintainers -- **Last Updated:** 2026-05-04 -- **Scope:** Revised task instructions for running Phase 0 bootstrap work in `franklesniak/macOSLab` with Codex. This document is a handoff prompt for the coding agent; it is not a protected repository governance file. -- **Related:** [`GETTING_STARTED_NEW_REPO.md`](../GETTING_STARTED_NEW_REPO.md), [`OPTIONAL_CONFIGURATIONS.md`](../OPTIONAL_CONFIGURATIONS.md), [`AGENTS.md`](../AGENTS.md), [Repository Copilot Instructions](../.github/copilot-instructions.md) +- **Last Updated:** 2026-05-05 +- **Scope:** Historical task instructions used for the initial Phase 0 bootstrap work in `franklesniak/macOSLab`. This document is retained for context; current repository rules live in the related governance and planning documents. +- **Related:** [`README.md`](../README.md), [`AGENTS.md`](../AGENTS.md), [Repository Copilot Instructions](../.github/copilot-instructions.md), [Phase 0 PR description](phase-0-bootstrap-pr-description.md), [macOSLab repository specification](planning/macOS-imaging-08c-repo-spec-final.md), [macOSLab ADRs](planning/macOS-imaging-08e-ADRs.md) ## Purpose -Use this document as the source prompt when asking Codex to perform Phase 0 bootstrap work for `franklesniak/macOSLab`. It incorporates the owner updates made on 2026-05-04: +This document is retained as historical context for the initial Phase 0 bootstrap work. Do not treat it as current task authorization; current protected-file authorization is recorded in [Phase 0 Bootstrap PR Description Draft](phase-0-bootstrap-pr-description.md) and governed by [Repository Copilot Instructions](../.github/copilot-instructions.md). + +The original prompt incorporated the owner updates made on 2026-05-04: - Do not use `franklesniak@users.noreply.github.com` as a contact email. - For general conduct/contact guidance, instruct people to contact the repository owners using the contact links on their GitHub profiles. @@ -28,8 +30,8 @@ You are working in `franklesniak/macOSLab`, a repository created from `franklesn Your job is to perform Phase 0 (Bootstrap) by walking through, in order, every relevant step in: -1. [`GETTING_STARTED_NEW_REPO.md`](../GETTING_STARTED_NEW_REPO.md) -2. [`OPTIONAL_CONFIGURATIONS.md`](../OPTIONAL_CONFIGURATIONS.md) +1. The now-removed template guide `GETTING_STARTED_NEW_REPO.md`. +2. The now-removed template guide `OPTIONAL_CONFIGURATIONS.md`. Tailor the result to this repository's actual goals. Skip steps that require manual GitHub-side action, and explicitly note each skipped step in the PR description. @@ -77,7 +79,7 @@ There is no Python source code and no Terraform / HCL. Therefore: - Purge all Terraform / HCL content from the template: CI jobs, examples, `.gitignore` entries, pre-commit hooks, Terraform-named directories, README sections, PR-template sections, Dependabot ecosystems, and related docs. -- Exception: do not delete or edit `.github/instructions/terraform.instructions.md` or any other protected instruction file. Instead, list "remove or de-link `terraform.instructions.md`" as an Open Question / Requested Authorization in the PR description. +- Historical exception: the original Phase 0 task did not authorize deleting or editing `.github/instructions/terraform.instructions.md` or any other protected instruction file. That authorization was granted separately on 2026-05-05 and applied in the protected-instruction cleanup. - Purge Python sample/source content and Python-specific CI jobs per ADR-0008. - Keep pre-commit itself and pre-commit hooks that happen to be implemented in Python, including `check-jsonschema`, `check-json`, `check-yaml`, `yamllint`, and `actionlint`. These are development tooling, not project Python code. - If all Python source code is removed, delete the Black and Ruff hook entries from `.pre-commit-config.yaml`. diff --git a/docs/phase-0-bootstrap-pr-description.md b/docs/phase-0-bootstrap-pr-description.md index 1648e30..82b0f6e 100644 --- a/docs/phase-0-bootstrap-pr-description.md +++ b/docs/phase-0-bootstrap-pr-description.md @@ -6,7 +6,7 @@ - **Status:** Draft - **Owner:** Repository Maintainers - **Last Updated:** 2026-05-05 -- **Scope:** PR-description content for the Phase 0 bootstrap branch. This records applied steps, skipped steps, manual owner actions, validation, and requested authorization. +- **Scope:** PR-description content for the Phase 0 bootstrap branch. This records applied steps, skipped steps, manual owner actions, validation, and protected-instruction cleanup. - **Related:** [Phase 0 bootstrap instructions](phase-0-bootstrap-codex-instructions.md), [macOSLab repository specification](planning/macOS-imaging-08c-repo-spec-final.md), [macOSLab ADRs](planning/macOS-imaging-08e-ADRs.md) ## Summary @@ -22,6 +22,8 @@ This Phase 0 bootstrap customizes the template repository for `franklesniak/macO - Customized README, CONTRIBUTING, issue templates, PR template, CODEOWNERS, package metadata, VS Code title, pre-commit, Dependabot, workflows, and `.gitignore`. - Removed non-protected Python sample/source files and Python package metadata. - Removed non-protected HCL sample files, HCL workflow/configuration, and HCL-specific docs. +- Removed unused Python and Terraform modular instruction files after explicit owner authorization. +- Updated protected instruction entry points to match the PowerShell/Markdown repository shape after explicit owner authorization. - Kept Markdown linting, nested Markdown linting, data-file validation, pre-commit, PowerShell/Pester CI, PSScriptAnalyzer settings, issue templates, PR template, CODEOWNERS, and governance files. - Kept the worked-example schema and `check-jsonschema` / `check-metaschema` hooks for Phase 0 validation. - Added required root phase TODO files, including branch-protection and Phase 7 evidence-schema replacement tracking. @@ -40,7 +42,7 @@ This Phase 0 bootstrap customizes the template repository for `franklesniak/macO - README customization: applied as a minimal Phase 0 README. - CONTRIBUTING customization: applied for PowerShell/Markdown and retained tooling. - CODE_OF_CONDUCT customization: applied. -- Copilot/protected instruction updates: skipped because protected instruction files require explicit owner authorization. +- Copilot/protected instruction updates: applied after explicit owner authorization on 2026-05-05. - Validation/testing: run locally as noted in this PR after edits. - Cleanup: applied for non-protected template artifacts that do not match macOSLab. @@ -53,10 +55,10 @@ This Phase 0 bootstrap customizes the template repository for `franklesniak/macO - Pre-commit configuration: applied; Black, Ruff, and HCL hooks removed, retained data-file and Markdown hooks kept. - Schema validation configuration: applied; worked example retained and Phase 7 replacement TODO added. - Markdown linting/workflow: kept. -- Copilot instruction configuration: skipped because protected instruction files require explicit owner authorization. +- Copilot instruction configuration: applied after explicit owner authorization on 2026-05-05. - CI workflow configuration: applied; Python and HCL CI removed, narrow aggregate pre-commit workflow added for retained tooling hooks. - Auto-fix pre-commit workflow: kept and updated to match retained hook set. -- Placeholder check workflow: kept. +- Placeholder check workflow: removed after repository initialization. - PowerShell CI workflow: kept, PowerShell 7.4 floor check added, Pester pinned to `5.7.1`. - Python template files: removed. - Pester test template: kept. @@ -77,11 +79,8 @@ This Phase 0 bootstrap customizes the template repository for `franklesniak/macO - Set repository About description and topics. - Verify GitHub Actions are green after the branch is pushed. -## Open Questions / Requested Authorization +## Open Questions -- Authorization requested: remove or de-link `.github/instructions/terraform.instructions.md` and remove inherited HCL references from protected instruction entry points. -- Authorization requested: remove or de-link `.github/instructions/python.instructions.md` and remove inherited Python references from protected instruction entry points. -- Authorization requested: update `.github/copilot-instructions.md`, `AGENTS.md`, `CLAUDE.md`, and `GEMINI.md` so their language tables, validation commands, and summaries match PowerShell/Markdown macOSLab. - Owner decision requested: whether to add the exact ADR-0009 `SECURITY.md` paragraph in Phase 1. ## Validation