Skip to content

fix: replace non-ASCII chars in PowerShell scripts for PS 5.1 compat#2708

Closed
mnriem wants to merge 1 commit into
github:mainfrom
mnriem:fix/2680-ps51-unicode-encoding
Closed

fix: replace non-ASCII chars in PowerShell scripts for PS 5.1 compat#2708
mnriem wants to merge 1 commit into
github:mainfrom
mnriem:fix/2680-ps51-unicode-encoding

Conversation

@mnriem

@mnriem mnriem commented May 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes #2680

PowerShell 5.1 (built-in on Windows) defaults to the system's legacy encoding when reading .ps1 files without a BOM. Multi-byte UTF-8 sequences for characters like (U+2713) are misinterpreted as individual bytes, one of which gets parsed as a quote character, causing TerminatorExpectedAtEndOfString parse errors.

Changes

PowerShell script fixes (5 files)

  • Replace [OK] and [FAIL] in string literals (common.ps1, initialize-repo.ps1)
  • Replace (em-dash) → -- in comments across all affected .ps1 files

Test fixes (2 files)

  • tests/integrations/test_cli.py — apply strip_ansi() to capsys output before asserting against Rich-formatted text
  • tests/integrations/test_integration_subcommand.py — import and apply strip_ansi() to CLI output assertions affected by Rich ANSI escape codes

Regression tests (1 new file)

  • tests/test_ps1_encoding.py — 10 parametrized tests scanning every shipped .ps1 file byte-by-byte, asserting all content is ASCII-only (≤ 0x7F). Includes a discovery sanity check.

Testing

  • Full suite: 3018 passed, 1 skipped (pre-existing Windows-only test), 0 failed
  • Negative case verified: injecting back into common.ps1 causes immediate test failure

…ithub#2680)

PowerShell 5.1 (built-in on Windows) defaults to legacy encoding when
reading .ps1 files without a BOM. Multi-byte UTF-8 sequences for
characters like checkmarks and em-dashes are misinterpreted, causing
TerminatorExpectedAtEndOfString parse errors.

Changes:
- Replace Unicode symbols with ASCII equivalents in all .ps1 files
  (checkmark -> [OK], ballot-x -> [FAIL], em-dash -> --)
- Add regression tests that scan all shipped .ps1 files for non-ASCII bytes
- Fix ANSI escape code stripping in test assertions for Rich-formatted output
Copilot AI review requested due to automatic review settings May 26, 2026 18:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses PowerShell 5.1 script parsing failures on Windows caused by non-ASCII characters being mis-decoded when .ps1 files are read without a BOM, and adds regression coverage to prevent reintroduction.

Changes:

  • Replaced non-ASCII symbols (e.g., , , em-dash) in shipped PowerShell scripts to ensure ASCII-only content.
  • Updated CLI/integration tests to strip Rich ANSI escape codes before making string assertions.
  • Added a regression test suite that scans shipped .ps1 files and fails on any non-ASCII bytes.
Show a summary per file
File Description
tests/test_ps1_encoding.py Adds regression tests enforcing ASCII-only bytes in shipped PowerShell scripts.
tests/integrations/test_integration_subcommand.py Normalizes CLI output by stripping ANSI codes before parsing/asserting table and messages.
tests/integrations/test_cli.py Strips ANSI codes from captured output before assertions to avoid Rich formatting affecting tests.
scripts/powershell/create-new-feature.ps1 Replaces non-ASCII em-dash in a comment for PS 5.1-safe encoding.
scripts/powershell/common.ps1 Replaces Unicode status glyphs and em-dash usage to keep script content ASCII-only.
extensions/git/scripts/powershell/initialize-repo.ps1 Replaces Unicode checkmark output and em-dash in comment for PS 5.1 compatibility.
extensions/git/scripts/powershell/git-common.ps1 Replaces non-ASCII em-dash in comment to keep the file ASCII-only.
extensions/git/scripts/powershell/auto-commit.ps1 Replaces non-ASCII em-dash in comment to keep the file ASCII-only.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 8/8 changed files
  • Comments generated: 2

Comment thread tests/test_ps1_encoding.py
Comment thread extensions/git/scripts/powershell/initialize-repo.ps1

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 8/8 changed files
  • Comments generated: 1

Comment thread tests/test_ps1_encoding.py
@mnriem

mnriem commented May 26, 2026

Copy link
Copy Markdown
Collaborator Author

Closing as superseded by PR #2709 (c7e0cac), which merged the same fix to main with two additional improvements from review feedback:

  1. glob("*.ps1")rglob("*.ps1") for recursive script discovery in regression tests
  2. Updated speckit.git.initialize.md docs to reflect the [OK] output change

Issue #2680 has been closed as completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

powershell 7 is an undocumented requirement? initialize-repo.ps1 seems to have an encoding issue hit going through the initial "Get Started"

2 participants