Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ on:
pull_request:
branches: [main]
types: [opened, reopened, ready_for_review, synchronize]
merge_group: # Merge-queue runs: validate the queued merge result so two
# PRs that are each green against an older main can't land a
# semantic conflict together.
push:
branches: [main] # PRs are covered by pull_request (incl. synchronize);
# scoping push to main avoids double-running every PR commit.
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ name: CodeQL
on:
pull_request:
branches: [main]
merge_group: # Merge-queue runs, so a required CodeQL check doesn't
# block queued merges (mirrors ci.yml).
push:
branches: [main] # PRs are covered by pull_request; scoping push to main
# avoids double-running every PR commit (mirrors ci.yml).
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ uv run pytest -m install # installs each init template's requirements fo

`check.sh` runs the default suite with a **90% branch-coverage gate** (`--cov-fail-under=90`). New code generally needs tests to clear that gate.

CLI output is pinned by **syrupy snapshot tests** (`tests/__snapshots__/*.ambr`). Changing help text, tables, or rendered output will fail those tests until you regenerate them with `uv run pytest --snapshot-update` and commit the updated `.ambr` files. The auto-format hook only touches `*.py`, and pre-commit's whitespace fixers deliberately skip `tests/__snapshots__/` (syrupy's indentation must stay byte-for-byte), so never hand-edit a snapshot — always regenerate.
CLI output is pinned by **syrupy snapshot tests** (`tests/__snapshots__/*.ambr`). Changing help text, tables, or rendered output will fail those tests until you regenerate them with `uv run pytest --snapshot-update` and commit the updated `.ambr` files. The auto-format hook only touches `*.py`, and pre-commit's whitespace fixers deliberately skip `tests/__snapshots__/` (syrupy's indentation must stay byte-for-byte), so never hand-edit a snapshot — always regenerate. The `--help` goldens are split per command group (`tests/test_snapshots_help_<group>.py`) so concurrent branches touching different commands regenerate *different* `.ambr` files; a new top-level command must be added to `HELP_GROUPS` in `tests/_snapshot_surface.py` (the partition guard in `tests/test_snapshots_help_groups.py` fails until it is).

The post-edit hook (`.claude/settings.json`) runs `ruff check --fix --unfixable F401` + `ruff format` on every edited `*.py`. `--unfixable F401` means a just-added import is **not** auto-deleted while it's momentarily unused — so adding an import in one edit and its usage in the next is safe. The flip side: a genuinely unused import survives the hook and only fails at `ruff check` in the gate, so still prefer making the import and its first usage land in the same edit.

Expand Down
3 changes: 2 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -276,4 +276,5 @@ DEP002 = ["fastapi", "python-dotenv", "python-multipart", "uvicorn"]
# coverage is read by scripts/mutation_gate.py and click by
# scripts/generated_code_compile_gate.py (dev-only gates run from check.sh, never
# shipped in the wheel), so deptry sees dev deps imported from non-test code.
DEP004 = ["fastapi", "httpx", "hypothesis", "pytest", "coverage", "click"]
# syrupy is imported by tests/_snapshot_surface.py (typing the snapshot fixture).
DEP004 = ["fastapi", "httpx", "hypothesis", "pytest", "coverage", "click", "syrupy"]
24 changes: 16 additions & 8 deletions scripts/check.sh
Original file line number Diff line number Diff line change
Expand Up @@ -209,10 +209,18 @@ fi
echo "==> no new static-analysis escape hatches"
# Existing escape hatches are tolerated for now; new ones must be refactored away or
# justified by changing this gate deliberately. Broad noqa/type-ignore/no-cover are
# checked by added diff lines. `Any` and `cast(` are count-gated against origin/main
# so mechanical edits to existing uses don't fail, but net-new uses do.
# checked by added diff lines. `Any` and `cast(` are count-gated against the
# merge-base with origin/main so mechanical edits to existing uses don't fail, but
# net-new uses do.
if git rev-parse --verify --quiet origin/main >/dev/null; then
escape_hatches="$(git diff -U0 origin/main -- aai_cli tests \
# Diff and count against the MERGE-BASE, not the origin/main tip (which the
# mutation gate and diff-cover already do). With many concurrent branches,
# main moves constantly: a tip-based baseline makes this gate fail on a branch
# the moment an unrelated merge lowers the count (e.g. removes an `Any`), even
# though the branch itself added nothing. The merge-base only moves when the
# branch itself rebases.
gate_base="$(git merge-base origin/main HEAD || echo origin/main)"
escape_hatches="$(git diff -U0 "$gate_base" -- aai_cli tests \
| rg '^\+.*(# type: ignore|# noqa|pragma: no cover)' || true)"
if [[ -n "$escape_hatches" ]]; then
printf '%s\n' "$escape_hatches"
Expand All @@ -226,25 +234,25 @@ if git rev-parse --verify --quiet origin/main >/dev/null; then
# env-gated marker suites (e2e/install) and live on origin/main, so they aren't
# added diff lines and don't trip this; a genuinely-needed new one must update this
# gate deliberately. Scoped to tests/ — production sleeps are fine.
test_shortcuts="$(git diff -U0 origin/main -- tests \
test_shortcuts="$(git diff -U0 "$gate_base" -- tests \
| rg '^\+.*(pytest\.skip\(|pytest\.xfail\(|@pytest\.mark\.(skip|xfail)|\btime\.sleep\()' || true)"
if [[ -n "$test_shortcuts" ]]; then
printf '%s\n' "$test_shortcuts"
echo "New test skip/xfail/time.sleep found; fix the test (or sync properly) or update the gate explicitly."
exit 1
fi

base_any_count="$({ git grep -n "Any" origin/main -- aai_cli tests || true; } | wc -l | tr -d '[:space:]')"
base_any_count="$({ git grep -n "Any" "$gate_base" -- aai_cli tests || true; } | wc -l | tr -d '[:space:]')"
work_any_count="$({ rg -n "Any" aai_cli tests || true; } | wc -l | tr -d '[:space:]')"
if (( work_any_count > base_any_count )); then
echo "New Any usage found: ${work_any_count} current vs ${base_any_count} on origin/main."
echo "New Any usage found: ${work_any_count} current vs ${base_any_count} at the merge-base with origin/main."
exit 1
fi

base_cast_count="$({ git grep -n "cast(" origin/main -- aai_cli tests || true; } | wc -l | tr -d '[:space:]')"
base_cast_count="$({ git grep -n "cast(" "$gate_base" -- aai_cli tests || true; } | wc -l | tr -d '[:space:]')"
work_cast_count="$({ rg -n "cast\\(" aai_cli tests || true; } | wc -l | tr -d '[:space:]')"
if (( work_cast_count > base_cast_count )); then
echo "New cast() usage found: ${work_cast_count} current vs ${base_cast_count} on origin/main."
echo "New cast() usage found: ${work_cast_count} current vs ${base_cast_count} at the merge-base with origin/main."
exit 1
fi
else
Expand Down
65 changes: 65 additions & 0 deletions tests/__snapshots__/test_snapshots_errors.ambr
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# serializer version: 1
# name: test_error_human_render_matches_snapshot[api_no_suggestion]
'''
Error: Transcription request failed: boom.

'''
# ---
# name: test_error_human_render_matches_snapshot[not_authenticated]
'''
Error: You're not signed in.
Suggestion: Run 'assembly onboard' for guided setup, 'assembly login' if you
have an account, or set ASSEMBLYAI_API_KEY.

'''
# ---
# name: test_error_human_render_matches_snapshot[plain_no_suggestion]
'''
Error: Something went wrong.

'''
# ---
# name: test_error_human_render_matches_snapshot[rejected_key]
'''
Error: Your API key was rejected.
Suggestion: Run 'assembly login' with a valid key, or set ASSEMBLYAI_API_KEY.

'''
# ---
# name: test_error_human_render_matches_snapshot[usage_with_suggestion]
'''
Error: Unknown voice 'nope'.
Suggestion: Run 'assembly agent --list-voices' to see the options.

'''
# ---
# name: test_error_json_render_matches_snapshot[api_no_suggestion]
'''
{"error": {"type": "api_error", "message": "Transcription request failed: boom."}}

'''
# ---
# name: test_error_json_render_matches_snapshot[not_authenticated]
'''
{"error": {"type": "not_authenticated", "message": "You're not signed in.", "suggestion": "Run 'assembly onboard' for guided setup, 'assembly login' if you have an account, or set ASSEMBLYAI_API_KEY."}}

'''
# ---
# name: test_error_json_render_matches_snapshot[plain_no_suggestion]
'''
{"error": {"type": "error", "message": "Something went wrong."}}

'''
# ---
# name: test_error_json_render_matches_snapshot[rejected_key]
'''
{"error": {"type": "not_authenticated", "message": "Your API key was rejected.", "suggestion": "Run 'assembly login' with a valid key, or set ASSEMBLYAI_API_KEY."}}

'''
# ---
# name: test_error_json_render_matches_snapshot[usage_with_suggestion]
'''
{"error": {"type": "usage_error", "message": "Unknown voice 'nope'.", "suggestion": "Run 'assembly agent --list-voices' to see the options."}}

'''
# ---
Loading
Loading