Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 17 additions & 11 deletions Docs/Architecture/01-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

## Purpose

`swift-mutation-testing` is a mutation testing CLI for Xcode + XCTest projects. It introduces controlled faults (mutants) into source code, runs the test suite for each one, and reports whether the tests detected the fault. The mutation score — the ratio of killed mutants to all testable mutants — measures the effectiveness of the test suite.
`swift-mutation-testing` is a mutation testing CLI for Swift projects (Xcode and SPM). It introduces controlled faults (mutants) into source code, runs the test suite for each one, and reports whether the tests detected the fault. The mutation score — the ratio of killed mutants to all testable mutants — measures the effectiveness of the test suite.

The tool never modifies the original project. All mutations happen inside isolated sandbox copies in `$TMPDIR`.

Expand All @@ -19,9 +19,9 @@ graph TD
CLI["CLI\n(SwiftMutationTesting · CommandLineParser)"]
CONFIG["Configuration\n(ConfigurationResolver · ProjectDetector)"]
DISCOVERY["Discovery\n(DiscoveryPipeline · Operators · Schematization)"]
EXECUTION["Execution\n(MutantExecutor · BuildStage · TestExecutionStage)"]
EXECUTION["Execution\n(MutantExecutor · FallbackExecutor · IncompatibleMutantExecutor\nBuildStage · TestExecutionStage · TestResultResolver)"]
REPORTING["Reporting\n(TextReporter · JsonReporter · HtmlReporter · SonarReporter)"]
INFRA["Infrastructure\n(ProcessLauncher · XCTestRunPlist · TestFilesHasher)"]
INFRA["Infrastructure\n(ProcessRunner · ProcessRequest · SPMProcessLauncher\nXCTestRunPlist · TestFilesHasher)"]

CLI --> CONFIG
CLI --> DISCOVERY
Expand All @@ -37,9 +37,9 @@ graph TD
| **CLI** | Argument parsing, subcommand routing, exit codes |
| **Configuration** | Config file parsing, CLI merge, auto-detection of scheme and destination |
| **Discovery** | Source file collection, AST parsing, mutant identification, schematization |
| **Execution** | Sandbox creation, build, simulator management, parallel test execution, result parsing, caching |
| **Execution** | Sandbox creation, build, simulator management, parallel test execution, result parsing (Xcode and SPM), fallback per-file builds, caching |
| **Reporting** | Progress output, mutation report generation (text, JSON, HTML, Sonar) |
| **Infrastructure** | Process lifecycle management, xctestrun plist manipulation, test file hashing |
| **Infrastructure** | Process lifecycle management (`ProcessRunner`, `ProcessRequest`, `SPMProcessLauncher`), xctestrun plist manipulation, test file hashing |

## Entry Point

Expand Down Expand Up @@ -68,36 +68,42 @@ flowchart LR
subgraph Discovery
FD[FileDiscoveryStage] --> PS[ParsingStage]
PS --> MD[MutantDiscoveryStage]
MD --> SS[SchematizationStage]
MD --> MI[MutantIndexingStage]
MI --> SS[SchematizationStage]
MI --> IRS[IncompatibleRewritingStage]
end
subgraph Execution
SF[SandboxFactory] --> BS[BuildStage]
BS --> TES[TestExecutionStage]
BS -- build failed --> FBP[Per-file fallback]
TES --> RP[ResultParser]
BS -- build failed --> FBP[FallbackExecutor\nper-file rebuild]
TES --> TR[TestResultResolver]
IME[IncompatibleMutantExecutor]
end
SS -- RunnerInput --> SF
SS -- incompatible mutants --> IME
IRS -- incompatible mutants --> IME
```

| Stage | Input | Output |
|---|---|---|
| `FileDiscoveryStage` | `DiscoveryInput` | `[SourceFile]` |
| `ParsingStage` | `[SourceFile]` | `[ParsedSource]` |
| `MutantDiscoveryStage` | `[ParsedSource]` | `[MutationPoint]` |
| `SchematizationStage` | `[MutationPoint]`, `[ParsedSource]` | `RunnerInput` |
| `MutantIndexingStage` | `[MutationPoint]`, `[ParsedSource]` | `[IndexedMutationPoint]` |
| `SchematizationStage` | `[IndexedMutationPoint]`, `[ParsedSource]` | `[SchematizedFile]`, `[MutantDescriptor]` |
| `IncompatibleRewritingStage` | `[IndexedMutationPoint]`, `[ParsedSource]` | `[MutantDescriptor]` |
| `SandboxFactory` | project path + schematized files | `Sandbox` |
| `BuildStage` | `Sandbox` | `BuildArtifact` |
| `TestExecutionStage` | `BuildArtifact` + mutants | `[ExecutionResult]` |
| `FallbackExecutor` | `RunnerInput` + `SimulatorPool` | `[ExecutionResult]` |
| `IncompatibleMutantExecutor` | incompatible mutants | `[ExecutionResult]` |
| `TestResultResolver` | `TestLaunchResult` + `ProjectType` | `TestRunOutcome` |

## Invariants

| Invariant | Enforcement |
|---|---|
| Original project is never modified | All mutations happen inside `$TMPDIR/xmr-<UUID>/` sandbox |
| `xcodebuild build-for-testing` runs exactly once | `BuildStage` builds once; `TestExecutionStage` uses `test-without-building` |
| Build runs exactly once for the normal path | `BuildStage` builds once (Xcode: `build-for-testing`, SPM: `swift build --build-tests`); `TestExecutionStage` uses `test-without-building` (Xcode) or `swift test --skip-build` (SPM) |
| No mutant results are lost or duplicated | `MutationCounter` tracks total; `withThrowingTaskGroup` accounts for every task |
| Mutant positions are accurate | UTF-8 offsets are preserved from AST through to final report |
| A cancelled task never permanently holds a simulator slot | `withTaskCancellationHandler` in `SimulatorPool.acquire` releases the slot on cancel |
Expand Down
65 changes: 55 additions & 10 deletions Docs/Architecture/02-discovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,18 @@

## Design

The discovery pipeline is a **linear chain of pure stages**. Each stage receives an immutable input, produces an immutable output, and has no side effects. `DiscoveryPipeline` is the entry point and orchestrates the four stages sequentially.
The discovery pipeline is a **linear chain of pure stages**. Each stage receives an immutable input, produces an immutable output, and has no side effects. `DiscoveryPipeline` is the entry point and orchestrates the six stages sequentially.

```mermaid
flowchart TD
IN[DiscoveryInput] --> FD[FileDiscoveryStage]
FD --> PA[ParsingStage]
PA --> MD[MutantDiscoveryStage]
MD --> SC[SchematizationStage]
MD --> MI[MutantIndexingStage]
MI --> SC[SchematizationStage]
MI --> IR[IncompatibleRewritingStage]
SC --> OUT[RunnerInput]
IR --> OUT
```

## Stages
Expand Down Expand Up @@ -52,16 +55,38 @@ Applies mutation operators to each parsed source and collects mutation points. R

Each operator walks the AST with its own visitor and emits a `MutationPoint` for every applicable node. Points are collected from all operators and all files, then returned as a flat list.

### SchematizationStage
### MutantIndexingStage

Transforms mutation points into the final `RunnerInput` consumed by the execution pipeline.
Assigns unique sequential IDs to each mutation point and classifies them as schematizable or incompatible.

| | |
|---|---|
| Input | `[MutationPoint]`, `[ParsedSource]` |
| Output | `RunnerInput` — schematized files, incompatible mutants, support file content |
| Output | `[IndexedMutationPoint]` — mutation point + unique ID + schematizable flag |

Each mutation point receives an ID in the format `swift-mutation-testing_<index>`, where `<index>` is a zero-based global counter. `TypeScopeVisitor` determines whether a mutation falls inside a function body (schematizable) or outside (incompatible). The indexed points are consumed by the next two stages.

### SchematizationStage

For each file, the stage separates schematizable mutations (inside function bodies) from incompatible ones (outside function bodies). Schematizable mutations are embedded into the source via `SchemataGenerator`. Incompatible mutations are stored as full file rewrites via `MutationRewriter`. See [Schematization](05-schematization.md) for a detailed breakdown.
Embeds all schematizable mutations into the source files via `SchemataGenerator`, producing `SchematizedFile` values and `MutantDescriptor` values for the execution pipeline.

| | |
|---|---|
| Input | `[IndexedMutationPoint]`, `[ParsedSource]` |
| Output | `[SchematizedFile]`, `[MutantDescriptor]` — schematized files and schematizable mutant descriptors |

For each file, the stage processes only the schematizable indexed points. Mutations are embedded into the source via `SchemataGenerator`, which rewrites function bodies to contain `switch __swiftMutationTestingID` blocks. See [Schematization](05-schematization.md) for a detailed breakdown.

### IncompatibleRewritingStage

Produces full-file rewrites for mutants that cannot be schematized (mutations outside function bodies, such as stored property initializers or global-scope expressions).

| | |
|---|---|
| Input | `[IndexedMutationPoint]`, `[ParsedSource]` |
| Output | `[MutantDescriptor]` — incompatible mutant descriptors with pre-computed `mutatedSourceContent` |

Each incompatible mutation point is applied to the source via `MutationRewriter`, producing a complete replacement source file stored in `MutantDescriptor.mutatedSourceContent`. These mutants are executed later by `IncompatibleMutantExecutor`, each requiring a separate build + test cycle.

## Mutation Operators

Expand All @@ -87,11 +112,12 @@ Mutations can be suppressed on a per-scope basis using the inline annotation `//

```
DiscoveryInput
├── projectPath — Xcode project root
├── projectPath — project root (Xcode or SPM)
├── projectType — ProjectType (.xcode or .spm)
├── sourcesPath — root for Swift file discovery
├── excludePatterns — glob patterns to skip
├── operators — list of active operator identifiers
└── scheme, destination, timeout, concurrency, noCache
└── timeout, concurrency, noCache

SourceFile
├── path — absolute path to the .swift file
Expand All @@ -108,11 +134,30 @@ MutationPoint
├── originalText — token(s) before mutation
├── mutatedText — token(s) after mutation
├── operatorIdentifier
└── replacement — ReplacementKind enum
└── replacementKind — ReplacementKind enum

IndexedMutationPoint
├── point — MutationPoint
├── id — unique ID (swift-mutation-testing_<index>)
└── isSchematizable — whether the mutation is inside a function body

MutantDescriptor
├── id — unique ID
├── filePath — absolute source file path
├── line, column — 1-based position
├── utf8Offset — byte offset
├── originalText — token(s) before mutation
├── mutatedText — token(s) after mutation
├── operatorIdentifier
├── replacementKind — ReplacementKind enum
├── description — human-readable mutation description
├── isSchematizable — schematizable or incompatible
└── mutatedSourceContent — pre-computed full source (incompatible only)

RunnerInput
├── projectPath
├── scheme, destination, timeout, concurrency, noCache
├── projectType — ProjectType (.xcode or .spm)
├── timeout, concurrency, noCache
├── schematizedFiles — [SchematizedFile] (one per modified source file)
├── supportFileContent — __swiftMutationTestingID global declaration
└── mutants — [MutantDescriptor] (all mutants, schematizable and incompatible)
Expand Down
Loading