Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ jobs:
node dist/cli/index.js scan examples/rails --pack rails --format json --output debtlens-rails-report.json
node dist/cli/index.js scan examples/compose --pack compose --format json --output debtlens-compose-report.json
node dist/cli/index.js scan examples/swiftui --pack swiftui --format json --output debtlens-swiftui-report.json
node dist/cli/index.js scan examples/ai-workflow --pack ai-workflow-drift --format json --output debtlens-ai-workflow-report.json
- name: Benchmark regression gate
run: npm run benchmark:ci
env:
Expand All @@ -55,3 +56,4 @@ jobs:
- run: node dist/cli/index.js scan examples/rails --pack rails --min-severity info --format markdown --output debtlens-rails-report.md
- run: node dist/cli/index.js scan examples/compose --pack compose --min-severity info --format markdown --output debtlens-compose-report.md
- run: node dist/cli/index.js scan examples/swiftui --pack swiftui --min-severity info --format markdown --output debtlens-swiftui-report.md
- run: node dist/cli/index.js scan examples/ai-workflow --pack ai-workflow-drift --min-severity info --format markdown --output debtlens-ai-workflow-report.md
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ All notable changes to DebtLens are documented here. This project adheres to
`ruby-dead-abstraction`, and `ruby-todo-comment` ([#198](https://github.com/ColumbusLabs/DebtLens/issues/198)).
- **Rails framework pack** (`rails`) combining Ruby core rules with `rails-route-sprawl`
and `rails-controller-sprawl` ([#198](https://github.com/ColumbusLabs/DebtLens/issues/198)).
- **`ai-workflow-drift` pack** with `ai-instruction-duplication` and
`ai-instruction-contradiction` for repository-local assistant instruction files.
The pack does not detect AI-authored code ([#214](https://github.com/ColumbusLabs/DebtLens/issues/214)).

### Changed

Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -477,6 +477,7 @@ also declare their discovery metadata, so selecting `python`, `vue`, `svelte`, `
| `expo` | React Native tuning for Expo Router projects |
| `ai-assisted-maintainer` | high-signal maintainability checks for assistant-heavy codebases; no authorship claims |
| `oss-maintainer` | library API surface, barrels, duplication, tests, and TODO debt |
| `ai-workflow-drift` | duplicated or contradictory AI assistant instruction files; no AI-authorship claims |

```json
{
Expand Down Expand Up @@ -932,7 +933,7 @@ and PR comment upsert, and `--diff-base` branch comparisons.

The architecture stays intentionally simple: a language-agnostic scan and reporting
layer with pluggable rule packs on top. Current shipped packs cover core TS/JS, React,
React Native, Next.js, Expo, Node, Python, Python web, Vue/Svelte SFC scripts, Kotlin, Swift, Ruby/Rails, Jetpack Compose, SwiftUI, and maintainer workflows. Additional
React Native, Next.js, Expo, Node, Python, Python web, Vue/Svelte SFC scripts, Kotlin, Swift, Ruby/Rails, Jetpack Compose, SwiftUI, maintainer workflows, and AI workflow instruction drift. Additional
packs expand from the same scan/reporting contract. See [`ROADMAP.md`](./ROADMAP.md) and
[`docs/rule-packs.md`](./docs/rule-packs.md).

Expand Down
2 changes: 1 addition & 1 deletion action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ inputs:
description: Compare against a git ref and report only introduced findings.
default: ""
pack:
description: Built-in rule pack preset (core, react, react-native, next, expo, node, python, python-web, vue, svelte, kotlin, swift, ruby, rails, compose, swiftui, ai-assisted-maintainer, oss-maintainer).
description: Built-in rule pack preset (core, react, react-native, next, expo, node, python, python-web, vue, svelte, kotlin, swift, ruby, rails, compose, swiftui, ai-assisted-maintainer, oss-maintainer, ai-workflow-drift).
default: ""
package:
description: Workspace package name to scan.
Expand Down
74 changes: 74 additions & 0 deletions docs/ai-workflow-drift-rfc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# AI Workflow Instruction Drift RFC

Status: **Prototype shipped (`ai-workflow-drift` pack)**

DebtLens teams increasingly maintain parallel instruction files for coding assistants:
`AGENTS.md`, `CLAUDE.md`, `.cursor/rules/**`, and `.github/copilot-instructions.md`.
Those files drift apart quickly, producing contradictory guidance for humans and agents.

This pack adds conservative, local checks for duplicated and conflicting instruction text.
It is a maintainability scanner extension, not an authorship detector.

## Target files

The pack scans repository-local instruction surfaces only:

| Path pattern | Purpose |
| --- | --- |
| `AGENTS.md` | Repository agent instructions (Cursor, Codex, and similar tools) |
| `CLAUDE.md` | Claude Code / Claude project instructions |
| `.github/copilot-instructions.md` | GitHub Copilot repository instructions |
| `.cursor/rules/**` (`*.md`, `*.mdc`) | Cursor rule files |

Detectors resolve files from the scan `context.files` list and, when needed, walk the
scan target on disk (similar to `config-drift` JSON discovery).

## Non-goals

- **Does not detect AI-authored code.** The pack inspects instruction markdown only.
- **No external telemetry.** Analysis stays on the local filesystem during `debtlens scan`.
- **No semantic policy enforcement.** The MVP uses normalized text duplication and a small
set of conservative contradiction patterns (for example, "always run tests" vs "skip tests").
- **No secret scanning.** Instruction files may contain credentials; this pack does not
upload, index, or phone home file contents.

## Prototype rules

| Rule id | Signal |
| --- | --- |
| `ai-instruction-duplication` | The same normalized instruction block appears in two or more target files |
| `ai-instruction-contradiction` | Conservative opposing directives appear across instruction files |

## File discovery model

The `ai-workflow-drift` pack declares `includeGlobs` for the instruction paths above.
`mergeConfig()` unions those globs into scan discovery when the pack is selected, while
detectors still filter `context.files` by instruction path patterns inside `detect()`.

Example:

```bash
debtlens scan examples/ai-workflow --pack ai-workflow-drift
```

## Privacy and security considerations

- **Local-only analysis:** DebtLens reads files from the scan target and optional git
staged overrides; it does not transmit instruction text to third-party services.
- **Sensitive content risk:** Instruction files sometimes embed API keys, internal URLs,
or customer names. Treat scan artifacts (JSON, SARIF, Markdown reports) like source
code output and restrict CI artifact retention when needed.
- **Conservative contradictions:** The contradiction rule intentionally under-reports to
avoid blocking legitimate tool-specific nuance. Review findings; do not treat them as
policy violations without human confirmation.
- **No provenance claims:** Findings describe instruction drift, not whether code was
written by an assistant.

## Future direction

- Optional `includeGlobs` overrides per repository config
- Richer contradiction templates with configurable vocabulary
- Cross-link suggestions to a canonical `AGENTS.md`
- Pack composition with `ai-assisted-maintainer` for code maintainability plus instruction drift

Track implementation in [#214](https://github.com/ColumbusLabs/DebtLens/issues/214).
2 changes: 1 addition & 1 deletion docs/example-report.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# DebtLens Report

Scanned **3** files with **26** rules in **174ms**.
Scanned **3** files with **28** rules in **174ms**.

## Summary

Expand Down
1 change: 1 addition & 0 deletions docs/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ these commands when evaluating output, writing docs, or checking a reporter chan
| SwiftUI screen | `debtlens scan examples/swiftui --pack swiftui --min-severity info` | SwiftUI large-view and state-sprawl rules. |
| Ruby service | `debtlens scan examples/ruby --pack ruby --min-severity info` | Ruby duplicate, large-method, thin-wrapper, and TODO rules. |
| Rails app | `debtlens scan examples/rails --pack rails --min-severity info` | Rails route/controller sprawl plus Ruby core rules. |
| AI workflow instructions | `debtlens scan examples/ai-workflow --pack ai-workflow-drift --min-severity info` | Duplicated and contradictory assistant instruction files. |
| Jetpack Compose screen | `debtlens scan examples/compose --pack compose --min-severity info` | Compose large-composable and state-hoisting rules. |
| Local plugin | `debtlens scan examples/plugin --config examples/plugin/debtlens.config.json --min-severity info` | Trusted local plugin loading and plugin rule output. |
| False-positive playground | `debtlens scan examples/false-positives --pack react --min-severity info` | Calibrated near-misses that should stay quiet. |
Expand Down
4 changes: 2 additions & 2 deletions docs/pack-chooser.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,6 @@ Action behavior are language-neutral; only file discovery and detectors are pack
| Mixed TS/Python/SFC/Kotlin/Swift monorepo | `core,python,vue,svelte,kotlin,swift` | `debtlens scan . --pack core,python,vue,svelte,kotlin,swift --format json` | Use package or path-scoped baselines; add `compose` or `swiftui` for UI modules. |
| Open-source library | `oss-maintainer` | `debtlens scan . --pack oss-maintainer --min-severity medium` | Prefer reports and issues before hard CI gates. |
| Ruby service or Rails app | `ruby` / `rails` | `debtlens scan . --pack rails --min-severity low` | Advisory first; review route and controller ownership before gating. |
| Agent-heavy repo with instruction files | `ai-workflow-drift` | `debtlens scan . --pack ai-workflow-drift --min-severity medium` | Review duplicated/contradictory guidance; not authorship detection. |

Future packs such as AI workflow instruction drift should reuse this same chooser shape
once their MVPs land.
Future packs should reuse this same chooser shape once their MVPs land.
3 changes: 3 additions & 0 deletions docs/rule-packs.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ For a user-facing selection table, see [`pack-chooser.md`](./pack-chooser.md).
| `ruby-todo-comment` | **ruby** | TODO/FIXME/HACK/temporary implementation comments in Ruby files | Low |
| `rails-route-sprawl` | **rails** | Rails `routes.rb` modules registering too many routes | Medium |
| `rails-controller-sprawl` | **rails** | Rails controllers with too many public actions | Medium |
| `ai-instruction-duplication` | **ai-workflow-drift** | The same normalized instruction block repeated across assistant instruction files | Medium |
| `ai-instruction-contradiction` | **ai-workflow-drift** | Conservative opposing directives across assistant instruction files | High |
| `compose-large-composable` | **compose** | Oversized or branch-heavy Jetpack Compose functions | Medium |
| `compose-state-hoisting` | **compose** | Composables that own many local state holders instead of hoisting state | Medium |

Expand Down Expand Up @@ -234,6 +236,7 @@ and Compose UI debt.
| `swiftui` | SwiftUI oversized views and local state sprawl | **Shipped** |
| `ruby` | Ruby duplicate methods, large methods, thin wrappers, and TODO debt | **Shipped** |
| `rails` | Ruby core rules plus Rails route and controller sprawl | **Shipped** |
| `ai-workflow-drift` | Duplicated or contradictory AI assistant instruction files | **Shipped** |
| `compose` | Jetpack Compose oversized composables and state-hoisting smells | **Shipped** |
| `expo` | Expo Router and RN app shell boundaries | **Shipped** (React Native tuning plus barrel tolerance) |
| `ai-assisted-maintainer` | Maintainability signals common in assistant-heavy codebases | **Shipped** |
Expand Down
38 changes: 38 additions & 0 deletions docs/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -1028,3 +1028,41 @@ When this is a false positive:
- the competing names belong to separate concepts rather than one overloaded domain term

Confidence: **0.62**. Co-occurring domain synonyms are often legitimate vocabulary, so this rule stays advisory.

## `ai-instruction-duplication`

Flags the same normalized instruction block repeated across assistant instruction files such as `AGENTS.md`, `CLAUDE.md`, `.cursor/rules/**`, and `.github/copilot-instructions.md`.

Why it matters: duplicated guidance creates maintenance overhead and makes it unclear which file is canonical when assistants load multiple instruction sources.

Good fixes:

- keep one canonical instruction file and link to it from tool-specific files
- tailor each file to tool-specific context instead of copy-pasting shared blocks

When this is a false positive:

- the repeated text is intentionally mirrored while one file is being migrated
- short boilerplate headers are duplicated but substantive guidance differs elsewhere in the file

Confidence: **0.82**. Normalized text equality is direct evidence, but some duplication may be deliberate during transitions.

## `ai-instruction-contradiction`

Flags conservative opposing directives across assistant instruction files, such as "always run tests" versus "skip tests".

Why it matters: contradictory assistant guidance produces inconsistent edits, failed CI expectations, and review churn.

Good fixes:

- reconcile policies into one canonical instruction
- scope exceptions explicitly ("skip tests only for docs-only edits") in the same section as the base rule

When this is a false positive:

- the policies apply to different contexts and are not actually opposing
- one file is deprecated and scheduled for removal

Confidence: **0.80**. Pattern matching is intentionally conservative and may miss nuanced conflicts or over-flag during migrations.

This pack does **not** detect AI-authored code. It inspects instruction markdown only and performs local analysis without external telemetry.
7 changes: 7 additions & 0 deletions examples/ai-workflow/.cursor/rules/testing.mdc
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
description: Testing policy for documentation-only edits
---

# Testing policy

Skip tests when making documentation-only changes.
5 changes: 5 additions & 0 deletions examples/ai-workflow/.github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Copilot instructions

## Formatting

Always run prettier on changed files before opening a pull request.
9 changes: 9 additions & 0 deletions examples/ai-workflow/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Agent instructions

## Testing

Always run the full test suite before committing changes.

## Style

Use TypeScript strict mode for all new files.
9 changes: 9 additions & 0 deletions examples/ai-workflow/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Claude project instructions

## Testing

Always run the full test suite before committing changes.

## Documentation

Update README examples when public APIs change.
14 changes: 14 additions & 0 deletions examples/ai-workflow/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# AI workflow drift examples

This fixture demonstrates duplicated and contradictory AI assistant instruction files.

Run:

```bash
debtlens scan examples/ai-workflow --pack ai-workflow-drift
```

Expected signals:

- `ai-instruction-duplication` for the repeated testing block in `AGENTS.md` and `CLAUDE.md`
- `ai-instruction-contradiction` for "always run tests" vs "skip tests"
7 changes: 7 additions & 0 deletions examples/false-positives/ai-workflow/.cursor/rules/review.mdc
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
description: Review checklist for assistant edits
---

# Review checklist

Request human review before merging risky authentication changes.
9 changes: 9 additions & 0 deletions examples/false-positives/ai-workflow/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Agent instructions

## Unit testing

Run unit tests for all application code changes.

## Type safety

Enable TypeScript strict mode in new modules.
9 changes: 9 additions & 0 deletions examples/false-positives/ai-workflow/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Claude project instructions

## Integration testing

Run integration tests when API contracts change.

## Documentation

Keep public README examples aligned with exported APIs.
35 changes: 32 additions & 3 deletions schema/debtlens.config.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,13 @@
"rails",
"compose",
"ai-assisted-maintainer",
"oss-maintainer"
"oss-maintainer",
"ai-workflow-drift"
]
},
{
"type": "string",
"pattern": "^\\s*(?:core|react|react-native|next|expo|node|python|python-web|vue|svelte|kotlin|swift|swiftui|ruby|rails|compose|ai-assisted-maintainer|oss-maintainer)(?:\\s*,\\s*(?:core|react|react-native|next|expo|node|python|python-web|vue|svelte|kotlin|swift|swiftui|ruby|rails|compose|ai-assisted-maintainer|oss-maintainer))*\\s*$"
"pattern": "^\\s*(?:core|react|react-native|next|expo|node|python|python-web|vue|svelte|kotlin|swift|swiftui|ruby|rails|compose|ai-assisted-maintainer|oss-maintainer|ai-workflow-drift)(?:\\s*,\\s*(?:core|react|react-native|next|expo|node|python|python-web|vue|svelte|kotlin|swift|swiftui|ruby|rails|compose|ai-assisted-maintainer|oss-maintainer|ai-workflow-drift))*\\s*$"
}
],
"description": "Built-in rule pack preset, or a comma-separated list of presets. Explicit rules override the pack."
Expand Down Expand Up @@ -125,7 +126,9 @@
"rails-route-sprawl",
"rails-controller-sprawl",
"compose-large-composable",
"compose-state-hoisting"
"compose-state-hoisting",
"ai-instruction-duplication",
"ai-instruction-contradiction"
]
},
{
Expand Down Expand Up @@ -801,6 +804,22 @@
"medium",
"high"
]
},
"ai-instruction-duplication": {
"enum": [
"info",
"low",
"medium",
"high"
]
},
"ai-instruction-contradiction": {
"enum": [
"info",
"low",
"medium",
"high"
]
}
},
"additionalProperties": {
Expand Down Expand Up @@ -1095,6 +1114,16 @@
"type": "number",
"minimum": 0,
"maximum": 1
},
"ai-instruction-duplication": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"ai-instruction-contradiction": {
"type": "number",
"minimum": 0,
"maximum": 1
}
},
"additionalProperties": {
Expand Down
3 changes: 3 additions & 0 deletions src/config/mergeConfig.ts
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,12 @@ function resolveIncludeGlobs(
if (cliOptions.include?.length) return cliOptions.include;
const base = resolveBaseIncludeGlobs(fileConfig, languageDiscovery);
const discoveryLanguages = languageDiscovery.languages.filter((language) => language !== DEFAULT_SOURCE_LANGUAGE);
const packIncludeGlobs = parsePackIds(cliOptions.pack ?? fileConfig.pack)
.flatMap((packId) => getRulePack(packId).includeGlobs ?? []);
return unique([
...base,
...includeGlobsForLanguages(discoveryLanguages),
...packIncludeGlobs,
]);
}

Expand Down
Loading