Skip to content

Diff viewer for test outputs — compare results between executions #17

@ThomasMalletCodra

Description

@ThomasMalletCodra

Description

Add a diff viewer that compares the output of a test between two executions (e.g., current vs. previous, or between two selected campaigns). This helps developers quickly identify what changed when a previously passing test starts failing — especially useful for regression debugging.

Current behavior

  • Only the latest test output is displayed.
  • No way to compare outputs between runs.
  • Debugging regressions requires manually saving and diffing outputs externally.

Expected behavior

  • A "Compare" action is available for any test that has been run at least twice.
  • Side-by-side or inline diff view highlighting added/removed/changed lines.
  • Color-coded diff: green for additions, red for deletions, yellow for changes.
  • Option to compare against:
    • Previous execution of the same test
    • A specific campaign (if campaign management is available)
    • A saved "baseline" output
  • Diff summary: "42 lines changed, 3 added, 5 removed".

Implementation ideas

  1. Store the last N outputs per test (configurable, default: 5) in .moduletester/history/.
  2. Use Python's difflib for generating unified or side-by-side diffs.
  3. Render diff in a QTextEdit or QWebEngineView with syntax highlighting.
  4. Add "Compare with previous" context menu action on test items.
  5. Optional: integrate with campaign management (Campaign management/Non regression — execution history and release comparison #9 ) for cross-campaign comparison.

Dependencies

Acceptance criteria

  • Last N test outputs are stored per test
  • Side-by-side or inline diff view is available
  • Diff highlights additions, deletions, and changes
  • "Compare with previous" action accessible from test context menu
  • Diff summary shows number of changed lines
  • Output history storage is configurable and can be cleared

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions