Skip to content

Show the rspec command output when the dry-run fails#27

Merged
tycooon merged 2 commits into
masterfrom
capture-rspec-dry-run-error
Jul 6, 2026
Merged

Show the rspec command output when the dry-run fails#27
tycooon merged 2 commits into
masterfrom
capture-rspec-dry-run-error

Conversation

@tycooon

@tycooon tycooon commented Jul 6, 2026

Copy link
Copy Markdown
Member

Problem

RunSpecs#example_ids runs rspec --dry-run --format=json --out <file> to enumerate examples for splitting. On failure it printed the contents of that --out JSON file — i.e. the example list — as the error:

RSpec dry-run failed. Output file contents:
{"version":"3.13.6","examples":[ ...thousands of lines... ]}

That's never the error. The real failure (a load error, or an exception raised at at_exit) goes to the process's stderr. execute streamed it via popen2e but never captured it, so the rescue had only the JSON file to fall back on. Worse, when the crash happens at at_exit — after rspec has already written a complete JSON file — that file contains no trace of the error at all, so the log is just a giant, useless JSON blob.

Fix

  • execute takes an opt-in capture: flag. When set, it accumulates the command's combined stdout+stderr and attaches it to the raised Error (new Error#output). Default is false, so the streaming behaviour of every other call is unchanged (and large test runs aren't buffered into memory).
  • RunSpecs runs the dry-run with capture: true and, on failure, fails with the captured output instead of the JSON file.

A failed dry-run now surfaces the actual error:

RSpec dry-run failed:
An error occurred while loading ./spec/broken_spec.rb.
LoadError: cannot load such file -- some_gem

Specs updated for the new behaviour; full suite is green with 100% line and branch coverage.

RunSpecs#example_ids ran the `rspec --dry-run` and, on failure, printed the
contents of rspec's `--out` JSON file (the example list) instead of the
actual error. The real error goes to the process's stderr, which `execute`
streamed but never captured, so the rescue only had the JSON to show — and
when the failure happens at exit (after the JSON is already written) that
file contains no trace of the error at all.

`execute` now optionally captures the command's combined stdout+stderr and
attaches it to the raised `Error`; `RunSpecs` runs the dry-run with
`capture: true` and fails with that output.
@tycooon
tycooon merged commit 0cc30c6 into master Jul 6, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants