Enhance flux-coverage: opcode tracking, multi-format reports, diffing, pytest plugin#3
Open
SuperInstance wants to merge 1 commit intomainfrom
Open
Enhance flux-coverage: opcode tracking, multi-format reports, diffing, pytest plugin#3SuperInstance wants to merge 1 commit intomainfrom
SuperInstance wants to merge 1 commit intomainfrom
Conversation
…, pytest integration, 29 tests - Track opcode coverage (247 opcodes) with hit counts and first-hit PCs - Enhanced branch coverage with per-branch-point taken/not-taken counts - Register coverage with read vs. written distinction - Report formats: terminal, JSON, HTML (styled), markdown - CoverageDiff for comparing two coverage runs - Overall weighted score across all metrics - FluxCoveragePlugin for pytest integration - 29 tests covering core, diff, and plugin functionality
![beta-devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/2443040?s=60&v=4){kind=small}
Comment on lines
+667
to
+677
| merged = self._reports[0] | ||
| for r in self._reports[1:]: | ||
| merged.opcodes_seen = len(set(merged.opcode_details) | set(r.opcode_details)) | ||
| merged.hit_instructions = max(merged.hit_instructions, r.hit_instructions) | ||
| merged.total_instructions = max(merged.total_instructions, r.total_instructions) | ||
| merged.registers_used = len(merged.register_read | r.register_read | merged.register_written | r.register_written) | ||
| merged.branches_taken = len(set(b.pc for b in merged.branch_details if b.taken) | | ||
| set(b.pc for b in r.branch_details if b.taken)) | ||
| merged.branches_not_taken = len(set(b.pc for b in merged.branch_details if b.not_taken) | | ||
| set(b.pc for b in r.branch_details if b.not_taken)) | ||
| return merged |
There was a problem hiding this comment.
🔴 generate_summary() mutates self._reports[0] by aliasing instead of copying
merged = self._reports[0] at line 667 creates a reference, not a copy. The subsequent loop (lines 669-676) directly mutates the first stored report's scalar fields (opcodes_seen, hit_instructions, total_instructions, registers_used, branches_taken, branches_not_taken). This corrupts the original report object in-place, meaning: (1) calling generate_summary() permanently corrupts self._reports[0], (2) the returned summary IS the first report (verified: s1 is r1 is True), and (3) any code holding a reference to that report sees its values change unexpectedly.
Reproduction showing mutation
plugin = FluxCoveragePlugin()
r1 = collect_coverage([0x18, 0, 42, 0x00]) # r1.opcodes_seen == 2
r2 = collect_coverage([0x18, 0, 10, 0x18, 1, 20, 0x20, 2, 0, 1, 0x00])
plugin.add_report(r1)
plugin.add_report(r2)
summary = plugin.generate_summary()
# r1.opcodes_seen is now 3 (was 2) — r1 is corrupted!
# summary is r1 — same objectPrompt for agents
In FluxCoveragePlugin.generate_summary() at coverage.py:667, `merged = self._reports[0]` creates a reference alias, not a copy. All subsequent mutations (lines 669-676) modify the original report stored in self._reports[0]. The fix should create a deep copy of the first report before mutating it. Additionally, the merge loop only updates scalar counters (opcodes_seen, hit_instructions, etc.) but does NOT merge the underlying detail data structures (opcode_details dict, register_read/register_written sets, branch_details list). This means the returned summary has inconsistent data — e.g., opcodes_seen=3 but len(opcode_details)=2. The fix should: (1) use copy.deepcopy or construct a new CoverageReport, and (2) actually merge opcode_details dicts, register_read/register_written sets, and branch_details lists from all reports.
Was this helpful? React with 👍 or 👎 to provide feedback.
Debug
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Comprehensive enhancement of the FLUX coverage analyzer with 29 tests.
Changes
FluxCoveragePluginfor collecting and aggregating coverage across sessionscollect_coverage()shorthand functionTest Results
29 tests passing across 3 test classes:
TestCoverageCore(15 tests): instruction, branch, register, path, opcode, and report format testsTestCoverageDiff(5 tests): diff improvement, markdown/terminal/JSON output, identical comparisonTestCoveragePytest(8 tests): plugin lifecycle, aggregation, edge cases