Skip to content

Waveform Diff Testing#266

Closed
Nikil-Shyamsunder wants to merge 6 commits into
nfa-to-dfafrom
waveform-testing
Closed

Waveform Diff Testing#266
Nikil-Shyamsunder wants to merge 6 commits into
nfa-to-dfafrom
waveform-testing

Conversation

@Nikil-Shyamsunder

@Nikil-Shyamsunder Nikil-Shyamsunder commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Do not merge until #264

Addressing #253. Most of the diff is the new runt runt/waveform config, the new ASCII output .expect files, and a bunch of shifts in the test because I removed the typechecking noise. Here are the main interesting things:

  • in graph_interp/main.rs and interp/main.rs, add support for printing the ASCII waveform output with --ascii-waveform
  • in the AST interpreter, before stepping the simulator we collect the DontCare/Concrete inputs and outputs and stick them in a waveform state which is just a FxHashMap<PortId, Vec<BitVecValue>>. An output is DontCare if any of its combinationally dependent inputs is DontCare. We do the same logic in graph_interpreter.rs.
  • there's also logic to actually pretty print the waveform in the format you had.

Diffing AST and Graph Interp

  • We want to diff results between the AST interpreter and the IR interpreter. They have different error messages, so really all we want to see is that the AST interpreter gives the same result for each trace (pass/fail) and the same waveform ouput. The problem is that there is no way to disable the error messages or route them to stderr instead of stdout.
  • To resolve this, I switched diagnostics (warning and error messaging) to go to stderr. The results ("Trace k excuted successfully/failed") , graph serialization with --graphout, and ASCII waveforms go to stdout. This applies to interpreter/graph-interpreter only, not to monitor/bi. I then switched the runt targets to use the binaries directly instead of cargo run, so that stderr is clean with only the error messages (it doesn't have the noise from cargo run. This way, the waveform tests can check the trace results are the same and the waveform is the same, without dealing with warning or error messages being different. It also removes all the annoying typechecking warnings from the diff, as in issue Disable Type Inference Info in test/serialization outputs #263.

@Nikil-Shyamsunder Nikil-Shyamsunder force-pushed the waveform-testing branch 2 times, most recently from 87c8807 to 7736659 Compare July 1, 2026 15:14
Nikil-Shyamsunder and others added 6 commits July 1, 2026 11:28
* multi-protocol lowering for concrete traces

* nfa to dfa

* fix bug in assignment merge semantics

* remove done from everything other than last protocol instance

* graphs are too hard to read -only serialize nodes reachable and dont include edges that are false

* testing

* fix the semantics of assignments

* some cleanup

* reorg

* fix the assignment semantics more

* debug wishbone internal assert issue

* clean up the lowerin

* improve one of the unorered assignment semantics tests
switched interpreter to output diagnostics on stderr and base errors + ascii wveform on stdout
janky thing to keep monitor on stdout only by specifying the err/out semantics in the test gen
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant