Skip to content

Fix flaky quoted-path-parsing property test#169

Merged
jbearak merged 1 commit into
mainfrom
fix-flaky-test
May 5, 2026
Merged

Fix flaky quoted-path-parsing property test#169
jbearak merged 1 commit into
mainfrom
fix-flaky-test

Conversation

@jbearak
Copy link
Copy Markdown
Owner

@jbearak jbearak commented May 5, 2026

Problem

The property test Quoted Path Parsing Property Tests > Property 2: Comma Boundary Respected > should capture all arguments before comma fails intermittently in CI.

The local arbitrary_identifier() generator in this file could randomly produce Stata reserved qualifier keywords (`if`, `in`) or prefix/file commands (`by`, `do`, etc.). When fast-check sampled such a keyword as a varlist argument to `display`/`list`/`gen`, the parser correctly treated it as a qualifier rather than a variable, so `varlist` came back as `undefined` and the assertion that `my_cmd_node.varlist?.length === my_args.length` failed.

Reproduction:

```
display if, opt → varlist: undefined, options: ["opt"]
display in, opt → varlist: undefined, options: ["opt"]
```

This is exactly the failure mode `AGENTS.md` warns about:

Stata treats `if` and `in` as qualifiers in many statement contexts, so generating them as "ordinary identifiers" can produce misleading property-test failures.

Fix

Delegate the file-local `arbitrary_identifier()` to the shared `arbitrary_non_reserved_identifier()` from `tests/property/generators/primitives.ts`, which filters out `RESERVED_QUALIFIER_KEYWORDS`, `PREFIX_COMMANDS`, and `FILE_COMMANDS`.

Validation

  • `bun run typecheck` clean
  • Full suite: 5532 pass, 0 fail
  • 10 consecutive runs of the affected test file pass cleanly

Conversation

The local arbitrary_identifier() generator could randomly produce Stata\nreserved qualifier keywords (if, in) or prefix/file commands (by, do,\netc.). When fast-check sampled such a keyword as a varlist argument to\ndisplay/list/gen, the parser correctly treated it as a qualifier rather\nthan a variable, causing intermittent CI failures in property tests like\n'should capture all arguments before comma'.\n\nDelegate to arbitrary_non_reserved_identifier() from the shared\ngenerators per the AGENTS.md guidance, which filters out\nRESERVED_QUALIFIER_KEYWORDS, PREFIX_COMMANDS, and FILE_COMMANDS.
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 5, 2026

Warning

Rate limit exceeded

@jbearak has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 22 minutes and 47 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 61fa7f2d-20b6-445a-a48e-5106e116b721

📥 Commits

Reviewing files that changed from the base of the PR and between 43962a9 and ec85036.

📒 Files selected for processing (1)
  • tests/property/quoted-path-parsing.prop.test.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-flaky-test

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@jbearak jbearak merged commit 4676b65 into main May 5, 2026
10 checks passed
@jbearak jbearak deleted the fix-flaky-test branch May 5, 2026 07:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant