Conversation
Adds 5 TOML filters for the Pulumi CLI, matching the existing OpenTofu/ Ansible/Terraform TOML-based pattern. Filters strip known noise (header banners, View Live URLs, policy pack loading, @ progress spinners, intermediate creating/updating/deleting rows, Duration, Node.js stack trace frames) while preserving resource change rows, policy violations, diagnostics, outputs, and summaries. Measured savings on a real neo-migration AWS project (~30 resources, Pulumi Cloud backend): pulumi up 65.8%, pulumi destroy 72.3%, preview --refresh 40.7%, refresh 34.9%, stack 29.3%. Lower savings on already- compact output (clean preview ~9%) because remaining content is signal per the correctness-over-compression design philosophy. Registers matching RtkRule in src/discover/rules.rs so the rewrite hook transparently proxies `pulumi <subcmd>` through `rtk pulumi`. Bumps BUILTIN_TOML filter count 59 -> 64 and updates the expected-filter list and concat-discoverability test. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Revert CHANGELOG.md (release-please CICD generates from commit msg) - Narrow pulumi-stack match_command to explicit subcommand whitelist (bare stack, ls, output, history, select, init, rm, rename, tag, unselect, change-secrets-provider). Excludes export (JSON state) and graph (DOT), which now passthrough unfiltered. - Lower savings_pct 70 -> 45 (measured mean across 5 subcommands). Populate subcmd_savings with measured values: up=66, destroy=72, refresh=35, preview=25, stack=29. JSON-safety (self-found during review validation): - All 5 pulumi subcommands support -j/--json; Rust regex crate has no lookahead so match_command can't negatively exclude --json. - Remove max_lines from all 5 filters; strip patterns don't match JSON structure so JSON passes through unfiltered. - Remove ^\\s*\\}\\s*$ strip pattern (was targeting Node.js stack trace trailers, but also matched legitimate JSON closing braces). Verified on neo-migration stack: - pulumi stack ls/output/history/export, preview --json, stack graph: byte-identical with RTK_NO_TOML=1 baseline (filter transparent). - Human-readable preview still compresses (~20% on clean state). - 10/10 inline pulumi tests pass; 1449/0 full cargo test suite. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…atterns Four bugs in `extract_pattern_path` caused `rtk grep` to misparse common invocations: - `-A 2 error src` → `2` was taken as the pattern (value-taking flags not recognised) - `-rn foo src` → `n` leaked to rg as `--replace` value (only standalone `-r` was stripped, not combined clusters like `-rn`) - `TODO src tests` → second path silently dropped (only one path kept) - `-i x agent -g '*.md'` → rg regex error (`*.md` became positional before `-g` could consume it) New implementation: - `VALUE_FLAGS_SHORT`: explicit byte-set of 2-char flags that consume one following token (`A`, `B`, `C`, `g`, `f`, `j`, `m`) - `process_flag()`: strips `r`/`R` from any single-dash cluster, drops `--recursive`; replaces the old inline `if arg == "-r"` loop - `extract_pattern_path` returns `(Vec<String>, Vec<String>, Vec<String>)` (patterns, paths, flags) — no more `Option<String>` for pattern - `-e` values collected separately; if any `-e` is present, all positionals are paths; otherwise first positional is the pattern - All patterns forwarded to rg as `-e` flags (BRE `\|`→`|` inline) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A combined cluster like -rne PATTERN or -rA 2 had the last flag's value consumed only if the cluster was exactly 2 chars. Longer clusters (e.g. -rne) fell through to process_flag, which stripped -r and emitted -ne — then -e ended up in flags without a value, causing rg to error. New unified short-flag handler: for any single-dash cluster, inspect the last byte. If it is -e or a VALUE_FLAGS_SHORT member, strip the r/R chars from the prefix, emit the prefix as a flag (if non-empty), and consume the next token as the value (pattern for -e, flag value otherwise). Long flags (--foo) are still routed through process_flag. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rtk diff treated files whose changes were all classified as modified (similar lines, e.g. "a: 1" vs "a: 2" in YAML/JSON) as identical, because the identical check only looked at added/removed counts. Report any non-empty change set as a difference, and exit 1 when files differ per diff convention (0 when identical). Fixes #2364
…hrough Five regressions on grep_arg_parsing vs develop: - `-ecarrot` matched "caot" — strip_r was applied to the inline value bytes. Fix: scan cluster left-to-right; stop at first value-taking letter and treat the remainder as the inline value without r/R stripping. - `-g*.rs` glob became `*.s` — same root cause, inline glob value r-stripped. - `--glob '*.md'` took `*.md` as a path — no VALUE_FLAGS_LONG existed; the value token fell through to the positional collector. Fix: add VALUE_FLAGS_LONG covering all rg long flags that take a value. - `--max-count 1 fn file` took `1` as the pattern — same missing long flag. - VALUE_FLAGS_SHORT expanded: add t/T (--type/--type-not), d (--max-depth), M (--max-columns) so inline short forms and space-separated forms parse correctly without mis-routing values to patterns or paths. - --regexp (long form of -e) now routes to patterns like -e does. - has_format_flag: add --count-matches, --json, --passthru, --files so these output-format modes get passthrough treatment instead of silent "0 matches in 0 files". 2183 tests pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… clippy fix
The left-to-right cluster scan now calls strip_r() at emit time on the
accumulated flag-letter prefix (raw_prefix), never on inline value bytes.
This makes the safety contract explicit and testable: strip_r("carrot")
produces "caot", documenting exactly why it must not touch value bytes.
Also fix pre-existing unused-mut in aws_cmd.rs (has_output_flag).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…y testable
The left-to-right scanning logic now lives entirely in parse_cluster(), which
returns a ClusterResult enum (Boolean or ValueTaking). extract_pattern_path
just dispatches on the result — no loop logic inline.
parse_cluster() is the only place that touches cluster bytes, so the full
contract is testable in isolation:
parse_cluster("ecarrot") → ValueTaking { prefix: None, flag: 'e', inline: "carrot" }
parse_cluster("rne") → ValueTaking { prefix: Some("n"), flag: 'e', inline: "" }
parse_cluster("g*.rs") → ValueTaking { prefix: None, flag: 'g', inline: "*.rs" }
parse_cluster("rn") → Boolean(Some("n"))
strip_r() remains as the focused r/R-stripping helper used internally by
parse_cluster (and tested directly to document the "carrot"→"caot" danger).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix(grep): stabilize argument parsing — trailing_var_arg, -v invert-match, --version passthrough, safe rg invocation
…se-identical fix(diff): report modified-only diffs and follow diff exit convention
The system-grep fallback (used when ripgrep is not installed) passed -rnHZ, relying on -Z for the NUL filename separator the match parser requires. -Z only means --null on GNU grep; on BSD/macOS grep it is an alias for --decompress (zgrep mode), so output is plain file:line:content with no NUL. parse_match_line() then matches zero filenames and every result collapses into "N matches in 0 files" with all lines hidden behind [+N more]. Use the long option --null instead, which both GNU and BSD grep define as "print a zero-byte after the file name". Related to #2310 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fix(hook): rewrite pytest under uv run
Exit codes >= 2 signal real errors in grep/rg (bad regex, tool crash, missing binary). rtk was only checking the narrow case exit_code == 2 with non-empty stderr, letting any other error exit masquerade as a clean "0 matches" false negative. Now any exit code >= 2 surfaces the error to stderr instead of printing a false zero-match result. Fixes #2461 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The test simulates a broken `rg` via a chmod'd shell script (from_mode, PermissionsExt) which is unix-only. The unconditional `use std::os::unix` broke compilation of the integration-test crate on the Windows CI target. Gate the whole file so Windows stays green while preserving unix coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wrap the rtk binary path expression per rustfmt to satisfy `cargo fmt --all -- --check` on CI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address KuSh review on #2465: - Move the `exit_code >= 2` decision into a pure `is_grep_error_exit` function and unit-test it directly (0/1 = normal, >=2 = error), instead of faking the rg binary in an integration test. - Drop the GREP_ERROR_EXIT const; the doc comment on the function conveys the grep/rg exit convention. - Remove tests/grep_error_test.rs (faked binary) in favour of the unit test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
run_commit had a branch that printed 'ok (nothing to commit)' and fell through to Ok(0) whenever git exited non-zero with a 'nothing to commit' message — so a no-op or hook-aborted commit was reported as success with exit 0. Gate strictly on output.status.success() via a CommitOutcome helper: any failure surfaces git's message and propagates the real exit code, matching native git. Same bug class as #1581 (push) and #1535 (stash). Fixes #2494
The compact status path only caught the "not a git repository" error and let every other failure (corrupt index, lock contention, broken refs) fall through to a formatted "Clean working tree" + Ok(0), masking a real git error. The non-compact path already guards on result.success(); apply the same guard to the compact path, keeping the friendly not-a-repo message and propagating git's real exit code otherwise. Part of #2497
run_worktree list mode never checked result.success() — a failed `git worktree list` (e.g. run outside a repo) was flattened to empty output + exit 0. The has_action branch already guards on success; apply the same guard to list mode and surface git's error with its exit code. Part of #2497
feat(oc): add Openshift CLI support with shared k8s filtering
On failing `dotnet test` runs the orchestrator prepended the full raw stdout ahead of the filtered summary. The `Failed Tests:` section already reproduces each failure (name + message + clipped stack) parsed from TRX/console, so every failure was printed twice — inflating output +65% vs raw, scaling linearly with failure count (issue #2501). Gate the raw-stdout prepend behind `test_needs_raw_fallback`: skip it when the structured section carries detail, keep it when the filter is blind (no failures parsed, or a parsed failure has empty detail — e.g. a self-closing <UnitTestResult> with no <ErrorInfo>, or a build failure / crash where nothing reaches failed_tests). Extracts two pure, unit-tested fns (`test_needs_raw_fallback`, `compose_failure_output`) so the orchestration decision is covered without running dotnet. Closes #2501 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
|
Tests are currently fail because oc added on develop isn't registered in the PASSTHROUGH allowlist checked by test_every_subcommand_is_classified. This test was introduced on master for a security fix. Fix: resync with master, then add "oc" to the PASSTHROUGH list. Fixed : #2514 |
RTK could emit more tokens than the underlying command on small inputs:
filters that add headers, summaries, re-indentation, or a tee hint, plus
synthetic no-result messages ("0 matches", "No stashes", "[docker] 0
containers") printed where the raw command emitted nothing. Both break the
Transparency principle and inflate tokens instead of saving them.
- core::guard::never_worse(raw, filtered) returns raw when the filtered form
has more tokens (reuses tracking::estimate_tokens), so RTK output is never
larger than the real command.
- runner::emit_guarded(filtered, hint, raw) composes body + tee hint, guards
the whole, prints, and returns what was shown so printed == tracked.
- run_captured_filter guards the run_filtered* family centrally; per-site
guards cover the remaining single-string filters.
- On empty raw, emit empty and preserve the exit code instead of a synthetic
no-result message (the messages were cosmetic with no dependents; #2461
reports the grep one as actively harmful).
- git stash show now propagates its exit code instead of masking a real
failure as Ok(0).
Resolves #2551.
fix(grep): correctly handle all flag shapes and never exceed raw output
Masking env secrets is not what core RTK (OSS) is for -- it compresses command output, it does not redact what the command itself already reveals. It also broke raw-vs-filtered consistency, which the never-worse guard relies on.
pipe filters piped stdin but bypassed the guard; route its output through never_worse like every other compressor.
Compare filtered output against the raw the agent's command would emit (numbered for -n), not against unnumbered content, so the guard keeps requested line numbers instead of dropping to plain.
…eline
docker ps/ps -a/images ran a plain command for the raw baseline and a
separate --format command for parsing, capturing the plain run with
unwrap_or_default(). A failed plain run while --format succeeded left
raw empty, and never_worse("", summary) dropped real output.
Run the agent's actual command first as the source of truth: on failure
show its output/exit faithfully; on success use its stdout as raw and
treat --format as a best-effort parse aid that falls back to raw. The
guard stays pure and always sees a real baseline. Parsers unchanged.
…orse-guard # Conflicts: # src/cmds/system/grep_cmd.rs
fix(core): never-worse output guard
Per review (#2498): the guard is self-explanatory; remove the comment to avoid noise.
Per review (#2498): self-explanatory guard; remove comment to avoid noise.
…-code fix(git): propagate exit code on git status failure in compact path
develop already propagates the exit code for failed git stash list/show (via the empty-stdout guard). Add regression tests so the masking-failure behavior (#2497) can't creep back.
…t-code test(git): cover exit-code propagation for git stash list/show failure
fix(git): propagate exit code when commit fails instead of reporting ok
…code fix(git): propagate exit code on git worktree list failure
- grep runs grep, rg runs rg: drop the substitution, forced --no-ignore-vcs, and BRE-to-rg translation - add `rtk rg` command (native ripgrep, sharing the same output filter) - split rewrite rule: grep to rtk grep, rg to rtk rg - record the agent's real command in tracking (was synthesized as "grep -rn") - emit nothing on a clean no-match (never-worse parity with the shared guard) - rename grep_cmd.rs to search.rs, now hosting both engines - cover engine faithfulness, ignore semantics, and rg savings with issue-referenced tests - benchmark the grep and rg paths
- line number always (the openable position); filename only when grep/rg prints one (multi-file, dir, -r, -H) - single file and stdin equal `grep -n` byte-for-byte: no synthetic filename, header, or lossy truncation - read piped stdin instead of injecting `.`; rg no longer searches the cwd on a pipe - pattern-less and format-flag commands run verbatim (rg --type-list, --files, grep -c/-o) - grouped/capped form only when capping actually shrinks the output - byte-for-byte tests vs grep -n across flags, pipes and edge cases (incl #1436)
- leading ^ anchor must stay scoped to the path (tenequm) - line ending in `:` keeps full content (BTCAlchemist) - rg surfaces a regex parse error as exit 2, not a silent 0 (jhagberg)
- grep_compress_test.rs -> search_compress_test.rs (tests the shared grep/rg filter) - binary match skipped as noise by default, -a opts back in - fix parse_flags doc: -I is an intentional binary skip, not a no-op
- drop the synthetic "search failed with exit code N" line (an rtk addition) - surface the engine stderr verbatim in every path, incl. partial match + error - remove the now-vestigial is_grep_error_exit (the "0 matches" it guarded is gone) - error/exit matrix tests vs real grep/rg (#2465: an error is never a silent no-match)
refacto(search): run the invoked engine instead of substituting rg for grep
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Feats
Fix