Skip to content

feat: reduce AI-agent flag/command errors#13

Open
weiminglong wants to merge 1 commit into
asksurf-ai:mainfrom
weiminglong:feat/reduce-agent-errors
Open

feat: reduce AI-agent flag/command errors#13
weiminglong wants to merge 1 commit into
asksurf-ai:mainfrom
weiminglong:feat/reduce-agent-errors

Conversation

@weiminglong

Copy link
Copy Markdown

Summary

Analysis of 194 recent agent errors collected from Claude Code / Codex usage showed ~40% traced to hermod doc strings (fixed in https://github.com/cyberconnecthq/hermod/pull/711) and a further group that the CLI can fix directly. This PR lands the four highest-ROI CLI improvements.

Changes

  1. Fix the broken root example. surf market-futures --symbol BTC in both README.md and cmd/surf/main.go Root.Example — market-futures has no --symbol flag. Replaced with surf market-price --symbol BTC, which is a real endpoint.

  2. Surface `[required]` + `example:` in `--help`. Param.Required and Param.Example were captured from the OpenAPI spec but never shown, so agents had to guess which flags were mandatory. New helpDescription() helper (cli/param.go) prepends [required] and appends (example: BTC) when those fields are set. No other code path changed.

  3. "Did you mean?" for unknown flags. Every API subcommand now has a FlagErrorFunc (cli/operation.go) that computes Levenshtein distance against its own flags and suggests the closest match (≤ 3 edits or substring match). Sort order prefers shorter names on ties so --query actually surfaces --q instead of only --quiet. Net effect: agents that guess --query, --timerange, --username get pointed at the real flag immediately.

  4. `surf list-operations --json`. Emits a machine-readable command catalog for agents — array of {name, method, group, required_flags[], optional_flags[], path_flags[], body_required} with each flag including type, description, default, example. Agents can now introspect the full command surface before calling, instead of trial-and-error.

Examples

`--help` now shows required + example:
```
--market-slug string [required] REQUIRED. Polymarket market slug (identifies a single Yes/No outcome)...
--symbol string Single token ticker symbol like `BTC`, `ETH`, or `SOL` (example: BTC)
```

Unknown flag → suggestion:
```
$ surf search-project --query bitcoin
Error: unknown flag: --query

Did you mean one of these?
--q
--quiet
```

JSON catalog (first entry):
```json
{
"name": "polymarket-markets",
"method": "GET",
"group": "Prediction Market",
"required_flags": [{"name": "market-slug", "type": "string", "required": true, "description": "..."}],
"optional_flags": [{"name": "limit", "type": "integer", "default": 20}, ...]
}
```

Test plan

  • `go test ./...` passes (all three packages, 36s)
  • New unit tests in `cli/suggest_test.go` cover `extractUnknownFlagName`, `levenshtein`, `suggestFlagNames`, `flagErrorFuncWithSuggestions`, and `helpDescription` variants
  • Built locally, ran `surf list-operations --json`, `surf polymarket-markets --help`, `surf search-project --query bitcoin` — all behave as expected
  • No changes to request path, response parsing, or auth

Out of scope (follow-ups)

  • Flag aliases layer (`--keyword`/`--query` as hidden aliases of `--q`) — the did-you-mean suggestion handles this case with a clear error; adding silent aliases is a bigger behavior change we can do in a follow-up if agents keep failing.
  • SKILL.md in this repo — the canonical agent skill lives in urania/surf-skills and is being updated there separately.

🤖 Generated with Claude Code

Analysis of 194 recent agent errors showed four CLI-side improvements that
would materially reduce guesswork. Implements all four:

1. Fix the broken README example. `surf market-futures --symbol BTC` was
   shown in both README.md and the root command's Example, but
   market-futures has no --symbol flag. Replaced with `surf market-price
   --symbol BTC`, which is a real endpoint.

2. Surface [required] + example in --help. `Param.Required` and
   `Param.Example` were captured from the OpenAPI spec but never shown in
   --help output, so agents couldn't tell which flags were mandatory. A
   new helpDescription() helper prepends "[required] " and appends
   "(example: BTC)" when those fields are set. No other code path touches
   the raw Description string.

3. Did-you-mean for unknown flags. Previously Cobra's default error was
   just "unknown flag: --query". Added a FlagErrorFunc on every API
   subcommand that computes Levenshtein distance against the command's
   own flags and suggests the closest (≤ 3 edits or substring match).
   `--query` → `--q`, `--timerange` → `--time-range`, etc. Sort order
   prefers shorter names on ties so `--query` actually surfaces `--q`.

4. Machine-readable catalog via `surf list-operations --json`. Emits a
   structured array of {name, method, group, required_flags[],
   optional_flags[], path_flags[], body_required} with each flag
   including type, description, default, and example. Agents can now
   introspect the full command surface before calling instead of trial-
   and-error. Human output paths unchanged.

Tests added in cli/suggest_test.go cover the new helpers. Full test suite
passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@weiminglong
weiminglong changed the base branch from main to stg April 21, 2026 02:04
@akasuv
akasuv changed the base branch from stg to main April 21, 2026 03:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant