[CLI] -f body.data silently swallows errors when response is {error:...}

[JamesDaizs]

Summary

When a request errors, the response shape is {"error": {"code":"INVALID_REQUEST", ...}} not {"data": ...}. With -f body.data, the JSONPath filter resolves to nothing and the CLI exits 0 with empty stdout. Agents conclude "no data exists" instead of "the API errored."

356 cases observed in a recent 6-runner CAIA-bench batch (~37k tool calls).

Reproduction

surf exchange-perp --pair BTCUSDT -o json -f body.data
# stdout: (empty)
# exit:   0
# Underlying response was: {"error":{"code":"INVALID_REQUEST","message":"NOT_FOUND: Symbol 'BTCUSDT:USDT'"}}

Other examples (cyberconnecthq/research-eval runs/):

• GT082 claude-opus-oauth surf exchange-perp --pair BTCUSDT -o json -f body.data → symbol mangling + silent swallow
• GT236 claude-opus-oauth surf web-fetch ... -f body.data | python3 -c ... → 502 hidden as empty
• GT061 claude-opus-oauth surf wallet-detail ... -f body.data.approvals → silent empty

Impact

Models reading the empty stdout fabricate "no recent activity" / "no data found" in their final reports. The eval team has to re-run without -f to find the real cause — costs significant debug time and pollutes hallucination metrics.

Suggested fix

When -f <path> resolves to empty AND response has an error key:

• Write the formatted error envelope to stderr
• Exit non-zero (e.g. 4)

if filterPath != "" && filterResult == nil {
    if errKey, ok := response["error"]; ok {
        fmt.Fprintln(os.Stderr, formatError(errKey))
        os.Exit(4)
    } else {
        fmt.Fprintf(os.Stderr, "surf: filter %q resolved to empty (response keys: %v)\n", filterPath, keys(response))
        os.Exit(5)
    }
}

Source

Forensic write-up at cyberconnecthq/research-eval → reviews/surf_cli_vulns.md (section A3).