From 676f34aa889aa1e956677e6fd06c81e363912380 Mon Sep 17 00:00:00 2001 From: Ari Mayer Date: Wed, 1 Jul 2026 20:27:35 -0400 Subject: [PATCH] feat(env-injection): value filters on resolved references (#138) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes item 3 of the #115 epic. A credential reference may end with "| filter" to transform the value before injection: ${pass:api/token | base64} # base64 a token ${pass:api | basicauth} # base64("username:password") for Basic auth --set 'AUTH=api|basicauth' # same, on the mapping forms Filters: base64 (standard), base64url (URL-safe), and basicauth (combines a credential's own username + password). One filter per reference. Design: - Parse once in the shared SplitPath (envmap): peel an optional "| filter", trim only when a pipe is present (filterless refs unchanged byte-for-byte), validate the name fail-closed, reserve the first '|'. New 4-return signature propagated to ParseSetSpec, parseTemplate, ParseManifest, so every surface (--set, manifest, ${pass:...}) gets it at once. - Carry via a new Filter field on envmap.Mapping and TemplateRef. - Apply client-side in resolver.ResolveValuesFiltered, after the backend returns raw values — the agent's values-only protocol is untouched, so output is identical in direct and daemon modes. basicauth resolves username and password via the normal value path (batched) and base64s "user:pass". - Single-source filter registry in internal/envmap/filter.go: IsKnownFilter (parse-time validation) and ApplyValueFilter (apply-time) both derive from one map, so they can't drift. Filters are not applied to the positional form (never parsed by SplitPath); on --set the '|' must be shell-quoted. Both documented. Tests: SplitPath filter parsing (spaces, colon form, empty/unknown/chaining/ multi-segment errors), filter registry, ResolveValuesFiltered (base64 + basicauth + unknown), and integration for inject (base64/basicauth/unknown), export (--set base64, filter-then-quote), and an agent-path case proving the filter is applied client-side while the daemon serves the raw value. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_014kVLjbUL4F4CkoA7RbMYy8 --- CHANGELOG.md | 1 + cmd/exec.go | 4 +- cmd/export.go | 5 +- cmd/inject.go | 21 +++++++-- docs/03-reference/command-reference.md | 18 ++++++++ internal/envmap/envmap.go | 59 +++++++++++++++++++----- internal/envmap/envmap_test.go | 45 +++++++++++++----- internal/envmap/filter.go | 43 ++++++++++++++++++ internal/envmap/filter_test.go | 55 ++++++++++++++++++++++ internal/envmap/manifest.go | 4 +- internal/envmap/template.go | 7 +-- internal/resolver/resolver.go | 58 +++++++++++++++++++++++- internal/resolver/resolver_test.go | 43 ++++++++++++++++++ test/integration/agent_test.go | 20 ++++++++ test/integration/export_test.go | 22 +++++++++ test/integration/inject_test.go | 63 ++++++++++++++++++++++++++ 16 files changed, 429 insertions(+), 39 deletions(-) create mode 100644 internal/envmap/filter.go create mode 100644 internal/envmap/filter_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 74a2210..ef8081f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- **Env-injection: value filters on resolved references** (#138) — a credential reference may end with `| filter` to transform the value before injection, resolved through the same shared grammar so it works in `${pass:...}` templates, `--set`, and `.pass-cli.toml` manifests: `${pass:api/token | base64}`. Filters: `base64` (standard) and `base64url` (URL-safe) encode a single value; `basicauth` combines a credential's own username and password into `base64("user:pass")` for an HTTP `Authorization: Basic` header (takes no field). One filter per reference; an unknown or empty filter is a hard parse error and nothing is injected (fail closed) — for `--set` and manifests this is caught before the vault is even opened. Filters are applied client-side after resolution, so the agent's values-only protocol is unchanged (identical output in direct and daemon modes). On the `--set` form the `|` must be shell-quoted; filters are not applied to the positional `` form. Item 3 of the #115 epic, completing it. - **Sync: configurable remote-probe timeout** (#137) — the pre-unlock metadata probe (`rclone lsjson`) was bounded at a hardcoded 8s, so a slow-but-alive remote on a high-latency backend would time out, be misclassified as failed, and enter the failure-backoff (#133) — deferring pulls/pushes for the whole window even though the remote was reachable. New `sync.probe_timeout_seconds` mirrors the `pull_ttl_seconds` tri-state: `0` uses the built-in default (8s), a positive value raises (or lowers) the bound, and a negative value disables it (unbounded probe). Only the metadata probe is bounded — the heavy pull/push transfers remain unbounded regardless. - **Background agent (`pass-cli agent`)** (#116) — an optional daemon that unlocks the vault once and holds it in memory, answering read-only credential lookups over a local unix socket so `exec`/`export`/`inject` need no master-password prompt and no key derivation on each call. It serves resolved field **values only** — the master password and derived key never cross the socket. Auto-locks after `--idle` inactivity (default 15m) and always after `--max-ttl` (default 8h), and locks + exits on SIGINT/SIGTERM. **`pass-cli agent start`** backgrounds the agent (unlock once on your terminal, then detach — no shell `&` needed); **`pass-cli agent serve`** (or bare `pass-cli agent`) runs it in the foreground. **`agent stop`** zeroes the resident secrets and stops the agent (freeing the socket so the next command falls back to direct-open; re-run `agent start` to re-establish it); `agent status` reports its state (never prints secrets). When no agent is running, every command **transparently falls back** to opening and unlocking the vault directly, so the agent is a pure optimization, never a dependency. Socket path: `$PASS_CLI_AGENT_SOCK`, else `$XDG_RUNTIME_DIR/pass-cli/agent.sock`, else `~/.pass-cli/agent.sock` (directory `0700`, socket `0600`). Connections are additionally authorized by peer credential — only a process owned by the same user may talk to the agent, and any failure to read the credential is a rejection (fail-closed): Linux via `SO_PEERCRED`, macOS via `getsockopt(LOCAL_PEERCRED)`. (Windows would use a named pipe + ACL; the Windows agent is not yet implemented and falls back to direct-open.) POSIX only for now; a Windows named-pipe transport is planned. - **`export` command — print shell statements that set credentials as env vars** (#115) — `pass-cli export` emits `export NAME='value'` for `eval`/`source`, the blessed replacement for `VAR="$(pass-cli get …)"`: `eval "$(pass-cli export --set GITHUB_TOKEN=github)"`. Uses the same mapping grammar as `exec` (repeatable `--set ENV_NAME=service[/field]` and the convenience form `pass-cli export `, which derives the name from the service), and the same `-f/--field` selection. `--format sh|fish|powershell` selects the shell syntax (default `sh`). Read-only like `exec` (records no usage, triggers no sync push). Because `export` output is meant to be eval'd, env names are validated against `[A-Za-z_][A-Za-z0-9_]*` before the vault is opened, and every field value is shell-quoted per target shell. diff --git a/cmd/exec.go b/cmd/exec.go index 0cea289..453e5d3 100644 --- a/cmd/exec.go +++ b/cmd/exec.go @@ -92,7 +92,7 @@ it is not process isolation.`, func init() { rootCmd.AddCommand(execCmd) - execCmd.Flags().StringArrayVar(&execSets, "set", nil, "map an environment variable to a credential: ENV_NAME=service[/field] (repeatable; ':field' also accepted)") + execCmd.Flags().StringArrayVar(&execSets, "set", nil, "map an environment variable to a credential: ENV_NAME=service[/field][|filter] (repeatable; ':field' also accepted; filters: base64, base64url, basicauth — quote the '|')") execCmd.Flags().StringVarP(&execField, "field", "f", "password", "field to inject for all mappings (username, password, category, url, notes, service)") execCmd.Flags().StringArrayVar(&execEnvFiles, "env-file", nil, "read KEY=${pass:service/field} template lines from a file (repeatable)") execCmd.Flags().StringArrayVar(&execFrom, "from", nil, "read ENV_NAME=service/field mappings from a .pass-cli.toml manifest (repeatable)") @@ -271,7 +271,7 @@ func runExec(cmd *cobra.Command, args []string) error { // ${pass:...} references) against the same read-only resolver. for _, e := range envEntries { value, rerr := envmap.RenderTemplate(e.Template, func(refs []envmap.TemplateRef) ([]string, error) { - return r.ResolveValues(templateMappings(refs), execField) + return resolver.ResolveValuesFiltered(r, templateMappings(refs), execField) }) if rerr != nil { return fmt.Errorf("env-file key %q: %w", e.Key, rerr) diff --git a/cmd/export.go b/cmd/export.go index cb6d357..5accb4b 100644 --- a/cmd/export.go +++ b/cmd/export.go @@ -8,6 +8,7 @@ import ( "github.com/spf13/cobra" "github.com/arimxyer/pass-cli/internal/envmap" + "github.com/arimxyer/pass-cli/internal/resolver" ) var ( @@ -67,7 +68,7 @@ blessed replacement for VAR="$(pass-cli get ...)", not as a replacement for 'exe func init() { rootCmd.AddCommand(exportCmd) - exportCmd.Flags().StringArrayVar(&exportSets, "set", nil, "map an environment variable to a credential: ENV_NAME=service[/field] (repeatable; ':field' also accepted)") + exportCmd.Flags().StringArrayVar(&exportSets, "set", nil, "map an environment variable to a credential: ENV_NAME=service[/field][|filter] (repeatable; ':field' also accepted; filters: base64, base64url, basicauth — quote the '|')") exportCmd.Flags().StringVarP(&exportField, "field", "f", "password", "field to export for all mappings (username, password, category, url, notes, service)") exportCmd.Flags().StringVar(&exportFormat, "format", "sh", "shell syntax: sh, fish, or powershell") exportCmd.Flags().StringArrayVar(&exportFrom, "from", nil, "read ENV_NAME=service/field mappings from a .pass-cli.toml manifest (repeatable)") @@ -140,7 +141,7 @@ func runExport(cmd *cobra.Command, args []string) error { } defer cleanup() - values, err := r.ResolveValues(mappings, exportField) + values, err := resolver.ResolveValuesFiltered(r, mappings, exportField) if err != nil { return err } diff --git a/cmd/inject.go b/cmd/inject.go index a9695a3..eb39534 100644 --- a/cmd/inject.go +++ b/cmd/inject.go @@ -8,6 +8,7 @@ import ( "github.com/spf13/cobra" "github.com/arimxyer/pass-cli/internal/envmap" + "github.com/arimxyer/pass-cli/internal/resolver" ) var ( @@ -32,11 +33,18 @@ connection string can be materialized in one step. pass-cli inject -i config.tmpl -o config.ini Reference syntax: - ${pass:service} the service's default field (see -f/--field) - ${pass:service/field} a specific field (username, password, url, ...) + ${pass:service} the service's default field (see -f/--field) + ${pass:service/field} a specific field (username, password, url, ...) + ${pass:service/field | flt} apply a filter to the value (see below) + +Filters (optional, one per reference): + base64 standard base64 of the value (e.g. a Bearer token) + base64url URL-safe base64 of the value + basicauth base64("username:password") from the credential (takes no field) Only ${pass:...} is special; $VAR, ${VAR}, and $(...) pass through untouched. An -unknown or malformed reference is a hard error and nothing is written. +unknown or malformed reference — including an unknown filter — is a hard error +and nothing is written. When the template is piped on stdin, unlock via the OS keychain: the master- password prompt also reads stdin, so a password prompt would consume the piped @@ -49,6 +57,9 @@ is on your terminal/pipe. Prefer 'exec' (child-scoped env) when you can; use Example: ` # Materialize a connection string echo 'redis://:${pass:redis/password}@cache:6379' | pass-cli inject + # A base64 Authorization header from a single credential's user:password + echo 'Authorization: Basic ${pass:api | basicauth}' | pass-cli inject + # Render a template file to a 0600 output file pass-cli inject -i .env.tmpl -o .env`, Args: cobra.NoArgs, @@ -98,7 +109,7 @@ func runInject(cmd *cobra.Command, _ []string) error { // Read-only via the shared resolver (no usage write, no sync push). All // references in the template resolve in a single batch call. rendered, err := envmap.RenderTemplate(string(input), func(refs []envmap.TemplateRef) ([]string, error) { - return r.ResolveValues(templateMappings(refs), injectField) + return resolver.ResolveValuesFiltered(r, templateMappings(refs), injectField) }) if err != nil { return err @@ -122,7 +133,7 @@ func runInject(cmd *cobra.Command, _ []string) error { func templateMappings(refs []envmap.TemplateRef) []envmap.Mapping { ms := make([]envmap.Mapping, len(refs)) for i, ref := range refs { - ms[i] = envmap.Mapping{Service: ref.Service, Field: ref.Field} + ms[i] = envmap.Mapping{Service: ref.Service, Field: ref.Field, Filter: ref.Filter} } return ms } diff --git a/docs/03-reference/command-reference.md b/docs/03-reference/command-reference.md index 8b63102..a517583 100644 --- a/docs/03-reference/command-reference.md +++ b/docs/03-reference/command-reference.md @@ -450,6 +450,24 @@ pass-cli exec \ -- ./run.sh ``` +**Value filters (`| filter`):** a reference may end with a filter that transforms the value before injection. The same grammar works everywhere a reference is parsed — `--set`, `.pass-cli.toml` manifests, and `${pass:...}` templates (`inject`, `exec --env-file`). + +| Filter | Result | +|--------|--------| +| `base64` | standard base64 of the value (e.g. a Bearer token) | +| `base64url` | URL-safe base64 of the value | +| `basicauth` | `base64("username:password")` from the credential — for an HTTP `Authorization: Basic` header; takes no field | + +```bash +# base64 a token; combine a credential's user:password for Basic auth +pass-cli exec \ + --set 'TOKEN=api/token|base64' \ + --set 'AUTH=api|basicauth' \ + -- ./run.sh +``` + +One filter per reference. An unknown or empty filter is a hard error and nothing is injected (fail closed); for `--set` and manifests it is caught before the vault is even opened. On the command line the `|` must be **shell-quoted** (as above). Filters are not applied to the positional `` form — use `--set` or a template. + #### Flags | Flag | Short | Type | Description | diff --git a/internal/envmap/envmap.go b/internal/envmap/envmap.go index 14edda0..bddbeff 100644 --- a/internal/envmap/envmap.go +++ b/internal/envmap/envmap.go @@ -34,6 +34,7 @@ type Mapping struct { EnvName string Service string Field string + Filter string // optional transform applied to the resolved value (e.g. "base64"); "" = none } // ParseSetSpec parses a single "NAME=service[/field]" spec into a Mapping. The @@ -44,26 +45,60 @@ func ParseSetSpec(spec string) (Mapping, error) { if !ok || name == "" || rest == "" { return Mapping{}, fmt.Errorf("invalid mapping %q: expected NAME=service[/field]", spec) } - service, field, err := SplitPath(rest) + service, field, filter, err := SplitPath(rest) if err != nil { return Mapping{}, fmt.Errorf("invalid mapping %q: %w", spec, err) } - return Mapping{EnvName: name, Service: service, Field: field}, nil + return Mapping{EnvName: name, Service: service, Field: field, Filter: filter}, nil } -// SplitPath splits a "service[/field]" credential reference into its service and -// optional field. It is the single home for the path separator, shared by -// --set, the project manifest, and ${pass:...} templates. +// SplitPath splits a "service[/field][ | filter]" credential reference into its +// service, optional field, and optional filter. It is the single home for the +// reference grammar, shared by --set, the project manifest, and ${pass:...} +// templates, so a filter added here works on every surface at once. // -// - Slash is preferred and wins: if the reference contains '/', it is split on -// '/', and any ':' in it is a literal character (the fragility fix). Exactly -// one slash — two segments, service/field — is accepted for now; three or -// more segments error, reserving vault/service/field for a future multi-vault. +// An optional trailing "| filter" is peeled first (on the first '|', which is +// thereby reserved in references like '/' and ':'). The filter name is validated +// against IsKnownFilter and an empty filter is an error — both fail closed at +// parse time (for --set and manifests, before the vault is opened). Whitespace is trimmed ONLY when a pipe +// is present, so a filterless reference is byte-for-byte the original behavior. +// The bare path is then split by splitServiceField. basicauth takes no field. +func SplitPath(ref string) (service, field, filter string, err error) { + path := ref + if left, right, hasPipe := strings.Cut(ref, "|"); hasPipe { + path = strings.TrimSpace(left) + filter = strings.TrimSpace(right) + if filter == "" { + return "", "", "", fmt.Errorf("empty filter after '|': %q", ref) + } + if !IsKnownFilter(filter) { + return "", "", "", fmt.Errorf("unknown filter %q in %q", filter, ref) + } + } + + service, field, err = splitServiceField(path) + if err != nil { + return "", "", "", err + } + + if filter == FilterBasicAuth && field != "" { + return "", "", "", fmt.Errorf("filter %q takes no field: %q", FilterBasicAuth, ref) + } + return service, field, filter, nil +} + +// splitServiceField splits a bare "service[/field]" path (filter already peeled) +// into its service and optional field. +// +// - Slash is preferred and wins: if the path contains '/', it is split on '/', +// and any ':' in it is a literal character (the fragility fix). Exactly one +// slash — two segments, service/field — is accepted for now; three or more +// segments error, reserving vault/service/field for a future multi-vault. // - Otherwise the legacy colon form applies, byte-for-byte the original // behavior: the first ':' separates service from an optional field. -// - With no separator at all, the whole reference is the service and the field -// is empty (the caller falls back to its default field). -func SplitPath(ref string) (service, field string, err error) { +// - With no separator at all, the whole path is the service and the field is +// empty (the caller falls back to its default field). +func splitServiceField(ref string) (service, field string, err error) { if strings.Contains(ref, "/") { segs := strings.Split(ref, "/") if len(segs) > 2 { diff --git a/internal/envmap/envmap_test.go b/internal/envmap/envmap_test.go index ff1beb9..c556be4 100644 --- a/internal/envmap/envmap_test.go +++ b/internal/envmap/envmap_test.go @@ -37,19 +37,25 @@ func TestParseSetSpec(t *testing.T) { wantErr bool }{ // --- legacy colon form (back-compat, unchanged) --- - {name: "service only", spec: "GITHUB_TOKEN=github", want: Mapping{"GITHUB_TOKEN", "github", ""}}, - {name: "colon field override", spec: "DB_USER=postgres:username", want: Mapping{"DB_USER", "postgres", "username"}}, + {name: "service only", spec: "GITHUB_TOKEN=github", want: Mapping{"GITHUB_TOKEN", "github", "", ""}}, + {name: "colon field override", spec: "DB_USER=postgres:username", want: Mapping{"DB_USER", "postgres", "username", ""}}, {name: "empty field after colon", spec: "K=svc:", wantErr: true}, {name: "empty service before colon", spec: "K=:username", wantErr: true}, {name: "no equals", spec: "NOEQUALS", wantErr: true}, {name: "empty service", spec: "K=", wantErr: true}, {name: "empty name", spec: "=github", wantErr: true}, // --- additive slash form (Phase 0b) --- - {name: "slash field", spec: "DB_USER=postgres/username", want: Mapping{"DB_USER", "postgres", "username"}}, - {name: "slash colon is literal in service", spec: "URL=my:svc/password", want: Mapping{"URL", "my:svc", "password"}}, + {name: "slash field", spec: "DB_USER=postgres/username", want: Mapping{"DB_USER", "postgres", "username", ""}}, + {name: "slash colon is literal in service", spec: "URL=my:svc/password", want: Mapping{"URL", "my:svc", "password", ""}}, {name: "slash empty field", spec: "K=svc/", wantErr: true}, {name: "slash empty service", spec: "K=/field", wantErr: true}, {name: "slash multi-segment reserved", spec: "K=vault/svc/field", wantErr: true}, + // --- filters (#138) --- + {name: "base64 filter", spec: "TOKEN=api/key|base64", want: Mapping{"TOKEN", "api", "key", "base64"}}, + {name: "base64url filter", spec: "TOKEN=api/key|base64url", want: Mapping{"TOKEN", "api", "key", "base64url"}}, + {name: "basicauth filter, no field", spec: "AUTH=api|basicauth", want: Mapping{"AUTH", "api", "", "basicauth"}}, + {name: "unknown filter", spec: "K=svc|bogus", wantErr: true}, + {name: "basicauth with field", spec: "K=svc/user|basicauth", wantErr: true}, } for _, tt := range tests { @@ -91,10 +97,11 @@ func TestValidEnvName(t *testing.T) { // slash-wins / colon-literal rule that later surfaces (manifest, templates) rely on. func TestSplitPath(t *testing.T) { tests := []struct { - ref string - wantSvc string - wantField string - wantErr bool + ref string + wantSvc string + wantField string + wantFilter string + wantErr bool }{ {ref: "github", wantSvc: "github", wantField: ""}, {ref: "postgres:username", wantSvc: "postgres", wantField: "username"}, @@ -105,21 +112,35 @@ func TestSplitPath(t *testing.T) { {ref: "vault/svc/field", wantErr: true}, // 3+ segments reserved {ref: "svc:", wantErr: true}, {ref: ":field", wantErr: true}, + // --- filters (#138) --- + {ref: "api/token|base64", wantSvc: "api", wantField: "token", wantFilter: "base64"}, + {ref: "api/token | base64", wantSvc: "api", wantField: "token", wantFilter: "base64"}, // spaces trimmed + {ref: "api/token|base64url", wantSvc: "api", wantField: "token", wantFilter: "base64url"}, + {ref: "github|base64", wantSvc: "github", wantField: "", wantFilter: "base64"}, // no field + filter + {ref: "my:svc/pw|base64", wantSvc: "my:svc", wantField: "pw", wantFilter: "base64"}, + {ref: "svc:field|base64", wantSvc: "svc", wantField: "field", wantFilter: "base64"}, // legacy colon + filter + {ref: "api|basicauth", wantSvc: "api", wantField: "", wantFilter: "basicauth"}, + {ref: "svc/field|", wantErr: true}, // empty filter sentinel + {ref: "svc/field| ", wantErr: true}, // whitespace-only filter + {ref: "svc/field|bogus", wantErr: true}, // unknown filter + {ref: "svc/field|base64|upper", wantErr: true}, // chaining rejected (unknown "base64|upper") + {ref: "api/token|basicauth", wantErr: true}, // basicauth takes no field + {ref: "vault/svc/field|base64", wantErr: true}, // multi-segment still guarded after peel } for _, tt := range tests { t.Run(tt.ref, func(t *testing.T) { - svc, field, err := SplitPath(tt.ref) + svc, field, filter, err := SplitPath(tt.ref) if tt.wantErr { if err == nil { - t.Fatalf("expected error, got service=%q field=%q", svc, field) + t.Fatalf("expected error, got service=%q field=%q filter=%q", svc, field, filter) } return } if err != nil { t.Fatalf("unexpected error: %v", err) } - if svc != tt.wantSvc || field != tt.wantField { - t.Errorf("SplitPath(%q) = (%q, %q), want (%q, %q)", tt.ref, svc, field, tt.wantSvc, tt.wantField) + if svc != tt.wantSvc || field != tt.wantField || filter != tt.wantFilter { + t.Errorf("SplitPath(%q) = (%q, %q, %q), want (%q, %q, %q)", tt.ref, svc, field, filter, tt.wantSvc, tt.wantField, tt.wantFilter) } }) } diff --git a/internal/envmap/filter.go b/internal/envmap/filter.go new file mode 100644 index 0000000..d263a24 --- /dev/null +++ b/internal/envmap/filter.go @@ -0,0 +1,43 @@ +package envmap + +import ( + "encoding/base64" + "fmt" +) + +// FilterBasicAuth is the one compound filter: it combines a credential's own +// username and password into base64("username:password"). It is NOT a value +// transform (it needs two fields), so it is applied in the resolver rather than +// by ApplyValueFilter — envmap only recognizes the name so "| basicauth" parses. +const FilterBasicAuth = "basicauth" + +// valueFilters are pure string->string transforms applied to a single resolved +// value after it comes back from the resolver. Keeping them in one map is the +// single source of truth: IsKnownFilter and ApplyValueFilter both derive from +// it, so parse-time validation and apply-time transformation cannot drift. +var valueFilters = map[string]func(string) string{ + "base64": func(s string) string { return base64.StdEncoding.EncodeToString([]byte(s)) }, + "base64url": func(s string) string { return base64.URLEncoding.EncodeToString([]byte(s)) }, +} + +// IsKnownFilter reports whether name is a recognized filter, matched exactly +// (lowercase). Used at parse time to reject an unknown filter before the vault +// is opened (fail closed). The set is the value filters plus basicauth. +func IsKnownFilter(name string) bool { + if name == FilterBasicAuth { + return true + } + _, ok := valueFilters[name] + return ok +} + +// ApplyValueFilter applies a value transform to a single resolved value. It +// handles only the value filters; basicauth is a compound filter resolved in the +// resolver and is rejected here. An unknown name is an error (fail closed). +func ApplyValueFilter(name, value string) (string, error) { + fn, ok := valueFilters[name] + if !ok { + return "", fmt.Errorf("unknown value filter %q", name) + } + return fn(value), nil +} diff --git a/internal/envmap/filter_test.go b/internal/envmap/filter_test.go new file mode 100644 index 0000000..6547c97 --- /dev/null +++ b/internal/envmap/filter_test.go @@ -0,0 +1,55 @@ +package envmap + +import "testing" + +func TestIsKnownFilter(t *testing.T) { + known := []string{"base64", "base64url", "basicauth"} + for _, n := range known { + if !IsKnownFilter(n) { + t.Errorf("IsKnownFilter(%q) = false, want true", n) + } + } + unknown := []string{"", "bogus", "BASE64", "Base64", "b64", "base64 "} + for _, n := range unknown { + if IsKnownFilter(n) { + t.Errorf("IsKnownFilter(%q) = true, want false", n) + } + } +} + +func TestApplyValueFilter(t *testing.T) { + // A value whose std base64 differs from base64url (contains bytes that map to + // '+' and '/' in the standard alphabet). + const raw = "\xfb\xff" // std: "+/8=", url: "-_8=" + tests := []struct { + name string + filter string + value string + want string + wantErr bool + }{ + {name: "base64 ascii", filter: "base64", value: "user:pass", want: "dXNlcjpwYXNz"}, + {name: "base64 std alphabet", filter: "base64", value: raw, want: "+/8="}, + {name: "base64url alphabet", filter: "base64url", value: raw, want: "-_8="}, + {name: "base64 empty", filter: "base64", value: "", want: ""}, + {name: "basicauth rejected here", filter: "basicauth", value: "x", wantErr: true}, + {name: "unknown", filter: "bogus", value: "x", wantErr: true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := ApplyValueFilter(tt.filter, tt.value) + if tt.wantErr { + if err == nil { + t.Fatalf("expected error, got %q", got) + } + return + } + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if got != tt.want { + t.Errorf("ApplyValueFilter(%q, %q) = %q, want %q", tt.filter, tt.value, got, tt.want) + } + }) + } +} diff --git a/internal/envmap/manifest.go b/internal/envmap/manifest.go index 288aec3..8bd6451 100644 --- a/internal/envmap/manifest.go +++ b/internal/envmap/manifest.go @@ -38,14 +38,14 @@ func ParseManifest(data []byte) ([]Mapping, error) { if !ValidEnvName(name) { return nil, fmt.Errorf("invalid environment variable name %q in manifest", name) } - service, field, err := SplitPath(mf.Env[name]) + service, field, filter, err := SplitPath(mf.Env[name]) if err != nil { return nil, fmt.Errorf("manifest entry %q: %w", name, err) } if service == "" { return nil, fmt.Errorf("manifest entry %q has an empty service", name) } - mappings = append(mappings, Mapping{EnvName: name, Service: service, Field: field}) + mappings = append(mappings, Mapping{EnvName: name, Service: service, Field: field, Filter: filter}) } return mappings, nil } diff --git a/internal/envmap/template.go b/internal/envmap/template.go index 5562fe4..8cd0222 100644 --- a/internal/envmap/template.go +++ b/internal/envmap/template.go @@ -5,10 +5,11 @@ import ( "strings" ) -// TemplateRef is one ${pass:service[/field]} reference found in a template. +// TemplateRef is one ${pass:service[/field][ | filter]} reference found in a template. type TemplateRef struct { Service string Field string // "" means the caller's default field + Filter string // optional value transform (e.g. "base64"); "" = none } const passMarker = "${pass:" @@ -35,14 +36,14 @@ func parseTemplate(s string) (literals []string, refs []TemplateRef, err error) return nil, nil, fmt.Errorf("unterminated %s...} reference: %q", passMarker, s[start:]) } inner := rest[:end] - service, field, e := SplitPath(inner) + service, field, filter, e := SplitPath(inner) if e != nil { return nil, nil, fmt.Errorf("invalid reference %s%s}: %w", passMarker, inner, e) } if service == "" { return nil, nil, fmt.Errorf("empty service in reference %s%s}", passMarker, inner) } - refs = append(refs, TemplateRef{Service: service, Field: field}) + refs = append(refs, TemplateRef{Service: service, Field: field, Filter: filter}) i = start + len(passMarker) + end + 1 } } diff --git a/internal/resolver/resolver.go b/internal/resolver/resolver.go index b45ce79..e3fb7f9 100644 --- a/internal/resolver/resolver.go +++ b/internal/resolver/resolver.go @@ -37,7 +37,7 @@ type Resolver interface { // resolver primitive value-only, so template rendering can reuse ResolveValues // without parsing a "NAME=" prefix back off. func ResolveEnv(r Resolver, mappings []envmap.Mapping, defaultField string) ([]string, error) { - values, err := r.ResolveValues(mappings, defaultField) + values, err := ResolveValuesFiltered(r, mappings, defaultField) if err != nil { return nil, err } @@ -48,6 +48,62 @@ func ResolveEnv(r Resolver, mappings []envmap.Mapping, defaultField string) ([]s return out, nil } +// ResolveValuesFiltered resolves each mapping's value and applies its Filter, +// returning one value per mapping in order. It is the filtered counterpart to +// Resolver.ResolveValues used by every injection surface (--set, manifests, and +// ${pass:...} templates). +// +// Filters are applied client-side, AFTER the backend returns raw values, so the +// agent's values-only protocol is untouched — base64 is identical whether a +// direct vault or a resident daemon served the value. Most mappings resolve one +// field; a "basicauth" mapping resolves TWO fields (username and password) of its +// service and combines them into base64("username:password"). Both sub-resolves +// go through the normal value path, so a socket backend still answers the whole +// request in one batch round-trip. +func ResolveValuesFiltered(r Resolver, mappings []envmap.Mapping, defaultField string) ([]string, error) { + // Expand each mapping into its sub-requests (1, or 2 for basicauth), recording + // how to fold the results back into one value per mapping. + type plan struct { + filter string + start int // index of this mapping's first value in the flat result + } + subs := make([]envmap.Mapping, 0, len(mappings)) + plans := make([]plan, len(mappings)) + for i, m := range mappings { + plans[i] = plan{filter: m.Filter, start: len(subs)} + if m.Filter == envmap.FilterBasicAuth { + subs = append(subs, + envmap.Mapping{Service: m.Service, Field: "username"}, + envmap.Mapping{Service: m.Service, Field: "password"}, + ) + continue + } + subs = append(subs, envmap.Mapping{Service: m.Service, Field: m.Field}) + } + + values, err := r.ResolveValues(subs, defaultField) + if err != nil { + return nil, err + } + + out := make([]string, len(mappings)) + for i, p := range plans { + switch p.filter { + case "": + out[i] = values[p.start] + case envmap.FilterBasicAuth: + // HTTP Basic auth: base64("username:password") (standard encoding). + out[i], err = envmap.ApplyValueFilter("base64", values[p.start]+":"+values[p.start+1]) + default: + out[i], err = envmap.ApplyValueFilter(p.filter, values[p.start]) + } + if err != nil { + return nil, err + } + } + return out, nil +} + // directResolver resolves against an already-unlocked, in-process VaultService. // The caller owns unlock/Lock; this type only reads. type directResolver struct { diff --git a/internal/resolver/resolver_test.go b/internal/resolver/resolver_test.go index b9c7259..2741361 100644 --- a/internal/resolver/resolver_test.go +++ b/internal/resolver/resolver_test.go @@ -2,6 +2,7 @@ package resolver import ( "bytes" + "encoding/base64" "os" "path/filepath" "testing" @@ -156,3 +157,45 @@ func TestDirectResolver_InvalidField(t *testing.T) { t.Fatal("expected error for invalid field, got nil") } } + +// TestResolveValuesFiltered covers the three filter cases: no filter (raw), +// base64 on a single field, and basicauth combining username+password. It also +// verifies an unknown filter (constructed directly, bypassing parse validation) +// propagates as an error. +func TestResolveValuesFiltered(t *testing.T) { + vs, _ := newUnlockedVault(t) + defer vs.Lock() + + r := NewDirect(vs) + defer func() { _ = r.Close() }() + + got, err := ResolveValuesFiltered(r, []envmap.Mapping{ + {Service: "github", Field: "password"}, // no filter -> raw + {Service: "github", Field: "password", Filter: "base64"}, // base64 of the value + {Service: "github", Filter: envmap.FilterBasicAuth}, // base64(user:pass) + }, "password") + if err != nil { + t.Fatalf("ResolveValuesFiltered: %v", err) + } + + want := []string{ + "s3cr3t-pw", + base64.StdEncoding.EncodeToString([]byte("s3cr3t-pw")), + base64.StdEncoding.EncodeToString([]byte("octocat:s3cr3t-pw")), + } + if len(got) != len(want) { + t.Fatalf("got %d values, want %d: %v", len(got), len(want), got) + } + for i := range want { + if got[i] != want[i] { + t.Errorf("value[%d] = %q, want %q", i, got[i], want[i]) + } + } + + // An unknown filter (bypassing parse-time validation) is a hard error. + if _, err := ResolveValuesFiltered(r, []envmap.Mapping{ + {Service: "github", Field: "password", Filter: "bogus"}, + }, "password"); err == nil { + t.Error("expected error for unknown filter, got nil") + } +} diff --git a/test/integration/agent_test.go b/test/integration/agent_test.go index e11109b..340b959 100644 --- a/test/integration/agent_test.go +++ b/test/integration/agent_test.go @@ -4,6 +4,7 @@ package integration import ( "bytes" + "encoding/base64" "fmt" "os" "os/exec" @@ -113,6 +114,25 @@ func TestIntegration_Agent_ExecResolvesViaSocket(t *testing.T) { } } +// TestIntegration_Agent_FilterAppliedClientSide proves a value filter is applied +// by the client even when a running agent serves the raw value over the socket: +// the agent protocol carries no filter, so base64 must happen client-side and +// match direct-mode output. +func TestIntegration_Agent_FilterAppliedClientSide(t *testing.T) { + configPath, password, service, secret := setupExecVault(t) + sockPath, _, _ := startAgent(t, configPath, password) + + out, stderr, err := runWithAgent(t, configPath, sockPath, + "exec", "--set", "TOK="+service+"|base64", "--", "sh", "-c", `printf %s "$TOK"`) + if err != nil { + t.Fatalf("filtered exec via agent failed: %v\nStderr: %s", err, stderr) + } + want := base64.StdEncoding.EncodeToString([]byte(secret)) + if strings.TrimSpace(out) != want { + t.Errorf("filtered exec via agent: got %q, want %q", strings.TrimSpace(out), want) + } +} + // TestIntegration_Agent_ExportResolvesViaSocket proves export also uses the agent. func TestIntegration_Agent_ExportResolvesViaSocket(t *testing.T) { configPath, password, service, secret := setupExecVault(t) diff --git a/test/integration/export_test.go b/test/integration/export_test.go index fa26318..108a14a 100644 --- a/test/integration/export_test.go +++ b/test/integration/export_test.go @@ -3,6 +3,7 @@ package integration import ( + "encoding/base64" "os/exec" "runtime" "strings" @@ -42,6 +43,27 @@ func TestIntegration_Export_RoundTrip(t *testing.T) { } } +// TestIntegration_Export_Base64Filter verifies a "| base64" filter is applied +// before shell-quoting, so the emitted statement eval's to the encoded value. +func TestIntegration_Export_Base64Filter(t *testing.T) { + if runtime.GOOS == "windows" { + t.Skip("uses POSIX sh") + } + configPath, password, service, secret := setupExecVault(t) + + stdin := helpers.BuildUnlockStdin(password) + stdout, stderr, err := helpers.RunCmd(t, binaryPath, configPath, stdin, + "export", "--set", "TOK="+service+"/password|base64") + if err != nil { + t.Fatalf("export failed: %v\nStderr: %s", err, stderr) + } + stmt := strings.TrimSpace(stdout) + want := base64.StdEncoding.EncodeToString([]byte(secret)) + if got := evalSh(t, stmt, "TOK"); got != want { + t.Errorf("eval base64: got %q, want %q\nstmt: %s", got, want, stmt) + } +} + // TestIntegration_Export_AdversarialSecret round-trips a secret whose value // contains quotes and command substitution — the end-to-end quoting proof. func TestIntegration_Export_AdversarialSecret(t *testing.T) { diff --git a/test/integration/inject_test.go b/test/integration/inject_test.go index 7959d04..8584d5c 100644 --- a/test/integration/inject_test.go +++ b/test/integration/inject_test.go @@ -3,6 +3,7 @@ package integration import ( + "encoding/base64" "os" "path/filepath" "runtime" @@ -113,3 +114,65 @@ func TestIntegration_Exec_EnvFile(t *testing.T) { t.Errorf("env-file injection: stdout = %q, want %q", strings.TrimSpace(stdout), want) } } + +// TestIntegration_Inject_Base64Filter verifies a "| base64" filter encodes the +// resolved value. +func TestIntegration_Inject_Base64Filter(t *testing.T) { + configPath, password, service, secret := setupExecVault(t) + + tmpl := filepath.Join(t.TempDir(), "b64.tmpl") + if err := os.WriteFile(tmpl, []byte("X=${pass:"+service+"/password | base64}\n"), 0600); err != nil { + t.Fatal(err) + } + + stdout, stderr, err := helpers.RunCmd(t, binaryPath, configPath, helpers.BuildUnlockStdin(password), + "inject", "--in-file", tmpl) + if err != nil { + t.Fatalf("inject failed: %v\nStderr: %s", err, stderr) + } + want := "X=" + base64.StdEncoding.EncodeToString([]byte(secret)) + "\n" + if stdout != want { + t.Errorf("inject output = %q, want %q", stdout, want) + } +} + +// TestIntegration_Inject_BasicAuthFilter verifies "| basicauth" combines the +// credential's username and password into base64("user:pass"). +func TestIntegration_Inject_BasicAuthFilter(t *testing.T) { + configPath, password, service, secret := setupExecVault(t) + + tmpl := filepath.Join(t.TempDir(), "auth.tmpl") + if err := os.WriteFile(tmpl, []byte("A=${pass:"+service+" | basicauth}\n"), 0600); err != nil { + t.Fatal(err) + } + + stdout, stderr, err := helpers.RunCmd(t, binaryPath, configPath, helpers.BuildUnlockStdin(password), + "inject", "--in-file", tmpl) + if err != nil { + t.Fatalf("inject failed: %v\nStderr: %s", err, stderr) + } + want := "A=" + base64.StdEncoding.EncodeToString([]byte("execuser:"+secret)) + "\n" + if stdout != want { + t.Errorf("inject output = %q, want %q", stdout, want) + } +} + +// TestIntegration_Inject_UnknownFilterFailsClosed verifies an unknown filter is a +// hard error that writes nothing. +func TestIntegration_Inject_UnknownFilterFailsClosed(t *testing.T) { + configPath, password, service, _ := setupExecVault(t) + + tmpl := filepath.Join(t.TempDir(), "bad.tmpl") + if err := os.WriteFile(tmpl, []byte("X=${pass:"+service+"/password | bogus}\n"), 0600); err != nil { + t.Fatal(err) + } + + stdout, _, err := helpers.RunCmd(t, binaryPath, configPath, helpers.BuildUnlockStdin(password), + "inject", "--in-file", tmpl) + if err == nil { + t.Fatalf("expected error for unknown filter, got output %q", stdout) + } + if stdout != "" { + t.Errorf("expected no output on failure, got %q", stdout) + } +}