Sensitive Data RBAC | securing the data-server grpc api

[steiler]

Sensitive Data RBAC — Design Document

This document outlines the available implementation options. Preferred choices are indicated with a ✓ symbol.

Background

include_sensitive: bool exists on GetIntentRequest and BlameConfigRequest in sdc-protos. The data-server render pipeline fully honors it — when true, redaction is skipped and plaintext values are returned. Today no caller sets it and no auth gate exists. Any process that can reach data-server's gRPC port can set the flag and receive unredacted data.

Two independent sensitivity sources compose at render time:

1. Schema-level — a leaf annotated with sdcio-ext:sensitive in a YANG overlay propagates LeafSchema.sensitive = true through schema-server
2. Intent path markers — sensitive_paths stored per-intent in the data-server cache, populated by config-server during K8s Secret resolution

Both sources are treated uniformly from an access-control perspective. There is no meaningful security boundary between them.

WatchDeviations has no include_sensitive field at all — streaming deviations are always redacted today, with no bypass mechanism.

---

Goals

• Allow authorized users to retrieve unredacted sensitive config values via northbound APIs
• data-server must be able to run standalone — no Kubernetes dependency, no config-server dependency
• config-server adds a Kubernetes-native authorization layer on top when running in-cluster
• Operators control which identities can request plaintext using standard, familiar tooling
• CLI tools expose the capability behind an explicit opt-in flag

---

Non-Goals

• Path-scoped access — granular per-YANG-path ACLs are out of scope; access is all-or-nothing
• Tier distinction — no separate roles for schema-sensitive vs. intent-sensitive data; both are treated as one sensitivity level
• Audit trail / structured access logging in data-server — deferred, possibly never needed
• Encrypt-at-rest — a separate KMS concern, unrelated to northbound visibility

---

Decision 1: Enforcement Architecture

Options considered

Option A — data-server only

A single interceptor in data-server validates caller identity before honoring include_sensitive. All clients, including config-server, go through one gate.

Rejected: config-server acts on behalf of K8s users whose identity lives in their bearer token, not in a cert. Propagating that identity into gRPC requires data-server to call K8s TokenReview or SubjectAccessReview, introducing a hard Kubernetes dependency into a component that is explicitly designed to run without Kubernetes.

---

Option B — config-server only

config-server performs a K8s SubjectAccessReview against the incoming caller's bearer token and only sets include_sensitive: true on the outbound data-server gRPC call if the SAR passes.

Rejected: sdctl talks directly to data-server gRPC, bypassing config-server entirely. This path is completely ungated unless data-server is network-isolated from sdctl — an operational constraint rather than a code contract. It cannot be relied upon as a security boundary.

---

Option C — both layers, defense in depth ✓

config-server performs a K8s SubjectAccessReview as its gate. data-server independently validates an mTLS client certificate. Each layer enforces autonomously. data-server remains Kubernetes-agnostic and works standalone.

Decision: Option C

kubectl-sdc  →  K8s API  →  config-server ──[SAR check]──► sets include_sensitive
                                   │
                                   │  gRPC + mTLS client cert (OU=sensitive-reader)
                                   ▼
                            data-server ──[cert OU check]──► honors include_sensitive

sdctl  ──────────────────────────────────  gRPC + mTLS client cert (OU=sensitive-reader)
                                   │
                                   ▼
                            data-server ──[cert OU check]──► honors include_sensitive

---

Decision 2: Auth Mechanism at data-server

Options considered

Option: K8s bearer token forwarding

config-server extracts the caller's SA token and forwards it via gRPC metadata. data-server calls K8s TokenReview and SubjectAccessReview to validate it.

Rejected: Introduces a Kubernetes dependency in data-server, breaking standalone deployment. Token audience and TTL semantics are fragile across a service boundary. Tight coupling to config-server as the only valid intermediary.

---

Option: Validated-role claim forwarding

config-server performs the SAR and, if the caller is authorized, forwards a validated_role: "sensitive-reader" claim as a proto field or gRPC metadata header. data-server trusts the claim.

Rejected: The claim is only as trustworthy as the network between config-server and data-server. Any process with network access can forge the header unless the connection is already mutually authenticated — at which point the mTLS certificate itself is the proof of identity and the separate claim is redundant.

---

Option: mTLS client certificate + configurable OU ✓

data-server verifies the TLS client certificate chain against a configurable CA. It checks for a required OU attribute on the certificate. No external calls are needed. Works fully standalone. The CA file and required OU value are configurable at startup via environment variables, allowing operators to bring their own PKI.

Decision: mTLS with configurable CA and OU

data-server startup environment variables:

DS_SENSITIVE_TLS_ENABLED=true
DS_SENSITIVE_TLS_CLIENT_CA_FILE=/etc/sdcio/sensitive-ca.crt   # PEM bundle
DS_SENSITIVE_TLS_CLIENT_OU=sensitive-reader                    # required OU value

Interceptor behavior:

1. If DS_SENSITIVE_TLS_ENABLED is false, include_sensitive is always treated as false.
2. If the client did not present a certificate, demote include_sensitive to false and continue — no error returned.
3. If the certificate chain does not validate against DS_SENSITIVE_TLS_CLIENT_CA_FILE, demote and continue.
4. If no certificate in the chain has OU == DS_SENSITIVE_TLS_CLIENT_OU, demote and continue.
5. All checks pass → honor the include_sensitive value from the request.

Clients without a matching certificate receive a fully redacted response. No error is returned and no information about the gate's existence is leaked.

---

Decision 3: config-server SAR Resource

Options considered

Option: get sensitiveconfigs

The SensitiveConfig CR already exists. No new handler would be needed.

Rejected: Semantically incorrect. get sensitiveconfigs means "read the encrypted payload CR." It does not express "reveal plaintext config values." Administrators writing RBAC policies would be confused by the mismatch between the verb and its actual effect.

---

Option: custom verb reveal on configs

kubectl auth can-i reveal configs expresses intent clearly.

Rejected: Custom verbs are valid K8s RBAC but are an uncommon pattern. Standard tooling, audit logs, and RBAC visualization tools handle subresources more naturally than custom verbs.

---

Option: get configs/sensitive subresource ✓

A standard K8s subresource. Idiomatic. Requires a new handler in config-server's aggregated API server, which is a small, contained addition.

Decision: configs/sensitive subresource

ClusterRole for sensitive readers:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: sensitive-reader
rules:
- apiGroups: ["config.sdcio.dev"]
  resources: ["configs/sensitive"]
  verbs: ["get"]

config-server performs a SubjectAccessReview against the incoming caller's bearer token before setting include_sensitive: true on the outbound data-server gRPC call. The resource and verb are hard-coded — they are not operator-configurable.

---

Decision 4: Cert Issuance Control

Options considered

Option: ClusterIssuer only, no request policy

Any principal that can create a Certificate CR can request any subject, including OU=sensitive-reader.

Rejected: Any namespace administrator would be able to obtain a sensitive-reader certificate on demand. This completely undermines the authorization model.

---

Option: Manual CertificateRequest approval

A cluster administrator manually patches the Approved condition on each CertificateRequest.

Rejected: Does not scale. Blocks automated certificate rotation for services. Operationally expensive.

---

Option: cert-manager approver-policy ✓

The CertificateRequestPolicy CRD from the cert-manager approver-policy project constrains exactly which subject attributes, usages, and issuers are permitted for a given policy. A standard K8s RBAC use binding on the policy controls which principals can trigger auto-approval.

Decision: cert-manager approver-policy

Setup:

# Policy: permits OU=sensitive-reader client auth certs only
apiVersion: policy.cert-manager.io/v1alpha1
kind: CertificateRequestPolicy
metadata:
  name: sensitive-reader-policy
spec:
  allowed:
    subject:
      organizations:
        values: ["sdcio"]
      organizationalUnits:
        values: ["sensitive-reader"]
    usages: ["client auth"]
  selector:
    issuerRef:
      name: sdcio-internal-ca
      kind: ClusterIssuer

# RBAC: controls who can use the policy
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: can-get-sensitive-cert
rules:
- apiGroups: ["policy.cert-manager.io"]
  resources: ["certificaterequestpolicies"]
  verbs: ["use"]
  resourceNames: ["sensitive-reader-policy"]

To grant an operator access: kubectl create clusterrolebinding alice-sensitive --clusterrole=can-get-sensitive-cert --user=alice

Without this binding, any Certificate CR requesting OU=sensitive-reader is automatically denied by the policy controller, regardless of who submits it.

Certificates are issued with a 24-hour TTL and are automatically rotated by cert-manager for service accounts. Human operators obtain their certificates via kubectl sdc cert get (see CLI section below).

---

Decision 5: Proto Changes

Options considered

Option: Replace bool include_sensitive with a structured SensitiveAccess message

Carry an identity proof inside the proto (bearer token, certificate DER, validated role name, access scope).

Rejected: This duplicates what TLS already provides at the transport layer. If mTLS is the authentication mechanism, the certificate is already exchanged during the handshake — re-encoding it in the proto payload is redundant and expands the attack surface. A scope field for partial access was also considered but rejected because access is all-or-nothing.

---

Option: Keep bool include_sensitive, enforce via transport ✓

The proto remains simple. The gRPC interceptor checks the mTLS certificate and either honors or silently zeros out the boolean. Authentication is a transport-layer concern; the proto carries only the intent.

Decision: Keep bool, add to WatchDeviationRequest

All changes land in a single proto release:

• WatchDeviationRequest: add bool include_sensitive = N — this field is missing today and the behavior is hardcoded to false with no bypass path
• GetIntentRequest.include_sensitive: no change (field 4, already exists)
• BlameConfigRequest.include_sensitive: no change (field 2, already exists)

---

Decision 6: Sensitivity Granularity

Options considered

Option: Two tiers — schema-sensitive vs. intent-sensitive

Separate roles for schema-annotated leaves (YANG sdcio-ext:sensitive) and operator-injected path markers (sensitive_paths on intents).

Rejected: There is no meaningful security boundary between the two. Both represent a deliberate decision by an operator that a value should not be visible northbound. Introducing separate roles adds RBAC complexity for administrators without a practical benefit.

---

Option: Path-scoped access

A role grants plaintext access only to specific YANG path prefixes.

Rejected: K8s RBAC has no native path-matching mechanism. Implementing it would require a custom admission webhook or OPA integration. No use case was identified that requires partial plaintext access.

---

Option: All-or-nothing ✓

One role. An authorized caller sees all sensitive values in plaintext. No partial access.

Decision: All-or-nothing

---

CLI Changes

kubectl-sdc

• Add --include-sensitive flag to the blame and runningconfig subcommands. When set, the request targets the configs/sensitive subresource path in config-server, which performs the SAR check before forwarding include_sensitive: true to data-server.

• Add kubectl sdc cert get subcommand with the following flow:

  1. Creates a Certificate CR (24h TTL, OU=sensitive-reader) in the caller's namespace
  2. Waits for the Ready condition
  3. Extracts tls.crt and tls.key from the resulting Secret to local files (e.g. ~/.sdcio/certs/)
  4. Prints a usage hint showing the correct sdctl invocation

  The subcommand requires the caller to already be bound to can-get-sensitive-cert — it is self-service but gated.

sdctl

• Add --tls-cert and --tls-key flags to data subcommands (blame, get-intent, list-intent, etc.)
• The certificate is obtained via kubectl sdc cert get or from the operator's own PKI

---

Open Questions

#	Question	Status
1	Exact K8s RBAC binding strategy — per-user, per-OIDC group, per-ServiceAccount namespace?	Organizational/team policy decision, not a technical one
2	Should WatchDeviations expose a privileged unredacted stream long-term, or remain always-redacted?	Deferred — the proto field is added in this release but behavioral requirements are not yet defined

---

Implementation Order

1. sdc-protos — add include_sensitive to WatchDeviationRequest, cut new release
2. data-server — mTLS interceptor, env var configuration, consume new proto
3. config-server — configs/sensitive subresource handler, SAR check, consume new proto
4. kubectl-sdcio — --include-sensitive flag on blame and runningconfig, kubectl sdc cert get subcommand
5. sdctl — --tls-cert / --tls-key flags on data subcommands
6. integration-tests — implement TC7 (currently skipped with a TODO tag)

[henderiw]

Alternative for Decision 3: two dedicated resources instead of configs/sensitive

I'd like to propose a variation on the config-server side of this. If the goal is to
express "who may read plaintext" as an RBAC rule, we can get that without a SAR call in
config-server, by mirroring the existing read endpoints with sensitive counterparts:

• sensitiverunningconfigs
• sensitiveblameconfigs

Both live in the same API group and return the same shape as runningconfigs /
blameconfigs. The only difference: when one of these endpoints is hit, config-server
sets include_sensitive: true on the outbound data-server call. The non-sensitive
endpoints keep passing false, unconditionally.

RBAC then looks like:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: sensitive-reader
rules:
- apiGroups: ["config.sdcio.dev"]
  resources: ["sensitiverunningconfigs", "sensitiveblameconfigs"]
  verbs: ["get"]

Why I prefer this over get configs/sensitive:

1. No SubjectAccessReview in config-server. As an aggregated API server we already
   run delegated authn/authz — the request is authorized against the resource before
   our handler ever sees it. A caller without the ClusterRole gets a 403 from the
   standard path. The configs/sensitive design still needs config-server to do an
   explicit SAR and then decide whether to flip a boolean; that's authorization logic
   we hand-roll instead of authorization we inherit. Fewer places to get it wrong.

2. Semantic fit. runningconfig and blameconfig aren't subresources of Config —
   they're their own read endpoints backed by data-server state. Hanging sensitive
   off configs implies a relationship that doesn't exist, and it doesn't express
   which view is being revealed. sensitiverunningconfigs says exactly what it grants.

3. Granularity for free. An operator can grant plaintext blame without granting
   plaintext running config, or vice versa. This isn't the path-scoped access rejected
   in Decision 6 — it's still all-or-nothing per view — but it costs us nothing and it
   maps onto how people actually reason about these two endpoints.

4. Audit logs are self-describing. The K8s audit entry names the sensitive resource
   directly; no need to correlate a get configs with a SAR to know plaintext was served.

Unchanged: Decisions 1, 2, 4, 5, 6 all still hold. data-server keeps its independent
mTLS + OU gate (sdctl still bypasses config-server entirely, so that layer is still
required), the proto keeps the plain bool, cert issuance stays on approver-policy.

CLI: kubectl-sdc keeps --include-sensitive on blame / runningconfig; the flag
simply routes to the sensitive resource instead of the normal one.

Open questions on my side:

• Naming collision with the existing SensitiveConfig CR — is sensitiverunningconfigs
  going to read as "the sensitive part of the running config" to someone scanning a
  ClusterRole? Alternatives: runningconfigs/sensitive and blameconfigs/sensitive as
  true subresources of the right parent, which keeps the subresource idiom from the
  original proposal but attaches it to the correct resource.
• WatchDeviations doesn't have an equivalent northbound resource in config-server, so
  it stays on the data-server mTLS gate only. That seems fine given Open Question 2.

Happy to be talked out of it if the SAR path buys something I'm not seeing.