Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

### Security

- Opt-in `ValidatingAdmissionPolicy` and binding to restrict the `node-data-broker` ServiceAccount to only updating its host Node resource, mitigating privilege escalation via node spec or annotation manipulation (audit F1 compliance).
- Removed unused RBAC verbs from the Topograph API server and node-data-broker ClusterRoles (least-privilege): API server `pods` rule dropped `get` (list-only), `daemonsets` rule dropped `list` (get-only), and the Slinky `configmaps` rule dropped `list`; node-data-broker `nodes` rule dropped `list` (get/update), and the InfiniBand `daemonsets`/`pods` rules dropped `list`/`get` respectively (get-only, list-only).

- node-observer ClusterRole no longer grants unused `get`; `nodes` list/watch now gated on `trigger.nodeSelector`.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
{{- if and .Values.nodeDataBroker.enabled .Values.nodeDataBroker.validatingAdmissionPolicy.enabled }}
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
name: {{ include "nodeDataBroker.fullname" . }}
labels:
{{- include "nodeDataBroker.labels" . | nindent 4 }}
spec:
failurePolicy: Fail

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

failurePolicy: Fail with a matchConstraints that selects every node UPDATE from any user means all node updates (including kubelet heartbeats) run through this CEL; a fault would deny them cluster-wide. Moving the SA check into spec.matchConditions (username == the broker SA) limits both the evaluation and the fail-closed blast radius to just the broker's requests.

matchConstraints:
resourceRules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["UPDATE"]
resources: ["nodes"]
matchConditions:
- name: match-node-data-broker-serviceaccount
expression: >-
request.userInfo.username == 'system:serviceaccount:{{ .Release.Namespace }}:{{ include "nodeDataBroker.serviceAccountName" . }}'
validations:
- expression: >-
has(request.userInfo.extra) &&
'authentication.kubernetes.io/node-name' in request.userInfo.extra &&
size(request.userInfo.extra['authentication.kubernetes.io/node-name']) > 0 &&
request.name == request.userInfo.extra['authentication.kubernetes.io/node-name'][0]
message: "node-data-broker ServiceAccount is only authorized to update its own Node resource (audit F1)."
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
name: {{ include "nodeDataBroker.fullname" . }}
labels:
{{- include "nodeDataBroker.labels" . | nindent 4 }}
spec:
policyName: {{ include "nodeDataBroker.fullname" . }}
validationActions: [Deny]
{{- end }}
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
suite: node-data-broker validating admission policy
templates:
- templates/nodeDataBroker/validatingadmissionpolicy.yaml
release:
name: chart-ci
namespace: topograph
capabilities:
majorVersion: "1"
minorVersion: "30"
tests:
- it: is omitted by default
asserts:
- hasDocuments:
count: 0

- it: is omitted when the component is disabled
set:
nodeDataBroker:
enabled: false
validatingAdmissionPolicy:
enabled: true
asserts:
- hasDocuments:
count: 0

- it: renders both resources when enabled
set:
nodeDataBroker:
enabled: true
validatingAdmissionPolicy:
enabled: true
asserts:
- hasDocuments:
count: 2

- it: renders correct API versions and kinds for ValidatingAdmissionPolicy
set:
nodeDataBroker:
enabled: true
validatingAdmissionPolicy:
enabled: true
documentIndex: 0
asserts:
- isKind:
of: ValidatingAdmissionPolicy
- equal:
path: apiVersion
value: admissionregistration.k8s.io/v1
- equal:
path: metadata.name
value: chart-ci-topograph-node-data-broker
- equal:
path: spec.failurePolicy
value: Fail

- it: renders correct API versions and kinds for ValidatingAdmissionPolicyBinding
set:
nodeDataBroker:
enabled: true
validatingAdmissionPolicy:
enabled: true
documentIndex: 1
asserts:
- isKind:
of: ValidatingAdmissionPolicyBinding
- equal:
path: apiVersion
value: admissionregistration.k8s.io/v1
- equal:
path: metadata.name
value: chart-ci-topograph-node-data-broker
- equal:
path: spec.policyName
value: chart-ci-topograph-node-data-broker
- equal:
path: spec.validationActions[0]
value: Deny

- it: configures the CEL expression and dynamic ServiceAccount matching
set:
nodeDataBroker:
enabled: true
validatingAdmissionPolicy:
enabled: true
serviceAccount:
create: true
name: custom-node-data-broker-sa
documentIndex: 0
asserts:
- equal:
path: spec.matchConditions[0].name
value: match-node-data-broker-serviceaccount
- equal:
path: spec.matchConditions[0].expression
value: "request.userInfo.username == 'system:serviceaccount:topograph:custom-node-data-broker-sa'"
- equal:
path: spec.validations[0].expression
value: >-
has(request.userInfo.extra) &&
'authentication.kubernetes.io/node-name' in request.userInfo.extra &&
size(request.userInfo.extra['authentication.kubernetes.io/node-name']) > 0 &&
request.name == request.userInfo.extra['authentication.kubernetes.io/node-name'][0]
- equal:
path: spec.validations[0].message
value: "node-data-broker ServiceAccount is only authorized to update its own Node resource (audit F1)."
6 changes: 6 additions & 0 deletions charts/topograph/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,12 @@
"startupProbe": { "type": "object" },
"serviceAccount": { "type": "object" },
"rbac": { "type": "object" },
"validatingAdmissionPolicy": {
"type": "object",
"properties": {
"enabled": { "type": "boolean" }
}
},
"env": { "type": "object" },
"initContainers": { "type": "array" },
"lifecycle": { "type": "object" },
Expand Down
5 changes: 5 additions & 0 deletions charts/topograph/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -312,6 +312,11 @@ nodeDataBroker:
rbac:
create: true

# Opt-in ValidatingAdmissionPolicy to scope node writes. Requires
# Kubernetes >= 1.30 (admissionregistration.k8s.io/v1) and ServiceAccountTokenPodNodeInfo enabled.
validatingAdmissionPolicy:
enabled: false

env: {}
initContainers: []
lifecycle: {}
Expand Down
16 changes: 16 additions & 0 deletions docs/engines/k8s.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,22 @@ Apply alongside the chart. A bundled template is under consideration.

The node-observer `ClusterRole` grants `pods [list, watch]` unconditionally, `apps/daemonsets [list, watch]` when `nodeDataBroker.enabled=true`, and `nodes [list, watch]` only when `nodeObserver.topograph.trigger.nodeSelector` is set. Set `nodeObserver.rbac.create: false` to suppress the `ClusterRole`/`ClusterRoleBinding` when managing RBAC externally.

### Node Data Broker write-scoping via ValidatingAdmissionPolicy

The node-data-broker's RBAC grants update permission on nodes cluster-wide. To enforce least-privilege scoping, the chart supports deploying an opt-in `ValidatingAdmissionPolicy` and its binding:

```yaml
nodeDataBroker:
validatingAdmissionPolicy:
enabled: true
```

When enabled, the policy matches node `UPDATE` operations performed by the broker's ServiceAccount and validates that the name of the node being modified matches the node name claim bound to the requester's ServiceAccount token (`authentication.kubernetes.io/node-name`).

**Prerequisites:**
1. A Kubernetes cluster running **Kubernetes 1.30+** with the **`ValidatingAdmissionPolicy` API enabled** (`admissionregistration.k8s.io/v1`).
2. ServiceAccount tokens carrying the node-binding identity claim (e.g., via `ServiceAccountTokenPodNodeInfo`). The policy denies updates from the broker ServiceAccount if this required node-name claim is missing from the request context.

## Validation and Testing

The Helm chart ships two layers of validation for operators.
Expand Down